The Self-Organizing Field - Semantic Scholar

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 0, NO. 0, MONTH 1996

1

The Self-Organizing Field Simone Santini Abstract— Many of the properties of the well-known Kohonen map algorithm are not easily derivable from its discrete formulation. For instance, the “projection” implemented by the map from a high dimensional input space to a lower dimensional map space must be properly regarded as a projection from a smooth manifold to a lattice and, in this framework, some of its property are not easily identified. This paper describes the Self-Organizing Field: a continuous embedding of a smooth manifold (the map) into another (the input manifold) that implements a topological map by selforganization. The adaptation of the Self Organizing field is governed by a set of differential equations analogous to the difference equations that determine weights updates in the Kohonen map. The paper derives several properties of the Self-Organizing Field, and shows that the emergence of certain structures on the brain—like the columnar organization in the primary visual cortex—arise naturally in the new model.

I. Introduction

T

HIS paper presents a continuous model of topology preserving map whose behavior is similar in many respects to that of a Kohonen map [11], [10], [1]. The model, which I call Self-Organizing Field, is an embedding of a lower dimensional smooth manifold (the map) into a higher dimensional one (the input manifold). The embedding selforganizes based on the geometry of the higher dimensional manifold, and its equilibrium point retains many of the desirable properties of Kohonen’s “topology preserving” map. The Kohonen map is a successful model of how topography preserving (or topological) maps can be formed by self-organization in the brain. The topological map is a common structure, and the same basic organization can be found in such functionally diverse area as the somatotopic map [16], the interaural delay map [9], and a number of maps in the visual system. In the following, I will consider mainly examples from the visual system. Maps of the visual field can be found in the retina, the ganglion cells, the Lateral Geniculate Nuclei (LGNs), and the primary visual cortex in the central visual pathway, as well as in the pretectum and the superior colliculus for the control of eye movements [17], [26]. Each map entails a deformation of the visual field whereby some areas have allocated larger portions of the map than other areas. For instance, the mapping from the visual field to the striate cortex is roughly logarithmic [25], with about 40% of the striate area being devoted to processing information from the central 5 degrees of the visual field (about 0.7% of the visual field). It should be noted that this distortion is not just a consequence of the fact that the human eye has a higher density or receptors in the fovea than in the peSimone is with the Department of Computer Science and Engineering, and with the Visual Computing Laboratory, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093-0114. E-mail: [email protected]. WWW: http://wwwcse.ucsd.edu/users/ssantini

riphery, but reveals a property of the mapping from the ganglion cells to the primary visual cortex. For instance, measurements in foveal and peripheral grating acuity reveal that cone density varies of a factor of 10 from the fovea to the periphery [3], [22], not enough to account for the anatomical data. Moreover, anatomical evidence suggests that there are about 1,000 striate cells per incoming LGN axon at the fovea, but only about 100 at the far periphery [26]. Combining this with estimates of ganglion cells to LGN convergence, it turns out that there are roughly 100 times more striate cells per cone in the fovea than in the far periphery. It is of great interest to investigate what kind of localized self-organizing mechanism can generate such maps. The connections along the central visual pathways determine a homeomorphism between different areas that, while maintaining the topological relations between nearby regions emphasizes some regions over others. This aspect of topographic maps is explained by the theory of Kohonen maps by proving that the distribution of units at equilibrium is a monotonically increasing function of the probability distribution of stimuli in the input space [21], [20]. It is more difficult for the theory of Kohonen maps, to give a proper explanation of the homeomorphic nature, and the related neighborhood preservation property of the map. The Kohonen map is an embedding of a discrete set into a continuous space, and it is not clear how to define the continuity of this embedding. One interesting solution [5], [4] is based on the definition of a similarity function on the map and in the input space. Then, a measure of the neighborhood preservation properties of the map can be defined as the correlation between the similarity of a pair of elements of the map and the similarity of their embeddings in the input space. This value is high if points with high similarity on the map are mapped to points with high similarity in the input space. A different definition states that a mapping is continuous if the probability that two units respond to a stimulus from the same object decreases as the distance between the two units, measured on the map, increases [23]. The Self-Organizing field, on the other hand, is an embedding of a manifold into another, and continuity can be defined in a natural way. The definition of continuity induced by the embedding is consistent with the definitions introduced above for the discrete map. Another problem for which the self-organizing field can provide a valuable tool is the explanation of the laminar organization of certain areas of the primary visual cortex. The primary visual cortex (area V1), for instance, is divided into 6 layers—layer 4 is further divided into 4 sublayers—. Axons from the LGN project to the spiny stellate cells in layers 4Cα and 4Cβ which form two maps of the


visual field. From these layers information is transmitted and passed to the other layers by the nonpiramidal cells, the other layers forming other maps of the visual field responding to different features for a given location. This connective structure gives rise to the columnar organization of the visual cortex [17]. A column (or cortical module) is formed by corresponding regions in the six layers which process data from the same region of the visual field. The Self-Organizing field proves that this structure is an equilibrium point for Kohonen-type learning rules whenever an input field is subject to the action of a transformation group. Finally, a few precisations. There is no clear agreement on what should be called the “dimension” of the Kohonen map. Some authors assume implicitly that the neurons are organized in a 2D sheet and say that a Kohonen map is m-dimensional when the dimension of the input space is m that is, when every neuron has m weights. In this paper, I will call “dimension” of the input map the dimension of the lattice connecting the neurons. According to this definition, a sheet of neuron is always a two-dimensional Kohonen network. A more precise definition of dimension is given in the following both for the Kohonen map and the Self-Organizing field. Also, in this paper, the Kohonen map is considered as a function from a lattice of neurons to the input space that positions each neuron in a given point in the input space. I will use Latin letters to indicate quantities belonging to the input space and indices ranging over the input space dimensions, and Greek letters to indicate quantities belonging to the Kohonen space and indices ranging on the dimensions of the Kohonen space. The paper is organized as follow: Sect. 2 is a restatement of the Kohonen algorithm: the algorithm is expressed in a form that allows an easy extension to the continuum. Sect. 3 introduces the Self-Organizing field, derives the equations describing the field evolution, and proves the basic theorems on the properties of the steady state solution of the field. Sect. 4 proves that, under certain circumstances, the Self-Organizing field can do the structural decomposition of high dimensional input spaces [14]. Sect. 5 discusses briefly the significance of the results of Sections 3 and 4 for the study of self-organization of maps in the brain.

2

element λ of the lattice, associate a point wλ ∈ M . For the Kohonen algorithm to work, we need to endow the lattice Λ with a metric, which we can do by using the following definition from lattice theory ([7]): Definition 1: Given a lattice Λ and two elements λ, ν ∈ Λ, we say that λ covers ν (ν−< λ) if: 1. ν ≤ λ, 2. there is no ξ ∈ Λ such that ν ≤ ξ and ξ ≤ λ. The chemical distance inside a lattice is defined as follows: Definition 2: Two elements λ, ν ∈ Λ are at chemical distance δ (∆c (λ, ν) = δ) if: 1. δ = 0 and λ = ν, 2. δ = 1 and λ−< ν or ν−< λ 3. δ > 1 and both the following are true: • ∃ξ ∈ Λ such that ∆c (λ, ξ) = 1, ∆c (ξ, ν) = δ − 1 • ∀µ 6= ξ : ∆c (λ, ξ) = 1, either ∆c (ξ, ν) is not defined or ∆c (ξ, ν) ≥ δ It is not difficult to see that the chemical distance has all the properties of a metric. With the notion of distance, it is possible to introduce in Λ the concept of neighborhood function: Definition 3: A function Ξ : Λ×Λ 7→ IR is a neighborhood function if Ξ(λ, ν) ≥ 0 and there exists a δ0 such that, for all λ, ν ∈ Λ, ∆c (λ, ν) > δ0 ⇒ Ξ(λ, ν) = 0

(1)

The largest δ0 such that (1) holds is the radius of the neighborhoods induced by the function Ξ. Let gij be the metric tensor of the manifold M . This induces a metric in M defined as: Z 1 2 d (x, y) = inf gij f˙i f˙j dt (2) f :f (0)=x,f (1)=y

0

where f : IR 7→ M is a curve in M joining the points x and y, and we have set def df . (3) f˙ ≡ dt The introduction of a metric tensor also endows the manifold with a connection ∂gil ∂gkl ∂gik Γhik = g hl + − , (4) ∂xk ∂xi ∂xl with a covariant derivative:

II. The Kohonen Map def

This section contains a brief description of the Kohonen map. Although there are excellent introduction to the map (every introductory book on neural networks has one; the reader may refer for instance to [10], [12], [8]), I included this introduction to present the main ideas behind the map in a language and with a formalism that is as close as possible to that that will be used in the next section for the SelfOrganizing field. I hope that the reader already familiar with the Kohonen map will find this a useful introduction to the less familiar model in the next section. Let M be a n-dimensional differentiable manifold, endowed with a metric structure, and Λ a lattice. To each

∇i u i ≡

∂uj + Γjmi um , ∂xi

(5)

and with a field of geodesics, q : IR 7→ M , defined by the equations d2 q j dq m dq i + Γjmi = 0. (6) 2 dt dt dt Given two points u, v ∈ M , let us call quv the geodesic joining them. In particular, there will be one geodesic such that quv (0) = u and quv (1) = v. Since quv : IR 7→ M , we have q˙uv (s) : IR 7→ Tq(s) M and, in particular, q˙uv (0) ∈ Tu M , i.e. i q˙uv (0) = quv ∂i . (7)


Since the curve quv is a geodesic, the vector q˙uv is translated by parallel displacement along the curve and therefore: i ∇i q˙uv =0

(8)

therefore, although the numerical values of the coordinates i q˙uv change from point to point on the curve, the “real” vector q˙uv does not change. In particular its length, which is a scalar, is constant. We can normalize the vector field q˙ by i q˙uv def . (9) uiuv ≡ kq˙uv (0)k Thus, uuv = uiuv ∂i .

(10)

is an unitary vector defining the direction of the geodesic i quv (s). The geodesic such that q˙uv = ui is parameterized with respect to the arc length. The presence of the metric tensor also induces a volume element ω in M . In the following, M will always be a probabilistic manifold (introduced in Appendix), for which it is Z ω=1 (11) M

The introduction of the probabilistic manifold is tantamount to the introduction, in an arbitrary manifold M 0 of a suitable probability distribution (see Appendix). Consider the discrete set L ⊂ M , consisting of the image of the points of Λ under the operator w: L = wΛ = {p ∈ M |∃ν ∈ Λ : wν = p}

(12)

L is the image of Λ under the immersion w. Given a point x ∈ M , the basis of the Kohonen theory is the determination of the point in L closest to the input point x. Let ζ(x) be such closest point. Moreover, let Ξ be a suitable neighborhood function over Λ. The evolution of the Kohonen network is described by a discrete time stochastic process. At time t, the sample x(t) is drawn with a prescribed probability p(x), and the immersions w are updated as: ∆i wν (x) = d(wν , x)uiwν ,x Ξ(wν , wζ(x) )

(13)

Equation (13) presents some difficulties in being extended to a continuous field. It represents a discrete process in which samples are drawn from the manifold M and applied to the Kohonen network. A better form for analysis can be obtained considering that, in general, we are not interested in the details of the trajectories followed by the neurons during learning, but just in the equilibrium position that the neurons reach at stochastic steady state at the end of the learning period. The following theorem characterizes this state: Theorem 1: The Kohonen network of eq. (13) is at stochastic steady state iff: Z d(wν , x)uiwν ,x Ξ(wν , wζ(x) )ω = 0 (14) M

3

where ω is the volume element induced by the Riemann metric g. Proof (sketch): From eq. (13), we have that Z Z 1 i d(wν , x)uwν ,x Ξ(wν , wζ(x) )ω = ∆i wν (x)ω (15) M M By constructionRof the probabilistic manifold M , an integral of the form M f (x)ω is equivalent to an integral in the Euclidean space weighted by the probability distribution of x, therefore: Z d(wν , x)uiwν ,x Ξ(wν , wζ(x) )ω = IE[∆i wν ] (16) M

from which the condition for stochastic steady state follows. 2 Based on this theorem, we can model the Kohonen network as a set of differential equations. The immersion wν changes and, thus, can be represented as a function wν (λν , t) : Λ × IR 7→ M that evolves with time according to: Z dw = d(wν , x)uiwν ,x Ξ(wν , wζ(x) )ω (17) dt M According to the view expressed in this section, a Kohonen map is a function w defined on a lattice (the map properly said) taking values in a manifold (the input space). This function evolves according to a set of differential equations in which a key role is played by neighborhood relations on the lattice, as well as by the metric of the input space. III. The Self-Organizing Field This section introduces a continuous model–the SelfOrganizing Field–which exhibits the same kind of behavior and is ruled by a similar learning law as the Kohonen map. The passage from the discrete map to the continuous field will give a better and deeper understanding of the properties underlying the map organization. In the discrete case, the Kohonen map was considered as a mapping from a lattice Λ to the input manifold M , characterized by the discrete set of points wν ∈ M , ν ∈ Λ. I will follow the same approach here but instead of being a discrete function, from a discrete set to a smooth manifold, the mapping will be a continuous function between two smooth manifold Let M be a m-dimensional probabilistic manifold, endowed with a Riemannian metric Φ with components gij , and a volume element Ω. Also, let Σ be a n-dimensional ˜ with components γνµ . manifold with Riemann metric Φ In analogy with the discrete case, I will use Latin letters to refer to quantities defined in M , and Greek letters to refer to quantities defined in Σ. Lowercase Latin indices span {1, . . . , m} and lowercase Greek indices span {1, . . . , n}. Let ψ : Σ 7→ M be a continuous immersion of Σ into M , ψσ ∈ M being the image of σ ∈ Σ under the transformation ψ, and let S = ψΣ. Indicate with (Υσ , β) a chart around σ, with coordinates (χ1 , . . . , χn ). Without loss of generality, we can assume


that β(σ) = (0, . . . , 0), and that χi are normal coordinates. Likewise, (Up , h) is a chart around p = ψσ with coordinates (x1 , . . . , xm ) such that h(p) = (0, . . . , 0) and xi are normal coordinates. When no confusion arises, the subscript of Υ and U will be dropped. Also, in analogy with the previous section, let d be the distance induces in M by the metric Φ, and δ : Σ×Σ 7→ IR+ ˜ be the distance induced in Σ by the metric Φ. A further specialization of the distance can be done by using the following theorem ([19], page. 187): Theorem 2: Let (x1 , . . . , xm ) be normal coordinates at a point p ∈ M . Then there exists a > 0 such that, if 0 < r < a, being B(p; r) a ball of center p and radius r, it holds: 1. Any two points in B(p; r) can be joined by a unique minimizing geodesic; 2. For any y, z ∈ B(p; r), d(y, z) is differentiable. This theorem ensures that, given s ∈ S, and a suitably close x ∈ M , the minimizing geodesic between the two is unique. I’ll say, in this case, that the couple (x, s) has the unique geodesic property. When theorem 2 holds, there are a number of other desirable properties that hold as well. For instance, in the hypotheses of theorem 2, it is possible to find a neighborhood N0 ⊂ Tp M such that the exponential mapping Expp : NP 7→ B(p; r) is a diffeomorphism ([19], page. 180). Just like in the previous section, call ζx the point of S such that d(x, ζx) = min d(x, s) (18) s∈S

Definition 4: Let M be a probabilistic manifold, and S ⊂ M a submanifold. If, for every x ∈ M , the unique geodesic property holds for the couple (x, x ¯), then M is concentrated around S. In terms of the manifold N underlying the probabilistic manifold M (see appendix), where the probability distribution f was defined, the definition implies that supp f is sufficiently concentrated around S to make the hypotheses of theorem 2 valid. One important property of ζ(x) in a concentrated manifold is the following: Theorem 3: If σ = ζ(x), then uψσ,x ∈ Tσ⊥ S

(19)

Proof: the proof is by contradiction. Suppose uψσ,x = γ + p, with p ∈ Tσ⊥ S and γ ∈ Tψσ S. Consider a curve q : [−1, 1] 7→ S, with q(0) = σ and q(0) ˙ = γ. Set d(t) = d(q(t), x)2 . In this case, we have: ˙ = −2huψσ,x , q(t)i d(t) ˙ Tψσ M

4

where the subscript Tψσ M indicates that the scalar product is computed in the tangent space of M at ψσ, and: ˙ d(0) = −2h(γ + p), γi Since γ and p are orthogonal, it is: ˙ d(0) = −2kγk2 < 0 Therefore, d(σ, x)2 cannot have an extremum along the curve q at t = 0. Since σ is an extremum if and only if it is an extremum along all the curves in S that pass through it, this proves the theorem 2 Finally, define the neighborhood function in Σ as Ξ : Σ × Σ 7→ IR+ Ξ[λ, ν] = F(δ(λ, ν))

(20)

where F is a non-negative, monotonically non-increasing function of its argument with compact support. With these quantities, let us define the function X : Σ 7→ Tψν M by associating to the point ν ∈ Σ the tangent vector X ∈ Tψν M whose components are given by: Z i X ν= Ξ ν, ψ −1 x ¯ d2 (ψν)uiψν,x Ω. (21) M

By this function, we associate to every point in Σ, a direction in Tψν M depending on all the points y ∈ M such that their projection y¯ belong to a given neighborhood of ν specified by Ξ. In eq. (21) there is the presence of a potentially discontinuous quantity: x ¯. When n < m, the association x 7→ x ¯ is – in certain circumstances – many-to-many, and the point x ¯ may suddenly “jump” from one location to another even for an infinitesimal change in x. This happens whenever the point x is at the same distance from two distinct points s1 and s2 in S, and these points are at minimal distance. In this case, an infinitesimal displacement towards s1 will cause x ¯ = s1 and an infinitesimal displacement towards s2 will cause x ¯ = s2 . However, we have the following property: Theorem 4: Let Z(x) = {σ ∈ Σ : d(x, σ) = d(x, ζ(x))}, and Q = {x ∈ M : |Z(x)| > 1}. Then Q has measure zero in M . Proof (sketch): The condition for x to be in Q is given by the following three relations, all defined from M × Σ × Σ → IR: d(x, ψσ1 ) − d(x, ψσ2 ) uψσ1 ,x uψσ2 ,x

= 0 ∈ Tσ⊥1 S ∈

Tσ⊥2 S

(22)

The second and third relations are the conditions for σ1 and σ2 to be projections, according to eq. (19), while the first equation states that the points are at the same distance. The second and third conditions in (22) state that the m


components of the projection of uψσ1 ,x on Tσ2 S are zero and therefore each one contains m conditions. Note that these conditions do not assure minimality, therefore the subset of M of points that satisfy (22) contains Q properly. The three conditions can be seen as the zero of a function g : M × Σ × Σ → IR2m+1 and therefore its solution set P is a submanifold of M × Σ × Σ of codimension 2m + 1 in M × Σ × Σ [15] and therefore has at most dimension n − 1. Since Q is contained in the projection of P over M , its dimension is also at most n − 1 and therefore Q is of measure zero in M . 2

5

Since the mapping ψ is continuous, it maintains certain topological properties of Σ, and transfers them to S. A noticeable example is the fundamental group: Theorem 6: ([2], page. 266) Let π1 (Σ; ν) denote the homotopy class of all loops at ν ∈ Σ. Then π1 (Σ; ν) is a group with product [f ][g] = [f ∗ g]1 .If ψ : Σ 7→ S is continuous, then ψ determines a homomorphism ψ∗ : π1 (Σ; ν) 7→ π1 (S; ψν)

(27)

by Since the set of singular points is of measure zero, integration eliminates the discontinuities and makes X into a continuous function although, in general, not smooth. If we regard ψ as the continuous analogous of the functions wν : Λ 7→ M , which defined the map in the discrete case, then Xν gives the direction in which ψν must evolve and therefore must be regarded as the learning rule for the Self-Organizing field. In the following it will be convenient to have the possibility to work with quantities defined entirely within M . To this end, define the field Y : S 7→ T M as: def

Y ≡ X ◦ ψ −1

(23)

The submanifold S changes with time as a consequence of the evolution of the immersion ψ. Let ψ(ν; t) be the immersion at time t. Definition 5: Let M be a probabilistic manifold. A submanifold S ⊂ M , homeomorphic image of a manifold Σ by ψ : Σ 7→ S is a Self-Organizing Field if ψ(ν; t) evolves according to: ˙ t) = Y ψ(ν; t) ψ(ν; (24) where Y (s; t) = (X ◦ ψ −1 )(s; t)

(25)

and X is defined as in eq. (21). Usually, we are interested in the stationary solutions of (24) that is, in steady state Self-Organizing Fields. The equilibrium states are characterized by the following theorem: Theorem 5: A submanifold S ⊂ M is a stationary solution of the Self-Organizing field equation (24) if and only if Y S ⊆ TS (26) Proof: Suppose the condition is true. In this case, Y is a vector field over S and, by (24), φ is the one-parameter group generated by Y . Because of this, the trajectory φ(σ; t) lies in S, which is therefore the invariant space of the SelfOrganizing field. If, on the other side, S is the invariant space of the SelfOrganizing field, then the trajectory φ(σ; t) lies in S and, from (24), it follows that Y is a vector field over S. 2

ψ∗ [f ] = [ψf ] (28) Thus S – the submanifold image of the Self-Organizing field – can only be a manifold with the same connectivity as Σ. When n = m and π1 (Σ; ν) = π1 (M ; ψν), we say that the Self-Organizing field has the possibility to achieve a perfect match. In the perfect match case, it is pretty easy to prove an analogous to the “co-density” property of Kohonen networks. In [20], Ritter proves that the density of neurons of a Kohonen map in a region R ⊂ M is proportional to IP[R]k , where IP is the probability distribution of input samples, and k is a suitable exponent. This property arises quite naturally for the SelfOrganizing field. Consider, for the sake of simplicity, the manifold IRn , endowed with a probability distribution IP[x], and an ndimensional Self-Organizing field Σ. The probability distribution IP[x] on IRn gives rise to the probabilistic manifold M. It is easy to see that, at equilibrium, the Self-Organizing def

field Σ will converge in such a way that ψΣ ≡ S = M since, in this case, d(ψσ, x) = 0, and thus, by (21), Xσ = 0, thus satisfying the hypothesis of Theorem 5. Consider a compact region U ⊆ IRn , with volume m(U ). By the definition of probabilistic manifold, it is immediate to see that the measure of the portion of Self-Organizing field whose image is in U is: Z Z ΩM = IP(x)ΩIRn (29) ιM U

U

where ι : M 7→ IRn is the natural immersion of M into IRn . This is exactly the property we were looking for: the higher the probability IP, the greater the volume of the Self-Organizing field that is enclosed in U . 1 If f and g are loops, with f (0) = f (1) = g(0) = g(1) = ν, [f ] denotes the homotopy class of f , i.e. the set of all the loops that can be transformed in the loop f by a continuous transformation. The concatenation [f ∗ g] is the loop:

def

[f ∗ g](t) ≡

f (2t) g(2t − 1)

0 ≤ t < 21 1 0), ∂y i = π n (xi ) (68) ∂xi The latter form is much more useful, since it allows to ex˜ as a functions of the coordinates press the coordinates of N of P . References [1]

Shun-Ichi Amari. Mathematical foundation of neurocomputing. Proceedings of the IEEE, 78(9):1443–1463, 1990. [2] William M. Boothby. An Introduction to Differentiable Manifolds and Riemannian Geometry. Pure and Applied Mathematics. Academic Press, 1975. [3] Christine A. Curcio, Kenneth R. Sloan, Orin Packer, Anita E. Hendrickson, and et.al. Distribution of cones in human and monkey retina: Individual variability and radial asymmetry. Science, 236(4801):579–582, May 1987. [4] Geoffrey J. Goodhill, Steven Finch, and Terrence J. Sejnowski. Quantifying neighbourhood preservation in topographic mappings. Technical Report INC-9505, Institute for neural computation, La Jolla, CA, 1995. WWW http://helmholtz.salk.edu/ geoff/. [5] Geoffrey J. Goodhill, Steven Finch, and Terrence J. Sejnowski. Optimizing cortical mappings. In David S. Touretzky, Michael C. Mozer, and Michael E. Hasselmo, editors, Advences in Neural Information Processing Systems, volume 8. MIT Press: Cambridge, MA, 1996. WWW http://helmholtz.salk.edu/ geoff/. [6] A. M. Granda and J. H. Maxwell. Neural Mechanisms of behavior in the pigeon. Plenum, New York, 1979. [7] George Gr¨ atzer. Lattice Theory. A Series of Books in Mathematics. W. H. Freeman and Company, 1971. [8] Robert Hecht-Nielsen. Neurocomputing. Addison-Wesley Pub. Co., 1989. [9] E. I. Knudsen, S. du Lac, and S. D. Esterly. Computational maps in the brain. Annual Review of Neuroscience, 10:41–65, 1987. [10] Teuvo Kohonen. Self-Organization and Associative Memory. Springer Series in Information Sciences. Springer-Verlag, 1987. [11] Teuvo Kohonen. The self-organizing map. Proceedings of the IEEE, 78(9):1464–1480, 1990.

9

[12] Bart Kosko. Neural Networks and Fuzzy Systems – A Dynamical Systems Approach to Machine Intelligence. Prentice Hall, Englewood Cliffs, NJ 07632, 1992. [13] D. Lovelock and H. Rund. Tensors, Differential Forms, and Variational Principles. Dover Books on Advanced Mathematics, 63. Dover Publications, Inc, New York, 1975, 1989. [14] Bruce MacLennan. Characteristics of connectionist knowledge representation. Technical Report CS-91-147, Computer Science Department, University of Tennessee, Knoxville TN, November 1991. [15] Antal Majthay. Foundations of Catastrophe Theory, volume 26 of Monographs, advanced texts and Surveys in Pure and Applied Mathematics. Pitman, 1985. [16] John H. Martin and Thomas M. Jessell. Modality coding in the somatic sensory system. In Eric R. Kandel, James H. Schwartz, and Thomas M. Jessell, editors, Principles of Neural Science, chapter 24, pages 341–352. Appleton & Lange, 1991. [17] Carol Mason and Eric R. Kandel. Central visual pathways. In Eric R. Kandel, James H. Schwartz, and Thomas M. Jessell, editors, Principles of Neural Science, chapter 30, pages 420–439. Appleton & Lange, 1991. [18] James T. McIlwain. Large receptive fields and spatial transformations in the visual system. International Review of Physiology, Neurophysiology II, 10:223–248, 1976. [19] T. Okubo. Differential Geometry. Monographs and Textbooks in pure and applied mathematics. Marcel Dekker, Inc, 270 Madison Ave. New York 10016, 1987. [20] Helge Ritter. Asymptotic level density for a class of vector quantization processes. IEEE Transaction on Neural Networks, 2(1):173–175, 1991. [21] Helge Ritter and K. Schulten. Convergence properties of Kohonen’s topology preserving maps: Fluctuations, stability and dimension selection. Biological Cybernetics, 60:59–71, 1988. [22] Jyrki Rovamo, Veijo Virsu, and Risto N¨ as¨ anen. Cortical magnification factor predicts the photopic contrast sensitivity of peripheral vision. Nature, 271(5640):54–56, January 1978. [23] Csaba Szepesv´ ari, L´ aszl´ o Bal´ azs, and Andr´ as L¨ orincz. Approximate geometry representation and sensory fusion. Neurocomputing, 1996. WWW: http://iserv.iki.kfki.hu/adaptlab.html. [24] B. Torr´ esani. Wavelets associated with representations of the affine Weyl-Heisenberg group. Journal of Mathematical Physics, 32(5):1273–1279, 1991. [25] David C. Van Essen and H. R. Newsome, William T.and Maunsell. The visual field representation in striate cortex of the macaque monkey: Asymmetries, anisotropies, and individual variability. Vision Research, 1984. [26] Hugh R. Wilson, Dennis Levi, Lamberto Maffei, Jyrki Rovamo, and Russel DeValois. The perception of form, retina to striate cortex. In Lothar Spillman and John S. Werner, editors, Visual Perception: The Neurophysiological Foundation. Academic Press, 1990.

Simone Santini received the Laurea degree in Electrical Engineering from the Universit` a di Firenze, Firenze (Italy) in 1990 and the M.Sc. degree in Computer Science from the University of California, San Diego in 1996. Currently, he is a Ph.D. candidate at the Department of Computer Science and the Visual Computing Laboratory at the University of California, San Diego. From 1990 until 1994 Simone was with the Dipartimento di Sistemi e Informatica, Universit` a di Firenze. He was visiting scientist at the Artificial Intelligence Lab of the University of Michigan in 1990 and at the IBM Almaden Research Center, San Jose, CA in 1993. Simone’s current research interests are models of preattentive similarity perception and similarity-based search in image databases.