Biol. Cybern. 82, 173±187 (2000)
Learning cortical topography from spatiotemporal stimuli J. Wiemer, F. Spengler, F. Joublin, P. Stagge, S. Wacquant Institut fuÈr Neuroinformatik, Ruhr-UniversitaÈt Bochum, D-44780 Bochum, Germany Received: 17 March 1999 / Accepted in revised form: 10 August 1999
Abstract. Stimulus representation is a functional interpretation of early sensory cortices. Early sensory cortices are subject to stimulus-induced modi®cations. Common models for stimulus-induced learning within topographic representations are based on the stimuli's spatial structure and probability distribution. Furthermore, we argue that average temporal stimulus distances re¯ect the stimuli's relatedness. As topographic representations re¯ect the stimuli's relatedness, the temporal structure of incoming stimuli is important for the learning in cortical maps. Motivated by recent neurobiological ®ndings, we present an approach of cortical self-organization that additionally takes temporal stimulus aspects into account. The proposed model transforms average interstimulus intervals into representational distances. Thereby, neural topography is related to stimulus dynamics. This oers a new time-based interpretation of cortical maps. Our approach is based on a wave-like spread of cortical activity. Interactions between dynamics and feedforward activations lead to shifts of neural activity. The psychophysical saltation phenomenon may represent an analogue to the shifts proposed here. With regard to cortical plasticity, we oer an explanation for neurobiological ®ndings that other models cannot explain. Moreover, we predict cortical reorganizations under new experimental, spatiotemporal conditions. With regard to psychophysics, we relate the saltation phenomenon to dynamics and interaction in early sensory cortices and predict further eects in the perception of spatiotemporal stimuli.
1 Introduction A functional interpretation of early sensory cortices is stimulus representation (Marshall et al. 1937; Merzenich et al. 1978). Stimuli are grouped to form representational units that are topographically represented in soCorrespondence to: J. Wiemer e-mail:
[email protected]
called cortical maps. The homunculus of the primary somatosensory cortex, retinotopy and orientation maps of the primary visual cortex, and frequency maps of the primary auditory cortex constitute well-known and wellstudied examples of such topographic representations. An increasing number of experiments reveal the plasticity of such maps (Jenkins et al. 1990; Allard et al. 1991; Garraghty and Kaas 1992). From a theoretical point of view, neural maps seem to be the result of selforganizing processes (von der Malsburg 1973), leading to an equilibrium that is based on the statistics of incoming stimuli (Kohonen 1982; Obermayer et al. 1992; Joublin et al. 1996). Furthermore, cortical maps may be understood in a dimension reduction framework (Durbin and Mitchison 1990). Certain neighborhood relations are preserved by these mappings of high-dimensional stimulus spaces onto the cortex. As a result of this preservation, local computations in stimulus space can also be performed locally in the cortex. In this context, the essential question arises according to which criteria neighborhood relations are mapped onto the cortex. Common models that simulate the formation and alteration of cortical maps consider only the spatial structure and the probability distribution of the stimuli (von der Malsburg 1973; Willshaw and von der Malsburg 1976; Kohonen 1995). In these models stimuli are drawn randomly, i.e. their temporal context is neglected. The resulting topographic structures depend on the coding of the stimuli and receptive ®elds (RFs) (Erwin et al. 1995; Mayer et al. 1998; Wiemer et al. 1999). Contrarily, we regard stimulus dynamics, i.e. the temporal course of incoming stimuli, to be of fundamental importance for the formation, alteration, and interpretation of cortical representations. Our point of view is based on the following arguments. 1. Recent neurobiological experiments concerning plasticity in the primary somatosensory cortex of adult monkeys demonstrate that synchronous stimuli are integrated (i.e. represented at one cortical location), whereas asynchronous stimuli [200±300 ms interstimulus interval (ISI)] are cortically segregated (i.e. repre-
174
sented at distant cortical locations; Wang et al. 1995; Spengler et al. 1996; Spengler et al. 1999). Here, the ISI plays a key role in guiding the processes of cortical reorganization (cf. Sect. 2). Conventional models of selforganization (e.g. Kohonen 1995) cannot explain these results (see Sect. 6.1 and Wiemer et al. 1998a,b). 2. Psychophysical experiments demonstrate systematic mislocalizations of localized somatosensory, visual, and auditory stimuli when their temporal distances lie in the range of up to several hundred milliseconds (Geldard 1982; Kilgard and Merzenich 1995; Cholewiak 1999). This phenomenon, called saltation, illustrates the continuous transformation of temporal distances into perceived spatial distances (cf. Sect. 5). The perceived spatial distance between two stimuli is composed of their real spatial distance and an ISI-dependent variation thereof. Small (large) ISIs lead to small (large) perceived distances. 3. An important aspect, which determines cortical representations, is the correlation between sensory and cortical dynamics (Garraghty and Kaas 1992), i.e. Hebbian learning (Brown et al. 1990; Weinberger 1996). However, sensory and cortical dynamics are not temporally separable, as assumed by previous models (von der Malsburg 1973; Kohonen 1995). Neural responses in the primary somatosensory and auditory cortex of the monkey for example, do not only depend on the currently applied stimulus but are additionally in¯uenced by earlier stimulations. Depending on the spatiotemporal (spectro-temporal) relation, such responses are inhibited or enhanced, even for ISIs up to 300 ms (Gardner and Costanzo 1980; Brosch et al. 1998). 4. Natural sensory stimulation develops continuously in time. Therefore, the temporal proximity of the incoming stimuli yields a metric that re¯ects the stimuli's relatedness and functional similarity. We argue that cortical representations could be formed and altered in accordance with such a time-based metric. Thereby, stimuli that follow each other closely in time can be regarded as belonging together and should be represented topographically close to each other (Wiemer et al. 1998a). We argue that the cortical reorganization observed by Spengler et al. (1996, 1999) and also by Wang et al. (1995) is the result of a self-organizing process based on spatiotemporal stimuli (Buonomano and Merzenich 1995). The experimental ®ndings and their interpretation are presented in Sect. 2. A model of self-organization that explains the observed cortical segregation of experimental stimuli is proposed in Sect. 3 and Sect. 4. It transforms temporal distances of the applied stimuli into spatial distances of the corresponding neural representations. These time-to-space transformations are achieved by a fundamental wave-like neural dynamics. This dynamics leads to shifts of neural stimulus responses that depend on the stimuli's spatiotemporal context. We speculate that such shifts contribute to the psychophysical saltation phenomenon. Our approach constitutes a framework in which this illusion can be explained from dynamics and interactions in primary sensory cortices. This is elaborated in Sect. 5. Our view is in accordance with the experimental ®nding that
neural responses in the primary somatosensory and auditory cortex depend on stimuli going back up to several hundred milliseconds. Furthermore, we predict features of cortical reorganization and psychophysical percepts under new experimental, spatiotemporal conditions (Sect. 4.3 and Sect. 5.2, respectively). With regard to cortical plasticity, our approach raises the following questions. To what extent does the formation of cortical topography rely on the temporal structure of the incoming stimuli? Does the transformation of temporal stimulus distances into cortical topography constitute a general principle of cortical selforganization? Furthermore, may this principle lead to a better understanding of the structure of early as well as of higher cortical areas, e.g. of the inferotemporal cortex (Tanaka et al. 1991; Wang et al. 1996)? Our approach is discussed in Sect. 6. We emphasize far-reaching implications (time-based interpretation of cortical topography, in¯uence on cortical reorganization, psychophysical saltation), indicate biological plausibility, and suggest extensions (more general stimuli and neural responses, learning of lateral connections). Finally, we summarize and present an outlook in Sect. 7. 2 Neurobiological experiment Many experiments on cortical plasticity can be understood in the context of self-organization based on stationary neural stimulus responses and Hebbian learning. In this section, we present neurobiological ®ndings that motivate an extension of the self-organizing processes to include the stimuli's temporal structure as well. 2.1 Experimental paradigm Spengler and colleagues (1996, 1999) trained monkeys on a tactile two-stimuli discrimination task for 5±7 months. In daily sessions of several hours, two vibrating bars (sinusoidal vibrations at 75 Hz, 80 ms duration and 100 lm amplitude) were applied to several ®nger segments. One of the bars stimulated synchronously the distal segments of the second, third, and fourth ®nger (denoted `bar A'), the other stimulated synchronously the proximal, middle, and partly the distal segment of the third ®nger (`bar B'). The two experimental stimuli overlapped slightly on the distal segment of the third ®nger (see the sketch of the experimental setup in Fig. 1a). In the task, the two stimuli were alternately applied with an ISI of 300 ms serving as background stimuli. As target stimuli the monkeys had to detect two consecutive stimuli of either bar A or bar B and retract their hand from the stimulator. The hand retraction was counted as (1) a `hit' when it occurred within the given reaction time window, (2) a `miss' when it was beyond the end of the reaction time window, and (3) a `false positive' when it preceded the presentation of the target stimuli (Fig. 1a). Hits were rewarded with
175
Fig. 1a±c. Neurobiological experiment. Monkeys were trained to discriminate two spatially overlapping tactile stimuli (`bar A', `bar B') that were alternately applied and temporally separated (interstimulus interval=300 ms) (a). The monkeys' task was to detect two consecutive stimuli of either bar A or bar B. After training, the animals' hand representations of the primary somatosensory cortex (layer IV, area 3b) were mapped electrophysiologically. A usual topographic representation with exclusively single segment receptive ®elds (RFs) is found on the control hemisphere (b). The experimental hemisphere reveals stimulus speci®c reorganizations (c). Synchronously stimulated ®nger segments are integrated while representations of the two experimental stimuli are segregated into dierent regions of the hand representation. The maps shown in b and c are reconstructed from a total of 230 and 231 microelectrode penetration sites, respectively
food pellets, misses and false positives were followed by cage light time out. 2.2 Experimental results The resulting hand representations in the primary somatosensory cortex (layer IV, area 3b) of the control and experimental hemisphere are shown in Fig. 1b and c, respectively. They were obtained by electrophysiological mapping in the anesthetized monkey after successful completion of the training (Wang et al. 1995; Spengler et al. 1996, 1999). The hand representation of the control hemisphere seems to be unaltered by experimental training. Figure 1b shows an orderly progression of cutaneous RF locations across lateral-to-medial (digit 1 to digit 5) and caudal-to-rostral (proximal to distal) surfaces of the hand digits. Borders between cortical representations of adjacent ®ngers and ®nger segments are distinct. In addition to the representation of glabrous digital skin surfaces, small islands of RFs located on the hand dorsum are scattered in the map. The representation of dorsal digits is fragmented and incomplete as in naive monkeys (Merzenich et al. 1987). In order to characterize the observed cortical reorganizations, we introduce the following terms: (1) representational distance is the spatial distance between cortical activities induced by two stimuli measured in cortical coordinates or in `parameter space' as, e.g. de®ned in population coding (Jancke et al. 1996); (2) integration is the fusion of dierent stimuli into one representation, or the reduction of their representational
distance; (3) segregation is the process of increasing representational distance. In contrast to the control hemisphere, the hand representation of the experimental hemisphere reveals extensive stimulus-speci®c reorganization. 1. Integration. In addition to single segment representation, some neurons have attained RFs extending over those ®nger segments that were synchronously stimulated in the experiment. The corresponding cortical regions where such neurons were found are marked as `bar A' ± (`bar B') speci®c multiple segment RFs in Fig. 1c. Concerning bar B, RFs extend over multiple segments of one ®nger (i.e. digit 3), concerning bar A, RFs even extend over multiple segments of dierent ®ngers. 2. Segregation. Although the experimental stimuli overlap spatially on the distal segment of the third ®nger, the corresponding cortical representations are not adjacent to one another but segregated into dierent regions of the hand representation. In the example in Fig. 1c, the segregation is revealed by a dorsal input band, i.e. a cortical region where the neurons' RFs are located on the back of the hand. The dorsal input band lies between the experimental stimuli's representations. It may result from unmasking of non-dominant dorsal aerents in the glabrous hand surface representation (see Schroeder et al. 1997). In another experiment, Wang et al. (1995) used a similar paradigm consisting of two parallel tactile bars, leading to similar results. In order to evaluate the origin of the neuronal changes, Wang et al. (1995) analyzed ventro-posterior thalamus response maps. They revealed no equivalent reorganization. Therefore, the representational plasticity appears to be of cortical origin.
176
3 The model A possible approach to interpret the presented experimental ®ndings lies in the assumption that knowledge about the trained task can in¯uence the topographic structure of the primary somatosensory cortex. Topdown processes could evoke task-speci®c changes of cortical activities. Changed neural activities and correlation-based learning could lead to the observed topographic reorganizations. However, little is known about the top-down in¯uence which higher cortical processes exert on primary cortical areas. On the other hand, the generalization of stimulusinduced learning from purely spatial stimuli to spatiotemporal stimuli is a natural step. Therefore, we address the question whether the above-stated experimental ®ndings can be explained by self-organizing processes that are based on spatiotemporal stimuli. Integration of synchronous stimuli can be understood as a direct consequence of Hebbian learning. The cortical segregation phenomenon is not as easily explained. It requires additional mechanisms, although not necessarily non-Hebbian learning. Our explanation in this section and in Sect. 4 is based on Hebbian learning and cortical dynamics. 3.1 Spatiotemporal stimuli We believe that the dynamics of incoming stimuli re¯ects the stimuli's relatedness with regard to their functional
meaning, and that stimulus dynamics is therefore important for the learning of topography. Accordingly, we model the ¯ux of incoming stimuli as a sequence of stimuli, s1 ; s2 ; s3 ; . . . ; sn ; sn1 ; . . . ;
1
in which the temporal proximity of consecutive stimuli is expressed in a corresponding sequence of ISIs, isi1 ; isi2 ; . . . ; isin ; . . . :
2
The presentation time tn of a stimulus sn is given by tn
nÿ1 X
isin0
t1 0 :
3
n0 1
We remark that this leads to an inhomogenous discretization of time. The stimuli's spatial structure is high-dimensionally coded; stimuli are activity patterns over an array of sensors (see Erwin et al. 1995; Riesenhuber et al. 1996; Wiemer et al. 1999). 3.2 Network architecture The model consists of three two-dimensional layers (Fig. 2). The ®rst layer is an array of N1s N2s sensors (`s' for sensors). High-dimensionally coded stimuli sn are applied to this layer at discrete times tn . The second layer consists of N1s N2s neural units and possesses additional
Fig. 2. Network architecture and generation of layer-3 response. The model consists of three two-dimensional layers: a sensory array to which arbitrary shaped stimuli can be applied, a second layer that temporally integrates the activity of the sensory array, and a third layer consisting of units with plastic connections that bring forth topography. Every sensory unit (i; j) is topographically connected to the corresponding unit (i; j) of the second layer; the weights of these connections are ®xed and set to unity. Each layer-3 unit (k; l) receives input via connection weights wkl;ij from all layer-2 units. In this framework, a layer-3 response is generated as follows. At time tn the stimulus sA has evoked a layer-3 response cA . An interstimulus interval isin later a dierent stimulus sB is applied and leads to a layer-2 activity that consists of a strong sB component, in addition to a more or less attenuated sA component. Due to neural dynamics (not shown in this ®gure, see Fig. 3) and interaction, the resulting feedforward activation is sharpened (c0B ) and shifted to form a layer-3 activity cB that depends on the applied ISI isin
177
Fig. 3. Neural dynamics causing a shift of activity. Two stimuli sA , sB are consecutively applied with a given interstimulus interval (ISI) cf. Fig. 2. At time tn isin , the layer-3 activity cA
tn , generated by stimulus sA
tn at time tn , has propagated into its surround. The stimulus sB
tn isin (applied at that time) generates a feedforward activation that is sharpened by recurrent inhibition (denoted c0B ) and shifted towards the prevailing wave front c0A to form the corresponding layer-3 response cB
tn isin (see main text for details)
dynamic properties. Every unit
i; j of this layer is connected to the corresponding sensor
i; j via ®xed connections that are set to unity. Each of the N1c N2c (`c' for cortical) neural units of the third layer is connected to all units of the second layer. The corresponding connection matrix W
tn consists of elements wkl;ij
tn that determine how strongly layer-2 neurons
i; j can activate layer-3 neurons
k; l via feedforward connections at time tn . We denote the weight vector wkl that speci®es the selectivity of neuron
k; l as the RF of that neuron. The weight vectors are learned during the process of self-organization and determine the resulting topographic structure. RFs and stimuli are coded in the same way. Using high-dimensional coding, we can interpret elements of the connection matrix biologically as eective strengths of connections between sensors and cortical neurons. In addition, we can model stimulus-induced changes in the spatial RF shape that are experimentally observed (multiple segment RFs, cf. Sect. 2.2). Finally, high-dimensional coding will facilitate the transfer of our model to other ®elds of signal processing, e.g. the processing of visual and auditory stimuli. 3.3 Neural activity and dynamics Activities in layers 2 and 3 are subject to dynamics. At times of stimulus presentation tn , the sensory activity s
tn is fed into layer 2. Between two stimulus presentations at times tn and tn1 , layer-2 activity p
tn decays (Fig. 2). This type of dynamics leads to a time-dependent averaging over successive incoming stimuli. As a consequence of the dynamics, the layer-2 activity p
tn isin approaches continuously the superposition of the presented stimuli, s
tn s
tn1 , when the ISI isin tn1 ÿ tn is reduced to zero. The functional relevance of this layer's dynamics lies in an interpolation between successive stimuli (Wallis and BuÈltho 1999). Thereby, stimuli that follow each other closely in time are
associated. In the framework of Hebbian learning, this can lead to the formation of invariant object representation (see Edelman and Weinshall 1991; Wallis 1996). Activity is fed from layer 2 to layer 3 at the times of stimulus presentation. As usual, we obtain the feedforward activation of layer 3, cff
tn , by multiplying the connection matrix W
tn with the layer-2 activity p
tn : cff
tn W
tn p
tn :
4
The connection strengths W
tn result from a learning process after n ÿ 1 adaptation steps at times t1 ; . . ., tnÿ1 (cf. Sect. 4). The layer-3 activity c
tn builds up from the current feedforward activation cff
tn , and the activity state of this layer, that has evolved out of the earlier activity c
tnÿ1 from time tnÿ1 to tn . The evolution of layer-3 activity between two stimulations is of wave-like type; excitation propagates into its neural surround. This dynamics may result from local interactions, e.g. between excitatory and inhibitory neurons (Wilson and Cowan 1973). The dynamics is fundamental in the sense of a general principle of locality; information can only be submitted with ®nite speed. In other words, ``eects propagate from point to neighboring point'' (Haag 1993). In the case of a monomodal and rotation symmetric layer-3 activity, an `elementary wave' propagates in two dimensions as shown in Fig. 3. Only idealized responses of this type will be used in our simulations (cf. Sect. 4). Assuming linear superposition, the dynamics corresponding to general activity patterns can be reduced to the superposition of elementary waves. As an analogue, one may think of water waves that a raindrop generates when it falls into a puddle.1 The functional relevance of layer-3 dynamics lies in the transformation of temporal coding into spatial 1 Water waves can reveal non-linear dynamics; they do not generally obey Huygens's principle of linear superposition (Mehaute 1976). We refer to water waves as a vivid example to illustrate the simplicity of this type of dynamics
178
Fig. 4a,b. ISI dependence of integration and segregation. The layer-3 response to a stimulus sB is shifted according to its current state of dynamics. For small ISIs, the wave evoked by an earlier stimulus sA is located near its initial response. A shift of activation towards the wave front leads to integration (a). In the case of large ISIs, the wave has already passed the location of maximal sB -induced feedforward activation. Therefore, a shift towards the wave front means segregation (b)
coding. In combination with feedforward activation, the wave re¯ects a spatiotemporal stimulus structure. It deforms feedforward activations and leads to layer-3 activities that incorporate spatiotemporal stimulus correlations. 3.4 Shifts of activation A layer-3 response c
tn is obtained by assuming the following principle of interaction: the feedforward activation cff
tn is sharpened by local recurrent inhibition (von der Malsburg 1973; Wilson and Cowan 1973; Amari 1980) and shifted towards the current wave front of this layer's dynamics (Fig. 3). In a neural ®eld model, and also in biological systems, these shift vectors can result from an asymmetry in neural excitability due to the dynamical state of the layer. The lengths of these shifts depend on the distances between the wave front and the center of feedforward activation. These distances result from the spatial distance between stimulus representations, the wave velocity, and the ISI used. The shift may also depend explicitly on the ISI re¯ecting the wave's time-dependent attenuation. An example for these interactions will be presented in Sect. 4.1. The interaction described above leads to two opposing situations that we denote integration and segregation. When the ISI between the two stimuli sA s
tnÿ1 , sB s
tn is small, the wave induced by sA is still located near its initial response. In this case, a shift of feedforward activation towards the wave front implies a decrease in representational distance, i.e. integration (Fig. 4a). On the other hand, when the ISI between the two stimuli sA , sB is large, the wave has already passed the cortical location of maximal feedforward activation induced by sB (Fig. 4b). Therefore, a shift towards the wave front leads to an increase of the representational distance, i.e. segregation. 3.5 Adaptation of weights In our model, topography is learned by the adaptation of the eective synaptic weights wkl;ij from the second to
the third layer. We assume Hebbian learning, i.e. weights are adapted as a function of the correlation between presynaptic and postsynaptic activity. Accordingly, we apply normalized Hebbian learning with layer-2 activity as the presynaptic component and layer-3 activity as the postsynaptic component. In addition, a second learning rule is applied in order to incorporate forgetting. The process is called homosynaptic depression: presynaptic activity in the absence of above-threshold postsynaptic activity causes synaptic depression (Brown et al. 1990; see also Sect. 4). Figure 5 illustrates the combined eect of postsynaptic shifts and homosynaptic depression leading to a decay of synaptic weights. 4 Numerical experiment The essential ideas of our approach were presented in the previous section. We now put this ansatz into concrete terms (Sect. 4.1). Our ®rst application is the `ontogenetic' formation of topographic structure (Sect. 4.2). We then apply our model to the experiment
Fig. 5. Illustration of unlearning. Presynaptic activity in the absence of (above-threshold) postsynaptic activity causes synaptic depression (`homosynaptic depression'). A shift of layer-3 activity exposes a cortical region that is highly selective for the applied layer-2 activity pattern. Especially in this region, large synaptic weights are reduced leading to the development of new neural selectivities
179
by Spengler and colleagues (Sect. 4.3). Finally, by changing the spatiotemporal structure of the experimental stimuli, we are led to predictions of cortical reorganization under new experimental conditions. 4.1 A simple form of the proposed model In our simulations, each of the three two-dimensional layers possesses N N units (N 30). In the following, layers 1, 2, and 3 are named sensory, precortical, and cortical layer, respectively. However, layer 2 may also represent an early cortical area, e.g. with regard to the representation of high-dimensional visual stimuli (Sect. 6.3). As sensory activity patterns, we apply highcoded Gauss-shaped stimuli s
tn dimensionally sij
tn i;j2f1;:::;N g , !
i ÿ in 2
j ÿ jn 2 sij
tn exp ÿ ;
5 2r2s1 2r2s2 T
centered at y
tn
in ; jn with widths rs1 , rs2 . At any time tn , the last two stimuli s
tnÿ1 , s
tn are temporally integrated to form the precortical activity p
tn s
tn
1 ÿ isinÿ1 =sI H
sI ÿ isinÿ1 s
tnÿ1 ;
6
with ISI isinÿ1 tn ÿ tnÿ1 , decay sI , and Heaviside function H() (H
x 0 for x 0, H
x 1 for x > 0; sI 4 in our simulations). We assume cortical responses c
tn ckl
tn to be of Gauss-shaped form !
k ÿ kn 2
l ÿ ln 2 ;
7 ckl
tn exp ÿ 2r2c centered at locations x
tn
kn ; ln T with width rc . During the time interval tnÿ1 ; tn , the earlier response c
tnÿ1 to the stimulation at time tnÿ1 propagates as a radially localized and symmetric wave with constant velocity v 1 into its surround (Fig. 3).2 At time tn , the location of the cortical response x
tn to the stimulus s
tn is computed by adding a shift D
tn to the center of cortical feedforward activation xff
tn
knff ; lffn T : x
tn xff
tn D
tn :
8
We de®ne the center of cortical feedforward activation xff
tn as the average position using feedforward activations cffkl
tn as weighting factors (see Eq. 4) and taking only high activations cffkl
tn > hff maxkl cffkl
tn into account (hff 0:8). The shift D
tn points from xff
tn towards the cortical wave front along the shortest connecting path [endpoint xw
tn ]. The length of this path determines the magnitude of the shift according to a non-linear function:
9 D
tn f jjxw
tn ÿ xff
tn jj : 2
The spatial scale is de®ned by the number of neurons, the choice v 1 introduces a time scale
We introduce two parameters that determine this function; the maximum length of the shift, j, and its decay constant for large distances, mj . The shift length is proportional to the distance between wave and feedforward activation for distances that are smaller than j. For larger distances, it decreases exponentially: x : 0xj
10 f
x j expÿ
x ÿ j=
mj j : x > j . We choose j 3 and mj 5. Having computed presynaptic and postsynaptic activity, p
tn and c
tn , we now change the synaptic strengths from the precortical to the cortical layer using the following learning rules: if a neuron
k; l of layer 3 responds strongly to a given input p
tn , i.e. ckl
tn hc (postsynaptic modi®cation threshold: hc 0:3), then its synaptic weights are adjusted towards this input: w0kl;ij
tn1 wkl;ij
tn ackl
tn pij
tn :
11
This learning rule is strictly Hebbian; the change of synaptic weights is proportional to presynaptic and postsynaptic activity. On the other hand, if a cortical neuron responds too weakly, ckl
tn < hc , its selectivity for the current input is reduced:
12 w0kl;ij
tn1 wkl;ij
tn 1 ÿ apij
tn : This applies to the neuron
k; l's connections
kl; ij that ful®ll wkl;ij
tn > hw maxij fwkl;ij
tn g (we choose hw 0:75). Presynaptic activity causes synaptic depression if it is not accompanied by above-threshold postsynaptic activity. A learning rule which incorporates unlearning was earlier introduced by Cooper et al. (1979) in order to explain the eect of visual experience on the speci®city of cortical neurons. This concept was elaborated by Bienenstock et al. (1982) who employed a variable postsynaptic modi®cation threshold that adapts according to averaged postsynaptic activity (Bear et al. 1987). In the following, it will suce to apply a ®xed postsynaptic modi®cation threshold. Each learning step is completed by multiplicative weight normalization: wkl;ij
tn1 P
w0kl;ij
tn1 ; 0 i0 j0 wkl;i0 j0
tn1
13
(von der Malsburg 1973). Thereby, the total aerent synaptic strength towards each cortical neuron is kept constant during learning: X wkl;ij
tn 1 for all tn :
14 ij
The proposed learning algorithm is outlined in Fig. 6. 4.2 Simulation of `ontogenesis' In the present subsection, we apply our model to idealized `natural' spatiotemporal stimuli. This leads to a topographic map that is in equilibrium with the pool
180
Figure 7a illustrates the formation of topography; at three dierent time steps (n 5 103 , 3 104 , 5 104 ), the cortical lattice is projected into stimulus space. Each neuron is represented by its RF center; centers belonging to neighboring units are connected by straight lines. Starting with randomly initialized synaptic weights (n 1), localized RFs are learned and arranged in topographic order, leading to a state of equilibrium. The synaptic weights, wkl;ij
nf 2 , and ®nal parameters a af , rc rf of the resulting network will be used in the following simulations of post-ontogenetic plasticity. Fig. 6. Sketch of the learning algorithm
4.3 Simulation of post-ontogenetic plasticity
of stimuli used. The results will serve as a starting point for the simulations of post-ontogenetic plasticity. We have argued that the relatedness of sensory activity patterns is re¯ected by their temporal nearness. In the simplest case of monomodal stimuli, this relation reduces to a correlation between position and time. Here, we choose Gauss-shaped stimuli of ®xed width (rs1 rs2 2) and at random (uniformly distributed) positions yn
in ; jn T 2 rs1 1; N ÿ rs1 2 . The ISI between two consecutive stimuli sn at yn and sn1 at yn1 is set proportional to their spatial distance isin
1 jjy ÿ yn1 jj : vs n
15
This proportionality describes approximately the average spatiotemporal correlations of natural somatosensory stimuli. It expresses the continuous development of local stimulations. The proportionality factor 1=vs (written as an inverse velocity) is selected to correspond to the previously ®xed sizes of the cortical layer and the stimuli's position range fvs
N ÿ 1 ÿ 2rs1 =
N ÿ 1 0:86g. In order to obtain global topographic order from randomly initialized weights, the learning rate, a, and the width of cortical response, rc , should decrease monotonicly during `ontogenesis'.3 We choose a n=n f f1 : n nf 1 a
n ai
ai
16 af : n > nf 1 . with a
1 ai 1, initial learning rate; a
n nf 1 af 0:002, ®nal learning rate (Fig. 7b; Ritter et al. 1990; Kohonen 1995).4 The duration of `ontogenesis' is set to nf 2 5 104 (nf 1 0:9 nf 2 ). The width of cortical response rc is reduced analogously; ai and af are substituted by ri 15
N =2 and rf 1:2, respectively. The choice of parameters is in accordance with Kohonen (1990); see also (Wiemer et al. 1999). 3 The question to what extent biological topographic structures are learned by the correlation of neural activity cannot be answered in general. Therefore, we assume random initialization as the `worst case'. Global order can alternatively be achieved by `polarity markers' (Willshaw and von der Malsburg 1976) 4 The learning rate a does not vanish but is kept at a small level af re¯ecting post-ontogenetic plasticity
In the experiment by Spengler et al. (1996), two classes of tactile stimuli have to be distinguished; the laboratory animals received experimental stimuli during daily training sessions and natural stimuli between two sessions. Accordingly, we subdivide the series of stimuli applied in our simulations into alternating subseries of experimental and natural stimuli (nexp 16, nnat 4). The two tactile bars A and B used as experimental stimuli (Fig. 1) are approximated by slightly overlapping, elongated Gaussian functions with half-widths rAsi , rBsi , i 1; 2 (rAs1 rBs2 5, rAs2 rBs1 2) located at yA , yB [Fig. 9a: yA
15; 11; yB
15; 19]. In accordance with our argumentation, we choose the ISI of the experimental stimuli, isiA;B , to be large compared to the stimuli's representational distance, i.e. we choose isiA;B > dAB
0;
v 1 :
17
The initial (post-ontogenetic) representational distance between the centers of feedforward activations, xff;A
0 and xff;B
0, is given by dAB
0 jjxff;A
0 ÿ xff;B
0jj. The activations are induced by the two stimuli A, B before the learning experiment (n is set to zero after `ontogenesis', isiA;B 16). A subseries of `natural' stimuli is constructed as in Sect. 4.2, with the temporal distances again being proportional to the spatial distances. In order to simulate the experiment, we apply the resulting series of stimuli to our model until a state of equilibrium is reached with regard to the representational distance of the experimental stimuli (Fig. 10). The numerically obtained cortical map reveals characteristic features that are neurobiologically observed. 1. Integration. Neural units have attained increased RFs that coincide with the extent of the experimental stimuli (Fig. 8). This result is a direct consequence of Hebbian learning. 2. Segregation. The representational distance between the two experimental stimuli is increased (Fig. 9). This result occurs as a consequence of wave-like cortical dynamics and Hebbian learning. Due to the unlearning by homosynaptic depression (Eq. 12), cortical neurons located between the experimental stimulus representations have attained less pronounced tuning curves. Former dominant synaptic weights are weakened and non-dominant connections
181
Fig. 7a,b. Simulation of `ontogenesis'. Each neuron is represented by its receptive ®eld (RF) center: centers belonging to neighboring units are connected by straight lines. Starting with randomly initialized synaptic weights (n 1), localized RFs are learned and arranged in topographic order leading to a state of equilibrium (a). The temporal course of learning rate a is shown in (b)
gration and segregation (Fig. 4). In order to demonstrate its eect, we rerun the numerical experiment described above with a dierent `small' experimental ISI, isi0A;B < dAB
0 (isi0A;B 4). In this case, the experimental stimuli's representational distance is consecutively reduced during the learning process, saturating at dAB
n 105 v isi0A;B (Fig. 9d, 10b). This cortical integration illustrates our prediction that temporal stimulus distances may be transferred into representational distances to form cortical topography. 5 Psychophysics
Fig. 8. Reorganization of RFs. Radially symmetric RFs result from `ontogenesis' as we show exemplarily for two neurons (left column, a). Simulating the experiment by Spengler et al. (1996) we observe stimulus induced shifts of RFs (cortical segregation) as well as changes in RF size and symmetry (feedforward integration, right column)
strengthened. We consider this phenomenon to be analogous to the formation of a dorsal input band observed experimentally (Sect. 2 and Wang et al. 1995). The neurobiological experiment reveals cortical reorganization at only one ®xed time chosen by the experimentalist for the electrophysiological mapping procedure. In the numerical experiment, we can additionally analyze the temporal course of the reorganization; we observe a continuous shift of cortical representations, saturating at a representational distance dAB
n 6:4 104 v isiA;B (Fig. 10a). Moreover, we are in a position to continue the experiment in the framework of our simulations by applying exclusively `natural' stimuli. Figure 9d shows the resulting reversibility, i.e. the system regains its initial state of equilibrium (apart from ¯uctuations). Its temporal course is included in Fig. 10 (n > 6:4 104 ). In our simulations, the experimental ISI constitutes the decisive quantity that dierentiates between inte-
We have introduced a model of cortical self-organization as a possible explanation for the neurobiological ®ndings presented in Sect. 2. It leads to shifts of cortical activation that depend on the spatiotemporal conditions of the incoming stimuli. In this section, we summarize psychophysical results that support such cortical processes. 5.1 The saltation phenomenon Spatially separated stimuli applied to the skin are systematically mislocalized in perception when their ISI lies in the range of up to several hundred milliseconds. This phenomenon is called saltation (Geldard and Sherrick 1972). It illustrates the transformation of temporal stimulus distances into perceived spatial distances. In a simple paradigm, Geldard and colleagues (1972) applied three stimuli of equal intensity to two loci of the skin, e.g. 5 cm or 10 cm apart on the back of the forearm (Fig. 11). The ®rst stimulus s1 was given at the locus l1 to assure the readiness of the experimental subjects. About 800 ms later, the second stimulus s2 was applied to the same locus l1 . An ISI isi2 later, the third stimulus s3 was applied to the second locus l2 . The subjects' task was to judge the position of the second stimulus. This estimated position varied systematically with the ISI isi2 between the second and third stimulus; small (large) ISIs lead to large (small) perceived shifts of the second
182
Fig. 9a±c. Topographic reorganization. Top row Lattice of neural units in stimulus space. Bottom row: Cortical activation induced by experimental stimuli shown in superposed form. Starting from a state of equilibrium (n 0) that represents the adult cortex before training (a), experimental stimuli induce cortical segregation (b) or integration (d) depending on the experimental ISI. The application of exclusively `natural' stimuli leads back to the initial state; we present the reversibility of cortical segregation (c). An asymmetry in the shift of the experimental stimuli's representations results from the stimuli's dierent orientation (see also Fig. 10)
Fig. 10a,b. Temporal course of reorganization. We present the temporal course of cortical position (left column) and representational distance (right column) for `large' experimental ISI (segregation, a) and `small' ISI (integration, b). Cortical position and representational distance saturate according to the stimuli's spatiotemporal correlations (n 6:4 104 and n 105 , respectively). Thereafter, only `natural' stimuli are applied demonstrating reversibility. We remark that in our simulations segregation progresses faster than integration. This ®nding depends crucially on the interaction function f expressing the ISI-dependent shift of cortical activation [see (6)]
stimulus towards the third stimulus. The perceived spatial distance between the two stimuli was found to be a monotonicaly increasing function of their ISI. We remark that the saltation phenomenon is clearly separable from apparent motion. In saltation, a mislocalized stimulus gives a concisely localized impression. Its position is determined by the ISI used. In contrast, apparent motion consists of ``a somewhat broadly localized, continuous, unbroken sweep'' (Geldard 1982). Geldard and colleagues analyzed the saltation phenomenon extensively by the described paradigm, which they called ``reduced rabbit''. They could demonstrate that saltation is not generated in the periphery but that it is a phenomenon of the central nervous system (Geldard 1982). Saltatory jumps penetrate, for example, anesthetized skin areas and they do not cross the body's midline, i.e. they re¯ect the brain's functional architecture in early sensory processing. Furthermore, saltation seems to be a universal phenomenon of sensory processing; visual and auditory analogs are described in Geldard and Sherrick (1974) and Hari (1995). Saltatory jumps can also be generated by monocular stimuli presented to the two eyes at slightly dierent positions in the visual ®eld (Geldard 1976). Geldard's ``reduced rabbit'' was symmetrized by Kilgard and Merzenich (1995). They introduced a fourth stimulus s4 at the second locus (Fig. 11b). While the perceived spatial distances between second and third stimuli were in accordance with Geldard's ®ndings, na-
183
Fig. 11a±c. Experimental paradigms for psychophysical saltation. Geldard (1976, 1982) introduced the `reduced rabbit paradigm' consisting of three stimuli at two loci: a ®rst stimulus s1 at locus l1 to assure readiness, a second stimulus s2 about 800 ms later also at l1 , and a third stimulus s3 an ISI isi2 2 [0, 300] ms later at the second locus l2 (a). Experienced experimental subjects perceived the locus of s2 to be shifted towards the attractant s3 . Kilgard and Merzenich (1995) symmetrized the `reduced rabbit paradigm' by the introduction of a fourth stimulus at the second locus (b). Naive subjects reported a mislocalization of both stimuli s2 , s3 . The stimuli's center of mass was in¯uenced by attention or expectation. A reduced symmetric paradigm consisting of only two stimuli was used by Cholewiak (1999, c). Here, experimental subjects were only asked to judge the distance between stimuli (or the spatial extent of stimulation if only one stimulus was felt), not their loci. For symmetry reasons, we assume in c that both stimuli are shifted towards each other
ive subjects generally observed shifts of both of these stimuli, the second and the third stimuli being shifted towards each other. Moreover, Kilgard and Merzenich (1995) found that the centers of the perceived stimuli depend on attention or expectation. Thus, the perception of two localized stimuli seems to be decomposed into two components. One process determines the distance between the stimuli and is unaffected by attention or expectation. Accordingly, we assume this process to re¯ect processing in early (primary) cortical areas. A second process determines the stimuli's center of mass; it is aected by attention or expectation and probably involves higher cortical areas. Cholewiak (1999) analyzed interactions in the perception of spatiotemporally localized stimuli by a symmetric two-stimuli paradigm. He found that the perceived spatial distance dperc
s1 ; s2 between the two stimuli s1 , s2 is composed of their true spatial distance dtrue
s1 ; s2 and an ISI-dependent variation D(ISI) thereof dperc
s1 ; s2 dtrue
s1 ; s2 D
ISI ;
18
(Fig. 11c). These results ®t well to the ISI-dependent shifts of cortical activations that we introduced in our model of cortical plasticity in order to simulate the neurobiological ®ndings of Sect. 2. 5.2 Application of our approach Although the saltation phenomenon has been known for many years, there is still no systematic theoretical approach towards its understanding. Here, we propose to transfer the model of cortical self-organization presented above to the perception of spatiotemporal stimuli. This oers a framework in which systematic
time-dependent shifts of cortical responses result from cortical dynamics and interaction. Accordingly, we predict the following psychophysical percepts. 1. Saltation should also occur for other stimulus parameters (besides sensory spatial coordinates) that are topologically represented in early cortices (e.g. frequency in audition, orientation in vision). 2. Saltation should extent to segregation, i.e. to increased perceived distances for `large' ISIs, where `large' is relative to the stimuli's representational distance. The corresponding time range (ISIs of about 200±400 ms) has not been systematically analyzed so far. According to our model, the extent of such segregating shifts might be smaller than those observed for small integrating ISIs. This asymmetry could result from a timedependent attenuation of cortical excitation. 6 Discussion 6.1 Importance of temporal stimulus structure for cortical topography We argue that the temporal order and proximity of the incoming stimuli re¯ect the stimuli's relatedness and functional similarity. As topographic representations re¯ect the stimuli's relatedness, these temporal stimulus aspects are important for learning in cortical maps. In the case of high-dimensional visual stimuli, this idea is elaborated in Sect. 6.3. The very goal of sensory processing is to provide a basis for the generation of adequate behavior. An organism must react to changes in its environment according to its needs. Thereby, the time scales of perception and behavior are linked to each other. There
184
is no bene®t, for example, of highly time-resolved perception if the assembled information cannot be transferred into behavior. In ecient stimulus representations, not all possible stimuli are dierentiated. Instead, stimuli are grouped into behaviorally relevant classes. We argue that cortical areas could use the temporal proximity of the stimuli to form ecient representations. Stimuli that follow each other very closely in time should be represented together (i.e. in one class); their dierentiation is behaviorally irrelevant. Stimuli that typically possess behaviorally relevant temporal distances to one another should be dierentiated. Average ISIs may serve as essential criteria that guide the corresponding integration and segregation processes. Previous models of topographic self-organization are based on the spatial structure and probability distribution of the stimuli (von der Malsburg 1973; Kohonen 1995). At discrete times, stimuli are presented to the neural network and neural properties are successively adapted according to some learning rule. Typically, the stimuli are chosen randomly according to their probability distribution and independent of earlier stimulus choices. In this framework, there are only the two extremes of synchronous and asynchronous stimulus presentation. Any two consecutive stimuli are temporally separated, i.e. they do not interact in the generation of network responses. The eect of an incoming stimulus only depends on the actual weights (and the learning rate) and not on the activations due to former stimuli. In the experiment of Spengler and colleagues (1996), stimuli are applied that are adjacent in space (spatial overlap on the third ®nger) but not in time. Why is the spatial neighborhood of the experimental stimuli not transformed into a cortical neighborhood? We argue that the observed cortical segregation of the experimental stimuli results from their temporal separation (by an ISI of 300 ms). A self-organizing process that is restricted to spatial stimulus patterns cannot segregate the corresponding representations. Therefore, we presented an approach to self-organization that extracts topography from spatiotemporal stimuli. The model transforms average temporal stimulus distances into representational distances. Our simulations of the neurobiological experiment support this view. 6.2 Time-based interpretation of cortical maps The presented model of stimulus-induced learning exhibits ISI-dependent representational distances. This constitutes an interpolation of neurobiological ®ndings. In other words, we assume that the ISI dependence of representational distances is not con®ned to a mere dierentiation of synchronous and asynchronous stimuli, but that this dependence extends to the whole range from zero to several hundred milliseconds. Our ansatz oers a time-based interpretation of well known cortical topographies. First, hand representations in cortical area 3b of monkeys typically do not re¯ect the proportions of the sensory dimensions. Given
the topography of the cortical hand representation (Merzenich et al. 1978; Jenkins et al. 1990; Recanzone et al. 1992), we notice a compression in the rostro-caudal direction (along represented ®ngers) relative to an expansion in the medio-lateral direction (across the dierent represented ®ngers). According to our model, this distortion may re¯ect the fact that the average ISI between stimuli on adjacent segments of the same ®nger is smaller than the average ISI between stimuli on adjacent segments of dierent ®ngers. Second, the roughly logarithmic structure of the retinotopic projection onto the primary visual cortex of monkeys may correspond to spatiotemporal correlations in the retinal ¯ow ®elds; the decrease in cortical magni®cation from foveal to peripheral visual coordinates correlates with an increase of retinal velocity due to selfmotion (Schwartz 1980; Lappe and Rauschecker 1995). 6.3 Representation of high-dimensional stimuli The Gaussian functions used in our simulations as experimental stimuli are sucient to illustrate our ideas and to generate simple topographic maps. However, we believe that our approach would also be highly valuable for the representation of high-dimensional stimuli. The Euclidean distances of high-dimensional stimuli do not necessarily re¯ect their relatedness. One may think of dierences in retinal projections generated by the translation and rotation of three-dimensional objects. Under natural conditions of stimulation, the latter geometrical transformations result from self and object motion. This leads to dierent views of single objects that are linked by temporal proximity. The resulting temporal proximity could be used by visual systems, not only to learn invariances in the selectivities of single cells (Edelman and Weinshall 1991; Wallis 1996), but also to topographically represent dierent single object views next to each other, i.e. to represent stimuli with similar functional meaning spatially similarly. We see the advantage of such a topographic coding scheme in its robustness; interpolation and extrapolation of stimuli can be performed in a functional meaningful space. Neurons in the anterior inferotemporal cortex (IT) of monkeys respond selectively to moderately complex visual stimuli and cluster in columnar regions (Tanaka et al. 1991). In addition, the selectivities of these neurons are highly plastic, they seem to constitute a visual associative long-term memory (Miyashita 1988). Furthermore, optical imaging analysis of the functional organization in IT suggests a ``continuous mapping of related features over a region around 1 mm in size'' (Wang et al. 1996). Therefore, we hypothesize that the inferotemporal cortex may represent topographically typical three-dimensional views according to a time-based metric. 6.4 Mapping stimulus frequency onto the cortex So far, we have demonstrated how self-organizing processes can transfer temporal stimulus distances into
185
representational distances. However, it is known from neurobiological experiments that the probability distribution of the stimuli is also mapped onto cortical topography; more frequently applied stimuli tend to capture larger cortical areas (Recanzone et al. 1992; Elbert et al. 1995; Pantev et al. 1998). The latter phenomenon can be easily incorporated in our approach. If we consider plastic intracortical connections that are adapted according to Hebbian learning, intracortical connections can be strengthened by propagating waves. Accordingly, more frequently stimulated cortical regions develop stronger intracortical connections, resulting in faster propagation and correspondingly increased representational areas. 6.5 Maladaptive plasticity Our approach may be relevant in the context of maladaptive plasticity. The phenomena phantom limb pain, focal dystonia, and dyslexia, for example, may result from pathological changes in cortical topology (Merzenich et al. 1993; Flor et al. 1995; Byl et al. 1996). Therapy including systematic alterations of the temporal structure of sensory stimuli may prevent further topographic deterioration and/or may be bene®cial to re-establish topographic representations in eected primary cortices. 6.6 Biological plausibility of wave-like dynamics From a theoretical perspective, the assumed wave-like dynamics is of a fundamental type (Sect. 3.3). We remark that the assumed wave-like dynamic does not have to be radially symmetric in single-trial analysis in order to exert its in¯uence on the learning process. It may be superimposed by ¯uctuations that are random or due to varying uncontrolled conditions of stimulation. However, the dynamics should be observable at least on average over several trials. There is some experimental evidence that wave-like dynamics occurs in biological neural networks. It may be realized in dierent forms: horizontal propagation of activity (Tanifuji et al. 1994; Prechtl et al. 1997; Bringuier et al. 1999), diusion of a volatile substance through cortical tissue (e.g. nitric oxide; Krekelberg and Taylor 19965 ), the propagation and dynamic interaction of chemical substances (e.g. calcium; Garaschuk et al. 1998) or of non-inactivating natrium currents (Taylor 1993). In the case of neural activity waves, the assumed type of dynamics was; for example observed by Tanifuji et al. (1994), who studied the propagation of excitation in slices of rat visual cortex. Prechtl and colleagues (1997) analyzed neural responses in the visual cortex of unanesthetized turtles. Their single-trial analysis reveals the coupling of spatial and temporal aspects of stimulus5 NO diuses at a speed of 2 mm/s, but seems to be spatially limited to volumes of tissue of about 10 lm (Montague and Sejnowki 1994)
evoked activity, e.g. propagating wave fronts of depolarization and hyperpolarization. More recently, Bringuier et al. (1999) conducted in vivo intracellular recordings to analyze subthreshold responses evoked by stimuli outside the classical RF. They observed a linear relation between the latency of postsynaptically evoked depolarization and the stimulus eccentricity to the minimal discharge ®eld. Their ®ndings suggest a radial wave of activity spreading at a constant speed of about 100 mm/s over a radius of more than 10 mm.6 6.7 Saltation Our ansatz oers a framework in which psychophysical saltation can be partly attributed to interactions and dynamics in early sensory cortices. The psychophysical saltation phenomenon transforms temporal stimulus distances into perceived spatial distances. It seems to be a general feature of sensory cortical processing as it occurs in several modalities (Geldard 1982; somatosensory, visual, and auditory; see Sect. 5.1). The phenomenon may prove to be analogous to the ISI-dependent shifts of cortical responses introduced in our model of self-organization. Assuming that shifted cortical stimulus responses result in ISI-dependent cortical reorganizations and lead to saltation, we predict that saltation extends functionally to additional topographically represented parameters and temporally (for `large' ISIs) to segregation. We remark that the present experimental evidence for saltatory jumps is restricted to interpolation; illusory localized stimuli vary between the two sensorily stimulated loci. The prediction of segregation implies the prediction of extrapolation; for speci®c spatiotemporal conditions, stimuli should also be perceivable outside the stimulated region (Sect. 5.2). 6.8 Neural ®eld dynamics In our simulations, we have focused on the time scale of long-term learning related to cortical reorganization. Stimulus responses were computed according to some basic rules. They were not generated in a dynamic way resulting from explicitly simulated interactions between feedforward activation and intracortical inhibition and excitation. In addition, stimulus responses were restricted to monomodal functions. The presented approach is exclusively based on processes that act locally in cortical coordinates (or in a parameter space). Therefore, time-dependent neural ®elds, as proposed by Wilson and Cowan (1973), oer a framework for re®nement leading to more realistic cortical stimulus responses and also to the assumed type of 6 With regard to the neurobiological ®ndings presented in Sect. 2, we do not know whether the reorganizational process has already saturated or reached anatomical limits. Therefore, we can only estimate a lower bound of a few millimeters per second for the cortical velocity
186
fundamental dynamics. They can be applied to achieve numerical results that allow a quantitative comparison with psychophysical and neurobiological data (Jancke et al. 1996; SchoÈner et al. 1997). 7 Summary and outlook We have presented an approach to the learning of cortical topography from stimulus dynamics, thereby stressing the importance of temporal stimulus structure. The resulting model was applied to the simulation of a neurobiological experiment. We have demonstrated how spatiotemporal correlations can lead to topography and deduced quantitative neurobiological and qualitative psychophysical predictions. Expressed in more general terms, our work puts forward the following question: To what extent do biological neural systems make use of the temporal structure of incoming signals to built up and alter their representation? We believe that this issue is essential for biological neural systems in order to speed up signal processing and increase its robustness, i.e. to provide a sound basis for the generation of adequate behavior. We believe that experimental tests of our predictions may constitute a further step towards a better understanding of cortical plasticity and cortical stimulus representation. Acknowledgements. We thank T. Burwick for stimulating discussions. We are also grateful to B. Sendho for his comments on earlier drafts of this manuscript. The work was supported by grant DFG, SFB 509.
References Allard T, Clark S, Jenkins W, Merzenich M (1991) Reorganization of somatosensory area 3b representation in adult owl monkeys after digital syndaktyly. J Neurophysiol 66:1048± 1058 Amari S (1980) Topographic organization of nerve ®elds. Bull Math Biol 42:339±364 Bear M, Cooper L, Ebener F (1987) A physiological basis for a theory of synapse modi®cation. Science 237:42±48 Bienenstock E, Cooper L, Munro P (1982) Theory for the development of neuron selectivity: orientation speci®city and binocular interaction in visual cortex. J Neurosci 2:32±48 Bringuier V, Chavane F, Glaeser L, Fregnac Y (1999) Horizontal propagation of visual activity in the synaptic integration ®eld of area 17 neurons. Science 283:695±699 Brosch M, Schulz A, Budinger E, Scheich H (1998) Enhancement of neuronal responses in sequences of pure tones in macaque auditory cortex. Soc Neurosc Abstr 24:400 Brown T, Kairiss E, Keenan C (1990) Hebbian synapses: biophysical mechanisms and algorithms. Annu Rev Neurosci 13:475±511 Buonomano D, Merzenich M (1995) Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267: 1028±1030 Byl N, Merzenich M, Jenkins W (1996) A primate genesis model of focal dystonia and repetitive strain injury [...]. Neurology 47:508±520 Cholewiak R (1999) The perception of tactile distance: in¯uences of body site, space and time. Perception, in press
Cooper L, Liberman F, Oja E (1979) A theory for the acquisition and loss of neuron speci®city in visual cortex. Biol Cybern 33:9±28 Durbin R, Mitchison G (1990) A dimension reduction framework for understanding cortical maps. Nature 343:644±647 Edelman S, Weinshall D (1991) A self-organizing multipleview representation of 3d objects. Biol Cybern 64:209±219 Elbert T, Pantev C, Weinbruch C, Rockstroh B, Taub E (1995) Increased cortical representation of the ®ngers of the left hand in string players. Science 270:305±307 Erwin E, Obermayer K, Schulten K (1995) Models of orientation and ocular dominance columns in the visual cortex: a critical comparison. Neural Comput 7:425±468 Flor H, Elbert T, Knecht S, Wienbruch C, Pantev C, Birbaumer N, Larbig W, Taub E (1995) Phantom-limb pain as a perceptual correlate of cortical reorganization following arm amputation. Nature 375:482±484 Garaschuk O, Hanse E, Konnerth A (1998) Developmental pro®le and synaptic origin of early network oscillations in the CA1 region of rat neonatal hippocampus. J Physiol (Lond.) 507.1:219±236 Gardner E, Costanzo R (1980) Temporal integration of multiplepoint stimuli in primary somatosensory cortical receptive ®elds of alert monkeys. J Neurophysiol 43:444±468 Garraghty P, Kass J (1992) Dynamic features of sensory and motor maps. Curr Opin Neurobiol 2:522±527 Geldard F (1976) The saltatory eect in vision. Sensory Processes 1: 77±86 Geldard F (1982) Saltation in somethesis. Psychol Bull 92:136±175 Geldard F, Sherrick C (1972) The cutaneous ``rabbit'': a perceptual illusion. Science 178:178±179 Geldard F, Sherrick C (1974) Princeton cutaneous research project. Technical Report 24, Princeton University Haag R (1993) Local quantum physics, 2nd edn, Springer, Berlin Heidelberg New York Hari R (1995) Illusory directional hearing in humans. Neurosci Lett 189: 29±30 Jancke D, Akhavan A, Erlhagen W, Giese M, Steinhage A, SchoÈner G, Dinse H (1996) Population coding in cat visual cortex reveals nonlinear interactions as predicted by a neural ®eld model. ICANN'96, Proceedings. Springer, Berlin Heidelberg New York pp 641±648 Jenkins W, Merzenich M, Ochs M, Allard T, Guic-Robles E (1990) Functional reorganization of primary somatosensory cortex in adult owl monkeys after behaviorally controlled tactile stimulation. J Neurophysiol 63:82±104 Joublin F, Spengler F, Wacquant S, Dinse H (1996) A columnar model of somatosensory reorganizational plasticity based on hebbian and non-hebbian learning rules. Biol Cybern 74:275±286 Kilgard M, Merzenich M (1995) Anticipated stimuli across skin. Nature 373:663 Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43:59±69 Kohonen T (1990) The self-organizing map. Proc IEEE 73:1464± 1480 Kohonen T (1995) Self-organizing maps. Springer, Berlin Heidelberg New York Krekelberg E, Taylor J (1996) Nitric oxide in cortical map formation. J Chem Neuroanal 10:191±196 Lappe M, Rauschecker J (1995) Motion anisotropes and heading detection. Biol Cybern 72:261±277 Malsburg C von der (1973) Self-organization of orientation sensitive cells in the striata cortex. Biol Cybern 14:85±100 Marshall W, Woolsey C, Bard P (1937) Cortical representation of tactile sensibility as indicated by cortical potentials. Science 85:388±390 Mayer N, Hermann M, Bauer H-U, Geisel T (1998) A cortical interpretation of assoms. Int ICANN'98, Proc Springer, Berlin Heidelberg New York, pp 361±366 Mehaute BL (1976) An introduction to hydrodynamics and water waves. Springer, Berlin Heidelberg New York
187 Merzenich M, Kaas J, Suhr M, Lin C (1978) Double representation of the body surface within cytoarchitectonic areas 3b and 1 in ``si'' in the owl monkey (aotus trivirgatus). J Comp Neurol 181:41±74 Merzenich M, Nelson R, Kaas J, Stryker M, Jenkins W, Zook J, Cynader M, Schoppmann A (1987) Variability in hand surface representations in areas 3b and 1 in adult owl and squirrel monkeys. J Comp Neurol 258:281±296 Merzenich M, Schreiner C, Jenkins W, Wang X (1993) Neural mechanisms underlying temporal integration, segmentation and input sequence representation: some implications for the origin of learning disabilities. Ann NY Acad Sci 628:1±22 Miyashita Y (1988) Neuronal correlate of visual associative longterm memory in the primate temporal cortex. Nature 335:817± 820 Montague P, Sejnowki T (1994) The predictive brain: Temporal coincedence and temporal order in synaptic learning mechanisms. Learning Memory 1: 1±33 Obermayer K, Blasdel G, Schulten K (1992) Statistical-mechanical analysis of self-organization and pattern formation during the development of visual maps. Phys Rev A 45:7568±7589 Pantev C, Oostenveld R, Engelien A, Ross B, Roberts L, Hoke M (1998) Increased auditory cortical representation in musicians. Nature 392: 811±814 Prechtl J, Cohen L, Pesaran B, Mitra P, Kleinfeld D (1997) Visual stimuli induce waves of electrical activity in turtle cortex. Proc Natl Acad Sci 94:7621±7626 Recanzone G, Mersenich M, Jenkins W, Grajski K, Dinse H (1992) Topographic reorganization of hand representation in cortical area 3b of owl monkeys trained in a frequency-discrimination task. J Neurophysiol 67:1031±1066 Riesenhuber M, Bauer H-U, Geisel T (1996) Analyzing phase transitions in high-dimensional self-organizing maps. Biol Cybern 75:397±407 Ritter H, Martinetz T, Schulten K (1990) Neuronale Netze. Addison-Wesley, Reading, Mass SchoÈner G, Kopecx K, Erlhagen W (1997) The dynamic neural ®eld theory of motor programming: arm and eye movement. In: Morasso P, Sanguinet V (eds) Self-organization, computational maps, and motor control. Elsevier, Amsterdam pp 271± 310 Schroeder C, Seto S, Garraghty P (1997) Emergence of radial nerve dominance in median nerve cortex after median nerve transection in an adult squirrel monkey. J Neurophysiol 77:522±526
Schwartz E (1980) Computational anatomy and functional architecture of striate cortex: a spatial mapping approach to perceptual coding. Vision Res 20:645±669 Spengler F, Hilger T, Wang X, Merzenich M (1996) Learning induced formation of cortical populations involved in tactile object recognition. Soc Neurosci Abstr 22:105 Spengler F, Hilger T, Wang X, Merzenich M (1999) Cortical plasticity underlying tactile object recognition learning Tanaka K, Saito H, Fukada Y, Moriya M (1991) Coding visual images of objects in the inferotemporal cortex of the macaque monkey. J Neurophysiol 66:170±189 Tanifuji M, Sugiyama T, Murase K (1994) Horizontal propagation of excitation in rat visual cortical slices revealed by optical imaging. Science 266:1057±1059 Taylor C (1993) Na currents that fail to inactivate. Trends Neurosci 16:455±460 Wallis G (1996) Using spatio-temporal correlations to learn invariant object recognition. Neural Netw 9:1513±1519 Wallis G, BuÈltho H (1999) Learning to recognize objects. Trends Cogn Sci 3:22±31 Wang G, Tanaka K, Tanifuji M (1996) Optical imaging of functional organization in the monkey inferotemporal cortex. Science 272:1665±1668 Wang X, Merzenich M, Sameshima K, Jenkins W (1995) Remodelling of hand representation in adult cortex determined by timing of tactile stimulation. Nature 378:71±75 Weinberger SCN (1996) Evidence for the hebbian hypothesis in experience-dependent physiological plasticity of neocortex: a critical review. Brain Res Rev 22:191±228 Wiemer J, Burwick T, von Seelen W (1999) Self-organizing maps for visual feature representation based on natural binocular stimuli Biol Cybern, in press Wiemer J, Spengler F, Joublin F, Stagge P, Wacquant S (1998a) A model of cortical plasticity: integration and segregation based on temporal input patterns. ICANN'98, Proc Springer, Berlin Heidelberg New York, pp 367±372 Wiemer J, Spengler F, Joublin F, Stagge P, Wacquant S, Leonhardt R (1998b) A model of cortical plasticity based on temporal input sequences. Soc Neurosci Abstr 24:637 Willshaw D, von der Malsburg C (1976) How patterned neural connections can be set up by self-organization. Proc R Soc Lond B 194:431±445 Wilson H, Cowan J (1973) A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue. Biol Cybern 13:55±80
