The following article was presented as an oral presentation at the Conference ICANN 98: Wiemer J, Spengler F, Joublin F, Stagge P, Wacquant S: A Model of Cortical Plasticity: Integration and Segregation based on Temporal Input Patterns. In Niklasson L, Boden M, Ziemke T (Eds): ICANN’98, Proceedings of the 8th International Conference on Artificial Neural Networks, Sk¨ovde, Schweden, 2-4 September 1998, (I):367-372. Springer-Verlag London (1998)
A Model of Cortical Plasticity: Integration and Segregation based on Temporal Input Patterns J. Wiemer, F. Spengler, F. Joublin, P. Stagge, S. Wacquant∗ Institut f¨ ur Neuroinformatik, Ruhr-Universit¨ at Bochum, Germany e-mail:
[email protected]
Abstract Early cortical areas reveal plastic topographic structures. The formation and alteration of these so-called cortical maps by self-organizing principles is often explained by Kohonen-like algorithms [7]. However, recent experiments concerning learning related reorganization of Area 3b of somatosensory cortex [1, 2] demonstrate task-specific cortical changes which cannot be explained by Kohonen’s model. We therefore propose a model of stimulus induced cortical plasticity that takes the temporal structure of afferent inputs into account, i.e. temporal stimulus distances are transformed into spatial distances of their cortical representations. The simulations agree with the above cited experimental results and predict the alteration of sensory cortices for different sequences of input patterns.
1
Integration/Segregation Task
Early sensory cortices represent sensory inputs from the external world in a topological way. They are to a large extent subject to stimulus induced plasticity [1, 2, 3] leading to cortical representations in which some stimuli are integrated and others are segregated. In order to make our point clear we define: representational distance as the distance between cortical activites induced by two stimuli, measured in cortical coordinates or in ’parameter space’ (e.g. as defined in population coding), integration as the fusion of different stimuli into one representation, or the reduction of their representational distance, and segregation as the process of further differentiating between stimuli by increasing their representational distance. Efficient stimulus representations require to define which of the incoming stimuli should be integrated and which should be segregated. We call this the integration/segregation task. Integration and segregation processes can be controlled by attentional topdown processes, and/or self-organizing processes using properties of the incoming stimuli. We consider self-organizing processes and put forward the following ∗ This
work was supported by DFG, SFB 509.
2
bar A
b) Control
D5d
c) Experiment
D5p D5m
bar B
D5d D3d
Experimental Stimuli
Background Stimuli
D3d
D2d
Bar B Bar A
D1p
time
D1d
false positive hit miss
Temporal Course of Training
3p
D2p
D2mp D12d
D1p
light off
D3d
3m3d
D2d 2mp D2m
D2p Palm
reward
D2d
D2m
ISI =300 ms
80 ms
D4p D3d
D3p
Target Stimuli
D4m
D4d D3m
Palm
D5p
D5m
D4d
D4m D4p
D1d
M C
D2p
Palm
D2m
Fig
a) Paradigm
D1p
M
1mm C
Receptive Field Characteristics: dorsum multi segment RF * non-cutaneous RF
Bar A specific multiple segment/finger RF Bar B specific multiple segment RF D: digit, p: proximal, m: medial, d: distal
Figure 1: Neurobiological Experiment; Sketch of experimental paradigm used for training (a), cortical hand representation in primary sensory cortex of control (b) and experimental hemisphere (c) [1]. theses: • The average temporal distance between two stimuli is a good measure of their similarity. • Early sensory cortex is structured by a self-organizing process that transforms average temporal stimulus distances into spatial distances of their cortical representations.
2
Neurobiological Experiments
Recent experiments on tactile object discrimination reveal that synchronous stimuli are integrated and asynchronous stimuli (200-300 ms interstimulus interval) are segregated in the cortex [1, 2]. Spengler et al. trained owl monkeys in a tactile two-bar recognition task for 5-7 months (cf fig. 1a). As background stimuli two vibrating bars A and B were applied alternately with an interstimulus interval (ISI) of 300ms. As target stimuli the monkeys had to detect two consecutive stimuli of either bar A or bar B. The resulting plastic reorganization is shown in figure 1c: receptive fields (RFs) show an integration effect, extending over synchronously stimulated finger segments (see existence of multiple segment RFs), while representations of the two bars A and B are segregated into different regions of the hand representation. The sensory neighbourhood between the two stimuli is not transformed into a cortical neighbourhood of their representations. Wang et al. used a similar experimental paradigm consisting of two parallel tactile bars and leading to similar results. In addition, they analyzed ventro-posterior thalamus response maps revealing no equivalent reorganization. Therefore the representational plasticity appears to be of cortical origin.
SB
a) Cortical shift
SA
b) Algorithm Initialization
dAB dw
For each learning step Choose spatio-temporal stimulus D
t+ISI
t
ISI . VW
t+ISI
Compute: 1. pre-cortical activity 2. cortical feedforward activation 3. propagated wave 4. resulting cortical activity Normalized Hebbian learning Homosynaptic depression
Figure 2: Integration/Segregation Model; on-going cortical activity due to former stimulation SA influences the location of the cortical response to an in-coming stimulus SB (a). Sketch of the Integration/Segregation Algorithm (b).
3
The Model
Hebbian learning can be seen as the underlying mechanism of integration of synchronous stimuli [4]. But cortical segregation of asynchronous stimuli requires an additional mechanism. As above postulated we assume the segregation distance (i.e. representational distance) to be ISI-dependent. As a consequence, interstimulus intervals are expressed in spatial distances in the stimulus representation. This time-to-space conversion can be generated by the propagation of neural activity into the neighbourhood of a cortical response. Such a wave produces in combination with the new stimulus a time-dependent shift of the resulting cortical activity. It allows a Hebbian learning mechanism to occur at an ISI-influenced cortical location. Our model of cortical plasticity is a three-layer model: • A sensory array on which arbitrary shaped stimuli can be applied. In the following we choose Gauß shaped stimuli. • A pre-cortical layer which integrates the activity of the sensor array using a fixed weight feedforward connectivity. The dynamics of this layer creates decaying responses. • A cortical layer with plastic connectivity coming from the pre-cortical layer. Gauß shaped cortical activities with one fixed width are assumed. The cortical response is computed as follows (cf fig. 2a): After the application of a first stimulus at time t, the resulting cortical activity wave spreads on the cortex at a speed VW 1 . When a second stimulus comes at time t + ISI, its cortical activation is shifted by an amount D in the direction to the wave front. This shift depends linearly on the distance dW to the wave front when dW is small (compared to the width of cortical activity) and decreases exponentially for larger dW . We use two learning rules: 1 Biological
data suggests VW to be in the range of several mm/s.
• Hebbian feedforward learning (in Kohonen-type normalization): ∆W (x, y, t) = α c(x, t) (s(y, t) − W (x, y, t)) (t: time, x: cortical coordinate, y: pre-cortical coordinate, W : synaptic weight, c: cortical activity, s: pre-cortical activity, α: learning rate), • activity-dependent ’Homosynaptic Depression’ [8], i.e. presynaptic activity in the absence of postsynaptic activity causes synaptic depression: ∆W (x, y, t) = −ǫ s(y, t) W (x, y, t) when s(y, t),W (x, y, t) are above and c(x, t) below fixed thresholds (ǫ: unlearning rate). A sketch of the algorithm is given in figure 2b.
4
Simulation Results
We use the above described model to simulate the experiment of Spengler et al. (cf fig. 3a,b). In addition, we vary the interstimulus interval leading to a prediction about experimental conditions that have not been examined so far (cf fig. 3d). The initial weights reflect topographically ordered synaptic weights. They are learned with the integration/segregation algorithm by applying ’natural’ stimuli (see below). In analogy to the neurobiological experiments we apply Experimental Phases, i.e. presentation of two alternating experimental stimuli A,B with fixed ISI, and ’Natural’ Phases, i.e. presentation of stimuli with ’natural’ spatio-temporal correlations (spatial distance proportional to temporal distance). These stimuli on their own lead to the initial state (cf fig. 3a). Figure 4 shows the temporal development of the reorganizing representation. After reaching a steady representation (besides fluctuations) we apply only ’natural’ stimuli and thereby demonstrate the reversibility of the process. The simulation results depend on the value of ISI relative to the ratio dAB (0)/VW , reflecting how far the cortical activity propagates during the ISI (dAB (0): initial representational distance between experimental stimuli).
5
Psychophysics
Spatially separated stimuli that excite the skin lead to systematical mislocalization in perception when their time difference lies in the range of 15 to 200 or even more milliseconds [5]. This phenomenon called Saltation illustrates the continuous transformation of temporal distances into perceived spatial distances
b) Segregation
c) Reversibility
d) Integration
RF centers
a) Initial state
n=16000
n=26000
n=16000
Cort. activation
n=0
Figure 3: Learning Results; top row: receptive field centers in stimulus space, bottom row: cortical activation induced by simultanous presentation of both experimental stimuli; initial state (a), segregation caused by ’large’ experimental ISI, i.e. ISI> dAB (0)/VW (b), reversibility of b (c), integration caused by ’small’ experimental ISI, i.e. ISI< dAB (0)/VW (d); learning step (cf fig. 2b) denoted by n.
and is described for different modalities (analogs in vision and audition). We regard the ISI-dependence of perceived spatial locations as a hint to dynamics of the kind assumed in our model.
6
Conclusion
Our model of cortical plasticity reveals ISI-dependent representational distances. This constitutes an interpolation of neurobiological findings. In other words, we assume that the ISI-dependence of representational distances is not confined to a mere differentiation of synchronous and asynchronous stimuli, but that this dependence extends to the whole range from 0 to several hundreds of milliseconds. This interpolation can functionally be interpreted as an efficient and quite general way to solve the integration/segregation task. We therefore regard the transfer of the integration/segregation model to other modalities (vision, audition) to be promising. The assumed cortical wave dynamics that guides the self-organizing process is supported by psychophysical findings and can be interpreted as an expectation about incoming stimuli. The influence of such a wave dynamics may be permanently present during ontogenesis leading to ISI-dependent cortical maps, or it may depend on attention reflecting a kind of ’segregation mode’ of the system. Temporal aspects of stimuli may constitute a key to understand the structure of cortical maps. E.g. hand representations in cortical area 3b of owl monkeys do not reflect the proportions of the sensory dimensions. Given the topography of the cortical hand representation (cf fig. 1b) we notice a compression in rostro-caudal direction relative to an expansion in medio-lateral direc-
b)
Representational distance
a)
Cortical position
bar A 20
10
0
bar B
0
10000
20000
Learning steps (n)
20
10
0
0
segregation integration
10000
20000
Learning steps (n)
Figure 4: Temporal Development of Segregation (diamonds) and Integration (squares), sampling every 2000 steps; after 16000 learning steps according to fig. 2b only ’natural’ stimuli are applied; centers of stimulus representation (vertical coordinate in fig. 3) (a), representational distance (b). tion. According to our model this distortion could reflect that the average time differences between stimuli on segments of the same finger are smaller than the ones of different fingers. In contrast to examples of cortical plasticity related to learning and improvements in discrimination ability, time dependent integration and segregation processes may play an important role in harmful side effects of input dependent cortical plasticity, e.g. focal dystonia, phantom-limb pain, and dyslexia [9].
References [1] Spengler F, Hilger T, Wang X, Merzenich MM. Learning Induced Formation of Cortical Populations Involved in Tactile Object Recognition. Soc Neurosc Abstracts 1996; 22(I):105 [2] Wang X, Merzenich MM, Sameshima K, Jenkins WM. Remodelling of hand representation in adult cortex determined by timing of tactile stimulation. Nature 1995; 378:71-75 [3] Joublin F, Spengler F, Wacquant S, Dinse HR. A columnar model of somatosensory reorganizational plasticity based on Hebbian and non-Hebbian learning rules. Biol Cybern 1996; 74:275-286 [4] Grajski KA, Merzenich MM. Hebb-type dynamics is sufficient to account for the inverse magnification rule in cortical somatotopy. Neural Comput 1990; 2:71-84 [5] Geldard FA. Saltation in Somethesis. Psychological Bulletin 1982; 92(I):136-175 [6] Tanifuji M, Sugiyama T, Murase K. Horizontal Propagation of Excitation in Rat Visual Cortical Slices Revealed by Optical Imaging. Science 1994; 266:1057-9 [7] Kohonen T: Self-Organizing Maps. Springer, Berlin, 1995 [8] Brown TH, Kairiss EW, Keenan CL. Hebbian Synapses: Biophysical Mechanisms and Algorithms. Annu Rev Neuroscience 1990; 13:475-511 [9] Merzenich MM, Schreiner C, Jenkins W, Wang X. Neural Mechanisms Underlying Temporal Integration, Segmentation [...]. Ann NY Acad Sci 1993; 628:1-22 [10] von der Malsburg C. Self-organization of orientation sensitive cells in the striata cortex. Kybernetik 1973; 14:85-100
