Controlling the Speed of Syn re Chains - Semantic Scholar

Controlling the Speed of Syn re Chains Thomas Wennekers and Gunther Palm University of Ulm, Department of Neural Information Processing, D-89069 Ulm, Germany

Abstract. This paper deals with the propagation velocity of syn re

chain activation in locally connected networks of arti cial spiking neurons. Analytical expressions for the propagation speed are derived taking into account form and range of local connectivity, explicitly modelled synaptic potentials, transmission delays and axonal conduction velocities. Wave velocities particularly depend on the level of external input to the network indicating that syn re chain propagation in real networks should also be controllable by appropriate inputs. The results are numerically tested for a network consisting of `integrate-and- re' neurons.

1 Introduction The concept of syn re chains has been introduced by Abeles [1] in order to explain precisely correlated spike events observable in the cortex on surprisingly large time-scales of up to hundreds of milliseconds. The main idea is that highly speci c spatio-temporal ring patterns occur in the brain in such a way that synchronously ring pools of neurons iteratively excite other pools, whereby a chain of activation evolves and propagates through the network. Experiments with trained awake monkeys show that the occurrence of syn re chain activation in frontal cortices is clearly correlated with behaviorally relevant events.[2] This observation relates the syn re chain idea to cognitive processes, although the explicit relation is still a matter of discussion. It is furthermore reasonable to believe that spatio-temporal excitation patterns similar to syn re chains also underly the neuronal generation of (elementary) movements. Since those can be executed at variable speed it seems promising to investigate neural network models capable of generating patterns of spike activity at a controllable speed. Syn re chain models have been analyzed for their memory capacity, the length of chains or the fusion of several chains in a single network [4, 5, 7]. Propagation velocities of activation have only recently been studied - numerically by Arnoldi and Brauer [4] and mathematically, using integro-dierential equations, by Arndt et al. [3]. Here we present a model of syn re chain activation consisting of a onedimensional chain of locally connected spiking neurons with `integrate-and- re'like dynamics. We derive a general exact but implicit formula for the propagation speed of travelling pulses as well as explicit approximative solutions for special cases. Background input can be used to control wave propagation speed, which also depends on the form of postsynaptic potentials and the coupling width. The theoretical results are checked numerically in section 3.

2 Theory Since in the original syn re chain model [1] all neurons in one syn re `node' re synchronously, each node may be represented by a single neuron. This leads to a one-dimensional model, which we assume to be in nitely extended. The velocity of pulse-like activation patterns can then be derived similar to the work of Idiart and Abbott [6] who analyzed the propagation of excitation in extended cortical tissue. In [6] a merely macroscopic point of view is taken, but the mathematical method is largely independent of the particularly chosen single unit model. In the sequel we apply it to spiking neurons. We require that the state of a neuron is given by a single variable y representing its membrane potential. A pulse-like output z is released whenever y reaches a given threshold #. After ring some mechanism should prevent the cell from ring immediately again for a certain time, but the details of implementation of such refractoriness do not matter at this point. The above requirements are satis ed by many types of arti cial spiking neurons, in particular the well known `integrate and re'-model. If a spike generated by cell x0 arrives at another cell x it evokes a post-synaptic potential (PSP) at x of a given form g(t) and a height K (x;x0). PSP's of several cells are supposed to sum linearly. Then we can write for the time-evolution of cell x: Z1

Zt

0j  j x ? x 0 0 dt0dx0 : (1) x ;t ?



y(x;t) = I (x;t) + c ?1 t For each neuron x the temporal output pattern of spikes z (x;t) in (1) has to be computed selfconsistently from y(x;t) according to the threshold mechanism described above. I (x;t) is some external input into the network and the double K (x ? x0)

0

g(t ? t0)z

integral in (1) represents neuronal interactions. It has the form of a spatiotemporal convolution, describing the linear summation of PSP's. Note that the spatial coupling kernel K is chosen shift-invariant, K (x;x0) = K (x ? x0 ); though not necessarily isotropic, e.g. K (x) = K (?x). Furthermore, we have included a nite propagation velocity c of signals in (1). This velocity represents the nite speed of action potentials along axonal bres and should not be confused with the speed of wave propagation computed below. Equation (1) supplemented with a suitable refractory mechanism describes the time-evolution of general excitation modes of the network. Now, we set I (x;t) = I (< #) and consider the special case that a solitary wave of spikes propagates to the right along the chain with velocity v. Thus we impose

  (2) z(x;t) =  t ? xv ; where  is the Dirac pulse and the time origin is chosen such that the wave arrives at x = 0 at time t = 0. But the cell at x = 0 can only re if it reaches threshold. Therefore selfconsistency requires y(0; 0) = #. Then, inserting (2) into (1) and evaluating the integrals for x = t = 0 leads to

#=I+

Z1 0

K (x)g(x)dx with  = v1 ? 1c :

(3)

This is an implicit equation determining the unknown parameter  and therefore the wave velocity v. Note, that c, the axonal conduction velocity, is contained in (3) only through the parameter . If we have solved (3) for a particular choice of K;g;I and # we can easily compute v for every c. Equation (3) has a simple physical meaning: assume for the moment that the spike train travels with an arbitrary xed speed. Then it evokes a compound PSP at x = 0 which for suitable K and g will be an unimodal function of time added to the background I . The compound PSP depends on the propagation speed: it is larger, when the wave travels faster, since then individual PSP's superimpose more synchronously and the compound PSP is less spread out. Now, by construction, the integral in (3) gives the value of the PSP at time t = 0, therefore (3) determines exactly that speed (respectively ) for which the membrane potential intersects the threshold at t = 0. (Strictly speaking there are generically two intersections: one on the leading and one on the trailing edge of the PSP. Of course, only the rst is physically meaningful.) Note that wave velocities derived from (3) will depend on I . Background input acts as a predepolarization; hence membrane potentials are nearer to the ring threshold and the compound PSP reaches threshold more quickly. Furthermore, since typically the integral in (3) will be bounded for arbitrary , there exists a minimum I for which (3) can have solutions. The biological meaning again is simple: if we inhibit cortical tissue suciently strong, then it will show no reverberating activation since the overall recurrent excitatory ecacy alone becomes too small. We proceed by specifying particular choices for g. Typical synaptic interaction functions are of the form

gn(t) = n!1n (t ?
)ne? t?
= (t ?
) ;n = 0; 1;


: (

)

+1

(4)

Here, (x) is the Heavyside step function, which we introduced to enforce causality.
 0 takes nite synaptic transmission delays into account, and  > 0 de nes the time-scale of the synaptic response. Using (4) we can compute an approximative solution of (3). To this end, we assume, that K is localized in space and the wave velocity is large, e.g.  small, which will be particularly the case when the cells are near threshold. Furthermore we assume
  . For xed n and k < n all (right handed) derivatives gn(k)(
+) vanish. Then we can expand g in (3) to lowest non-vanishing order yielding the following condition for :

 := # ? I = gnn (
+) ( )

Z1 0

K (x)(x ?
)n(x ?
)dx :

(5)

Furthermore we choose the rectangular coupling kernel

K (x) = K (a ? x)(a + x) ;

(6) where K0 and a measure strength and range of the synaptic interactions. The choice is somewhat arbitrary and in particular applications one may solve (5) or 0

(3) numerically with other choices. Here, the simple form of (6) together with (5) allows us nally to compute explicit formulas for the wave velocity 



 ?1 n = 0 :  = v1 ? 1c =
a 1 ? aK 0 ! r 2

n = 1 :  = a + x 1 + 1 + 2 ax ; with x = a2K : 0

(7) (8)

3 Simulations We tested the results of the previous section by simulating a long discrete chain of N = 512 `integrate and re' neurons. Beside implementing the time-evolution of membrane potentials y(i;t);i = 1; 2;


N , given by Eqn. (1), the following refractory mechanism was chosen: after reaching the threshold value # = 1 the membrane potential of neuron i was reset to zero and exponentially relaxed back to the input value I . This was eectively done by adding ? exp(?(t ? tf )=T ) to y(i;t) after a spike at t = tf with a time constant T long enough to prevent cells from ring more than once during each simulated wave. The PSP-function and coupling kernel were chosen as in (4) and (6) with n = 0 and n = 1;  = 10:0;
= 1:0;K0 = 0:1 and a = 50. The axonal conduction velocity was c = 300:0. (Here and below, all velocities are measured in units of cells/time.) Waves were evoked by forcing a sucient number of cells at one end of the chain to re. Figure 1 combines results from theory and simulations. Figure 1a displays the dependency of the wave velocity on =K0 = (# ? I )=K0 for n = 0 and n = 1.  := # ? I measures the distance between the external (subthreshold) Input and the ring threshold of cells. In Fig. 1a we have scaled  by K0 since only the fraction =K0 enters in (5), (7), and (8). Thus, if we change the synaptic strength K0 the curves in Fig. 1a still remain valid. The solid lines represent exact solutions derived from (3) by numerical root nding (bisectioning). Dashed lines display the approximations (7) and (8), which t the exact curves quite well. Asterisks indicate simulation results. First observe that the velocities for n = 1 are appreciable smaller than for n = 0. This, because g jumps for n = 0 discontinuously to its maximum value as soon as the transmission delay
is over, whereas the PSP's for n = 1 grow slowly and reach a stronger in uence on succeeding cells much later; thereby delaying the wave. Figure 1a shows further, that the velocity increases with increasing input or coupling strength as it should be expected, and that there exists a critical input necessary for the stable development of syn re chains (dotted vertical lines). This is signi cantly higher for n = 1 because then PSP's are broad and relatively small; therefore it is harder for them to reach threshold. If I tends to # Eqns. (7) and (8) both give a limiting  of :02. For c = 1 this would yield a maximum velocity of v = 50. The value becomes somewhat lowered (to  42) due to the nite value of c (cf. the equation for  in (3)). Since biological conduction velocities are large the maximum cortical wave speed will

a

b

Fig.1. Wave velocities for dierent parameter variations. Solid: exact solutions; dashed: approximations; asterisks: simulation results. (See text for details.)

probably be mainly determined by synaptic transmission delays as long as only localized pools of cells are considered. The PSP-functions (4) are characterized by two time-constants
and  , representing the synaptic transmission delay and time scale of PSP variations. Combining
and  with the range a of the coupling kernel yields two characteristic velocities a=
= 50 and a= = 5. The rst, as mentioned above, is an upper bound for the highest possible speed. It can be shown that the second gives the order of the minimumspeed if
  and a=
 c. Then these two velocities approximately determine the range of possible velocity variations. Since rise-times of real synaptic PSP's are probably not much larger than synaptic transmission delays [1] it is opportune to ask for the range of speed-modulation when
  . This is considered in Fig.1b, where  has been varied for xed
= 1:0. Displayed are the maximum speed of 42 (which is independent of  , cf. (7) and (8)) and the minimum speeds derived from (3) by root- nding. As can be seen the possible range of variation only weakly depends on  . Wave velocities also depend on the coupling width a. Changing a leaves the qualitative results intact, but absolute values and ranges of speed variation may change. Furthermore, in discrete chains with small a more complex dynamical phenomena can occur (e.g. discontinuities and hysteresis in v(I )); lack of space prevents us from going into details. Nearest neighbor couplings a = 1, as in the original syn re chain model [1], together with n = 0 present a somewhat singular case, insofar as then ring times between nodes are always
independent of I . However, any nite slope of g abolishes this eect.

4 Discussion Under general conditions we have derived the propagation speed of pulse-like waves in networks of spiking neurons. The velocities show a marked dependency

on the level of external input. By this mechanism wave speeds should also be controllable in real cortices. However, our results present new problems concerning both of the initially mentioned phenomena, syn re activity in frontal cortex and movement control. The syn re idea has originally been developed to explain highly precise correlations of spike events, occuring with an accuracy of less than 3ms for temporal delays between participating spikes of up to hundreds of milliseconds. Changing velocities of syn re activity contradict such precise timing and the repetition of exact spike patterns. Only the constancy of background activation over long time intervals could explain such a pattern. This possibility is supported by [2]. The opposite reasoning applies to movement control. Looking at Fig. 1b reveals that the speed can be modulated roughly by a factor of 2-5, otherwise time-constants have to be chosen in an unreasonable way. This means, it is not very likely that the proposed mechanism can account for the whole dynamical range of speeds with which subjects are able to conduct speci c movements. Nonetheless this mechanism might still help to support ne-tuning of submodules in a more complex architecture of movement control. Let us nally mention, that the proposed mechanism also occurs in other architectures than our abstract model (cf.[4, 3]). Small random variations in single cell or PSP-parameters (;#;
;K ) can be considered theoretically yielding averaged forms of Eqn. (3) (see also [1], chapter 7). Similar calculations furthermore apply to replay speeds in associative memories which store pattern sequences.

Acknowledgement This work has been supported by DFG, Pa 268/8-1.

References 1. Abeles, M.: Corticonics: neural circuits of the cerebral cortex. Cambridge University Press, Cambridge UK, 1991 2. Abeles, M., Bergman, H., Gat, I. Meilijson, I., Seidemann, E., Tishby, N., and Vaadia, E.: Cortical Activity Flips Among Quasi Stationary States. PNAS 92 (1995) 8616{8620 3. Arndt, M., Erlhagen, W., and Aertsen, A.: Propagation of Syn re Activity in Cortical Networks - a Dynamical Systems Approach. In: Lappen, B. and Gielen, S.: Neural Networks: Arti cial Intelligence and Industrial Applications. Proceedings of the Third Annual SNN Symposium on Neural Networks. Springer, Berlin, 1995 4. Arnoldi, H.M.R., Brauer, W.: Synchronization without oscillatory neurons. Biol.Cybern. 74 (1996) 209{223 5. Bienenstock, E.: A model of the neocortex. Network 6 (1995) 179{224 6. Idiart, M.A.P., Abbott, L.F.: Propagation of excitation in neural network models. Network 4 (1993) 285{294 7. Palm, G.: On the internal structure of cell assemblies. In Aertsen, A. (ed) Brain Theorie, pp 261-271. Elsevier Science Publishers, Amsterdam, 1993 This article will appear in: Proceedings of the ICANN 96, Bochum, 17-19 July.