Ulrich Roth, Axel Jahnke, and Heinrich Klar. TU-Berlin, Institut für Mikroelektronik, Jebensstr. 1, D-10623 Berlin. Experimental results suggest that the time ...
HARDWARE REQUIREMENTS FOR SPIKE-PROCESSING NEURAL NETWORKS Ulrich Roth, Axel Jahnke, and Heinrich Klar TU-Berlin, Institut für Mikroelektronik, Jebensstr. 1, D-10623 Berlin Experimental results suggest that the time structure of neuronal spike trains can be relevant in neuronal signal processing. In view of these results, a shift of interest from analog neural networks to spikeprocessing neural networks has been observed. For tasks like image processing the simulation of these networks has to be performed with the speed of biological neural networks. We investigated the performance of available hardware and showed, that the required performance for large networks could not be achieved. According to these results we formulated concepts for the design of dedicated hardware for spike-processing neurons. For an efficient hardware implementation it is necessary to know the requisite precision for computations. Through simulations with fixed-point numbers we examined the effects of word length limitation on the behaviour of a spike-processing network. The network was able to perform its basic task as long as the word length does not fall below a certain limit. On this basis we derived conditions for the lower bound of the requisite word length.
1 Introduction In the eighties interest in artificial neural networks was revived by the incorporation of statistical methods and analogies in physical systems, e.g. the back-propagation algorithm and the Hopfield model. This led to the well-known growth of this field. For a few years there has been a strong tendency towards a return to biology and towards including more details of neuronal signal processing. The background of this shift of interest is the experimental proof of stimulus-induced synchronized brain activity [Eckh88] [Gray89]. Together with the Correlation Theory by von der Malsburg [Mals86] this results in the assumption, that temporal correlation of activity might be used by the brain as a code to bind features to one object and to segregate one object from others. The synchronised firing of neuronal assemblies could serve as a versatile and general mechanism for feature binding, pattern segmentation and figure/ground separation. How the brain accomplishes these fundamental tasks for invariant object recognition is an old questions of brain science. Invariant object recognition is a natural and easy task for human beings and animals. But it is a difficult and intricate problem in pattern processing for machine vision and until now largely unsolved in a real world environment. Various model neurons and network architectures have been presented which allowed to
reproduce the essential phenomena of synchronized activity in simulation studies. To mention only a few: [Eckh89] [Hart92] [Gers93] [Somp90] [Horn91] [Koen91] [Spor89], see also the references therein. The majority of these model neurons share the following properties: spike trains as incoming signals which induce time-varying potentials at a synapse and outgoing spike trains which originate from a threshold function processing the combination of the postsynaptic potentials. Some models incorporate a time-varying threshold modelling the relative refractory period and/or include axonal delay times. Recently Maas presented a stimulating theoretical analysis of the computational power of spiking neurons [Maas94]. In section 2 we describe the particular model neuron and the model network we chose for our investigations. Unfortunately simulations of these more complex model neurons demand more computer resources. Our first question was whether there the need for specific hardware if someone wants to simulate large networks with thousands of neurons in order to tackle technical problems or to model brain areas. To answer this question we investigated the simulation times on workstations and state of-the-art neurocomputer, like the CNAPS and the SYNAPSE. We present the results in section 3. On this basis we investigated the requirements for an efficient design of a digital accelerator for spike-processing neurons. Through simulations with fixed-point numbers we examined the effects of word length limita-
tion. In section 4 we present the results and derive conditions for the lower bound of the requisite limitation value and the upper bound of the quantization step. Finally, in section 5 we summarize the requirements for an efficient architecture of an accelerator for spike-processing neurons.
2 Spike-Processing Neurons 2.1 Model Neuron The particular model neuron we chose for our investigation is shown in Fig 1. This neuron is basically the same as the one introduced by Reitboeck, Eckhorn et al. [Eckh89]. Incoming spikes x(t) ∈ { 0, 1 } are weighted and induce time-varying potentials u(t) at a synapse which change on a time scale much longer than a single spike. There are two types of input potentials: a feeding potential uf(t), used for direct inputs, and a linking potential ul(t), used for lateral and feedback connections. The linking potential acts modulatory onto the feeding potential. In this way it shifts the temporal occurrence of the spikes without affecting the mean firing rate of the neuron and, thus, supports synchronization in interconnected model neurons. Furthermore there is a time-varying threshold potential ut(t) which enables the model neuron to act as a local nonlinear oscillator. Figure 1: Block diagram of the model neuron wij 1
rt
ut(n) T
T
ul(n) u0
rl
-+
uf(n) T xj(n)
um(n)
wt T yi(n)
rf
The response functions model a postsynaptic potential and are realized as first-order recursive digital filter in this discrete version with a relaxation factor r = exp ( ( – T ) ⁄ τ ) (2.1) where T denotes the basic time unit for one simulation step, normally T = 1 ms. The feeding, the linking and the threshold potentials are described by the following equa-
tions: u f(n) = ∑ w j x j(n) + r f u f(n – 1) j u l(n) = 1 + ∑ w j x j(n) + r l u l(n – 1) j u t(n) = ( w t y(n) + r t u t(n – 1) )
(2.2-2.4).
The combination of the input potentials yields the membrane potential u m(n) = u f(n) ⋅ u l(n) (2.5) The neuron emits a spike if the membrane potential becomes greater than the sum of threshold potential and a static offset uo, i.e. y(n) = H(u m(n) – ( u t(n) + u 0 ) ) (2.6). 2.2 Model network As model network for the study of simulation times we chose a enlarged version of the neural network presented by Reitboeck, Stoecker et al. [Reit93]. Our network consists of a two-dimensional layer of 128x128 neurons. Each neuron receives an input signal to its feeding input from its corresponding pixel in the input image. The input image has the same size as the layer of neurons. Furthermore each neuron is connected to its eighty nearest neighbours via linking inputs in a square of 9x9 neurons. The linking weights decrease exponentially with the distance between the neurons. This type of interconnection is identical for all neurons in the network, except for those close to a border of the layer. Moreover, there is a global inhibitory neuron to which all neurons of the layer are connected. This neuron sums up all outgoing spikes and feeds back the resulting inhibitory potential ui(n) to all neurons. The membrane potential um(n) of each neuron of the layer is, thus, u m(n) = u f(n) ⋅ u l(n) – u i(n) (2.7) with u i(n) = ∑ w i y i(n) + r i u i(n – 1) i
(2.8).
If the network is presented an input image with two or more objects it is able to bind together pixels which belong to one object and to separate one object from others. All neurons which receive input from one object are forming one assembly that is defined by the synchronized firing of its member. Neurons which receive input from an other object also form an assembly. But the synchronized assemblies of different objects fire with different relative phases to each other,
thus, separating one object from others. For a throughout explanation how the network accomplishes this task see [Reit93].
3 Simulation of the model network 3.1 Available Hardware Despite simulating spike-processing neural networks on a general-purpose computer such as a workstation, we can use some of the existing neurocomputer. On the one hand there exist the so-called general-purpose neurocomputers allowing the simulation of a broad class of neural networks. Examples are the CNAPS from Adaptive Solutions, the SNAP from Hecht-Nielsen and the SYNAPSE-1 from Siemens AG. Otherwise, the so-called neuroemulators are designed for specific neural network with the intention to gain more computational power. Recently only one neuroemulator for networks with spiking neurons is known, which has been developed at the TU Berlin. 3.2 Reference network In order to compare different hardware approaches, we choose the network of section 3 as reference. Before continuing, some special properties of this network should be discussed concerning the simulation. When simulating spike-processing neurons on digital computers, the time t proceeds in discrete basic time units. A time slice denotes the simulation of one basic time unit. In order to compare the simulated networks with biological neural networks, we have to compute the artificial neurons with the speed of biological neurons. Therefore the simulation of one time slice have to be finished in less than one millisecond. This could roughly be defined as minimal realtime requirements. A conventional method for storing the network topology is the use one matrix with dimension N x N (N denotes the numbers of neurons). This, however, leads to a very large but sparsely occupied matrix for the reference network due to both the local connectivity and the high number of neurons. To reduce the memory requirements, the topology of sparsely connected networks is commonly stored in two related matrices C and W, each with dimension N x NC (NC denotes the maximum number of connections per neuron). Each row i of C contains several numbers cm cor-
responding to the neurons connected to neuron ni. Each cell of W contains the weight wm corresponding to a connection cm in C. Matrix C could be ordered in two opposed ways: 1. The number cm denotes the neuron nj, to which neuron i sends a spike. 2. The number cmdenotes the neuron nj, from which the neuron receives to which neuron ni sends its spikes. The first method should be called sender oriented or spike distributing, the second receiver oriented or spike collecting [Hart93]. Let the network activity be a measurement of the average number of neurons spiking (the so-called active neurons) per one time slice. Usually a low network activity has been observed in networks with the model neuron described in 2.1 [Reit93]. Thus, the spike-distributing method has to be faster than the spike-collecting method by order of log N. In the reference network, we can also exploit the fact, that the connection between two neurons depends solely on the distance vector d. Therefore we only need to store a small weight vector containing NC values. Calculating the distance vector on-line can also be done with little effort when referencing the neuron by its Cartesian coordinates. Examining the simulation of the network, each time slice can be divided in two steps: 1. In step 1, the numbers of the active neurons spiking the previous time slice have to be distributed or to be collected. According to the incoming spikes, each neuron will increment its potentials uf, ul and us by the corresponding weight. 2. During step 2, each neuron has to relax its potentials and to calculate um(n). Obviously the time needed for step 1 depends on the network activity, while the time for step 2 is rather independent of network activity. 3.3 Experimental results First of all we simulated the network on a conventional computer (Sparc-10, ft = 40 Mhz) Times reported in table 1 are measured by a profiler while simulating and averaging over a large number of time units. Activity of the network has been controlled by changing the value of the stimulus. The spikes are distributed via on-line
calculating the connectivity in a spike distributing manner. As one would expect, most of the time is spent executing step 2, even for high activity. Furthermore, the time needed for one time slice exceeds the value of 1 millisecond by order of 40. Table 1: Simulation Sparc-10 - activity : network activity (percent) - step1: increment of potentials (ms) - step2: relaxation & calculate um (n) (ms) - time slice : Σ (step1, step2) (ms) activity
0.01
0.21
0.8
1.5
2.5
step1
1
4
7.4
13.5
17.7
step 2
39
39.2
39.2
39.3
39.3
time slice
40
43.2
46.6
52.8
57
Table 2: Simulation CNAPS: CP1 activity
0.01
0.21
0.8
1.5
2.5
step1
0.4
41.5
98.2
160
184
step 2
1.6
1.7
1.8
2
2
time slice
2
43.2
100
162
186
Table 3: Simulation CNAPS: CP2 Activity
0.01
0.21
0.8
1.5
2.5
step1
0.1
2.4
4.2
4.2
4.2
step 2
1.4
1.6
1.8
1.8
1.8
time unit
1.5
4
6
6
6
To reduce the simulation time, we implemented the network on a CNAPS-256 from Adaptive Solutions [Hamm90]. The SIMD parallel computer has 256 processing elements (PE´s) and local memory of 4 KB each. There are two 8b-buses accessible, for input and output, respectively. We simulated two different implementations of the network. It should be noted in particular, that we have to calculate the connectivity on-line. Otherwise a network of this complexity could not be simulated on the CNAPS-256 due to the roughly limited size of available memory. A more irregular network of the same size would require about 5 MB for C and W. The two implementations - CP1 and CP2 - are mainly different with respect to the method to calculate the connectivity:
1. CP1 is a straightforward implementation of the network. The 16384 neurons are distributed to all PE‘s in such a way, that every second neuron of row k of neuron mesh is allocated to PE2*k, while the resting 64 neurons of row k are allocated to PE2*k+1. The simplified algorithm for step 1 is shown here: 1. Distributes number of active neurons 2. -- Parallel code: for all neurons if connected to active neuron increment potential 3. If more active neurons, go to 1.
Note, that the spikes are distributed over the bus, but each PE have to collect the spikes. The results are shown in table 2. As one would expect, the time needed for step 1 is dominant even for moderate activity. Furthermore, the overall performance is worse than these of the serial implementation. This behaviour is caused by both the unfavourable distribution of the neurons and the spike collecting on each PE. 2. Therefore, we chose a different approach for CP2 and divided the network in 64 rectangular sectors with 16 * 16 neurons each. Neurons with an identical position p relative to a sector are assigned to the same PE. Position p is stored in its local memory. The algorithm for step 1 is shown here: 1. PEj with active neurons distributes pj 2. -- Parallel code on PEi: calculate distance vector di = pi - pj 3. for all active neurons distribute index of active neuron -- Parallel code on PEi: incr. potential of neuron with same index -- end of parallel code 4. Go to 1. for next PEj
Note, that now each PE distribute the spike to its local neurons. Table 3 shows the computation times. The time needed for step 1 has been reduced by the order of 10 and is also nearly independent of the activity A. The results indicates, that the realtime requirements are almost met CP2. Further improvement should be possible. Nevertheless, the good results were only achieved due to the strong regularity of the used network. Otherwise, the needed memory for the matrixes C and W would lie beyond the available local memory, even when switching to larger CNAPS systems. 3.3 Some additional remarks Another neurocomputer, the SYNAPSE from
Siemens AG, exhibits a high-performance for a broad class of conventional neural networks [Rama93]. One of the key points of SYNAPSE is the reusing of weights from a small subset of neurons for several consecutive pattern before computing the next subset. When simulating e.g. networks with spike processing neurons, one time slice has to be computed after the other. Thus, the SYNAPSE is unsuitable for our reference net or, generally spoken, for the simulation of neural networks processing dynamic pattern. As mentioned above, a neuroemulator has been developed at the institute of microelectronic at the TU Berlin [Pran93]. The underlying neuron model is the model originally presented by Eckhorn et al. [Eckh89], which includes a response function for each synapse. The chip incorporates 16 identical filter blocks and one output block, each containing all parameters required for 16 neurons, allowing therefore the simulation of 16 neurons with 16 synapses each. Due to the full cascadability of this chip, the reference network could be calculated with a network of these chips. The time needed for one time slice is independent of network size and activity. One time slice could be simulated in less than 0.001 ms. However, the unacceptable number of 5184 chips is needed. 3.4 Summary of the results Our investigation indicates, that the considered hardware do not meet the above formulated realtime requirements for large spike-processing neural networks. The reasons are: - missing parallelism (Sparc-10), - inefficient spike-collecting (CP1), - limitation to small networks (CP2), - limitation to static neural networks (SYNAPSE) - unreasonable amount of hardware due to inflexibility concerning the neuron model (neuroemulator) These points have to be taken into consideration for the development of dedicated hardware for spike-processing neurons.
4 Resolution analysis For our study of the requisite precision we used a simplified one-dimensional version of the network described in section 2. In that way we could reduce the simulation times but the essential effects of word length limitation remains the same. In a chain of 128 neurons each neuron is connected to sixteen neighbours to the left and
sixteen neighbours to the right with linking weights of equal magnitude. The input image consists now of two static lines of 32 pixels. Table 4 lists the settings of the weights and the relaxation factors . Table 4: Parameter Settings of the Chain Network weight
relax. fac.
feeding pot.
1.3
0.904837
linking pot.
0.1
0.367879
inhibitory pot.
0.3
0.904837
threshold pot.
40
0.920044
Before each simulation run we initialized the potentials of each neuron with randomly chosen values of a prior simulation run. The network needs a mechanism which breaks the initial symmetry in order to separate the two objects. The network forms now two assemblies which fire about every 28 simulation steps. The firing of the two assemblies occurs with a phase difference of π, i.e. about every 14 simulation steps fires one assembly. If the basic time unit for each simulation step is defined as 1 ms, the firing interval of the assemblies corresponds to an oscillation of nearly 36 Hz. An input signal arriving with a fixed period k leads to a peak value of the potential which can be calculated as the limit value of the sum over a geometric series according to the following equation w (4.1). u peak = ------------k1–r At the perfectly synchronized network the linking, the inhibition and the threshold potential reach their minimum value at the firing time and their peak value one time step later. The minimum value can be computed as u min = r k u peak (4.2). Table 5 displays the resulting values for these potentials together with the feeding potential which reach a constant value and the membrane potential which is calculated according to eqn. (2.7). The value of the static offset was 5. These values have been verified with the simulation results. For our investigation of the effects of word length limitation we proceeded as follows: we changed for one type of potential at a time the
Table 5: Peak and minimal values at the perfectly synchronized chain network upeak
Figure 2: Performance measure Q vs. limited word length for the potentials of the chain network in fixed-point representation
umin
feeding pot.
13.66
13.66
linking pot.
4.2
1
inhibitory pot.
12.74
3.14
membrane pot.
44.63
10.52
threshold pot.
44.30
4.35
representation from floating-point to unsigned fixed-point number. These fixed-point numbers i-bit integer part and f-bit fraction part - were used to store the results of the computations in finite length registers. A value exceeding the maximum representable number, the limitation value vl = 2i-2-f, was set to this limitation value. Quantities smaller than the minimum representable number, the quantization step q = 2-f were truncated. For every determined word length i+f and scaling (i,f) we performed 20 simulation runs with different initialization. We judged the performance of the network by two valuation criteria: the maximum quantity of neurons firing at one time step Nf an the standard deviation of the firing times of one assembly σ. The scaled values are combined in the following performance measure Q = σ ⋅ Nf (4.3). We reduced the word width and - if necessary changed the scaling as long as the performance measure exceeded a certain value and the network was able to perform the basic task of binding and separation. We applied this procedure to each type of potential. The results (Fig. 3) indicate that finite resolution has no significant influence on the performance of the network as long as the word length does not falls below a certain limit. This proves again, that the network performance does not depend critically on accurate parameter settings, or - assuming the influence of finite resolution as additive noise - that the network is astonishing robust against noise. The limit of the word length is between 5 and 7 bits for the various types of potentials. Going beyond the limit results in a break down of the network performance. The network was able to perform its basic task with a minimal word length of 5 bits and a scaling of (3,2) for the feeding potential, 6 bits and
feeding potential
Q
1 0.8 0.6 0.4 0.2 0
0
5
10
15
20
25
30
linking potential Q 1 0.8 0.6 0.4 0.2 0
0
5
10
15
20
25
30
inhibitory potential
Q 1 0.8 0.6 0.4 0.2 0
Q
0
5
10
15
20
25
30
membrane potential 1
0.8 0.6 0.4 0.2 0
0
5
15
20
25
30
threshold potential
Q
1 0.8 0.6 0.4 0.2 0
10
0
5
10
15
20
word length
25
30
(1,5) for the linking potential, 4 bits and (2,2) for the inhibitory potential, 5 bits and (5,0) for the membrane potential and finally 7 bits and (6,1) for the threshold potential. As we see, there is no serious problem for all potentials except the threshold potential, if the limitation value is smaller than their peak value. The limitation of the various potentials has, however, different effects on the network performance. Now we will examine the effects of finite resolution for the various potentials in detail. A limited feeding potential lowers the oscillating frequency, but the network is able to form two assemblies as long as the feeding potential has the chance to become greater than the static offset. The peak value of the linking potential is only reached when nearly all connected neighbours fire at the same time step. This happens, however, only if one assembly is perfectly synchronized, and the linking influence is no longer necessary. A limited linking potential reduces the quantity of neighbours which can exhibit a synchronizing influence at one time step. The simulation results indicate that this quantity should be greater than a distinct value in order to allow the neurons of one assembly to synchronize themselves. This value lies between 9 and 19 neighbours for the model network, corresponding to a scaling of (0,5) resp. (1,5). A limited inhibitory potential shortens the firing periods of the assemblies and the firing period between two assemblies. The two assemblies are no longer firing with a phase difference of π. These effects impair the network performance, but the network is able to separate two objects as long as the limitation value is greater than the minimum value of the inhibitory potential. A limited membrane potential effects the network performance during the initial synchronizing phase. At this stage large values of the membrane potential are necessary to force a neuron which fired a few time steps earlier to fire again, now synchronized with his assembly. Therefore, the limitation value of the membrane potential should be in the same range as the sum of maximum threshold potential and static offset. The only potential which must never be limited is the threshold potential, because a limited threshold potential results in a lasting firing of the neuron.
Summarizing, we can deduce the lower bound for the limitation value of the feeding, the linking, the inhibitory and the membrane potential. The limitation value needs only to be in the range of 2n with n = ld(u max) . (4.4) The limitation value of the threshold potential has to be greater than 2n+1. The maximum value of the various potentials can be computed in advance if the sum of the weights and the minimal period kmin of incoming spikes are fixed: Σw -. u max = ----------------1 – r kmin
(4.5)
Now we will analyse the influence of the quantization step q on the network performance. Computation with truncation has two effects: weights with smaller magnitude are added and the values of the potentials are lowered after each multiplication with the relaxation factor r. Therefore, smaller values are reached for the various potentials compared to the floating-point case. Furthermore, the potentials are decaying faster and arrive at zero, when their value falls below q. Computing with fixed-point numbers leads to an impulse response of finite length, if no rounding is applied. Smaller values for the feeding, linking and inhibitory potential give rise to almost the same effects as the limitation of the maximum values. The firing period is changed and the capacity to bind and separate is reduced in nearly the same way as we described before. At the threshold potential a greater quantization step results in a shorter firing period of the neuron due to the steeper decay of the potential. The network performance is only slightly impaired as long as the quantization step does not become greater than a certain limit value. In the following we will derive a upper bound for the requisite quantization step. At every potential the minimal input quantity, i.e. the minimal weight wmin, should at least have an influence over a distinct time. We will call this time influence length kinf, measured in discrete time steps. Now we demand as upper bound for q that q < w min r kinf
t
(4.6)
where [ ]t denotes the computation with truncation. Calculating the right expression yields
r kinf – r kinf + 1 q < w min -------------------------------, (4.7) 2 – r – r kinf + 1 if we assume that the maximum error q occurs at each computation step. The simulation results validate this upper bound for the quantization step.
5 Conclusion The investigation of the existing hardware shows, that there is still a need for an specific neurocomputer dedicated for spike processing neural networks. An architecture for such a neurocomputer should include following features: - parallelism - distribution of spikes - efficient algorithm to compute the connectivity for regular networks - flexibility concerning the neuron model - simulation of one time slice in less than 1 ms (independent of the network activity) For an efficient hardware design it is necessary to know the effects of word length limitation. The results of our examination indicates, that there is no need for floating-point precision. The network was able to perform its basic task of binding and separating objects as long as the word length of the potentials does not falls below a certain limit. On this basis we derived conditions for the lower bound of the request word length. We checked our results with different parameter settings and verified the derived conditions for the requisite resolution: The minimal word length depends only on the minimal weight, the sum of the weights and the firing characteristics of each potential
Acknowledgment We would like to thank R. Eckhorn and G. Hartmann for stimulating discussions, M. Stoecker for providing us the source code of his network and M. Pfister for supporting our work with the CNAPS. Special thanks to L. Bala, who did a lot of the programming work. This work has been supported in part by the Deutsche Forschungsgemeinschaft (DFG) under Grant No. Kl 918/1-1.
References [Eckh88] Eckhorn R, Bauer R, Jordan W, Brosch M, Kruse W, Munk M. Reitboeck HJ (1988) Coherent oscillations: A mechanism of feature linking in the visual
cortex ? Biol Cybern 60:121-130 [Eckh89] Eckhorn R, Reitboeck HJ, Arndt M, Dicke P (1989) Feature linking via stimulus-evoked oscillations: Experimental results from cat visual cortex and functional implication from a network model. Proc ICNN I:723-730 [Gers93] Gerstner W, Ritz R, Hemmen JL van (1993) A biologically motivated and analytically soluble model of collective oscillations in the cortex. Biol Cybern 68:363-374 [Gray89] Gray CM, Singer W (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proc Natl Acad Sci USA 86:1698-1702 [Hamm89] Hammerstrom D (1990) A VLSI Architecture for High-Performance, Low-Cost, On-Chip Learning. Proc. IJCNN II: 537-543. [Hart92] Hartmann G (1992) Hierarchical neural representations by synchronized activity: a concept for visual pattern recognition. In: Neural Network Dynamics, edited by j. G. Taylor et al. (Springer, Berlin et al.) pp 356-370 [Hart93] Hartmann G (1993) 1. Workshop zum Förderungsschwerpunkt „Elektronisches Auge“. Summary: 10-19. [Horn91] Horn D, Usher M (1991) Segmentation and Binding in an oscillatory neural network. Proc IJCNN II:243-248 [Koen91] Koenig P, Schillen W (1991) Stimulus-dependent assembly formation of oscillatory responses: I. Synchronization. Neural Comput 3(2):155-166 [Maas94] Maas W (1994) On the computational complexity of networks of spiking neurons. NeuroCOLT Technical Report NC-TR-94-021 [Mals86] Malsburg C von der, Schneider W (1986) A neural cocktail-party processor. Biol Cybern 54:29-40 [Pran93] Prange SJ, Klar H (1993) Cascadable Digital Emulator IC for 16 Biological Neurons. Proc ISSCC: 243-244. [Rama92] Ramacher U, Beichter J, Brüls N (1991) Architecture of a General Purpose Neural Signal Processor. Proc IJCNN I: 443-446. [Reit93] Reitboeck HJ, Stoecker M, Hahn C (1993) Object separation in dynamic neural networks. Proc IEEE Int Conf Neural Networks II:638-641 [Somp90] Sompolinsky H, Golomb D, Kleinfeld D (1990) Global processing of visual stimuli in a neural network of coupled oscillators. Proc Natl Acad Sci USA 87:7200-7204 [Spor89]] Sporns O, Gally JA, Reeke GN, Edelman GM (1989) Reentrant signalling among neuronal groups leads to coherency in their oscillatory activity. Proc Natl Acad Sci USA 86:7265-7269
