生体の構造に学んだサッカード モデルとその集積回路化 浅井 哲也, 米津 宏雄 豊橋技術科学大学 電気・電子工学系 〒 441-8580 愛知県豊橋市天伯町雲雀ヶ丘 1-1
E-mail:
[email protected],
[email protected] 視野に突然現れた視覚対象に対して反射的に目を向ける人工サッカード システム を提案する。集積回路化を目的とし てモデル化されたシステムは 、網膜から上丘 へのマップ，上丘および上丘からモーターニューロンへのマップ構造を含む。本 報告では 、数値および 回路シミュレーションにより、提案したシステムが視覚対 象に対してサッカード を起こし 、高速度・高精度サッカード のために自己学習す べきパラメータが 、上丘-モーターニューロン間のマップおよび上丘の広域抑制強 度であることを示す。 サッカード , 上丘, アナログ集積回路, ニューラルネットワーク
A neuromorphic object-capturing circuit based on biological saccadic systems Tetsuya Asai, Hiroo Yonezu Department of Electrical and Electronic Engineering, Toyohashi University of Technology
1-1, Hibarigaoka, Tempaku-cho, Toyohashi 441-8580, Japan E-mail:
[email protected],
[email protected] A simple object capturing circuit which is based on the biological structure of the superior colliculus and retina is proposed for realizing an artificial saccadic system. The function of the superior colliculus, an afferent and efferent mappings are considered in the circuit. Results of numerical and SPICE simulations reveal that the proposed circuit could shift the gaze toward the target in the absence of learning processes. Furthermore, it is clarified that candidates of parameters required for tuning the model are the gain and space constants of an efferent mapping and strength of lateral inhibition in the superior colliculus. saccade, superior colliculus, analog integrated circuit, neural network
R/cosθ
1 Introduction It is essential for living things to pay attention to unknown objects in order to confirm whether the objects have a harmful influence to oneself or not. A saccadic neural system is believed to be responsible for such attentional behavior and captures visual targets on the retinal fovea by turning the eyeball. An artificial object capturing system which realizes the saccadic functions seems to be particularly attractive since it leads to wide practical applications. Horiuchi and Koch (1996) successfully demonstrated the artificial saccadic system. Their one dimensional system is largely based on traditional saccadic models (Jurgens et al. 1981; MacKenzie and Lisberger 1986) in which afferent,efferent maps and the superior colliculus (SC) do not explicitly appear. It is quite important to develop hardwares by mimicking the structure of biological systems since the hardware could give us opportunities to investigate unknown functions of those systems. In this article we propose a simple saccadic model which is based on the biological structure of the SC and retina and is suitable for an analog circuit implementation.
2 One dimensional object capturing system The description of the proposed circuit system is divided into four stages; i) a motor coordinate for determining the relation between the visual field and its projection on the retina according to the angle of the eyeball; ii) an afferent mapping function from the retinal ganglion cells to the SC; iii) a competitive neural network for eliminating noises and selecting important information from the retinal images; iv) an efferent mapping function from the SC deep layer to motor neurons for driving the eyeball toward the target.
2.1 Motor coordinates In this article, one dimensional visual field is considered, as shown in Fig. 1. Suppose that the gaze is not fixed to the target. When the target appears at the position µ on the screen, the reflection of the target on the retina does at the position µ as long as assuming that the visual angle and θ are equal to zero. Turning the eyeball θ yields the gaze shift toward the target with the amount of k tan θ, where k represents the distance between the screen and the retina when θ = 0. Hence, the angle required for fixing the fovea to the target can be expressed as µ (1) θ = tan−1 . k The width of the target on the retinal axis (y) is shrunk as compared with that of the target on the screen axis (x) with the ratio of cos θ. Thus, by defining f(x) the light intensity of the target on the screen, the intensity on the retina making the position of the fovea zero on the y-axis is given by
screen 0
µ=ktanθ (target)
x
k
0 (fovea)
θ y
retina
R Fig. 1. Visual field (screen) and retinal motor coordination. The visual angle is assumed to be zero. The target appears on the screen (x-axis) at the position of µ, while corresponding image does on the retina (y-axis) with the scale factor of cos θ.
y + k tan θ . (2) cos θ For large visual angle, the same strategy could be taken without loss of generality. R(y) = f
2.2 Mapping from the retina to the SC The center of the retina including the fovea has 1:1 ratio between the photoreceptors and the superficial layer of the SC, while the far periphery has a ratio of 300:1 (Kandel et al. 1991). This is because the number of the photoreceptors exponentially decreases from the fovea to the end of the retinal sheet. This structure makes the mapping function between the retina and SC logarithmic, as shown in Fig. 2. Actually, the function obtained from physiological experiments (Ottes et al. 1986) is expressed as u = b · ln(1 + y/a),
(3)
where b and a the gain and space constants, u and y the SC and retinal coordinates, respectively(See Fig. 2c). Note that employing the function results in a gradual shift in resolution.
2.3 Competitive layer in the SC To generate a specific eye movement, the selection of neural activity that encodes a specific displacement vector needs to be occur. It is expected that lateral inhibition within the SC may function for this neural selection.
According to Wurtz and Munoz (1994), the lateral inhibition is so widely distributed in the SC. This implies that inhibitory connections among neurons in the intermediate layer may be regarded as all-to-all connections. Based on this idea, we employ the Lotka-Volterra (LV) competitive neural network as the intermediate and deep layer of the SC. The LV equation which describes competitive behavior among N identical neurons is given as (Fukai and Tanaka 1997) X zj , i = 1, . . . , N (4) τz z˙i = zi γ + Wi − zi − λ
rostral
caudal
neurons in the SC deeper layer
P1
PN (Pi < Pi+1 ) motor neuron driving signal (a)
optic nerve
The number of the ganglion cells
where zi is the activity of the ith neuron, γ represents an input which is nonspecific to each neuron, Wi represents neuron-dependent inputs and ε is a small positive constant. Each neuron has a self-inhibitory connection of the strength normalized to unity and λ is the relative strength of all-to-all lateral inhibitory connections among different neurons. The steady-state solutions to (4) can be classified into three types, i.e., winner-takeall (WTA), winners-share-all (WSA) and variant winnertake-all solutions, each of which represents qualitatively different selection behavior. Among the solutions, the WSA solution in which a certain number of neurons remain activated in steady states is particularly useful owing to the robustness in the selection of inputs from a noisy environment.
0
z i, Pi, z iPi
j6=i
zi Pi
z iPi
1
i (b)
Fig. 3. Efferent mapping from the SC neurons to the motor neuron. (a) Connections between the SC (upper) and the motor neuron. SC rostral neurons have small movement contribution, while caudal neurons do large contribution (Pi < Pi+1 ). The output of the motor neuron is given by the weighted average of the SC neurons, which generates a motor signal to drive the eyeball. (b) The efferent mapping function determined by the output of the SC and the synaptic connections between the SC neurons and the motor neuron.
The retinal image with logarithmic mapping function: W (u) = R(b · ln(1 + u/a)) distance from fovea
(b)
u
fovea
(a)
SC
y (c)
Fig. 2. Afferent mapping from the retina to the superficial layer of the SC. (a) Schematic image of the arrangement and connection between ganglion cells of the retina and SC superficial neurons. (b) The distribution of the ganglion cells. The number of the cell decreases toward the end of the retina. (c) The afferent mapping function from retinal (y) to SC (u) axis.
(5)
is given to the LV network by assuming discrete distribution: Wi = W (idu), b · ln(1 + idu/a) + k tan θ , =f cos θ
ganglion cells
N
(6)
where i and du represent the position and discrete step, respectively. Since the neurons of the LV network could be regarded as that of the deep layer of the SC, zi represents the activity of the neuron of the layer. After the competition, neurons receiving the large inputs from the retina will be activated, which yields a specific motor signal to drive the eyeball.
2.4 Mapping from the SC to oculomotor neurons The trajectory of the saccade is based on vector summation of the movement tendencies provided by each
member of the population of active neurons in the SC deep layer or be determined by a weighted average of the vector contributions of each neuron in the active population (Lee et al. 1988). In the one dimensional saccadic system proposed in this article, the vector summation is simply expressed as a liner summation: (output) =
N 1 X zi , N
V com
λ
ΣIout,i H-cell
E-cells 1
i
N
(7)
i
where zi represents the activity of the neuron in the SC deeper layer. It has been clarified that the saccade amplitude increases by stimulating the caudal neurons in the SC deep layer, while it decreases by approaching to the rostral pole of the SC (Robinson 1972). This implies that the rostral neurons has small movement contributions, while the caudal neurons has large contributions to the eye movement, as shown in Fig. 3a. According to Opstal and Gisbergen (1989), the movement contribution of ith neuron is given by Pi = A(exp(idu/B) − 1),
(8)
where i represents the distance from the rostral neuron, A and B the gain and space constants, respectively. Based on those results, the sum of the movement contributions of each neuron in the active population could be expressed as N 1 X zi Pi . m= N
(9)
i
Figure 3b shows that the filled area corresponding to m becomes small as i → 1, which implies that the saccade amplitude decreases as the populational activity moves toward the rostral pole of the SC. Finally, the dynamics for driving the eyeball could be assumed as (10) τθ θ˙ = m. Note that τθ must be sufficiently small as compared with τz in (4) since the motor signal should be determined after the competition in the intermediate and deep layer of the SC. 3 Analog circuit implementation In the section, we propose analog circuits and devices which realize the mapping functions and the LV network discussed in the previous section. 3.1 Device structure of the afferent mapping The logarithmic afferent map discussed in Sect. 2.2 can easily be implemented on the chip by arranging photodiodes according to the distribution of the retinal photoreceptors in which the number of the receptors exponentially decreases from the fovea to the end of the retinal sheet (See Fig. 2). This structure makes the implementation quite efficient since the sparse area of the photodiodes on the chip could be used for implementing the LV circuit and the efferent mapping circuit.
Iin,1
Iin,i
Iin,N (a)
Iin,i Iout,i
V com
ΣIout,i
V com M1,i (b)
(c)
Fig. 4. Schematic images of the LV circuits. (a) A network topology for the N -dimensional LV neural circuit with O(N ) complexity. The circuit consists of the H-cell and N E-cell circuits. (b) An inhibitory neuron (H-cell) circuit composed of five MOS transistors and two current sources. (c) An excitatory neuron (E-cell) circuit composed of four MOS transistors. The E-cell circuits are connected with the H-cell circuit.
3.2 SC competitive circuit The LV integrated circuit (LVIC) has been designed and fabricated by Asai et al. (1997). The measured results of the fabricated LVICs indicate that the reliability of the WSA selection in the noisy environment is considerably improved as compared with conventional WTA circuits. The proposed LV circuit consists of N excitatory cells (E-cells) and one inhibitory cell (H-cell) as shown in Fig. 4a. The H-cell acts as a global inhibitor which receives output currents of all the E-cells and inhibits those Ecells, as shown in Fig. 4a. Figures 4b and 4c represent the H-cell and E-cell circuits, respectively. The output currents of the retinal devices are given to each E-cell through a pMOS transistor in Fig. 4c, while the output current of the ith E-cell is given by the current of M1,i . The output currents of the LV circuit are given to the efferent mapping circuit by mirroring the current of M1,i for each E-cell. 3.3 Efferent mapping circuit The efferent mapping circuit generates the motor signal by summing the output currents of the LV circuit with weighted synaptic connections, as shown in Fig. 3a. Figure 5 represents the mapping circuit realized by a capacitance C, 3N − 2 transistors. The input currents of the
f(x), R(x) and W(x)
1
f(x)=R(x) 0.8 0.6
W(x)
0.4 0.2 0 -0.5
0
0.5
1
1.5
x
(a)
circuit are represented by current sources (I1 , I2 , . . . , IN ) in the figure. Those currents are regulated by the transistors which are connected with the sources and flow into the capacitance. Gate terminals of the ith transistor (Vi ) is connected with the neighboring transistors through the pass transistors. The pass transistors act as a resistive network. Since a tip and an end of the resistive network are connected with voltage terminals VA and GND, the potential distribution of the gate voltages of the transistors become exponential. The common gate voltage VB determines the space constant B in (8), while VA does the gain constant A.
4 Simulation results In this section, several numerical and SPICE simulations are conducted for showing the behavior of the proposed model and circuits. Figure 6 represents the light intensity of the target image on the screen (f) and its projection on the retina (R) and superficial layer of the SC (W ). In the simulations, the intensity of the target is assumed to be gaussian, which is given by f(x) = exp(
−(x − µ)2 ), 2σ 2
(11)
where µ and σ are set at unity and 0.1, respectively. The distance between the screen and the retina k, the gain b and space constants a are fixed at unity. Images on the screen yield the same image on the retina when θ = 0, that is f(x) = R(x), as shown in Fig. 6a. Turning the eyeball results in the shift of R(x), which shifts W (x) towards x = 0, as well (Fig. 6b). Equation (1) predicts that θ must be 45deg in order to distribute the retinal projection around x = 0(Fig. 6c). Note that the distribution of W (x) is asymmetric in Figs 6a and 6b, while the distribution is symmetric in Fig. 6c due to the afferent mapping function (3). Figure 7 represents the input and output distributions of the competitive layer. The input distribution is given by (6), where N , du, λ and γ are set at 100, 0.04,
W(x)
0.8
R(x)
f(x)
0.6 0.4 0.2 0 -0.5
0
0.5
1
1.5
x
(b) 1
f(x), R(x) and W(x)
Fig. 5. The efferent mapping circuit which generates the motor driving voltage Vout . The common gate voltage VB determines the space constant B of the map, while VA does the gain A.
f(x), R(x) and W(x)
1
f(x)
0.8 0.6 0.4
R(x) W(x)
0.2 0 -0.5
0
0.5
1
1.5
x
(c) Fig. 6. The light intensity of the target image on the screen (f ) and its projection on the retina (R) and superficial layer of the SC (W ) for θ = 0 (a), θ = 25deg (b) and θ = 45deg (c).
0.8 and 1.0, respectively. After the competition, five neurons around the center of the gaussian remain activated, as shown in Fig. 7b. The maximum output value of the neuron in the output layer is smaller than that of the input layer due to the existence of the five winners. The efferent mapping function used in the simulation is shown in Fig. 8. The gain A, du and the space constants B are set at (exp(N du) − 1)−1 , 0.04 and unity, respectively. The function is normalized to have a unit value when i = 100. SPICE simulations are conducted for the proposed efferent mapping circuit. Figure 9 shows output voltages of the circuit for Ii = 100nA, Ij6=i = 0A. When caudal (rostral) neuron is activated, Vout becomes large (small). The movement contribution for the ith neuron is controlled by changing VA for the amplitude and VB for the space gradient. Namely, VA determines the maximum
1
10
0.9
(10V, 10V)
(V A , V B)
0.8
(10V, 2V)
0.7 1
V out [ V ]
Wi
0.6 0.5 0.4
(1V, 10V) 0.1
0.3 0.2
(1V, 2V)
0.1 0 0
20
40
60
80
100
i
caudal
0.01 1
2
rostral 3
4
5
6
7
8
9
10
i Fig. 9. SPICE simulation results of the efferent mapping circuit with N = 10. The maximum output voltage (i = 1) is determined by VA . The sloop of the distribution becomes steep for small VB (2V), while the sloop does gentle when VB is large (10V).
(a) 1
0.8
50
zi
0.6
45
A=2.0
40
0.4
A=1.0
35
θ[deg]
0.2
0 0
20
40
60
80
A=0.5
30 25 20
100 15
i
10
(b) Fig. 7. The input (a) and output (b) distributions of the competitive layer. The strength of the lateral inhibition is set at 0.8 so that five neurons would be activated in equilibrium states.
5 0 0
50
100
150
200
150
200
time
(a) 50 45
λ=0.6
1 40
λ=0.8
35
θ[deg]
0.8
Pi
0.6
λ=1.0
30 25 20 15
0.4
10 5
0.2 0 0
50
100
time
0 0
20
40
60
80
100
i Fig. 8. Efferent mapping function which determines the strength of the synaptic connection between the SC neurons and the motor neuron. The function is normalized to have a unit value when i = 100.
output voltage (i = 1), while the sloop of the distribution depends on VB . Figure 10 represents the time course of θ(t). The time constants τd and τθ are fixed at 1.0 and 0.1, respectively. Changes in the gain of the efferent mapping function (8) yields significant changes in the performance of the proposed model, as shown in Fig. 10a. Small gain (A = 0.5) requires long saturation time, while the equilibrium value of θ is close to the correct value (45deg) since the gain determines the total value of the input magnitude in (10) and the error signal around i = 1. The accuracy
(b) Fig. 10. Dependence of the efferent gain (a) and the strength of the lateral inhibition (b) on time courses of the proposed saccadic system.
could be improved by decreasing the space constant B. Thus, the gain A must be large for ensuring the quick saccade, while the space constant should be small for obtaining the accuracy. The performance of the proposed model depends on the strength of the lateral inhibition λ in the competitive layer, as shown in Fig. 10b. When λ = 1 in which the equilibrium state of the competitive network has the WTA solution, the equilibrium value of θ is closest to 45deg since only winner is activated at the position of i = 1. Decreasing the strength of λ yields several neurons around i = 1, which makes small error in θ due to the efferent mapping function. Note that λ must be con-
200 180 160 140 120 100 80 time 60 40 20
z i( t ) 1.6 0 0
20
40
i
60
80
100 0
Fig. 11. Time course of the neurons in the SC output layer. Activity of the neurons shifts toward the rostral pole of the SC.
trolled according to the environment. When the system operates in the noisy environment, λ should be smaller than unity since the result of the competition is unreliable. Since the equilibrium value of θ and saturation time for each λ do not change significantly, λ can be controlled without balancing the system performance. Figure 11 shows the time course of the activity in the SC output layer. The gain A and λ are set at unity and 0.8, respectively. The activity moves across the SC toward the rostral pole like a moving hill during the saccade. Remember that the LV network used in the proposed model has initial value independence on the subsequent neural selection. If a traditional competitive network was employed in the SC layer, the outputs never change once the competition was finished, which results in no saccadic movement for the appearance of the visual target. This implies that employing the Lotka-Volterra network is necessary for generating the saccades and supporting the existence of the long-lange inhibition in the SC. 5 Discussion In previous saccadic system proposed by Horiuchi and Koch (1996), the input signal of the system is a fixed motor error. According to the magnitude of the error, a burst unit fires and integrates its output. The output of the unit is subtracted from the error by the local feedback. The output of the burst unit is connected to a neural integrator to generate a tonic voltage to sustain the oculomotor. The parameter being trained in the system is a burst gain which is used to inhibit an overshooting of the saccade. In the system, the motor error is corrected by integrating the local velocity during the saccades. A direction-selective motion detector (Horiuchi and Koch 1996) is required for obtaining the velocity. The proposed system determines the amount of the movement by the local feedback including the motor system in the absence of learning processes. The remaining learning process is a fine tuning to generate quick and precise saccade. Primary candidate of tuning the system is the efferent mapping function described by (8). The space constant B and gain A can easily be controlled by the pass transistors in the efferent mapping circuit.
The second candidate is the strength of the lateral inhibition λ in the competitive layer. The strength determines not only the number of winners but also the magnitudes of the activated neurons. Those parameters should be learned adaptively for various targets. In the saccadic system, a cost function that determines the amount of accuracy of the gaze shift is given to the system in the absence of teachers, which implies that a self-organizing approach could be considered in the processes. References 1. Asai T, Ohtani M, Yonezu H (1997) Analog integrated circuits for the Lotka-Volterra competitive neural networks with a winners-share-all solution. Proceedings of the MicroNeuro’97:91–98 2. Fukai T, Tanaka S (1997) A simple neural network exhibiting selective activation of neuronal ensembles: from winner-takeall to winners-share-all. Neural Computation 9:77–97 3. Jurgens R, Becker W, Kornhuber HH (1981) Natural and druginduced variations of velocity and duration of human saccadic eye movements: evidence for a control of the neural pulse generator by local feedback. Biol Cybern 39:87–96 4. Kandel RE, Schwartz HJ, Jessell MT (1991) In:Principles of neural science. Prentice Hall International, London 5. Horiuchi KT, Koch C (1996) Analog VLSI circuits for visual motion based adaptation of post-saccadic drift. Proceedings of the MicroNeuro’96:60–66 6. Lee C, Rohrer HW, Sparks DL (1988), Population coding of saccadic eye movements be neurons in the superior colliculus. Nature 332:357–360 7. MacKenzie A, Lisberger GS (1986) Properties of signals that determine the amplitude and direction of saccadic eye movements in monkeys. J Neurophysiol 56:196–207 8. Opstal VJA, Gisbergen MAJV (1989) A nonlinear model for collicular spatial interactions underlying the metrical properties of electrically elicited saccades. Biol Cybern 60:171–183 9. Ottes FP, Gisbergen MAJV, Eggermont JJ (1986) Visuomotor fields of the superior colliculus: a quantitative model. Vision Res 26:857-873 10. Robinson DA (1972) Eye movement evoked by collicular stimulation in the alert monkey, Vision Res 12:1795–1808 11. Wurtz RH, Munoz DP (1994) In: Fuch FA (eds) Organization of saccade related neurons in monkey superior colliculus. Contemporary ocular motor and vestibular research: a tribute to David A. Robinson, Springer-Verlag, Berlin
This article was processed by the author using the LaTEX style file cljour2 from Springer-Verlag.
