Computation of different optical variables of looming ... - CiteSeerX

© 1998 Nature America Inc. • http://neurosci.nature.com

articles

Computation of different optical variables of looming objects in pigeon nucleus rotundus neurons Hongjin Sun1,2 and Barrie J. Frost1 1

Visual and Auditory Neurosciences Laboratory, Department of Psychology, Queen’s University, Kingston, Ontario, K7L 3N6, Canada

2

Present address: Department of Psychology, McMaster University, Hamilton, Ontario, Canada, L8S 4K1


Correspondence should be addressed to B.J.F ([email protected])

Three types of looming-selective neurons have been found in the nucleus rotundus of pigeons, each computing a different optical variable related to image expansion of objects approaching on a direct collision course with the bird. None of these neurons respond to simulated approach toward stationary objects. A detailed analysis of these neurons’ firing pattern to approaching objects of different sizes and velocities shows that one group of neurons signals relative rate of expansion τ (tau), a second group signals absolute rate of expansion ρ (rho), and a third group signals yet another optical variable η (eta). The ρ parameter is required for the computation of both τ and η, whose respective ecological functions probably provide precise ‘time-to-collision’ information and ‘early warning’ detection for large approaching objects.

Gibson’s ecological approach emphasized the importance of dynamic visual information for both perception and the control of ongoing behavior1,2. He and others3 have argued that when an animal interacts with its environment, several distinct types of visual motion pattern may occur, each specifying different classes of events, which in turn require different behavioral responses. For example, motion of a small part of the retinal image relative to other parts of the image that remain stationary, or move differently, usually indicates the presence and trajectory of a moving object or animal. In contrast, its path through space produces visual flow patterns across the entire retina, which can inform the moving animal of its rate of motion and heading through its environment. The relative rate and direction of motion of objects located at different distances (motion parallax) that result from self motion can help to specify the three-dimensional (3D) layout of the environment, whereas the symmetrical expansion of the retinal image of an object can indicate it is on a direct collision course (looming) toward the observer4–8. For many species, including humans, the image of an object looming toward them typically elicits an escape or avoidance response. This has been demonstrated experimentally in fiddler crabs, fishes, frogs, turtles, chicks, monkeys and humans9–16. Moreover, in these studies, the responses do not require stereo cues, as they can be elicited by monocular stimuli, and they remain essentially invariant even when the looming object’s size, shape and starting distance are changed. Several other studies have shown, in a variety of species, that locomotor behavior of the moving animal itself toward stationary features in its environment is also controlled by monocular information associated with the rapid expansion of features in the retinal image 17–22 . Lee 17 proposed that the expansion of the image of an object that is approaching (or the environment that the observer approaches) can trigger a behav296

ioral response, and that the precise timing of the response is controlled by the optical variable called τ (tau), which is equal to the inverse of the relative rate of expansion of the looming object. As the object approaches, the relative rate of expansion increases and τ decreases; if the movement velocity is constant, τ is equal to the ‘time-to-collision’, the time that will elapse before the object will collide with the observer. A number of recent studies have investigated and questioned the possibility that the brain mechanisms underlying visuomotor control tasks may involve the computation of tau23–26. Figure 1a depicts a spherical object moving on a direct collision course toward an observer’s eye. For monocular viewing of such looming stimuli, the most salient feature is expansion of the object’s image on the retina. Instantaneous image size of the object, or the angle θ subtended by the object on the retina, is the first-order information available to the nervous system. As the object looms toward the eye, θ increases approximately exponentially with time, so does the rate of change of visual angle θ′ or rate of expansion ρ (rho). θ, ρ and any other mathematical combinations of θ and ρ (including τ) could potentially be calculated and encoded by the brain and used to predict the time of arrival of the object. Several studies have found looming-sensitive neurons in the visual system of locusts27–29, and we have previously shown that neurons in the nucleus rotundus of pigeons (which is homologous to the pulvinar in the mammalian thalamus) are selectively activated in response to looming objects at a fixed time prior to an impending collision30. Little is known, however, about which particular optical variables are being extracted and used by humans or other animals to perform these computations. Here we report the presence of several different types of looming-selective neurons in the dorsal zone of the nucleus rotundus of pigeons, each computing a different optical parameter (variable) of image expansion. One group nature neuroscience • volume 1 no 4 • august 1998


articles


signals the absolute rate of image expansion (ρ), another group signals the relative rate of expansion (τ), and the third group signals yet another optical parameter of time (η), which has been proposed to model the response of locust looming-sensitive neurons29. The change in time course of these neuronal responses as we varied the time course of optical variables has helped to elucidate their function.

Results We examined the temporal response patterns of looming-sensitive neurons in the nucleus rotundus (Rt) of pigeons, using a moving ‘soccer ball’ stimulus, as previously described30. From a total of 234 neurons recorded from 57 pigeons, 85 (36%) were identified as looming neurons and studied in detail. These looming neurons had large receptive fields (60°–120°), and when monocularly presented with object motion in many different 3D directions, they revealed sharp directional tuning curves that were selective for looming objects approaching on a direct collision course toward the bird30,31. These neurons responded only during simulation of an approaching object, but not for simulation of the birds’ self-motion toward stationary objects32. Subsequent anatomical reconstruction of the locations of the recorded neurons showed that they were concentrated in the dorsal region of Rt. After establishing an Rt neuron’s looming selectivity, we examined which optical variable of the looming object these neurons encoded. The most direct way to test this is to vary the time course of optical variable changes, by presenting looming objects of different sizes or velocities. (We kept movement velocity constant during each approach.) We found that the whole population of looming neurons could be classified into three types. One class of neurons always initiates their response at the same time-to-collision, regardless of the object’s size and velocity. The other two groups of neurons, in contrast, both fired earlier for larger (or slower) objects, but could be clearly distinguished from each other by different patterns of firing rate that increase immediately after the onset of the response and by the timing of the peak responses. Response patterns (peri-stimulus time histograms or PSTHs) for one neuron typical of each of the three classes is shown in Fig. 2. Figure 2a shows different responses as the size of the looming object was varied (from 10 to 50 cm), whereas Fig. 2b shows different responses for the same cells as velocity was varied (from 150 to 750 cm per s). The main effect of varying size or velocity was on the time course of the response, and there was no systematic effect on the magnitude of the peak responses. For neurons like neuron A (Fig. 2a), the response occurred at the same time prior to collision, regardless of the stimulus size, whereas for neurons like neurons B and C (Fig. 2a), the response occurs earlier for larger looming objects. What is different between neurons B and C is that neuron B maintained a similar firing pattern after the onset of the responses, whereas the firing pattern of neuron C varied systematically after the onset of firing. This is evident from the shallower slopes in the ascending phase of the responses and earlier peaks for objects of larger sizes in neuron C. Figure 1b–e illustrates the time course of several different optical variables that could potentially be encoded in the brain, as a spherical object moves on a direct collision course toward an observer’s eye. Solid and dotted lines indicate large and small objects respectively. Figure 1b depicts the time course of the τ(t) function. When an object moves directly towards the observer at a constant velocity V, at time t, for small values of θ, the τ(t) function can be expressed in equation 1 as follows: nature neuroscience • volume 1 no 4 • august 1998

articles

a

Looming object

retina

D(t) Moving at a constant velocity v

b τ

d ρ

τ Threshold

Threshold

1/τ 0

Time

Time

0

e

c η

θ Threshold

Time

0

Time

0

Fig. 1. A schematic diagram of a spherical object of diameter d directly approaching an animal’s eye (a). When an object moves at constant velocity V, at time t, it is at distance D(t) away from the eye, and subtends a visual angle of θ(t). Time course of four optical variables, τ (and 1/τ), θ, ρ, and η derived from the edges of two spherical objects of different sizes, moving at a constant speed directly toward the eye are shown in (b), (c), (d) and (e) respectively (solid line corresponds to larger sphere, dashed line to smaller sphere). Horizontal lines represent hypothetical threshold values for onset of neuronal firing. Time was plotted backwards from the time of collision, t = 0.

τ(t) ≈

θ(t) θ′(t)

(1)

As shown in Fig. 1b, the size of the object has no effect on the value of τ (or 1/τ) at a given time point. In fact, because τ equals time-to-collision if movement velocity is constant, it provides accurate timing predictions with an object of unknown size and velocity17. Because neurons like neuron A always initiated firing at the same time-to-collision (Fig. 2), these neurons seem to be encoding τ. Figure 1c and d depicts the time course for θ and ρ. These two functions increase faster for larger objects, so if a neuron initiates firing when a threshold value of θ or ρ is reached, it should fire earlier for larger objects. Neurons like neuron B (Fig. 2a) all behaved in this manner. In addition to the response onset time, it is interesting to look at the temporal pattern of the response after the onset of the response, to see whether the neuronal firing patterns actually follow the temporal profile of any of these optical variables. Although constraints on maximum firing rates would prevent neurons from continuing to code θ or ρ immediately before col297


lision, when the value of these variables becomes very large (Fig. 1c and d), it is still possible to ask whether the firing rate might track these variables in the early stages of the response. As shown in Fig. 1c and d, if the neuronal firing rate follows either θ or ρ, the rate of increase in the early phase of firing should be faster for larger objects. However, we found no systematic difference in the slopes of the PSTH’s in neurons like neuron B (Fig. 2), indicating that they do not track θ or ρ but instead start to fire when one or other of these variables reaches threshold. The third class of neurons, exemplified by neuron C (Fig. 2a), does not appear to track θ or ρ either, because they showed shallower slopes for larger objects, which does not correspond to either function. Instead, the responses of these neurons showed a distinct peak, which occurred earlier for larger objects. A more suitable optical variable to describe this pattern would be some other mathematical function of time that becomes negatively related to θ when θ reaches large values. The η function, proposed by Hatsopoulos and colleagues29 to model locusts’ looming-detector responses, seems to fit these requirements well. The η function is represented in equation 2 as follows:

time courses (Fig. 2b) closely parallel those found for the size manipulation (Fig. 2a) for the same three neurons. Although the size and velocity manipulations produce identical retinal image expansion profiles, in the real world, and our simulated 3D space, the size and velocity of objects vary independently. We then used the data from these size and velocity manipulation experiments to examined the time course of looming responses more quantitatively. Onset of response was arbitrarily defined as the point where firing rate first reached 33% of maximal rate. We then measured the time-to-collision (Tc) for both the response onset and the response peak, that is, the time interval between these points and the moment of collision (Tconset and Tcpeak). These empirically obtained values were then compared against the quantitative predictions generated from mathematical transformations of the optical variables described above. Because τ equals time-to-collision regardless of object size and velocity, if a neuron’s onset of response is triggered by a threshold of τ (τTh), Tconset should always be constant. If δ represents response latency, then Tconset can be defined (equation 3) as follows:

η(t) = C × θ′(t) × e-αθ(t)

Tconset = τTh - δτ

(2)

(3)

Size

Here, C is a proportionality constant that controls the overall If a neuron is encoding other optical variables, however, there magnitude of the response, and α is a constant for a particular Fig. 2. Based on the differNeuron A Neuron B Neuron C neuron that controls the a ences in the time course of (τ) (ρ) (η) inhibitory damping as the excithe neuronal responses relatatory response become very tive to the moment of collilarge. The η(t) function has a dission, the looming sensitive tinct peak, and its ascending neurons in nucleus rotundus phase slope is shallower for larghave been classified into three er objects (Fig. 1e), which is the distinct classes. This figure opposite of θ or ρ functions. In shows the response pattern neurons like neuron C (Fig. 2a), (post-stimulus time histhe responses showed distinct tograms) for a typical neuron peaks, and larger objects proin each of the three classes duced shallower slopes, which (neuron A, B, and C for τ, ρ, matches what we expect from the and η respectively) to a series theoretical η function, thus sugof stimuli (a simulated moving gesting neurons in this class actusphere with a soccer-ball patally track the optical variable η. tern) of varying sizes (a) or varying velocities (b), moving Object velocity was also varied b along the direct collisionwhile object size was held constant course path toward the bird. (Fig. 2b). If one only examines Responses are the sum of five retinal image size, variation of trials and are referenced to velocity (or more precisely, the time zero, which is the time inverse of velocity) actually creates when the stimulus would have the equivalent change of visual collided with the bird. The angle over time as produced by simulated path was 15 m in the object size manipulation. To length. In (a), velocity for neudescribe this more intuitively, ron B was 375 cm per s and doubling the object’s velocity for neurons A and C was 500 would require the object to start cm per s. In (b), object size its movement at twice the distance was 30 cm for all three neufrom the eye (if we maintain the rons. Note that for neuron A, same movement duration). This the timing of the response would create the same visual angle remains invariant despite subchange over time as would occur stantial changes in size and Time-to-collision if an object half the size was velocity, whereas for neuron B moved at the original velocity. As and neuron C, the timing depends on object size and velocity, with larger or slower objects evoking an expected, when velocity was earlier response. Further quantitative examination suggests that neurons A, B and C encode optical manipulated, neuronal response variables τ, ρ, and η respectively. Velocity


articles

298

nature neuroscience • volume 1 no 4 • august 1998


articles

b Response onset Tc (sec)

Response peak Tc (sec)

a


Square root of (object diameter/velocity)

Object diameter/velocity

Fig. 3. Quantitative examination of the timing of the responses of same three neurons shown in Fig. 2. Empirical measurements of the time intervals between response onset or response peak and time of collision ( Tconset and Tcpeak respectively) for each neuron are plotted as a function of the square root of d/V (a) and d/V (b) respectively. Best-fit linear functions were also generated for each plot. The dataset is the same as for Fig. 2; data from manipulation of object size and object velocity were pooled together for each cell. For neuron A, Tconset did not differ much over a range of values of the square root of d/V (a); for neuron B, however, the plot of Tc onset as a function of the square root of d/V demonstrates a very strong linear relationship (a). For neuron C, the plot of Tcpeak as a function of d/V also clearly demonstrates a strong linear relationship (b). This analysis further supports the idea that neurons A, B and C encode optical variables τ, ρ, and η respectively. l Neuron A (τ), G Neuron b (ρ), P Neuron C (η).

should be specific mathematical relationships between Tconset or Tcpeak and the object size d (and approaching velocity V). For variable ρ, if a neuron starts to fire when a threshold of ρ (ρTh ) is reached during a looming object’s approach, Tconset has an approximately linear relationship with the square root of d/V, and is represented (equation 4) as: 1 d Tconset ≈ ρ × — (4) V - δρ EFF FFF Th

!ê

For visual angle θ, if a neuron starts to fire when a threshold of θ (θTh)is reached, Tconset has a linear relationship with d/V: Tconset =

1

θTh 2 × tan( —– ) 2

d -δ × — θ V

(4.1)

If a neuron tracks the η function, Tcpeak would have a linear relationship with d/V: α × — d -δ Tcpeak = — η 2 V

(5)

Experimentally obtained data for response times were compared to these theoretical predictions. Empirical measurements of Tc (both Tconset and Tcpeak) from the responses of each of the three neurons shown in Fig. 2 are plotted in Fig. 3 as a function of the square root of d/V and d/V respectively. Best-fit linear functions were also generated for each plot. The Tc data for object size and velocity manipulation were pooled together for each cell. For neuron A, because there are minimal differences in Tconset values for looming objects of different sizes and velocity, the best-fit line (Tc onset versus square root of d/V) is rather flat (Fig. 3a, cf. equation 3). The empirical Tconset values closely match the theoretical prediction that the neuron would start to fire when τ reaches a certain threshold value. It is thus very likely that neuron A is signaling τ or ‘time-to-collision’. Although most of our tests were done under constant velocity condition, we also tested the responses of the τ neurons when the movement velocity of the looming object was not constant during nature neuroscience • volume 1 no 4 • august 1998

approach. We found that when an object approaches with deceleration or acceleration, the timing of the onset of the response of these τ neurons matched what would be predicted if the neuron responds to the same τ value, as revealed in the constant velocity situation. This strengthens the conclusion that these neurons are indeed computing τ and thus signaling accurately the time-to-collision when the movement velocity is constant. It also provides an approximate estimate of time-to-collision even when the movement velocity is not constant. It is evident that empirical Tconset values for neuron B produces a tight linear relationship to the square root of d/V (Fig. 3a, cf. equation 4). Moreover Tconset data for neuron B does not produce a strong linear relationship to d/V (plot not shown, cf. equation 4.1). This suggests that neuron B encodes ρ and not θ. Moreover, according to equation 4, the exact value of the threshold for ρ can be derived from the slope of the linear relationship. For neuron B, the regression line (R2 = 0.988) has a slope of 4.1, thus the calculated threshold was 3.4 degrees per second. Similarly Tcpeak values for neuron C are linearly related to d/V (see Fig. 3b, cf. Equation 5). This is consistent with the notion that neuron C tracks the η function. From the slope of equation 5, one can obtain the exact value of α, which is the constant in the visual angle component of the η function shown in equation 5. Thus the complete η function can be generated. The regression line for neuron C (R 2 = 0.994) has a slope of 12.6, whereas α was 25.2. From Fig. 3b, one can also see that there is a weak relationship between Tc peak and d/V for neuron B (R2 = 0.195). Therefore neuron B clearly does not encode η. However, there is a strong linear relationship between Tconset and square root of d/V (R2 = 0.929) for neuron C (Fig. 3a). However, this does not contradict the notion that neuron C encodes the η function. This is because in the initial stages of the η function, the contribution of the θ’(t) or ρ component is dominant, and only in the later stages does the exponential component produce a negative inhibitory contribution to the overall product of the function. 299


articles

b R2 for reponse peak Tc

% of drop-off in firing rate at the time of collision

a

Fig. 4. Quantitative examination of the timing of the response for the population of nucleus rotundus looming-sensitive neurons when presented with approaching objects that varied in size or velocity. In (a), the variances (standard deviation) of Tconset were plotted along the x-axis, and the average drop-off in firing rates at the time of collision, relative to the response peak (%), was plotted along y-axis. The R2 values (b) and slope values (c) of the two linear relationships (Tc onset versus square root of d/V and Tcpeak versus d/V) were plotted against each other. The data points are clustered in three separate regions in each plot, and the group membership of each neuron is consistent across (a), (b) and (c). Therefore this population of neurons can be classified into three distinct groups (τ, ρ or η neurons). p τ neuron, G ρ neuron, P η neuron.

R2 for response onset Tc

c

Slope for response peak Tc


Variance in response onset Tc

Slope for response onset Tc

A series of similar quantitative analyses were performed for the population of Rt looming selective neurons. From the total of 85 looming neurons found and studied in detail, 70 neurons produced complete sets of qualitative and quantitative analyses, which indicated that 37 were τ neurons, 20 were ρ neurons and 13 were η neurons. None of the looming-sensitive neurons were found to encode absolute visual angles θ (i.e., matching equation 4.1). The data for the remaining 15 neurons were discarded because of insufficient data points for a regression analysis (although the timing of their responses and the shape of their response histograms suggest that they most likely were 3 τ, 8 ρ and 4 η neurons) An important question is whether these response properties represent three distinct subgroups of looming-sensitive cells, or whether they form a continuum. To address this, a number of criteria were used to classify the 70 looming neurons for which complete data sets were available. The initial classification made no prior assumption of the underlying optical variable that these neurons may be encoding. After examining Tconset, we found that one group of neurons showed very little variation as object size and velocity was varied (τ neurons), whereas the remaining neurons showed much larger variation (η or ρ neurons). Those neurons with large systematic variances in Tconset could be further subdivided into two groups based on the shape of their response histograms. We found that one of these groups showed a substantial drop-off in firing rate after reaching their peak response rate, and a greatly reduced response was evident at the time of collision (η neurons). In contrast, the firing rate of the other group remained at a fairly high level for some time (ρ neurons). These response characteristics are displayed in Fig. 4a, where three distinct clusters are apparent. 300

In Fig. 4a, the variance (standard deviation) of the Tconset produced by variation of the size and velocity of looming objects is plotted for each cell along the x-axis, and its average drop-off (or reduction) in firing rate at the time of collision, relative to the response peak (in percentage), is plotted along the y-axis. The looming cells clearly cluster in three separate regions in the plot. One cluster (τ neurons) has very small variances in Tconset compared to the other two groups, and their response drop-off is small (i.e., the responses remain at a high level even at the moment of collision). A second cluster (ρ neurons), like τ neurons, also show a small drop-off after peak firing, yet have much larger variation in Tc onset for different stimulus sizes/velocities, compared to that of τ neurons. The third cluster (η neurons) exhibits a significant drop-off in firing rate after peak firing compared to the other two groups. Although this initial classification reveals the existence of three distinct groups of rotundus looming neurons, Fig. 4b and c further demonstrates quantitative relationships between the timing of each cell’s response to object properties, by testing against certain predictions from possible underlying optical variables. The fits of each neuron with respect to the two possible linear relationships (Tconset versus square root of d/V and Tcpeak versus d/V) outlined above are plotted against each other, for both R2 values (Fig. 4b) and slope values (Fig. 4c). Again the data points of the Rt looming neuron population cluster in three separate regions (Fig. 4b and c). The cluster of data points with very low slope values (Fig. 4c) for both Tconset and Tcpeak (along both x and y-axis) functions, clearly belongs to τ neurons, and indicate that for these cells, Tc onset or Tcpeak show little systematic variation with changes in d/V (object size/velocity), even though the R2 values for Tconset are not always very small (Fig. 4b, x-axis). nature neuroscience • volume 1 no 4 • august 1998


articles

Box 1. Mathematical appendix As shown in Fig. 1a, when a spherical object (diameter d) approaches directly towards an animal’s eye at constant velocity V, at time t, it is at distance D(t) away from the eye, and subtends a visual angle of θ(t). T0 t Tc = T0 - t

Total movement duration, i.e., from the start of the movement until the moment when the object reaches the eye Time since the start of the movement Time-to-collision, i.e., time that will elapse before the object will collide with the eye

If a neuron responds when a threshold of θ′(t), i.e., ρTh is reached,

TcTh = To - tTh ≈

And, if we consider a response latency (δ)

Tconset ≈

(1) τ can specify time-to-collision (Tc)

!ê

1 d × — V - δρ ρ EFF FFF Th

(4)

From Fig. 1a,

Thus, Tconset for ρTh has a linear relationship with

–d θ(t) 2 tan[—–] = –— 2 D(t)

with slope of

1

θ(t) 2 2 × [cos —– ] 2

1 —— ρTh and intercept of -δρ EFF FFF

d — !ê V

1 Thus, the threshold of the rate of expansion, ρTh = —— slope2

When both sides are differentiated with time (t), and V>0 © 1998 Nature America Inc. • http://neurosci.nature.com

!ê

1 d × — V ρ EFF FFF Th

From Equation A,

–d 2 ×V × θ′(t) = [D(t)]2

To - t =

1

θ(t) 2 × tan —– 2

d × — V

Dividing the first equation by the second,

If a neuron reponds when threshold θ is reached,

sin[θ(t)] D(t) = θ′(t) V

TcTh = To - tTh =

For small values of θ(t), this becomes

If we then consider response latency (δ)

θ(t) D(t) ≈ θ′(t) V

Tconset =

If we define τ(t) = Then τ(t) ≈

θ(t) θ′(t)

(1)

Therefore, the optical variable τ(t) can specify time-to-collision (Tc) when the movement velocity is constant.

(2) The time-to-collision (Tc) when ρ or θ reaches threshold

Because ρ(t)>0, θ(t)>0 and they are monotonic functions, through their inverse functions unique t values can be generated for certain ρ(t) or θ(t) values.

–d θ(t) 2 d × —— 1 –— tan(—– ) = =— 2 D(t) 2V To - t 1 d V — × (To-t)2 + — 4V d

d -δ × — θ V

(4.1)

d , Thus, Tconset for θTh has a linear relationship with — V

1

θTh 2 × tan( —– ) 2

and intercept of -δθ

(3) The time-to-collision (Tc) when η reaches peak value η(t) = C × θ′(t) × e-αθ(t)

(2)

At the peak of the η(t) function, η′(tpeak) = 0

[θ′′(tpeak) = α × [θ′(tpeak)]2 (A) (B)

!êêêê !ê

d × – 1–– - — ∴To - t = — θ′(t) 4V

1

d × — V

{η′(t) = C × [θ′′(t) × e-αθt + e-αθt × (-α) × (θ′(t))2]

From Fig. 1a,

∴θ′(t) =

θTh 2 × tan (—– ) 2

θTh 2 × tan( —– ) 2

with a slope of

D(t) = Tc V

1

d — V

In our experiment,

d « 1 (s) and — 1 > 5 (s/rad) — ––– 4V θ′(t)Th d 1 » — ∴— ––– θ′(t)Th 4V d 1 × — ––– ∴To - t ≈ — V θ′(t)

!êê !ê


(C)

Based on Equation B

2V × (T - t) × [θ′(t)]2 θ′′(t) = — o d

(D)

Combining Equations C and D,

– × d– [To-tpeak = α 2 V If a neuron tracks the function η and a response latency (δ) is considered,

Tcpeak = α – × d– - δη 2 V

(5)

Thus, Tcpeak for the peak of the function η has a linear relationship with d with a slope of α– and intercept of -δ — η. 2 V

301



articles

For the function involving Tconset (along x-axes), the cluster of data points with high values of R2 (Fig. 4b) and not very small values of slope (Fig. 4c) clearly belongs to ρ neurons. The fact that, along the y-axes (Tcpeak), they do not show high R2 values (Fig. 4b) or slopes (Fig. 4c) distinguishes them from the third cluster of cells (η neurons). This third cluster of cells shows high values of R2 (Fig. 4b) and slopes (Fig. 4c), which indicates they are most likely η neurons. However, looking at the x-axes (Tconset), we find some of these neurons also have high values of R2 (Fig. 4b) and slope (Fig. 4c). This is because of the large contribution of the θ’(t) variable in the η function in the early stages, as discussed previously for neuron C (Fig. 3a). Overall, of the 20 ρ neurons, 15 revealed a very strong linear relationship in Tconset versus square root of d/V (R2>0.9), and the 5 others had R2 values between 0.7–0.9. For the 13 η neurons, regression analysis (Tcpeak vs. d/V) revealed that 9 neurons had R 2 >0.9 and 4 neurons had R 2 values between 0.6–0.9. In addition, the group membership of each neuron is completely consistent across figures 4a, b and c. Therefore, the combination of these different criteria clearly indicate the existence of three distinct neuronal groups, which appear to encode three different optic variables of looming objects.

Discussion It is interesting to look at the functional implications of these three types of Rt looming detectors. The three cell types found here might represent different sources of sensory information that are required to produce different behaviors, or given the nature of the mathematical functions themselves, some might also serve as a building blocks for the computation of other optical variables. Detecting time-to-collision from τ of an approaching object moving in depth would be a valuable strategy for survival, because it provides accurate predictions of the motion of an object of unknown size and velocity. Indeed, it was found that Rt looming neurons’ responses were followed by a sharp increase in EMG activity of the pigeon’s major flight muscle and also heart rate30, both of which are associated with natural defensive responses, such as escape and avoidance behavior. The ρ neurons signal the rate of visual angle change, which increases nearly exponentially as a looming object approaches very close to the eye (as shown in Fig. 1c); therefore, the value of ρ carries important information about the timing of the looming motion. By responding to certain levels of ρ, these neurons could provide animals with gross estimates of the time-to-collision of a looming object. For animals living in a stable environment, the motion of predators may be more or less stereotypical (the size and velocity of particular predators is relatively stable), and therefore certain values of ρ could be hardwired to provide a gross estimate of time-to-collision. In addition, given the usefulness of rate-of-expansion information, ρ neurons most likely serve as building blocks for other neurons to calculate more complicated optical variables, such as τ and η. As for η neurons, one of the key features of the η function is that Tcpeak is linearly related to the ratio of object size and velocity (see equation 5). For different-sized objects moving at the same speeds, a larger object would reach its peak firing rate earlier. This feature might have certain evolutionary benefits for prey like pigeons, as larger objects often represent a greater danger or reflect a higher probability that the object is a predator. These neurons might quickly and simply detect incoming large objects and thereby also signal ‘urgency’ to elic302

it quick defensive actions. Indeed there have been a number of behavioral studies in animals and humans suggesting that instantaneous object size is a cue that triggers motor action in the face of impending collision33,34. Importantly, although the looming response was elicited by simulated motion of isolated objects, no response was seen when the object was combined with a background that underwent the same transformation32. The latter stimulus mimics the effect of self-motion, and so it seems likely that self-motion, and hence information about impending collision with stationary objects, is processed in separate brain structures from those that code looming objects. Methods Methods have previously been described elsewhere 31. We recorded extracellularly from anaesthetized (ketamine/rompun) pigeons (Columba livia), which were positioned in a stereotaxic instrument. This project was reviewed by the Queen’s University Animal Care Committee and certified as in compliance with the guidelines of the Canadian Council of Animal Care. Visual stimuli were presented monocularly on a rear-projection tangent screen (120o x 120o), which was placed 40 centimeters in front of one of the bird’s eyes. The visual stimulation system consisted of a high resolution (1280 x 1024 pixels) Electrohome Trinitron color projector (ECP4000) and a Silicon Graphics IRIS 4D/310GTX computer graphics system. The visual object was designed as a paneled ‘soccer ball’ with alternating black and white panels, with the area of black and white in equal proportions to control for overall luminance changes. The soccer ball could move in different directions in simulated three-dimensional space. The computer calculates the appropriate projective geometry and displays the transformed images in sequence in real time (at 60 frames per s) to yield smoothly moving and transforming animation.

After a single unit was isolated, three-dimensional directional tuning curves were obtained to determine whether a cell selectively responded to looming stimuli. Then random sequences of stimuli differing in either size or velocity were presented to determine the precise time course of responses to these parametric variations.

Acknowledgements The authors wish to thank T. Kripalani and S. David for excellent technical assistance and D. Fleet and N. Troje for helpful discussions and comments on the manuscripts. HJS was supported by a Postgraduate Scholarship from the Natural Science and Engineering Research Council (NSERC) of Canada. This work was supported by an NSERC grant OGP0000353 and an Alexander von Humboldt Research Award to BJF.

RECEIVED 11 MARCH: ACCEPTED 21 MAY 1998 1. Gibson, J. J. The Senses Considered as Perceptual Systems (Houghton Mifflin, Boston, 1966). 2. Gibson, J. J. The Ecological Approach to Visual Perception (Houghton Mifflin, Boston, 1979). 3. Nakayama, K. & Loomis, J. M. Optical velocity patterns, velocity-sensitive neurons and space perception: a hypothesis. Perception 3, 63–80 (1974). 4. Frost, B. J., Wylie, D. R. & Wang, Y.-C. in Perception and Motor Control in Birds (eds Davies, M. N. O. & Green, P. R.) 248–269 (Springer-Verlag, Berlin, 1994). 5. Longuet-Higgins, H. C. & Prazdny, K. The interpretation of moving retinal image. Proc. R. Soc. Lond. B 206, 358–397 (1980). 6. Wylie, D. R., Kripalani, T. & Frost, B. J. Responses of pigeon vestibulocerebellar neurons to optokinetic stimulation. I. Functional organization of neurons discriminating between translational and rotational visual flow. J. Neurophysiol. 70, 2632–2646 (1993). 7. Roy, J.-P. & Wurtz, R. H. The role of disparity-sensitive cortical neurons in signalling the direction of self-motion. Nature 348, 160–162 (1990). 8. Tanaka, K. & Saito, H. Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the Macaque Monkey. J. Neurophysiol. 62, 626–641 (1989).




articles

9. Schiff, W. Perception of impending collision: A study of visually directed avoidant behaviour. Psychol. Monogr. 79, 1–26 (1965). 10. Dill, L. M. The escape response of the zebra danio (Brachydanio rerio). I. The stimulus for escape. Anim. Behav. 22, 771–722 (1974). 11. Ingle, D. J. & Shook, B. L. in Brain Mechanisms of Spatial Vision (eds Ingle, D. J., Jeannerod, M. & Lee, D. N.) 229–258 (Martinus Nijhoft, Dordrecht, 1985). 12. Hayes, W. N. & Saiff, E. I. Visual alarm reactions in turtles. Anim. Behav. 15, 102–108 (1967). 13. Tronick, E. Approach response of domestic chicks to an optical display. J. Comp. Physiol. Psychol. 64, 529–531 (1967). 14. Schiff, W., Caviness, J. A. & Gibson, J. J. Persistent fear responses in rhesus monkeys to the optical stimulus of ‘looming’. Science 136, 982–983 (1962). 15. Bower, T. G. R., Broughton, J. M. & Moore, M. K. Infant responses to approaching objects: an indicator of response to distal variables. Percept. Psychophys. 9, 193–196 (1970). 16. Ball, W. & Tronick, E. Infant responses to impending collision. Science 171, 818–820 (1971). 17. Lee, D. N. A theory of visual control of braking based on information about time-to-collision. Perception 5, 437–459 (1976). 18. Lee, D. N. The optic flow field: The foundation of vision. Philos. Trans. R. Soc. Lond. B Biol. Sci. 290, 169–179 (1980). 19. Lee, D. N., & Reddish, P. E. Plummeting gannets: a paradigm of ecological optics. Nature 293, 293–294 (1980). 20. Lee, D. N., & Reddish, P. E. Visual regulation of gait in long jumping. J. Exp. Psychol. Hum. Percept. Perform. 8, 448–459 (1982). 21. Sun, H.-J., Carey, D. P. & Goodale, M. A. A mammalian model of opticflow utilization in the control of locomotion. Exp. Brain Res. 91, 171–175 (1992). 22. Wagner, H. Flow-field variables trigger landing in flies. Nature 297, 147–148 (1982).


23. Lee, D. N. & Young, D. S. in Brain Mechanisms and Spatial Vision (eds Ingle, D. J., Jeannerod, M. & Lee, D. N.) 1–30 (Martinus Nijhoff, Dordrecht, 1985). 24. Regan, D. & Hamstra, S. J. Dissociation of discrimination thresholds for time to contact and for rate of angular expansion. Vision Res. 33, 447–462 (1993). 25. Tresilian, J. R. Four questions of time to contact: A critical examination of research on interceptive timing. Perception 22, 653–680 (1993). 26. Wann, J. P. Anticipating arrival: Is the tau margin a specious theory? J. Exp. Psychol. Hum. Percept. Perform. 22, 1031–1048 (1996). 27. Rind, F. C. & Simmons, P. J. Orthopteran DCMD neuron: A reevaluation of responses to moving objects. I. Selective responses to approaching objects. J. Neurophysiol. 68, 1654–1666 (1992). 28. Simmons, P. J. & Rind, F. C. Orthopteran DCMD neuron: A reevaluation of responses to moving objects. II. Critical cues for detecting approaching objects. J. Neurophysiol. 68, 1667–1682 (1992). 29. Hatsopoulos, N., Gabbiani, F. & Laurent, G. Elementary computation of object approach by a wide-field visual neuron. Science 270, 1000–1003 (1995). 30. Wang, Y. & Frost, B. J. Time to collision is signalled by neurons in the nucleus rotundus of pigeons. Nature 356, 236–238 (1992). 31. Wang, Y. C., Jiang, S. & Frost, B. J. Visual processing in pigeon nucleus rotundus: luminance, colour. motion and looming subdivisions. Visual Neurosci. 10, 21–31 (1993). 32. Frost, B. J. & Sun, H.-J. in From Living Eyes to Seeing Machines (eds Srinivasan, M. V. & Venkatesh, S.) 80–103 (Oxford Univ. Press, Oxford, 1997). 33. DeLucia, P. R. Pictorial and motion-based information for depth perception. J. Exp. Psychol. Hum. Percept. Perform. 17, 738–748 (1991). 34. Robertson, R. M. & Johnson, A. G. Retinal image size triggers obstacle avoidance in flying locusts. Naturwissenschaften 80, 176–178 (1993).

303


news and views

Collision-avoidance: nature’s many solutions Gilles Laurent and Fabrizio Gabbiani


h(t)(arbitrary (arbitrary units) units) η(t)

-1) 1/timetotocollision collision(sec (s-1) 1/Time

Angularspeed speed(deg/sec) (deg/s) Angular

aa na. From these variables, Anyone who was led to an observer can derive believe that catching a some of the characterisbird simply requires tics of the object’s sprinkling salt on its tail θ d motion, as well as predict can probably testify that the time of impending this method does not collision. The big queswork well. Whether or tion is, how does the not salt on the tail has a v brain behind the exposed paralyzing effect, live b b cc retina do it? birds rarely give you the 1/τ(t) One possibility is that opportunity to test the 50 500 3 1 η(t) neurons (or neuronal cirmethod. In fact, most 2.5 40 400 0.8 cuits) compute ρ(t), the animals have evolved 2 30 300 0.6 rate of angular expanmany parallel warning 1.5 ρ(t) 20 200 0.4 sion, thus indirectly systems to escape such 1 tracking object approach. undesirable encounters θ(t) 10 100 0.2 0.5 τ(t) This solution is reasonwith predators. The smell 0 0 0 0 ably simple and could of coyote urine makes -3 -2 -1 0 -3 -2 -1 0 lead to an appropriate wild mice freeze, the Time Time Time to to collision collision (sec) (s) Timeto tocollision collision(sec) (s) avoidance command sound of bat calls makes when the firing rate of flying crickets dive, and Fig. 1. Elementary kinematics of object approach on a collision course these neurons crosses a the sight of an approach(a) Schematics of a typical looming experiment1. An object of fixed size d certain threshold. It also ing car makes humans approaches the eye at a constant velocity v. During approach, the angle θ(t) run (or brake). Among subtended by the object and its rate of increase [ρ(t) = θ’(t)] both grow non- has an advantage: warning signals, those linearly. (b) Time course of θ(t) and ρ(t) during approach (v = 300 cm per s, because large objects coming from moving d = 30 cm). (c) Corresponding time course of the functions τ(t), τ–1(t) and start to appear big earlier than small ones, the predators present a com- η(t) (with α = 16, see eq. 2 in ref. 1). threshold will be crossed plex challenge to the earlier during approach brain: how can their of large objects, leaving dynamic physical characmore time for escape. The solution has teristics indicate with little ambiguity of the brain and, second, that one of its downside too, in that the rate of that they represent a looming danger? these solutions, despite its apparent expansion increases faster as collision Work by Sun and Frost1 on page 296 of complexity, is found unchanged in very approaches. If this parameter (the rate different animals (birds1 and insects2). this issue of Nature Neuroscience proof expansion) is represented by a firing poses a set of solutions expressed by the An approaching object on a collision rate, the neuron’s firing may saturate dynamic responses of specialized visual course projects an expanding image on close to the critical time. neurons in pigeons. What makes this the retina (Fig. 1a). If the approach To avoid this, Mother Nature found work all the more interesting to us is velocity is constant, the angle θ(t) suba solution: the fast expansion can be that it indicates, first, that several diftended by the object grows nonlinearly slowed down by dividing the rate of ferent computations—each with its own in a near-to-exponential fashion. This angular expansion by an exponential advantages and disadvantages—are carcan be seen in Fig. 1a, where the growth function of the object’s angular size ried out in parallel in the same region of θ is greater over the late than over the [η(t) = θ’(t) / e α.θ(t)] (Fig. 1c). When early half of the object’s approach. Similarly, the rate at which this angle the object is far away, the growth of the Gilles Laurent and Fabrizio Gabbiani are at the expands (ρ(t) = θ’(t)) itself increases numerator dominates and η(t) (or the Division of Biology, California Institute of nonlinearly (Fig. 1b). For a given object firing rate of the neuron representing Technology, 139-74, approach velocity, specific attributes of it) increases. When the object comes Pasadena, California 91125, USA angular expansion (size, velocity, accelnearer, the denominator gains relative e-mail: GL ([email protected]) or FG ([email protected]) eration) are thus projected on the retiinfluence because of its exponentiation Angular Angular size size (deg) (deg)


How does the brain sense looming danger? A new study shows that specialized visual neurons in pigeons carry out several different computations in parallel to analyze signals from approaching objects such as predators.

261


news and views

10

d1/v1

5 0

-3 -2 -1 0 Timeto tocollision collision(sec) (s) Time

2 1.5 1 0.5 0

0

0.05 0.1 0.15 0.2 d/v (s) (sec) d/v

Fig. 2. To classify neurons into different groups, Sun and Frost1 compared the instantaneous firing rate (PSTHs) of their recorded neurons with different functions of time such as θ(t), ρ(t), and η(t) (in a1, b1 and c1, respectively) for different values of d/v (d1/v1 = 0.02 s, d2/v2 = 0.1 s, d3/v3 = 0.2 s in a-c). Each function also predicts specific relations between the stimulus and the PSTH. (a) If the relationship follows θ(t), the time at which a given angular threshold is crossed (a1) should be linearly related to d/v (a2). (b) If the relationship follows ρ(t), the time at which a given angular velocity threshold is crossed (b1) should be linearly related to (d/v)1/2 (b2). (c) If the relationship follows η(t), the time of the peak in the PSTH (c1) should be linearly related to d/v (c2). Sun and Frost’ s results1 indicate the presence of two classes corresponding to (b) and (c) but none to (a). In addition, a third class has a constant firing threshold, consistent with a τ computation.

and slows down the increase in firing rate. Another advantage is that, at a critical time preceding collision, η(t) reaches a peak. This peak occurs when (and thus signals that) the object subtends a particular angle, given by 2tg -1 (2/α), constant for a given neuron. There are two problems associated with η, however. First, to know that this angle has been reached, downstream circuits need to detect a peak firing rate. This is not trivial, and because peak detection requires comparing successive values, this operation requires time, when time is in short supply. Second, the peak signals an angle, not an actual size; it thus confounds a large object that is farther away with a small object that is near. Less time would therefore be available for escape from a small and rapidly approaching object than from a large, slow moving one. Here also, there is a potential solution. It consists of calculating yet another variable, the time-to-contact τ(t) or its inverse (Fig. 1c). Tau is relatively easy to compute when θ(t) is small and when the approach velocity is constant: τ ≈ θ(t)/θ’(t). This measure, introduced by Gibson3 and studied behaviorally in diving birds by Lee and Reddish 4, has the advantage that it is independent of the object size or approach velocity. Tau or 1/τ(t) gives a running value of the 262

bb22

ρ(t)

25

2

20

Tthres (s) Tthres (sec)

15

2.5

1

15 10 5

1.5 1 0.5

0 0 -3 -2 -1 Time to collision (s)

0 0.1 0.2 0.3 0.4 0.5 1/2 (s1/2 1/2 1/2 (d/v) )

(

cc11

)

(

)

cc22 η(t)

1

2

0.8

Tpeak (sec) (s) Tpeak

d3 /v3

bb1

Angularspeed speed(deg/sec) (deg/s) Angular

θ(t)

Tthres (sec) (s) Tthres

20

aa22

d2 /v2

η(t) (arbitrary units) η(t) (arbitrary units)


Angular Angular size size (deg) (deg)

aa11

0.6 0.4 0.2 0

-3 0 -2 -1 Time Timeto tocollision collision(sec) (s)

time before collision. By setting an appropriate threshold, it becomes possible to trigger a motor reaction at a constant delay prior to the anticipated collision. The downsides of the τ computation are that it provides no information about object size or velocity and that the mathematical approximation τ ≈ θ/θ’ is not valid when θ is large. Remarkably, each of these three potnetial solutions is reflected in the properties of neurons in nucleus rotundus of pigeons 1 , an area homologous to the mammalian inferior caudal pulvinar5— a thalamic nucleus with visual inputs from the superior colliculus and which projects to occipital, parietal and temporal cortices. Sun and Frost1 identified three groups of neurons that respond to approaching objects on a collision course by comparing the dynamics of their firing rates with kinematic functions such as θ(t), ρ(t) or η(t) for different object sizes and approach velocities. (See Fig. 2 for a summary of their methods.) One group of neurons shows firing profiles that are best described by a ρ computation. A second group shows peaked firing profiles, best fitted by a η computation. This is particularly interesting to us because the η algorithm was first derived2 to describe the responses of DCMD, a looming-sensitive neuron in locusts6–8. To find such

1.5 1 0.5 0

0

0.05 0.1 0.15 0.2 d/v (sec) (s) d/v

a remarkably similar solution in such distant species (which interestingly have similar predators) supports the idea that similar problems engender similar computational solutions. Finally, Sun and Frost describe a third group of neurons whose onset of activity during approach is independent of object size or velocity, suggesting a τ-style computation. The existence of this last class of neurons was previously reported by Frost’s group9. The properties of these neurons were confirmed in this report and analyzed further, allowing their clear distinction from the other two neuronal and computational clusters. Why are these results important? First, they focus on the dynamics of neuronal responses—rather than mean responses—as relevant neuronal signals. An early attempt at this was made by Rind and Simmons 8 in their study of locust DCMD responses. Second, the results indicate that the brain reconstructs object approach using several parallel (and possibly serial) computations. Each one provides a different piece of information about the state of the environment, and the animal thus presumably makes an informed decision on the basis of these different inputs. But how? It will be interesting to study how downstream circuits interpret these parallel messages, so as to make the best motor



news and views


decision (for example, “duck, but not too early, so as to prevent course correction by predator”). Does one signal dominate the others (see ref. 9)? If so, under what circumstances? Or does some combination of these signals guide behavioral responses? It will also be fascinating to study the cellular mechanisms by which these different computations are carried out. Do the ρ neurons for example, provide inputs to the τ and η neurons, allowing size and velocity signals to be combined? If so, how are these opera-

tions implemented biophysically? If the τ neurons really fire when the variable τ crosses a given threshold, how is this threshold set and held constant? These are some of the many interesting questions that remain to be answered, but Barrie Frost and colleagues are getting us closer to this target.

3. Gibson, J. J. The Ecological Approach to Visual Perception (Houghton Mifflin, Boston, 1979). 4. Lee, D. N. & Reddish, P. E. Nature 293, 293–294 (1980). 5. Karten H. J., Cox, K. & Mpodozis, J. J. Comp. Neurol. 387, 449–465 (1997). 6. Palka, J. J. Insect. Physiol. 13, 235–248 (1967). 7. Schlotterer, G. R. Can. J. Zool. 55, 1372–1376 (1977).

1. Sun, H. & Frost, B. J. Nature Neuroscience 1, 296–303 (1998).

8. Rind, F. C. & Simmons, P. J. J. Neurophysiol. 68, 1654–1666 (1992).

2. Hatsopoulos, N., Gabbiani, F. & Laurent, G. Science 270, 1000–1003 (1995).

9. Wang, Y. & Frost, B. J. Nature 356, 236–238 (1992).

Neurogenic control of the cerebral microcirculation: is dopamine minding the store?

rons cannot account in full for the rapidity and spatial definition of the increases in cerebral blood flow produced by neural activity. It was therefore proposed that selected neurons project to cerebral blood vessels and regulate cerebral blood flow by influencing vascular diameter directly. Although it is well known that cerebral blood vessels are closely associated with neural processes, there has been a long-standing controversy as to whether this proposal is correct because clear morphological and functional evidence linking central neural processes to contractile elements of the cerebral vessel wall was lacking2. Now Krimer and colleagues provide new evidence supporting a direct role of central vascular terminals in the dynamic regulation of the cerebral microcirculation. Using immunocytochemical techniques to visualize dopaminergic terminals both at the light and electron microscopic level, they demonstrate that processes from central dopaminergic neurons terminate in close contact with penetrating arterioles and cerebral capillaries in the cerebral cortex. The density of the dopaminergic vascular innervation varies regionally, being greatest in dopaminerich regions of the frontal, sensorimotor and entorhinal cortices. The dopaminergic processes terminate either directly on the vascular basal lamina or on perivascular astrocytic end-feet. In arterioles, the terminals are located on penetrating arterioles, which are important in the distribution of cerebral blood flow. At the capillary level, the terminals are almost invariably located in close proximity to pericytes, which are contractile cells embedded in the microvascular basal lamina. Krimer and colleagues also show that microapplication of dopamine in the vicinity of cerebral microvessels by iontophoresis, produces vasoconstriction in approximately 50% of the microvessels

Costantino Iadecola Cerebral blood flow is highly regulated by neural activity. New anatomical and functional evidence suggests that dopamine neurons may play a key role in this process. It has long been known that cerebral blood vessels receive abundant projections from central and peripheral neurons, but the functional significance of this innervation has remained elusive. A new study by Krimer and colleagues on pages 286–289 of this issue of Nature Neuroscience provides evidence that central dopaminergic neurons are uniquely positioned to control the cerebral microcirculation and that they may participate in the regulation of cerebral blood flow by neural activity. The ability to map changes in cerebral blood flow produced by neural activity using functional imaging has proven to be one of the most powerful tools for localization of function in the behaving human brain1. Yet the fundamental mechanisms linking neural activity to cerebral blood flow are not fully understood. The brain, perhaps more than any other organ of the body, has the intrinsic ability to regulate its own blood flow with a high degree of Costantino Iadecola is at the Laboratory of Cerebrovascular Biology and Stroke, Department of Neurology, University of Minnesota, Box 295 UMHC, 420 Delaware Street S.E., Minneapolis, Minnesota 55455, USA email: [email protected]

spatial and temporal precision 2 . The amount of flow that each brain region receives is directly related to the functional activity of that region. Thus, if the neural activity increases, for example in specific areas of the cerebral cortex during somatosensory or visual stimuli, cerebral blood flow to the activated regions also increases. Cerebral arteries travel on the surface of the brain (pial arteries) and then enter the brain parenchyma (penetrating arterioles), giving rise to smaller arterioles and capillaries, which supply the tissue with nutrients and remove waste. Cerebral blood vessels have the ability to contract and relax in response to a wide variety of stimuli, thereby decreasing or increasing flow to the different areas of the brain. The mechanisms of this activitydependent regulation of blood flow have been investigated for many decades. A widely accepted hypothesis, articulated by Roy and Sherrington in 1890, is that working neurons release vasoactive agents in the extracellular space, which reach blood vessels by diffusion and produce relaxation of vascular smooth muscles3. However, diffusion of vasoactive metabolites, such as nitric oxide, K+ and H+ ions and adenosine from active neu-


263

Time-to-Contact, Advances in Psychology series Heiko Hecht & Geert J. P. Savelsbergh (Eds.) 2003 Amsterdam: Elsevier - North-Holland

CHAPTER 2: The Biological Bases of Time-to-collision Computation

Barrie J. Frost Queen’s University, Kingston, Ontario, Canada

Hongjin Sun McMaster University, Hamilton, Ontario, Canada

ABSTRACT We begin the chapter by arguing that there may be several neural mechanisms that have evolved for computing time-to-collision (TTC) information as a way of controlling different classes of action. We then focus on single unit mechanisms responsible for processing the impending collision of a moving object towards a stationary observer. After discussing TTC processing in the invertebrate visual system, we describe our own work involving neurons in the pigeon nucleus rotundus that respond exclusively to visual information relating to objects that are approaching on a direct collision course, but not to visual information simulating movement towards those same stationary objects. Based on the recorded neuronal responses to various manipulations of the stimuli, we classified these looming sensitive neurons into three different types of looming detectors based on the temporal differences in neuronal responding relative to the moment of collision. We also described quantitative models for these looming detectors as a way of explaining their physiological response properties.

14

Barrie J. Frost and Hongjin Sun

1. Introduction Information about the time-to-collision or time-to-contact (TTC) has important consequences for the survival of countless species and for their skilled interaction with both the inanimate and animate objects in their environments. As a consequence it appears very probable that there may be several neural mechanisms that have evolved to compute TTC information to control different classes of action, and even different mechanisms within the same animal for different functions. For example, it appears unlikely that mechanisms that have evolved in birds for avoidance of rapidly approaching objects such as predators, where critical and rapid evasive maneuvers are required, are the same mechanisms that control pinpoint landing on branches. In the former case the motion of the approaching predator will primarily determine TTC, whereas in the latter it is only the self-motion of the animal approaching the stationary branch that determines TTC. Of course there may be many instances where both self-motion and motion of another animal determine TTC. For an excellent review that puts TTC in a much broader context the reader is referred to a paper by Cutting, Vishton and Braren (1995). One way that may help conceptualize these factors is to subdivide the primary stimulus determinants of TTC on the one hand, and the general nature of responses controlled by the information on the other, and place these in a simple 2 x 2 table as illustrated in Table 1. Here we have divided the world simply into stationary and moving objects on the vertical axis, and

Source of Looming Stimulus

Behavioural Output

Self-motion towards stationary objects

1

• • •

Insect’s landing Bird’s landing Human or gerbil’s approaching towards target

2

• •

Avoiding obstacles Avoiding cliffs and drop offs

Moving objects (towards and away from the observer)

3

• • • •

Pursuit – prey capture Pursuing mates Flock formation Ball catching

4

• •

Predator avoidance Avoiding aggressive encounters

Approach

Avoidance

Table 1: Situations Requiring TTC Information

the behavioural responses into approach and avoidance on the other. Examples of TTC studies falling in cell 1 (Stationary objects/Approach behaviour) are the landing response of the milkweed bug, Oncopeltus fas-

The Biological Bases of Time-to-collision Computation

15

ciatus (Coggshall, 1972) and the fly (Wagner, 1982). The aerodynamic folding of gannet wings just prior to their entry into water (Lee & Reddish, 1981), birds landing on stationary perches (Lee, Davies, Green & Van der Weel, 1993) or human subjects braking to avoid collision with stationary barriers (Sun & Frost, 1997) or gerbil behaviour of running towards target (Sun, Carey & Goodale, 1992) are other examples that fall into this category. Examples of behaviour falling in cell 2 (Stationary objects/Avoidance) would involve negotiating paths through a cluttered environment where obstacles have to be avoided. This might include steering around barriers, and avoiding holes or sudden drop offs. There appear to be few studies of TTC detection in this category, but Cutting et al.’s (1995) study of path interceptions may be relevant. Prey capture by predators and pursuit chasing during mating could well satisfy entry into cell 3 (Moving objects/Approach), although not all studies of this behaviour have focused on TTC information. Ball catching behaviour, and batting in sports seem also to be appropriate exemplars of this category. Escape from rapidly approaching predators or threatening rivals in territorial mating would be prime example of cell 4 (Moving objects/Avoidance). Throughout the animal kingdom the sight of a rapidly approaching object almost universally signals danger and elicits an escape or avoidance response. When confronted with such a looming stimulus, the visual system must determine precisely the 3D flight path, and compute the TTC of the object, to provide the information necessary for eliciting and controlling the appropriate evasive action (Fishman & Tallaroco, 1961; Schiff et al., 1962; Schiff, 1965; Hayes & Saiff, 1967; Tronick, 1967; Bower et al., 1970; Ball & Tronick, 1971; Dill, 1974; Ingle & Shook, 1983; Yonas & Granrud, 1985). Our own work on neurons in the pigeon nucleus rotundus of pigeons clearly fits in this category because these neurons respond only to the direct collision course of approaching objects (Wang & Frost, 1992, Sun & Frost, 1998), and not to simulation of the movement of pigeons toward the same stationary objects (Sun & Frost, submitted). Also the work on locust looming detectors would fit this category because of the demonstrated elicitation of jumping and flying by the same expanding stimulus patterns that optimally excites the Lobula Giant Movement Detector (LGMD) and the Descending Contralateral Movement detector (DCMD) neurons (Rind & Simmons, 1992, 1999; Hatsopoulos, Gabbiani, Laurent, 1995). Of course it should be remembered that the necessity to compute TTC first requires that any object or surface be indeed on a collision course if the present path of the observing or approaching animal is maintained. Gibson (1979) in his classic work on ecological optics suggests that symmetrical expansion of the images of objects specifies direct approach along a course that will ultimately result in collision with continuous motion. The advantage of using

16


this strategy is that it can be computed using monocular information alone. It is possible that animals with well-developed binocular stereoscopic visual systems, and subpopulations of neurons that respond to stereoscopic motion directions specifying object-motion paths directly toward the animal, might also be used for TTC computations. In this work and his other writings, Gibson also made the clear distinction between collisions with stationary objects occasioned by the motion of the observer (row 1 in Table 1) and other cases where it is the approaching object’s motion itself that will result in collision if it continues along this path (row 2 in Table 1). In this chapter we will focus primarily on research that falls in cell 4 simply because it appears that most of the empirical studies about neural mechanisms of TTC have used stimulus arrangements that simulate events that fall into this category, that is, a rapidly approaching object on a direct collision course with the observing animal, and which might therefore require some sort of evasive action or avoidance response on the part of the animal. In the other category of object motion where the observing animal is trying to arrange a collision with the moving object such as the prey capture behaviour of dragonflies (Olberg, Worthington & Venator, 2000), similar processing mechanisms may occur.

2. TTC in the invertebrate visual system Flying insects have long been used as model systems because they exhibit spectacular aerial performance and accomplish this with relatively simple neural computational mechanisms. Moreover since the same neurons can be identified from animal to animal the neural circuitry is often amenable to analysis. Two such neurons, LGMD and DCMD in the locust, that are synaptically linked, have been shown to be selectively responsive to approaching objects (Rind & Simmons, 1992; Rind, 1997; Hatsopoulos et al., 1995). Neurons that respond to changes in depth have also been found in optic lobes of the hawk moth, Manduca sexta (Wicklein & Strausfeld, 2000), but these may be examples of neurons computing approach and recession for the control of self motion, in this case controlling the hovering flight in front of flowers during nectar collection, rather than for the computation of TTC. Because the DCMD neurons can be readily recorded extracellularly, have very large receptive fields, and respond well to the movement of objects, they have been studied extensively for many years. Schlotterer (1977) was the first to use approaching stimuli to show that DCMD neurons were more responsive to approaching objects than other 2D patterns of movement. Rind and her colleagues, and Laurent and his colleagues have extensively studied these neurons using a variety of stimuli and confirmed that symmetrical expansion gener-


17

ated by an approaching stimulus object is the critical stimulus variable that optimally fires these cells. The allocation of the LGMD - DCMD neurons to cell 4 of our schema presented in Table 1 is justified by their connection to pre-motor interneurons and motor-neurons known to be involved in flying and jumping (Burrows & Rowell, 1973; Pearson et al., 1980; Simmons, 1980). This is further supported by the studies of Robertson and Johnson (1993a, 1993b) who have shown in tethered, flying locusts, that approaching objects elicit a steering avoidance response when the approaching object reaches a critical angular size, thus indicating that some thresholding probably occurs in this pathway. What are the critical features of a symmetrically expanding image that these locust DCMD neurons are responding to that generates their specificity to approaching objects? From an analysis of the several possible cues available in the monocular image Rind and her colleagues (Simmons & Rind, 1992; Rind & Simmons, 1992) have shown that these neurons do not register changes in overall luminance since they respond in a similar manner when light objects approach as when dark objects approach, and their responses were much smaller to sudden luminance change per se. Divergence of two lines moving in opposite directions, to partially represent the opposite contours of a symmetrically expanding object, also did not adequately stimulate DCMD neurons, but increasing the amount of edges in the Receptive Field (RF) and increasing the velocity of edges appeared to be the critical trigger features. Judge and Rind (1997; see also Rind & Simmons 1999) have shown that these locust looming sensitive neurons are very tightly tuned to the direct collision course. Stimulation of the locust retina in one area suppresses LGMD response to a second stimulus presented elsewhere in the visual field, thus indicating there are lateral inhibitory mechanisms operating. Indeed if the appropriate experiments were to be performed one might well find the RF characteristics are similar to those found in the tectofugal or collicular-pulvinar pathway of vertebrates where a directionally specific, double opponent RF organization occurs to ensure that these cells respond to moving objects, and not to the large patterns of optic flow produced by the animal's self motion (Frost 1978; Frost, Scilley & Wong, 1981; Frost, Cavanagh & Morgan, 1988, Sun, Zhao, Southall & Xu, 2002). From a functional point of view this also implies that the LGMD neurons might be interested in approaching objects that fall into cell 4 of our matrix, and not in the locust's approach toward stationary features in its environment. According to Rind and Simmons (1999) the specificity of the LGMD for approaching images is generated by a “critical race over the dendrites of the LGMD in the optic lobe”. The two competitive forces in the race are the excitation produced by the moving edges of the expanding image, and lateral inhibition mediated by neurons in the medulla also synapse onto the LGMD. Rind and Bramwell (1996) have produced a neural network model which seems to support this view and have also shown through electron microscopy that the anatomical

18


arrangement of presynaptic connections to LGMD are compatible with this interpretation. Hatopoulus, Gabbiani and Laurent (1995) have also investigated the LGMD of locusts, and shown that this neuron fires with an increasing rate as an object approaches, then peaks, and drops off just before collision occurs. They have shown that the responses are typically brisker for fast moving or smaller objects, but the peak firing rate does not appears to solely depend on the approach speed or object size. They describe the peak as always exhibiting a constant latency after the time at which the object reaches a fixed angular threshold size on the eye (Gabbiani, et al, 1999). These authors have suggested that the behaviour of the LGMD is best described by the following equation:

f(t) = C × θ ′(t ) × e-αθ (t)

(1)

Here, θ is visual angular subtense, C is a proportionality constant, and is a constant for a particular neuron. In contrast to Rind and her associates view, these authors in recent papers (Gabbiani, et al, 2001; Gabbiani et al., 2002) have suggested that the LGMD post-synaptically multiplies an excitatory and inhibitory input via two different parts of LGMD neuron’s dendritic tree. In order to provide evidence in support of this model these authors (Gabbiani et al., 2002) have selectively activated and deactivated pre-and post synaptic inhibition, and have found that it is post-synaptic inhibition that plays a critical role in shaping the temporal response profile of these neurons, and this indicates that the multiplication takes place within the LGMD neuron itself. These findings are noteworthy for two reasons: in the first place they show in a detailed way how these computations which provide information about looming object are accomplished within the neural machinery of the LGMD and its synaptic connections, and secondly they provide one of the first pieces of clear evidence for how multiplication (and division) is accomplished in the nervous system.

3. Neurons that compute tau in the pigeon brain A number of behavioral studies have revealed that the tectofugal pathway in vertebrates might be involved in processing the visual information necessary for generating such escape or avoidance action. Electrical stimulation experiments indicated that the anuran optic tectum is involved in triggering both prey-catching and also various kinds of avoidance behaviours (Ewert, 1984), and ablation of the optic tectum resulted in abolition of all visually guided preycatching and visual avoidance behaviour (Bechterew, 1984, cf: GrüsserCornehls, 1984). Electrical or chemical stimulation of the superior colliculus in


19

rats also results in defensive-like reactions (Redgrave et al., 1981; Sahibzada et al., 1986; Dean et al., 1988) and is associated with large increases in blood pressure and heart rate (Keay et al., 1988). Pigeons with bilateral lesions of the optic tectum or/and the nucleus rotundus not only showed substantial impairment in intensity, colour, and pattern discrimination, but also exhibited severe deficits in visually guided orientation, escape or avoidance behaviour (Hodos & Karten, 1966; Hodos, 1969; Hodos & Bonbright, 1974; Jarvis, 1974; Bessette & Hodos, 1989). Wild rats with collicular lesions may ignore an approaching human (Blanchard et al., 1981) and similar results have been reported in hamsters and gerbils (Ellard & Goodale, 1986; Northmore et al., 1988). This evidence provides a vivid illustration of the importance of the tectofugal pathway in guiding orientation, detecting approaching objects, and generating escape or avoidance behaviours. Over the years several investigators have claimed that they have encountered cells that respond specifically to objects approaching the eye on a direct collision course. For example, as early as 1976 Grüsser and Grüsser-Cornehls (1976) and later Ewert (1984) reported that some frog and toad tectal neurons respond vigorously to stimuli moving on paths directly towards the eye. However from these early studies many of the appropriate controls were not performed to conclusively exclude the possibility that these neurons were not simply responding to some aspect of the lateral motion of an approaching stimulus. It should be remembered that as an approaching image expands, obviously there will be 2D motion of the edges of the object and its textures and if the expansion is placed asymmetrically over a standard 2D directionally specific motion it could artifactually stimulate the neuron to give a false impression it is responsive to approaching stimuli. We also had encountered cells we thought were responding to the direct approach path of moving objects in 1983, but it was only when we had extremely well-controlled stimuli, which we could systematically vary in their simulated 3D paths, that we could finally convince ourselves that these neurons were indeed coding some aspect of 3D motion. In 1992 Wang and Frost showed that some neurons located in the dorsal posterior regions of the nucleus rotundus of pigeons responded specifically to the direct approach direction of a soccer ball pattern. Using the 3D imaging capabilities of a Silicon Graphics computer we were able to move this soccer-ball stimulus in any trajectory in 3D space. By performing very time consuming 3D tuning curves on these cells we were able to show that this rotundal subpopulation would only respond when the soccer ball stimulus was on a direct collision course with the bird’s head. We chose a soccer ball because the spaceaverage mean luminance did not change as this stimulus expanded and contracted in size (especially when the object moved against a stationary background with the same texture pattern), and it provided many moving and expanding/contracting elements that might be necessary for these neurons to re-

20


spond. Earlier studies, and often some more recent ones, use a simple expanding square or circle where changes in luminance obviously occur concurrently with the expansion/contraction of stimuli, and this necessitates several other controls to rule out this variable as the major contributor to the responsiveness of the neuron. Also other studies have not specifically performed 3D tuning curves to quantify the true directional tuning of neurons of this type. Figure 1 shows the typical 3D tuning curves of one of these rotundal neurons.

Figure 1: A. A soccer-ball-like stimulus pattern consisting of black and white panels, was moved along simulated 3D trajectories 45° apart in spherical coordinates. The diagram illustrates the 4 planes along which stimuli were moved. B. A typical single neuron from the nucleus rotundus of pigeons exhibiting clear selectivity for a looming visual stimulus. Firing rate is plotted for the different directions of motion of the soccer-ball stimulus in 3D space. Each direction of motion was presented 5 times in a randomly interleaved sequence, and the values plotted represent the mean number of spikes for each 3D direction. Note that in the standard X-Y (tangent screen plane, or Azimuth = 90°) plot, there is no indication of directional preference, and firing rate is quite low. However, for the 0°azimuthal plane (Z-axis) there is a strong preference for stimuli directly approaching the bird (0°). Polar tuning plots for directions specifying the azimuthal 45° and 135° planes likewise show no strong preference for any direction. Thus it is only the direct collision course or looming direction that produces an increased response in these neurons, and this pattern of activity was typical of the 27 neurons studied in the dorsal posterior area of the nucleus.


21

Even with 26 directions, 45 degrees apart these are relatively crudetuning curves, so in a few cases we have used a much narrower range of directions after having first performed the broad 3D tuning and found them to be very tightly tuned indeed (Sun & Frost, submitted). In fact the half-width and half-height of detailed tuning curves like those illustrated in Figure 2 is about 3 degrees, where the rotation is around a point halfway along the

Figure 2: Fine tuning of two cells located in the pigeon dorsal nucleus rotundus. First these cells were presented with a soccer ball stimulus that moved in 26 directions 45 degrees apart in 3D space, and they only responded to the direct collision course direction. The graphs shown here are the fine grained tuning curves and show that when the soccer ball, which traveled along a simulated path of 15 meters, was rotated by small amounts each time passing through the center of the path, the cells reduced their firing substantially. The few degrees of rotation of the path indicate that now the soccer ball would travel in a “near miss” and not collide with the bird.

simulated 15 meter path taken by the approaching stimulus. This means in simple terms that a stimulus that depicts a very “near miss” of the bird’s head will only fire the neurons minimally, and one that is a clear “miss” will not evoke any response at all. Perhaps the most important defining character of these neurons’ responses, in addition to their sensitivity to the direct collision course direction, was the constant time they fired before collision, irrespective of the size of the simulated approaching soccer-ball, or of its approach velocity (Wang & Frost, 1992). This indicated to us that these neurons might well be computing the opti-

22


cal variable tau that had been suggested by Lee (1976) to provide important information about the TTC with approaching objects. In our original paper (Wang & Frost, 1992) we reported that there were a variety of times before collision that the population of neurons exhibiting these characteristics showed. This can be seen in Figure 3A.

Figure 3: A. Distribution of different response onset time for 27 looming cells from the dorsal posterior zone of nucleus rotundus of pigeons. Although different cells exhibit different values of TTC, individual cells (B) show remarkable consistency even when velocity or size of stimulus is varied.

But for a particular neuron, the variation in its response onset was remarkably constant (see Figure 3B) on repeated trials and with stimuli of different sizes and velocities. This variation in the population is precisely what is needed if other factors, such as recognition of what the incoming object is, can jointly influence the time selected to perform escape responses of different sorts, each of which may have characteristic time requirements for their optimal deployment.


23

Clearly the characteristics of these rotundal neurons suggested to us that they might be involved among other things in predator avoidance and thus fall in to cell 4 of our classification system shown in Table 1. To provide some evidence for this we tested a few birds under a lightened anesthesia at the conclusion of their recording session. Under deep anesthesia no electromyelograhic signals (EMGs) are obtained from animals, but as the anesthetic is lightened and clearly before any pain stimuli can be experienced, it is possible to obtain good clear muscle responses. These responses do not result in any overt movement of the animal, but can be very useful in indicating what major muscle groups might be involved in a response system normally associated with a stimulus. In this case we recorded from the large pectoralis muscles that power the wings for flight. When we presented the approaching soccer-ball stimulus under these conditions we found that first the tau cells responded with their characteristic maintained burst of firing, then the pectoralis EMGs occurred some 200 milliseconds later, and then finally the heart rate went more slowly up to levels near 300% of the resting rate. These responses again were incredibly specific, and only occurred when the soccer-ball was on a direct collision course with the bird. Near misses and directions 180 degrees away showed no increased EMGs or increases in heart rate. Data typical of these experimental findings can be seen in Figure 4. Although only correlative, we feel this constellation of activity in these tau neurons, and the increased wing EMG and heart rate are indicative of a flight response elicited by the rapidly approaching ball. In a more detailed recent study Sun and Frost (1998) have again confirmed the presence of a population of neurons in nucleus rotundus of pigeons that only respond when the soccer-ball stimulus is on a direct collision course with the bird’s head. Additionally, we found that these neurons only responded when our computer simulated the approach of a moving object towards the bird (stimuli falling into cell 4 of the matrix), and not when the complex stimulus pattern was configured to simulate the bird moving towards the same stationary soccer-ball (stimuli falling into cell 1, or possibly 2 of the matrix) (Sun & Frost, submitted).

24


Figure 4: Heart rate and pectoralis muscle EMGs recorded concurrently with single cell response rate from a looming selective rotundal neuron. Note that the "looming cell" begins firing first, then the muscle response occurs, and then heart rate increases dramatically when the soccer-ball looms toward the bird (A). B. No responses occur when the ball moves along the same path but in the opposite direction directly away from the bird. Bars under the visual response histograms indicate the duration of the visual stimulus. Data collection for the looming-selective neuron was terminated with stimulus offset. The neuronal data and EMGs represent the summed activity over 5 sweeps of the stimulus whereas the heart rate data represent the means and standard errors for the same 5 sweeps. Simulated size of stimulus was 30 cm, path length 15 m, and velocity 375 m/s.

To do this we placed the soccer ball against a stationary background, which consisted of a checkerboard pattern. When the soccer-ball was moved in a trajectory towards the bird (symmetrical expansion) while the background remained stationary, the neurons responded in the typical way and identically to the case where no background was present. However, when the background was moved along the same trajectory as the soccer-ball, so as to show a similar but delayed expanding pattern, the neurons did not respond. This latter configuration formed the precise simulation of the bird approaching a stationary soccer-ball that remained a constant distance in front of the background “brick wall”. The stimulus conditions simulating a soccer ball approaching the bird, and the bird approaching a stationary soccer ball are shown in Figure 5.


25

Figure 5: Schematic diagram of stimulus conditions presented to pigeons. The left portion of the figure illustrates the kind of object movement and its background in the simulated display, while the right portion represents the image change on the screen for the corresponding simulation illustrated on the left. In A, the soccer ball object is presented against a blank background, and the “path” simulates direct approach toward the bird. In B, the same looming object is presented against a stationary textured checkerboard background. In C, both object and textured background move (at the same speed) toward the bird, which simulates the bird’s self-motion toward a stationary soccer ball. Note that the expansion pattern of motion of the object is identical in the three conditions.

It must be emphasized that the expansion pattern of the soccer-ball was identical in these to two cases of moving object and moving bird simulations, yet in the former the cells responded vigorously, while in the latter they were essentially silent. Figure 6 shows the responses of a tau neuron to an approaching soccerball, and also a simulation of the bird approaching a stationary soccer-ball where the rate of expansion is identical in both cases.

26


Figure 6: Comparison of the responses Peri-Stimulus Time Histograms (PSTHs) for a single tau neuron to a series of stimuli (soccer-ball) of varying sizes swept along the direct collision course path toward the bird. Responses are the sum of 5 sweeps and are referenced to time zero, which is the time when the stimulus would have contacted the bird. The looming object was presented against a white non-textured background (A), a stationary textured checkerboard background (object-motion) (B), and a looming background moving at the same speed behind the object (C), as shown in Figure 5. The latter condition simulated the approach of the animal toward the stationary ball and background (selfinduced motion). Responses were similar to those produced by the looming object against a blank background and a stationary checkerboard background. The magnitude of responses (maximal firing rate) were similar across different object sizes. Note that the neuron did not fire to the self-motion display, even though the soccer ball’s image was expanding in the same way in B and C. This implies this tau neuron is exclusively selective for “object motion in depth”. The simulated path for the object was 15 m in length and the simulated object size varied from a diameter of 10 cm to 50 cm. Velocity was 375 cm/s.


27

We have also found that not all of the neurons in nucleus rotundus appear to be computing the tau function (Sun & Frost, 1998). Histological examination indicated that those neurons were distributed in a larger anatomical region (dorsal rotundus) as opposed to dorsal posterior rotundus in our earlier discovery by Wang and Frost (1992). In fact, half of the neurons seem to be clearly responding in this fashion, that is, they suddenly start firing at a particular and constant time before the collision event and maintain this high firing rate throughout the remainder of the approach sequence. Roughly a quarter of the neurons that show selectivity to an approaching object show a response that begins earlier for larger objects, or soccer-ball stimuli approaching at slower velocities. In detailed mathematical arguments and quantification of the timing of the response, Sun and Frost (1998) show that these neurons are computing the rate of expansion, rho, of the approaching object. Finally, the remaining quarter of the looming neurons appear to be computing the very same function which best describes the locust looming detector (Hatsopoulos et al., 1995; Gabbiani et al., 1999). An example of the response patterns of each of these three classes of neuron is shown in Figure 7. Sun and Frost (1998) show that on several multidimensional plots these three classes of neurons form very distinct and tight clusters which indicate that there is not some simple underlying continuum that we have arbitrarily divided into three separate groups, but that these are genuine types of neurons each computing the following three functions.

(1) Rho

ρ (t ) = θ ′(t)

(2)

(2) Tau

τ (t) ≈

θ (t) θ ′(t )

(3)

(3) Eta

η(t) = C × θ ′(t ) × e-αθ (t)

(4)

28

Barrie J. Frost and Hongjin Sun Neuron a (τ)

Neuron b (ρ)

Neuron c (η)

Figure 7: Based on the differences in the time course of the neuronal responses relative to the moment of collision, the looming sensitive neurons in nucleus rotundus have been classified into three distinct classes. This figure shows the response pattern (PSTHs) for a typical neuron in each of the three classes (neuron a, b, and c for tau, rho, and etc respectively) to a series of stimuli (a simulated moving sphere with a soccer-ball pattern) of varying sizes (A) and of varying velocities (B), moving along the direct collision course path toward the bird. Responses are the sum of 5 trials and are referenced to time zero, which is the time when the stimulus would have collided with the bird. The simulated path was 15 m in length. In (A) velocity for neuron b was 375 cm/s and for neurons a and c was 500 cm/s. In (B), object size was 30 cm for all three neurons. Note that for the neuron a, the timing of the response remains invariant despite substantial changes in size and velocity, whereas for neuron b and neuron c, the timing depends on object size and velocity, with larger or slower objects evoking an earlier response.


29

An example of such clustering is shown in Figure 8 taken from Sun and Frost (1998).

Figure 8: Quantitative examination of the timing of the response for the population of nucleus rotundus looming-sensitive neurons when presented with approaching objects that varied in size or velocity. The variances (standard deviation) of Tconset were plotted along the x axis, and the average drop-off in firing rates at the time of collision, relative to the response peak (%), were plotted along y axis. The data points are clustered in three separate regions, therefore this population of neurons can be classified into three distinct groups ( ○ tau neuron, ▲ rho neuron, ● etc neuron). B

B

What is rather amazing about these findings is that all looming cells are accounted for. Each rotundal neuron that responds specifically to an object approaching on a collision course with the bird is either a rho neuron, a tau neuron or an eta neuron. Usually in single unit recording studies there are “junk” or “intermediate” categories required for neurons that don’t seem to fit the major classifications that apply to the other cells. But here we have no residual cells to account for!

30


The function of the tau neurons is quite clear; they can provide the animal with useful information about the TTC of the object that is approaching on a direct collision course. The rho cells obviously compute rate of expansion information that is required to compute tau, that is, the denominator in the tau equation. Also the eta function also contains a rate of expansion term and rho neurons could provide essential input into this computation also. Indeed the multiplication and division required in these three equations that describe the total set of looming detectors we have found in the pigeon nucleus rotundus may well be performed by biophysical operations similar to those described in the recent papers by Peña and Konishi (2001) and Gabbiani et al. (2002). In this latter paper Gabbiani et al. (2002) suggest that the eta operation is implemented within single LGMD neurons by exponentiation of the sum of a positive excitatory postsynaptic potential representing the logarithm of angular velocity, and a negative postsynaptic potential representing angular size.

4. Model We have developed neuronal models to explain the physiological response properties of our pigeon looming sensitive neurons. These models were created based on the physiological responses (both qualitative and quantitative data) recorded to various stimulus conditions (including direct manipulation of various optical variables that could be specified by a looming object). The models also take into account the physiological response properties and anatomical connection of the optic tectum that sends a major input to nucleus rotundus. For these rotundal looming detectors, the RF could be composed of a radial arrangement of concentric arrays of RFs of simple local motion detectors (possibly tectal neurons), with the centre of expansion overlapping with the centre of RF radial layout. These tectal neurons would respond to movements that are oriented radially from the centre of the concentric array and they then converge onto rotundal looming detectors. This arrangement is shown in Figure 9.


31

Figure 9: General model of receptive field (RF) organization of rotundal looming sensitive neurons. The RF is composed of a number RF subunits, each of which corresponds to the RF of a tectal local motion sensitive neuron. These tectal RFs are arranged on the circumference of a series of concentric circles (or rings) with different radii. The converging input from each concentric ring of RF subunits enable both spatial and temporal summation to signal symmetrical image expansion. Note that this general model explains qualitatively the vigorous firing of rotundal looming neurons to symmetrical image expansion.

The spatial and temporal summation of the activity of these small RFs could provide a basis for the rotundal neurons’ large receptive field size, the strong directional selectivity for motion along the direct collision course (indicated by symmetrical expansion from a stationary centre). The opponentdirection centre-surround organization of these tectal units (Frost & Nakayama, 1983) could contribute to the silence of these rotundal neurons to stimulation with a self-motion display, in which local segments of the background move in the same direction as that of the object in the adjacent region across the object boundary. A series of quantitative models were also formulated to account for the specific properties of the time course of rotundal neurons response to impending collision. When an object approaches the eye, the visual angular subtense θ(t) and rate of change of visual angle θ'(t) form the basic building blocks for the

32


calculation of those optical variables that could signal impending collision. For example, TTC can be signaled by the optic variable tau, which can be specified by the ratio of θ' (t) over θ(t) (equal to 1/tau). One way to generate this ratio is through response to the instantaneous values of θ'(t) and θ(t) individually by two sets of neurons, and then through neuronal interaction to generate the ratio. Alternatively, if the instantaneous value of θ'(t) can be encoded (perhaps through the velocity selectivity of local motion detectors, e.g. tectal neurons), then visual angle θ(t) could be registered through the spatial location of the RF of the responding local motion detector relative to the centre of the concentric array of RFs of these local motion detectors. With the centre of image expansion overlapping with the centre of such a concentric array of RFs, a fixed ratio of θ'(t) over θ(t) could be hardwired in the brain to create a threshold for the optical variable 1/tau, so that Rts neurons would only start to fire when the predefined ratio increase to a fixed level. Consequently, such neurons would only fire when TTC decrease to a threshold level as the object approaches (see Figure 10).

A

B

A'

preferred velocity

A

radius of the circle of subunits

Figure 10: Computation of tau. In this model, rate of expansion can be signalled by the spatial summation of the ring of subunits, each with a certain local velocity selectivity (indicated by the size of arrow in A). The instantaneous visual angle could be registered by the distance of this ring of RFs from the centre of the circle at the same time (radius of expansion). To calculate tau, the preferred velocity of each local tectal unit should be linearly related to the radius of the concentric ring of RF subunits located at the same distance (shown in B).

A similar quantitative model of the other two types of rotundal cells has been provided in a recently submitted paper (Sun & Frost, submitted). Our models include alternative one and two component versions where size encoding is realized either from intrinsic pattern of RF organization (one component model)


33

or from the simultaneous coding of visual angle in spatial overlapping neurons, and then converges onto the rotundal neuron. These neuronal models provide explanations for the various physiological responses found in different classes of rotundal looming detectors. Not only are they potentially informative for understanding the neuronal coding of motion in depth, but these models could provide important insights for robotics and machine vision. A neural net model In an interesting recent paper Karanka-Ahonen, Luque-Ruiz and LópezZamora (2002) present the results of a neural network employing backpropagation, implemented in T learn, that learned to predict collision of objects of different sizes that moved towards it. The network consisted of 3 layers with 47 nodes, 40 of which were input nodes, three were hidden nodes, one was the output node and there were three context nodes. The nodes of each layer were completely connected to those of the preceding layer and connections from the context nodes to the context nodes were one to one and reciprocal. After training the network was tested with new objects that differed in size and distance from the original training set and it was found that 75% of collisions were correctly predicted. Interestingly, the behaviour of the hidden units appeared to be very similar to some of the units we found in the pigeon nucleus rotundus, in that the output units predictive response came earlier for larger objects, like our eta cells.

5. Conclusions In this chapter we suggest there may be several different classes of neurons that are each specialized to compute TTC information for different subsets of tasks critical for the survival of animals in their natural environments. We then examine one class of these neurons in detail and find there may be several subroutines required before the tau or eta computations can be performed. What is interesting is that there appears to be convergence in solutions between looming sensitive neurons found in invertebrates and vertebrates in that the eta function describes well some of the neurons in both locust and pigeon visual systems. The functional significance of having both tau and eta neurons is still not obvious, although it is provocative that learning networks appear to give rise to both types of hidden neurons.

34


REFERENCES Ball, W. & Tronick, E. (1971). Infant responses to impending collision. Science, 171, 818-820. Bechterew, W. (1884). Über die Function der Vierhügel. Pflügers Arch, 33, 413. Bessette, B. B. & Hodos, W. (1989). Intensity, colour, and pattern discrimination deficits after lesions of the core and belt regions of the ectostriatum. Visual Neuroscience, 2, 27-34. Blanchard, D. C., William, G., Lee, E. M. C. & Blanchard, R. J. (1981). Taming of wild rattus norvegicus by lesions of the mesencephalis central gray. Physiological Psychology, 9, 157-163. Bower, T. G. R., Broughton, J. M. & Moore, M. K. (1970). Infant responses to approaching objects: an indicator of response to distal variables. Perception and Psychophysiology, 9, 193-196. Burrows, M. & Rowell, C. H. F. (1973). Connections between descending visual interneurons and metathoracic motoneurons in the locust. Journal of Comparative Physiology, 85, 221234. Coggshall, J. C. (1972). The landing response and visual processing in the milkweed bug, Oncopeltus fasciatus. Journal of Experimental Biology, 57, 401-413. Cutting, J. E., Vishton, P. M. & Braren, P. A. (1995). How we avoid collisions with stationary and moving obstacles. Psychological Review, 102, 627-651. Dean, P., Mitchell, I. J. & Redgrave, P. (1988). Response resembling defensive behaviour produced by microinjection of glutamate into superior colliculus of rats. Neuroscience, 24, 501-510. Dill, L. M. (1974). The escape response of the zebra danio (Brachydanio rerio). I. The stimulus for escape. Animal Behaviour, 22, 771-722. Ellard, C. G. & Goodale, M. A. (1986). The role of the predorsal bundle in head and body movements elicited by electrical stimulation of the superior colliculus in the Mongolian gerbil. Experimental Brain Research, 71, 307-319. Ewert, J.-P. (1984) Tectal mechanisms that underlie prey-catching and avoidance behaviours in toads. In H. Vanegas (Ed.), Comparative neurology of the optic tectum (pp. 247-416). New York, NY:Plenum Press. Fishman, R. & Tallaroco, R. B. (1961). Studies of visual depth perception. II. Avoidance reaction as an indicator response in chicks. Perceptual and Motor Skills, 12, 251-257. Frost, B. J. (1978). Moving background patterns alter directionally specific responses of pigeon tectal neurons. Brain Research, 151, 599-603. Frost, B. J., Cavanagh, P. & Morgan, B. (1988). Deep tectal cells in pigeons respond to kinematograms. Journal of Comparative Physiology, 162, 639-647. Frost, B. J. & Nakayama, K. (1983). Single visual neurons code opposing motion independent of direction. Science, 220, 744-745. Frost, B. J., Scilley, P. L. & Wong, S. C. P. (1981). Moving background patterns reveal double opponency of directionally specific pigeon tectal neurons. Experimental Brain Research, 43, 173-185.


35

Gabbiani, F., Krapp, H. G., Koch, C. & Laurent, G. (2002). Multiplicative computation in a looming-sensitive neuron. Nature, 420, 320-324. Gabbiani, F., Krapp, H. G., & Laurent, G. (1999). Computation of object approach by a widefield, motion- sensitive neuron. Journal of Neuroscience, 19, 1122-1141. Gabbiani, F., Mo, C. H., & Laurent, G. (2001). Invariance of angular threshold computation in a wide-field looming-sensitive neuron. Journal of Neuroscience, 21, 314-329. Gibson, J. J. (1979). The ecological approach to visual perception (Houghton Mifflin, Boston). Grüsser, O. J. & Grüsser-Cornehls, U. (1976). Neurophysiology of the anuran visual system. In R. Llinas & W. Precht (eds.), Frog neurobiology. (pp. 297-385). New York, NY: Plenum Press. Hatsopoulos, N., Gabbiani, F. & Laurent, G. (1995) Elementary computation of object approach by a wide-field visual neuron. Science 270, 1000-1003. Hayes, W. N. & Saiff, E. I. (1967). Visual alarm reactions in turtles. Animal Behaviour, 15, 102108. Hodos, W. (1969). Color discrimination deficits after lesions of the nucleus rotundus in pigeons. Brain, Behavior and Evolution, 2, 185-200. Hodos, W. and Bonbright, J. C. (1974). Intensity difference thresholds in pigeons after lesions of the tectofugal and thalamofugal visual pathways. Journal of Comparative and Physiological Psychology, 87, 1013-1031. Hodos, W. & Karten, H. J. (1966). Brightness and pattern discrimination deficits in the pigeon after lesions of nucleus rotundus. Experimental Brain Research, 2, 151-167. Ingle, D. J. & Shook, B. L. (1983). Action-oriented approaches to visuo-spatial brain functions. In D. Ingle, M. Jeannerod & D. Lee (Eds.). Brain mechanisms of spatial vision. (pp. 229258). Dordrecht:Matinus Nijhoft. Jarvis, C. D. (1974). Visual discrimination and spatial location deficits after lesions of the tectofugal pathway in pigeons. Brain, Behaviour and Evolution, 9, 195-228. Judge, S. & Rind, F. C. (1997). The Locust DCMD, a movement-detecting neurone tightly tuned to collision trajectories. Journal of Experimental Biology, 200, 2209-2216. Karanka-Ahonen, J. T., Luque-Ruiz, D. & López-Zamora, M. (2002). A neural network that acquires the capacity of predicting collision from the increase in the retinal size of objects. XIV Congreso de la Sociedad Espaňola de Psicologia Comparada. Keay, K. A., Redgrave, P. & Dean, P. (1988). Cardiovascular and respiratory changes elicited by stimulation of rat superior colliculus. Brain Research Bulletin, 20, 13-26. Lee, D. N. (1976). A theory of visual control of braking based on information about time-tocollision. Perception 5, 437-459. Lee, D. N., Davies, M. N. O., Green, P. R. & Van der Weel, F. R. (1993). Visual control of velocity of approach by pigeon when landing. Journal of Experimental Biology, 180, 85-104. Lee, D. N. & Reddish, P. E. (1981). Plummeting gannets: a paradigm of ecological optics. Nature, 293, 293-294. Northmore, D. P. M., Levine, E. S. & Schneider, G. E. (1988). Behaviour evoked by electrical stimulation of the hamster superior colliculus. Experimental Brain Research, 73, 595605.

36


Olberg, R. M., Worthington, A. H. & Venator, K. R. (2000). Prey pursuit and interception in dragonflies. Journal of Comparative Physiology A, 186, 155-162. Pearson, K. G., Heitler, W. J. & Steeves, J. D. (1980). Triggering of locust jump by mutimodal inhibitory interneurons. Journal of Neurophysiology, 43, 257-278. Peña, J. L. & Konishi, M. (2001). Auditory spatial receptive field s created by multiplication. Science, 292, 249-252. Redgrave, P., Dean, P., Souki, W. & Lewis, G. (1981). Gnawing and changes in reactivity produced by microinjections of picrotoxin into the superior colliculus of rats. Psychopharmacology, 75, 198-203. Rind, F. C. (1997). Collision avoidance: from the locust eye to a seeing machine. In M.V. Srinivasan & S. Venkatesh (Eds), From Living Eyes to Seeing Machines (pp 105-125). Oxford:Oxford University Press. Rind, F. C. & Bramwell, D. I. (1996). Neural network based on the input organization of an identified neuron signaling impeding collision. Journal of Neurophysiology, 75, 967-985. Rind, F. C. & Simmons, P. J. (1992). Orthopteran DCMD neuron: A reevaluation of responses to moving objects. I. Selective responses to approaching objects. Journal of Neurophysiology, 68, 1654-1666. Rind, F. C. & Simmons, P. J. (1999). Seeing what is coming: building collision-sensitive neurones. Trends in Neurosciences, 22, 215-220. Robertson, R. M. & Johnson, A. G. (1993a). Retinal image size triggers obstacle avoidance in flying locusts. Naturwissenschaften, 80, 176-178. Robertson, R. M. & Johnson, A. G. (1993b). Collision avoidance of flying locusts: Steering torques and behaviour. Journal of Experimental Biology, 183, 35-60. Sahibzada, N., Dean, P. & Redgrave, P. (1986). Movements resembling orientation or avoidance elicited by electrical stimulation of superior colliculus in rats. Journal of Neuroscience, 6, 723-733. Schiff, W. (1965). Perception of impeding collision: A study of visually directed avoidant behaviour. Psychological Monographs: General and Applied, 79, 1-26. Schiff, W., Caviness, J. A. & Gibson, J. J. (1962). Persistent fear responses in rhesus monkeys to the optical stimulus of ‘looming’. Science, 136, 982-983. Schlotterer, G. R. (1977). Response of the locust descending movement detector neuron to rapidly approaching and withdrawing visual stimuli. Canadian Journal of Zoology, 55, 13721376. Simmons, P. (1980). Connexions between a movement-detecting interneurone and flight motoneurones of a locust. Journal of Experimental Biology, 86, 87-97. Simmons, P. J. & Rind, F. C. (1992). Orthopteran DCMD neuron: A reevaluation of responses to moving objects. II. Critical cues for detecting approaching objects. Journal of Neurophysiology, 68, 1667-1682. Sun, H.-J., Carey, D. P. & Goodale, M. A. (1992). A mammalian model of optic-flow utilization in the control of locomotion. Experimental Brain Research, 91, 171-175. Sun, H.-J. & Frost, B. J. (1997). The effect of image expansion on a human target-directed locomotion task tested in virtual reality. Society for Neuroscience Abstracts,23


37

Sun, H.-J. & Frost, B. J. (1998). Computation of different optical variables of looming objects in pigeon nucleus rotundus neurons. Nature Neuroscience, 1, 296-263. Sun, H.-J., Zhao, J., Southall, T. L., & Xu, B. (2002). Contextual influences on the directional responses of tectal cells in pigeons. Visual Neuroscience, 19, 133-144. Tronick, E. (1967). Approach response of domestic chicks to an optical display. Journal of Comparative and Physiological Psychology, 64, 529-531. Wang, Y. & Frost, B. J. (1992) Time to collision is signalled by neurons in the nucleus rotundus of pigeons. Nature, 356, 236-238. Wagner, H. (1982). Flow-field variables trigger landing in flies. Nature, 297, 147-148. Wicklein, M. & Strausfeld, N. J. (2000). Organization and significance of neurons that detect change of visual depth in the hawk moth Manduca sexta. Journal of Comparative Neurology, 424, 356-376. Yonas, A. & Granrud, C. (1985). The development of sensitivity to kinetic binocular and pictorial depth information in human infants. In D. Ingle, M. Jeannerod, & D. Lee (Eds.), Brain Mechanisms and Spatial Vision (pp. 113-145). Amsterdam: Martinus Nijhoff Press.

38


SUBJECT INDEX Action.......................................................36 Auditory ...................................................37 motion 14, 16, 17, 19, 20, 25, 26, 30, 31, 32, 33, 35, 36 tau 22, 23, 25, 26, 27, 28, 29, 30, 32, 33 Flow .........................................................38 optic flow ............................................17

Information...............................................14 Looming ...................................................14 Tau ..................................................... 18, 27 local ........................................ 30, 31, 32 Time to collision ......................................38 trajectory ............................................ 19, 24 Velocity....................................................26

A Neural Network Model for the Estimation of Time-to-Collision Ling Wang1, Hongjin Sun2, and Dezhong Yao1 1

Center of NeuroInformatics, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China [email protected] 2 Department of Psychology, Neuroscience and Behaviour, McMaster University, Hamilton, Ontario, L8S 4K1, Canada

Abstract. Artificial Neural Networks (ANNs) which are derived from Biological Neural Networks (BNNs) are enhanced by many advanced mathematical techniques and have become powerful tools for solving complicated engineering problems. Integrating BNNs with mature ANNs is a very effective method of solving intricate biological problems and explaining neurophysiological data. In this paper we propose a neural network model that explains how the brain processes visual information about impending collisions with an object - in particular, how time-to-collision information is caculated in the brain. The model performs extremely well as a result of incorporating physiological data with the methods involved in the development of ANNs. By implementing this novel compuational neural network model, the results of the simulation demonstrate that this integrative approach is a very useful and efficient way to deal with complicated problems in neural computation.

1 Introduction Artificial neural networks (ANNs) that are derived from biological neural networks (BNNs), are often generated through the integration of many advanced techniques, e.g., mathematic methods, computer science, artificial intelligence etc. ANNs have become very useful tools for solving many complex engineering problems. This is partly because ANNs have many highly desirable and efficient properties and capabilities including: nonlinearity, input-output mapping, adaptivity, evidential response, contextual information, fault tolerance, VISI implementability, uniformity of analysis and design, and neurobiological analogy. The biological brain can be considered as a highly complex, nonlinear, and parallel computer (information-processing system). The brain has the capability to organize its structural constituents, known as neurons, as a way of constructing elaborate neural networks that perform certain computations (e.g., pattern recognition, perception and motor control, etc) at speeds many times faster than the fastest digital computer in existence today. Neural networks play a central role in an animals survival by virtue of the flexible manner in which they are able to interact with a complex and dynamic J. Wang et al. (Eds.): ISNN 2006, LNCS 3973, pp. 614 – 619, 2006. © Springer-Verlag Berlin Heidelberg 2006

A Neural Network Model for the Estimation of Time-to-Collision

615

environment. While neurophysiological data provides information regarding the communication between individual neurons, computational modeling approaches provide important insights in the communication that occurs within an entire network of neurons. One important application of neural networks is in the field of computational vision. The visual system provides us with a representation of the environment around us and more importantly guides us in effectively interacting with the environment. One relevant area of vision research relates to the neural computations involved in the processing of visual motion in 3-dimentional space. For example, behaviorally, it has been demonstrated that the TTC (Time-to-Collision - that is, the time elapsed before a looming object on collision course will reach the eye) is of profound importance to an animals survival by allowing them to avoid impending danger and escaping as soon as possible [1]. But how the brain processes the visual information to obtain TTC is less known. Based on the results of well-grounded artificial neural network techniques and some electrophysiological data on birds [2,3], we propose a novel computational neural network model to simulate the brain mechanisms involved in the information processing of visual motion-in-depth, including the estimation of TTC.

2 TTC-Neural Network 2.1 Physiological Basis of the Visual Processing of TTC For an animal, it is critical to be able to respond quickly and accurately to a looming object on a collision course. When an object moves towards the eyes, its retinal image size increases, and when it moves away, its retinal image size decreases. Dynamically changing the retinal image size, even of a stationary object, can produce a sensation of motion-in-depth [4,5]. Lee [5] proposed that the expansion of the retinal image of an approaching object could trigger a behavioral response and that the precise timing of the response is controlled by the optical variable called τ which happens to be equal to TTC. If the object is approaching at a constant speed, τ is equal to the inverse of the relative rate of expansion of the looming object. This can be expressed as τ = sinθ/θ′ ≈ θ/θ′ , where θ is the visual angle subtended by the looming object and θ′ is the derivative of θ over time. While, there are many behavioural studies suggesting that τ can be a very useful source of information for various aspects of visual-motor control in humans and animals [1], very few studies have illustrated how the brain might actually compute such information. It has been discovered that particular neurons in the pigeon brain selectively respond to displays of an object moving on a collision course directly towards the eye [2]. More importantly, the onset of the responses always occurs at a constant time before the object would reach the bird. This remained true for various object movement velocities and various object sizes, suggesting that these neurons encode τ by responding to a threshold value of τ [1, 2, 6]. Although it has been demonstrated empirically that such neurons can perform the computation, it is not clear how the brain BNN is organized to process this visual information to obtain TTC information.

616

L. Wang, H. Sun, and D. Yao

2.2 The Framework of the Integrated Network On the basis of both the ANN techniques and the electrophysiological findings related to collision detections in pigeons [1,2], we have previously reported some simple and convenient neural network models to simulate this biological neural network for the estimation of TTC [7,8]. However, one limitation of these earlier models is that the activation in first hidden layer could not be easily explained in physiological terms. In the current paper, we proposed an improved spatio-temporal integrated computational neural network model. To build such a model, we assumed that, as a result of learning, the neural network structure is established using optimal parameters, which have also been observed in physiological findings [9]. The framework of this integrated computational model is summarized in the Flow chart 1. This figure illustrates the entire flow of visual information from the point at which information about the approaching object is initially received, to the final acquisition of TTC information. The retinal image of the object serves as the input for the entire informational process. Subsequently, the primary perceptual system in H1 receives information about the visual angle è and as a result, sums the responses from the total of n neurons which constitutes the output of H1 and in turn serves as the inputs of H2. Next, based on the changing visual angle over time, the advanced temporal processing system in H2 detects the looming object, represented by the number of responding neurons n, and determines Flow chart 1. Visual information process whether H2_Output should be produced regarding TTC. Finally the H2_Output TTC is multiplied by the responding neuron number n of the looming object to generate the final TTC information output.

3 Implementation of the Network 3.1 Implementation of Spatial Neural Network The first level of the computational model is a spatial neural network (SNN), starting from the input (retinal image of the object) to the H1_Output. Its main role is to perceive visual angle θ and to work out the total responding neurons number n from the initial retinal image. The SNN’s structure and information processing mechanisms are proposed based on a simple perceptron neural network and some physiological findings. The physiological and anatomical basis of this structure may be located in the earlier stage of the bird’s tectofugal pathway from the retina to the optic tectum.


617

Fig.1 provides a visual interpretation of this SNN structure (Fig.1a) and information process (Fig.1b). We assume that the retinal image of the external boundary of the object would trigger the responses from all of the corresponding neurons whose receptive fields overlap with that part of the space. As a result, the spatial separation of the farthest two neurons would provide information about the visual angle, i.e. θ=|Rl-Rr|, and the number of neurons whose receptive fields correspond with the outer edge of the retinal image would be summed Fig. 1. SNN structure and information process together to n. Therefore, if the same object subtends at a different offset angle (ä in Fig.1b), then H1_Output would indicate the same visual angle θ but different responding neuron number n. This could occur, for example, if the object is on a direct collision at either ä=0 (indicated by the right orbit in Fig. 1b), or at ä=5°(indicated by the left orbit shown in Fig. 1b). It is conceivable that the more meaningful variables related to object motion on a collision course would be the object’s size and velocity. However, from the retinal point of view, the first order input is the retinal image size and the change of retinal image size and position. Consequently, we make use of the retinal image size as the input of this SNN, which should be evaluated at the retinal level before feeding into the neural network simulating processing in the higher brain areas. 3.2 Implementation of TNN The second level of our model is a temporal neural network (TNN) starting from the H1_Output and leading to the final output of the entire network (TTC), shown in Fig. 2. TNN structure(a) and information process(b) Fig.2 and also seen in [7, 8]. The processing at this level could be achieved through biological struc-tures such as the optic tectum, the nucleus rotundus and even the core of the extrostriatum which represents the later stage of the tectofugal pathway in birds.

618

L. Wang, H. Sun, and D. Yao

We employed a back-propagation neural network (BPNN) model. Here, we in Fig.2, making adopted the traditional "logsig" as a transfer function in step afferent sequential èi in different input ranges in logsig function by means of weights of opposite signs and biases with big differences. This accomplishes the task of simulating èi and 1/èi. In order to obtain information about TTC, we use the theoretical value of Tau computed through the equation 1/τ≈θ’/θ, to act as the teacher. The final error curve can be derived from the difference between the theoretical value of Tau and the real output TTC of the network. The real output of the model is a serial time sequence, which makes the output unit suitable for a threshold unit. If given a proper threshold, this will allow us to directly predict an impending collision. We also take advantage of the adaptive learning rate and the auxiliary momentum item, by adopting a gradient descend algorithm to improve the algorithm Table 1. The final optimal parameters and promote the NN performance. We assume that BPNN has some optimal of TNN parameters to respond to similar inputs quickly. We present a quick searching method in light of minimal square error and best neural network performance for obtaining the optimal parameters. It is conceivable that this optimization reflects millions of years of evolution through competition and adaptation to the surrounding environment. The final parameters are shown in Table 1 where Wij is the connective weight from the ith layer to jth layer and bij is the corresponding bias.

②

4 Results After training the neural network becomes adaptive to the inputs. Given similar inputs, it could quickly acquire accurate TTC information. Fig. 3 shows that the difference between the theoretical value and the network output in terms of the minimum mean square error is in the e-6 range. The neurons in the middle stages can also have biological implications [7, 8]. In addition, the integrated spatio-temporal neural net-work can also be applied to different collision scenarios, such as identifying the dis-tinction between different forms of motion-indepth (e.g. self-motion vs. looming[3]), different movement directions (i.e., at different offset angles[6]), and different object sizes and velocities[2]. The physiological implica-tions and their consistency with the experimental animal results have been discussed elsewhere Fig. 3. The final and middle layer results. The MSE of a, b, [7, 8] and c are 1.14e-005, 2.69e-005 and 1.48e-006 respectively. .


619

5 Conclusion and Discussions Our integrated spatio-temporal neural network is a simple and convenient computational model. Each level can be organized by using an adaptive integration of the NN algorithm and biological data. The properties of the model are consistent with response patterns observed in animals and also match the minimum consumption principle. This neural network not only allows one to obtain accurate TTC information but also has clear physiological significance. The network makes use of neurons from the entire tectofugal pathway, which had been shown to have characteristic responses to looming objects. After all, it is perhaps conceivable that this may recruit the entire visual information stream starting from the initial retinal image to gain high level information about TTC. The fact that this model performs extremely well indicates that combining physiological findings with an ANN is a very fruitful and powerful approach. Together, the well-designed structure and nearly perfect performance of the model, it not only provides a good approximation of brain mechanisms, but could also provide important insights into machine vision and robotics.

Acknowledgement This work was supported in partly by the 973 Project No.2003CB716106 and NSFC No. 30525030, 60571019.

References 1. Hecht, H., Savelsbergh, G.J.P. (ed.): Time-to-contact, Advances in Psychology Series, Amsterdam: Elsevier – North Holland (2004) 2. Sun, H.J., Frost, B.J.: Computation of Different Optical Variables of Looming Objects in Pigeon Nucleus Rotundus Neurons, Nature Neurosci. 1 (1998) 296-303 3. Frost, B. J., Sun, H.J.: The Biological Basis of Time to Collision Computation, In: Hecht, H., Savelsbergh, G.J.P. (ed.): Time-to-contact, Advances in Psychology Series, Amsterdam: Elsevier – North Holland (2004) 13-37 4. Gibson, J.J.: The Ecological Approach to Visual Perception, Houghton Mifflin, Boston (1979) 5. Lee, D.N.: A Theory of Visual Control of Braking Based on Information about Time-tocollision, Perception 5 (1976), 437-459 6. Wang, Y.C., Frost, B.J.: Time to Collision Is Signalled by Neurons in the Nucleus Rotundus of Pigeons. Nature 356 (1992), 236-238 7. Wang, L., Yao, D.Z., Sun, H.J.: A Simple Computational Model for the Estimation of Time-to-collision. In: Zhang, Y.T., Xu, L.X., Roux, C., Zhuang, T.G., Tamera, T., Galiana, H.L. (eds.): Proceedings of the 27th IEEE EMBS annual Conference (2005) 8. Yao, D.Z., Wang, L.: Visual Information Processing in Direct Collision Course —A Simple Computational Model. In: He, J.P., Gao, S.K., Lin, J.R. (eds.): Proceedings of International Conference on Neural Interface and Control (2005) 131-134 9. Guo, A.K.: Biological Neural Network, Acta Biophysica Sinica 12 (1991) 615-622

Visual Neuroscience (2002), 19, 133–144. Printed in the USA. Copyright © 2002 Cambridge University Press 0952-5238002 $12.50 DOI: 10.1017.S0952523802191127

Contextual influences on the directional responses of tectal cells in pigeons

HONG-JIN SUN, JIAN ZHAO, TRACY L. SOUTHALL, and BIN XU Department of Psychology, McMaster University, Hamilton, Ontario, Canada, L8S 4K1 (Received October 17, 2000; Accepted January 3, 2002)

Abstract Contrary to the traditional view that receptive fields are limited in spatial extent, recent studies have indicated that the response of neurons to a local stimulus within the receptive field can be modulated by stimulation of the surrounding region. Here we quantified the nature of these contextual effects on visual motion responses of neurons in the pigeon’s optic tectum using standard extracellular recording techniques. All of the cells tested responded well to a test spot moving across their receptive fields. When a background pattern was moved in the same or in a similar direction as that of the test spot, the responses of most deep tectal neurons to the test spot were maximally inhibited. Movement of the background in the opposite or near opposite direction produced minimal inhibition or even facilitation. For some deep tectal neurons, this directionally selective modulation by the moving background was maintained when the background motion was paired with different movement directions of the test spot (including both the preferred and least preferred directions). Thus, this selectivity for opposing motion was independent of the absolute direction of either the test spot or the background, a finding which is consistent with the results reported by Frost and Nakayama (1983), although they did not include all test spot directions. For some other neurons, identified here for the first time, the background movement selectively modulated the response only when the test spot moved in the neuron’s preferred directions. These neurons lost selectivity for opposing motion when the test spot moved in nonpreferred directions. The significance of these contextual effects on the motion response of tectal neurons may be related to how the brain distinguishes self-induced motion from object motion and segregates figure from ground. Keywords: Single-cell recording, Tectum, Visual motion, Direction, Center-surround interactions

as the test spot. The responses of these same neurons were much less inhibited, or even facilitated, when the background moved in the opposite direction to the test spot. A similar effect has been found in the striate cortex of the cat (Hammond, 1985; Gulyas et al., 1987; Orban et al., 1987), area MT of the monkey (Allman et al., 1985b; Born & Tootell, 1992), superior temporal area of the monkey (Tanaka et al., 1986), superior colliculus of the monkey (Bender & Davidson, 1986; Davidson & Bender, 1991) and the cat (Mandl, 1985), and the tectum of the toad (Tsai & Ewert, 1988) and the pigeon (Frost, 1978; Frost et al., 1981). In a study of pigeons’ tectum, Frost and Nakayama (1983) extended their previous findings by pairing many directions of background movement with different directions of test spot movement. They found that a group of neurons were selective for opposing motion between the test stimulus and the background, rather than the absolute directions of either the test spot or the background. Although they showed this to be true over a range of directions of the test spot, they did not include the anti-preferred (null) directions of the test spot in their protocol. This does not completely address whether the modulation is only related to the preferred direction or invariant to the direction of the test spot. The aim of the present study was to determine the magnitude of the modulation when a patterned background moved in various

Introduction The “classical” receptive field of a cell in the visual system refers to a region of the visual field in which a stimulus can elicit a response. These receptive fields have traditionally been considered limited in spatial extent and tuned to specific stimulus attributes. Stimulation outside the area by itself does not evoke a response. However, it has been discovered that if an additional stimulus is presented outside the receptive field at the same time that a stimulus is presented inside the receptive field, then the response of the cell may be modulated. These results suggest that the way in which a cell responds to a stimulus is dependent on the context within which that stimulus is presented (Sterling & Wickelgren, 1969; Nelson & Frost, 1976, 1978, see reviews by Allman et al., 1985a and by Gilbert, 1998). In their examination of motion-sensitive neurons, von Grünau and Frost (1983) found that, in the cat’s lateral suprasylvian visual areas, the responses to motion of a test spot were profoundly inhibited when a background pattern moved in the same direction

Address correspondence and reprint requests to: Hong-Jin Sun, Department of Psychology, McMaster University, 1280 Main Street West, Hamilton, Ontario, Canada, L8S 4K1. E-mail: [email protected]

133

134 directions relative to the preferred and nonpreferred directions of movement of the test spot. The prevalence, the magnitude of the response modulation, and the direction for which the neurons were selective under background modulation from the neuronal population were analyzed quantitatively. By comparing the effect of selective modulation by the background when the test spot moved in preferred versus nonpreferred directions, we found that the neurons originally thought to be sensitive to relative motion (Frost & Nakayama, 1983) should be actually divided into at least two types. One type encodes opposing motion between the test spot and background regardless of the test spot direction, and a second type is selective for opposing motion only when the test spot moves in the preferred directions. Materials and methods Pigeons (Columba livia) were anesthetized with a ketamine0 rompum mixture (50 mg ketamine, 5 mg rompum0kg initially and supplemented by hourly 12.5 mg ketamine, 1.25 mg rompum0kg, i.m.). They were positioned in a Kopf stereotaxic instrument to record single-cell responses from various sites across the tectum. Bone and dura were removed over an area on the left side of the brain, exposing the dorsal lateral tectal surface, and the right eye was sutured open. The animal’s temperature was maintained at 40– 428C by a heated water pad. All procedures were approved by the McMaster University Animal Care Committee and certified to be in compliance with the guidelines of the Canadian Council of Animal Care. Recording and data collection Standard extracellular recordings were made in the pigeon’s tectum. Parylene-coated tungsten microelectrodes (exposed tips: 10– 13 mm long; 2 MV at 1 kHz, A-M Systems, Inc, Carlsborg, WA) were used to record extracellular potentials. Penetrations were made in the dorsal lateral parts of the tectum which correspond to the projection of the fovea and the surrounding 20 deg of the visual field. Responses were recorded from the stratum griseum et fibrosum superficiale of the tectum through to the underlying ventricle (approximately 200–1300 mm in depth from the tectal surface). A systematic series of penetrations, in a direction perpendicular to the tectal surface, were made through the tectum. A stepping motorized hydraulic microdrive system with digital readout (Frederick Haer and Co., Brunswick, ME) was used to advance electrodes radially through the tectum. By zeroing the micron counter on contact with the surface of the tectum and noting the depth when the electrode emerged into the ventricle, it was possible to localize the electrode position within the major sublayers of the tectum. The signals were amplified by an A-M 1800 (A-M Systems, Inc.) differential preamplifier with a band-pass filter. The amplified signals were played over an audio monitor and displayed on an oscilloscope (Tektronix 5120) allowing the isolation of single units. The raw signal was also fed through a window discriminator (Neurofeedback Instruments, La Jolla, CA; BME441) in order to collect quantitative data from single units. Standardized square-wave pulses, generated from the window discriminator and each representing an action potential, were collected by a custom data collecting system which transferred the interspike intervals to a SGI computer after each stimulus presentation. These accumulated responses were used to produce peri-stimulus-time-histograms (PSTHs). The computer controlled both the collection of neuronal spikes and the presentation of stimuli.

H.-J. Sun et al. Visual stimuli Visual stimuli were presented on a rear-projection tangent screen which was placed 40 cm in front of the bird’s eyes and subtended a visual angle of 120 deg 3 100 deg. The visual stimulation system consisted of a SGI 4D0310GTX computer graphics system and a high-resolution LCD projector (Mitsubishi, Japan; LVP-X50). The size, position, shape, and motion of a test spot against a textured or nontextured (uniform white) background was controlled by the software. The textured background (60 deg 3 50 deg) consisted of a random distribution of black dots ranging from 0.5 deg to 2 deg in size and positioned from 0.5 deg to 2 deg apart. The test spot and the black dots in the background (both with luminance of 3 cd0m 2 ) were superimposed on a uniform background (23 cd0m 2 ). To restrict the background effect to the region beyond the classical receptive field, the region around the receptive field of the tectal neuron was masked (through software) to selectively conceal only the background but not the test spot (see Fig. 1 for a schematic illustration of the stimulus configuration). To achieve this same effect, masking in previous studies (e.g. Frost et al., 1981) was accomplished using a physical mask (e.g. a piece of cardboard) while the background and the test spot were presented from opposing sides of the tangent screen. In these studies, the background was back-projected while the test spot was frontprojected. The present method of masking, however, avoids the possible limitations of front-projection caused by the obstruction of the projection by the animal and the recording apparatus. The present method also allows for precise control of the stimulus luminance. In our experiments, the size of the mask was controlled to be about 2–3 times the diameter of the region of the excitatory receptive field to ensure that the background did not simulate any part of the classical receptive field. Testing procedures and data analysis Upon isolation of a tectal cell, the receptive-field boundary was mapped using a small moving spot. Only movement-sensitive neurons were selected for this study. The neuron’s optimal responses for test spot movement were tested in order to find the optimal test spot size and velocity. The optimal responses were determined by testing a wide range of values for a certain parameter (e.g. velocity) and locating the peak response from the tuning curves. To generate a direction tuning curve, a test spot of optimal size and velocity was moved across the receptive field against a stationary background in eight different directions (45 deg apart). Subsequently, responses were recorded for systematic combinations of test spot and background movement directions. Each testing condition was presented five to eight times in a randomly interleaved sequence. An individual neuron was typically recorded for 3 to 5 h (depending on the stability of the recording) in order to allow for testing of the complete series of manipulations. The total number of spikes evoked by a given stimulus was used as a measure for neuronal response (the stimulus movement duration was 6 s for all the trials). The stimulus velocity was always held constant at the optimal level, and thus the total number of spikes was a better measure than peak firing rate, which was sometimes more variable because of multiple peaks or peaks of odd shapes in the response histogram. Most tectal units exhibited little spontaneous firing. However, when spontaneous firing did occur, the amount was subtracted from the total number of spikes obtained during stimulus presentation. To quantify the background effect, we generated background direction tuning curves by varying the direction in which the

Contextual influences on motion responses

135

Fig. 1. The response of a neuron showing clear selectivity for opposing motion. In the diagram on the left in each panel, the central black spot indicates the test spot, the central white square indicates the mask over the receptive field, and the textured pattern represents the background. The white and black arrows indicate the movement direction of the test spot and background, respectively. The polar response plot on the right in each panel shows the total number of spikes as a function of test spot direction with background stationary (A) or background direction when the test spot moved forward (0 deg, B), backward (180 deg, C), upward (90 deg, D), and downward (270 deg, E). Each direction of motion was presented six times in a randomly interleaved sequence. Each data point represents average number of spikes over the six sweeps (all standard errors are smaller than the size of the data labels). The dotted outer circle on each of the four polar plots (B–E) represent responses to the moving test spot under the stationary background condition.

textured background moved when the test spot was moving in a particular direction. R min and R max were defined as the minimal and maximal response on the background direction tuning curves. The responses when the background moved were then compared to the responses when the background was stationary ~R stat ) in order to estimate the magnitude of the background effect. Two variables were calculated: R min 0R stat 2 1 and R max 0R stat 2 1, which quantify the amount of response modulation compared to the baseline response (i.e., when the background was stationary) in the two extreme cases. R min 0R stat 2 1 quantifies maximal background inhibition and R max 0R stat 2 1 quantifies minimal inhibition or maximal facilitation. For both variables, negative values indicate inhibition; positive values indicate facilitation and a value of 0 signifies no background effect. For example, a negative value for R min 0R stat 2 1 indicates the proportion of response reduction due to background inhibitory effect (while R min 0R stat indicates the percentage of response that remained after background motion was introduced). Paired t-tests comparing R min versus R stat and R max versus R stat were used to examine whether the background effects were significant. We recorded the neuronal responses for eight background directions (in equal intervals of 45 deg within a circle) for the same

number of stimulus repetitions. We thus considered the magnitude of neuronal firing (number of spikes) for each of the eight directions ~u1 through u8 ! as a frequency of occurrence, fi , in a certain direction within a circle. If we define the total neuronal firing in all directions as n, we may compute the sample mean angle u,N which is the mean vector of the response bias toward a certain direction (Batschelet, 1981; Zar, 1999; also see Swindale, 1998).

r5

!S ( D S ( D fi cos ui

2

fi sin ui

1

n

2

n

,

( fi cos ui cos uN 5

sin uN 5

!S !S

n

( fi cos ui n

D S 2

1

( fi sin ui n

( fi sin ui ( fi cos ui n

D S 2

1

( fi sin ui n

D

,

D

.

2

2

136 We calculated the mean angle uN from cos uN and sin u.N The mean angle is the best estimate of the neurons’ preferred direction. The variable r is a measure of concentration or sharpness of directional tuning. The value of r varies from 0 (when there is so much dispersion that a mean angle could not be described) to 1 (when all the data is concentrated in one direction, i.e. when the neuron only responded to one direction). We also performed significance tests of the directional selectivity for each direction tuning curve. The Rayleigh test was used to determine whether there was a mean direction among the population of data sampled (Zar, 1999), that is, whether there was a directional preference in the direction tuning curve. The modified Rayleigh test for uniformity versus a specified mean angle (Zar, 1999, page 618) was also used. For this study, based on previous results (Frost et al., 1981, Frost & Nakayama, 1983), it is reasonable to expect that, at least for some neurons, the background directional tuning curve should be biased towards the direction opposite to the test spot direction. Thus, in performing the modified Rayleigh test, we predicted that the tuning curve would not be uniform (not a circle) and it would have a specified mean relative direction of 180 deg (i.e. when the test spot and background move in opposite directions). We further investigated whether this directionally selective background modulation effect was consistent among different test spot directions; in other words, we tested whether the neurons were selective for opposing motion regardless of the test spot direction. A small amount of variation in this mean vector would indicate true relative direction tuning and a large amount of variation would indicate the converse. To compare different background direction tuning curves tested at different directions of test spot motion, we used the Watson-Williams test, a circular version of a multisample test (Zar, 1999, page 625). If the result of this test is not significant, we conclude that the different circular distributions have the same sample mean direction preference.

Histology Responses were recorded from neurons located over a range of depths from 200 mm to 1300 mm in depth from the tectal surface. The depth was determined by zeroing the microcounter on the microdrive upon reaching the tectal surface and was verified by piercing the ventral cavity below the tectum. At the end of the testing, small electrolytic lesions (10 mA tip negative, 10 s) were made to the recorded site. After a few hours, the animal was deeply anesthetized with pentobarbital sodium and perfused intracardially with 0.75% saline followed by 10% formol saline. The brain was extracted, cut in serial sections (each 30 mm) in the coronal plane, and stained with cresyl violet. The electrode tracks were reconstructed by reference to the lesions and traces of the electrode penetrations. The electrode depths obtained from the reading on the microdrive were found to be consistent with tip location verified by histology.

Results The results were based on observations of 85 tectal cells in 43 pigeons. Most tectal units responded vigorously to movement of a test spot on the tangent screen and their receptive fields generally varied from 2 deg to 15 deg in size. In general, receptive-field size tended to increase as the depth of the electrode tip increased. The optimal moving test spot ranged from 3 deg to 12 deg in size and

H.-J. Sun et al. from 10 deg to 80 deg0s in velocity. Most neurons responded well over a wide range of test spot velocities. About 78% of the tectal units showed directional preferences. There was a greater tendency for pigeon tectal neurons to prefer forward motion (posterior to anterior in the visual field) than any other direction and very few preferred backward motion (similar results have been found by Frost & DiFranco, 1976 and by Sun & Frost, 1997a). A few neurons were tuned to a narrow range of directions, but most were broadly tuned for direction of motion. The responses of two broadly tuned neurons are shown in two polar response plots (Figs. 1A & 2A), which are functions of the directions of movement of the test spot. Both neurons preferred forward (0 deg) motion and responded less to backward motion (around 180 deg 6 45 deg). Selective response modulation in typical neurons For some neurons, particularly neurons in deep tectal layers, the response modulation by the background was independent of the direction of motion of the test spot. Figs. 1B–1E show four background direction tuning curves that were obtained by plotting the total spikes evoked by the motion of the test spot in one of the four directions (0, 180, 90, & 270 deg), when the background also moved in each of the eight directions. In general, when the background moved in the same direction as the test spot, the response to the test spot was strongly inhibited. When the background direction moved in a different direction from the test spot, inhibition was reduced. Inhibition was minimal (or facilitation occurred) when the test spot and the background moved in opposite directions. Due to the fact that this pattern of background modulation was similar for all test spot directions, it suggested that this neuron was selective for a relative movement direction (opposing motion in particular) between the test spot and the background. More importantly, this selectivity was independent of the absolute direction of either the test spot or the background. For other neurons, also in deep layers of the tectum, the pattern and the degree of selective modulation varied with particular directions of movement of the test spot. Figs. 2B–2E illustrate the response of such a neuron. When the test spot was moved in the preferred direction (forward movement, 0 deg) (Fig. 2B), there was clearly a bias toward the opposite direction of background movement. However, when the test spot was moved in the nonpreferred direction of 180 deg (Fig. 2C), the bias for certain directions was different or disappeared. In other words, in the nonpreferred direction, the background modulation was no longer related to opposing motion between the test spot and the background. For this kind of neuron, the difference in background effect was not limited to the preferred direction as opposed to the opposite direction (0 deg vs. 180 deg in Fig. 2A). As shown in the test spot direction tuning curve (Fig. 2A), the response to test spot motion alone at 270 deg was greater for this neuron than the response at 90 deg. The background direction tuning curve for test spot motion of 270 deg (Fig. 2E) was biased toward the opposite direction while the tuning curve for 90 deg (Fig. 2D) was not. Although most neurons in the tectum were affected by simultaneous background motion, the magnitude and pattern of the background effect varied considerably among cells. In the following sections, we provide a quantitative analysis of the background effect for the population of neurons recorded. We first address the background effect when the test spot was moved in the preferred direction (out of the eight possible test spot directions when the test spot moved alone, see Fig. 1A or 2A), then we return to the


137

Fig. 2. The response of a neuron showing selectivity for opposing motion only when the test spot moved in its preferred directions. Other details are as in Fig. 1.

response modulation when the test spot moved in the direction opposite to the preferred direction (null direction).

The magnitude of the background effect in the preferred test spot direction After we obtained background direction tuning curves for movement of the test spot in its preferred direction, the magnitude of the response modulation at different background directions was compared to the response in the absence of background motion. Fig. 3 illustrates the distribution of the two indices for the relative magnitude of background modulation: R min 0R stat 2 1 (Fig. 3A) and R max 0R stat 2 1 (Fig. 3B). For at least one background direction, most of the neurons (72085, 85%) showed the effect of background inhibition (paired t-test, comparing R min and R stat , P , 0.05). The inhibition reduced the response of 49% (42085) of the neurons by more than 50%, that is, R min 0R stat 2 1 , 20.5 (Fig. 3A). For another background direction (typically the one opposite to the direction where R min occurred), maximal response occurred. R max 0R stat 2 1 averaged 20.25 (minimal inhibition) and for 24% (20085) of the neurons, it exceeded 0, indicating response enhancement (Fig. 3B). Data from the 15% (13085) of sampled neurons which failed to show a reliable background effect (i.e. no significant difference between R min and R stat ) were excluded from further analysis. These neurons tended to be located in the superficial layers of the tectum.

The background direction tuning in the preferred test spot direction For most neurons, when the background moved in the same direction as the test spot, the response to the test spot was maximally inhibited. When the background and test spot moved in opposite directions, the inhibition was typically minimal (sometimes even facilitated). Thus, the result of background modulation could be selective for the relative direction of 180 deg between the test spot and background. In the subsequent part of this paper, the direction of the background is expressed in terms of the direction relative to the test spot direction. Fig. 4A illustrates the distribution of the mean vectors for background direction in the tuning curve when direction of the background was manipulated and the test spot motion was held constant at the neuron’s preferred direction (established when tested with the test spot alone). For most cells, the background direction vector occurs mostly within a range of 180 deg 6 45 deg in relative direction (i.e. the opposite direction of movement of the center and surround) (Fig. 4A). Another dimension of the tuning curve is the extent to which the data is concentrated on one direction. The variable r in describing a circular distribution is an indicator of the degree of concentration. The relationship between an r value and its mean vector when the test spot moved in the neurons’ preferred direction is illustrated in Fig. 4B. The variation of the r value signifies the variation in magnitude of the directionally selective effect of the

138

H.-J. Sun et al. tionally selective background effects [66072, including those with selectivity for 180 deg (64072) or otherwise (2072)], and those showing nonselective effects (6072). Background effect in the null test spot direction

Fig. 3. Characteristics of the magnitude of the background modulation when the test spot was moved in a neuron’s preferred direction and the background direction was varied ~n 5 85). A: Distribution of the ratio R min 0R stat 2 1 showing the relative proportion of maximal inhibition as a function of the response when the background was stationary. B: Distribution of the ratio R max 0R stat 2 1 showing the proportion of minimal inhibition or even facilitation as a function of the response when the background was stationary.

background. For most neurons, such a selective background effect was reliable (high r values). However, for a small number of neurons, the r values were small, indicating that the background inhibited the motion response similarly, regardless of the relationship between the direction of the background and the test spot. Thus, the background effect for these neurons was not selective. To further describe the directional nature of the background effect, we ran a test of significance (the Rayleigh test) for each tuning curve to determine whether there was a directional preference. The results indicated that for most neurons (66072) the tuning curves were not uniform (not a circle) and hence the neurons had a significant ~P , 0.05) directional preference. For six other neurons, the result for the Rayleigh test was not significant, indicating no directional preference. For these six neurons, the background inhibited the motion response similarly regardless of the relative direction between the background and the test spot, thus the background effect for these neurons was not selective. We then ran the modified Rayleigh test (Zar, 1999, page 618) to examine whether the tuning curves had a specified mean angle of 180 deg in relative direction. While most neurons (64066) showed significant ~P , 0.05) results indicating they had a specific mean relative direction of 180 deg, two neurons failed to show significant results for the modified Rayleigh test (but significant for the Rayleigh test), indicating they had a directional preference for some other relative directions. Fig. 4A displays the mean vector for all the tectal neurons that showed a reliable background effect ~R min significantly smaller than R stat , 72 out of the total 85 neurons sampled) including both cells showing direc-

Fig. 5 shows the background direction selectivity when the test spot moved in the direction opposite to the preferred direction (null direction). Fig. 5A illustrates the distribution of the mean vectors for background direction when the test spot moved in the null direction. The mean vectors occur mostly within a range of 180 deg 6 90 deg in relative direction, which is a much larger range than that found for testing using the preferred direction of movement of the test spot (cf. Fig. 4A). The tests of significance (the Rayleigh test) indicated that for most neurons (62072) the tuning curves were not uniform (not a circle) and hence indicated a directional preference. The modified Rayleigh test showed that, for movement of the test spot in the null direction, 14 out of the 62 cells had a directional preference other than 180 deg, far more than the case for movement of the test spot in the preferred direction (2066, see Fig. 4A). The relationship between the r value and the mean vector when the test spot moved in the null direction is illustrated in Fig. 5B. Note that some neurons showed a directional preference to the relative directions far away from 180 deg and did so with high r values. Fifty seven out of the 72 neurons showed significant directional preference in the Rayleigh test for both the preferred and null test spot directions. Fig. 6A showed the relationship for the background direction vector when the test spot moved in the preferred and null directions. There was a much greater variation among neurons in the direction vector at the null test spot direction (along vertical axis) compared to the preferred direction (along horizontal axis). For some neurons (like the neuron shown in Fig. 1), mean tuning direction was similar for both the preferred and null directions of test spot motion (centered around 180 deg, along both horizontal and vertical axis). For some other neurons (like the neuron in Fig. 2), although the direction vector concentrated around 180 deg for the preferred direction (along horizontal axis), for the null direction, there were greater variations in the direction vectors. The relationship of the r value when the test spot moved in the preferred ~rp ! and null ~rn ! directions is plotted in Fig. 6B. There was a weak relationship between the two directions (with a correlation coefficient of 0.26). Overall, rp values (mean 5 0.12) were significantly greater than rn (mean 5 0.09) (paired t-test, P , 0.005). We ran statistical tests to evaluate the extent to which such selective background modulation was consistent among different test spot directions; in other words, whether the neurons encode relative direction regardless of the test spot direction. We directly compared the circular distributions that constitute each tuning curve using the Watson-Williams test (Zar, 1999, page 625), which is a circular equivalent of the ANOVA test. As shown in Fig. 6A, among the 57 neurons showing significant directional preference in the Rayleigh test for both the preferred and null direction, 17 of these neurons did not show a significant difference in relative directional tuning when we compared the tuning curves for preferred direction versus its opposite direction (e.g. Figs. 1B vs. 1C). This suggests that these 17 neurons encoded the same relative direction between the test spot and background. All 17 neurons also showed significant results in the modified Rayleigh test (Zar,


139

Fig. 4. Characteristics of the individual relative direction tuning curves when various background movement directions were paired with test spot movement in a neuron’s preferred direction ~n 5 72). A relative direction of 0 deg indicates the same movement direction between the background and the test spot, and a relative direction of 180 deg indicates the opposite movement between the background and the test spot. A: Distribution of the mean vector direction uN (expressed in relative directions). B: Relationship between the r value, which is the measure for direction selectivity (degree of concentration in one direction in a circular distribution) and mean vector direction. Each point represents data for a single cell.

1999, page 618) indicating that the tuning curves had a specified mean angle of 180 deg in relative direction (note that the data for those neurons are located around 180 deg along both axis in Fig. 6A). Other neurons (40057) showed a significant difference in the two sample Watson-Williams test (e.g. Figs. 2B vs. 2C), indicating they did not always encode relative direction between the test spot and background. For the 17 neurons that showed the same relative direction preference for the preferred and null direction, we also compared background direction tuning curves under multiple test spot directions using the multisample Watson-Williams test, involving four background tuning curves (e.g. the test spot directions of 0, 180, 90, & 270 deg; e.g. Figs. 1B–1E) or six tuning curves (adding two more test spot directions; e.g. 135 deg and 315 deg). Eleven neurons did not show significant differences in directional preference (seven neurons tested in four directions, four neurons tested in six directions), indicating they truly encoded relative direction between the test spot and background, regardless of the test spot directions.

The relation between selectivity for background direction and test spot direction Fig. 7 illustrates the relationship between background direction tuning and test spot directional tuning, rt (tuning sharpness generated from the test spot direction tuning curves when background was stationary). Generally, there was a very weak relationship between the rp and rt (correlation coefficient 5 0.20, Fig. 7A), between the rn and rt (correlation coefficient 5 20.03, Fig. 7B), and between the rp-rn and rt (correlation coefficient 5 0.21, Fig. 7C). The variable rp-rn represents the difference score for the background directional tuning when the test spot moved in the preferred direction compared to that in null direction. Overall, rp values (mean 5 0.12) were significantly greater than rt (mean 5 0.09) (paired t-test, P , 0.005). Discussion For most of the neurons in the deep tectum, the response to the motion of the test spot could be modulated by background motion.

140

H.-J. Sun et al.

Fig. 5. Characteristics of the individual relative direction tuning curves when various background movement directions were paired with test spot movement in the direction opposite to the neuron’s preferred direction (null direction) ~n 5 72). A: Distribution of the mean vector direction uN (expressed in relative directions). B: Relationship between the r value and the mean vector direction u.N

The tectal neuron’s responses were maximally inhibited when the background moved in a direction the same as, or similar to, the test spot direction and were least inhibited, or even facilitated, when they moved in opposite, or nearly opposite, directions. The present results are in general agreement with the initial reports of surround modulation in the pigeon tectum (Frost, 1978; Frost et al., 1981), in the superior colliculus of the monkey (Bender & Davidson, 1986; Davidson & Bender, 1991), and in area MT in monkey (Allman et al., 1985b; Tanaka et al., 1986; Lagae et al., 1989). We found that some neurons were able to encode relative directions between the test spot and the background. The selective modulation pattern was maintained when the background was paired with all tested directions of the test spot. This selectivity for relative direction was independent of the absolute direction of either the test spot or the background. It is important to note that for these neurons, the tuning curve for the relative direction was always similar for two opposite test spot directions, even for neurons with a high degree of test spot directional preference. In this case, when the test spot moved in the null test spot direction and when the background was moving in the opposite direction, the responses of the neurons were usually enhanced relative to the response under stationary background. This kind of relative direc-

tion encoding neuron has also been identified in the superior colliculus of the monkey (Bender & Davidson, 1986; Davidson & Bender, 1991). In pigeon tectum, Frost and Nakayama (1983) found a group of neurons that were selective for opposing motion between the test stimulus and the background over a range of test spot directions, although they did not include nonpreferred directions of movement. One difference between the neurons in pigeon tectum and that in the monkey colliculus is that the pigeon tectal neurons showed some facilitation to anti-phase movement while neurons in the monkey colliculus did not. However, the tuning curves for relative direction were similar between our results and those found in monkey. The tuning curves reported in our study were broader than that of the single tuning curve from pigeon tectum plotted by Frost and Nakayama (1983) and no quantitative information for the neuronal populations was reported in their study. In this study, we identified a new group of neurons which were selective modulated by the background (selective for opposing motion) only when the test spot was moved in preferred directions. When the test spot was moved in nonpreferred directions, the selectivity for opposing motion disappeared. It is unlikely this was due to the relatively low level of baseline firing at the nonpreferred


141

Fig. 6. Scatter plot of mean vector direction uN (A) and r value (B) when the test spot moved in a neuron’s preferred direction and its opposite direction. Each point represents data for a single cell. For mean vector data (A), only the neurons showing directional preference for both preferred and null directions are plotted ~n 5 57). Test of difference of direction vectors between the preferred and null direction revealed that 17 neurons were selective for the same relative direction between the test spot and background. For r value (B), all the neurons showing background effect are plotted ~n 5 72).

directions. If this were the case, the neurons with a higher rt value (generally with low firing rate at the null direction) should exhibit weaker background direction tuning at the null direction (lower rn !. However, as shown in Fig. 7B, there was no relationship between the rt and rn . Similarly, as shown in Fig. 7C, there was also no relationship between rt and the difference score between rp and rn , which provides information about the consistency of the background direction tuning among different test spot directions. Our results regarding the background effect suggest that there are at least four kinds of tectal motion-selective neurons. The motion responses of the first kind of neuron are not affected by the background movement. These neurons tend to have a very small receptive field and are located in the superficial layers of the tectum. They represent low-level motion processing since they do not integrate information beyond their “classical” receptive field. In this study, we focused on the background effect, and therefore did not sample many neurons in the superficial layers. Consequently, the proportion of this kind of neuron in the population of neurons tested in this study may not represent the true proportion within the entire tectum. The motion responses of the second kind of neuron are affected by the background movement. However, the effect is not selective

for the relative direction between the test spot and background. This type of neuron was not identified by Frost and his colleagues (Frost, 1978; Frost et al., 1981) but it has been found in cat colliculus (Rizzolatti et al., 1974). The motion responses of the third kind of neuron (such as the neuron shown in Fig. 2) are selectively modulated by the moving background (preferring opposing motion) only when the test spot moves in the neuron’s preferred direction. When the test spot moved in a nonpreferred direction, the relative directional preference differed or disappeared. This type of neuron was identified for the first time in the current study. These deep tectal neurons integrate information from the regions outside the receptive fields, although they do so only when the preferred directional signal is present in the receptive field. In fact, since the selectivity for opposing motion is greatest only when the test spot moved in the preferred directions, their responses demonstrate the existence of an interaction between the contextual effect and the neuron’s intrinsic directional preference. As a result, the firing rate of these neurons can potentially indicate the absolute direction of the motion of the test spot. The motion responses of the fourth kind of neuron (such as the neuron shown in Fig. 1) are always selectively modulated in the

142

Fig. 7. Scatter plot of rp (background direction selectivity when the test spot moved in the neuron’s preferred direction) versus rt (direction selectivity measure generated from test spot direction tuning curve) (A), rn (background direction selectivity when the test spot moved in the neuron’s null direction) versus rt (B), and rp 2 rn versus rt (C). Each point represents data for a single cell ~n 5 72).

same way by the moving background, which suggests that these neurons encode relative direction regardless of the direction of the test spot. The responses of these neurons are similar to the neurons in the superior colliculus of the monkey studied by Davidson and Bender (1991). Both the pigeon tectal and monkey collicular

H.-J. Sun et al. neurons represent a high level of abstraction in motion processing. These deep tectal neurons detect a relative directional difference between motion inside and outside of the receptive field. They can be considered to have an antagonistic, center-surround organization with respect to motion (Nakayama & Loomis, 1974; Frost & Nakayama, 1983; Nakayama, 1985). Such double-opponent processing can be achieved in any absolute direction, as long as the direction inside and outside the receptive field are opposite to each other. Due to the fact that nonpreferred directions were not tested, the neurons identified by Frost and Nakayama (1983) may include both the third and fourth types of neurons described above. Although there were neurons that clearly exhibited certain patterns of response modulation and thus could be classified into the four distinct groups (e.g. the two neurons shown in Figs. 1 & 2), there were other neurons whose degree of response modulation by the background fell into a region between the categories. For example, as shown in Fig. 6A, the degree of similarity between the mean vector of the preferred background direction at the preferred and null test spot direction was widely distributed over the graph. The data points fell on a continuum that ranged from those that were very close to 180 deg (high consistency for relative direction between preferred and null direction) to the those that were very far from that (low consistency for relative direction). Similarly, the overall effect of background modulation was weak for some neurons. They thus fell between the categories of background modulated neurons and background ineffective neurons (the first type of neurons described above). The response modulation by the background found in deep tectal neurons suggests that these neurons play a role in signalling local motion while ignoring coherent motion over a large area of the visual field. Object movement relative to the background would trigger large responses. However, body, head, or eye movements cause a large area of the retinal image to move together, a pattern that would elicit little response from these tectal neurons. It has been proposed that the tectofugal pathway primarily processes visual “object motion,” which is in contrast to the optic flow produced by the observer’s self movement (Frost, 1985, 1993; Frost et al., 1990; Frost & Wylie, 2000). The latter is generally believed to be processed by the accessory optic system referred to as the third visual pathway (Wylie & Frost, 1999a,b; also see a review by Simpson, 1984). The anatomically segregated system for object and self-motion has also been shown in the fly (Egelhaaf et al., 1988). Similar functional groupings of neurons responding to either object or self-motion have been shown in area MT (Born, 2000) and MST (Tanaka et al., 1986; Eifuku & Wurtz, 1998) of the monkey. Contextual modulation of tectal response has been used to construct a neural model to explain how the brain distinguishes object motion and self-motion in three-dimensional space (Sun & Frost, 1997b). Some neurons in the pigeon nucleus rotundus, which receives massive projections from the tectum, signal impending collision of an approaching object (Sun & Frost, 1998) but these neurons do not respond to simulation of the observer moving toward an object along the same trajectory (Frost & Sun, 1997). The receptive fields of these looming-sensitive neurons could be composed of a radial arrangement of concentric arrays of receptive fields of relative motion detectors (like the tectal neurons of the third and fourth type described above), with the center of expansion overlapping the center of receptive-field radial layout. These tectal neurons would respond only to relative movement between the test spot and its surround and then provide converging input onto rotundal looming detectors. Through this kind of receptive-

Contextual influences on motion responses field organization, the rotundal looming detectors could respond well to an approaching object, but would not respond to simulation of approach by the observer, in which the relative motion across the boundary of the looming object is minimal. Besides providing neural signals for distinguishing object motion and self-motion, the existence of deep tectal cells with different types of background modulation can explain the ability to separate figure from ground based on motion cues alone (Frost et al., 1988). Using kinematograms (the motion domain equivalence of random dots stereograms), Frost et al. (1988) have shown that these neurons were able to segregate different regions of the image into separate objects by using motion as the only physically defining characteristic. In kinematograms, the test spot can only be visible if it moves differentially from the background. Tectal responses to objects defined by kinematograms were quite similar in their responses to objects defined by luminance contrast. This suggests that the deep tectal cells were capable of separating figure from ground based on a motion signal alone. The ability of birds to distinguish a figure from its surrounding is a necessary survival technique for avoiding prey or finding food. The second, third and fourth types of tectal neurons described above would collectively achieve these functional roles. These different types of center-surround interactions may reflect complementary functions in refining various aspects of visual motion representation: distinction between object motion and self-motion, detection of absolute direction of the test spot, and segregation of figure from ground.

Acknowledgments The authors wish to thank Drs. B. Frost, D. Maurer, D. Jones, K. Murphy, and R. Racine for helpful discussions and comments on the manuscript and to thank L. Schell for proof reading of the manuscript. This work was supported by grants from the Natural Science and Engineering Research Council of Canada and the Canadian Foundation for Innovation to HongJin Sun.

References Allman, J., Miezin, F. & McGuinness, E. (1985a). Stimulus specific responses from beyond the classical receptive field: Neurophysiological mechanisms for local-global comparisons in visual neurons. Annual Review of Neuroscience 8, 407– 430. Allman, J., Miezin, F. & McGuinness, E. (1985b). Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception 14, 105–126. Batschelet, E. (1981). Circular Statistics in Biology. London: Academic Press. Bender, D.B. & Davidson, R.M. (1986). Global visual processing in the monkey superior colliculus. Brain Research 381, 372–375. Born, R.T. (2000). Center-surround interactions in the middle temporal visual area of the owl monkey. Journal of Neurophysiology 84, 2658–2669. Born, R.T. & Tootell, R.B.H. (1992). Segregation of global and local motion processing in primate middle temporal visual area. Nature 357, 497– 499. Davidson, R.M. & Bender, P.B., (1991). Selectivity for relative motion in the monkey superior colliculus. Journal of Neurophysiology 65, 1115–1133. Egelhaaf, M., Hausen, K., Reichardt, W. & Wehrhahn, C. (1988). Visual course control in flies relies on neuronal computation of object and background motion. Trends in Neuroscience 11, 351–358. Eifuku, S. & Wurtz, R.H. (1998). Response to motion in extrastriate area MSTl: Center-surround interactions. Journal of Neurophysiology 80, 282–296. Frost, B.J. (1978). Moving background patterns alter directionally specific responses of pigeon tectal neurons. Brain Research 151, 599– 603.

143 Frost, B.J. (1985). Neural mechanisms for detecting object motion and figure-ground boundaries, contrasted with self-motion detecting system. In Brain Mechanisms and Spatial Vision, ed. Ingle, D.J., Jeannerod, M. & Lee, D.N., pp. 415– 441. Dordrecht: Martinus Nijhoff Publishers. Frost, B.J. (1993). Subcortical analysis of visual motion: Relative motion, figure-ground discrimination and self-induced optic flow. In Visual Motion and its Role in the Stabilisation of Gaze, ed. Miles, F.A. & Wallman, J., pp. 159–175. Amsterdam: Elsevier. Frost, B.J. & DiFranco, D.E. (1976). Motion characteristics of single units in the pigeon optic tectum. Vision Research 16, 1229–1234. Frost, B.J. & Nakayama, K. (1983). Single visual neurons code opposing motion independent of direction. Science 220, 744–745. Frost, B.J., Scilley, P.L. & Wong, S.C.P. (1981). Moving background patterns reveal double-opponency of directionally specific pigeon tectal neurons. Experimental Brain Research 43, 173–185. Frost, B.J., Cavanagh, P. & Morgan, B. (1988). Deep tectal cells in pigeons respond to kinematograms. Journal of Comparative Physiology A: Sensory, Neural and Behavioral Physiology 162, 639– 647. Frost, B.J., Wylie, D.R. & Wang, Y. (1990). The processing of object and self-motion in the tectofugal and accessory optic pathways of birds. Vision Research 30, 1677–1688. Frost, B.J. & Sun, H.-J. (1997). Visual motion processing for figure0 ground segregation, collision avoidance, and optic flow analysis in the pigeon. In From Living Eyes to Seeing Machines, ed. Srinivasan, M.V. & Venkatesh, S., pp. 80–103. Oxford: Oxford University Press. Frost, B.J. & Wylie, D.R.W. (2000). A common frame of reference for the analysis of optic flow and vestibular information. International Review of Neurobiology 44,121–140. Gilbert, C.P. (1998). Adult cortical dynamics. Physiological Reviews 78, 467– 485. Hammond, P. (1985). Visual cortical processing: Texture sensitivity and relative motion. In Brain Mechanisms and Spatial Vision, ed. Ingle, D.J., Jeannerod, M. & Lee, D.N., pp. 389– 414. Dordrecht: Martinus Nijhoff Publishers. Gulyas, B., Orban, G.A., Duysens, J. & Maes, H. (1987). The suppressive influence of moving textured backgrounds on responses of cat striate neurons to moving bars. Journal of Neurophysiology 57, 1767– 1791. Lagae, L., Gulyas, B., Raiguel, S.E. & Orban, G.A. (1989). Laminar analysis of motion information processing in macaque V5. Brain Research 496, 361–367. Mandl, G. (1985). Responses of visual cells in cat superior colliculus to relative pattern movement. Vision Research 25, 267–281. Nakayama, K. & Loomis, J.M. (1974). Optical velocity patterns, velocity sensitive neurons and space perception: A hypothesis. Perception 3, 63–80. Nakayama, K. (1985). Biological image motion processing: A review. Vision Research 25, 625– 660. Nelson J.I. & Frost, B.J. (1976). The neural mechanism of orientation contrast. Society for Neuroscience Abstracts 2, 1083. Nelson, J.I. & Frost, B.J. (1978). Orientation selective inhibition from beyond the classic visual receptive field. Brain Research 139, 359– 365. Orban, G.A., Gulyas, B. & Vogels, R. (1987). Influence of a moving textured background on direction selectivity of cat striate neurons. Journal of Neurophysiology 57, 1792–1812. Rizzolatti, G., Camarda, R., Grupp, L.A. & Pisa, M. (1974). Inhibitory effect of remote visual stimuli on visual responses of cat superior colliculus: Spatial and temporal factors. Journal of Neurophysiology 37, 1262–1275. Simpson, J.I. (1984). The accessory optic system. Annual Review of Neuroscience 7, 13– 41. Sterling, P. & Wickelgren, B.G. (1969). Visual receptive fields in the superior colliculus of the cat. Journal of Neurophysiology 32, 1–15. Sun, H.-J. & Frost, B.J. (1997a). Motion processing in pigeon tectum: Equiluminant chromatic mechanisms. Experimental Brain Research 116, 434– 444. Sun, H.-J. & Frost, B.J. (1997b). Neuronal models of looming detectors in the nucleus rotundus of pigeon. Investigative Ophthalmology and Visual Science (Suppl.) 38/4, S362. Sun, H. & Frost, B.J. (1998). Computation of different optical variables of looming objects in pigeon nucleus rotundus neurons. Nature Neuroscience 1, 296–303.

144 Swindale, N.V. (1998). Orientation tuning curves: Empirical description and estimation of parameters. Biological Cybernetics 78, 45–56. Tanaka, K., Hikosaka, K., Saito, H., Yukie, M., Fukada, Y. & Iwai, E. (1986). Analysis of local and wide-field movements in the superior temporal visual areas of the macaque monkey. Journal of Neuroscience 6, 134–144. Tsai, H.J. & Ewert, J.-P. (1988). Influence of stationary and moving textured backgrounds on the response of visual neurons in toads (Bufo bufo L.). Brain, Behavior and Evolution 32, 27–38. von Grünau, M. & Frost, B.J. (1983). Double opponent-process mechanism underlying receptive field structure of directionally specific cells

H.-J. Sun et al. of cat lateral suprasylvian visual area. Experimental Brain Research 49, 84–92. Wylie, D.R.W. & Frost, B.J. (1999a). Complex spike activity of qs Purkinje cells in the ventral uvula and nodulus of pigeons in response to translational optic flow. Journal of Neurophysiology 81, 256–266. Wylie, D.R.W. & Frost, B.J. (1999b). Responses of neurons in the nucleus of the basal optic root to translational and rotational flow fields. Journal of Neurophysiology 81, 267–276. Zar, J.H. (1999). Biostatistical Analysis. 4th edition. Upper Saddle River, New Jersey: Prentice-Hall.

Exp Brain Res (1997) 116:434–444

© Springer-Verlag 1997

R E S E A R C H A RT I C L E

&roles:H.-J. Sun · B.J. Frost

Motion processing in pigeon tectum: equiluminant chromatic mechanisms

&misc:Received: 6 April 1996 / Accepted: 17 March 1997

&p.1:Abstract Recent psychophysical and neurophysiological studies have suggested that, in mammals, there are interactions between the P (colour processing) and M (motion processing) visual pathways, which were previously believed to be parallel and separate. In this study, the role colour information plays in the coding of object motion was determined in the tectofugal pathway of pigeons. The responses of motion-sensitive neurons in the tectum to moving stimuli formed by chromatic contrast were recorded extracellularly using standard single-unit recording techniques. A moving coloured object was presented on a uniform (opponent coloured) background (e.g. blue-on-yellow, red-on-green and black-on-white). Through systematically manipulation of the luminance contrast between object and background, an equiluminant condition was generated. It was found that, at chromatic equiluminance, the majority of cells maintain some level of response. The mean magnitude of the response at equiluminance was about one-third of the response at maximal contrast to the same chromatic border. These results suggest that tectal units can detect motion of a pattern defined by a pure colour contour, although the strength of output is considerably weaker than that for the movement of patterns formed by luminance contrast. &kwd:Key words Single-cell recording · Tectum · Visual motion · Equiluminance · Pigeon&bdy:

Introduction Although controversial in recent years, it has been accepted generally that colour and motion information are processed by separate, parallel pathways. This notion is H.-J. Sun · B.J. Frost (✉) Visual and Auditory Neurosciences Laboratory, Department of Psychology, Queen’s University, Kingston, Ontario, Canada K7L 3N6 Tel.: +1-613-545-2484, Fax: +1-613-545-2499, e-mail: [email protected]&/fn-block:

supported by anatomical, physiological and behavioural evidence (Livingstone and Hubel 1984, 1987b, 1988). In mammals, the major projection from the retina is to the lateral geniculate nuclei (LGN), which in monkeys consists of four parvocellular layers and two magnocellular layers. Parvocellular layers of LGN efferents project to V1 and eventually to V4, while magnocellular layers project to different layers of V1 and then the middle temporal area (MT; Hubel and Livingstone 1987; Livingstone and Hubel 1984, 1987a; Shipp and Zeki 1985). Electrophysiological studies have demonstrated that these anatomically defined pathways are also functionally separate. Motion information is primarily processed in a pathway possibly originating in the retina and subsequently involving the magnocellular layers of the LGN, while colour is processed in pathways connected to LGN parvocellular layers (Livingstone and Hubel 1988). The question is then to what extent are these two pathways functionally independent. If the two pathways are completely separate in the brain, perception of motion defined solely by colour (under an equiluminant condition) would not be possible. Nevertheless, recent psychophysical studies have shown that perception of motion of chromatic stimuli still occurs at equiluminance, although dramatically attenuated, thus suggesting that there are interactions between the two pathways (Cavanagh and Anstis 1991; Cavanagh and Favreau 1985). Several electrophysiological studies have also shown similar results. When recordings were made in area MT of the macaque, which has been thought to specifically process visual motion information (Maunsell and Newsome 1987), Saito et al. (1989) found that half of the motion-sensitive neurons studied responded to equiluminant stimuli, though the magnitude of the responses dropped to one-third of the maximal response, when optimal luminance contrast was presented. Gegenfurtner et al. (1994) also found that some MT neurons responded to equiluminant stimuli, although these neurons’ thresholds for equiluminance were considerably higher than the thresholds revealed from behavioural studies. Dobkins

435

and Albright (1994) also demonstrated that MT neurons were capable of coding chromatic contrast information. While visual information processing has been studied most extensively in the geniculocortical pathway, there is also the second visual pathway; the extrageniculocortical visual pathway. This pathway does not include the LGN or the striate cortex. In mammals, the superficial portion of the superior colliculus projects to the lateral posterior nucleus and/or pulvinar of the thalamus. These thalamus nuclei then project to several extrastriate cortical areas, including the lateral suprasylvian visual areas (in cats) and area MT (in monkeys; Creutzfeldt 1988; Karten and Shimizu 1989). The geniculocortical and extrageniculocortical system might both exhibit parallel processing while being complementary to each other (Karten and Shimizu 1991). The parallel arrangement of these two visual processing pathways is not limited to mammals. There is strong evidence suggesting that both the geniculocortical and extrageniculocortical visual pathway in mammals are homologous to their avian counterparts, namely the thalamofugal pathway (retina-nuclei opticus principalis thalami-Wulst) and tectofugal pathway (retina-optic tectumthalamic nucleus rotundus-ectostriatum; Karten 1969; Shimizu and Karten 1993). The tectofugal system participates in a considerable variety of visual informationprocessing tasks. In fact, this system is the major visual ascending pathway in the majority of nonmammalian vertebrates. Most birds, especially those with laterally placed eyes, have a much less well developed thalamofugal pathway (Engelage and Bischof 1993), whilst the tectofugal pathway has been well documented as the most prominent pathway (Cohen and Karten 1974; Karten 1969). Neurobehavioural studies of the two pathways in pigeons have indicated that, for tasks testing intensitydifference threshold, size-difference threshold, visual acuity and visual search, tectofugal lesions produce severe visual deficits (Hodos and Bonbright 1974; Hodos et al. 1984, 1986, 1988; Kertzman and Hodos 1988; Macko and Hodos 1984), while thalamofugal pathway lesions produce only minimal effects (Hodos et al. 1984; Macko and Hodos 1984; Pasternak and Hodos 1977). Given the prominence of the tectofugal pathway in birds, it is reasonable to expect that different classes of visual information will be processed in this pathway. In fact, in addition to motion, the tectofugal pathway is known to mediate some visual functions performed primarily by the geniculocortical pathway in mammals. For example, lesions in tectofugal structures produce significant deficits on colour, brightness, pattern, and size discrimination (see review, Hodos 1993). Just as parallel visual information processing streams have been found within the geniculostriate visual system of mammals, there are a number of anatomical, electrophysiological and behavioural studies showing multiple parallel channels within the tectofugal pathway. In birds, neurons in different sublayers of the optic tectum project to distinct regions of the nucleus rotundus (Hunt and Künzle 1976; Karten and Revzin 1966; Revzin and Kar-

ten 1966–1967) and, from these regions, rotundal neurons project topographically to several subdivisions of the ectostriatum (Benowitz and Karten 1976; Karten and Hodos 1970). Studies of the responses of single cells to a variety of visual stimuli have also revealed some evidence for parallel processing of visual information about luminance, colour, pattern and object motion in two-dimensional (2D) and three-dimensional (3D) space in this pathway (Granda and Yazulla 1971; Jassik-Gerschenfeld and Hardy 1984; Revzin 1979; Wang et al. 1993; Yazulla and Granda 1973). Having witnessed the intensive debate on whether there exist separate and parallel processing streams, and the possible interactions between magnocellular and parvocellular pathway in the geniculostriate system of mammals, it is important to ask similar questions of the tectofugal pathway of birds. It has been clearly established that the tectofugal pathway is involved in processing motion information. Tectal neurons in birds, like superior colliculus neurons in mammals, respond optimally and often exclusively, to a visual stimulus moving across the receptive field, and their receptive fields consist of a relatively small excitatory receptive field and a large inhibitory surround (Frost and DiFranco 1976; Frost and Nakayama 1983; Hughes and Pearlman 1974; Jassik-Gerschenfeld and Guichard 1972; Jassik-Gerschenfeld et al. 1970). Some of these motion-sensitive tectal cells show a directional preference or specificity, while the majority of them respond to a wide range of directions. It has been proposed that the tectofugal pathway primarily processes visual “object motion”, which contrasts with the optic flow produced by the observer’s self-movement (Frost 1982, 1985, 1993; Frost and Sun 1997; Frost et al. 1990, 1994). The latter is generally believed to be processed by the accessory optic system (AOS), the third visual pathway (see review, Simpson 1984). Unlike mammals, very little is known about how colour information is processed in the central nervous system (CNS) of birds, although behaviourally there is strong evidence that birds are able to process colour information (see review, Varela et al. 1993; Wright 1979). There is some evidence showing that colour processing occurs in the tectofugal pathway. Behavioural studies following lesions of the nucleus rotundus (the equivalent structure to pulvinar), which is the processing nucleus between tectum and ectostriatum (possibly the equivalent structure to MT in monkey), have revealed deficits in colour discrimination (Hodos 1969). Furthermore, colour opponent-units have been found in the nucleus rotundus (Maxwell and Granda 1979; Wang et al. 1993; Yazulla and Granda 1973). Since the tectum receives a major input from retina and sends a massive projection to the nucleus rotundus (Hunt and Künzle 1976; Karten and Revzin 1966; Revzin and Karten 1966–1967), it is quite likely that colour processing occurs in the tectum itself. Indeed, Jassik-Gerschenfeld et al. (1977) found that the majority of the tectal units in their study responded to static chromatic stimuli that were suddenly changed in wavelength, while luminance was held constant.

436

In the experiments reported here, we examined the extracellular responses of single cells from the pigeon’s tectum to pure colour stimuli. We determined whether motion-sensitive neurons in the tectum were also sensitive to chromatic stimuli. An object moving against a background can be detected not only by luminance contrast but also by colour contrast. If presentation of a chromatic object moving against a equiluminant background still triggers the neuron’s responses, a colour signal could be used by this neuron for motion analysis. If, however, the responses disappear under equiluminant conditions, this neuron must be encoding motion through luminance information alone.

Materials and methods Pigeons (Columba livia) were anaesthetized with a ketamine/rompum mixture (50 mg ketamine, 5 mg rompum/kg initially and supplemented by hourly 12.5 mg ketamine, 1.25 mg rompum/kg i.m.). They were positioned in a Kopf stereotaxic instrument to record single-cell responses from various sites across the tectum. Bone and dura were removed over an area on the left side of the brain exposing the tectal surface, and the right eye was sutured open. The animal’s temperature was maintained at 40–42° C by a heated water pad. This project was reviewed by the Queen’s University Animal Care Committee and certified as in compliance with the guidelines of the Canadian Council of Animal Care. Recording and data collection Standard extracellular recordings were made in the pigeon’s tectum. Parylene-coated tungsten microelectrodes (exposed tips 10–13 µm long; 2 MΩ at 1 kHz; A–M Systems) were used to record extracellular potentials. Responses were recorded starting from the stratum griseum et fibrosum superficiale (SGFS) of the tectum (approximately 300 µm in depth from the tectal surface) through to the underlying ventricle (approximately 1500 µm in depth from the tectal surface). A systematic series of penetrations were made through the tectum surface (with electrodes perpendicular to the tectal surface). A stepping motorized hydraulic microdrive system with digital readout (Frederick Haer) was used to advance electrodes radially through the tectum. By zeroing the micron counter on contact with the surface of the tectum and noting the depth of the electrode submerged in the ventricle, it was possible to localize electrode position precisely within the major sublayers of the tectum. The signals were amplified by an A–M 1800 (A–M Systems) differential preamplifier with a band-pass filter. The amplified signals were played over an audio monitor and displayed on a digital storage oscilloscopy (B & K, Precision 2520) allowing the isolation of single units. The raw signal was also fed through a window discriminator (Neurofeedback Instruments, BME441) in order to collect quantitative data from single units. Standardized square-wave pulses, generated from the window discriminator, each representing an action potential,were collected by a custom-made data collecting system, which transferred the interspike intervals to the IRIS computer after each stimulus presentation, and these accumulated responses were used to produce peristimulus-time histograms (PSTHs). The computer controlled both the collection of neuronal spikes and the presentation of stimuli.

Table 1 Colour projector phosphor characteristics (CIE Commission Internationale de l’Eclairage)&/tbl.c:& Colour

Red Green Blue a

Wavelength at peak energy (nm)

CIE chromaticity coordinatesa x

y

630 530 450

0.66 0.29 0.14

0.33 0.56 0.07

An international standard for primary colours&/tbl.:

a visual angle of 80°×60°. The visual stimulation system consisted of a high-resolution (1280×1024 pixels) Electrohome Trinitron colour projector (ECP4000) and a Silicon Graphics IRIS 4D/310GTX computer graphics system. This system permits independent control of red, green and blue beam intensities. Although this system did not permit variation of wavelength, it permitted qualitative manipulation of “colour” by mixing beams to produce equivalents of spectral and non-spectral colour. The software used in the system also allowed control of the size, position, shape and motion of an object against a background of uniform colour. The luminance and colour of both object and background could be manipulated independently. The luminance of object and background at each colour were calibrated by a chroma meter (Minolta CS100). The spectral peaks of the three beams of the Electrohome ECP4000 are given in Table 1. Qualitative testing and quantitative data collection Neurons were selected only if they responded to luminance-defined object motion. Upon isolation of a tectal cell, the receptive field boundary was mapped, using either a light spot from a handheld ophthalmoscope or a small dark spot. Neurons’ optimal responses for object movement were then tested using a black and white pattern in order to find the preferred velocity, object size and direction (tested with eight movement directions, 45° apart). A moving coloured object was then presented on a uniform (opponent-coloured) background (e.g. red-on-green, blue-on-yellow, black-on-white, and so on). When the luminance of either the stimulus object or background was kept constant at an intermediate level (range 5–30 cd/m2) while the luminance level of the other was varied over a wide range in small steps, there must be a pair of values when the luminance of the object and background are perceptually identical for a particular “observer” (in this case a neuron). At such equiluminant levels, presumably a “luminancesensitive” neuron’ response will be minimal compared with various other luminance ratios. Because luminance ratios at equiluminance might vary across different neurons, it is necessary to investigate for each cell the response to various luminance ratios for the same colour combination. To locate such equiluminant levels, an approximate equiluminant level was obtained initially through presentation of a wide range of luminance contrasts using large steps. A finer tuning curve was then obtained (using up to 256 levels of luminance contrasts in 0.02 log unit steps) around the equiluminant level obtained from the coarser set of contrasts. If necessary, these procedures were repeated several times in order to find the exact equiluminance condition that produced the minimum response. The luminance level that produced a minimum response was considered the “equiluminant condition” for that cell and for that particular object and background colour combination.

Results Visual stimuli Visual stimuli were presented on a rear-projection tangent screen, which was placed 40 cm in front of the bird’s eyes and subtended

The present results were based on observations made on a total of 60 tectal cells in 29 pigeons. Most tectal units responded vigorously to movement of an object on the

437

Fig. 1A–C The magnitude of the response of a tectal neuron to a moving object is plotted against the object-background luminance contrast. A Depicts the response to pure luminance contrast between object and background (a grey-grey pattern; B/W). The neuron’s response significantly decreased around equiluminance and actually dropped to zero at the exact equiluminant point. B, C The response to the opponent pairs (red-green, R/G, and blue-yellow, B/Y, respectively). The response to the two colour combinations dropped to almost zero when the luminance ratio was close to 0.1 log unit for the red-green colour combination and 0.6 for the blueyellow combination, suggesting these were the equiluminant conditions for this cell&ig.c:/f

Fig. 2A–C The magnitude of the response of another typical neuron to a moving object is plotted against the object-background luminance contrast. A Depicts the response to pure luminance contrast between object and background (a grey-grey pattern). This neuron’s response significantly decreased around equiluminance and actually dropped to zero at the exact equiluminant point. However, the response of this cell was maintained at a quite high level in the equiluminant chromatic condition, about one-third of the response at maximum contrast. This occurred for both red-green (B) and blue-yellow (C) colour combinations&ig.c:/f

tangent screen. Only movement-sensitive neurons were selected in this study. These neurons responsed to objects moving from 10 to 160°/s in velocity. Their respective fields generally varied from 2° to 20° in size. About half of the tectal units (total 34) showed directional preference. Among them some were tuned to a narrow range of directions, while most others were broadly tuned for direction of motion. To test chromatic responses, two different objectbackground colour combinations were used: red and green, blue and yellow. Of the 60 cells in the sample, 37 cells were tested for the full range of luminances for both red-green and blue-yellow combinations. For the remaining cells only one colour combination was tested. Most tectal units exhibited little spontaneous firing. However, when it did occur, the spontaneous firing was

subtracted from the total number of spikes obtained during stimulus presentation. Only the incremental spikes have been attributed to neuronal response to the stimuli. General responses The response behaviours of two typical tectal neurons are illustrated in Figs. 1 and 2. In Figs. 1A and 2A, the magnitude of the response to a moving object is plotted against the object-background luminance contrast (a grey-grey pattern). Both cells’ response significantly decreased around equiluminance and actually dropped to zero at the exact equiluminant point. Figure 1B and C illustrates the response to the opponent pairs (red-green

438

and blue-yellow, respectively) in addition to luminance contrast. For this cell, the response to the two colour combinations dropped to almost zero when the luminance ratio was close to 0.1 log unit for the red-green colour combination and 0.6 for the blue-yellow combination. This suggests that this cell cannot signal motion at certain luminance contrast levels, which is presumably the equiluminance point of the cell. A different pattern of response for chromatic stimuli was found in other cells, an example of which is shown in Fig. 2B, C. There was only a relatively shallow dip in the magnitude of the response of the cell to chromatic luminance contrasts for both red-green (Fig. 2B) and blueyellow (Fig. 2C) colour combinations. The response of the cell was maintained at a quite high level (about onethird of the maximum) even at equiluminance. The majority of tectal cells showed a response behaviour similar to the second case, where the minimum response was, on average, about one-third of the maximal response. The key to successful implementation of this experimental manipulation is to ensure that the various decrements of the luminance contrast do not miss the point of equiluminance. In this experiment, for each cell, a number of contrast levels (with about 24 contrast levels in each series) were presented. First, a series of luminance contrasts with a rather large contrast interval between each condition were presented. Then, based on the results of this series, finer tuning was obtained by presenting a series of contrasts in smaller steps around the point of minimum response. The process was repeated until the contrast intervals were close to the system’s limits. The resolution of the computer system allows 256 luminance levels. The luminance contrast steps can be as small as 0.02 log units in the mid-luminance range. In fact, in most cases, there was little need to test with such small luminance steps. From Figs. 1A and 2A, in which achromatic stimuli were used to obtain pure luminance contrasts, it is evident that the decline in the responses of these units occurred over quite a wide range of luminance contrasts, with a half-width at half-height of more than 0.1 log units. Given that the fine resolution tuning curve was obtained with step sizes typically of 0.05 log units, it is unlikely that the null point could be overlooked. Before concluding that the residual response at equiluminance is a response to a pure colour contour, it is necessary to consider the effect of the transient luminance changes generated at the edge of the object by chromatic aberration. Because tectal cells generally show high sensitivity to luminance contrast, such a luminance change might cause the residual response. Using small achromatic stimuli 0.25°, 0.5° and 1° in width, moving against a background, we tested six cells with the largest luminance contrast (10:1; black on white, and white on black). It was found that the cells generally did not respond to 0.25° stimuli. A 0.5° stimulus only produced a small response in two of the six cells tested. A 1° stimulus produced some response, although still far below optimal for all six cells. Based on the analysis by Saito et

al. (1989), it would be safe to say that chromatic aberration (which could spread boundaries over a range smaller than 0.05°) is not the cause of the residual response at equiluminance. We therefore concluded that the residual response of tectal cells at equiluminance has its origin largely in a chromatic contrast input. Although the magnitude of the mean response decreases significantly at equiluminance compared with the response at maximum luminance contrast, tectal cells can still detect motion of a pattern composed of a pure colour contour. Minimum-maximum response ratios To evaluate quantitatively the responsiveness of tectal neurons to the movement of equiluminant colour contrasts, we calculated the ratio between the magnitude of the response at the equiluminant point (minimal response point) and response to the same colour stimulus with the largest luminance contrast. The latter were derived from the mean of two responses, one from an object-background luminance ratio of 1:10 (−1 log unit) and the other from a ratio of 10:1 (1 log unit). Because the neurons almost always produced maximal firing at these two maximal luminance contrasts, the response ratio mentioned above can be called simply the minimal-maximal response ratio (Min/Max). A Min/Max ratio of zero indicates that there is no neuronal response at equiluminance, while a Min/Max ratio of 1 indicates that the neuron responds at equiliuminance as much as at maximal luminance contrast. A Min/Max ratio of 0.5 shows the neuronal response at equiluminance is only half of the response at maximal luminance contrast. It was found that this Min/Max response ratio for both red-green and blue-yellow combinations varied across cells from 0 to 0.9, with a mean of 0.40 for the red-green combination (n=52) and 0.34 for the blue-yellow combination (n=45). Figure 3 shows the distribution of Min/Max response ratios for the red-green (Fig. 3, top) and blue-yellow (Fig. 3, bottom) combinations. It is quite evident that the Min/Max response ratio varied widely across the population of neurons tested, but no distinct peak was apparent in either distribution. An analysis of the Min/Max ratio for red-green and blue-yellow combinations and the tectal depth from which they were recorded revealed no systematic pattern. In fact neurons that exhibited both a substantial residual response at chromatic equiluminance and no response at equiluminance appeared to be scattered throughout the various tectal laminae. This does not imply that there is no spatial or laminar segregation of chromatically sensitive neurons, but simply that their responses might have been recorded from radial axonal processes as well as from some located in different laminae.

439

Fig. 4 A scatter diagram showing the relationship of minimum/maximum (MIN/MAX) response ratios between the redgreen and blue-yellow conditions measured on the same cells. See text for detailed description&ig.c:/f

Fig. 3 The distribution of the ratios between the minimal response at equiluminance and the response at maximal luminance contrast to the same chromatic contrast (Min/Max ratio). It is quite evident that the Min/Max response ratio varied widely across the population of neurons tested. The mean was 0.40 for the red-green combination and 0.34 for the blue-yellow combination&ig.c:/f

Chromatic properties that make up the colour contrast In order to examine the sensitivity of tectal cells to movements of pure colour contours, and also their sensitivity to the kind of colour at the same time, two different colour combinations were tested. It is of interest to ascertain whether the same tectal cells show the same or different levels of response at equiluminance for different colour combintions. A scatter diagram (Fig. 4) represents the relationship of Min/Max response ratios between the red-green and blue-yellow conditions measured on the same cells. For the majority of cells (about two-thirds) those with a large Min/Max response ratio for one colour combination tended to also have a large ratio for the other colour combination, while cells with a small Min/Max response ratio for one colour combination tended to also have a small ratio for the other colour combination. However, the general trend was that the plotted points were distributed below the 45° line, which represents equal responses to red-green and blue-yellow contrast, i.e. the line of Min/Max ratio (r–g)=Min/Max ratio (b–y). In other words, the general trend was for cells to respond to redgreen slightly more than blue-yellow.

It is also important to note that, for some other cells (about one-third), the difference in the responses to the two colour combinations was quite large. This indicated that some cells might be more responsive to one colour combination than the other. To examine this issue quantitatively, we arbitrarily defined a cut-off criterion, categorizing those cells with Min/Max response ratios larger than 0.3 as “colour-sensitive cells”. It was found that of the 37 cells that were tested with both red-green and blue-yellow combinations, 26 units (70%) met the criteria of colour-sensitive cells. Of these 26 colour-sensitive units, 17 (65%) responded to both colour combinations, while 6 (23%) responded only to the red-green combination, and 3 (12%) responded only to the blue-yellow combination. Eleven out of the thirty-seven units tested (30%) gave little response at chromatic equiluminance (less than 30% of the maximal response) and were thus classified as luminance contrast units. Distribution of the luminance ratio at equiluminant points The distribution of the luminance ratio of red-green and blue-yellow at which each cell’s response dropped to its minimum is shown in Fig. 5 top and bottom, respectively. The distributions were generally narrow, although there were a few units somewhat remote from the mode. The peak in luminance ratio distribution was 1.4:1 (0.15 log unit) for red-green and 5.0:1 (0.7 log unit) for blueyellow.

440

Fig. 5 The luminance ratios between red and green, and between blue and yellow, at which a cell’s response dropped to its minimum is shown at the top (red-green) and bottom (blue-yellow), respectively. The peak in the luminance ratio distribution is 1.4:1 (0.15 log units) for red-green and 5.0:1 (0.7 log units) for blueyellow conditions&ig.c:/f

Discussion Although the magnitude of the response of most tectal units decreased at certain object-background luminance ratios for chromatically contrasting stimuli, for some cells, the response still remained at a substantial level. We therefore concluded that tectal units can detect motion of a pattern composed of pure colour-contours, though the strengh of the output is considerably weaker than for the movement of patterns with luminance contrast. Because very fine steps of luminance contrast were used, judging from the contrast response function of cells to the achromatic stimulus, the experiment eliminates the possibility of a confound arising from residual luminance contrast. Moreover, the failure of very small objects to produce a tectal response suggests that chromatic aberration was not responsible for the residual response at equiluminance. Colour processing in birds Colour is an important dimension in the vision of many animals. There is overwhelming and conclusive evidence

from behavioural experiments and ecological observations that birds are able to process colour information. Pigeons, for example, have been extensively studied on wavelength discrimination and colour mixture (Blough 1957, 1972; Hamilton and Coleman 1973; Wright 1972, 1979), and their discriminative performance was found to be comparable with human data. In fact, birds’ colour vision matches and perhaps even surpasses that of humans in several ways. The retinal structures that make this impressive chromatic performance understandable have been known for some time. Using microspectrophotometry, Bowmaker (1977) identified at least three different cone photo pigments. These pigments are located in cones that also have prominent oil droplets with different spectral transmissions operating as cut-off filters. Birds have retinal constituents capable of giving rise to a tetra- or even a pentachromatic space (Bowmaker 1980; Norren 1976; Romeskie and Yager 1976). It is important to note that the light filters produced by the retinal oil droplets are selective at the individual receptor level, affecting the absorbance spectra of only the individual cone on which the droplet is situated (Muntz 1972). The different combinations of cone pigments with oil droplets are commonly considered the source of spectral sensitivity variation between receptors. Thus, visual neurons fed by different cone-oil drop combinations may have sensitivity variations. This could be the reason that the neuronal responses to the same colour contour generally vary from cell to cell and why some of the tectal cells in the present study even fall into distinct sensitivity groups (for different colours). While there are several studies concerned with the retinal photobiological mechanism and behavioural aspects of birds’ chromatic abilities, there are very few physiological analyses of neural mechanisms of colour vision (see review, Varela et al. 1993). One possible reason for this may be that attention was first directed to the thalamofugal pathway, whose colour-processing functions have been well studied in mammalian species. In fact, studies of the nucleus opticus principalis thalami (the avian homologue of the LGN of mammals) revealed no colour-selective responses (Maxwell and Granda 1979). However, in a much smaller thalamic area, the ventral lateral geniculate nucleus (GLv), colour-opponent neurons (50%) have been reported (Maturana and Varela 1982). GLv apparently does not exist in mammals and it is not part of either the thalamofugal or tectofugal ascending pathways. Even though colour sensitivity is found in this structure, bilateral lesions of the GLv do not lead to sustained deficits in colour discrimination (Varela et al. 1993). Therefore, it appears that the thalamofugal pathway in birds does not play a significant role in colour processing. The most likely candidate for colour processing in birds is the tectofugal pathway. Given the prominance of the tectofugal pathway, it is likely that colour information, as well as motion, is processed in this pathway. Indeed, a number of studies suggest tectofugal structures

441

participate in colour processing. Behavioural studies of birds with rotundal lesions have revealed deficits in the discrimination of intensity and colour of stimuli (Hodos 1969). Rotundal units have been reported to be selective for the wavelength of light stimuli (Maxwell and Granda 1979; Granda and Yazulla 1971) and exhibit opponent colour processing (Wang et al. 1993; Yazulla and Granda 1973). Given that the tectum provides a major input to the nucleus rotundus, it is most probable that the tectum itself also processes colour information. Moreover, the fact that the other known colour-processing nucleus, GLv, connects reciprocally and topographically to the tectum and receives afferent connections from the Wulst (Crossland and Uchwart 1979), but does not connect to any other visual centres, further suggests that the tectum is involved in colour processing. Earlier neurophysiological studies support this view. Jassik-Gerschenfeld et al. (1977) used stimuli consisting of a stationary spot illuminated with light of changing wavelengths. By substituting stimuli of the same luminance but of different colours, they showed that a majority (61 of 73) of the pigeon tectal units responded to the change of colour, but no distinct groups of chromatic sensitivity were found. Unfortunately, no quantitative description of the responses were provided, and it was not reported whether these same cells responded to motion as well. Varela et al. (1983) also recorded from pigeon tectal units and found that 35% of the tectal cells responded to some chromatic property. Our study confirmed and extended previous findings and showed that a substantial number of motion-sensitive tectal units responded to chromatic stimuli at equiluminance. Moreover, a majority of the units responded to colour contour regardless of the chromatic properties that make up the colour contrast. This is similar to the response of MT neurons to equiluminant chromatic stimuli. Dobkins and Albright (1994) reported that MT neurons could respond to the movement of the stimulus in the direction of the nearest chromatically defined borders, even when the colour contrast that made up the border alternated over time. It appears that MT neurons respond to the colour contrast while ignoring the chromatic information that creates the contrast. The exact role these cells play in processing of colour information is not clear. In the current study, there were also a number of units that seemed to particularly respond to either red-green or blue-yellow colour combinations. Exactly which colour (in the colour pair that made up the colour border) the neurons responded to was hard to determine with the small visual stimulus patterns that we employed (a small square of 2°–4°). When a chromatic object passes through the receptive field, the colour contrast of the leading and trailing edges are opposite. It is possible that the cell responds to either or both leading or trailing edges (or contrasts). In future research,it might be possible to use a long chromatic bar, which would provide a long delay between the responses to the leading edge and re-

sponses to the trailing edges. This way the relative contributions of either edge or direction of the colour contrast could be differentiated. Comparison with behaviourally determined spectral sensitivity functions One fundamental aspect of colour vision is the spectral sensitivity of the animal, which describes the variation in threshold as a function of wavelength. The first thorough behavioural assessment of the photopic spectral sensitivity of pigeons was determined by Blough (1957) and was later replicated in several laboratories. Recent studies report differences in spectral sensitivity as a function of location in the visual field (Martin and Muntz 1978, 1979). Using spectral sensitivity data to adjust stimuli differing in wavelength so that they are equiluminant has been a common practice in psychophysical research on colour. In this experiment, equiluminant points were determined from tectal neuronal responses. It is interesting to compare these physiologically determined equiluminant points with behaviourally determined spectral sensitivity. In the present experiment, based on single-cell responses to various levels of luminance contrast, the redgreen luminance ratio at equiluminance is around 0.15 log units (equivalent to a luminance ratio of 1.4:1). In other words, luminance for a red stimulus (630 nm) has to be 0.15 log units higher than that of the green stimulus (530 nm) to make it equiluminant. This result is comparable with the spectral sensitivity found in behavioural tests. In the spectral-sensitivity functions found by Martin and Muntz (1979) for pigeons, the relative sensitivity at 630 nm is about 0.2 log units lower than that at 530 nm. The blue-yellow luminance ratio at equiluminance found in this experiment was around 0.7 log units. In other words, the luminance for blue (450 nm) has to be 0.7 log units higher than that of the yellow (estimated to be around 575 nm) to make it equiluminant. From the same behaviourally determined spectral-senstivity curve mentioned above, the relative sensitivity at 450 nm is about 1.0 log units lower than that at 575 nm. While the neuronally determined equiluminance for red-green is relatively close to the red-green sensitivity difference found in the spectral-sensitivity curve, the equiluminance for blue-yellow does not exactly match blue-yellow sensitivity difference. It seems the neuronal sensitivity difference between blue and yellow is not as large as that found in behaving pigeons, and a number of factors might have contributed to this difference. First, spectral sensitivity describes the variation in threshold as a function of wavelength (by assuming all stimuli at threshold are equal in luminance), while, in the equiluminant condition found in the current study, both object and background luminance are well above threshold. It is not impossible that, at supra-threshold levels, the system performs “magnitude estimation” differently (e.g. with different rates of amplification) relative to the

442

performance for theshold detection for different colours. In other words, there might to be some non-linearity in the colour-sensitivity system from threshold to suprathreshold levels. Secondly, the stimulus presentation system employed here did not allow precise control of wavelength composition. Therefore, the slight differences between our results and behavioural data might result from the broader wavelength range that was employed compared with the “monochromatic” stimuli typically used in spectral-sensitivity studies. Moreover, there is a lack of homogeneity in the distribution of cones and oil drops in the pigeon’s retina. Our spatial sampling of neurons was restricted to only a limited region of the visual field, while the pigeon’s psychophysical responses might be influenced by information integrated over much wider areas of the retina. It is important to note that pigeon’s retina can be divided into two separate areas according to the proportions of the different oil droplet types contained in them. The red field, which is located in the dorsotemporal quadrant of the retina, owes its appearance to a large portion of the oil drops that transmit only long-wavelength light, while the yellow field, which represents the rest of the retina, contains oil drops which transmit most of the visual spectrum. The shape of the spectral-sensitivity function has been found to vary depending on which retinal field the pigeon uses to view the stimulus (Martin and Muntz 1979; Wright 1979). Two different spectralsensitivity functions were obtained when a behavioural paradigm was used that separately targeted either the red or yellow fields of the pigeon retina (Martin and Muntz 1979). In our recordings of single-unit responses in the tectum, all of the neuron’s receptive fields were within the yellow field’s projection area, and thus the spectralsensitivity function obtained from the yellow field was used to compare with the single-cell data. Our results are in closer agreement with the spectral function for the yellow field rather than data found in the red field. Parallel processing compared with co-processing of motion and colour information in the same cell Parallel processing of different kinds of visual information in the mammalian geniculostriate system has long been accepted by most researchers. In birds, the optic tectum, rather than the LGN, is the major processing centre for ascending visual information, and the tectofugal pathway has been proposed to carry several different streams of information, at least in birds with laterally placed eyes. The nucleus rotundus has been shown to be involved with processing information about colour (Granda and Yazulla 1971; Hodos 1969; Yazulla and Granda 1973), luminance (Hodos and Karten 1966) and motion (Revzin 1979). In most of these studies, typically only one class of stimuli was tested. Moreover, little anatomical localization data were provided to see whether the neuronal specificities were segregated. Wang et al. (1993) systematically tested nucleus rotundus neurons

with a battery of stimuli possessing different attributes. By using computer-generated stimuli and systematically varying recordings sites, combined with histological confirmation, their studies provided quantitative data that revealed several functionally distinct subdivisions within the nucleus rotundus, where at least four classes of visual selective cells are clustered; areas specializing in luminance, colour, 2D motion directions and looming stimuli. These functional divisions closely matched histological and acetylcholinesterase subdivisions found by others (Benowitz and Karten 1976; Martinez-de-la-Torre et al. 1990). These results suggest that, as in mammals, different types of visual processing may be executed in parallel in the visual system of birds. Although the study by Wang et al. (1993) generally supported the idea that colour and motion information are processed in different clusters of neurons, given their spatial proximity within the nucleus rotundus, it is possible that the neuronal interaction between these subdivisions might occur in nucleus rotundus. In fact, some preliminary testing showed that some rotundus neurons gave responses to combinations of colour and motion (S. Jiang, Y.C. Wang and B.J. Frost, personal communication). The obvious next step is to test the independence of colour and motion processing in the nucleus rotundus. The current study suggests that, in the tectum, motion-processing units can use colour-contrast information in addition to luminance differences. This result is quite consistent with mammalian studies that demonstrate interactions occur between mechanisms for colour and motion processing, although the luminance channel provides a much stronger motion signal than does the colour channel (Dobkins and Albright 1994; Gegenfurtner et al. 1994; Saito et al. 1989). &p.2:Acknowledgements The authors wish to thank T. Kripalani and S. David for technical assistance and S. Marlin for helpful discussions and comments on the manuscripts. H.-J.S. was supported by a Postgraduate Scholarship from the Natural Science and Engineering Research Council (NSERC) of Canada. This work was supported by an NSERC grant OGP0000353 to B.J.F.

References Benowitz LI, Karten HJ (1976) Organization of the tectofugal visual pathway in the pigeon: a retrograde transport study. J Comp Neurol 167:503–520 Blough DS (1957) Spectral sensitivity in the pigeon. J Opt Soc Am 47:827–833 Blough PM (1972) Wavelength generalization and discrimination in the pigeon. Percept Psychophys 4:342–348 Bowmaker JK (1977) The visual pigments, oil droplets and spectral sensitivity of the pigeon. Vision Res 17:1129–1138 Bowmaker JK (1980) Colour vision in birds and the role of oil droplets. Trends Neurosci 3:196–199 Cavanagh P, Antis S (1991) The contribution of color to motion in normal and color-deficient observers. Vision Res 31:2109–2148 Cavanagh P, Favreau OE (1985) Color and luminance share a common motion pathway. Vision Res 25:1595–1601 Cohen DH, Karten HJ (1974) Structural organization of avian brain: an overview. In: Goodman IJ, Schein MW (eds) Birds: brain and behavior. Academic Press, New York, pp 29-73

443 Creutzfeldt OD (1988) Extrageniculo-striate visual mechanisms: compartimentalization of visual functions. In: Hicks TP, Benedek G (eds) Vision within extrageniculo-striate systems. Elsevier, Amsterdam, pp 307–320 Crossland WJ, Uchwart CJ (1979) Topographic projections of the retina and optic tectum upon the ventral lateral geniculate nucleus of the chick. J Comp Neurol 185:87–106 Dobkins KR, Albright TD (1994) What happens if it changes color when it moves? The nature of chromatic input to macaque visual area MT. J Neurosci 14:4854–4870 Engelage J, Bischof H-J (1993) The organization of the tectofugal pathway in birds: a comparative review. In: Zeigler HP, Bischof H-J (eds) Vision, brain, and behavior in birds. MIT Press, Cambridge, Mass, pp 137–158 Frost BJ (1982) Mechanisms for discriminating object motion from self-induced motion in the pigeon. In: Ingle DJ, Goodale MA, Mansfield R (eds) Analysis of visual behaviour. MIT Press, Boston, Mass, pp 177–196 Frost BJ (1985) Neural mechanisms for detecting object motion and figure-ground boundaries, contrasted with self-motion detecting system. In: Ingle DJ, Jeannerod M, Lee DN (eds) Brain mechanisms and spatial vision. Martinus Nijhoff, Dordrecht, pp 415–441 Frost BJ (1993) Subcortical analysis of visual motion: relative motion, figure-ground discrimination and self-induced optic flow. In: Miles FA, Wallman J (eds) Visual motion and its role in the stabilisation of gaze. Elsevier, Amsterdam, pp 159–175 Frost BJ, DiFranco DE (1976) Motion characteristics of single units in the pigeon optic tectum. Vision Res 16:1229–1234 Frost BJ, Nakayama K (1983) Single visual neurons code opposing motion independent of direction. Science 220:744–745 Frost BJ, Sun H-J (1997) Visual motion processing for figure/ground segregation, collision avoidance, and optic flow analysis in the pigeon. In: Srinivasan MV, Venkatesh K (eds) From living eyes to seeing machines. Oxford University Press, Oxford, pp 80–103 Frost BJ, Wylie DR, Wang Y (1990) The processing of object and self-motion in the tectofugal and accessory optic pathways of birds. Vision Res 30:1677–1688 Frost BJ, Wylie DR, Wang Y-C (1994) The analysis of motion in the visual systems of birds. In: Davies MNO, Green PR (eds) Perception and motor control in birds. Springer, Berlin Heidelberg New York, pp 248–269 Gegenfurtner KR, Kiper DC, Beusmans JMH, Carandini M, Zaidi Q, Movshon JA (1994) Chromatic properties of neurons in macaque MT. Vis Neurosci 11:455–466 Granda AM, Yazulla S (1971) The spectral sensitivity of single units in the nucleus rotundus of pigeon, Columba livia. J Gen Physiol 57:363–384 Hamilton WF, Coleman TB (1973) Trichromatic vision in the pigeon as illustrated by the spectral discrimination curve. J Comp Physiol 15:183–191 Hodos W (1969) Color discrimination deficits after lesions of the nucleus rotundus in pigeons. Brain Behav Evol 2:185–200 Hodos W (1993) The visual capabilities of birds. In: Zeigler HP, Bischof H-J (eds) Vision, brain, and behavior in birds. MIT Press, Cambridge, Mass, pp 63–76 Hodos W, Bonbright JC (1974) Intensity difference thresholds in pigeons after lesions of the tectofugal and thalamofugal visual pathways. J Comp Physiol Psychol 87:1013–1031 Hodos W, Karten HJ (1966) Brightness and pattern discrimination deficits in the pigeon after lesions of nucleus rotundus. Exp Brain Res 2:151–167 Hodos W, Macko KA, Bessette BB (1984) Near-field acuity changes after visual system lesions in pigeons. II. Telencephalon. Behav Brain Res 13:15–30 Hodos W, Weiss SR, Bessette BB (1986) Size-threshold changes after lesions of the visual telencephalon in pigeons. Behav Brain Res 21:203–214 Hodos W, Weiss SR, Bessette BB (1988) Intensity difference thresholds after lesions of ectostriatum in pigeons. Behav Brain Res 30:43–53

Hubel DH, Livingstone MS (1987) Segregation of form, color, and stereopsis in primate area 18. J Neurosci 7:3378–3415 Hughes CP, Pearlman AL (1974) Single unit receptive fields and the cellular layers of the pigeon optic tectum. Brain Res 80:365–377 Hunt SP, Künzle H (1976) Observations on the projections and intrinsic organization of the pigeon optic tectum: an autoradiographic study based on anterograde and retrograde axonal and dendritic flow. J Comp Neurol 170:153–172 Jassik-Gerschenfeld D, Guichard J (1972) Visual receptive fields of single cells in the pigeon’s optic tectum. Brain Res 40:303–317 Jassik-Gerschenfeld D, Hardy O (1984) The avian optic tectum: neurophysiological and behavioral correlations. In: Vanegas H (ed) Comparative neurology of the optic tectum. Plenum Press, New York, pp 649–686 Jassik-Gerschenfeld D, Minois F, Conde-Courtina F (1970) Receptive field properties of directionally selective units in the pigeon’s optic tectum. Brain Res 24:407–421 Jassik-Gerschenfeld D, Lange RV, Ropert N (1977) Response of movement detecting cells in the optic tectum of pigeons to change of wavelength. Vision Res 17:1139–1114 Karten HJ (1969) The organization of the avian telencephalon and some speculations on the phylogeny of the amniote telencephalon. Ann NY Acad Sci 167:164–179 Karten HJ, Hodos W (1970) Telencephalic projections of the nucleus rotundus in the pigeon (Columba livia). J Comp Neurol 140:35–51 Karten HJ, Revzin AM (1966) The afferent connections of the nucleus rotundus in the pigeon. Brain Res 2:368–377 Karten HJ, Shimizu T (1989) The origins of neocortex: connections and lamination as distinct events in evolution. J Cogn Neurosci 1:291–301 Karten HJ, Shimizu T (1991) Are visual hierarchies in the brains of the beholders? Constancy and variablity in the visual system of birds and mamals. In: Bagnoli P, Hodos W (eds) The changing visual system: maturation and aging in the central nervous system. Plenum Press, New York, pp 51–59 Kertzman C, Hodos W (1988) Size-difference thresholds after lesions of thalamic visual nuclei in pigeons. Vis Neurosci 1:83–92 Livingstone MS, Hubel DH (1984) Anatomy and physiology of a color system in the primate visual cortex. J Neurosci 4:309–356 Livingstone MS, Hubel DH (1987a) Connections between layer 4B of area 17 and thick cytochrome oxidase stripes of area 18 in the squirrel monkey. J Neurosci 7:3371–3377 Livingstone MS, Hubel DH (1987b) Psychophysical evidence for separate channels for the perception of form, color, movement, and depth. J Neurosci 7:3416–3468 Livingstone MS, Hubel D (1988) Segregation of form, color, movement, and depth: anatomy, physiology, and perception. Science 240:740–749 Macka KA, Hodos W (1984) Near-field acuity after visual system lesions in pigeons. I. Thalamus. Behav Brain Res 13:1–14 Martin GR, Muntz WRA (1978) Spectral sensitivity of the red and yellow oil droplets fields of the pigeon (Columba livia). Nature 620–621 Martin GR, Muntz WRA (1979) Retinal oil droplets and vision in the pigeon (Columba livia). In: Granda AM, Maxwell JH (eds) Neural mechanisms of behavior in the pigeon. Plenum Press, New York, pp 307–325 Martinez-de-la Torre M, Martinez S, Puelles L (1990) Acetylcholinesterase-histochemical differential staining of subdivisions within the nucleus rotundus in the chick. Anat Embryol 181:129–135 Maturana HR, Varela FJ (1982) Color-opponent responses in the avian lateral geniculate: a study in the quail (Coturnix cortunix japonica). Brain Res 247:227–241 Maunsell JHR, Newsome WT (1987) Visual processing in monkey extrastriate cortex. Annu Rev Neurosci 10:363–401 Maxwell JH, Granda AM (1979) Receptive fields of movementsensitive cells in the pigeon thalamus. In: Granda AM, Max-

444 well JH (eds) Neural mechanisms of behavior in the pigeon. Plenum Press, New York, pp 177–197 Muntz WRA (1972) Inert absorbing and reflecting pigments. In: Dartnall HJA (ed) Photochemistry of vision. (Handbook of sensory physiology, vol VII, part 1). Springer, Berlin Heidelberg New York, pp 529–565 Norren DB (1976) Two short wavelength sensitive cone systems in pigeon, chicken and daw. Vision Res 15:1164–1166 Pasternak T, Hodos W (1977) Intensity difference thresholds after lesions of the visual Wulst in pigeons. J Comp Physiol Psychol 91:485–497 Revzin AM (1979) Functional localization in the nucleus rotundus. In: Granda AM, Maxwell JH (eds) Neural mechanisms of behavior in the pigeon. Plenum Press, New York, pp 165–175 Revzin AM, Karten HJ (1966–1967) Rostral projections of the optic tectum and the nucleus rotundus in the pigeon. Brain Res 3:264–276 Romeskie M, Yager D (1976) Psychophysical studies of pigeon color vision. I. Photopic spectral sensitivity. Vision Res 16:501–505 Saito H, Tanaka K, Isono H, Yasuda M, Mikami A (1989) Directionally selective response of cells in the middle temporal area (MT) of the macaque monkey to the movement of equiluminous opponent color stimuli. Exp Brain Res 75:1–14 Shimizu T, Karten HJ (1993) The avian visual system and the evolution of the neocortex. In: Zeigler HP, Bischof H-J (eds) Vi-

sion, brain, and behavior in birds. MIT Press, Cambridge, Mass, pp 103–114 Shipp S, Zeki S (1985) Segregation of pathways leading from area V2 to areas V4 and V5 of macaque monkey visual cortex. Nature 315:322–325 Simpson JI (1984) The accessory optic system. Annu Rev Neurosci 7:13–41 Varela FJ, Letelier JC, Marin G, Maturana HR (1983) The neurophysiology of avian color vision. Arch Biol Med Exp 16:291–303 Varela FJ, Palacios AG, Goldmsmith TH (1993) Color vision in birds. In: Zeigler HP, Bischof H-J (eds) Vision, brain, and behavior in birds. MIT Press, Cambridge, Mass, pp 77–98 Wang YC, Jiang S, Frost BJ (1993) Visual processing in pigeon nucleus rotundus: luminance, color, motion, and looming subdivisions. Vis Neurosci 10:21–30 Wright AA (1972) Psychometric and psychophysical hue discrimination functions for the pigeon. Vision Res 12:1447–1464 Wright AA (1979) Color vision psychophysics: a comparison of pigeon and human. In: Granda AM, Maxwell JH (eds) Neural mechanisms of behavior in the pigeon. Plenum Press, New York, pp 89–127 Yazulla S, Granda AM (1973) Opponent-color units in the thalamus of the pigeon. (Columba livia). Vision Res 13:1555–1563

Computation of different optical variables of looming ... - CiteSeerX

Computation of different optical variables of looming ... - CiteSeerX

Suggest Documents

Computation of different optical variables of looming ... - CiteSeerX

Robust Computation Of Optical Flow - CiteSeerX

Quantum computation over continuous variables

Specialized Architectures for Optical Flow Computation - CiteSeerX

Large Displacement Optical Flow Computation without ... - CiteSeerX

Reliable and Efficient Computation of Optical Flow - CiteSeerX

Computation of Discontinuous Optical Flow by Domain ... - CiteSeerX

Translation of variables for different applications in

Optical Holonomic Quantum Computation

Computation of surface roughness using optical correlation

Synthesis and optical properties of different colloidal ... - CiteSeerX

Effect of Environmental Variables - CiteSeerX

Looming Vulnerability

ROBUST MULTIRESOLUTION COMPUTATION OF ... - CiteSeerX

NUMERICAL COMPUTATION OF COMPLEX ... - CiteSeerX

Foundations of Field Computation - CiteSeerX

Comparing Models of Computation - CiteSeerX

1 ANALYSIS OF THE INFLUENCE OF THE DIFFERENT VARIABLES ...

Effect of Different Variables on the Size Distribution of Barium

An Adaptive Bias in the Perception of Looming Auditory ... - CiteSeerX

Performance analysis of different multi-user optical

Structural and Optical Properties of Different ...

Radiative effect of cirrus with different optical

The computation of linear and nonlinear optical constants of ...