Exp Brain Res (1999) 129:615–628
© Springer-Verlag 1999
R E S E A R C H A RT I C L E
Ignace Th.C. Hooge · Jaap A. Beintema A.V. van den Berg
Visual search of heading direction
Received: 26 May 1998 / Accepted: 8 June 1999
Abstract When we move along we frequently look around. How quickly and accurately can we gaze in the direction of heading? We studied the temporal aspects of heading perception in expanding and contracting patterns simulating self-motion. Center of flow (CF) eccentricity was 15°. Subjects had to indicate the CF by making a saccade to it. A temporal constraint on the response time was introduced, because stimuli were presented briefly (1 s). On average, subjects needed two saccades to find the CF. Initial saccades covered about 50–60% of the distance between the fixation point and the CF. Subjects underestimated the eccentricity of the CF. The systematic radial error ranged from –2.4° to –4.9°. The systematic tangential error was small (about 0.5°). The variable radial error ranged from 2.7° to 4.6°. We found a relation between saccade onset time and saccade endpoint error. Saccade endpoint error decreased with increasing saccade onset time, suggesting that saccades were often fired before the heading processing had been completed. From the saccade onset times, saccade endpoint errors and an estimate for the saccadic dead time (interval prior to the saccade during which modification is impossible 70 ms), we estimated the heading processing time (HPT 0.43 s). In three out of four subjects, HPT was longer for trials simulating backward movement than for trials simulating forward movement. For each saccade we determined whether it reduced the distance error. The second saccade reduced the error more effectively per time unit than the initial saccade. On the basis of this finding, we suggest that visual processing that occurs during the saccadic dead time of the first saccade is used in the preparation of the second saccade. I.Th.C. Hooge (✉) Department of Comparative Physiology, Neuroethology Group, Utrecht University, PO Box 80085, 3508 TB Utrecht, The Netherlands e-mail:
[email protected] I.Th.C. Hooge · J.A. Beintema · A.V. van den Berg Helmholtz School for Autonomous Systems Research, Department of Physiology, Erasmus University Rotterdam, Rotterdam, The Netherlands
Key words Human · Saccades · Latency · Heading · Visual search
Introduction Gibson (1966) coined the term “optic flow” for the motion pattern received by a moving vantage point. The optic flow pattern is expanding if the observer moves forward and contracting if the observer moves backward. The center of flow (CF) coincides with the heading direction or the direction from which the observer recedes in the case of the eye not rotating relative to the environment. In natural situations the eye is not stationary in the head. What kind of eye movements occur during exposure to expanding or contracting patterns? Lappe et al. (1998) measured monkey’s eye movements during exposure to expanding and contracting patterns. Epochs of slow eye movements that were directed along the local flow and saccades that directed gaze to a new location alternated. Thus the smooth eye movements in successive intervals were often differently directed, but otherwise rhythmic alternation between saccadic and smooth eye movements was reminiscent of optokinetic nystagmus (OKN). Remarkably, the gain of the slow eye movements was quite low (0.2–0.6). Duration of the slow phase (which can be seen as an intersaccadic interval) was of the order of 400 ms. Heading detection can take place during the slow phases. Because the eyes are moving continuously, the CF is not stationary on the retina but shifts abruptly due to saccades and continuously due to slow eye movements. All models of heading attempt to find the retinal direction of heading from one instance of the flowfield (Rieger and Lawton 1985; Hildreth 1992; Perrone and Stone 1994; Lappe and Rauschecker 1994; van den Berg and Beintema 1997; Royden 1997). They ignore the difficulty that this direction is not stable over time. From heading discrimination experiments it is known that the heading discrimination threshold is between 1° and 4°. This is measured both during fixation and during eye movements (van den Berg 1992; Royden et al.
616
1992). Subjects had to compare the heading direction with one (or more) lines that were presented directly following stimulus presentation. Thresholds for a rotating and stationary eye did not differ. For example, in the experiment of Royden et al. (1992), rotation rate of the eye was 5°/s. During the presentation time of 1.25 s, the heading direction shifts 6.25° along the retina. However, the average heading error was much smaller (1.9°) and comparable to thresholds for a stationary eye. To transform the heading direction from retinal coordinates to head centric coordinates, it has to be corrected for the actual orientation of the eye. Thus, humans appear to be able to cope with the problem of change of retinal heading direction due to eye movements. This implies that the representation of the heading direction has to be detected and updated fast enough; otherwise we would have expected large shifts in the responses to the CF. What do we know about the time the brain needs to detect the heading direction? From many experiments it is known that a presentation time of 1 or 2 s is sufficient for an observer to find out the heading direction (van den Berg 1992; van den Berg and Brenner 1994). Even shorter presentation times still allow observers to detect their heading direction. In the experiment by te Pas et al. (1998), presentation time was 228 ms. After stimulus presentation, subjects had to indicate with a cursor where they saw the center of flow (CF). A presentation time of 228 ms was ample to enable subjects to point to the CF with average pointing errors that varied from 0° to 4°. Pointing errors were larger than in conditions having a longer presentation time. Another example of a heading task with short presentation times is provided by the experiment by Crowell et al. (1990). Observers had to detect a horizontal difference in heading direction between two expanding patterns. The discrimination threshold decreased inversely proportional to presentation time when presentation time was shorter than 300 ms. For longer presentation times the threshold was constant. The results of both studies suggest that 300 ms is sufficient to detect the heading direction. However, stimulation time must not be confused with available processing time (Salthouse and Ellis 1980). Processing of visual stimuli may continue after stimulus presentation. Thus, heading discrimination tasks and pointing tasks are not suitable for the investigation of heading processing time because there are no constraints on the response time. The results of Crowell et al. (1990) and te Pas et al. (1998) only provide an estimation of the minimal stimulation time. However, we do not want to exclude the possibility that 300 ms is ample to detect the direction of heading. To measure the heading processing time we need a method that gives a temporal constraint on the processing. A way to estimate the upper boundary of heading processing is to measure the time required to look with a saccade to the CF. The relation between saccade onset time and saccade endpoint error may reveal temporal aspects of heading processing. Saccade endpoint error is defined as the gaze angle between the heading direction
and saccade endpoint. For saccades that are generated before the visual processing is complete, we expect saccade endpoint error to decrease with increasing saccade onset time. Beforehand we do not know whether saccades will be made before the CF is located. However, an example of saccades generated independently of the immediate visual task is found in the results of Hooge and Erkelens (1998). Their stimulus consisted of a hexagonal pattern of 35 “C”s and one “O”. The gaps in the “C”s pointed in the direction of the “O”. Subjects were asked to find the “O” by making saccades from one “C” to another “C”. They were explicitly instructed to make saccades in the direction of the gap of the “C” fixated. Twenty to 40% of the saccades were made in directions other than indicated by the gap in the “C”. These incorrectly directed saccades were preceded by short fixation times, whereas correctly directed saccades were preceded by longer fixation durations. Thus, the relation between the direction error and the fixation duration reflects the state of the analysis process as a function of time. In a heading task in which subjects are asked to fixate the CF, we expect the same to occur. We expect saccade endpoint error to increase with decreasing saccade onset time. Is the saccade onset time a good measure for heading processing time? The time preceding a saccade consists at least of time needed for visual processing and for saccade preparation. These two processes may occur in parallel and may overlap in time (Viviani 1990; Hooge and Erkelens 1996, 1998). This means that if subjects are able to fixate the CF after one saccade, the saccade onset time can be interpreted as the upper boundary for the processing time of a heading task. Just before the eyes leave there is a period during which the saccade can no longer be stopped or changed. This period is called the saccadic dead time. The saccadic dead time is estimated to be 70 ms (Hooge et al. 1996). Direction and amplitude of a saccade reflect the state of the processing at the start of the saccadic dead time. Thus, the heading processing cannot be equated to the saccade onset time, as this would overestimate the real heading processing time by the saccadic dead time. Our first question in the present study was: How much time does the brain need to detect the CF in an expanding or contracting pattern? To find an answer to this question we engaged four subjects in a heading task. Subjects were asked to fixate the CF. Saccade endpoint errors were related to saccade onset times. If the time preceding the initial saccade is too short to determine the CF, subjects will need more than one saccade to look to the CF. If so, this experimental method also enables us to study saccade patterns during the search for the CF.
Materials and methods Subjects Four male subjects (IH, JB and AB were the authors) participated in the experiments (ages 27–40 years). All subjects were experienced in wearing scleral coils for eye movement recording.
617
Fig. 1 Set-up. Subjects sat in front of a large screen and looked at a perspective projection of a moving box filled with dots. The moving box caused an expanding (forward movement) or contracting (backward movement) pattern on the screen. The center of flow (CF) corresponded to the heading direction. Eye movements were measured with the scleral coil method (not drawn)
Stimuli Stimuli were backprojected (Sony VPH 1270QM projection television; only the red tube was used) on a translucent screen (distance 1.5 m; size 59.6°×57.6°) in a completely darkened room. The frame rate of the projection television was 120 Hz. To prevent head movements, a bite-board was used. Subjects looked monocularly with the right (IH, JB and MF) or the left eye (AB). Stimuli were generated on an SGI Onyx workstation. To simulate forward or backward self-motion, we presented on the screen a perspective projection of a moving box filled with dots (Fig. 1). The projection of the moving box caused an expanding (forward self-movement) or contracting (backward movement) pattern on the screen. The center of flow (CF) corresponded to the heading direction. Directions of the movement of the box were chosen such that the CF was projected in a random direction at 15° eccentricity from the center of the screen. The simulated box (13.76×13.20×8.00 m) contained 1152 red dots. Each dot had diameter of 0.8°. The distance between the observer and the front of the simulated box was 8 m at stimulus onset. During 1 s, forward or backward motion of the box (speed 3 m/s) was simulated. Subjects never saw the complete box. They viewed the stimulus through an aperture (size 57.6°×59.6°). Therefore, depending on the displacement of the box, the number of dots visible on the screen varied. On average 277 moving dots were visible at the beginning of the trial. At the end of the trial on average 217 (forward movement) or 317 (backward movement) dots were visible. Dot size was constant (i.e., did not scale with distance).
which they were receding. If found, they had to fixate that direction until the end of the trial. To estimate the heading processing time, we need a transition from large to small saccade endpoint errors as a function of saccade onset time (see “Introduction”). To evoke sufficient early saccadic responses (express saccades; Fischer and Ramsperger 1984) in 50% of the trials, the fixation marker disappeared 200 ms before stimulus onset (gap trials). In the remaining 50% of the trials (no gap trials), the fixation marker remained visible during stimulus presentation. Inclusion of gap trials is expected to broaden the distribution of initial saccade onset times, which is desirable because we do not know in advance the value of the heading processing time. In the case of forward movement, subjects looked at an expanding pattern. In such a pattern, individual dots move away from the CF. For backward motion, however, the local motion is towards the CF. In equal amounts, we mixed trials with forward and backward movement. Thus, the direction of the local motion was not informative of the direction of the CF. If an observer relies only on the local motion to quickly look at the CF, we expect a high proportion of early responses that carry the line of sight away from the CF if: (1) the observer expects forward movement when the movement is backward or (2) the observer expects backward movement when the movement is forward. In summary, the experiment consisted of 400 trials. There were four types of trial: (1) forward/gap, (2) backward/gap, (3) forward/no-gap and (4) backward/no-gap. These trials (100 per type) were mixed in random order. Subjects did not receive practice before doing the experiment. Eye movement recordings Movements of the right (IH, JB and MF) or left eye (AB) were measured with an induction coil mounted in a scleral annulus in an a.c. magnetic field (Skalar eye position meter 3020, Delft, the Netherlands). This method was first described by Robinson (1963) and refined by Collewijn et al. (1975). The horizontal and vertical eye orientations were measured at a sampling rate of 500 Hz. Eye orientation measurements started 500 ms before presentation of the optic flow pattern and continued for 1.5 s. The signals were fed through a low-pass analogue filter with a cutoff frequency of 250 Hz, prior to sampling by the computer. Data were stored on disk for offline analysis. Data analysis Data were analyzed offline. A computer program searched for saccades with a velocity threshold of 10.0°/s. The precise moment of saccade onset was determined by computing the mean and SD of the presaccadic velocity in an interval ranging from 100 to 50 ms before the initially detected start of the saccade. The instant at which velocity exceeded the mean presaccadic velocity with 3 SD was taken as saccade onset. Saccade offset was determined by using the postsaccadic velocity in an analogous way (see also van der Steen and Bruno 1995). Saccades shorter than 12 ms and saccades that started before appearance of the optic flow pattern were excluded from the analysis.
Results Effect of gap on saccade onset time
Procedure A trial started with the presentation of a fixation marker (a red dot, size 0.8°) at the center of the screen. Subjects were asked to fixate the marker that remained visible between 1.5 and 2.8 s. Subsequently, the stimulus appeared for 1 s. Subjects were asked to indicate with their eyes the heading direction or the direction from
Did the introduction of the gap paradigm broaden the distribution of saccade onset times for initial saccades? Figure 2 shows distributions of saccade onset times of initial saccades of both the gap and the no-gap trials. Except for subject JB, distributions were shifted slightly
618
Fig. 2 Distributions of onset times of initial saccades. To evoke early saccadic responses, the gap paradigm was introduced in 50% of the trials. In a gap trial, the fixation marker disappeared 200 ms before stimulus onset. Fat lines show the distribution of initial saccade onset times obtained from trials that were preceded by a gap. Thin lines show the distribution of initial saccade onset times obtained from the “no gap” trials. Data were pooled in bins of 0.05 s. We did not find express saccades
when the stimulus was preceded by a gap (Fig. 2). Mean latencies were 50 ms shorter for MF, 60 ms shorter for AB and equal for JB and IH. The saccade onset time distributions are unimodal. We did not find express saccades. Initial saccades with latencies shorter than 120 ms were rare. Thus, for three subjects we were able to broaden the distribution of first latencies. However, the effect of the inclusion of gap trials was small. Effect of simulated self-movement direction If simulated movement is forward, local velocities point in the direction opposite to the CF. If simulated movement is backward, local velocities point in the direction of the CF. Were saccade parameters different for backward and forward ego-motion? If a subject knows whether he is moving backward or forward, the local velocity cue may help to find the CF. Figure 3a shows the fraction of saccades away from the CF for trials with
Fig. 3A–C Incorrectly directed initial saccades. A Fraction of initial saccades having an unsigned direction error (|ß|) larger than 90°. Dashed bars indicate the fraction obtained for trials in which simulated motion was forward; black bars indicate the fraction ob-
tained for trials in which simulated motion was backward. B Onset time of the initial saccade. Dashed bars indicate the onset time obtained for trials in which simulated motion was forward; black bars indicate the onset time obtained for trials in which simulated motion was backward. Error bars indicate standard errors of the mean. C Onset time of the initial saccade. Dashed bars indicate the onset time for initial saccades that went in the direction of the CF; white bars indicate the onset time for saccades that went in the direction opposite to the CF. Error bars indicate standard errors of the mean
619 Fig. 4A–C Amplitudes and number of saccades. A shows the distribution of saccade amplitudes. B depicts the mean saccade amplitude. Error bars denote standard error of the mean. C depicts the mean number of saccades per trial
Fig. 5A–C Distribution of the number of saccades, saccade amplitude and fixation duration. A depicts the number of trials that contained one, two or more saccades. B depicts the mean saccade amplitude versus saccade number. C depicts fixation duration versus saccade number. Fixation duration is the time between two saccades. Error bars denote standard error of the mean
simulated forward and backward movement. Saccades away from the CF are those initial saccades that have an unsigned direction error that is larger than 90°. The fraction is obtained by dividing by the total number of initial saccades. Direction error is defined by the angle (screen coordinates) between the vector from the saccade start point to the saccade endpoint and the vector between the saccade start point and the CF. The majority of the initial saccades (Fig. 3, 61–95%) went in the correct direction. In the backward condition we found a higher fraction of saccades that carry away the line of sight from the CF than in the forward condition (Fig. 3a). Differences between the two fractions (χ2-test for two independent samples; Siegel 1956) were significant for subjects AB (P
