1991; Ferster 198 1; Fischer and Krueger. 1979; Freeman and Ohzawa 1990; Joshua and Bishop 1970; ...... Collett 199 1) . In a subsequent study it was reported ...
JOL'RNALOF NEUROPHYSIOLOGY Vol. 76, No. 5. November 1996. Printed in U.S.A.
Neural Processing of Stereopsis as a Function of Viewing Distance in Primate Visual Cortical Area Vl YVES TROTTER, SIMONA CELEBRINI, BRIGITTE STRICANNE, SIMON THORPE, AND MICHEL IMBERT Centre de Recherche Cerveau et Cognition, Centre National de la Recherche ScientiJique, Faculte’ de Mdecine de Rangueil, Universite’ Paul Sabatier, 31062 Toulouse Ce’dex, France SUMMARY
AND
CONCLUSIONS
I. The influence of viewing distance on disparity selectivity was investigated in area Vl of behaving monkeys. While the animals performed a fixation task, cortical cells were recorded extracellularly in the fovea1 representation of the visual field. Disparity selectivity was assessed by using static random dot stereograms (RDSs) through red/green filters flashed over the central fixation target. To determine the influence of the viewing distance, a color video monitor was positioned at fixed distances of 20,40, or 80 cm. The same RDSs with the same angular size of dots were used at the three distances. 2. Disparity sensitivity was tested on 139 cells, of which 78 were analyzed at two or more distances and the rest (61) at a single distance. When disparity selectivity was analyzed at a given distance, about half the cells were found to be selective at 40 or 80 cm, but only a third at 20 cm. Near cells were 2 1.5 times more common than far cells at all three distances. The latency distribution of the responses of disparity-selective (OS) cells was similar at all three distances, with a mean distribution centered around 60 ms. 3. Changing the viewing distance drastically affected the neural activity of the Vl neurons. The visual responsiveness of 60 of 78 cells (77%) was significantly changed. Disparity selectivity could be present at a given distance and absent at other(s), with often a loss of visual response. This emergence of disparity coding was the strongest effect (28 of 78 or 36%) and occurred more frequently from short to long distances. Among the cells that remained disparity insensitive at all recorded distances (3 1 of 78 or 40%)) about half also showed modulations of the amplitude of the visual response. For cells that remained DS at all recorded distances ( 13 of 78 or 17%), changing the viewing distance also affected the sharpness (or magnitude) of disparity coding in terms of level of visual responsiveness and those changes were often combined with variations in tuning width. In only two cells did the peak of selectivity type change. Finally, the activity of four DS cells was not affected at all by the viewing distance. 4. Another effect concerned the level of ongoing activity (OA), defined as being the neural activity in darkness preceding the flash of the visual stimulus while the monkey was fixating the small bright target. Changing the viewing distance resulted in significant changes in OA level for more than half of the cells (41 of 78 or 53%). The most common effect was an increase in OA level at the shorter distance. The modulations of both visual responsiveness and OA could occur simultaneously, although they often had opposite signs. Indeed, the two effects were statistically independent of each other, i.e., modulations of visual responses were not related to the level of excitability of the neurons. 5. Control experiments were performed that showed that the effects of changing the viewing distance were not due to the retinal patterns in that the modulations of visual responsiveness were independent of the dot density. Seventeen cells were also tested for a possible effect of vergence by the use of prisms. When there was 2872
0022-3077/96
$5.00
Copyright
an effect of distance, it could be closely or partially reproduced by using prisms. These controls, together with the effects observed on OA, strongly suggest that the modulations of neural activity of the Vl neurons by the viewing distance are extraretinal in origin, probably proprioceptive. 6. The modulation of visual responsiveness by the viewing distance in the primary visual cortex indicates that integration of information from both retinal and extraretinal sources can occur early in the visual processing pathway for cortical representation of three-dimensional space. A functional scheme of three-dimensional cortical circuitry is discussed that shows cortical areas where disparity selectivity and modulations of visual activity by the angle of gaze have been described so far.
INTRODUCTION
Some of the most important sources of information that the visual system can use to help perform accurate motor acts are the binocular cues that underlie depth perception. They include two main aspects: stereoscopic and distance perception. Stereoscopic perception is derived from retinal disparity between the images projected on both retinas and gives information about the relative distance separating the objects in space, independently of the fixation point or eye position. From a geometric point of view, the actual distance between the objects and the observer (absolute distance) can be calculated by combining the angle of convergence with the retinal disparity images of the object (Blakemore 1969; Collewijn and Erkelens 1990; Ogle 1962b). Psychophysical experiments in humans strongly suggest that distance perception combines retinal disparity signals and egocentric distance signals related to ocular vergence, accommodation, and retinal vertical disparity (for reviews see Bishop 1989; Foley 1980). These egocentric cues are also potentially involved in depth constancy, which refers to the fact that the perceived three-dimensional (3-D) shape of an object appears to remain constant at different viewing distances despite dramatic changes in the amount of disparity. Theoretically, any object should appear to flatten when it is moved away from the observer because the size of a retinal image changes approximately in inverse proportion to viewing distance whereas retinal disparities change in inverse proportion to the square of viewing distance. To achieve this depth constancy phenomenon, the decrease in retinal disparity as a function of the viewing distance has to be internally compensated, probably with the use of egocentric cues like those mentioned above (Bradshaw et al. 1996; Ritter 1977; Wallath and Zuckerman 1963: for reviews see Bishop 1989;
0 1996 The American
Physiological
Society
VIEWING
DISTANCE
EFFECTS
ON
DISPARITY
CODING
2873
between 1 and 4 s. A slight dimming of the target was the signal Foley 1980; Gogel 1977; Ogle 1962a,b; Ono and Comerford for the monkeys to release the lever (within 0.5 s) in order to 1977). From a physiological point of view, the problem of the get a reward (a drop of water). The target fixation was checked relationship between perceived depth and the viewing dis- monocularly by the use of an infrared eye-tracking system (Bouis, Freiburg, Germany; Bach et al. 1983). The monkeys were trained tance has attracted surprisingly little attention, considering to fixate in any sequence order at three distances (20, 40, or 80 the great interest that exists in stereoscopic vision. As a cm) that roughly correspond to the distances of manipulation, preresult, nearly all studies on the neural mechanisms of depth hension, and exploration of surrounding space. At the end of the perception have concentrated on stereopsis. Area Vl is the experiment the monkeys returned to the home cage, where fruit first cortical area where inputs from each eye are integrated. was given and also additional water when necessary. Receptive fields of a binocular neuron may be matched or slightly mismatched in the two eyes. Consequently, a given Visual stimulation stimulus falling in corresponding or slightly different posiStatic RDSs, generated through red (wratten #26) and green tions on the retinas will produce binocular interactions that (wratten #59) filters located in front of the eyes, were flashed for depend on this angular difference. The property of disparity selectivity was first demonstrated in the primary visual cor- 300 ms on a color video monitor at six disparity values (from -0.6 to +0.3” or from -0.3 to +0.6”). Care was taken, by changing the tex of the anesthetized, paralyzed cat (Barlow et al. 1967; position of the filters regularly, not to favor negative or positive Nikara et al. 1968; Pettigrew et al. 1968). Numerous studies disparity values. The RDS pattern was a 128 x 128dot array of have followed in areas 17 and 18 in cat (Bishop et al. 197 1; rectangular dots (typical dot density from 5 to 40% according to DeAngelis et al. 1991; Ferster 198 1; Fischer and Krueger the neuron’s best response), which appeared as a flat surface 1979; Freeman and Ohzawa 1990; Joshua and Bishop 1970; (width 18”, height 14’ of visual angle) floating in front of or behind Lepore et al. 1992; LeVay and Voigt 1988; Maske et al. a fixation point in the center of the screen. The distance of fixation was physically changed by moving the monitor on rails in a tunnel 1984, 1986; Trotter et al. 1993; von der Heydt et al. 1978)) in sheep (Clarke et al. 1976), and in monkey (Hubel and to fixed distances of either 20, 40, or 80 cm. No disparity cue (0 Livingstone 1987; Hubel and Wiesel 1970). Studies per- disparity value) was introduced in the fixation point, so that the monkey had no problem in fusing it binocularly, as indicated by formed in areas Vl and V2 of the behaving monkey showed the constant level of performance at the three distances. The same the existence of three main categories of cells in normal visual pattern was used at each fixation distance, but to keep the conditions of binocular vision : “near,” “far,” and “tuned” angular size of the stimulus constant, the dimensions of the images cells (Poggio and Fischer 1977). The use of random dot were doubled each time the fixation distance was increased by a stereograms (RDSs) confirmed the existence of these cells factor of 2, which resulted in a fixed angular dot size of 9 x 7 and their classification was expanded to “tuned near” (TN) min of arc. The luminance levels through colored filters were equated for 1 cd/m* at the three distances of fixation. and “tuned far” (TF) cells (Gonzalez et al. 1993a,b; Poggio To test the effect of ocular vergence on the neural activity, two et al. 1985, 1988; Poggio 1995). All these studies addressed wedge prisms (base out) were placed in front of the eyes. By neural mechanisms of stereopsis in anesthetized or alert animals at a fixed viewing distance, and therefore with no ma- choosing appropriate power of prisms according to the interpupillary distance, one can reproduce an angle of vergence that normally nipulation of visual convergence. We propose in the present exists at a given distance of fixation according to the formula study to investigate how stereoscopic and viewing distance, D = 100 e (X - y)lx* y, where D is the power of the prism (in i.e., real depth, interact in cortical area VI of the behaving diopters), i is interpupillary distance, e = i/2, x is distance of monkey. Some of the results have appeared earlier in short fixation, and y is distance to the intersection of optical axis with reports (Trotter et al. 1991b, 1992a,b). prisms. For example, for an interpupillary distance of 2.6 cm, two l
METHODS
Animals Two monkeys (i&zcaca mulatta) were prepared for single-cell recordings by surgically implanting a head holder and stainless steel recording chambers (20 mm diam) over cortical area VI while the monkey was under general anesthesia (induction by 0.5 ml ketamine 500, followed by 30 mg/kg pentobarbital sodium). Antibiotics ( local terramycin and thiophenicol, 30 mg kg -’ day -’ im) and postsurgical care were given to the monkeys to ensure a rapid recovery. All experimental protocols were performed according to the Public Health Service policy on use of laboratory animals. A skiascopic examination of the eyes revealed no optical abnormality. l
l
Task During the training, monkeys were seated in a primate chair with the head restrained. The animals had to detect the presence of a small bright target at the center of a video monitor screen and signal its appearance by pulling a lever. The monkeys had to maintain the gaze fixed on the target for random periods of times of
l
prisms of 5 diopters each had to be applied in front of the eyes to reproduce an angle of vergence equivalent to that of 25 cm of distance during fixation at 80 cm. The second monkey was trained to fixate at 80 cm of distance with two prisms of 5 diopters each inducing a convergence toward 25 cm while RDSs were flashed on the video monitor screen.
Recording The stainless steel recording chamber covered the fovea1 projection of the visual field. A small 3- to 4-mm-diam hole was made under Ketamine anesthesia; through this hole a recording electrode could be advanced with the use of a custom-built microdrive (Chubuck, Baltimore, MD). Typically, we were able to make recordings on roughly lo- 15 days before the dura became too thick to allow the recording electrodes to pass. At this point a new aperture was made and the process was repeated. In this way it was possible to record from the same hemisphere for a number of months. Single-unit activity was recorded with the use of varnish-coated tungsten electrodes (Frederick Haer, Brunswick, ME; thin tips, impedance typically 10 Ma) and a commercially available amplification and discrimination system (Bak Electronics, Rockville, MD). The resulting digital pulses were then sent to a PC-AT computer equipped with a laboratory interface board (Scientific
2874
Y. TROTTER, S. CELEBRINI,
B. STRICANNE,
Solutions, Solon, OH) that was used to control the experiments and record all the relevant events (binwidth 1 ms) . At the end of the last recording sessions, electrocoagulations were made at the recording sites to confirm that the recordings lay within area Vl. The animals were killed with barbiturates and perfused with a saline rinse followed by 10% form01 solution. Stainless steel pins were inserted within the recording chamber to delimit the cortical area used for recording. According to the known cortical representation of the fovea in the rhesus monkey (Poggio 1980), the recordings were all from the central 5” of the visual representation. The occipital lobe was processed for histological examination with the use of cresyl violet staining on sections 100 pm in thickness.
S. THORPE, AND M. IMBERT
RESULTS
Disparity sensitivity was tested on 139 cells, of which 78 were analyzed at more than one distance (2 distances, N = 39; 3 distances, N = 39). For the other cells, testing was only completed for one of the three distances (N = 61) .
Disparity selectivity at a given viewing distance
Disparity selectivity was found in similar proportions at 40 cm (56 of 104 cells or 54%) and 80 cm (44 of 79 or 56%). On the other hand, the amount of selectivity dropped significantly at 20 cm ( 15 of 48 cells or 31%) (x2; P < 0.025; df = 2) (Fig. 1A). The repartition of the cell types, Data analysis however, was similar (x2; P > 0.5) at the three viewing At each viewing distance and for all disparity values, the same distances. The near cells were significantly more common RDSs were presented randomly interleaved 9-15 times for 300 ( 1.5 times) than the far and tuned cells at the three distances; ms. Neural activity was displayed on the screen in the form of dot difference of repartition of near, far, and tuned (TE and TI)
displays. The latency of the response was determined by manually positioning a cursor with a precision of 1 ms at the beginning of the visual response. Quantitative analysis was performed starting from that point. “Ongoing activity” (OA) was defined as the activity preceding the visual response in darkness while the monkey was fixating the target for 400 ms with no other visual cue.
Statistical analysis An analysis of variance, [ANOVA, 1 factor (disparity); P < 0.051 was performed on visual responsesduring the 300 ms following the onset latency. This was done for each cell and at each distance of fixation to determine the presence of disparity selectivity. Then, for each cell recorded at two or three distances, an ANOVA [ 2 factors (disparity and distance)] was performed to evaluate the influence of distance and disparity on visual responses. For each cell that exhibited a significant effect of distance and disparity selectivity, a cross-correlation analysis was performed that compared the disparity tunings obtained at different distances. This test involves calculating a regression between the mean values obtained at distance A and distance B for each of the six disparity values (95% confidence interval). A value R close to 1 (P < 0.05) indicates that the shapes of the tuning curves are similar for the two distances. In addition, the effect of the distance on OA of each cell was tested by the use of a one-factor ANOVA (distance, P < 0.05). Results of the statistical analysis were in good agreement with visual inspection of the rasters and of the corresponding tuning curves.
W Near+TN 0
Far+TF
Cell classijication Two major types of disparity-selective (DS) cells were recognized as previously described (Poggio and Fischer 1977; Poggio et al. 1985, 1988). The first category includes cells with a sharp peak response tuned around O”, either tuned excitatory (TE) or tuned inhibitory (TI). The second category contains all the cells that respond selectively to RDSs outside the fixation plane. It includes those that are sharply tuned around 0.15 or 0.3” just in front the fixation plane (TN), or behind (TF), with no excitatory response at 0”. Cells with excitatory responseson a larger scale of disparities with a slope crossing 0” most of the time were classified either as far for positive values or as near for negative values. For quantitative analysis, TN and TF cell types were pooled with near and far types, respectively. These cell types seem to have similar mechanisms that generate their selectivity profiles (Poggio 1995; Poggio et al. 1988). Cells with no disparity selectivity were classified as disparity insensitive (DI).
n 20
40 Distance
80 (cm)
FIG. 1. Disparity selectivity at 3 viewing distances in area Vl. A: proportion of disparity-selective (DS) cells. Notice that there is a lower proportion of disparity cells at 20 cm than at the other distances. B: distribution of the 3 main categories of cells as a function of the viewing distance. At all 3 distances near and tuned near (TN) cells are found in higher proportions than the other categories of cells, far and tuned far (TF), tuned excitatory (TE), and tuned inhibitory (TI).
VIEWING
DISTANCE
EFFECTS
ON
DISPARITY
CODING
cells is shown at 20 cm (x2; P < 0.05), at 40 cm (x2; P < O.OOl), and at 80 cm (x2; P < 0.001) (Fig. 1B). If TN (and TF) cells were pooled with the TE and TI cells rather than with near (and far) cells for quantitative analysis (see METHODS), the near cells would become twice as common as far cells at 20 cm and 3 times more common at 40 or 80 cm. This overrepresentation is thus independent of the way cells were ranged.
2875
N = 43
Latency The response latencies of the DS cells are reported in Fig. 2 for each tested distance. Most neurons responded with between 50 and 80 ms of latency. The latencies could be as short as 34 ms and rarely exceeded 100 ms. The mean was centered around 60 ms, with no significant difference between the three viewing distances or between cell types (ttest; P > 0.4). 20 cm: mean 65.4 t 13 (SD) ms, median value 61 ms 40 cm: mean 62.4 t 14 ms, median value 61 ms 6-
ul z
20cm N=15
75 4tE a E 3
2.
=
'30
, 40
50
60
70
80
90
100
110
10
Emergence
Sharpening
change
selec
Type of modulation FIG. 3. Repartition of the modulation types of neural activity provoked by changing the viewing distance. Emergence means that a neuron is either not visually responsive or disparity insensitive (DI) for at least 1 distance and is DS at the other distance(s). Sharpening indicates that the neuron is DS at all tested distances but has a significantly higher amplitude of visual responsiveness at a particular distance, often associated with changes in some features of the tuning curve. Finally, change of selectivity means that a neuron changes its category of selectivity as a function of viewing distance.
80 cm: mean 61 t 13 ms, median value 58 ms tuned (TE and TI): mean 60.6 t 13.7 ms near and far: mean 62.9 t 13.9 ms
1
The distributions of latencies at the three distances for all categories of DS cells are similar to those recorded for orientation-selective cells in area VI of the monkey (Celebrini et al. 1993; Maunsell and Gibson 1992; Nowak et al. 1995).
40cm N=56
Effects of changing the viewing distance on neural activity ' 30
40
50
60
70
80
90
100
90
100
110
I
" 30
40
50
60
70
80
110
Latency (ms) 2. Frequency histograms of response latencies for DS cells tested at 20, 40, or 80 cm viewing distances. The 3 distributions are centered around 60 ms and are similar at the 3 distances (20 cm: 65.4 t 13 ms, mean + SD, median value 61 ms; 40 cm: 62.4 + 14 ms, median value 61 ms; 80 cm: 61 2 13 ms, median value 58 ms. FIG.
ON VISUAL RESPONSIVENESS. Among cells that were tested at two or three viewing distances, 77% (60 of 78 cells) exhibited significant changes [ANOVA, 1 factor (distance) ; P < 0.051 in visual responsiveness. Both DS cells and DI cells were affected. In our sample, 47 cells of 78 (60%) were DS cells at least at one distance, and 31 cells remained DI cells at all recorded distances. DS cells. The viewing distance changed the level of visual response for 43 of the 47 cells (9 1%) that were DS for at least one distance, with similar effects on all three cell types. In many cases the modulation was such that the neuron failed to show disparity selectivity at all for one or more of the viewing distances. In other cases the neuron was selective at all the tested viewing distances, but the degree of selectivity changed. We thus classified the modulations in two main categories: emergence of disparity selectivity and sharpening of disparity coding. Only rarely did we observe changes in disparity selectivity type (Fig. 3). Emergence of disparity selectivity. The most common efEFFECTS
Ql w
Y. TROTTER, S. CELEBRINI,
2876
B. STRICANNE,
(near cell in Fig. 5A and TE cell in Fig. SS), whereas both cells are fairly silent at 40 cm. For the cells that were tested systematically at all three distances (N = 39), we looked at what distance disparity selectivity did emerge. In other words, did cells become DS at shorter or at longer distances? The proportions of DS cells (28% at 20 cm, 49% at 40 cm, and 51% at 80 cm) are similar to those obtained in the whole population (Fig. 1A). There was a significantly higher proportion of cells that were DS at 40 or at 80 cm than at 20 cm (x2; P < 0.01) (Fig.
feet was the presence of disparity selectivity at certain distance(s) but its absence at the other tested distance(s). Sometimes a DS cell at a given distance could even be visually unresponsive when recorded at another viewing distance. Emergence of disparity selectivity occurred for 28 of the 43 DS cells (or 65%). In Fig. 4 is shown an example of a cell that is clearly a TN cell for a stimulus presentation at 40 and 80 cm, but that exhibits almost no visual response at 20 cm. Two additional examples are shown in Fig. 5, where in both cases disparity selectivity is present at 80 cm
View. dist. : 40 cm
View. dist. : 20 cm
100
100. 60.
.8$60I Y ‘5 ul 40. ‘”
4
6 -
O-0.6
0.3
-0.0
Disparity (deg.)
0.3
0.A
S. THORPE, AND M. IMBERT
a
IM
4 -0.6
-0.3
-0.0
0.3
Disparity (deg.)
View. dist. : 80 cm
#
a ii
-0.6
-0.3
-0.0
0.3
Disparity (deg.)
FTG. 4. Top: example of a dot display of the activity of a neuron in area Vl as a function of the viewing distance. Response of the neuron (between dashed vertical lines) to the flash of a static random dot stereogram (see METHODS) at different disparity values, in front (negative values), behind (positive values), or in the plane of fixation (0 value). Trials were randomly interleaved 12 times at each distance and the sequence of tested distances was 80, 20, and 40 cm. Preceding the visual response is the “ongoing activity” (OA). Bottom: corresponding tuning curves, with OA level on the right. Vertical bars: SEs. It can be seen clearly that this neuron is a TN-type neuron at 40 and 80 cm of distance [analysis of variance (ANOVA); 1 factor (disparity); P = 0.0001 for both] but is silent at 20 cm (P = 0.69). This neuron was thus classified as an emergence-type neuron. Note also that the OA in this particular example is much lower at 20 cm than at the other distances (P < 0.05 )
=
VIEWING
0
DISTANCE
40cm
EFFECTS
ON
DISPARITY
2877
CODING
80
A 80cm
60
40
20
of
.
* . -0.3
’ 0.0
-
Disparity
’ 0.3
-
’ 0.6
8 0:A
0
I
-0.3
(deg.)
W DI cells only at that distance RI DS cells only at that distance
40 cm
80 cm
Viewing distance FIG. 6. Emergence as a function of viewing distance. Histograms represent the distribution of DS cells of the emergence-type (N = 19) among cells recorded at all 3 distances (N = 39). Black bars: cells that are DI types only at the distance indicated underneath and are DS types at the other 2 distances (not indicated in the histogram). Hatched bars: cells that are DS types only at the distance indicated underneath and DI types at the other 2 distances (not indicated in the histogram). These distributions explain the significantly lower percentage of DS cells recorded on the whole Donulation at 20 cm in Fig, 1 A: it also shows that emergence of disnaritv selectivity occurs from short to longer distances rather than the reveise. 1
L
0.0
Disparity
6). Could this difference be due to an inability of the monkey to maintain a steady binocular fixation at 20 cm? If this were the case, one could expect a significant increase in the level of variability of the visual response at the shorter distance. However, this was not observed, because the variability was not significantly different between the three distances (ANOVA, distance P > 0.15). Sharpening. Some cells were DS type at all tested distances with the same peak preference, but differed significantly in amplitude of the visual response. In other cases these modulations were associated with changes in the width
20 cm
.
d
I
0.3
-
FIG. 5. Disparity selectivity tuning curves of 2 other neurons of the emergence-type type demonstrating complete gating. A : cell is near type at 80 cm (A; P = 0.0002) and is visually suppressed at 40 cm ( q J; P = 0.87). To the right of the curves are the corresponding OA levels with matching symbols. Note the decrease of the OA level at 40 cm. B: 2nd example of a cell that is TE at 80 cm (P = 0.0001) and silent at 40 cm (P = 0.15).
1
0.6
0:A
(deg.)
of the tuning curves. One or both of these two effects were seen in 13 of 43 or 30% of cells. An example of the combination of both is presented in Fig. 7A. The cell is a TF cell at all three distances, but the tunin .g is broader at 20 cm than at the other distances. It is at 40 cm that the cell is the most sharply tuned with a higher level of visual response. Another example is shown in Fig. 7B, where the cell exhibits only changes in amplitudes of visual responsiveness. This is a far type with a higher level of response at 80 cm than at 40 cm. Change in disparity selecti vity type. Changes in disparity selectivity type were only observed in 2 cells of 43 ( 0.1. B: cell is a far type with no change in tuning width but a significantly higher level of responsiveness at 80 cm (ANOVA, 2 factors; disparity P = 0.0001; distance P = 0.00 1; interaction term P = 0.58; correlation value R = 0.99).
higher level of visual response at distances other than 20 cm (ANOVA; P < 0.05) (Fig. lo), with the result that there was no correlation between the maximum level of OA and the viewing distance where visual responsiveness was highest. Together these observations imply that the two types of modulations, OA and/or visual responsiveness, cannot both be related to the level of excitability of the neuron. Controls The modulations could originate from different sources, either physiological or artifactual. We tested the most probable of them. Cl
40cm
A
80cm
. 8
SEQUENCE OF THE VIEWING DISTANCE. When recording the activity of a given neuron, we carefully checked that it was the same neuron that was recorded during the course of the tests by storing the shape of the spike with the use of a digital scope. This trace was then used as the reference for all subsequent spike activity for that particular neuron. There was no systematic order in the sequence of the tests, which started at any viewing distance with no assumption of an effect at a particular distance. On several occasions we repeated the first sequence of tests at the end of the course of recordings to be sure that there was no change in the level of responsiveness. Changes in the excitability of the neurons during the course of the tests or between tests, which could last up to 45 min, might possibly occur and produce changes in variability of the response. To test this possibility, analysis of the variability of the visual response [ log( mean/variance)] as a function of the sequence order of the tested distances was performed and revealed no significant differences in variabilities [ ANOVA, 1 factor (order sequence); P > 0.21. TEMPORAL
PATTERN. In addition to the efforts made to maintain the pattern on the retina constant for all viewing distances, controls were performed to ensure that the modulation of disparity selectivity could not be explained by subtle shifts in dot position on the retina and therefore in the receptive fields. If this were the case, changing the dot pattern of the RDS by changing the dot density would result in different kinds of modulations of the visual responsiveness, independently of the viewing distance. For 11 cells, disparity sensitivity was tested at two or more different dot densities for each tested distance. Cells often preferred a certain dot density ranging from 10 to 30%. Figure 11A shows an example of such a cell, which is more sensitive to 20% dot density than to 10%. But in both RETINAL
3020lo-
Disparity (deg.) FIG.8. Example of modulation of the level of responsiveness of a DI cell. This cell is visually unresponsive at 40 cm ( q ), whereas the visual response is clearly present at Sd cm with no sensitivity to disparity values [ANOVA, 2 factors; disparity P = 0.7; distance P = 0.0001; interaction term P = 0.91.
VIEWING
DISTANCE
EFFECTS
20 cm
-0.6
-0.3 Disparity
ON
DISPARITY
2879
CODING
40 cm
0.0
0.3
-0.6
(deg.)
80 cm
-0.6 Disparity
(deg.)
Disparity
(deg.)
FIG _ 9. Example of a neuron that shows modulations of both OA (- -) and of visual responsiveness (-). Both amplitude of the visual response and the disparity selectivity (far type) are higher at longer distances (40 and 80 cm) than at the shorter distance (20 cm) where the cell is IX type (ANOVA, 2 factors; disparity P = 0.05; distance P = 0.09; interaction term P = 0.03 ) . It is also at that short distance that the OA level is the highest; it then decreases monotonically with longer distances to become almost 0 at 80 cm (ANOVA, P < 0.05). This indicates that changes in disparity selectivity can occur independently of changes in the level of excitability of the neuron.
stimulus conditions the visual responsiveness drops drastically when tested at 20 cm, thus independently of the retinal pattern. The tuning curves remain similar at 40 and 80 cm. In all 11 cells tested, changing the dot densities did not change the effect of the viewing distance when present. Changing the dot density also did not change the disparity selectivity category (near/far/tuned). Figure 11 B shows another example of a cell whose disparity selectivity is similar at the three viewing distances, with a moderate effect of the distance in both stimulus conditions. This indicates that cell H
Max 0.A correlated
Cl Max 0.A
a E
2 :E c .-; =to 8 6 8
uncorrelated
to max visual to max
resp.
visual
resp.
80
60
40
20
20 cm
40 cm Distance
inducing
80 cm max
0.A
FIG. 10. Histogram showing the distribution of cells with the highest level of OA as a function of viewing distance. All these cells were tested at the 3 distances and showed a significant effect of distance on both OA and visual response. Most of the cells have a significantly higher level of OA at 20 cm. Hatched area: percentage of cells having the maximum visual responsiveness for a distance other than 20 cm. This shows that modulations of both visual responsiveness and OA are not necessarily correlated and consequently that the modulation of visual responsiveness is independent of the level of excitability of the neuron.
category is essentially independent of the retinal pattern, and that the effects of viewing distance, when present, cannot be explained by that retinal pattern. Similarly, changes in the level of OA with viewing distance cannot be accounted for by any simple retinal differences and thus must have an extraretinal origin very likely related to oculomotor vergence and/or accommodation. VERGENCE ANGLE. Seventeen neurons (6 DS cells and 11 DI cells) were tested with prisms. The influence of the distance was also tested on these cells except in one case. An effect of both the distance and prisms was observed on visual responsiveness for 11 of the 16 cells. These included both DS and DI cells. We used the same statistical analysis as we used for testing the effects of distance. Four cells that did not show an effect of the viewing distance also did not show an effect with prisms. The use of prisms replicated closely (5 cells) or partially (intermediate effect, 3 cells) the effect of changing the distance. In three cases the effect of introducing the prisms was different from that of changing the distance in that it produced a drop in visual responsiveness. One possible reason for the incomplete reproduction of some effects when prisms were used might be the fact that we reproduced a distance of 25 cm instead of 20 cm. Another explanation is a possible role played by accommodation, which has not been tested in the present experiments. Figure 12 shows an example in which the cell is poorly disparity sensitive at 80 cm, but is clearly DS at 20 cm of fixation. However when prisms are added at 80 cm to reproduce an angle of convergence similar to that found close to 20 cm in this monkey, the disparity selectivity becomes similar to that occurring naturally at 20 cm. In this example, the variability of the visual response appears higher with prisms, which is likely attributable to adaptation. This trend was, however, not significant in the population of cells tested with prisms. Indeed, the variability of the response at 80 cm of fixation does not differ significantly from that with prisms or from that at 20 cm (t-test, P > 0.2), nor did the variability differ between responses recorded without prisms at 20 and 80 cm (t-test; P > 0.8).
Y.
Dot density: 140 120
10%
TROTTER,
0 20cm 0 40cm A 80cm
S. CELEBRINI,
B.
STRICANNE,
S. THORPE,
Dot density:
AND
M.
IMBERT
20%
14c
1 1
#
0 1
-0.6
-0.3
0-A
-0.3
0.3
0.0
Disparity
The use of prisms changed significantly (ANOVA; P < 0.05) the OA levels of six cells, reproducing the effect of changing the distance for half of the cells and producing a drop in the OA level for the other half. Although the sample of cells tested with prisms is too small to draw definitive conclusions, the results indicate that changing the vergence angle can reproduce to some extent the effects of changing the viewing distance. DISCUSSION
The neural activity of a large majority of cells in area Vl is modulated by the viewing distance so that disparity selectivity may be present, or better expressed, at a given viewing distance. This phenomenon is independent of the retinal pattern. The OA of about half of the neurons is also modulated, being often significantly higher at the shortest distance. Moreover, the use of prisms can reproduce to some extent the effects of the viewing distance, which suggests that vergence is implicated in this phenomenon. The finding that visual responsiveness is modulated by viewing distance in the primary visual cortex indicates that integration of information from both retinal and extraretinal sources can occur early in the visual processing pathway for cortical representation of 3-D space. Disparity
selectivity at a jixed viewing distance
About half of the cortical cells were DS at 40 cm as well as 80 cm viewing distances. This proportion is higher than
0.6
FIG. 11. Disparity tuning curves for 2 cells that were recorded at 3 distances with random dot stereograms of 2 dot densities, 10% and 20%. A : cell is more sensitive to 20% dot density than to lo%, but in both stimulus conditions the visual responsiveness drops drastically at 20 cm of distance whereas disparity selectivity (TN) remains similar at 40 and 80 cm (ANOVA; disparity, distance; P = 0.0001). The 2nd cell in B is an example in which disparity selectivity is clearly present at the 3 distances at 10% or 20% dot densities but shows a moderate effect of the distance (ANOVA, distance; P < 0.02) (all correlation values between 0.87 and 0.99).
0.A
(deg.)
that reported previously with the use of RDSs at a distance of 100 cm ( -30% in area VI by the use of dynamic RDSs; Poggio et al. 1985, 1988). The higher proportion in our situation may result from methodological reasons like static/ dynamic RDSs and rectangular/dot components of the RDSs, and from the fact that we used large RDSs ( 18 X 14” of visual angle), thus not limited to the receptive field area. It was indeed reported that “best responses to cyclopean figures were obtained with patterns of relatively larger sizes, such as square figures at any orientation and several times the size of the conventionally defined receptive fields of the Vl or V2 neuron” (Poggio et al. 1985). Furthermore, it has been reported under conditions of stabilized vision in the awake monkey that small receptive fields may exhibit powerful inhibitory zones surrounding the classical excitatory receptive field (Gur and Snodderly 1987). This surrounding inhibition of the receptive fields in area VI is not usually taken into account for receptive field size measurements, which are thus underestimated (Dow et al. 198 1). This may partly explain why the size of an effective random dot figure is usually larger than the size of the “binocular response field” mapped with a narrow bar, suggesting that the “cyclopean field” is more widely connected to the retinal input than is the field defined with solid bars (Poggio 1995). Moreover, large RDSs probably activate other neurons in the neighboring network with similar properties, which can potentiate the activity of the neuron under study. The use of large stereograms may therefore reveal more disparity-sensitive neurons because binocular inhibitory in-
VIEWING
DISTANCE
EFFECTS
ON
DISPARITY
CODING
2881
80 cm + 10 diopt.
020cm Aaocm IOO-
IOO-
80-
80-
60-
40-
20-
o(-
-0.3
0.0
0.3
0.6
0.A
Disparity (deg.)
01
-0.3
0.0
0.3
0.6
0:A
Disparity (deg.)
FIG. 12. Effect of wedge prisms (base out) on disparity sensitivity. This cell is clearly selective to negative disparity values (near type) when the monkey fixates at 20 cm ( 0). This disparity selectivity almost disappears at 80 cm, as does the visual response ( A ) . At that distance of fixation, 2 wedge pri sms of 5 diopters each were applied in front of each eye of the monkey. The power of the prisms was chosen by calculation so as to reproduce an angle of vergence simi lar to that existing at -20 cm of normal viewing distance. Recordings were performed -5 min after the application of the prisms. As a result, the visual response (right curve) reappears with a disparity selectivity similar to that seen at 20 cm when normal viewing is used (correlation value R = 0.9). The higher variability of the visual response is likely attributable to adaptation to prisms.
teractions play a crucial role in generating depth sensitivity profiles (Ferster 198 1; Trotter et al. 1993). There is a common trend for all distances to find more near cells than far cells ( 1.5 times). Although the predominance of near cells has not been reported previously in Vl, a preference for near disparities has been reported in area V2 when cyclopean figures are used (von der Heydt et al. 1995)‘) and near cells are almost twice as common as far cells in cortical area MT (Maunsell and Van Essen 1983). These three areas are known to be anatomically interconnected (for review see Felleman and Van Essen 199 1) . Disparity selectivity has been shown to exist in the visual pathway as far as the parietal cortex, cortical area LIP. However, the properties of DS cells differ from area Vl to area LIP, suggesting a possible cortical specialization (for review see Trotter 1995 ) . One of their possible roles would be to participate in the initiation of vergence eye movements, either slow or fast, according to their spatiotemporal characteristics. Indeed, most light-sensitive cells in LIP respond with latencies ranging between 70 and 130 ms (Barash et al. 1991) , and near and far cells that are seen are coarsely tuned, with tuning widths >4” (Gnadt and Mays 1995). These spatiotemporal properties would make these cells ideally suited to elicit vergence eye movements with latencies of - 130- 170 ms in monkeys (Cumming and Judge 1986). In area VI, the latencies of most cells, either tuned or near and far, are much shorter (see RESULTS), with a range of between 40 and 80 ms, and have much finer disparity tuning curves (near and far; tuning width < 1”). Disparity cells with such characteristics could be used to elicit vergence movements of short latencies up to 60 ms obtained with large textured patterns and small disparity steps (Busettini et al. 1996). Disparity selectivity as a jimction of viewing distance VISUAL RESPONSIVENESS. We classified the effects of changing viewing distance under two categories: emergence
and sharpening of the retinal disparity coding. This classification is arbitrary, because we have no information about what would happen outside or inside the range of distances used. However the neural effects refer to variations of the gain of the neural response, a phenomenon that has been described in numerous parts of the brain ( see below). The most striking effect in the present study was the high proportion of cells falling within the emergence classification, because 65% of DS cells are not disparity sensitive or are even visually unresponsive at a certain viewing distance, often at the shortest. The second effect combines in several occasions variations in bandwidth of disparity selectivity and variations of amplitude of responses, but with no change in peak selectivity, resulting in a better expression of disparity coding at a given distance (30% of DS cells). Both effects probably belong to a continuum ranging from a partial to a complete gating of disparity coding by viewing distance. PARTICULARITY OF THE SHORT DISTANCE OF FIXATION. At a distance of 20 cm, only a third of cortical cells were found to be DS. One can wonder why disparity cells should be less represented at a shorter distance. Even though eye recordings were made monocularly, problems related to convergence and/or accommodation impairments at that distance appear unlikely for several reasons. First, skiascopic examinations of the eyes did not reveal any optical abnormality. Second, if there had been a problem of fixation of the visual target (with 0” disparity) at 20 cm, the variability in the level of visual activity (DS cells or DI cells) would have been higher than that recorded at longer distances. However, this was not the case, because there was no significant differences in the level of variability of visual response between the three distances (see RESULTS). Also, DS cells were found in similar proportions in both monkeys at 20 cm (34 and 3 1% of DS cells, respectively) and had the same properties as those recorded at longer distances, as clearlv shown bv the exam-
Y. TROTTER,
S. CELEBRINI,
B. STRICANNE,
ples of tuning curves in Fig. 11 B. Finally, 20 cm of viewing distance for a monkey with an interpupillary distance of 2.6 cm, which roughly corresponds to 46 cm for a human with an interpupillary distance of 6 cm, is not a limited distance for normal binocular fixation or fusion. Shorter viewing distances have been tested in other laboratories, at 10 and 5 cm, with no report whatsoever of impairment of binocular fixation [eye movements tested by electrooculogram (Sakata et al. 1985 ) or coil techniques (Colby et al. 1993 )] . A plausible explanation for a lower level of disparity coding at 20 cm is that the neural computation at short distances could use additional cues than retinal disparity, as has been shown in the case of cortical area VIP (Colby et al. 1993). Retinal and extraretinal facturs implicated modulations
in the
DISPARITY. Theoretically retinal vertical disparity could be used to compute viewing distance (Bishop 1989; G&ding et al. 1995; Mayhew 1982; Mayhew and LonguetHiggins 1982). However, this could not be verified by psychophysical experiments (Cumming et al. 199 1; Sobel and Collett 199 1) . In a subsequent study it was reported that vertical disparity can provide a cue to viewing distance, but only when the field of view is sufficiently large, 70 X 80” (Bradshaw et al. 1996; Rogers and Bradshaw 1993). Our experiments used smaller stimuli ( 18 X 14”) that were projected in the frontoparallel plane, and, according to our calculations, which take into account the fact that we recorded cells well within the central representation of the fovea, the maximum vertical disparity shift produced at 5” of horizontal and vertical retinal eccentricities could be 3.3 min of arc at 20 cm, 1.6 min of arc at 40 cm, and 0.8 min of arc at 80 cm. These values are too small to account for the strong modulations observed at the three viewing distances, although modulations of horizontal disparity by vertical disparity have been shown to occur in area VI. The minimum vertical disparity required for inducing modulations is much larger, ~30 min of visual angle ( see Fig. 4 in Gonzalez et al. 1993b). So it would seem unlikely that the gain modulation we have observed could be accounted for entirely by such small vertical disparities. Oculomotor cues instead could be involved in this phenomenon, as suggested by the effects on the OA. OCULUMOTOR CUES. The OA is the activity of the neuron recorded during the period of the fixation before the presentation of the 3-D visual stimulus. Clearly, modulations of that activity by the viewing distance cannot possibly be produced by changes in the retinal input, because the only visual stimulus present was the fixation point (with no disparity). Consequently, such modulations seen in 53% of the cells are presumably produced by nonretinal factors related to viewing distance cues. Interestingly, among cells that showed such effects, most showed higher OA when the monkey was fixating at a short distance (Fig. 10). A similar effect was observed in a few cells when appropriate powers of prisms were used that increase the convergence angle. These observations are reminiscent of those made in the dorsal lateral geniculate nucleus (LGNd) with the use of prisms in the behaving monkey (Richards 1968): about a third of recorded cells had their spontaneous activity VERTICAL
S. THORPE,
AND
M.
IMBERT
altered by convergence that affected differently the categories of cells. Indeed, activity of “ON ” cells slowed down during convergence and increased during divergence, whereas the ‘ ‘OFF ’ ’ cells showed the reverse effect. Thus one possibility is that the neural effects on the OA occurring at the cortical level reflect extraretinal influences originating at the LGNd level. However, the extraretinal signals may operate at the same time in both LGNd and visual cortex, because disparity selectivity is not present in LGNd but appears in area VI, where modulations of visual responsiveness and of OA can occur independently. At the cortical level, the modulations are also reminiscent of those shown in area 7a, where visual fixation neurons increase their discharge rate when the animal fixates at
