Dissociating oculomotor contributions to spatial and feature-based ...

J Neurophysiol 110: 1525–1534, 2013. First published July 3, 2013; doi:10.1152/jn.00275.2013.

Dissociating oculomotor contributions to spatial and feature-based selection Donatas Jonikaitis1,2 and Jan Theeuwes2 1

Allgemeine und Experimentelle Psychologie, Ludwig-Maximilians Universität München, Munich, Germany; and 2Department of Cognitive Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands Submitted 16 April 2013; accepted in final form 1 July 2013

Jonikaitis D, Theeuwes J. Dissociating oculomotor contributions to spatial and feature-based selection. J Neurophysiol 110: 1525–1534, 2013. First published July 3, 2013; doi:10.1152/jn.00275.2013.—Saccades not only deliver the high-resolution retinal image requisite for visual perception, but processing stages associated with saccade target selection affect visual perception even before the eye movement starts. These presaccadic effects are thought to arise from two visual selection mechanisms: spatial selection that enhances processing of the saccade target location and feature-based selection that enhances processing of the saccade target features. By measuring oculomotor performance and perceptual discrimination, we determined which selection mechanisms are associated with saccade preparation. We observed both feature-based and space-based selection during saccade preparation but found that feature-based selection was neither related to saccade initiation nor was it affected by simultaneously observed redistribution of spatial selection. We conclude that oculomotor selection biases visual selection only in a spatial, feature-unspecific manner. eye movements; feature-based attention; saccades; spatial attention RAPID EYE MOVEMENTS, OR SACCADES,

direct our gaze to the parts of visual scene we wish to inspect in detail. However, there is more to a saccade than meets the eye, as processes associated with saccade preparation alter visual information processing even before the eye movement starts. In particular, visual selection is biased towards the saccade goal, as perceptual discrimination improves and perceived contrast increases at that location (Hoffman and Subramaniam 1995; Kowler et al. 1995; Deubel and Schneider 1996; Deubel 2008; Jonikaitis and Deubel 2011; Rolfs et al. 2011; Rolfs and Carrasco 2012). These effects are a consequence of saccade preparation, as activity in oculomotor brain structures increases the activity in visual areas representing saccade target location (Moore and Armstrong 2003; Armstrong et al. 2006; Ruff et al. 2006; Armstrong and Moore 2007; Noudoost and Moore 2011), which is accompanied by improved visual selection (Moore and Fallah 2001; Carello and Krauzlis 2004; Moore and Fallah 2004; Ruff et al. 2006). Modulation of visual processing is thought to aid saccade target selection (Moore 1999) by enhancing representations of stimuli to which saccades are planned. There are two possible explanations of how saccade target selection modulates processing of the visual field. First, saccade target selection could generate feature-unspecific spatial bias (Fig. 1A, top). This hypothesis is based on the idea that signals from feature maps (e.g., luminance, orientation, color, and motion) and topdown signals are combined into a priority map that represents salient and task relevant regions and is used by the saccadic system (Itti and Koch 2000, 2001). Crucially, the priority map

Address for reprint requests and other correspondence: D. Jonikaitis, Allgemeine und Experimentelle Psychologie, Leopoldstr 13, 80802, Munich, Germany. www.jn.org

represents this information without access to the particular features that made the locations salient or relevant (Fecteau and Munoz 2006). Saccadic target selection relying on a priority map can therefore generate only feature-unspecific spatial biases that lead to spatial attention shifts. Second, saccade target selection could generate featurespecific bias (Fig. 1A, bottom). This hypothesis is based on observations that the oculomotor system represents featurematching signals (Bichot and Schall 1999; Egner et al. 2008; Ipata et al. 2009; Greenberg et al. 2010; Liu et al. 2011; Zhou and Desimone 2011) and that feature-based target selection in oculomotor structures precedes selection in earlier visual areas (Zhou and Desimone 2011). Even though these studies suggest such a link, none of them have directly investigated whether this feature-matching signal is related to oculomotor selection. Furthermore, behavioral studies reporting spatial effects of saccade target selection did not measure feature-based selection (Hoffman and Subramaniam 1995; Kowler et al. 1995; Deubel and Schneider 1996; Montagnini and Castet 2007; Deubel 2008; Jonikaitis and Deubel 2011; Rolfs et al. 2011), and the few studies that measured feature-based selection show conflicting results (Born et al. 2012; White et al. 2013). In the current study, we systematically evaluated the two above alternative explanations. Specifically, two types of selection are expected to modulate perceptual performance in tasks measuring distribution of attentional processing differently. Spatial attention selects a limited number of locations, irrespective of object features (Müller and Rabbitt 1989; Nakayama and Mackeben 1989; Carrasco 2011), and is known to shift to the saccade target even if task requires to attend elsewhere (Kowler et al. 1995; Deubel and Schneider 1996; Deubel 2008). Therefore, one can expect competition between saccade target selection and spatial selection at nonsaccade locations. Feature-based attention (FBA) is assumed to enhance the feature-specific representations throughout the visual field, without an apparent limit on number of locations attended (Martinez-Trujillo and Treue 2004; Melcher et al. 2005; Maunsell and Treue 2006; Wegener et al. 2008; Liu and Mance 2011; White and Carrasco 2011). Therefore, one can measure FBA allocation to nonsaccade locations through the visual field. In two experiments, participants made saccadic eye movements while performing a perceptual discrimination task at different locations across the visual field. We systematically manipulated the role of spatial and feature-based selection. In experiment 1, we measured how space- and feature-guided saccade target selection affected perceptual discrimination at peripheral locations to which saccades were never made. In experiment 2, we quantified the relationship between saccade target selection and perceptual discrimination at the saccade

0022-3077/13 Copyright © 2013 the American Physiological Society

1525

1526

FEATURE ATTENTION BEFORE SACCADES

A

Saccadic map

Feature selective maps Attention

1. Featuretarget selection

Spatial attention shift irrespective of target feature

2. Matchingfeature target selection

FBA shift through the

B Experiment 1: Spatial cue

Experiment 1: Color cue

Experiment 1: Combined cue

Non-match

Match Probe stream

C

Experiment 2: Spatial cue

...

Experiment 2: Color cue

Distractor stream

...

C 25 ms

Time

Fig. 1. A: theoretical predictions. 1) Feature-unspecific target selection. Selecting a saccade target biases visual selection irrespective of the saccade target features, resulting here in a spatial attention shift only toward the saccade target location. 2) Matching feature target selection. Selecting a saccade target biases visual selection as a function of the saccade target feature. Here, objects matching the saccade target feature get enhanced visual processing, leading to feature-based attention allocation throughout the visual field. B: display configuration in experiment 1. In the Spatial-cue task, participants were instructed to shift their gaze from the central fixation to the saccade target as indicated by the central cue. In the Color-cue task, saccade target color was selected and instructed at the beginning of each block. In the Combined-cue task, saccade target was 1 of 4 color-shape combinations, selected and instructed block-wise. Green and blue square frames mark locations of the stream of vertical Gabor patch and masks alternating every 25 ms. At different time relative to the onset of the saccade cue (from ⫺150 – 450 ms), attention was probed with the presentation of tilted Gabor for 25 ms. Stimuli are not to scale. C: display configuration in experiment 2. Here probe discrimination performance was measured with probes presented at the potential saccade locations.

target location by examining differences between spatially and feature-guided saccade target selection.

Germany. Experiments were designed according to the ethical requirements specified by corresponding institution even though no institutional review board (or corresponding institution) ethics approval was needed for experiments that do not involve invasive methods.

METHODS

Participants

Setup

In experiment 1, 12 participants completed the Spatial-cue task, 16 completed the Color-cue task, and 9 completed the Combined-cue task. In experiment 2, 10 participants completed the Color-cue task, and 13 completed the Spatial-cue task. Participants’ age was between 21 and 30 yr, vision was normal or corrected to normal, and except for the author (D. Jonikaitis) all were naive as to the purpose of the studies. The experiments were all conducted in accordance with the Declaration of Helsinki. Experiments were carried out in Allgemeine und Experimentelle Psychologie, Ludwig-Maximilians Universität München, Munich,

Eye position was recorded with an SR Research EyeLink 1000 tower-mounted eye-tracker (sampling rate of 1,000 Hz), calibrated before each session and whenever necessary. Experiments were controlled by an Apple desktop computer and implemented in Matlab (MathWorks, Natick, MA) using Psychophysics and Eyelink toolboxes (Watson and Pelli 1983; Brainard 1997; Cornelissen et al. 2002). Stimuli were presented at a viewing distance of 70 cm on gamma-linearized 20-in. Sony GDM-F500R screen (1,024 ⫻ 768 pixels, vertical refresh rate of 120 Hz).

J Neurophysiol • doi:10.1152/jn.00275.2013 • www.jn.org

FEATURE ATTENTION BEFORE SACCADES

Procedure In every task (with exception in the Combined-cue task), participants were asked to make a saccade while we measured perceptual discrimination performance for stimuli presented at different locations in space. We used perceptual discrimination task as a measure of attention allocation to different locations and stimuli (Deubel and Schneider 1996; Rolfs et al. 2011). In experiment 1, we measured attention allocation to peripheral locations while participants made leftward or rightward saccades. The direction of the saccades was cued using a spatial cue, a color cue, or by combining the color and the shape of the cue (see Fig. 1B). In experiment 2, we measured attention allocation to the saccade target while participants made leftward or rightward saccades. The direction of the saccades was cued using a color cue or a spatial cue. Experiment 1: Spatial-cue task. In this task, we measured whether perceptual discrimination performance at two peripheral locations was affected by whether those locations match or did not match the saccade target color. Figure 1B shows stimulus configuration. Display is composed of a fixation target (black circle: 2.5 cd/m2; diameter: 0.75°) and two potential saccade targets (one green: 29 cd/m2, CIE Yxy 29.0, 0.30, 0.56; one blue circle: 28.5 cd/m2, CIE Yxy 28.4, 0.18, 0.13; presented at 7° to the left or the right of the fixation, diameter 0.75°) and two discrimination-task-related objects (presented 7° above and below the fixation, diameter 2°), all shown on a gray background (73 cd/m2, CIE Yxy 75.7, 30.2, 31.8). Discriminationtask-related objects consisted of green and blue square frames containing vertical Gabor patches (frequency: 2.5 cycles-per-degree, 100% contrast, random phase on each presentation, SD of Gaussian window mask covering the Gabor: 0.8°) alternating every 25 ms with noise masks (each pixel gray value from Gaussian distribution with mean of 0.5, where 0 is black and 1 is white). Each trial started with participants fixating on a fixation target (display duration from Gaussian distribution: 1,000 ms, SD of 300 ms). After the fixation period, a line (length: 0.4°) pointing to the left or to the right indicated the direction of the saccade (display duration: 600 ms, trial repeated at the end of the block if correct saccade was not detected). Between 150 ms before and 450 ms following the saccadecue onset (time selected randomly), a perceptual probe, a tilted Gabor patch, was presented in the stream. No more vertical Gabor patches were presented after the probe, leaving noise masks alternating with blanks at both locations. At the end of each trial, participants reported the probe orientation (clockwise or counterclockwise from vertical). Participants completed a minimum of three 1-h sessions of the task. If saccade planning results in an enhancement of processing of features matching saccade target object, we should observe perceptual discrimination improvement at the saccade target color-matching locations. Importantly, we measured FBA at two peripheral locations (upward or downward), to which no saccades had to be directed, in contrast to two studies that measured FBA at locations that were potential saccade targets, thus confounding discrimination benefits arising because of saccade target selection with potential FBA benefits (Gersch et al. 2009; Born et al. 2012). Experiment 1: Color-cue task. In this task, we measured whether perceptual discrimination performance at two peripheral locations is affected by the locations that match or did not match the saccade target color. Participants made a saccade to a color-matching target (color randomly selected and instructed block-wise, color targets appeared after fixation period). Participants completed a minimum of four 1-h sessions of the task. Experiment 1: Combined-cue task. In this task, we measured contributions of saccade planning to spatial attention and FBA effects by comparing discrimination performance in saccade and fixation conditions. Saccade target was defined by a combination of color and shape: green or blue, and square or circle (4 possible combinations, target selected randomly and instructed block-wise). Saccades were made if color and shape combination was present (50% of trials,

1527

saccade condition), fixation was kept if the combination was absent (50%, fixation condition). Participants completed a minimum of six 1-h sessions of the task. Experiment 1: threshold procedure. The threshold procedures was completed before each session and task. Stimulus configuration was similar as in above tasks, except for the fixation position (at the center of the display, 7° left or 7° right of the fixation). One-hundred percent valid color cue presented after fixation period (frame around Gabor patch and mask stream changed color) indicated location of the probe (presented 100 –200 ms after the cue onset). We used QUEST staircases (Watson and Pelli 1983) to adjust the Gabor patch tilt angle needed to reach 82% correct discrimination performance at the cued locations. Two interleaved staircases were run: one for the fixation at the display center (simulating presaccadic display, with probe eccentricity of 7°), and one for the fixation to the left or right of the display center (simulating postsaccadic display, with probe eccentricity of 9.9°). Those threshold values were used during the main task by presenting the appropriate probe tilt depending on whether the eye was at fixation or at the saccade target. Experiment 2: Spatial-cue task and Color-cue tasks. Discrimination performance was measured at the saccade target location instead of peripheral locations. In both tasks, alternating Gabor patch/mask streams were presented at the saccade target location. Participants completed a minimum of four 1-h sessions of the task. Experiment 2: threshold procedure. Same as in experiment 1, except that only central fixation was presented (thus stimulating only presaccadic stimulus eccentricity for experiment 2). Data Analysis We detected saccades offline using an algorithm based on eye velocity changes (Engbert and Kliegl 2003). Saccades were accepted if end-point and start-point error was ⬍3°, and saccade latency was ⬎100 ms and ⬍500 ms. In experiment 1, we accepted 29,510 trials in the Color-cue task, 13,776 in the Spatial-cue task and 25,165 in the Combined-cue task. In experiment 2 we accepted 19,341 trials in the Color-cue task and 28,748 in the Spatial-cue task. Probe discrimination performance was calculated for probes shown (using probe offset time) within specified time-bins and is expressed as the percentage of correct orientation discriminations for each time bin (chance level is 50%). Probes presented during the saccade were excluded. Statistical comparisons included repeated-measures ANOVAs and paired t-tests. RESULTS

Experiment 1 In experiment 1 (Fig. 1B), we investigated whether saccade preparation affected spatial and feature-based selection throughout the visual field. In two versions of the experiment, we measured discrimination performance at two peripheral locations while saccades were directed either to targets selected on the basis of their locations (Spatial-cue task) or of their colors (Color-cue task). Discrimination performance served as a measure of spatial attention allocation at those locations (Müller and Rabbitt 1989; Nakayama and Mackeben 1989; Carrasco 2011). Discrimination performance at the saccade target color-matching location vs. the nonmatching location was used as a measure of feature-based attention (Melcher et al. 2005; Wegener et al. 2008; Liu and Mance 2011; White and Carrasco 2011). Figure 2A shows temporal dynamics of discrimination performance in the Spatial-cue task. In this task we investigated whether spatially guided saccade preparation coactivates selec-

J Neurophysiol • doi:10.1152/jn.00275.2013 • www.jn.org

1528

A

FEATURE ATTENTION BEFORE SACCADES

B

C

D

Fig. 2. Experiment 1. A: probe discrimination performance in the Spatial-cue task. Legend depicts color-matching (red) and nonmatching (black) probe locations; saccade is depicted as a large arrow. Error bars represent SE. Lines below the graph represent cumulative saccade onset probability. B: feature-based-attention effect (i.e., discrimination performance for the color-matching minus color-nonmatching location) for probes presented [⫺150 – 0] or [200 –350] ms in the Spatial-cue task. Statistical comparisons (repeated measures t-test): *significant difference; ns, absence of difference. C and D: Color-cue task, same convention as A and B.

tion of the saccade target-matching features throughout the visual field. We observed that discrimination performance improved at both probe locations over time. However, spatially guided saccade target selection did not evoke feature-specific selection across the visual field, because discrimination performance was not differentially influenced depending on whether or not the probe location was the same or different color as the saccade target. Repeated-measures ANOVA showed main effect of probe presentation time [F(5,55) ⫽ 5.46, P ⬍ 0.001], no effect of probe-saccade target color match [F(1,11) ⫽ 0.69, P ⫽ 0.42], and no interaction between two factors [F(5,55) ⫽ 1.44, P ⫽ 0.22]. Figure 2B shows the FBA effect for probes presented before the saccade cue (⫺150 – 0 ms) and probes presented at the time FBA effects are known to occur (200 – 350 ms) (Stoppel et al. 2012). We observed comparable FBA effects for probes presented at both time intervals [t(11) ⫽ 0.89, P ⫽ 0.38]. This further indicates that spatially guided saccade preparation did not evoke feature-specific selection in the visual field. Thus our results confirm a previous study (White et al. 2013), but contradict another (Born et al. 2012). The color-matching effects reported by Born and colleagues in their Spatial-cue task (Born et al. 2012) are likely due to their nonbalanced visual displays (more objects matched saccade target color), which results in bottom-up driven priming effects (Theeuwes and Van der Burg 2007). Figure 2C shows temporal dynamics of discrimination performance in the Color-cue task. Here, we investigated whether feature-guided saccades lead to FBA effects throughout the visual field. Discrimination performance at the color-matching peripheral location was better than at nonmatching location and this difference increased over time. Repeatedmeasures ANOVA showed main effect of probe presentation time [F(5,75) ⫽ 19.31, P ⬍ 0.001], main effect of probesaccade target color match [F(1,15) ⫽ 26.34, P ⬍ 0.001], and interaction between two factors [F(5,75) ⫽ 4.66, P ⬍ 0.001]. Figure 2D shows that the FBA effect doubled from ⬃6% for probes presented before the saccade cue to ⬃13% after the saccade cue [t(16) ⫽ 3.12, P ⬍ 0.01]. However, the fact that we observed that feature-guided saccade preparation is associ-

ated with an increase in feature-based selection does not necessary show that saccade target selection and feature-based selection are generated from the same underlying machinery. Two different mechanisms could account for these results. First, saccade target selection could lead to FBA allocation to all target-matching locations. Second, FBA effect might arise through a mechanism unrelated to saccade target selection. We addressed this question in the Combined-cue task. In this task, participants made a saccade if target matched instructed shape-color combination and fixated if target matched only color but not the shape (Fig. 1B). This paradigm leads to two specific predictions. First, since spatial selection is biased towards saccade target during saccade preparation (Kowler et al. 1995; Deubel and Schneider 1996; Castet et al. 2006; Montagnini and Castet 2007; Deubel 2008), we should observe differences in spatial attention allocation at peripheral locations during saccade condition compared with the fixation condition. Second, if saccade preparation affects feature-specific selection, we should also observe FBA effect size differences during saccade condition compared with fixation condition. Figure 3A shows the temporal dynamics of probe discrimination performance in the Combined-cue task. Two findings of note can be seen in this figure. First, discrimination performance at peripheral locations (regardless of target color) increased over time, and differed between fixation and saccade trials. Repeatedmeasures ANOVA showed main effect of probe presentation time [F(6,48) ⫽ 52.20, P ⬍ 0.001], main effect of trial type [F(1,8) ⫽ 35.59, P ⬍ 0.001], and interaction between the two factors was approaching significance [F(6,48) ⫽ 1.99, P ⫽ 0.08]. This finding shows that allocation of spatial attention differed between saccade and fixation trials, which we discuss in more detail below. Second, discrimination performance at the color-matching peripheral location was better than at nonmatching location and this difference increased over time for both fixation and saccade trials. Repeatedmeasures ANOVA over FBA effect showed main effect of probe presentation time [F(6,48) ⫽ 2.47, P ⬍ 0.05], no effect of trial type [F(1,8) ⫽ 1.06, P ⫽ 0.33], and no interaction between the two factors [F(6,48) ⫽ 0.93, P ⫽ 0.48]. This finding shows that FBA was evoked both during fixation and saccade trials to a

J Neurophysiol • doi:10.1152/jn.00275.2013 • www.jn.org

FEATURE ATTENTION BEFORE SACCADES

A

B

1529

C

Saccade target

Saccade

Fixation

Fig. 3. Experiment 1. A: probe discrimination performance in the Combined-cue task. Legend depicts color-matching and nonmatching probe locations; saccade is depicted as large arrow (color and shape combination is present; saccade condition) or if no saccade is made (color and shape combination is absent; fixation condition). Lines below the graph represent cumulative saccade onset probability. Error bars are SE. B: discrimination performance for probes presented [⫺150 – 0] or [200 –350] ms for saccade and fixation conditions irrespective of color match/nonmatch. Statistical comparisons (repeated measures t-test): *significant difference; ns, absence of difference. C: Feature-based attention (FBA) effect for probes presented [⫺150 – 0] or [200 –350] ms for saccade and fixation conditions.

similar magnitude. To further understand the effects of spatial and feature-based selection, we considered them separately for probes presented before saccade cue and after the saccade cue. Figure 3B shows the spatial attention effects for saccade and fixation conditions. Before the saccade cue appeared (⫺150 – 0 ms), discrimination performance at both peripheral locations was comparable for saccade and fixation trials [t(8) ⫽ 0.45, P ⫽ 0.65]. During the saccade target selection interval (200 – 350 ms, only trials with presaccadic probe presentation) discrimination performance was worse during saccade than fixation trials [t(8) ⫽ 3.64, P ⬍ 0.01]. This analysis shows that saccade preparation redistributed spatial selection through the visual field, as spatial attention was withdrawn from nontarget locations during saccade preparation (Kowler et al. 1995; Deubel and Schneider 1996; Deubel 2008). Figure 3C shows FBA effect for saccade and fixation conditions. FBA was comparable between fixation and saccade trials for both precue [⫺150 – 0 ms; t(8) ⫽ 0.67, P ⫽ 0.52] and postcue periods [200 –350 ms; t(8) ⫽ 0.87, P ⫽ 0.40]. This analysis shows that

A

saccade preparation did not affect allocation of feature-based selection through the visual field. Experiment 2 While experiment 1 found no effects of saccade preparation on FBA throughout the visual field, in experiment 2 we evaluated the relationship between saccade target selection and spatial and feature-based selection at the saccade target location (Fig. 1C). Figure 4 shows probe discrimination performance at saccade target and nontarget locations. Figure 4A shows temporal dynamics of probe discrimination performance in the Spatial-cue task. Discrimination performance increased at both probe locations over time, but much more so at the saccade target location compared with nontarget location. Repeated-measures ANOVA showed main effect of probe presentation time [F(5,60) ⫽ 35.94, P ⬍ 0.001], main effect of probe location [F(1,12) ⫽ 28.56, P ⬍ 0.001], and interaction between the two factors [F(5,60) ⫽ 20.43, P ⬍ 0.001]. These

B

Fig. 4. Experiment 2. A: probe discrimination performance in the Spatial-cue task. Legend depicts saccade and nonsaccade probe locations, and saccade is depicted as large arrow. Error bars are SE. Lines below the graph represent cumulative saccade onset probability. Filled symbols depict significant t-test comparisons for each time point against first time point. B: Color-cue task.

J Neurophysiol • doi:10.1152/jn.00275.2013 • www.jn.org

1530

FEATURE ATTENTION BEFORE SACCADES

results therefore suggest that saccade target selection resulted in clear spatial attention shift toward that location. Figure 4B shows similar results for the Color-cue task. Repeated-measures ANOVA showed main effect of probe presentation time [F(5,40) ⫽ 23.94, P ⬍ 0.001], main effect of probe location [F(1,8) ⫽ 103.04, P ⬍ 0.001], and interaction between the two factors [F(5,40) ⫽ 19.20, P ⬍ 0.001]. These results show that similarly as in the Spatial-cue task attention is shifted toward the saccade target. To determine the relationship between saccade preparation and spatial and feature-based selection, we evaluated the temporal relationship between saccade initiation and visual selection. We previously reported that spatial selection is coupled to saccade planning: the shorter the saccade latency, the earlier the attention shifts to the saccade goal (Jonikaitis and Deubel 2011). Figure 5A shows discrimination performance at the saccade target location for short and long (below and above

median latency for each participant) saccade latencies in the Spatial-cue task. Discrimination performance differed for short latency saccades compared with long latency saccades. Figure 5B shows that discrimination performance for probes presented before the saccade cue (⫺150 – 0 ms) is comparable for trials with short and long latency saccades [t(12) ⫽ 1.01, P ⫽ 0.33]. On the other hand, discrimination performance for probes presented during the saccade preparation (50 –200 ms after the cue) was better for short compared with long latency saccade trials [t(12) ⫽ 3.45, P ⬍ 0.01]. Further comparisons of individual participant data (Fig. 5C, circles) suggest that this true for majority of participants, with most of the data points above the unity line when comparing short vs. long saccade latency trial performance for probes presented 50 –200 ms after the saccade cue. Better discrimination performance for probes presented after the saccade cue for short latency saccade trials can be ex-

A

B

C

D

E

F

G

H

Fig. 5. Relationship between saccade latencies and time course of attentional allocation in the experiment 2. A: probe discrimination performance at the saccade target location in the Spatial-cue task for short-long latency saccades, as defined by dividing latencies into below (short) and above (long) median latency for each subject. Lines below the graph represent cumulative saccade onset probability. Color scheme represents short-long saccade latencies and is same for discrimination performance and cumulative saccade latencies. Error bars are SE. B: discrimination performance for probes presented [⫺150 – 0] or [50 –200] ms at the saccade target location. Statistical comparisons (repeated measures t-test): *significant difference; ns, absence of difference. C: probe discrimination performance at the saccade target location for short-long latency saccades for probes presented 50 –200 ms after the saccade-cue onset. Each data point represents individual participant. Data points above unity line indicate earlier improvement in discrimination performance for short latency saccades: experiment 2, Spatial-cue task shown as red dots; Rolfs et al. (2011), experiment 4 shown as rightward triangles; Rolfs et al. (2011), experiment 1 shown as upward triangles; Jonikaitis and Deubel (2011), experiment 2 shown as black squares. D: probe discrimination performance at the saccade target location in the Spatial-cue task for short-long latency saccades, represented relative to the saccade onset time. Lines below the graph represent cumulative saccade-cue onset probability, with cue appearing short time before saccade onset for short latency saccades and long time before saccade onset for long latency saccades. E, F, and H: Color-cue task. F and G: experiment 2, Color-cue task shown as red dots; experiment 1, Color-cue task shown as squares; experiment 1, Combined-cue task shown as triangles. J Neurophysiol • doi:10.1152/jn.00275.2013 • www.jn.org

FEATURE ATTENTION BEFORE SACCADES

plained either by an improvement in attentional selection in short saccade latency trials (representing an upward shift in performance function) or by earlier saccade target selection in short saccade latency trials (representing a leftward shift in performance function). We addressed this question by reanalyzing the data with respect to the saccade onset. If our results can be explained by earlier saccade target selection in short latency trials, then locking discrimination performance relative to saccade onset should remove differences between short and long latency trials. Figure 5D shows that discrimination performance for short and long latency saccades does not differ as a function of saccade-cue onset time and only differs as a function of saccade onset time (Fig. 5A). Repeated-measures ANOVA showed main effect of probe presentation time [F(3,36) ⫽ 54.29, P ⬍ 0.001], no effect of saccade-cue onset time [F(1,12) ⫽ 0.002, P ⫽ 0.96], and no interaction between the two factors [F(3,36) ⫽ 0.87, P ⫽ 0.46]. In other words, spatial selection varied as a function of the saccade latency, indicating close coupling between saccade initiation and attentional selection. Temporal coupling between spatial selection and saccade initiation is also supported by reanalysis of previously published data from our lab. Figure 5C, squares and triangles, presents data from three additional experiments utilizing spatial cues (Jonikaitis and Deubel 2011; Rolfs et al. 2011). In all tasks, we observed similar results: spatial attention shifted to the saccade target earlier if saccade latencies were short (all P ⬍ 0.05) and discrimination performance for short and long latency saccades was comparable before saccade onset (all P ⬎ 0.05). Combined these results show an association between spatial selection at the saccade target location and saccade latencies. Notably, we observed different pattern of results in the Color-cue task (Fig. 5, E–H). Figure 5E shows discrimination performance at the saccade target location for short and long saccade latencies in the Color-cue task. Discrimination performance at the saccade target location was comparable for both latency saccades. Figure 5F shows that discrimination performance cannot be distinguished for trials with short and long latency saccades neither for probes presented before the saccade cue [⫺150 – 0 ms before the cue, t(9) ⫽ 0.89, P ⫽ 0.39] nor after the saccade cue [⫺150 – 0 ms before the cue, t(9) ⫽ 0.05, P ⫽ 0.95]. Further comparisons of individual participant data confirm this result (Fig. 5G, circles), as individual participants data are scattered evenly above and below the unity line when comparing short vs. long latency trials performance for probes presented 50 –200 ms after the saccade cue. Last, Fig. 5H shows that discrimination performance varied as a function of saccade-cue presentation time and suggests that discrimination performance at the saccade target location did not covary with saccade latencies. This finding further corroborates the independence between saccade target and feature-based selection. Finally, we reasoned that if the saccades start faster in the color-guided saccade task in experiment 1 (i.e., saccadic latency is short), then FBA might shift earlier to peripheral color-matching locations. However, we neither observed any such effect in the Color-cue task (Fig. 5G, squares) nor in the Combined-cue task (Fig. 5G, triangles). Therefore, saccade target selection was not associated with temporal dynamics of FBA in any of three tasks participants completed.

1531

DISCUSSION

In the current study, we evaluated whether saccade target selection biases visual selection in a spatial, feature-unspecific, or a feature-specific manner. We found strong evidence for the first and none for the second alternative. In experiment 1, we observed that spatial (Spatial-cue task) and feature-based attention (Colorcue task) modulated discrimination performance at locations to which saccades were never made. However, when we compared a saccade condition to a fixation condition (Combined-cue task), we observed that saccade preparation affected distribution of spatial processing irrespective of the relevance of the feature. This suggests that saccade target selection biases the perceptual system in a feature-unspecific manner. In experiment 2, we observed that both spatially (Spatial-cue task) and feature (Color-cue task)guided saccades result in similar discrimination improvement at the saccade target location. However, latencies of the saccades were related to the discrimination performance time course only in the Spatial-cue task (also the case for three other data sets from previously published work; Jonikaitis and Deubel 2011; Rolfs et al. 2011), but not in the Color-cue task (also true for 2 other data sets from experiment 1). This last point indicates separation between saccade target selection and feature-based selection. We observed feature-based selection in the visual field during feature-guided saccade tasks. Feature-based selection during saccade planning has been observed in a few behavioral studies (Gersch et al. 2009; Born et al. 2012); however, those studies did not measure whether saccade preparation evokes FBA shifts. Further, neurophysiological studies have observed feature-based selection during saccade preparation (Bichot et al. 2005; Zhou and Desimone 2011) and have suggested that it could originate in areas assumed to represent saccade target selection such as the frontal-eye-fields (FEF) or the lateral intraparietal sulcus (Egner et al. 2008; Greenberg et al. 2010; Liu et al. 2011; Zhou and Desimone 2011). Our findings clarify those results, in showing that even though feature-based selectivity is apparent during saccade planning, saccade planning is unlikely to directly affect feature-specific selectivity. We based this conclusion on the following observations. First, in the Spatial-cue task of experiment 1, saccade target selection did not lead to concomitant feature-specific activation, ruling out automatic coupling between these two processes. Second, in the Combined-cue task of experiment 1, saccade target selection biased spatial selection irrespective of object features, thus showing different effects of saccade planning on spatial and feature-based selection. Third, in the Color-cue tasks of both experiments 1 and 2, shorter saccade latencies were not associated with earlier selectivity at the feature-matching locations, thereby ruling out the possibility that saccade target selection could directly bias featurespecific activation. Combined these findings suggest that even though feature-specific selection is clearly observed during saccade preparation, saccade target selection alters spatial, but not feature-specific, processing. Our results are consistent with the notion that there are two selection mechanisms: a spatial one that is closely related to saccade target selection and a feature-based one that is not driven by the oculomotor selection. Spatial selection mechanism could be described as a priority map, which represents top-down attentional selection and bottom-up salience at each location in space (Itti and Koch 2000; Thompson and Bichot 2005; Fecteau and Munoz 2006; Bisley and Goldberg 2010),

J Neurophysiol • doi:10.1152/jn.00275.2013 • www.jn.org

1532

FEATURE ATTENTION BEFORE SACCADES

calculated as a summed spatial signal across all feature space, without feedback to feature-specific signals. In this view, saccade target selection can bias only spatial selection, irrespective of feature specific values. Such priority maps have been demonstrated in psychophysical studies (Donk and van Zoest 2008; Zehetleitner et al. 2011), have been formalized in computational models (Itti and Koch 2000), and have been suggested to be located in saccade planning areas such as FEF, lateral intraparietal sulcus, or superior coliculus (SC) (Thompson and Bichot 2005; Fecteau and Munoz 2006; Bisley and Goldberg 2010). In parallel to spatial selection, feature-based selection could assign higher weights to task-relevant features, resulting in enhanced representations at all feature-matching locations. This selection mechanism could still be used to guide saccades in a spatial manner, as only feature-matching locations would provide higher weighted signal and nonmatching locations would be tagged with lower weights. If this interpretation is correct, then the origin of the feature-based visual biases remain unknown, even though few studies have observed independent spatial and feature-based attentional modulations using behavioral and neurophysiological paradigms (Treue and Martinez-Trujillo 1999; Hayden and Gallant 2005, 2009; Cohen and Maunsell 2011). Neurophysiological findings are inconclusive concerning the contributions of oculomotor structures to feature-based selection. Those studies that suggest such a contribution have not demonstrated direct causality, as has been demonstrated for spatial attention using transcranial magnetic stimulation, inactivation, or microstimulation (Moore and Fallah 2001; Ruff et al. 2006; van Ettinger-Veenstra et al. 2009; Lovejoy and Krauzlis 2010; Zenon and Krauzlis 2012). In addition, the relationship between feature-based selection and saccade target selection within FEF, intraparietal sulcus, or SC remains unclear, as few studies distinguish visual feature selection from saccade target selection (Gottlieb et al. 1998; Sato et al. 2001; Shadlen and Newsome 2001; Sato and Schall 2003; Juan et al. 2004). Indeed, it has been shown that even in experimental situations when object features/identity are task relevant, FEF neurons seem to robustly represent spatial selection signals (Monosov et al. 2010, 2011; Clark et al. 2012). However, even though oculomotor structures have been demonstrated to modulate spatial selection, the relationship between spatial selection and saccade preparation is less direct than suggested by models proposing direct equivalence between two (Rizzolatti et al. 1987; Sheliga et al. 1997). Separate neural populations within FEF mediate visual target selection and saccade preparation (Sato and Schall 2003; Thompson et al. 2005; Schafer and Moore 2011; Gregoriou et al. 2012), effectively dissociating spatial attention and saccade preparation. Further, as distinct neural populations within FEF are interconnected with visual cortex and with the SC, areas that are considered to be recipients of visual selection and saccade preparation signals, respectively (Pouget et al. 2009; Gregoriou et al. 2012), it is visual selection cells that likely mediate spatial selection via connections to visual processing areas (Gregoriou et al. 2012). However, as there is a close behavioral association between spatial selection and saccade preparation (Kowler et al. 1995; Deubel and Schneider 1996; Baldauf et al. 2006; Jonikaitis and Deubel 2011), oculomotor structures might be best regarded as a functional network underlying spatial selection and saccade preparation (Awh et al. 2006),

even though dissociable visual and saccade preparation signals can be observed within that network. How feature-based selection interacts with this network underlying spatial selection and saccade preparation still remains to be defined, even though our findings suggest stronger dissociation between feature-based selection and saccade preparation than between spatial selection and saccade preparation. Dissociating two separate mechanisms for spatial saccade target selection and feature-specific biases might offer behavioral advantages. In foraging situations, which require the selection of as many task-relevant objects as possible, information about task-relevant features with every intervening eye movement could simply be lost due to spatial bias induced by the saccade planning. This is especially evident in situations where perceptual discrimination performance at nonsaccade locations decreases during saccade preparation and is associated with impoverished moving stimulus tracking (Jonikaitis et al. 2009) or memory (Bays and Husain 2008) at attended locations after the saccade. Limited interactions between saccade target selection and feature-based selection, on the other hand, could leave unimpaired selectivity for task relevant features, even in the case of intervening eye movements. In this view, dissociation between saccade planning and feature-based selection offers an adaptive advantage and supports the ability to react to constantly changing visual world despite intervening eye movements. GRANTS This research was supported by Deutsche Forschungsgemeinschaft Grant JO 980/1–1 (to D. Jonikaitis). DISCLOSURES No conflicts of interest, financial or otherwise, are declared by the author(s). AUTHOR CONTRIBUTIONS Author contributions: D.J. and J.T. conception and design of research; D.J. performed experiments; D.J. analyzed data; D.J. and J.T. interpreted results of experiments; D.J. prepared figures; D.J. and J.T. drafted manuscript; D.J. and J.T. edited and revised manuscript; D.J. and J.T. approved final version of manuscript. REFERENCES Armstrong KM, Fitzgerald JK, Moore T. Changes in visual receptive fields with microstimulation of frontal cortex. Neuron 50: 791–798, 2006. Armstrong KM, Moore T. Rapid enhancement of visual cortical response discriminability by microstimulation of the frontal eye field. Proc Natl Acad Sci USA 104: 9499 –9504, 2007. Awh E, Armstrong KM, Moore T. Visual and oculomotor selection: links, causes and implications for spatial attention. Trends Cogn Sci 10: 124 –130, 2006. Baldauf D, Wolf M, Deubel H. Deployment of visual attention before sequences of goal-directed hand movements. Vision Res 46: 4355– 4374, 2006. Bays PM, Husain M. Dynamic shifts of limited working memory resources in human vision. Science 321: 851– 854, 2008. Bichot NP, Rossi AF, Desimone R. Parallel and serial neural mechanisms for visual search in macaque area V4. Science 308: 529 –534, 2005. Bichot NP, Schall JD. Effects of similarity and history on neural mechanisms of visual selection. Nat Neurosci 2: 549 –554, 1999. Bisley JW, Goldberg ME. Attention, intention, and priority in the parietal lobe. Annu Rev Neurosci 33: 1–21, 2010. Born S, Ansorge U, Kerzel D. Feature-based effects in the coupling between attention and saccades. J Vis 12: pii 27, 2012. Brainard DH. The Psychophysics Toolbox. Spat Vis 10: 433– 436, 1997.

J Neurophysiol • doi:10.1152/jn.00275.2013 • www.jn.org

FEATURE ATTENTION BEFORE SACCADES Carello CD, Krauzlis RJ. Manipulating intent: evidence for a causal role of the superior colliculus in target selection. Neuron 43: 575–583, 2004. Carrasco M. Visual attention: the past 25 years. Vision Res 51: 1484 –1525, 2011. Castet E, Jeanjean S, Montagnini A, Laugier D, Masson GS. Dynamics of attentional deployment during saccadic programming. J Vis 6: 196 –212, 2006. Clark KL, Noudoost B, Moore T. Persistent spatial information in the frontal eye field during object-based short-term memory. J Neurosci 32: 10907– 10914, 2012. Cohen MR, Maunsell JR. Using neuronal populations to study the mechanisms underlying spatial and feature attention. Neuron 70: 1192–1204, 2011. Cornelissen FW, Peters EM, Palmer J. The Eyelink Toolbox: eye tracking with MATLAB and the Psychophysics Toolbox. Behav Res Methods Instrum Comput 34: 613– 617, 2002. Deubel H, Schneider WX. Saccade target selection and object recognition: evidence for a common attentional mechanism. Vision Res 36: 1827–1837, 1996. Deubel H. The time course of presaccadic attention shifts. Psychol Res 72: 630 – 640, 2008. Donk M, van Zoest W. Effects of salience are short-lived. Psychol Sci 19: 733–739, 2008. Egner T, Monti JM, Trittschuh EH, Wieneke CA, Hirsch J, Mesulam MM. Neural integration of top-down spatial and feature-based information in visual search. J Neurosci 28: 6141– 6151, 2008. Engbert R, Kliegl R. Microsaccades uncover the orientation of covert attention. Vision Res 43: 1035–1045, 2003. Fecteau JH, Munoz DP. Salience, relevance, and firing: a priority map for target selection. Trends Cogn Sci 10: 382–390, 2006. Gersch TM, Kowler E, Schnitzer BS, Dosher BA. Attention during sequences of saccades along marked and memorized paths. Vision Res 49: 1256 –1266, 2009. Gottlieb J, Kusunoki M, Goldberg ME. The representation of visual salience in monkey parietal cortex. Nature 391: 481– 484, 1998. Greenberg AS, Esterman M, Wilson D, Serences JT, Yantis S. Control of spatial and feature-based attention in frontoparietal cortex. J Neurosci 30: 14330 –14339, 2010. Gregoriou GG, Gotts SJ, Desimone R. Cell-type-specific synchronization of neural activity in FEF with V4 during attention. Neuron 73: 581–594, 2012. Hayden BY, Gallant JL. Time course of attention reveals different mechanisms for spatial and feature-based attention in area V4. Neuron 47: 637– 643, 2005. Hayden BY, Gallant JL. Combined effects of spatial and feature-based attention on responses of V4 neurons. Vision Res 49: 1182–1187, 2009. Hoffman JE, Subramaniam B. The role of visual attention in saccadic eye movements. Percept Psychophys 57: 787–795, 1995. Ipata AE, Gee AL, Bisley JW, Goldberg ME. Neurons in the lateral intraparietal area create a priority map by the combination of disparate signals. Exp Brain Res 192: 479 – 488, 2009. Itti L, Koch C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res 40: 1489 –1506, 2000. Itti L, Koch C. Computational modelling of visual attention. Nat Rev Neurosci 2: 194 –203, 2001. Jonikaitis D, Deubel H, de’Sperati C. Time gaps in mental imagery introduced by competing saccadic tasks. Vision Res 49: 2164 –2175, 2009. Jonikaitis D, Deubel H. Independent allocation of attention to eye and hand targets in coordinated eye-hand movements. Psychol Sci 22: 339 – 347, 2011. Juan CH, Shorter-Jacobi SM, Schall JD. Dissociation of spatial attention and saccade preparation. Proc Natl Acad Sci USA 101: 15541–15544, 2004. Kowler E, Anderson E, Dosher BA, Blaser E. The role of attention in the programming of saccades. Vision Res 35: 1897–1916, 1995. Liu T, Hospadaruk L, Zhu DC, Gardner JL. Feature-specific attentional priority signals in human cortex. J Neurosci 31: 4484 – 4495, 2011. Liu T, Mance I. Constant spread of feature-based attention across the visual field. Vision Res 51: 26 –33, 2011. Lovejoy LP, Krauzlis RJ. Inactivation of primate superior colliculus impairs covert selection of signals for perceptual judgments. Nat Neurosci 13: 261–266, 2010. Martinez-Trujillo JC, Treue S. Feature-based attention increases the selectivity of population responses in primate visual cortex. Curr Biol 14: 744 –751, 2004. Maunsell JR, Treue S. Feature-based attention in visual cortex. Trends Neurosci 29: 317–322, 2006.

1533

Melcher D, Papathomas TV, Vidnyánszky Z. Implicit attentional selection of bound visual features. Neuron 46: 723–729, 2005. Monosov IE, Sheinberg DL, Thompson KG. Paired neuron recordings in the prefrontal and inferotemporal cortices reveal that spatial selection precedes object identification during visual search. Proc Natl Acad Sci USA 107: 13105–13110, 2010. Monosov IE, Sheinberg DL, Thompson KG. The effects of prefrontal cortex inactivation on object responses of single neurons in the inferotemporal cortex during visual search. J Neurosci 31: 15956 –15961, 2011. Montagnini A, Castet E. Spatiotemporal dynamics of visual attention during saccade preparation: independence and coupling between attention and movement planning. J Vis 7: 8.1–16, 2007. Moore T, Armstrong KM. Selective gating of visual signals by microstimulation of frontal cortex. Nature 421: 370 –373, 2003. Moore T, Fallah M. Control of eye movements and spatial attention. Proc Natl Acad Sci USA 98: 1273–1276, 2001. Moore T, Fallah M. Microstimulation of the frontal eye field and its effects on covert spatial attention. J Neurophysiol 91: 152–162, 2004. Moore T. Shape representations and visual guidance of saccadic eye movements. Science 285: 1914 –1917, 1999. Müller HJ, Rabbitt PM. Reflexive and voluntary orienting of visual attention: time course of activation and resistance to interruption. J Exp Psychol Hum Percept Perform 15: 315–330, 1989. Nakayama K, Mackeben M. Sustained and transient components of focal visual attention. Vision Res 29: 1631–1647, 1989. Noudoost B, Moore T. Control of visual cortical signals by prefrontal dopamine. Nature 474: 372–375, 2011. Pouget P, Stepniewska I, Crowder EA, Leslie MW, Emeric EE, Nelson MJ, Schall JD. Visual and motor connectivity and the distribution of calcium-binding proteins in macaque frontal eye field: implications for saccade target selection. Front Neuroanat 3: 2, 2009. Rizzolatti G, Riggio L, Dascola I, Umiltá C. Reorienting attention across the horizontal and vertical meridians: evidence in favor of a premotor theory of attention. Neuropsychologia 25: 31– 40, 1987. Rolfs M, Carrasco M. Rapid simultaneous enhancement of visual sensitivity and perceived contrast during saccade preparation. J Neurosci 32: 13744 – 13752, 2012. Rolfs M, Jonikaitis D, Deubel H, Cavanagh P. Predictive remapping of attention across eye movements. Nat Neurosci 14: 252–256, 2011. Ruff CC, Blankenburg F, Bjoertomt O, Bestmann S, Freeman E, Haynes JD, Rees G, Josephs O, Deichmann R, Driver J. Concurrent TMS-fMRI and psychophysics reveal frontal influences on human retinotopic visual cortex. Curr Biol 16: 1479 –1488, 2006. Sato T, Murthy A, Thompson KG, Schall JD. Search efficiency but not response interference affects visual selection in frontal eye field. Neuron 30: 583–591, 2001. Sato TR, Schall JD. Effects of stimulus-response compatibility on neural selection in frontal eye field. Neuron 38: 637– 648, 2003. Schafer RJ, Moore T. Selective attention from voluntary control of neurons in prefrontal cortex. Science 332: 1568 –1571, 2011. Shadlen MN, Newsome WT. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86: 1916 –1936, 2001. Sheliga BM, Craighero L, Riggio L, Rizzolatti G. Effects of spatial attention on directional manual and ocular responses. Exp Brain Res 114: 339 –351, 1997. Stoppel CM, Boehler CN, Strumpf H, Krebs RM, Heinze HJ, Hopf JM, Schoenfeld MA. Spatiotemporal dynamics of feature-based attention spread: evidence from combined electroencephalographic and magnetoencephalographic recordings. J Neurosci 32: 9671–9676, 2012. Theeuwes J, Van der Burg E. The role of spatial and nonspatial information in visual selection. J Exp Psychol Hum Percept Perform 33: 1335–1351, 2007. Thompson KG, Bichot NP. A visual salience map in the primate frontal eye field. Prog Brain Res 147: 251–262, 2005. Thompson KG, Biscoe KL, Sato TR. Neuronal basis of covert spatial attention in the frontal eye field. J Neurosci 25: 9479 –9487, 2005. Treue S, Martinez-Trujillo JC. Feature-based attention influences motion processing gain in macaque visual cortex. Nature 399: 575–579, 1999. van Ettinger-Veenstra HM, Huijbers W, Gutteling TP, Vink M, Kenemans JL, Neggers SF. fMRI-guided TMS on cortical eye fields: the frontal but not intraparietal eye fields regulate the coupling between visuospatial attention and eye movements. J Neurophysiol 102: 3469 –3480, 2009.

J Neurophysiol • doi:10.1152/jn.00275.2013 • www.jn.org

1534

FEATURE ATTENTION BEFORE SACCADES

Watson AB, Pelli DG. QUEST: a Bayesian adaptive psychometric method. Percept Psychophys 33: 113–120, 1983. Wegener D, Ehn F, Aurich MK, Galashan FO, Kreiter AK. Feature-based attention and the suppression of non-relevant object features. Vision Res 48: 2696 –2707, 2008. White AL, Carrasco M. Feature-based attention involuntarily and simultaneously improves visual performance across locations. J Vis 11, 2011.

White AL, Rolfs M, Carrasco M. Adaptive deployment of spatial and feature-based attention before saccades. Vision Res 85: 26 –35, 2013. Zehetleitner M, Hegenloh M, Müller HJ. Visually guided pointing movements are driven by the salience map. J Vis 11, 2011. Zenon A, Krauzlis RJ. Attention deficits without cortical neuronal deficits. Nature 489: 434 – 437, 2012. Zhou H, Desimone R. Feature-based attention in the frontal eye field and area V4 during visual search. Neuron 70: 1205–1217, 2011.

J Neurophysiol • doi:10.1152/jn.00275.2013 • www.jn.org