Neuropsychologia 39 (2001) 937– 949 www.elsevier.com/locate/neuropsychologia
Processing of irrelevant visual motion during performance of an auditory attention task Geraint Rees a,b,*, Chris Frith b, Nilli Lavie c b
a Institute of Cogniti6e Neuroscience, Uni6ersity College London, Alexandra House, 17 Queen Square, London WC1N 3AR, UK Wellcome Department of Cogniti6e Neurology, Institute of Neurology, Uni6ersity College London, 12 Queen Square, London WC1N 3BG, UK c Department of Psychology, Uni6ersity College London, Gower Street, London WC1E 6BT, UK
Received 15 November 2000; received in revised form 15 January 2001; accepted 17 January 2001
Abstract The extent to which irrelevant perception of visual motion distractors can be modulated by manipulating auditory load in a relevant task was assessed with Positron Emission Tomography (PET) and behavioural experiments. Subjects performed an auditory task and ignored an irrelevant visual motion stimulus, under two conditions. In a low load condition, subjects were asked to detect words spoken in a loud voice among words spoken in a quiet voice, while in a high load condition they attempted to detect bisyllabic words among monosyllabic and trisyllabic words. We found that motion-related visual areas were strongly activated by the irrelevant motion stimulus, compared to a static stimulus, under both conditions of load in the auditory task. In a second behavioural experiment, the duration of the motion after-effect was similarly unaffected by adaptation under low or high auditory load. These results are in clear contrast with the strong modulation of irrelevant motion processing by 6isual load, as reflected in the duration of the motion after effect (Section 6) and neural responses in motion-related visual areas (Rees et al., Science, (1997) 278, 338). These findings support the claim that attentional capacity is restricted within but not between sensory modalities, and indicate that processing of 6isual distractors may occur whenever there is sufficient 6isual capacity to process them, despite being task- and modality-irrelevant. © 2001 Elsevier Science Ltd. All rights reserved. Keywords: PET; Attention; Load; Crossmodal; Distractors; After-effect
1. Introduction Directing attention to a particular stimulus in the environment often entails ignoring other stimuli elsewhere. However, the extent to which irrelevant stimuli can be successfully ignored has been a topic of enduring controversy in psychology for the last four decades. During that time, many studies suggested that irrelevant stimuli could be successfully excluded from perceptual processes (supporting an ‘early selection’ view of attention e.g. [43]). However, many other studies found that irrelevant stimuli are perceived and interfere with relevant processing, and thus concluded that attention can only affect post-perceptual processes such as * Corresponding author. Present address: Institute of Cognitive Neuroscience, University College London, Alexandra House, 17 Queen Square, London WC1N 3AR, UK. Tel.: +44-20-76795496; fax: +44-20-78132835. E-mail address:
[email protected] (G. Rees).
memory or response selection (a ‘late selection’ view, e.g. [11,40]). To date the extent to which attention can result in prevention of irrelevant perception thus remains a controversial issue (for a review see [27]). Recent functional imaging studies have provided a new source of evidence that bears on this longstanding debate. Functional MRI (fMRI) and Positron Emission Tomography (PET) have been used to demonstrate that attention can modulate perception, with smaller brain responses to task-irrelevant compared to task-relevant stimuli in extrastriate visual cortex [5,7,8,33] and sometimes even in primary visual cortex [18,29,37]. Thus, these recent studies seem to provide new evidence in favour of an early selection view of attention, namely that attention can result in selective perception. However, all of these neuroimaging studies have only compared brain responses to attended and unattended stimuli. It is therefore, not clear whether the effects of attention are mainly due to enhanced perception of
0028-3932/01/$ - see front matter © 2001 Elsevier Science Ltd. All rights reserved. PII: S0028-3932(01)00016-1
938
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
attended stimuli or attenuated perception of unattended stimuli. Indeed, single cell electrophysiology experiments in monkeys show that the processing of unattended stimuli can be modulated by attention [30,45] and it has been suggested that attention operates by attenuating distractors rather than by changing the processing of attended stimuli [32]. However, as previous neuroimaging studies have not manipulated the identity and presence of irrelevant stimuli, they cannot directly address the critical issue of the early and late selection debate: namely the extent of processing for irrelevant stimuli. Moreover, the results from both the neuroimaging and single cell studies cannot provide a satisfying resolution to the early versus late selection debate, as it is hard to explain on the basis of these early- selection results why attention would sometimes fail to affect perception altogether (as shown by the numerous reports of late selection in psychology; for reviews see [22,27]). Lavie [23,24,27] has recently suggested that a resolution to the early and late selection debate may be found if a hybrid model of attention, that combines aspects from both views, is considered. According to this model, the extent to which irrelevant distractors can be excluded from perception depends on the level of load that is incurred while processing relevant stimuli. Situations of high load in relevant processing will exhaust perceptual capacity, leaving none of this capacity available for distractor processing, and so inevitably excluding them from perception. However, in situations of low load, any spare capacity left over from the less demanding relevant processing will ‘spill over’ to the processing of irrelevant distractors. Thus, on this model early selection is predicted for situations of high perceptual load, while late selection is predicted for situations of low perceptual load. Several behavioural studies have supported these predictions, by showing that distractor effects on RTs (e.g. response competition effects [14]; or negative priming, [40]) were always found in conditions of low load and could only be eliminated by higher load in relevant processing (e.g. [23,25,26]). Although framed in terms of cognitive processes, Lavie’s load model makes clear predictions concerning neural responses to distractors. Responses in sensory cortices that reflect distractor perception should be found only in conditions of low load in relevant processing, and be reduced by higher processing load. We tested these predictions in an earlier study using fMRI [35]. Subjects performed a linguistic task on visually presented words while attempting to ignore irrelevant dots that could be either static or moving. Despite the motion distractors being irrelevant and ignored, both behavioural measures of motion after-effect and functional imaging measures of motion perception in V5/ MT and other related visual areas (e.g. V1/V2 border) indicated distractor processing under conditions of low
load in the primary task (i.e. ‘late’ selection). However, when processing load was increased, motion-specific activity due to the irrelevant stimuli was reduced in these visual areas (i.e. ‘early’ selection). Thus, neural responses to distractors in our previous study were consistent with either early selection or late selection, depending on the load in a relevant task, in support of the predictions we derived from the load model. The purpose of the present study was to investigate whether load effects are modality specific or can be obtained across the visual and auditory modalities. To meet this goal we designed a functional imaging study in which the distractor and target were presented in different sensory modalities. While subjects performed a task on auditory stimuli, an unrelated visual distractor, either moving or static, was displayed. Previous studies have demonstrated that people can share attentional capacity efficiently between stimuli as long as these are presented between modalities rather than within modality (e.g. [2,31,44]). If this is the case then we should find that distractors are processed regardless of the perceptual load in the auditory modality of our experiment, as both auditory conditions should leave sufficient capacity for 6isual perception. Such results will provide strong support for our claim that people are unable to ignore visual distractors despite their being irrelevant and presented in another modality, whenever there is sufficient capacity available for their visual processing. As we know from our previous study that the high load task was sufficiently engaging to eliminate processing of irrelevant motion distractors, we used a very similar high load task in the present study except that the word stimuli were now presented in the auditory rather than the visual modality. By asking subjects to perform either easy or hard tasks on identical streams of spoken words, we were able to evaluate the perception of irrelevant visual motion as a function of auditory perceptual load. Manipulating attentional load in a different modality to the irrelevant distractors also allowed us to examine the question of whether attentional capacity is shared between modalities. As discussed above, we predicted that visual distractor perception would be found in both conditions of auditory perceptual load if capacity limits exist within modality but not between modalities. If, however, general capacity limits exist across modalities [4] then as with our previous unimodal study, distractor perception will occur only in conditions of low perceptual load, and will be reduced by increasing the auditory load. Another motivation for the current study was to rule out a possible alternative account for our previous results. Our unimodal study seems to provide the first clear demonstration of a late selection result in functional imaging. In the low load condition we found that moving distractors evoked strong activity in V5/MT,
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
compared to static distractors, despite their irrelevancy and explicit instructions to subjects to ignore them. This finding has potentially an interesting implication; it implies that distractor processing is not under voluntary control. Distractor processing depends on the level of load in the display, rather than on subjects’ intentions to ignore the distractors. However, our late selection result may be open to some criticism. As described above, the linguistic task involved presentation of words at fixation, and although the presentation rate was fairly slow (1 word/s), the onsets and offsets of the visual stimuli may evoke a motion signal (flicker) that might activate motion-sensitive visual areas [42]. Moreover, this motion signal was perhaps more noticeable in the low load condition which involved purely visual processing (subjects had to detect upper vs. lower case words) than in the high load condition which required also some phonological processing (subjects had to detect bisyllabic words among monosyllabic and trisyllabic words). This criticism cannot account for our results, which showed an interaction between load and distractor motion. Nonetheless it seemed desirable to assess the processing of motion distractors that subjects attempt to ignore, while performing in a relevant auditory task that does not involve any motion signal at all. To measure brain activity we chose to use PET rather than fMRI. Functional MRI is a noisy technique, and ambient noise in the bore of the scanner may exceed 100 dB. Although auditory perception is possible within the magnet with appropriate sound delivery systems, it is not possible to achieve a truly low load condition for the auditory system, as intrusive background noise is always evident to some extent. Thus we used PET, which is virtually silent.
2. Experiment 1
2.1. Method 2.1.1. Subjects Five right-handed subjects (four men, one woman, mean age 34.8 years, and range 26– 43years) gave their informed consent to take part in the imaging study. The local ethics committee approved the study, and permission to administer radioactive substances was obtained from the Advisory Committee on Radioactive Substances (UK). 2.1.2. Experimental paradigm Subjects were presented with auditory and visual stimuli while undergoing scanning. Attention was always directed to the auditory stimuli alone. Subjects were instructed that the visual stimulus was irrelevant, could interfere with performance, and should be ignored at all times.
939
Auditory stimuli consisted of single spoken words presented at two different volume levels (quiet and loud) sequentially over insert earphones with a stimulus onset asynchrony of 1.5 s. Words were high frequency concrete nouns of between one and three syllables, drawn from exactly the same lists as those used in our previous study [35]. In separate scans, subjects performed one of two tasks on identical lists of words presented in random order. In the low load task, they were asked to press a button whenever they heard a loud word. The volume at which the words were presented was arranged so that the loud words were clearly audible, while the quiet words were barely audible. For the high load task, subjects were asked to press a button whenever they heard a bisyllabic word among the mixture of trisyllabic and monosyllabic words presented at loud and quiet volumes. There was no correlation between the volume at which a word was presented and its syllable content. Thus, in the high load task, subjects were forced to listen to and analyse all the words, including the quiet words. The auditory task was presented for a period of approximately 60 s during the scanning frame of 90 s. This resulted in presenting a list of 40 words per frame; 10 words were targets (25%), presented in random intervals in the stream. Note that stimuli were physically identical during high load and low load tasks, so any difference in brain activity reflects task performance rather than stimulus characteristics. During scanning, subjects were asked to fixate on a fixation point positioned at the centre of a computer monitor placed comfortably in their line of sight, at a distance of 45 cm. An irrelevant visual stimulus was presented on the screen during performance of the auditory task. This was composed of 400 white dots, each subtending approximately 0.1°, placed randomly around the fixation point. The total size of the display was approximately 36° horizontally and 27° vertically. There were two conditions. In the no-motion condition, the dots remained static throughout the scanning frame. In the motion condition, the dots moved radially away from the fixation point at a constant velocity of approximately 8°/s. As dots left the edge of the screen, they were re-plotted at the centre so that the dot density remained constant over time. Prior to scanning, subjects were shown the visual stimuli and practised fixation. The requirement for fixation during the scanning frame was emphasised before and during the experiment. Direct observation of eye position during scanning for four subjects confirmed accuracy of fixation during the scanning frame. This experiment therefore, represents a 2× 2 factorial design where two factors are independently manipulated: the presence of a moving visual stimulus and the level of load in the auditory task. Each combination of load and distractor type was presented three times
940
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
while brain activity was measured with Positron Emission Tomography (PET) to give a total of 12 scans per subject. Each subject received a different order of the conditions of load and distractor. It was emphasised to subjects that the low load task required only cursory attention to the quiet stimuli, while the high load task would be challenging and requires identification of every stimulus.
2.1.3. Functional imaging and data analysis Cerebral blood flow (CBF) scans were obtained using an ECAT EXACT HR + scanner (CTI Siemens, Knoxville, TN) for each subject during task performance, while lying in a quiet, darkened room. The scanner was operated in 3D mode with septa retracted. A venous cannula to administer the tracer was inserted in the left antecubital fossa vein of the subject, and radioactivity was administered as a bolus of approximately 350 MBq H15 2 O over 20 s followed by a saline flush. Twelve PET emission scans were collected over 2 h, with an 8 min interval between scans. This represents three replications of each combination of the two experimental factors. Integrated radioactivity counts were accumulated over a 90 s acquisition period, beginning with the rising phase of radioactivity counts in the head, used as an index of regional CBF. A transmission scan was collected prior to the emission scans to correct for the attenuating effect of extra-cerebral tissue. Data were analysed with SPM99 (Wellcome Department of Cognitive Neurology, http://www.fil.ion. ucl.ac.uk/spm) running under MATLAB 5.3 (The Mathworks, Inc., Sherbourn MA). Each subject’s PET scans were realigned, spatially normalised to the stereotactic space of Talairach and Tournoux [39] and smoothed with a Gaussian kernel of 14 mm [16,17]. Condition-specific activation was estimated using the General Linear Model, with global normalisation (to a mean of 50–100 ml/min) using a subject-specific ANCOVA. Appropriately weighted linear contrasts between the experimental conditions identified activated areas on a voxel-wise basis. The resulting set of t values constituted a Statistical Parametric Map (SPM{T}) which was globally thresholded at a level of P B0.001, uncorrected for multiple comparisons. Due to the very large number of voxels, here we report only those loci that survived a statistical threshold of P B0.05 after correction for multiple comparisons, except where specifically discussed in the text.
3. Results
the high load condition, only 66.1% (SD= 11.3%) of the syllable targets were correctly detected. This difference is highly significant, (F(1, 4)= 40.56; P B 0.001), and confirms that our manipulation of load in the auditory task was effective. Despite poorer performance in the high load condition, subjects made a similar number of responses in both low load (average 9.9 responses per epoch: SD= 2.2) and high load (average 8.9 responses, SD= 1.3). This difference was not significant (t= 3.03, P\ 0.05). Thus the two conditions were balanced for motor output, and the presence of false positives in the high load condition confirms that subjects were attempting to perform the task.
3.2. Functional imaging To recapitulate we predicted that, if capacity limits for perception exist only within modalities, activity related to 6isual motion distractors should be found under both conditions of low and high auditory perceptual load, as both conditions should leave capacity available for visual processing. If however, capacity limits hold across the modalities, motion related activity in visual cortex should be seen under conditions of low auditory load, but not under conditions of high auditory load. Our results provided support for the first prediction.
3.2.1. Mo6ing 6ersus static 6isual distractors A comparison of the conditions with moving distractors versus static distractors (across the conditions of load) provided a measure of the main effect of distractor processing. Extensive and confluent activation (which may reflect the relatively low spatial resolution of PET) was seen throughout posterior cortical visual areas (Fig. 1 Table 1). As the experiment was carried out using PET, assignment of activity to functionally defined areas with retinotopic mapping was not possible. However, the activated areas overlapped with coordinates and surface anatomy previously reported [9,19] for areas V1, V2, V3A, the ‘Kinetic Occipital’ area [13] and V5/MT bilaterally [47]. The locations of the most significant activations in these areas are shown in Table 1. All of these areas also showed highly significant activation when motion was compared with no motion, separately for the low and high load conditions (simple main effects of visual motion, see Fig. 2). Qualitatively, we noted that the simple main effects of motion (comparison of motion and no-motion distractor conditions separately for low and high load) were of comparable strength and anatomical distribution under both levels of load.
3.1. Beha6ioural Subjects correctly detected 99% (SD= 2.2%) of the auditory targets in the low load condition. However, in
3.2.2. High 6s. low auditory load The comparison of conditions where subjects performed the high auditory load task with the low audi-
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
tory load task revealed activation in left frontal operculum and left posterior basal temporal gyrus (Brodmann area 37). Both these areas have been consistently implicated in the retrieval of phonological information about words [34]; for example when word production is contrasted with semantic decisions on auditory stimuli [46], or when reading is contrasted with viewing consonant letter strings. Both types of contrast would be expected
941
to emphasise processes involved in phonological retrieval and ‘sounding out’ words [34] which were involved in our syllable judgement task. Bilateral inferior and middle frontal gyri were also activated in this comparison. Enhanced activity in frontal regions has been seen previously with increased working memory demands of a task [28]. Lesions to the right inferior frontal gyrus in a similar location to that activated
Fig. 1. Main effects of visual motion. Areas where activity produced by the irrelevant visual motion stimulus was greater than that produced by the irrelevant static visual stimulus, irrespective of auditory task demands. The activations are overlaid onto a T1 weighted anatomical template image in the stereotactic space of Talairach & Tournoux [39]. Upper panel (A). An axial slice (Z = +0 mm) showing activation in motion-sensitive areas V5/MT and the Kinetic Occipital area. The centers of each of the circles superimposed on the image are placed at the previously identified Talairach co-ordinates of right and left V5/MT (A and B from [47]) and right and left Kinetic Occipital areas (C and D from [13]). Lower panel (B). A higher slice (Z = +10 mm) showing bilateral activation of area V3a in the same statistical comparison. The locations of these activated areas (P B 0.05, corrected for multiple comparisons) are listed in Table 1.
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
942
Table 1 Co-ordinates and Z scores for activation produced by the presence of irrelevant visual motion, compared to an irrelevant static stimulus, for the regions of interest (motion-sensitive areas)a Cortical area
Left V5 [47] Right V5 [47] Left Kinetic Occipital [13] Right Kinetic Occipital [13] Left V3a [9] Right V3a [9]
Talairach co-ordinates
T-value
X
Y
Z
−44 40 −26 32 36 −28
−70 −70 −92 −92 −84 −92
0 0 4 0 16 10
7.03 5.43 9.72 13.33 5.26 10.56
a Shown in the table are loci where differential activity was greater during the comparisons of irrelevant visual motion with no visual motion, independent of auditory load. Only the most significant peaks within each area of activation are reported in the table (PB 0.05, corrected for multiple comparisons).
in the present study may give rise to disorders of attention [21]. Thus the enhanced activity in these frontal areas may indicate the increased demands on attentional control with higher load Fig. 3. In addition a number of subcortical areas in the midbrain, brainstem and cerebellum were also activated (Table 2). The reverse comparison of low auditory load and high auditory perceptual load revealed activity in three left-hemisphere areas. These may represent activation during performance of the low load task, or functionally relevant deactivation during performance of the high load task.
3.2.3. Interaction between auditory perceptual load and 6isual motion This comparison represents the difference between differential activation due to the moving visual distractor stimulus (compared to static) comparing low auditory load and high auditory load. This identifies those areas that respond more strongly to the effects of irrelevant visual motion when auditory load is low. There were no areas of interaction between distractor motion and load within the visual cortical areas (both striate and extrastriate) described in Table 1 at the overall statistical threshold. Even at a lower threshold of PB0.001, uncorrected for multiple comparisons, no interaction was found apart from a single voxel (co-ordinates {30–98 4}; T=3.29). Not only was the interaction weak and only just above the lowered statistical threshold, but this voxel was isolated and located in right superior Brodmann area 18. Its importance is therefore unclear.
4. Experiment 2 The results from the high load condition in the present study contrast with our previous imaging findings [35] of reduced responses to 6isual motion distractors in motion sensitive areas in conditions of high 6isual load. We considered the possibility that two minor differences in paradigm between the studies might have contributed to the difference in results. In the unimodal study, visual motion distractors were
Fig. 2. Activity evoked by irrelevant visual stimulation under different levels of auditory perceptual load. Regional cerebral blood flow (normalised to a global mean of 50 ml/100 ml per min) is shown as a function of experimental condition for left V3A (A), left V5/MT (B) and the right Kinetic Occipital area (C). These areas correspond to the areas shown in Fig. 1. Error bars represent the inter-subject standard deviation of the blood flow measurement. Note that in all areas the activity evoked by the motion stimulus is greater than the no motion stimulus, whether under conditions of low auditory perceptual load or under conditions of high auditory perceptual load. For each of these areas (and all the other areas listed in Table 1), there was no detectable interaction between auditory perceptual load and the presence of irrelevant visual motion.
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
943
tional manipulation ineffective? The radiation dose limitations of PET preclude a direct comparison within the same subjects, so we examined this alternative account in a behavioural experiment. Prolonged exposure to visual motion, followed by viewing of a static stimulus, produces an illusory perception of opposing motion in the static display that fades over time. This motion after-effect (MAE) is contingent on V5/MT activity [41] and has been found to be sensitive to attention (5, 35). We therefore, measured the duration of the MAE in four further subjects following adaptation to ignored visual motion under conditions of high and low auditory load, using the same manipulation of load as in Section 2. The displays and word lists used were identical to those described above, with two important modifications. First, a small blank ellipse was placed at the centre of the visual motion display. Second, the auditory words were presented at the higher rate of 1 Hz. These differences were introduced to make the paradigm and display used exactly comparable to that used previously in our unimodal imaging study.
4.1. Method 4.1.1. Subjects Four right-handed subjects (three men, one woman, mean age 25.1 year, and range 21–32) gave their informed consent to take part in the behavioural study, which was approved by the local ethics committee. None of the subjects had participated in Section 2.
Fig. 3. Main effects of auditory perceptual load. Areas where activity during high auditory perceptual load was greater than that during low auditory perceptual load. Activated areas are shown superimposed on the left lateral (A), right lateral (B) and inferior (C) surfaces of the T1-weighted anatomical template image in Talairach space. The locations of the activated areas (P B 0.05, corrected for multiple comparisons) are listed in Table 2.
presented outside a small central blank ellipse, and the attended word stream was presented visually at a higher overall rate (1 vs. 1.5 s ISI in the current cross modal study). Perhaps the existence of motion distractors in the centre in the present study made it harder to ignore them, or the lower rate of presentation made the atten-
4.1.2. Stimuli and procedure Subjects viewed a computer monitor from 60 cm in a dimly illuminated room, on which was presented a stimulus identical to that used in Section 2 except for the presence of a blank ellipse placed at the centre of the display. The blank ellipse obscured the moving dots (but not the fixation point) in that area. The ellipse subtended approximately 3° (vertical) by 4° (horizontal), as used in the previous unimodal visual study (33). While viewing this visual motion stimulus, subjects listened through insert earphones to the same sets of words used during scanning runs in Section 2, except that the words were presented at the faster frequency of 1 Hz. The high and low auditory load tasks were identical. After the sequence of words finished, the moving dots were replaced by a static display of dots at the same overall density, and subjects were asked to indicate the duration of the MAE by button press. Subjects were shown the stimulus and apparatus beforehand to ensure that they were familiar with the task requirements, but not otherwise given practice. Subjects completed four measurements of MAE duration after adapting under either high or low auditory load, with the order of measurement counterbalanced across subjects.
944
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
Table 2 Co-ordinates and Z scores for activation evoked by changing auditory perceptual load of the relevant taska Cortical area
Talairach co-ordinates X
T-value
Y
Z
High auditory perceptual loadBLow auditory perceptual load L post basal temporal gyrus (BA37) −56 L frontal operculum −34 L inf/middle frontal gyrus (BA 45) −52 R inf/middle frontal gyrus (BA 9/45) 46 R inferior frontal gyrus (BA 45) 36 R middle frontal gyrus (BA 46) 42 R midbrain 4 R brainstem 10 L cerebellum −36
−56 22 28 32 22 46 −36 −24 −64
−20 6 22 32 4 16 −20 −4 −28
6.37 5.00 (corrected P= 0.09) 5.67 6.17 6.00 5.81 6.11 6.20 5.97
Low auditory perceptual loadBHigh auditory perceptual load L precentral/postcentral gyrus −32 L inferior temporal/fusiform gyrus −56 L supramarginal gyrus −46
−24 −22 −58
54 −30 36
5.96 5.71 5.39
a
Shown in the table are loci where differential activity was greater during the comparisons of high auditory load with low auditory load; and the inverse comparison. Only the most significant peaks within each area of activation are reported in the table (PB0.05, corrected for multiple comparisons, except where specifically stated).
5. Results
6. Experiment 3
Subjects correctly detected 98% (SD= 1.4%) of the auditory targets on average in the low perceptual load condition. However, in the high perceptual load condition, only an average of 54% (SD=17%) of the targets were correctly detected. This difference is highly significant, (F(1, 6)=25.79; P B 0.002), and confirms that perceptual load was manipulated appropriately by the two tasks. Note that the performance difference comparing high and low load is slightly greater than that found in our imaging experiment, which could be the result of the smaller number of trials run in the behavioural task (and thus reduced practice) and/or the increased load at the higher word presentation rate. Despite this, auditory load had little influence on the MAE duration. Fig. 4 presents the duration of the MAE as a function of load for each subject. There was no difference in the duration of the MAE comparing high and low auditory load, neither in the group (F(1, 24)=1.85; P \0.10) nor in individual subjects (All t’s B 1.5; all P-values \0.10). These results are consistent with the imaging findings we report in Section 2, with no apparent reduction in MAE duration despite increasing auditory load. As we discuss above (in Section 1) these results contrast with our previous imaging findings that irrelevant perception of visual motion distractors can be modulated by increasing the load in a 6isual task. We therefore, conducted a third experiment to examine whether a similar manipulation of load in linguistic judgements to the one currently used in Sections 2 and 4 can indeed modulate perception of irrelevant visual motion distractors when load is manipulated in a 6isual task.
6.1. Methods 6.1.1. Subjects Four new right-handed subjects (three male, one female) gave their informed consent to take part in the
Fig. 4. Behavioural effects of load in an auditory task on the motion aftereffect (MAE) from irrelevant distractors. MAE duration for four different subjects (see Section 4.1) after adapting to the visual motion stimulus under either high or low auditory load. No significant differences in MAE duration are seen.
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
945
Fig. 5. Behavioural effects of load in a visual task on the motion aftereffect (MAE) from irrelevant distractors. Left panels MAE duration for four different subjects (see Section 6.1) after adapting to the visual motion stimulus under either high or low visual load. A significant reduction in MAE duration is seen under high load in every subject. Right panel Comparison of relative reduction in MAE duration under high load in Sections 4 and 6. A value of 1 indicates no reduction in average MAE duration under high load; larger values indicate greater duration under high load while smaller values indicate a reduction of the MAE duration with high load.
experiment, which was approved by the local ethics committee. None of the subjects had participated in Sections 2 and 4. Initial report of parts of these data has been made previously [35].
6.1.2. Stimuli and procedure The stimuli and procedure were the same as in Experiment 2 except for the following changes. Single words (each measuring approximately 2° across) drawn from the lists used in Sections 2 and 4 were presented serially inside the blank ellipse at the centre of the display. No auditory stimuli were presented. The words could be presented in either upper or lower case. Subjects were asked to monitor this word stream and indicate by button press when a target appeared. In the low visual load condition the target were words displayed in upper case; in the high visual load targets were bisyllabic nouns. Subjects adapted to irrelevant visual motion during task performance in the same manner as for Experiment 2, subsequently indicating the duration of the MAE. 6.1.3. Results Subjects correctly detected 80% (SD=11%) of the visual targets on average in the low perceptual load condition. However, in the high perceptual load condition, only an average of 63% (SD=5%) of the targets were correctly detected. This difference is significant, (F(1, 6)= 8.25; PB0.05), and confirms that perceptual
load was manipulated appropriately by the two tasks. Note that the performance difference comparing high and low load is slightly lower than that in Experiments 1 and 2, so this experiment had less power than the others did to detect any effect of perceptual load on MAE duration. Despite this, visual load had a strong influence on the MAE duration. Fig. 5 presents the duration of the MAE duration as a function of visual load for each subject. There was a highly significant decrease in the duration of the MAE comparing high and low auditory load, both in the group (F(1, 48)= 35.68; PB0.001) and in individual subjects (All t’s\1.92; all P-valuesB 0.05 one tailed). These behavioural results stand in striking contrast to the results from Experiment 2. Fig. 5B presents the relative reduction in MAE duration for the four subjects from this unimodal experiment, compared to that seen in the crossmodal situation of Experiment 2. To facilitate this comparison, MAE durations have been normalised within subject to the mean MAE duration under low load, as different subjects have different overall MAE durations. While a substantial reduction of around 20% in the MAE duration is seen comparing high and low 6isual load, there is no reduction whatsoever comparing high and low auditory load. Moreover, this experiment used exactly the same stimulus presentation (in terms of presentation rate, and display
946
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
configuration) as the one used in our earlier imaging and behavioural studies of visual load. Thus we can rule out alternative accounts to the difference in neuroimaging findings between the unimodal experiments and the cross-modal experiments in terms of differences in stimulus presentation.
7. Discussion
7.1. Processing of 6isual distractors during the auditory task is consistent with late selection In this study we were able to identify the extent to which visual cortex responded to an irrelevant motion stimulus during performance of an auditory task. We found that strong activation was produced throughout motion-sensitive visual cortex by a moving irrelevant stimulus compared to a static irrelevant stimulus. This activation was seen both when the auditory task was easy (low perceptual load) and when the auditory task was very demanding (high perceptual load), with no significant difference between the activation in these two conditions. Thus our study showed that visual motion distractors are nonetheless processed despite their being irrelevant to a current task, with clear instructions to ignore them. Although these findings suggest that distractors are processed in the same way in the two auditory load conditions, an alternate possibility to consider is that PET is insensitive to relevant differences in distractor processing. For example, attentional load might increase the selectivity of tuned sensory neurons [38]. This might not affect the PET signal in a cortical sensory area since fewer neurons would be activated, but their level of activity will increase, leaving the overall activity unchanged. However, for moving visual stimuli, existing evidence does not support such a hypothesis, as attention increases the gain of directionselective neurons in visual cortical area V5/MT without narrowing the direction-tuning curves [45]. In such a situation the PET signal in cortical sensory areas would only remain unchanged if the gain remained unchanged (i.e. independent of attention). Our findings that distractor-related activity was independent of attentional load thus provides support for a late selection view, where distractor processing is assumed to be independent of attention. As mentioned earlier, several imaging studies have previously reported attentional modulation of distractor processing and have thus supported an early selection view of attention [5,8,33,35]. However, these previous studies have typically presented target and distractor stimuli within a single modality. Their early selection results can be accommodated with the present late selection results found for distractors and targets in different modalities by adapting a hybrid model for attention (according to
which both early and late selection exist) to account for cross-modal situations as well. As discussed earlier (in Section 1), Lavie (1995) has suggested such a hybrid model for attentional selection. According to her model, the level of load in the processing of a relevant task determines whether irrelevant distractors are processed (i.e. late selection) or can be successfully ignored (i.e. early selection). Early selection can only be found under conditions of high load, which serve to exhaust attention in relevant processing. Otherwise, in situations of low load on relevant processing, late selection is found because spare capacity from the relevant processing will spill over to distractor processing as well. This load model has only been applied to unimodal studies of selective attention, and our present results suggest an important qualification of this model for the situation of crossmodal selection. The findings of the present study reinforce the model’s claim that distractors are always processed whenever there is capacity available for their processing. Thus, visual distractors were processed (i.e. late selection) regardless of auditory load because manipulations of auditory perceptual load still left free capacity for 6isual perception. Interestingly, any free visual capacity appears to be allocated to processing of distractor stimuli which people attempt to ignore. Thus, our study implies that distractor processing is not under voluntary control: visual distractors are processed whenever there is sufficient visual capacity for their perception. The finding that irrelevant visual distractors are processed regardless of auditory load also provides evidence against an alternative account for our previous demonstration that late selection is found under low visual load [35], in terms of a failure to manipulate attention in easy tasks. The behavioural data from the present study clearly show that general task difficulty was increased, with reduced accuracy under high load. This increase in task difficulty is consistent with our imaging findings of greater activation in frontal areas that are typically associated with attention. Nevertheless, despite the clear increase in task difficulty with high auditory load, visual distractors still evoked robust differential brain activity reflecting their processing.
7.1.1. E6idence for capacity limits within but not between modalities The results from the high load condition in the present study stand in direct contrast to our previous imaging result of reduced responses to 6isual motion distractors in motion sensitive areas in conditions of high 6isual load. Since the high load task was very similar between the two experiments (it involved a syllable judgement in both cases) the contrast in the results between these studies is strongly supportive of the idea that attentional capacity is restricted within a modality but not between modalities. This possibility is
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
supported by our behavioural measurement of the motion after-effect duration. While the MAE duration was strongly influenced by high 6isual load in Section 6, the duration was unchanged under situations of high auditory load in Section 4. These findings are consistent with recent behavioural work reported by Spence and colleagues [6]. They replicated our finding of a strong effect of visual load on the motion after-effect, but showed little or no behavioural influence of visual load on processing of irrelevant auditory information or vice versa. Similarly, previous ERP findings are also consistent with our new neuroimaging data. Ahlo and colleagues found that deviance related negativities to infrequent targets in the visual modality are unaffected by increasing auditory processing load, and vice versa [1]. Our findings converge with the results of these studies to show that distractor perception depends on attention as long as processing load is manipulated within a modality, and is independent of attention when load is manipulated in a different modality. These results thus point to separate capacities for cortical areas that subserve processing in different modalities and shared attentional capacity for any cortical areas which are involved in processing stimuli within the same modality. As we mention earlier (in Section 1) relatively few studies have compared interference between two sources of input within and between modalities, but the findings of these studies are suggestive that attentional capacity may be restricted within but not between sensory modalities. Treisman and Davies found that searching for a target among distractors in different modalities was more efficient than searching within the same inputs in the same modality [44]. More recently, Duncan and colleagues [12] found that attending to two streams of letters in the same modality (e.g. two visual streams, or two auditory streams) produced interference in target identification (a phenomenon termed the ‘attentional blink’). However, no such interference was found when the streams were in different modalities (i.e. a visual stream combined with an auditory stream). They argued that these findings suggested that any restriction on concurrent attention to more than one target was modality-specific, implying that the major source of attentional restrictions must lie in modalityspecific systems. The present findings are consistent with this view. However, it is important to note that our findings, that brain activity related to motion distractors and the MAE are modulated by visual load, but unaffected by auditory load, do not imply that all limitations or resources are necessarily modality specific. For example, even though Treisman and Davies [44] observed better performance dividing attention within a modality, there was still a deficit relative to focused attention. Similarly, other groups have observed cross modal interference in target identification
947
paradigms similar to that used by Duncan [3], and a definitive conclusion regarding the nature of capacity limits in situations that produce the ‘attentional blink’ remains elusive at present. A few functional imaging studies have claimed that attending to one sensory modality may lead to general suppression of sensory input in other sensory modalities. In these studies decreased activity in primary or association cortex of task-irrelevant sensory modalities (e.g. decreased blood flow in auditory cortex during performance of a visual task, or vice-versa) has been found [15,20]. However, in none of these studies were stimuli presented in the task-irrelevant modality to directly assess task-irrelevant sensory processing. Decreases in activity in task-irrelevant sensory cortex may therefore represent non-specific factors such as arousal, rather than stimulus processing per se. Moreover, many other imaging studies have failed to find similar cross modal effects (e.g. [8]). Shulman systematically examined this inter-experiment variability by performing a meta-analysis of nine functional imaging studies of attention [36]. He found little evidence to support suppression of auditory cortex during visual attention and concluded that ‘these results are inconsistent with a model in which the precortical input to task-irrelevant sensory cortical areas is broadly suppressed’ (p. 193). The present study supports this conclusion, for the specific case of auditory attention and task-irrelevant visual processing. Qualitatively, it is interesting to note that we observed weak and inconsistent decreases in overall activity in some visual areas as auditory perceptual load increased (e.g. compare high and low load in Fig. 2b). If we had not independently measured distractor processing by comparing two distractor types, it might have been erroneously concluded, that such decreases in visual cortex activity as auditory attentional demands increased, reflected crossmodal suppression of visual processing. However, all of these areas showed strong activation comparing irrelevant visual motion and a static stimulus under both high and low auditory perceptual load, with no significant interaction (Fig. 2). Thus, the differential sensitivity of these visual areas to a task-irrelevant visual stimulus was retained regardless of increases in auditory load. These observations clearly illustrate the necessity of independently manipulating both task-relevant and task-irrelevant processing in order to understand the processing of ignored, task-irrelevant stimuli. Finally, it should be noted that our conclusion that capacity limits may only exist within but not between modalities obviously does not rule out other forms of interactions between sensory modalities. The present study involved independent auditory and visual inputs, with subjects being asked to explicitly ignore one source of stimulation. However, whenever a single event gives
948
G. Rees et al. / Neuropsychologia 39 (2001) 937–949
rise to concurrent and correlated stimulation in different modalities to which the subject must attend (for example, as in the case of speech), cross modal integration is likely to be found [10].
8. Conclusion In conclusion, strong activation of visual cortex and a robust motion after effect can be produced by an irrelevant and ignored visual motion stimulus during concurrent auditory task performance, regardless of whether the relevant task is of high or low auditory load. These results stand in clear contrast to the modulation of visual motion processing by high 6isual load as indexed by the duration of MAE in the present study as well as activity of visual cortex in a previous study [35]. Our findings are consistent with viewing the major source of attentional restrictions on task performance as lying within modality-specific subsystems.
Acknowledgements This work was supported by the Wellcome Trust (GR, CF), Medical Research Council (UK) grant number G9805400 (NL), and the Human Frontiers Science Program (NL). We thank Richard Frackowiak for comments on the paper.
References [1] Alho K, et al. Intermodal selective attention. II. Effects of attentional load on processing of auditory and visual stimuli in central space. Electroencephalogr Clinical Neurophysiology 1992;82(5):356 – 68. [2] Allport DA, Antonis B, Reynolds P. On the division of attention: a disproof of the single channel hypothesis. The Quarterly Journal of Experimental Psychology 1972;24(2):225 – 35. [3] Arnell KM, Jolicoeur P. The attentional blink across stimulus modalities: evidence for central processing limitations. Journal of Experimental Psychology Human Perception and Performance 1999;25(3):630 – 48. [4] Broadbent DE. Perception and Communication. London: Pergamon, 1958. [5] Buchel C, et al. The functional anatomy of attention to visual motion. A functional MRI study. Brain 1998;121(Pt 7):1281 – 94. [6] Chan J, Spence C, Simm A. Crossmodal attention and the ‘perceptual load’ hypothesis. In: Experimental Psychology Society. UK: Cambridge, 2000. [7] Chawla D, Rees G, Friston KJ. The physiological basis of attentional modulation in extrastriate visual areas. Nature Neuroscience 1999;2(7):671 –6. [8] Corbetta M, et al. Selective and divided attention during visual discriminations of shape, color, and speed: functional anatomy by positron emission tomography. Journal of Neuroscience 1991;11(8):2383 – 402. [9] DeYoe EA, et al. Mapping striate and extrastriate visual areas in human cerebral cortex. Proceedings of the National Academy Sciences of the United States of America 1996;93(6):2382 – 6.
[10] Driver J, Spence C. Crossmodal attention. Current Opinion in Neurobiology 1998;8(2):245 – 53. [11] Duncan J. The locus of interference in the perception of simultaneous stimuli. Psychological Review 1980;87(3):272 – 300. [12] Duncan J, Martens S, Ward R. Restricted attentional capacity within but not between sensory modalities. Nature 1997;387(6635):808 – 10. [13] Dupont P, et al. The kinetic occipital region in human visual cortex. Cerebral Cortex 1997;7(3):283 – 92. [14] Eriksen B, Eriksen C. Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception and Psychophysics 1974;16:143 – 9. [15] Fiez JA, et al. PET studies of auditory and phonological processing: effects of stimulus characteristics and task demands. Journal of Cognitive Neuroscience 1995;7:357 – 75. [16] Friston KJ, et al. Spatial realignment and normalisation of images. Human Brain Mapping 1995;2:165 – 89. [17] Friston KJ, et al. Statistical parametric mapping in functional imaging: a general linear approach. Human Brain Mapping 1995;2:189 – 210. [18] Gandhi SP, Heeger DJ, Boynton GM. Spatial attention affects brain activity in human primary visual cortex. Proceedings of the National Academy of Sciences of the United States of America 1999;96(6):3314 – 9. [19] Hasnain MK, Fox PT, Woldorff MG. Intersubject variability of functional areas in the human visual cortex. Human Brain Mapping 1998;6(4):301 – 15. [20] Haxby JV, et al. The functional organization of human extrastriate cortex: a PET-rCBF study of selective attention to faces and locations. Journal of Neuroscience 1994;14(11 Pt1):6336 –53. [21] Husain M, Kennard C. Visual neglect associated with frontal lobe infarction. Journal of Neurology 1996;243(9):652 – 7. [22] Kahneman D, Treisman A. Changing views of attention and automaticity. In: Parasuraman R, Davies DR, editors. Varieties of Attention. London: Academic Press, 1984. [23] Lavie N. Perceptual load as a necessary condition for selective attention. Journal of Experimental Psychology Human Perception and Performance 1995;21(3):451 – 68. [24] Lavie N. The role of capacity limits in in selective attention: behavioural evidence and implications for neural activity. In: Braun J, Koch C, editors. Neural Circuits of Attention. Cambridge, MA: MIT Press, 2000. [25] Lavie N, Cox S. On the efficiency of visual selective attention: efficient visual search leads to inefficient distractor rejection. Psychological Science 1997;8(5):395 – 8. [26] Lavie N, Fox E. The role of perceptual load in negative priming. Journal of Experimental Psychology Human Perception and Performance 2000;26(3):1038 – 52. [27] Lavie N, Tsal Y. Perceptual load as a major determinant of the locus of selection in visual attention. Perception and Psychophysics 1994;56(2):183 – 97. [28] Manoach DS, et al. Prefrontal cortex fMRI signal changes are correlated with working memory load. NeuroReport 1997;8:545 – 9. [29] Martinez A, et al. Involvement of striate and extrastriate visual cortical areas in spatial attention. Nature Neuroscience 1999;2(4):364 – 9. [30] McAdams CJ, Maunsell JHR. Effects of attention on orientation-tuning functions of single neurons in macaque cortical area V4. Journal of Neuroscience 1999;19(1):431 – 41. [31] McLeod P. A dual task response modality effect: support for multiprocessor models of attention. The Quarterly Journal of Experimental Psychology 1977;29:651 – 67. [32] Moran J, Desimone R. Selective attention gates visual processing in the extrastriate cortex. Science 1985;229(4715):782 – 4. [33] O’Craven KM, et al. Voluntary attention modulates fMRI activity in human MT-MST. Neuron 1997;18(4):591 – 8.
G. Rees et al. / Neuropsychologia 39 (2001) 937–949 [34] Price CJ. The functional anatomy of word comprehension and production. Trends In Cognitive Sciences 1998;2(8):281 – 8. [35] Rees G, Frith CD, Lavie N. Modulating irrelevant motion perception by varying attentional load in an unrelated task. Science 1997;278:1616 – 9. [36] Shulman GL, et al. Top-down modulation of early sensory cortex. Cerebral Cortex 1997;7(3):193 –206. [37] Somers DC, et al. Functional MRI reveals spatially specific attentional modulation in human primary visual cortex. Proceedings of the National Academy of Sciences of the United States of America 1999;96(4):1663 –8. [38] Spitzer H, Desimone R, Moran J. Increased attention enhances both behavioral and neuronal performance. Science 1988;240(4850):338 –40. [39] Talairach J, Tournoux P. Co-Planar Stereotaxic Atlas of the Human Brain. New York: Georg Thieme Verlag, 1988. [40] Tipper SP. The negative priming effect: inhibitory priming by ignored objects. The Quarterly Journal of Experimental Psychology 1985;A37(4):571 –90.
949
[41] Tootell RB, et al. Visual motion aftereffect in human cortical area MT revealed by functional magnetic resonance imaging. Nature 1996;375(6527):139 – 41. [42] Tootell RB, et al. Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging. Journal of Neuroscience 1995;15(4):3215 – 30. [43] Treisman A, Geffen G. Selective attention: perception or response? The Quarterly Journal of Experimental Psychology 1967;19(1):1 – 17. [44] Treisman AM, Davies A. In: Kornblum S, editor. Attention and Performance IV. London: Academic, 1973:101 – 17. [45] Treue S, Trujillo CM. Feature-based attention influences motion processing gain in macaque visual cortex. Nature 1999;399:575 – 9. [46] Warburton E, et al. Noun and verb retrieval by normal subjects. Studies with PET. Brain 1996;119(Pt 1):159 – 79. [47] Watson JD, et al. Area V5 of the human brain: evidence from a combined study using positron emission tomography and magnetic resonance imaging. Cerebral Cortex 1993;3(2):79 –94.
.
