*Manuscript including abstract Click here to view linked References
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
Title: Task probability and report of feature information: what you know about what you „see‟ depends on what you expect to need
Michael Pilling & Angus Gellatly Oxford Brookes University
Corresponding author Dr. Michael Pilling, Department of Psychology, Oxford Brookes University, Headington Campus, Oxford, OX3 0BP United Kingdom Tel: +44 (0) 1865 483788; Fax: +44 (0) 1865 483887 Email:
[email protected]
Keywords: visual awareness, task probability, dimensional attention, set size PsychInfo codes: 2323, 2346 Word count
Author note Some results were initially reported at the European Society for Cognitive Psychology conference September 2011, San Sebastian, Spain.
1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
Abstract We investigated the influence of dimensional set on report of object feature information using an immediate memory probe task. Participants viewed displays containing up to 36 coloured geometric shapes which were presented for several hundred milliseconds before one item was abruptly occluded by a probe. A cue presented simultaneously with the probe instructed participants to report either about the colour or shape of the probe item. A dimensional set towards the colour or shape of the presented items was induced by manipulating task probability –the relative probability with which the two feature dimensions required report. This was done across two participant groups: One group was given trials where there was a higher report probability of colour, the other a higher report probability of shape. Two experiments showed that features were reported most accurately when they were of high task probability, though in both cases the effect was largely driven by the colour dimension. Importantly the task probability effect did not interact with display set size. This is interpreted as tentative evidence that this manipulation influences feature processing in a global manner and at a stage prior to visual short term memory.
Abstract word count=191
2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
1. Introduction A major role of attention is to prioritise information received by the senses in accordance with its behavioural significance (Broadbent, 1958). In vision, spatial location seems to be the principal basis on which this attentional prioritisation occurs (Broadbent, 1982; Posner, 1980; Tsal & Lavie, 1988; Lamy & Tsal, 2001). However we can also prioritise by selectively attending to particular feature dimensions, e.g. colour, or for items possessing certain values on these dimensions, e.g. red items, (see Rossi, & Paradiso, 1995; Kumada, 2001; Maunsel & Treue, 2006, Stojanoski & Niemeir, 2011; Zhang & Luck, 2009; Adams & Chambers, 2012). Attention, whether towards locations, feature dimensions or values, seems to be intimately linked with conscious visual experience. Indeed, it is argued that attention may be a necessary condition for an observer to demonstrate visual awareness, or at least for awareness of specific details about an object or scene (Posner, 1994; Rensink, O‟Regen & Clarke, 1997; O‟Regan, & Noe, 2001; Dehaene Changeux, Naccache, Sackur & Sergent, 2006; Hsieh, Colas & Kanwisher, 2011; though see Lamme, 2003; Koch & Tsuchiya, 2007). The role of attention in visual awareness has been assessed largely through investigation of two phenomena: change blindness (Rensink et al. 1997), the striking inability to detect large changes in visual scenes, and inattentional blindness, the failure under conditions of misdirected attention to notice the appearance of unexpected objects (Mack & Rock, 1998; Simons & Chabris, 1999). Both phenomena attest to the same principle: that we habitually fail to notice things that would be clearly visible if we knew what to look for and where (Jenson, Yao, Street & Simons, 2011). The change blindness phenomenon is most obviously demonstrated in the flicker paradigm (Rensink, et al., 1997; Rensink, 2000) in which an original version and a modified version of a display or picture (e.g. a jet aeroplane with a missing engine in one version) are presented in a cycling sequence interleaved with a blank screen. Despite actively searching for it, observers typically require several iterations of the cycle to notice changes in the scene. Inattentional blindness is demonstrated using a task in which participants are instructed to monitor a stimulus or set of stimuli (e.g. track some moving circles and count how frequently they cross the horizontal midline); on one critical trial, an unexpected object is introduced (e.g. a red cross moving along the horizontal midline), which participants typically fail subsequently to report even though it passed right in front of their eyes.
3
Attentional manipulations strongly determine the extent to which changes or 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
unexpected events are noticed. Much of the work looking at the role of attentional allocation in change blindness and inattentional blindness has concentrated on spatial attentional manipulations (e.g. Rensink et al., 1997; Tse, 2004; Newby & Rock, 1998; Most, Simons, Scholl, & Chabris, 2000; Smith, & Schenk, 2008; Umemoto, Scolari, Vogel & Awh, 2010). However a body of work has also performed experimental manipulations which influence how observers allocate attention across different stimulus feature values and dimensions. In inattentional blindness, the frequency with which unexpected objects are noticed has been shown to vary according to the type of visual information to which an observer is instructed to attend; unexpected objects are most likely to be noticed by observers when they possess feature values consistent with the attended features of the primary task (Mack & Rock, 1998; Simons & Chabris, 1999; Most, Scholl, Clifford and Simons, 2005; Most & Astur, 2007). For instance, Most et al. (2005) found that participants viewing a display consisting of moving black and white circles and squares were far more likely to notice the appearance of an unexpected grey circle when they had been instructed to monitor circular objects than when instructed to monitor squares. It seems that when participants had an attentional set for „circles‟ this augmented the perceptual salience of the grey circle to a point to which it crossed the awareness threshold (Most et al., 2005). Comparable effects of attentional set are found in the change blindness literature. Unlike in inattentional blindness, here the observer‟s attention is typically directed towards an entire feature dimension, rather than a particular feature value. It is with such dimensional effects that the current paper is primarily concerned. The perceptual consequences of manipulations of dimensional attentional set (sometimes referred to as dimensional weighting, e.g. Krummenacher & Müller, 2012) are analogous to those described for manipulating attention towards particular feature values. In the change blindness paradigm it has been repeatedly demonstrated that changes are most likely to be noticed when occurring on the stimulus dimension coinciding with the dimensional set of the observer (Aginsky & Tarr, 2000; Triesch, Ballard, Hayhoe, & Sullivan, 2003; Droll, Hayhoe, Triesch, & Sullivan, 2005; Austen & Enns, 2000; Austen & Enns, 2003; Beck, Angelone, Levin, Peterson, & Varakin, 2008; Wegener, Ehn, Aurich, Galashan & Kreiter, 2008; van Lamsweerde & Beck, 2011). For example, if in the flicker task participants are cued that a change will involve an item‟s colour, they are more efficient in spotting such changes against a baseline where either no cue or invalid cue information is given prior to the trial (Aginsky & Tarr, 2000; Wegener et al. 2008). 4
Similar effects in change blindness can also be found by inducing a dimensional set in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
a less directed manner, by varying the demands of a primary task given concurrently with the change detection task (Triesch et al., 2003; Droll et al., 2005), or by manipulating the probabilities with which different sorts of change occur across experimental trials (Austen & Enns, 2000; Austen & Enns, 2003; Beck, et al., 2008; van Lamsweerde & Beck, 2011). Using the flicker paradigm Austen & Enns (2000, 2003), induced a dimensional set by having certain changes occur with a higher probability in an experiment, the type of higher probability change being varied across participant groups. Changes tended to be detected most efficiently when they were consistent with the dimensional set presumably induced by the trial probability manipulation. For instance, with face stimuli participants who mostly received trials in which the change occurred in the identity of a face stimulus were more efficient at detecting identity changes than they were at detecting changes in facial expression. Participants experiencing mostly changes in facial expression displayed an effectively converse pattern of results. Similar change probability effects have also been reported for simple visual objects such as coloured geometric shapes (Beck, et al., 2008; van Lamsweerde & Beck, 2011). Van Lamsweerde & Beck (2011) used a one-shot change detection task in which displays consisting of several such shapes were presented interleaved by a blank field; the two displays were the same apart from a change either to the colour, shape or location of one of the items. An attentional set was induced by giving participants an initial block of trials in which one type of change was more frequent than the other two. For three groups of participants either colour, shape or location changes occurred most frequently. Change detection was then measured in a subsequent trial block in which each type of change occurred equally often. Change detection was measured across all trials by the ability to identify the changed object in the post-change display. Results indicated that change detection was governed largely by participants‟ experiences of change probability in the preceding trial block, participants generally being more accurate in reporting the type of change presented most frequently in this initial block1. Thus, findings from both the change blindness and inattentional blindness literatures are consistent with attentional set influencing what we know about what our eyes tell us. How might such influence occur? Where attention is being directed to a single feature value it is 1
The authors report effects for both colour and shape dimensions but not for location. They argue that location may have a special role as a stimulus dimension and may be a default component of object representations irrespective of shifts in feature-based attention.
5
possible that spatial attention may play a mediating role in determining performance. Such 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
claims have been made in the context of findings from visual search (Shih & Sperling, 1996; Moore & Egeth, 1998; cf. Anderson, Müller & Hillyard, 2008), the argument being that cueing a feature value serves to direct attention to locations containing that particular feature. A similar mechanism could account for some of the earlier described findings. For instance in the work of Most et al. (2005) on inattentional blindness, inducing an attentional set for „square‟ items might influence the distribution of spatial attention in a way that it becomes biased towards locations where objects possessing this feature are currently located. Were this the case the sudden appearance of an unexpected item which also possessed this feature value would increase the likelihood that such an item received some spatial attention and, consequently, increase the probability of it being consciously perceived. Even where task conditions direct an observer towards an entire feature dimension spatial attention could sometimes play a mediating role. Indeed, Austen & Enns (2003) interpret their effect as a consequence of differential spatial filtering of the stimuli used. Feature dimensions such as face identity and face emotion are often prominent at different scales of spatial frequency (e.g. Schyns, Bonnar & Gosselin, 2002); it is thus plausible that when task conditions emphasise a particular feature of these stimuli that there is a corresponding change in the scale at which spatial attention operates leading to differences in change detection performance. However other findings cannot have arisen via spatial attention, however conceived; for instance, in the earlier described study of van Lamsweerde & Beck (2001) the manipulated feature dimensions (colour and shape) were not ones which can be selected by spatial filtering. How else could such effects of dimensional set manipulation arise? Van Lamsweerde & Beck (2011) give two possible interpretations of their results, while also noting that their data could not distinguish between them. One possibility they offer is that dimensional set influences the sort of information about object features which is selectively encoded in visual short term memory (VSTM). A further possibility is that dimensional set influences not feature representation itself but the order in which feature information is compared across the pre- and post-change representations: a change being more likely to be detected if it is present in the initial comparison process because the representation will be less subject to decay. To summarise, there is clear evidence that dimensional set influences the accuracy of change detection. What is less clear is the basis of these effects and, more specifically, whether the visual representations themselves are influenced. Two experiments explore this 6
question further by determining what influence dimensional set has on what observers can 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
explicitly report about the features of objects in a display they have viewed. Both experiments utilise an immediate memory probe paradigm (Wolfe, Reinecke & Brawn, 2006; Reinecke, Rinck & Becker, 2006; Pilling & Gellatly, 2011). This paradigm is particularly suited to our research question because it requires only a simple perceptual judgement on the part of the observer, who is merely being required to report a feature of a single probed item immediately after viewing it (rather than perform a visual comparison across two sequentially presented displays as required by the more ubiquitous change detection paradigm). In the standard form of the immediate memory paradigm, a display containing a variable number of items (e.g. coloured shapes) is presented and viewed for several hundred milliseconds before one item is abruptly covered by the probe, participants then making an unspeeded decision about an aspect of the probe item e.g. was it a red or green disk, was it a left or right tilted bar? Despite the long viewing time, the probe‟s immediacy and the absence of spatial uncertainty about the required perceptual decision, participants can be surprisingly poor on this task (Wolfe et al., 2006), performance decreasing proportionally with the number of display items (Pilling & Gellatly, 2011). In the present experiments, dimensional set towards one of two feature dimensions (colour, shape) is manipulated by varying task probability: Individual trials require report of either colour or shape of the probed item, the report dimension being indicated only when the probe appears (see Figure 1). Task probability is varied across two participant groups; one group is given trials with a high task probability of colour, the other a high task probability of shape. The expectation is that the two groups will differ in their dimensional set as a consequence of this manipulation. If the manipulation is successful in influencing what observers know about the items when probed then we should find systematic differences in report accuracy between the groups. Specifically groups should be most accurate in reporting about the feature which is concordant with their induced set. The experiments also have a second related purpose, which is to determine the global extent of any putative effects of attentional set. There are two ways in which attentional set could influence of the processing of feature information in vision. One is by influencing only a limited number of spatially attended display items; a second is by influencing all display items in a global manner. According to the first possibility, participants spatially select a small subset of (around four) display items for representation in VSTM (e.g. Luck & Vogel, 1997; Wolfe et al., 2006; Zhang & Luck, 2008) with dimensional set influencing only what feature information is represented of these selected items. If this is the case, then as set size 7
increases the proportion of items influenced by dimensional set will diminish. The resulting 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
data pattern will therefore show any difference in feature report to be greatest with small display set sizes and diminish as set size is increased. According to the second possibility, attentional set is not restricted to a small number of items, but instead is applied to all items in the display. Consistent with this possibility, there is substantial behavioural and physiological evidence that feature-based attention tends to operate in a uniform manner across the visual field (e.g. Sàenz, Buraĉas & Bonton, 2003; Ling, Liu & Carrasco, 2009; Liu & Hu, 2011; White & Carrasco, 2011). If such global effects occur then a different pattern of data would be expected in our experiments. Specifically, any difference in feature report should be of constant magnitude irrespective of set size. We report two experiments which manipulate task probability while varying display set size with the aim of resolving the above issues. These experiments should provide insight into the relationship between dimensional set and what we know about viewed objects.
2. Experiment 1 In Experiment 1 participants were required to report the colour or the shape of one probed item in a display containing a variable number of items. Individual display items each had of one of three possible values on each of the two feature dimensions of colour and shape. When the occluding probe appeared participants reported, by means of a key press, what they thought was the feature value of the probe item. The probability of report of the two feature dimensions (colour, shape) was varied across two participant groups.
2.1. Method 2.1.1. Participants. Thirty-eight participants, all normal or corrected-to-normal in visual acuity and with normal colour vision, performed the task. Participants were drawn from the undergraduate and postgraduate population of Oxford Brookes University; some received course credit for participation. Half were allocated to the high colour task probability group and half to the high shape task probability group. 2.1.2. Stimuli and equipment. Stimuli were presented on a 19” CRT monitor viewed at an approximate distance of 120 cm. The monitor was controlled by a computer running software purpose written in Microsoft Visual Basic® (V. 6.0). This performed the appropriate stimulus 8
and task randomisations, with Microsoft DirectX® routines utilised for rendering the 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
graphical stimuli. These stimuli consisted of circles (0.7° diameter), equilateral triangles (0.7° base) or plus signs (0.7° in height and width). Each shape was either red (CIE x, y chromaticity coordinates of .456/.347; luminance = 9.30 c/m2), blue (.194/.230; 9.42 c/m2) or green (.260/.443; 9.72 c/m2). These were presented on a neutral grey background area (8.65 c/m2) which subtending an angle of 13°×15°) which was surrounded by a black (0.22 c/m2) border that filled the rest of the screen. Stimuli were arranged on the grey background in a pseudo-random fashion with the constraint that items were always a minimum distance of 0.5 ° apart. The probe consisted of a black square (1°) that was large enough to fully occlude any of the items. 2.1.3. Procedure. Each trial began with a blank screen followed by presentation of the stimulus display, which was shown for a randomised time between 1000 ms-1500 ms, after which the probe appeared and covered the item for report. There were always 9, 18, or 36 items in each display; this set size manipulation was varied pseudorandomly across trials. Simultaneous with the occluding square‟s appearance a high- or low- pitched cue tone was presented that indicated the attribute to be reported (high = report colour; low = report shape). On colour trials participants had to press either the “I”, “O” or “P” key on a standard computer keyboard to report the colour as red, green or blue; on shape trials they had to press the “X”, “C” or “V” key to report the shape as circle, triangle or cross (the „shape keys were disabled on shape trials and the colour keys disabled on colour trials). Keys were appropriately relabelled for the experiment. Participants were instructed to make an unspeeded response and to ensure that they press their intended key. Responses initiated the next trial after a 500 ms interval. An error tone was sounded when an incorrect response was given. A total of 350 trials were given in each session. In the experiment, the proportion of report colour and report shape trials depended on the participant group allocation. One group (high colour probability group) had a ratio of 7:3 colour-to-shape trials, for the other group (high shape probability group) this ratio was 3:7. Before starting the experimental trials participants first performed some demonstration trials. They were then given an initial practice block of thirty trials. In this practice block an equal frequency of colour and shape trials was given irrespective of group. The purpose of the practice trials was to ensure that all participants were familiar with the colour and shape tasks and their associated responses before commencing the experiment.
9
***Insert Figure 1 about here*** 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
2.2. Results Mean percentage correct responses for the two tasks are given in Table 1 for the two groups. One sample t-tests showed that performance was above chance for both tasks for all set size combinations (min. t=5.39, p
