Change Blindness and Its Determinants

MASARYK UNIVERSITY Faculty of Social Studies Department of Psychology

Change Blindness and Its Determinants The Role of High Level Scene Factors, Emotions, and Personality in Change Detection

Dissertation Mgr. Michaela Porubanová

Advisor: Mrg. Radovan Šikl, Ph.D. Brno 2013

Hereby I declare that I have written this dissertation independently and that all cited resources have been listed in the references.

June 1st, 2013

Signature ..........................

2

Personal Acknowledgements I would like to thank my supervisor, Mgr. Radovan Šikl, Ph.D., for reviews and suggestions during the course of writing the dissertation. Secondly, Daniel Shaw for helping me with the methodological part of my second study (script writing). Most of all, I would like to thank my family, Jeffrey Alan, and all my friends who have supported me and believed in me greatly. I would also like to thank Prof. Arien Mack, who welcomed me to her lab at the New School for Social Research during my last semester of PHD studies and provided me a supportive environment for writing my dissertation. Lastly, I would like to thank Prof. Ivo Čermák, who has been a very influential figure in my academic career, both personally, and as a mentor.

Chapter Acknowledgements

Chapter 6 (Experimental Study I) is a chapter that is being published as an article in peerreviewed journal: Porubanová, M. & Šikl, R. (in press). What is our attention blind to? Studia Psychologica; and was published as a part of conference proceedings: Porubanova, M. & Sikl, R. (2010). The role of various categories of changes in the inducement of change blindness. Perception 39 ECVP Abstract.Supplement.

Chapter 3 (Scene Perception) was written based on conference proceedings: Porubanova, M. (2010). Scene perception. Sborník prací z konference Kognice 2010.

Chapter 7 (Experimental Study II) is a chapter that is based on a manuscript in preparation for publishing in a journal focused on cognition and emotion.

The chapters have been written and used with the consent of co-authors.

3

The faculty of voluntarily bringing back a wandering attention, over and over again, is the very root of judgment, character, and will. No one is compos sui if he have it not. An education which should improve this faculty would be the education par excellence. But it is easier to define this ideal than to give practical instructions for bringing it about. ~William James, Principles of Psychology The theory of the body is already a theory of perception. ~M. Merleau-Ponty Not only are we are blind to many aspects of our personal visual world, we are also surprisingly unaware of this fact. ~ S. Blackmore 4

Abstract Every moment, we perceive visual information in the environment, and regardless of the complexity of the information, we have a very powerful impression that the world is there, at our disposal, and the way we visually represent it is veridical to its actual image. However, the discovery of the phenomenon of change blindness, or an extreme difficulty in detecting changes in a visual field when a disruption in visual continuity occurs, has undermined the notion that visual representations are intricate, and suggests that memory for visual scenes might be very poor (O’Regan, 1992; O’Regan & Noë, 2001; Rensink, 2000). Change blindness has since become one of the most well-known and well-studied phenomenon in the study of visual attention and visual working memory, and has provided powerful methods to study visual attention in general. This dissertation can be split into two different parts. The first part deals with understanding the role of high-level factors in scene perception using the change blindness (flicker) paradigm. The second part focuses on the role of the emotional context of scenes and personality traits on change blindness performance. In the first study, we were interested in understanding whether there are high-level scene factors that influence change detection. Our experiment was motivated by several facts. Firstly, scene perception research shows that the human visual system has a tendency to attend predominantly to informative and important parts of a scene (Buswell, 1935; Henderson, 2003; Loftus & Mackworth, 1978; Yarbus, 1967), and that in active tasks such as change detection, attending to information with high visual saliency (i.e. spatial frequency, color, and intensity) is inhibited (Henderson, 2003; Henderson & Hollingworth, 2003; Turano, Geruschat, & Baker, 2003). Therefore attention is rather dependent on high-level factors (Beck et al., 2004; Hollingsworth & Henderson, 2000; Rensink et al., 1997). Taken together, the goal of the first part of the dissertation was to explore what high-level scene information is considered by the human visual system informative and important. In the experiment, observers inspected various sets of two real-world scene images which featured a changed aspect. The images were shown alternating in sequential order (i.e., the flicker task) and the observers were asked to click on the location where the change had 5

occurred. The results demonstrated that not all changes present in a scene have the same likelihood to be detected. The changes that seem to be dominant in attracting attention were found to be changes occurring in the central region, context-related changes, probable changes, and changes occurring within a figure. Our results are consistent with results of other studies. For instance, research on scene perception has shown that changes in the central region are detected faster than those in the marginal (Rensink et al., 1997), changes inconsistent with the scene context are detected faster than those consistent with the scene context (Hollingsworth & Henderson, 2000), probable changes faster than improbable ones (Beck et al., 2004), changes in a figure are detected to a greater extent than ones in the background (Mazza et al., 2005; Turatto et al., 2002). Interestingly, the detection of changes occurring within close proximity to a figure was most difficult. This indicates that when searching for changes in scenes, parts of the scenes close to the most powerful attractors are being shadowed, and therefore seem to be ignored by selective attention. This could be ascribed to the role of expectations in change detection tasks. Therefore, we believe that specifically in explicit change detection tasks, an individual might use a certain heuristic that helps her scan the scene. The study discusses the significance of the categories under which a stimulus falls in terms of its non-visual or semantic properties. The goal of the second study focused specifically on three areas of the effect of emotion on attention: 1. the role of emotional context in change detection; 2. the role of emotional states in change detection; and 3. the role of time-invariant personality traits in change detection. In the study, standardized and validated images from IAPS (International Affective Picture System, Lang, Bradley, & Cuthbert, 2005) were used in order to examine the role of emotional context (created by real-world scenes with various degrees of valence and arousal) on the ability to detect changes within those images. It was believed that change detection would be affected by the emotional significance of the scenes via two mechanisms. The first mechanism is the modification of attentional scope (FOV), which would be greater for positive than negative images, allowing more relevant information to be processed simultaneously and resulting in faster change detection performance. For instance, research found a correlation between FOV and reaction time in an intentional change detection tasks (such as flicker) (Pringle, Irwin, Kramer, & Atchley 2001). We found the greatest accuracy of change detection for changes occurring in positive and negative scenes (compared with neutral scenes). Secondly and most importantly, changes in 6

negative scenes (and also positive scenes, though not with statistical significance) were detected faster than changes in neutral scenes. The results show that emotional significance of a scene does have an effect on spatial attention. Quick and successful change detection in flicker tasks requires spatial attention to be widened and maintenance of the information selected by attention from the particular scene area. Much research has provided evidence for the prioritized processing of emotional stimuli. Additionally, it is clear from our study that the emotional significance of visual scenes has a differential effect on attention. Similarly, Anderson & Phelps (2001) showed that attentional blink for the second target identification is alleviated when the second target represents an emotionally significant stimulus (in that case, an emotional word). Most et al. (2005), using the RSVP paradigm (rapid serial visual presentation), demonstrated that presenting an emotional stimulus prior to the onset of a target hinders target detection. Clearly, emotional stimuli not only enjoy prioritized processing but utilize attention resources and create interference when tasks require a brief presentation of neutral stimuli. Secondly, we looked at the role of individuals’ affective states in change blindness. Many studies suggest that positive emotional states, when compared with neutral or negative states, increase the field of view and reduce attentional biases in iconic memory tasks (Kuhnbandner et al., 2011), flanker tasks (Rowe et al., 2007), or Navon’s global-local processing task (Gasper & Clore, 2002). In the current study, however, no experimental manipulation of emotional states took place; rather each participant assessed her own emotional state prior to the beginning of change detection task. Based on the results, overall emotional states did not correlate with the change detection latency performance to a great extent. Only ‘feeling nervous’ positively correlated with the overall change detection performance. ‘Feeling alert’ negatively correlated with change detection performance, but only for neutral images. The results support the hypothesis that negative emotions (in our case nervousness) impaired performance in attentional task. Secondly, positive emotions, such as feeling alert, improved reaction times. In the part concerning the role of time-invariant personality traits in change detection, we used Cloninger’s Temperament and Character Inventory to assess participants’ personality traits. The relationship between Harm Avoidance, a temperament dimension that refers to sensitivity toward aversive stimuli and can be perceived as one’s trait anxiety, in relation to the performance in attentional task was especially of interest to the current study.

7

The results show that Harm Avoidance in particular is associated with responses to emotional stimuli processing. The association is, interestingly, to positive, neutral, as well as with the overall change detection performance, but not with negative stimuli processing. Interestingly, Empathy has been negatively correlated with the performance in all emotion categories, indicating that greater scores in Empathy lead to shorter detection times. The overall results indicate that visual attention and its allocation is navigated via different external (high-level scene factors, or emotional information), or internal (emotional states, or personality traits) factors, and the reason of their role in influencing visual attention are discussed in the dissertation.

8

Table of Contents Abstract...................................................................................................................... .................... 5 Chapter 1 Introduction .................................................................................................................... 12 1.1 Motivation and Problem Statement .......................................................................................... 12 1.2 Thesis Outline .......................................................................................................................... 14

Chapter 2 Change Blindness................................................................................................... 15 2.1 Introduction ............................................................................................................................. 15 2.2 Change Blindness and the Nature of Visual Representations .................................................... 16 2.3 Change Blindness and Visual Attention .................................................................................... 18 2.4 Summary.................................................................................................................................. 19 References ..................................................................................................................................... 21

Chapter 3 Scene Perception .................................................................................................... 25 3.1 What is Scene Perception? ................................................................................................. 25 3.2 Eye Movements and Scene Perception ................................................................................. 27 3.3 High-Level Scene Perception............................................................................................... 30 3.4 Theories of Attentional Guidance in Scene Perception ......................................................... 33 3.4.1 Coherence Theory ................................................................................................ 34 3.4.2 Visual Memory Theory of Scene Perception ........................................................ 35 3.5 The Nature of Scene Representations ................................................................................... 37 3.6 Summary ............................................................................................................................. 39 References ............................................................................................................................... 40

Chapter 4 Emotion and Attention ......................................................................................... 45 4.1 Introduction ........................................................................................................................ 45 4.2 Interplay between Attention and Emotion ............................................................................ 46 4.2.1 Emotional Guidance of Attentional Allocation ..................................................... 49 4.3 Emotional Memory for Scenes ............................................................................................. 51 4.3.1 The Arousal Model of Memory Narrowing ......................................................... 52 4.3.2 The Valence Model of Memory Narrowing .......................................................... 53 4.4 Interindividual Differences and Attention Allocation .......................................................... 54 4.5 Summary ............................................................................................................................. 56 References ............................................................................................................................... 57

9

Chapter 5 Introduction to the Experimental Part ...................................................................... 61 5.1 Research Identification........................................................................................................ 61 5.2 Research Objectives: Experimental Study I.......................................................................... 61 5.3 Research Objectives: Experimental Study II ........................................................................ 62

Chapter 6 Experimental Study I .................................................................................................... 63 6.1 Introduction: Research Rationale ........................................................................................ 63 6.2 Method ............................................................................................................................... 67 6.2.1 Participants .......................................................................................................... 67 6.2.2 Procedure ............................................................................................................ 67 6.2.3 Stimuli ................................................................................................................. 69 6.3 Results ................................................................................................................................ 69 6.3.1 Percentage of Detected Trials............................................................................... 69 6.3.1.1 Descriptive Statistics ............................................................................ 69 6.3.1.2 Change Detection Accuracy.................................................................. 70 6.4 Discussion and Conclusion.................................................................................................. 72 6.5 Summary ............................................................................................................................. 74 References ................................................................................................................................ 76

Chapter 7 Experimental Study II ................................................................................................. 80 7.1 Introduction: Research Rationale ........................................................................................ 80 7.2 Method ............................................................................................................................... 84 7.2.1 Participants .......................................................................................................... 84 7.2.2 Procedure ............................................................................................................ 84 7.2.3 Measures ............................................................................................................. 85 7.2.3.1 Change Detection Method ................................................................... 85 7.2.3.2 Affective States Measures..................................................................... 86 7.2.3.3 Personality Measures ............................................................................ 87

7.2.4 Stimuli ................................................................................................................. 88 7.3 Results ................................................................................................................................ 91 7.3.1 Change Detection ................................................................................................ 91 7.3.1.1 Descriptive Statistics ............................................................................ 91 7.3.1.2 Change Detection Accuracy.................................................................. 92 7.3.1.3 Change Detection Latency .................................................................... 93 10

7.3.2 Change Detection and Emotional States ............................................................... 95 7.3.3 Change Detection and Personality Determinants .................................................. 97 7.4 Discussion and Conclusion.................................................................................................. 99 7.4.1 Brief Recapitulation of the Study ......................................................................... 99 7.4.2 Attentional Processing of Emotional Scenes ....................................................... 100 7.4.2.1 Discussion ......................................................................................... 100 7.4.2.2 Gender Differences............................................................................. 102 7.4.2.3 Limitations ......................................................................................... 102 7.4.3 The Role of Emotional States in Change Detection ............................................ 103 7.4.3.1 Discussion ......................................................................................... 103 7.4.3.2 Limitations ......................................................................................... 104 7.4.4 The Role of Affective Predispositions in Change Detection Involving Emotional Scenes ........................................................................................................................ 105 7.4.4.1 Discussion ......................................................................................... 105 7.4.4.2 Limitations ......................................................................................... 106 7.5 Summary ........................................................................................................................... 107 References .............................................................................................................................. 108

11

Chapter 1: Introduction 1.1 Motivation and Problem Statement

My master thesis was focused on exploring the link between working memory and change detection, specifically whether working memory is utilized differently in incidental, i.e., task-irrelevant, versus intentional, i.e., task-specific change detection. Following my passion and fascination for using this powerful paradigm to understand the nature of attention, consciousness, memory and interaction between them, this dissertation is devoted to my further research endeavor on this topic. The phenomenon of change blindness has been a greatly debated topic in the study of visual attention, visual consciousness, and their mutual interaction. Change blindness, the profoundly striking inability to detect changes in our visual environment, provides further insights into the nature of visual representations, prioritization of representing certain visual information over other, and vulnerability of visual information to decay. A seminal paper on change blindness by Rensink and his colleagues in 1997 introduced the flicker paradigm, a task in which two images alternate while being separated by a brief blank screen and quite frequently even substantial changes are difficult to notice. This paradigm is used vastly nowadays in investigating allocation of attention, preferential processing of various stimuli, and thus determining stimulus saliency. It is well known that many different factors, both low-level and high-level, make a contribution to guiding attention. The role of low-level factors has been explored greatly in the research of visual attention, while the role of high-level factors did not enjoy such an ardent interest. The overarching goal of this dissertation was to explore how three different high-level determinants modulate guidance of visual attention in scene perception; specifically, high-level scene factors, emotion, and personality. The dissertation employs the most popular paradigm used in the study of change blindness, i.e., flicker task, to understand the nature of scene perception in respect of those 12

factors. The dissertation is organized into two different parts. The first part deals with understanding of the role of high-level factors, while the second part focuses on the role of emotional context of scenes, and personality traits on change blindness performance. The first part investigated the role of various semantic or high-level stimulus attributes in the study of scene perception. Many studies have studied the role of low-level stimulus properties in attentional guidance. However, it is important to understand how knowledge acquired by visual system through everyday encounter with particular types of stimuli, or scene knowledge, influences processing of new stimuli. Therefore, in the first study, we were interested in understanding the contribution of those high-level scene factors to change detection. The changes in our study could fall under the categories of: probability, relevance to context, localization within figure versus ground, or localization within central or marginal scene regions. Thus, the goal of the first part of the dissertation was to identify what high-level scene information is considered by human visual system informative and important, and therefore resulting in faster change detection. Secondly, another area of high-level factors of visual attention was explored. Traditionally, visual attention was perceived as a unitary concept, unaffected by such factors as emotions or personality traits. Recent research pointed out the prioritization of emotional information by attention, as well as attentional modulation by personality traits. The goal of the second study focused specifically on those areas of the effect of emotion and personality on attention: a, the role of emotional context in change detection; b, the role of emotional states in change detection; and c, the role of time-invariant personality traits in change detection. Real-world scenes with various degrees of valence and arousal were used in flicker paradigm in order to examine the influence of emotional factors on scene perception. Positive and Negative Affect Schedule (PANAS) was included in order to evaluate initial level of one’s emotions, and Cloninger’s Temperament and Character Inventory (TCI) was used to assess individuals‘ personality traits. The overall results indicate that visual attention and its allocation is navigated via different external (high-level scene factors, or emotional information), or internal (emotional states, or personality traits) factors, and the reason of their role in influencing visual attention are discussed in the dissertation.

13

1.2 Thesis Outline

The thesis is organized into 7 chapters. Chapters 1 to 4 introduce theoretical background for the research problems outlines in experimental chapters 5 to 7.

Chapter 1 introduces the basic research problem and questions that were the objectives of the dissertation. Chapter 2 introduces the phenomenon of change blindness. It is mainly involved in outlining how changes blindness occurs. The Chapter also included research on scene perception, as it is essential for understanding change blindness, because change blindness uses almost absolutely scenes from real-world. Chapter 3 provides important insights into mechanisms underlying scene perception, the role of eye movements in attention allocation, as well as empirical evidence concerning high-level scene perception. The chapter provides theoretical background for the Experimental Study I (Chapter 6). Chapter 4 presents theoretical, as well as empirical grounds for the Experimental Study II (Chapter 7). The research concerning interaction between attention and emotion, and attention and personality traits is discussed. Chapter 5 addresses the motivation for the two research studies. Chapter 6 introduces the Experimental Study I that examines the role of high-level scene factors in attention allocation through paradigms of change blindness. Chapter 7 or the Experimental Study II, explores the impact of emotions (both emotional context of scenes and participants‘ emotional states) and personality on change detection performance.

14

Chapter 2: Change Blindness When we perceive visual information in the environment, regardless of the complexity of the information, we have a very powerful impression that the world is out there, at our disposal, and the way we visually represent it is veridical to its actual image. However, ever since its discovery, the phenomenon of change blindness has undermined the notion that visual representations are intricate, and has suggested that scene memory might be rather poor (O’Regan, 1992; O’Regan & Noë, 2001a; Rensink, 2000). Change blindness has ever since become one of the most well-known and well-studied phenomenon in the study of visual attention and visual working memory, and provided powerful methods to study visual attention in general.

2.1 Introduction

Change blindness is the inability to consciously perceive a changing stimulus in the visual environment, and subsequently not being able to report it. Even visual changes of a great magnitude can go undetected. Such changes can include abrupt and robust changes ranging from a conversation partner changed to another person (Levin & Simons, 1997; Simons & Levin, 1998), disappearance of large background objects (Rensink, O’Regan, & Clark, 1997), the exchange of the heads of two cowboys (Grimes, 1996), or the enlargement of an entire scene (Blackmore, Brelstaff, Nelson, & Troscianko, 1995), to name a few. The failure to detect such dramatic changes illustrated the importance of visual attention to conscious perception. In change blindness experiments, the important role of focused attention in change detection is manifested by an elimination of the motion transient that accompanies a change under normal circumstances. Once a brief transient is introduced to a scene, creating a disruption in visual continuity, change blindness occurs. Those disruptions that increase the

15

likelihood of the occurrence of change blindness were represented a blank screen introduced between two scenes (a paradigm dubbed the flicker technique) (Rensink et al., 1997), eye movements (Grimes,1996; Hollingworth & Henderson, 2002; Hollingworth, Williams, & Henderson, 2001; McConkie & Currie, 1996), color or luminance transients (Arrington, Levin, & Varakin, 2006), mudsplashes (i.e., “blobs” superimposed on the image; O’Regan, Rensink, & Clark, 1999), motion cuts (Levin & Simons, 1997; Simons & Levin, 1998), or eye blink (Bridgeman, Hendry, & Stark, 1975; O’Regan, Deubel, Clark, & Rensink, 2000). Interestingly, change blindness can arise in the absence of a disruption, when a change is gradual, without accompanying motion signals (Simons, Franconeri, & Reimer, 2000). In the most popular approach to study change blindness, the flicker task, an original and a modified image are presented in successive alternations, while a black screen is interjected between them resulting in a disruption in visual continuity. Once a brief blank is interposed between those two alternating images, it results in change blindness, even though the change is large and under normal conditions (without a blank slide) would be easily detectable (Noë, Pessoa, & Thompson, 2000). Interestingly, change blindness occurs in other sensory modalities, such as in olfaction (Sela & Sobel, 2010), auditory (Eramudugolla, Irvine, McAnally, Martin, & Mattingley, 2005; Gregg & Samuel, 2008; Vitevitch, 2003), or tactile perception (Auvray, Gallace, HartcherO’Brien, Tan, & Spence, 2008; Gallace, Tan, & Spence, 2006).

2.2 Change Blindness and the Nature of Visual Representations

Change blindness has served as an indispensable tool in the study of the mechanisms important for the creation of visual representations. Working with a disruption of visual continuity, postulations about the extent and complexity of visual representations created during each fixation were made. Specifically, Noë (2002) proposed the idea that the impression of an intricate visual world might be just an illusion, suggesting the nature of perceptual consciousness (i.e., visual working memory) is of an enactive or sensorimotor nature (Noë, 2002, 2005; Noë, Pessoa, & Thompson, 2000; O’Regan, 1992; O’Regan & Noë, 2001a, b). This account 16

emphasized the close coupling of perception and action following a Gibsonian tradition that deemed visual representations as being unnecessary for perception (Gibson, 1966; Noë, 2005), or only action-relevant information as needing representation (Bridgeman & Tseng, 2011; Tseng & Bridgeman, 2011). The nearby-hand effect corroborates this account by illustrating the prioritized processing of visual information that is located in a close proximity of the dominant hand (see Tseng, Bridgeman, & Juan, 2012 for a review). However, this account has been widely criticized by empirical evidence demonstrating the memory’s capacity for storing enormous amounts of information with great detail (Brady, Konkle, Alvarez, & Oliva, 2009; Landman, Spekreijse, & Lamme, 2003), and that change blindness does not necessarily occur due to the absence of visual representation (Wan, Ambinder, & Simons, 2009), but rather due to a failure in the comparison of the pre- and postchange visual representations (Simons, 2000; Simons & Rensink, 2005), especially in the absence of attention (Dretske, 2004; Lamme, 2003; Landman et al., 2003). As encoding, retention, and comparison of the pre- and post-change information are necessary precursors of successful change detection, a rupture in any of the mechanisms can essentially result in change blindness (Wan et al., 2009). Given a sufficient time for encoding, visual memory representations can be very intricate even in the face of distractors (Brady et al., 2009; Konkle, Brady, Oliva, & Alvarez 2010). It is important to mention that although change detection implies ‘detecting a change', in the majority of change blindness experiments participants are prompted to not only report on whether a change occurred (change detection), but also to identify and localize the change (Beck & Levin, 2003). Turatto & Bridgeman (2005) showed that change identification is almost immaculate for high-priority stimuli, but very inaccurate for stimuli of marginal interest. Presumably, the identity of attended visual information (most likely that of central interest) is retained through a transfer to visual working memory and therefore is available for comparison with upcoming visual stimuli (Beck & Levin, 2003; Turrato & Bridgeman, 2005).

17

2.3 Change Blindness and Visual Attention

Change blindness has provided insights into the nature of visual representations, visual awareness, and visual memory; contributions of attention and memory to visual awareness; and insights into how the visual system operates at large (Jensen, Yao, Street, & Simons, 2011; Wan et al., 2009). Focused attention is a necessary precursor to successful change detection (Rensink et al., 1997; Rensink, 2005), but attending to does not alone lead to detection of a change (Levin & Simons, 1997; Simons & Levin, 1998; Smith, Lamont, & Henderson, 2012). Instead, five steps are necessary for the ability to detect a change (Jensen et al., 2011; Simons, 2000): fixating the change location; encoding the information at the fixation location before the change has occurred; encoding the information at the fixation location after the change has occurred; comparing those representations; and consciously realizing the difference between the two representations. However, this process can be modulated by other factors that can lead to attraction of attention. In fact, the likelihood of change blindness for a particular stimulus is contingent on several factors, such as the importance of the stimulus to the scene context (Rensink et al., 1997) or to the ongoing task (McCarley et al., 2004; Triesch, Ballard, Hayhoe, & Sullivan, 2003), the semantic consistency between the stimulus and the scene (Hollingworth & Henderson, 2000), the magnitude of the change (i.e, rotation versus deletion) (Henderson & Hollingworth, 2003), the probability the change will occur (Beck, Angelone, & Lewin, 2004; Porubanova & Sikl, in press), on the availability of resources (McCarley et al., 2004), whether the stimulus was foveated (Henderson & Hollingworth, 1999), or the stimulus’ visual saliency (McCarley et al., 2004). Therefore, attention allocation is highly dependent on other visual or semantic factors than attention alone (i.e., the deployment of eye movements toward a particular stimulus) does not guarantee the overcoming of change blindness. For instance, a persistence of change blindness has been shown even in the face of a fixating on the target location. For instance, when pre- and post-change regions were fixated, subjects failed to see 60% of changes if the change was a deletion of an object, but 90% of rotations of an object (Henderson & Hollingworth, 1999), indicating that even when attention is allocated to the target, there can still be a failure in 18

change detection. However, the role of attention in change detection has been demonstrated in various studies (Mack & Rock, 1998; Rensink, 2000; Scholl, 2000). For instance, Scholl (2000) showed that cuing a location of a changing object by using an abrupt onset or color singleton, leads to faster change detection in comparison to uncued objects. Rensink (2000) found reaction times to change detection contingent on the stimulus set size, suggesting the role of attention in serial scene exploration. Performance in change blindness is determined by expertise as well, suggesting the importance of utilization of attentional resources in change detection. For instance, action video players show greater (and different) search patterns when exploring the scene than non-video players (Clark, Fleck, & Mitroff, 2011); football players are more sensitive than novices to football-related changes (Werner & Thies, 2000); and chess players are faster at detecting meaningful changes to chessboard configurations (Reingold, Charness, Pomplun, & Stampe, 2001). Similarly, attentional preferences are shown in patients with Williams syndrome (a disorder characterized by learning difficulties, but possessing great social skills), who show superior change detection for information pertaining to social interactions (Tager-Flusberg, Plesa-Skwerer, Schofield, Verbalis, & Simons, 2007), and patients with specific anxieties show preferential attention for stimuli that are related to their anxiety (Mayer, Muris, Vogel, Nojoredjo, & Merckelbach, 2006; McGlynn, Wheeler, Wilamowska, & Katz, 2008). Therefore, enhanced visual attention abilities enable better performance in change detection, as shown in the research on attentional cuing, or top-down processing demonstrated through expertise influence.

2.4 Summary

To conclude, change blindness has become not only an important phenomenon in the study of visual perception and attention, but the experimental paradigms used to study change blindness initially (the flicker paradigm in particular), have been adopted as powerful methods in studying visual attention in general in different populations, or even modalities. Change blindness provides important insights, as it demonstrates an astonishing potential for failure to 19

notice even information that occurs right in front of our eyes. Furthermore, and maybe even more importantly, change blindness represents a tool demonstrating the engagement or application of other cognitive or executive functions in conscious perception (attention, working memory, long-term memory).

20

References: Arrington, J. G., Levin, D. T., & Varakin, D. A. (2006). Color onsets and offsets, and luminance changes can cause change blindness. Perception, 35, 1665–1678. Auvray, M., Gallace, A., Hartcher-O’Brien, J., Tan, H. Z., & Spence, C. (2008). Tactile and visual distractors induce change blindness for tactile stimuli presented on the fingertips. Brain Research, 1213, 111-119. Beck, M. R., Angelone, B. L., & Levin, D. T. (2004). Knowledge about the probability of change affects change detection performance. Journal of Experimental Psychology: Human Perception & Performance, 30(4), 778-791. Beck, M. R., & Levin, D. T. (2003). The role of representation volatility in recognizing pre- and postchange objects. Perception & Psychophysics, 65, 458–468. Blackmore, S. J., Brelstaff, G., Nelson, K., & Troscianko,T. (1995). Is the richness of our visual world an illusion? Transsaccadic memory for complex scenes. Perception, 24, 1075-1081. Brady, T. F., Konkle, T., Oliva, A., & Alvarez, G. (2009). Detecting changes in real-world objects: The relationship between visual long-term memory and change blindness. Communicative and Integrative Biology, 2(1), 1-3. Bridgeman, B., Hendry, D., & Stark, L. (1975). Failure to detect displacement of the visual world during saccade eye movements. Vision Research, 15, 719–722. Bridgeman, B., & Tseng, P. (2011). Embodied cognition and the perception–action link. Physics of Life Reviews, 8(1), 73–85. Clark, K., Fleck, M. S., & Mitroff, S. R. (2011). Enhanced change detection performance reveals improved strategy use in avid action video game players. Acta Psychologica, 136, 67-72. Dretske, F. (2004). Change blindness. Philosophical Studies, 120, 1–18. Eramudugolla, R., Irvine, D. R. F., McAnally, K. I., Martin, R. L., & Mattingley, J. B. (2005). Directed attention eliminates ‘Change deafness’ in complex auditory scenes. Current Biology, 15, 1108– 1113. Gallace, A., Tan, H. Z., & Spence, C. (2006). The failure to detect tactile change: a tactile analogue of visual change blindness. Psychonomic Bulletin Review, 13, 300–303. Gibson, J. J. (1966). The senses considered as perceptual systems. Boston: Houghton Mifflin. Gregg, M. K., & Samuel, A. G. (2008). Change deafness and the organizational properties of sounds. Journal of Experimental Psychology: Human Perception & Performance, 34(4), 974-991. Grimes, J. (1996). On the failure to detect changes in scenes across saccades. In K. Akins (Ed.), Perception: Vancouver Studies in Cognitive Science (pp. 89-110). Oxford, UK: Oxford University Press.

21

Henderson, J. M., & Hollingworth, A. (1999). The role of fixation position in detecting scene changes across saccades. Psychological Science, 10, 438-443. Henderson, J. M., & Hollingworth, A. (2003). Eye movements and visual memory: Detecting changes to saccade targets in scenes. Perception & Psychophysics, 65, 58-71. Hollingworth, A., & Henderson, J. M. (2000). Semantic informativeness mediates the detection of changes in natural scenes. Visual Cognition, 7, 213-235. Hollingworth, A., & Henderson, J. M. (2002). Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception & Performance, 28, 113136. Hollingworth, A., Williams, C. C., & Henderson, J. M. (2001). To see and remember: Visually specific information is retained in memory from previously attended objects in natural scenes. Psychonomic Bulletin & Review, 8, 761-768. Jensen, M. W., Yao, R., Street, W. N., & Simons, D. J. (2011). Change blindness and inattentional blindness. WIREs Cognitive Science, 2, 529–546. Lamme, V. (2003). Why visual attention and awareness are different. Trends in Cognitive Sciences, 7, 12–18. Landman, R., Spekreijse, H., & Lamme, V. A. F. (2003). Large capacity storage of integrated objects before change blindness. Vision Research, 43(2), 149-164. Levin, D. T., & Simons, D. J. (1997). Failure to detect changes to attended objects in motion pictures. Psychonomic Bulletin & Review, 4, 501-506. Konkle, T., Brady, T. F., Alvarez, G. A., & Oliva, A. (2010). Scene memory is more detailed than you think: the role of scene categories in visual long-term memory. Psychological Science, 21(11), 1551-1556. Mack, A., & Rock, I. (1998). Inattentional blindness. Cambridge, MA: MIT Press. Mayer, B., Muris, P., Vogel, L., Nojoredjo, I., & Merckelbach, H. (2006). Fear-relevant change detection in spider fearful and non-fearful participants. Journal of Anxiety Disorders, 20, 510–519. McCarley, J. S., Vais, M. J., Pringle, H., Kramer, A. F., Irwin, D. E., & Strayer, D. L. (2004). Conversation disrupts change detection in complex traffic scenes. Human Factors, 46, 424–436. McConkie, G. W., & Currie, C. B. (1996). Visual stability across saccades while viewing complex pictures. Journal of Experimental Psychology: Human Perception & Performance, 22(3), 563581. McGlynn, F. D., Wheeler, S. A., Wilamowska, Z. A., & Katz, J. S. (2008). Detection of change in threatrelated and innocuous scenes among snake-fearful and snake-tolerant participants: data from the flicker task. Journal of Anxiety Disorders, 22, 515–523. Noë, A. (2002). Is the visual world a grand illusion? Journal of Consciousness Studies, 9(5/6), 1-12.

22

Noë, A. (2005). What does change blindness teach us about consciousness? Trends in Cognitive Sciences, 9(5), 218. Noë, A., Pessoa, L., & Thompson, E. (2000). Beyond the grand illusion: what change blindness really teaches us about vision. Visual Cognition, 7, 93-106. O’Regan, J. K. (1992). Solving the “real” mysteries of visual perception: the world as an outside memory.Canadian Journal of Psychology, 46, 461-488. O’Regan, J. K., Deubel, H., Clark, J. J., & Rensink, R. A. (2000). Picture changes during blinks: Looking without seeing and seeing without looking. Visual Cognition, 7, 191-211. O’Regan, J. K., & Noë, A. (2001a). A sensorimotor account of vision and visual consciousness. Behavioral & Brain Sciences, 24 (5), 939–1031. O’Regan, J. K., & Noë, A. (2001b). What it is like to see: A sensorimotor theory of perceptual Experience. Synthese, 129(1), 79–103. O’Regan, J. K., Rensink, R. A., & Clark, J. J. (1999). Change blindness as a result of “mudsplashes”. Nature, 398, 34. Porubanova, M., & Sikl, R. (in press). What is our attention blind to? Studia Psychologica. Reingold, E. M., Charness, N., Pomplun, M., & Stampe, D. M. (2001). Visual span in expert chess players: evidence from eye movements. Psychological Science, 12, 48–55. Rensink, R. A. (2000). Visual search for change: A probe into the nature of attentional processing. Visual Cognition, 7, 345-376. Rensink, R. A. (2005). Change blindness. In L. Itti, G. Rees, & J. K. Tsotsos (Eds). Neurobiology of Attention (pp. 76-81). San Diego, CA: Elsevier. Rensink, R. A., O'Regan, J. K., & Clark, J. J. (1997). To see or not to see: The need for attention to perceive changes in scenes. Psychological Science, 8, 368-373. Scholl, B. J. (2000). Attenuated change blindness for exogenously attended items in a flicker paradigm. Visual Cognition, 7, 377–396. Sela, L., & Sobel, N. (2010). Human olfaction: a constant state of change-blindness. Experimental Brain Research, 205(1), 13-29. Simons, D. J. (2000). Current approaches to change blindness. Visual Cognition, 7, 1–16. Simons, D. J., Franconeri, S. L., & Reimer, R. L. (2000). Change blindness in the absence of a visual disruption. Perception, 29(10), 1143 – 1154. Simons, D. J., & Levin, D. T. (1998). Failure to detect changes to people in a real-world interaction. Psychonomic Bulletin & Review, 5, 644–9. Simons, D. J., & Rensink, R.A. (2005). Change blindness: Past, present, and future. Trends in Cognitive Sciences, 9(1), 16–20. 23

Smith, T. J., Lamont, P., & Henderson, J. M. (2012). The penny drops: Change blindness at fixation. Perception, 41(4), 489 – 492. Tager-Flusberg, H., Plesa-Skwerer, D., Schofield, C., Verbalis, A., & Simons, D. J. (2007). Change detection as a tool for assessing attentional deployment in atypical populations: the case of Williams syndrome. Cognition, Brain, Behavior, 11, 491–506. Triesch, J. J., Ballard, D., Hayhoe, M., & Sullivan, B. (2003). What you see is what you need. Journal of Vision, 3, 86-94. Tseng, P., & Bridgeman, B. (2011). Improved change detection with nearby hands. Experimental Brain Research, 209, 257-69. Tseng, P., Bridgeman, B., & Juan, C. H. (2012). Take the matter into your own hands: A brief review of the effect of nearby-hands on visual processing. Vision Research, 72, 74-77. Turatto, M., & Bridgeman, B. (2005). Change perception using visual transients: object substitution and deletion. Experimental Brain Research, 167, 595-608. Vitevitch, M. S. (2003). Change deafness: the inability to detect changes between two voices. Journal of Experimental Psychology: Human Perception and Performance, 29, 333–342. Wan, X. I., Ambinder, M. S., & Simons, D. J. (2009). Change blindness. In T. Baynes, A. Cleeremans, & P. Wilken (Eds.), The Oxford Companion to Consciousness (pp. 130-133). Oxford, UK: Oxford University Press. Werner, S., & Thies, B. (2000). Is “change blindness” attenuated by domain-specific expertise? An expert-novices comparison of change detection in football images. Visual Cognition, 7(1/2/4), 163-173.

24

Chapter 3: Scene Perception The goal of this chapter is to provide information about scene perception, i.e., the structure of scenes, the role of eye movements in scene perception, theories of scene perception and the role of cognitive factors in scene perception. An understanding of scene perception might be crucial in identifying the link between change blindness and its determinants, as examined in our Experimental Study I. Questions concerning the reasons why a particular change (as part of a scene) was or was not detected and how fast each change was detected might be elucidated by drawing this connection.

3.1 What is Scene Perception?

Wherever we look, there is a scene to be perceived. Scene perception is an activity we conduct every single day (almost) at any given time, and there is very limited time in our lives when our perception does not involve scene perception. Scene perception refers to perception of an environment which entail objects, their relative locations and expectations about future outcomes (Rensink, 2000). Henderson and Hollingsworth (1999) define a scene as a semantically coherent percept of a natural environment that consists of background and discrete, individuated objects presented in a spatially meaningful way. It is important to mention that scenes are often accompanied by the synonymous adjectives ‘natural’ or ‘real-world’, indicating that understanding of scene perception refers to “visual material” as encountered during everyday perception (Henderson & Ferreira, 2004). In this way, scene perception deals with stimuli very different from those used in many psychophysical experiments. Thus scene perception can be referred to as ‘real-world perception’. All natural scenes contain structures, such as features, surfaces, objects and events with specific semantic and syntactic limitations such as those encountered in the everyday world (Henderson & Ferreira, 2004). The background, events and 25

objects contained in a scene are crucial for further information processing, subsequent thinking and actions that one executes when perceiving a scene (Triesch, Ballard, Hayhoe, & Sullivan, 2003). Chun (2003) equates the understanding of scene perception to the understanding of vision because of its predictive power in explaining the mechanisms underlying everyday vision, while Rensink (2003) sees scene perception as a special case of visual perception. In a review on scene perception, Henderson and Hollingworth (1999) list the following key features of scenes: spatial configuration (the locations of objects relative to each other, e.g., cutlery on a table); object shapes that correlate together (e.g., kitchen, cutlery, sink, kettle); temporal structure (movements of objects within a scene allowing for an action; integration of scene over time). ‘Scene gist’ is another important component of scenes (Rensink, 2000), Even though scene perception might be perceived as completely effortless by an observer (Biedermann, 1981), its underlying complexity shows otherwise. Questions, such as what a scene entails, what aspects are represented, and what are the processes underlying scene perception, have been a substantial part of extensive scene perception research for many decades. Chun (2003) points out that a categorization or unitary criteria of scene description are still lacking. Despite of this, some consensus exists: • Scenes are complex. Every scene possesses a great amount of information that is far beyond the grasp of the visual system. Therefore, attentional mechanisms play a huge role in determining which ‘scene regions’ (or objects, events) will be further encoded and processed, and the complexity of scenes and the disability of our visual system to process every information result in quite remarkable gaps in our perception of the visual world (as demonstrated of examples of change blindness, inattentional blindness, and attentional blink). In scene perception two categories of cues can draw attention to themselves: a, ‘bottom-up cues’: salient visual features such as color, size, orientation, motion direction; b, ‘top-down cues’: such as perceptual set, scene context, novelty.

26

• Scene structure is stable (invariant). Scenes are represented by structures that are regular irrespective of their complexity and variability. We can find plenty of expected, well-known, familiar objects, events, or scene-level properties. Even novel scenes have resemblance with already seen ones. For instance, even though one has never driven on a new, specific road, she has the experience of driving on similarly constructed roads; roads with similar physical properties. Anticipatory processes play an important role in scene perception. • Scenes are disposed with contextual information. Very rarely do objects occur in a scene without rich contextual information such as features, events, and surfaces creating a global visual context.

One of the ways of determining how context- and scene-meaning are relevant how ongoing tasks guide attention has been examined by studies using eye movement tracking, which will be discussed in the following chapter.

3.2. Eye Movements and Scene Perception

Eye movements represent an essential mechanism through which visual attention can be allocated in order to process information embedded in a scene. Even though eye movements help visual attention focus on a specific part of a visual scene, the necessary mechanism for conscious processing of the information is visual attention itself (Carrasco, 2011; Cohen, Alvarez, & Nakayama, 2012; Cohen, Cavanagh, Chun, & Nakayama, 2012; Mack & Rock, 1998), since the allocating work of eye movements does not always correspond to conscious visual information processing. Saccades (or rapid eye movements) are accompanied by fixations— pauses in attending to specific parts of the scene. During each brief saccadic eye movement, visual information processing is repressed (Matin, 1974), and all incoming visual information at the 27

time of the saccadic movement is selectively blocked (i.e., saccadic suppression; Bridgeman, Hendry, & Stark, 1975). The saccades repeatedly occur three-to-four times per second, and fixations between each saccade last on average 330 ms (Henderson, 2007). The variability in fixation time varies; individual fixation time depends on properties like an objects’ luminance, or the nature of the ongoing task. For instance, durations are longer for memorization tasks in comparison with visual search tasks (Henderson, 2007), and are dependent also on the individual’s arousal accompanying the scene exploration (Kaspar et al., 2013). The location of the fixation point, as well as the width of the fixation position, is indicative of the processes underlying scene perception (Henderson & Hollingworth, 1999). When viewing real-world scenes, one has a powerful impression that she processes all information cast onto retina. However, this “fallacy” about the nature of the visual system, already at the level of anatomy of the visual system (starting with perturbations in visual processing during saccades) has been demonstrated in various experiments. In a study by Grimes (1996), a rather large change in a visual scene occurred during an eye movement. For example, when a child in a playground scene was enlarged by 30% of its original size and adjusted accordingly in depth, detection of the change was very poor. In a research study by Blackmore, Brelstaff, Nelson, & Trosciansko (1995), in which real-life images were presented, every change presented in an image was accompanied by the movement of the whole image in a random direction. Therefore, in order to detect any change, a person had to execute eye movements. If the original image was displaced in a way emulating the saccade while a change occurred, only half of the overall changes were detected. In an initial experiment using the flicker paradigm (Rensink et al., 1997), two alternating scenes were interjected with a blank screen, while this sequence was being repeated until a participant detected a change between the two scenes. Participants had difficulties detecting changes, and the detection time was inversely correlated with the importance of the aspect of the scene to the scene’s context. The results of those studies ignited a fervent debate concerning the nature of visual representations (see Chapter 2). Wolfe (1999) states that visual representation remains active for a very short period of time after the physical stimulus is being extinguished and refers to this phenomenon as visual amnesia, because rather than an absolute absence of visual representation, a memory for that information is missing. However, not all information, not even subliminally presented visual 28

information, seems to be vulnerable to decay. Certain properties are more likely to be retained; attention optimizes the visual system’s limits by prioritizing representations of the relevant locations or features while inhibiting irrelevant information (Bradley, 2009; Carrasco, 2011). Even though detecting changes in visual environments has an evolutionary advantage (Beck, Ress, Frith, & Lavie, 2001), it has been shown that subjects are poor at detecting changes if visual continuity is disrupted (Fernandez-Duque, Grossi, Thornton, & Neville, 2003; Levin & Simons, 1997; McCarley et al., 2004; O'Regan, Rensink, & Clark, 1999; Rensink, O’Regan, & Clark, 1997, 2000; Scholl, 2000; Simons, 1996; Simons & Rensink, 2005), and therefore in situations of visual disruption, a need for representing relevant information might be even more important in guiding human behavior (Henderson & Holingworth, 1999; Rensink et al., 1997). For instance, Loftus and Mackworth (1978) conducted research on eye movements in scenes which was driven by the fact that gaze in scene perception is focused and attracted to ‘informative areas of the scene’. Areas of scenes that were unexpected, or had a lower probability to appear in the scene, were fixated upon for longer time (e.g., an octopus in a farm scene), and more fixations were allotted to those objects. In a similar experiment (Loftus, Loftus, & Messo, 1987), subjects viewed scenes which contained a non-threatening or threatening item in a scene (e.g. a bank check versus a gun) while their eye movements were recorded. As a result, subjects allotted their attention most often toward the central objects in a threatening scene while they did not remember details outside of the central area of the scene. On the other hand, in non-threatening scenes, they allotted their attention more equally throughout the whole scene. Another experiment equalized the latency of fixations across neutral and upsetting events in scenes (Christianson, Loftus, Loftus, & Hoffman, 1991). Regardless of this constraint, information about central objects in upsetting scenes was recalled to a greater extent in comparison with that of neutral scenes. Similarly, more recent research by Rensink et al. (1997) showed that, mostly, information of high-interest and great significance is retained across individual fixations. In an experiment by Hollingworth, Schrock, and Henderson (2001), target objects presented in flicker paradigm were either deleted or rotated by 90° during saccades. Change detection tended to occur when the eyes were fixated on the changing region. Additionally, when participants were able to move their eyes freely (in opposition to being asked to maintain a central fixation), the change detection was higher. The results indicate a close link between 29

fixation position and target detection. This suggests that both fixation position and overt attention are relevant factors in change detection. Further evidence that the allocation of visual attention is important in successful change detection in flicker paradigm is provided by various research studies (Boyer, Smith, Yu, & Bertenthal, 2011; Rensink et al., 1997; Tse, 2004; Wolfe, 1999), showing that the relevant and informative parts of the scenes are prioritized in attentional processing. Furthermore, the meaning of a fixated region seems to be linked to the probability that an area will be fixated upon, because meaningful objects usually differ from the rest of the scene (Henderson, 2007). One might think that the findings suggest that our visual representations from each fixation do not necessarily convey every single little detail, but merely retain the information about the most relevant scene objects and areas of the scene. However, although our phenomenal consciousness possesses rich, detailed, very intricate representations of the world, access consciousness enables the retaining of a rather small portion of the initially present information (Block, 2007). Therefore, it would be misleading to think that visual representations retained from successive views are very poor in their nature, but rather to believe that the nature of visual percept does not necessarily correspond to the memory trace left from the initial view.

3.3 High-Level Scene Perception

A long-term quest for understanding how humans make sense of the constant flow of information “attacking” our senses has been active since the work of Immanuel Kant. Kant (1781) spoke about the faculty of Sensibility, that is, a receptive capacity to detect raw, not interpreted sensory information; and the faculty of Understanding, i.e., organizing those raw data into a meaningful experience of the environment. Both faculties can be seen as representing the low-level and high-level perception, respectively. How do we organize this vast, chaotic information bombarding our retina? Our ability to perceive and process all incoming information in parallel is hindered by the capacity limits of our cognition. Therefore, attention systems subserve the goal to pick only a portion of all information for more detailed scrutiny (Chun & Wolfe, 2001; Lamme, 2003). Both 30

stimulus-based and observer-based factors can guide attentional selection (Brosch, Pourtois, Sander, & Vuilleumier, 2011). Stimulus-based properties can be related to either low-level salience features, such as size, color or contrast (Egeth & Yantis, 1997; Treisman & Gelade, 1980), while high-level features encompass voluntary, top-down mechanisms mediated via an observer’s internal state and expectations regarding specific objects, locations of parts of scenes, as well as scene memory. Being exposed to scene regularities (spatial layout, object shapes, and location correspondences) enables an individual to appropriate attention allocation to visual information or objects relevant to the ongoing behavior (Chun, 2003). Rensink (2000) refers to high-level scene processing as “concerned with issues of meaning”. (p.155), such that the identifying of a scene as a beach activates an expectation of objects that could occur in that scene (sand, sea, beach umbrellas, seagulls) and therefore involves the synthesis of a priori knowledge with the ongoing results of attentional processing (Silva, Groeger, & Bradsaw, 2006). In short, high-level scene perception explores the interaction of perception and cognition when viewing natural scenes (Henderson & Hollingworth, 1999).

Figure 1: Representation of a scene (modified according to Rensink, 2000).

gist

country -side

Schema “countryside”

object

layout

31

For example, since most scene layouts remain invariant, an understanding what can or cannot occur in the scene occurs (Hollingworth, 2008), i.e., establishment of scene schema. Schema can be defined as an abstract representation of a particular scene (Hollingworth, 2008) encompassing information such as objects, information about the objects’ appearance, and the spatial location of objects and their relationship (Mandler & Parker, 1976). Mandler & Parker (1976) also point out the importance of actions in the scene, i.e., “what can be inferred to be happening in the picture” (p. 39), but also the types of objects that can be expected in the scene and their importance for the ongoing task (Friedman, 1979). When discovering a presence of an unexpected object in a scene, re-assessment of that object and scene’s gist takes place, leading to a new association between the two (Rensink, 2000). The effect of ‘scene knowledge’, i.e., an understanding of what can be expected to occur in a specific scene, on memory for those objects has been widely studied and the results are rather conflicting. Some studies suggested that memory for schema-consistent information is superior to memory for schema-inconsistent information (Silva et al., 2006), while others suggest the contrary (De Graef, Christiaens, & d'Ydevalle, 1990; Loftus & Mackworth, 1978; Pezdek et al., 1989). The effect of scene representation on subsequent object identification has been also shown (Palmer, 1975), illustrating how the activation of scene memory enables faster processing of congruent stimuli and hinders recognition of stimuli that is incongruent with the scene. Similar to change detection tasks, detection is accelerated for features central to scene context when compared to less central information (Hollingworth, Williams, & Henderson, 2001; Rensink et al., 1997; Turrato & Bridgeman, 2005). In terms of immediate scene knowledge, familiarity with a scene affects attention allocation for that scene, resulting in an initial scanning of previously changed locations and, subsequently, previously changed objects at their new locations (Becker & Rasmussen, 2008). When viewing scenes, the initial fixations tend to aggregate around the more informative regions (Antes, 1974; Loftus & Mackworth, 1978; Mackworth & Morandi, 1967), although other studies have concluded that initial fixations are controlled by visual (not semantic) factors (Mannan, Ruddock, & Wooding, 1996). Once semantic analysis of the scene occurs after the first fixation, an important factor in determining the deployment of eye movements, i.e., attention allocation, is semantic informativeness (Henderson & Hollingworth, 1999; Hollingworth, 2008).

32

Another important piece of evidence concerning the role of context in object identification (and therefore attention allocation in scene perception) comes from the phenomenon called ‘contextual cuing’. Contextual cuing refers to a process of guiding attention to certain information in a scene based on a previous memory for context of a scene, resulting in rapid target detection (Chun, 2003). Chun and Jiang (1998) and Peterson and Kramer (2001) examined the visual search for novel and old contexts. Subjects detected targets presented in the old scenes faster than in the new ones. This phenomenon was dubbed the ‘contextual cuing effect’ due to the facilitating function of context in guiding attention to specific objects in scenes. Furthermore, selective attention is important in contextual cuing, because only attended contextual information provides facilitation for further visual information processing. This demonstrates the importance of attention in implicit learning (as context is not learned intentionally and explicitly). Therefore, context augments the identification of objects consistent with the scene (Chun, 2003), although the top-down influence can be impeded by scene inversion (Kelley, Chun, & Chua, 2003) or scene regions being jumbled (Biederman, Glass, & Stacy, 1973).

3.4 Theories of Attentional Guidance in Scene Perception

A great emphasis has been put on the role of attention in scene perception, an emphasis drawn mostly from change detection research (Hollingworth & Henderson, 2002), but a great body of research on other phenomena also implies many conclusions about attentional mechanisms involved in scene perception. As such, sophisticated selection methods are involved in focusing on and selecting only a small part of the scene, mainly reflecting the behavioral relevance of the individual elements (Chun & Jiang, 1998). Two main and competing theories of scene perception were proposed by Rensink (2002) and Henderson and Hollingworth (2002). Coherence Theory in scene perception (Rensink, 2002) posits that the retention of volatile object representations or proto-objects occurs when attention is not focused on them. Attention, therefore, is a necessary precursor of any visual representation. In contrast, Visual Memory

33

Theory of scene perception (Hollingworth & Henderson, 2002) states that robust object representations from scene explorations are retained.

3.4.1 Coherence Theory Rensink (2002) proposed that only one object at a time is properly attended to and consequently represented, and focused attention plays a primary role in the construction of the representation. The coherence theory of attention, introduced by Rensink, suggests that attention helps construct a stable representation of an object, which enables successful change detection. Normally, when a change is accompanied by a motion signal, any change is easily detected. However, when the visual continuity of scene perception is compromised, attention is necessary to encode the information carrying the difference (change). This suggests that attention is connected with coherence, meaning consistency, both in representational sets, as well as in spatiotemporal consistency. Moreover, structures that are created by attention are brief with the latency of attentional focus.

The main assumptions of Coherence Theory of Attention (Rensink, 2000, 2002) are:

1, Proto-objects, as low-level formations, are being created across the visual field (prior to attention being focused on them). Due to the volatility of those proto-objects, they are constantly being replaced with new stimuli presented at the same retinal location.

2, Focused attention gathers a small amount of those proto-objects from the flow, creating a stable formation with coherence over time and space. Therefore, any new stimulus (a change) is considered to be different from the former, stable one.

3, An object retains its coherence only when attention is focused on it, once the object is out of attentional focus, it loses its coherence.

According to this theory, a change in a scene can be detected only when attention is directly focused on that part of the scene. ‘Nexus’, a group consisting of proto-objects forming a 34

local hierarchy, enables information within it to be easily accessible. Nexus properties differ from object files (Irwin, 1992) by being limited solely to abstract properties, whereas object files consist of non-visual information as well. Coherence Theory basically states that objects that are not attended to cannot enter the consciousness; therefore the ability to report changes concerning unattended scenes is impaired. This proposition is consistent with research on inattentional blindness. Mack and Rock (1998) showed that unattended parts of visual scenes go unnoticed; participants are unable to report information that has been modified through the scene. However, more than being a perceptional mechanism, this seems to be a mechanism of memory, and even though the information is perceived, it is not encoded in memory (Dretske, 1997). Therefore, attention seems to be a necessary mediator toward conscious perception; volatile representations are constantly created in the presence and absence of attention. Only one object (or part of scene) can be represented coherently at any time (Rensink, 2000). ‘Virtual representation’ is a term used for a representation that is created at a time for current intents and purposes, and all other representations available in the vicinity that allows to exert less of processing and using of memory resources. Therefore, all the virtual representations constitute a network of available information. Chun (2003) suggests that attention deals with the extraction of important information from scenes and with how this information is utilized to guide behavior.

3.4.2 Visual Memory Theory of Scene Perception Hollingworth and Henderson (2002) suggested that rather robust visual object representations are maintained after attention is directed away from the target. They believe that the visual object representations during online processing (especially in change detection tasks when two different images alternate) are being stored in visual short-term memory. The main difference in comparison with the Coherence Theory is that Hollingworth and Henderson (2002) emphasize the importance of short-term and long-term memory in the online perception of scenes rather than attention augmented only by visual short-term memory, as suggested by Rensink (2002). This account is supported by evidence concerning the role of semantic knowledge in scene representation (Hollingworth et al., 2001), and the selection and storage of ostensibly behavior-relevant stimuli. The model is based on several assumptions:

35

1, The ‘construction’ of scene representation is of higher level nature (object identity and meaning) abstracted from low-level sensory attributes; 2, The higher level representations are projected onto the scene layout creating an ‘object file’ that represents both visual short-term memory (sensory) as well as conceptual short-term memory (higher level); 3, Locking the former into a specific spatial location leads to the construction of ‘longterm object file’;

4, Withdrawal of attention is followed by the decay of short-term memory representations and retention of long-term of object files. Attended areas of a scene contribute to the formation of a ‘scene map’, a fairly detailed representation; 5, ‘Attention allocation’ plays a key role in the comparison of the retrieval of long-term object files with the ongoing perceptual processing;

6, Once a scene is removed, the long-term object file contains a scene map and local object codes.

The model posits that the visual representation of a scene is rather intricate and possesses a great deal of detail. Scanning of the scene through eye movements and attention allocation controls visual information encoding and retrieval. The most important distinction of this model is the emphasis on the storage of visual representations in the long-term memory after attentional withdrawal and a fast decay of the information in visual short-term memory.

36

3.5 The Nature of Scene Representations

The limitations and constraints imposed on the visual system (e.g., saccadic eye movements, the high-acuity of visual information limited to the foveal part of the retina, rather unreliable transsaccadic memory) represent a difficulty in the process of creating visual representations (Hollingworth & Henderson, 2002) that would be veridical to the actual visual environment while also durable. As Intraub (2007) wrote: “In a sense, it as if the world is always viewed through a “window”: an imperfect window, with graded clarity” (p. 454). This refers to the fact that the greatest visual acuity is enabled by the foveal region, which provides a perfect resolution of incoming retinal information of only 2° of visual angle. Therefore, movement during perception is necessary in order to achieve an immaculate, high-resolution percept. Although change blindness research suggests that a rather sparse representation is created from individual fixations (Rensink, 2000; Triesch et al., 2003); research on iconic memory demonstrates that quite a bit of information is processed, while only very little information is transferred to working memory for later recall. Attention determines what is to be remembered; conscious perception entails attention, as documented by studies on inattentional blindness (Mack & Rock, 1998), change blindness (Rensink et al., 1997) or attentional blink (Raymond, Shapiro, & Arnell, 1992). A normal state of attention (i.e., when attention is distributed) enables a less robust, but richer impression of the percept (Mack, 2002). However, focused attention allows for attenuation of perception even at a very low level. It has been suggested that attention amplifies even preattentive information processing, such as contrast discrimination, texture segmentation and acuity (Carrasco, 2011). In terms of an understanding of visual representations, Irwin (1992; Gordon & Irwin, 1996) proposed the ‘object file’ theory of transsaccadic memory that refers to assumptions about the nature of scene representations. The most important factor in deciding which visual information will be represented and which will not, is allocation of visual attention as already mentioned. When selected by attention, a temporary object file is concocted by integrating the 37

visual features of the object with its spatial location. Across saccades, object files are integrated within short-term memory (Irwin, 1992). According to this theory, scene information encoded across saccades is limited to three sources: active object files conveying visual codes; activated conceptual nodes in long-term memory, and schematic representation containing conceptualsemantic properties (the gist of the scene). According to object file theory, when attention is directed to a certain object, a temporary representation of the object is constructed. Either a ‘perceptual representation’, i.e., to the structural arrangement of a scene, or a ‘conceptual representation’, i.e., semantic information that is deduced from the viewing of the scene, can be created (Oliva, 2005). When the object changes in some way, a comparison between the previous and the current object file is made. Most importantly, an object file gathers information about the objects and compares it to the information retained in conceptual memory. Therefore, visual representations throughout the individual saccades seem to be dependent on higher-level processing (Riesenhuber & Poggio, 1999), i.e., an a priori existent scene or conceptual knowledge. As such, prior experience helps us predict what is to be expected in a scene and what is not. Hollingworth and Henderson (2002) suggest: “If visual representations are accumulated from previously fixated and attended regions, they must be in a form significantly more abstract than sensory representations” (p. 114). They further state that human gaze control behaves intelligently because it engages not only the currently viewed visual information, but also shortterm and episodic memory for the ongoing scene, and also stored long-term visual, spatial, and semantic information for analogous scenes (Hollingworth & Henderson, 2002). In this respect, Hochberg (1986) proposed that the current percept is embedded within ‘schema’, a construct encompassing both the memory for previous views, as well as anticipatory estimations of information in accordance with the observer’s actions or prospects. As previously suggested even a single view of a scene can trigger or create an expectation about the parts of the scenes that did not undergo "thorough scanning” (Hochberg, 1986). The nature of schema is seemingly very abstract, sketch-like (Noë, 2002; Noë, Pessoa, & Thompson, 2000; O’Regan, 1992), rather than depicting exact, metric parameteres of the scene if it has been not been selected by attention (Mack & Rock, 1998; Sperling, 1960), or when interrupted by new information and not updated (Chun & Potter, 1995; Raymond et al., 1992).

38

3.6 Summary

Research in scene perception provides important insights into the understanding of everyday visual information processing. Endogenous attention picks out behavior-relevant information in order to accomplish successful goal-oriented operations. Therefore, next to lowlevel visual saliency, high-level scene factors contribute to viewing behavior. The empirical quest in understanding this issue will be presented in the first experimental study.

39

References: Antes, J. R. (1974). The time course of picture viewing. Journal of Experimental Psychology, 103, 62-70. Beck, D. M., Rees, G., Frith, C. D., & Lavie, N. (2001). Neural correlates of change awareness and change blindness. Nature Neuroscience, 4(6), 645-650. Becker, M. W., & Rasmussen, I. P. (2008). The guidance of attention to objects and locations by longterm memory of natural scenes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(6), 1325-1338. Biederman, I., Glass, A. L., & Stacy, E. W., Jr. (1973). Searching for objects in real-world scenes. Journal of Experimental Psychology, 97, 22-27. Biederman, I. (1981). On the semantics of a glance at a scene. In M. Kubovy & J. R. Pomerantz (Eds.), Perceptual Organization (pp. 213-263). Hillsdale, New Jersey: Lawrence Erlbaum. Blackmore, S. J., Brelstaff, G., Nelson, K., &Troscianko, T. (1995). Is the richness of our world an illusion? Transsaccadic memory for complex scenes. Perception, 24(9), 1075-1081. Block, N. (2007). Consciousness, accessibility, and the mesh between psychology and neuroscience. Behavioral and Brain Sciences, 30, 481–548. Boyer, T. W., Smith, T. G., Yu, C. &Bertenthal, B. I. (2011). Visual attention and change detection. In L. Carlson, C. Hölscher, & T. Shipley (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society. Bradley, M. M. (2009). Natural selective attention: Orienting and emotion. Psychophysiology, 46, 1-11. Bridgeman, G., Hendry, D., & Stark, L. (1975). Failure to detect displacement of visual world during saccadic eye movements. Vision Research, 15, 719-722. Brosch, T., Pourtois, G., Sander, D., & Vuilleumier, P. (2011). Additive effects of emotional, endogenous, and exogenous attention: Behavioral and electrophysiological evidence. Neuropsychologia, 49, 1779-1787. Carrasco, M. (2011). Visual attention: The past 25 years. Vision Research, 51, 1484-1525. Christianson, S., Loftus, E. F., Loftus, G. R., & Hoffman, H. (1991). Eye fixations and accuracy in detail memory of emotional versus neutral events. Journal of Experimental Psychology: Learning, Memory, & Cognition, 17, 693-701. Chun, M. M. (2003). Scene perception and memory. In D. Irwin & B. Ross (Eds.), Psychology of Learning and Motivation: Advances in Research and Theory: Cognitive Vision, Vol. 42 (pp. 79108). San Diego, CA: Academic Press. Chun, M. M., & Jiang, Y. (1998). Contextual cueing: Implicit learning and memory of visual context guides spatial attention. Cognitive Psychology, 36, 28-71.

40

Chun, M. M., & Potter, M. C. (1995). A two-stage model for multiple target detection in rapid serial visual presentation. Journal of Experimental Psychology: Human Perception and Performance, 21, 109-127. Chun, M. M., & Wolfe, J. M. (2001).Visual attention. In E. B. Goldstein (Ed.), Blackwell's Handbook of Perception, Vol. 9 (pp. 272-310). Oxford, UK: Blackwell. Cohen, M. A., Alvarez, G. A., & Nakayama, K. (2011). Natural scene perception requires attention. Psychological Science, 22(9), 1165-1172. Cohen, M. A., Cavanagh, P., Chun, M. M., & Nakayama, K. (2012). The attentional requirements of consciousness. Trends in Cognitive Sciences. 16, 411-417. De Graef, P., Christiaens, D., & d’Ydevalle, G. (1990). Perceptual effects of scene context on object identification. Psychological Research, 52, 317-329. Dretske, F. (1997). What good is consciousness? Canadian Journal of Philosophy, 27(1), 1-15. Egeth, H. E., & Yantis, S. (1997). Visual attention: Control, representation, and time course. Annual Review of Psychology, 48, 269-297 Fernandez-Duque, D., Grossi, G., Thornton, I. M., & Neville, H. J. (2003). Representation of change: Separate electrophysiological markers of attention, awareness, and implicit processing. Journal of Cognitive Neuroscience, 15(4), 491-507. Friedman, A. (1979). Framing pictures: The role of knowledge in automatized encoding and memory for gist. Journal of Experimental Psychology: General, 108, 316-355. Gordon, R., & Irwin, D. (1996). What’s in an object file? Evidence from priming studies. Perception & Psychophysics, 58, 1260-1277. Grimes, J. (1996). On the failure to detect changes in scenes across saccades. In K. Akins (Ed.), Vancouver studies in cognitive science: Perception (pp. 89–110). New York: Oxford University Press. Henderson, J. M. (2007). Regarding scenes. Current Directions in Psychological Science, 16, 219-222. Henderson, J. M., & Ferreira, F. (2004). Scene perception for psycholinguists. In J. M. Henderson & F. Ferreira (Eds.), The interface of language, vision, and action: Eye movements and the visual world (pp. 1–58). New York: Psychology Press. Henderson, J. M., & Hollingworth, A. (1999). High-level scene perception. Annual Review of Psychology, 50, 243-271. Hochberg, J. (1986). Representation of motion and space in video and cinematic displays. In K. J. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance (pp. 1–64). New York: John Wiley & Sons. Hollingworth, A. (2008). Visual memory for natural scenes. In S. J. Luck & A. Hollingworth (Eds.), Visual memory (pp. 123-162). New York: Oxford University Press. 41

Hollingworth, A., & Henderson, J. M. (2002).Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception & Performance, 28, 113136. Hollingworth, A., Schrock, G., & Henderson, J. M. (2001). Change detection in the flicker paradigm: The role of fixation position within the scene. Memory & Cognition, 29, 296-304. Hollingworth, A., Williams, C. C., & Henderson, J. M. (2001). Scene context and change blindness: Memory mediates change detection. Psychonomic Bulletin & Review, 8, 761-768. Intraub, H. (2007). Scene perception: The world through a window. In M. A. Peterson, B. Gillam, & H. A. Sedgwick (Eds.), In the mind’s eye: Julian Hochberg on the perception of pictures, films, and the world (pp. 454-466). New York, NY: Oxford University Press. Irwin, D. (1992). Memory for position and identity across eye movements. Journal of Experimental Psychology: Learning, Memory, & Cognition, 18, 307-317. Kant, I. (1781, 1998). The Critique of Pure Reason. New York, NY: Kessinger Publishing. Kaspar, K., Hloucal, T. M., Kriz, J., Canzler, S., Gameiro, R. R., et al. (2013). Emotions’ impact on viewing behavior under natural conditions. PLoS ONE, 8(1): e52737. Kelley, T. A., Chun, M. M., & Chua, K. P. (2003). Effects of scene inversion on change detection of targets matched for visual salience. Journal of Vision, 3(1), 1–5. Lamme, V. A. F. (2003). Why visual attention and awareness are different. Trends in Cognitive Sciences, 7, 12-18. Levin, D. T., & Simons, D. J. (1997). Failure to detect changes to attended objects in motion pictures. Psychonomic Bulletin & Review, 4, 501–506. Loftus, E. F., Loftus, G. R., & Messo, J. (1987). Some facts about "weapon focus". Law and Human Behavior, 11, 55-62. Loftus, G. R. & Mackworth, N. H. (1978). Cognitive determinants of fixation location during picture viewing. Journal of Experimental Psychology: Human Perception and Performance, 4, 565-572. Mack, A. (2002). Is the visual world a grand illusion? A response. Journal of Consciousness Studies, 9, 102-10. Mack, A., & Rock, I. (1998). Inattentional Blindness. Boston, MA: MIT Press. Mackworth, N. H., & Morandi, A. J. (1967). The gaze selects informative details within pictures. Perception & Psychophysics, 2, 547-552. Mandler, J. M., & Parker, R. E. (1976). Memory for descriptive and spatial information in complex pictures. Journal of Experimental Psychology: Human Learning and Memory, 2(1), 38-48. Mannan, S. K., Ruddock, K. H., & Wooding, D. S. (1996). The relationship between the locations of spatial features and those of fixations made during visual examination of briefly presented images. Spatial Vision, 10, 165-188. 42

Matin, E. (1974). Saccadic suppression: A review and an analysis. Psychological Bulletin, 81, 899-917. McCarley, J. S., Vais, M. J., Pringle, H. L., Kramer, A. F., Irwin, D. E., & Strayer, D. L. (2004). Conversation disrupts change detection in complex traffic scenes. Human Factors, 46, 424-436. Noë, A.(2002). Is the visual world a grand illusion? Journal of Consciousness Studies, 9 (5/6), 1-12. Noë, A., Pessoa, L., & Thompson, E. (2000). Beyond the grand illusion: what change blindness really teaches us about vision. Visual Cognition, 7, 93-106. Oliva, A. (2005). Gist of the scene. In L. Itti, G. Rees, & J. K. Tsotsos (Eds.), The Encyclopedia of Neurobiology of Attention (pp. 251-256). San Diego: Elsevier. O’Regan, J. K. (1992). Solving the “real” mysteries of visual perception: the world as an outside memory. Canadian Journal of Psychology, 46 (3), 461-488. O’Regan, J. K., Rensink, R. A., & Clark, J. J. (1999). Change blindness as a result of “mudsplashes”. Nature, 398, 4. Palmer, S. E. (1975). The effects of contextual scenes on the identification of objects. Memory & Cognition, 3(5), 519-526. Peterson, M. S., & Kramer, A. F. (2001). Attentional guidance of the eyes by contextual information and abrupt onsets. Perception & Psychophysics, 63, 1239-1249. Pezdek, K. et al. (1989). Memory for real world scenes: The role of consistency with schema expectation. Journal of Experimental Psychology: Learning, Memory & Cognition, 15, 587-595. Raymond, J. E., Shapiro, K. L., & Arnell, K. M. (1992). Temporary suppression of visual processing in an RSVP task: an attentional blink? Journal of Experimental Psychology: Human Perception and Performance, 18, 849-860. Rensink, R. A. (2000). Scene perception. In A. E. Kazdin (Ed.), Encyclopedia of Psychology, Vol. 7 (pp. 151-155). New York, NY: Oxford University Press. Rensink, R. A. (2003). Visual attention. In L. Nadel (Ed.), Encyclopedia of Cognitive Science (pp. 509515). London: Nature Publishing Group. Rensink, R. A., O’Regan, J. K., & Clark, J. J. (1997). To see or not to see: The need for attention to perceive changes in scenes. Psychological Science, 8, 368–373. Rensink, R. A., O’Regan, J. K., & Clark, J. J. (2000). On the failure to detect changes in scenes across brief interruptions. Visual Cognition, 7, 127–145. Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2, 1019–1025. Scholl, B. J. (2000). Attenuated change blindness for exogenously attended items in a flicker paradigm. Visual Cognition, 7, 377–396. 43

Silva, M. M., Groeger, J. A., & Bradshaw, M. F. (2006). Attention-memory interactions in scene perception. Spatial Vision, 19, 9-19. Simons, D. J. (1996). In sight, out of mind: When object representations fail. Psychological Science, 7, 301-305. Simons, D. J., & Rensink, R. A. (2005). Change blindness: Past, present, and future. Trends in Cognitive Sciences, 9(1), 16-20. Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs: General & Applied, 74, 1-30. Treisman, A. M., & Gelade, G. (1980). A feature integration theory of attention. Cognitive Psychology, 12, 97-136. Triesch, J., Ballard, D. H., Hayhoe, M. M., & Sullivan, B. T. (2003). What you see is what you need. Journal of Vision, 3, 86-94. Tse, P. U. (2004). Mapping visual attention with change blindness: New directions for a new method. Cognitive Science, 28, 241–258. Turrato, M., & Bridgeman, B. (2005). Change perception using visual transients: object substitution and deletion. Experimental Brain Research, 167, 595-608. Wolfe, J. M. (1999). Inattentional amnesia. In V. Coltheart (Ed.), Fleeting memories (pp. 71–94). Cambridge, MA: MIT Press.

44

Chapter 4: Emotion and Attention 4.1 Introduction

Every instant, our brain is exposed to an unlimited amount of stimuli, be it visual, auditory, or tactile, which at times can be overwhelming and exhausting; a condition reflected in subsequent directed attention (Berman, Jonides, & Kaplan, 2008). For the reason that attentional resources and maintenance of information in working memory are limited, the brain has evolved mechanisms to quickly and effectively choose the most relevant and salient information to manage the task at hand. But what kind of information is necessarily salient or relevant? Much research has suggested that emotional information is one kind of salient stimuli and is prioritized in attentional processing. In this sense, both attention and emotion accentuate our sensitivity to surrounding information. This chapter will deal with understanding of how selective attention is modulated by the affective relevance of sensory information. Two decades ago, the most prevalent paradigm in cognitive psychology research disregarded emotional states in the study of cognition, and most research in the field of interaction between cognition and emotion concentrated on effects of the former on the latter (Lazarus, 1991). The impact of emotion, whether in the form of an emotional stimulus or emotional state of an individual, was neglected, as the majority of research did not include the effects of emotion on cognition and had subjects perform tasks that involved relatively neutral stimuli while the subjects were also in a relatively neutral emotional state. Research of past two decades has changed rapidly in this sense, as a great effort has been made to identify the relationship between emotion and cognition, specifically the effects of emotional states on subsequent cognitive performance. The effect of emotion on cognition is extremely relevant as our cognitive resources are limited in terms of their capacity processing, therefore the brain needed to develop mechanisms for how to deal with these restrictions (Vuilleumier, 2005). Much of the information an individual is exposed to must be detected quickly, often under the threshold for conscious processing. This necessity has evolved to help 45

rapid detection of particularly threatening and survival-relevant cues in the environment in order to provide instantaneous decisions and behavior that can guide one from dangerous situations (LeDoux, 2000). Imagine going through a jungle and almost stepping on a spider. The quick response to avoid this threatening stimulus is enabled via an automatic processing of the stimulus that represents the immediate danger. Automatic responses to threatening stimuli, in particular, are perceived as hardwired and available at the birth, as they induce rapid, adaptive behavioral responses (Cummins & Cummins, 1999). The main goal of this chapter is to introduce an understanding of the interplay between emotion and attention, especially the attentional biases induced by various emotional states or emotional stimuli and to examine whether emotional cues are followed by more efficient attentional processing.

4.2 Interplay between Attention and Emotion

Emotions can be defined as short-lived experiences that modulate one’s thoughts, actions or physiological responses, resulting in specific action tendencies (Frederickson & Braningan, 2005). Emotional states thus impact the mind and subsequent behavior by creating specific bodily states that lead to appropriate actions. For example, fear induces bodily states by creating a readiness for flight, whereas amusement leads to a tendency to sustain the specific behavior that creates it. From an evolutionary perspective, the specific action tendencies resulting from induced emotion represent behavior that has proven to be beneficial in terms of our ancestors’ survival (Tooby & Cosmides, 1990) by inducing either ‘approach’ (in case of a positively evaluated emotional state) or ‘avoidance’ (if a threat is encountered) (Elliot & Covington, 2001; Lang, 2000). Emotions have many dimensions, among which one is ‘valence’, or the perceived pleasure or displeasure induced by an emotional stimulus. Some researchers also refer to emotional valence as the hedonistic value of emotion. Another dimension is ‘arousal’, or the bodily activation that a certain emotion elicits, which varies from feeling calm to feeling excited. 46

Both dimensions have subjective values and pertain to subjective states of pleasantness or un-pleasantness (for valence) or subjective states of feeling activated or deactivated (for arousal) (Feldman Barrett, 1998). Another dimension is ‘motivation direction’, i.e., approach or avoidance (withdrawal), referring to subsequent behavior of going toward or away from an object. In many models of interaction between attention and emotion, it is believed that positive emotion widens attentional scope (Derryberry & Tucker, 1994; Frederickson, 1998, 2001). This attentional broadening for positive emotion is widely supported by empirical evidence (Fernandes, Koji, Dixon, & Aquino, 2011; Frederickson & Branigan, 2005; Rowe, Hirsh, & Anderson, 2007). The seminal work of Easterbrook (1959) showed that states related to avoidance behavior combined with tense arousal lead to a narrowing of attentional focus. On the contrary, states inducing elated arousal have the opposite effect, leading to a broadening of attentional focus. Frederickson (1998, 2001) developed a theory called the Broaden-and-Build Theory of positive emotions, which describes the role of positive emotion in attentional scope. It further posits that the role of positive and negative emotions in attention is complementary. Negative emotions momentarily narrow the scope of mental and behavioral “options” (called thought-action repertoires) through activating the specific action tendencies (such as flight or fight). Conversely, positive emotions expand those thought-action repertoires resulting in a greater scope of thoughts and actions than normally assumed (exploring, reveling, and playing). Frederickson further discusses the adaptive nature of positive and negative emotions. While negative emotions play importance in short-term behavioral survival (through motivating certain actions), positive emotions are adaptive in long-term ways (such as health, friendships, knowledge etc.) (Frederickson & Braningan, 2005) and serve as psychological adaptations. It has been suggested that mood influences also a variety of cognitive mechanisms, such as cognitive control (Martin & Kerns, 2011), broadening of attentional scope (Rowe et al., 2007), alerting (Jiang, Scolaro, Bailey, & Chen, 2011) or spatial memory (Brunye, Mahoney, Augustyn, & Taylor, 2009). The effect of attention broadening due to positive emotions can be also detrimental, because attention is much more prone to interference from spatially distant distractors. For example, Rowe et al. (2007) presented a flanker task in which the central target was surrounded by flankers (distractors) at three different distances from the target. The greater the distance, the 47

slower the reaction time for the target was observed for positive-mood participants compared to neutral or negative-mood participants. This relationship between positive affect and attention has, however, been contested by recent research (Brunye et al., 2009), suggesting that the relationship is moderated by approach motivation (Gable & Harmon-Jones, 2008). Gable and Harmon-Jones (2008) showed in their study that attentional scope does increase due to positive affect, but only when the approach motivation is low (i.e., does not lead to desire, but rather to a joyful emotion), whereas positive emotion resulting in high-approach motivation (i.e., desire) leads to a decrease in the attentional scope (due to increased focused attention). This is a particularly interesting finding, as one must wonder whether desire itself does not create a negative emotional state, as it indicates an attentional bias toward the desired stimulus. Similar evidence was documented in a study using dot detection task (Prause, Janssen, & Hetrick, 2008). In the dot detection task (Mathews & MacLeod, 2002), two stimuli (one “critical”— usually emotional, and another one neutral) are presented simultaneously for a very brief time and subject is supposed to detect a dot that appears in one of the two stimuli location after the two stimuli disappear from the screen. The detection time is believed to be an indicator of attentional bias toward the specific stimulus. Participants who had higher level of sexual desire performed much worse (the reaction times were much longer) in the dot detection task in comparison with participants with low level of sexual desire. Another study of Gable and Harmon-Jones (2010) showed a similar trend for negative emotions. While sadness (low motivation negative emotion) broadened attentional focus, disgust (high motivation negative emotion) caused attentional narrowing. Thus, not only emotional valence alone, but also approach motivation contributes to changes in attentional scope. In general, the empirical evidence shows that positive emotions broaden attention, while simultaneously impairing working memory storage capacity (Martin & Kerns, 2011). When in induced positive emotional states, participants’ working memory capacity was much lower in comparison with participants in neutral states, which was attributed to continuously spreading activation of information in working memory (Davelaar, Goshen-Gottstein, Ashkenazi, Haarmann, & Usher, 2005). However, performance in both the Stroop and flanker tasks was unaffected. The study by Martin and Kerns (2011) is in discordance with the previously

48

mentioned study by Rowe et al. (2007) where the flanker task was predicted by the emotional state of an individual (see above).

4.2.1 Emotional Guidance of Attentional Allocation Emotionally salient stimulus is a kind of information that quickly elicits attentional responses and modulates attention in order to induce greater information selection (Lang, 2000). Lang (2000) refers to an interplay between attention and emotion through understanding attentional allocation as a dependent on specific physiological states, either by appetitive or defensive systems. Those systems govern attentional allocation by either broadening or focusing attention. Again, Lang (2000) sees those systems as having evolved in our ancestors due to their remarkable evolutionary advantages. When referring to an emotional stimulus from an external environment inducing emotional reactions, we talk about bottom-up emotions (in comparison with top-down emotion generation based on previous memories, or conceptual knowledge). The stimulus that elicits emotions is deemed to have certain physical attributes that make it inherently emotional (McRae, Misra, Prasad, Pereira, & Gross, 2011). For instance, an emotion such as joy is evoked by a presentation of cute animals. From an evolutionary perspective, emotions are regarded as mechanisms responding to the detection of perceptual features of emotional stimuli (Aggleton, 1992; LeDoux, 2000) in which the limbic system, specifically the amygdala plays a fundamental role (Vuilleumier, 2005).The amygdala is informed about the emotional significance of a stimulus quickly, even before the stimulus is perceived consciously (Whalen, 1998). Subsequently, the amygdala, via projections to visual cortices, modifies perception and attention (Vuilleumier & Huang, 2009). Thus, the role of the amygdala in increasing an organism´s vigilance and processing of biologically relevant stimuli with predictive value is apparent (Whalen, 1998). Furthermore, the magnitude of the emotionality of a stimulus determines the breath of activation in primary modality-specific brain regions (Vuillemier, 2005). Much research suggests that emotion-laden stimuli are processed automatically, without a need for focused attention, even when awareness of the stimulus presence is lacking (Vuillemier, 2005). Vuilleumier (2005) suggests that this pre-attentive and automatic emotional stimulus processing enables a preference toward relevant stimuli. Evidence for the automatic processing of emotional events is abundant, indicating that emotion modulates attention in a 49

reflexive way (Vuilleumier & Huang, 2009). The evidence comes from research studies documenting that even when attentional resources are rather fully utilized, emotion may still enhance attention. For instance, if a target is emotional in a visual search task, target detection is faster than with a neutral target (Öhman, Flykt, & Esteves, 2001). Anderson (2005) used a paradigm of attentional blink. In this paradigm, visual stimuli are presented in rapid succession at the threshold of conscious visual processing (SOA=80-100ms). A subject is asked to detect two stimuli that are somehow visually distinct from the rest of the stimuli (e.g., by color or category— letters among numbers). Attentional blindness arises when the two to-be-detected visual stimuli are in close temporal proximity, most often when the second target is presented within 100-500ms from the first target. The evidence from study by Raymond, Shapiro, and Arnell (1992) suggests that ‘attention blinks’, i.e., attentional processing of the first target, utilizes resources for a short period of time. However, Anderson (2005) demonstrated that when emotional words are presented, the attentional blink is attenuated, suggesting that the processing of information with emotional significance is not contingent on attentional resources. Similarly, detection of a second target in a rapid serial visual presentation is impaired when the preceding target has negative affective valence (Most, Chun, Widders, & Zald, 2005). Clearly, attention is modulated by emotional information via faster attentional orienting. Specifically, attention is exhausted by and biased toward emotional events. However, this is mostly limited to active search tasks (Frischen, Eastwood, & Smilek, 2008), while emotional events that are task irrelevant impose a greater attentional disengagement from target stimuli, potentially resulting in attentional capture (Eimer & Kiss, 2008), and in a diminished processing of other non-emotional stimuli (Nummenmaa, Hyona, & Calvo, 2006). Vuilleumier and Huang (2009) propose that the increased attention does not have to result from the emotional significance of a stimulus, but rather from a configuration of certain perceptual features of the stimulus. To conclude, from an evolutionary perspective, the facilitation or attenuation of attention and perception by emotion has an adaptive function. A preference for processing stimuli eliciting emotional responses (i.e., stimuli that can cause potential threat) has had a survival value in our ancestors’ past, leading to an automatic attentional orientation to emotional stimuli.

50

4.3 Emotional Memory for Scenes

Each individual probably experiences vivid and perceptually detailed memories that are of an emotional quality or were encoded and consolidated during a very emotionally arousing state. One well-known emotional memory phenomena is flashbulb memories- vivid and almost haunting memories of extremely emotional events that often accompany mental disorders, such as in case of post-traumatic stress disorder. Why is information with emotional significance remembered easily, vividly and is almost immune to forgetting? The following subchapter will discuss the determinants that constitute emotional memory and why emotional memories are powerful and persistent in our cognition. The emotional content of sensory information can impact memory for details concerning that information (Dolcos, LaBar, & Cabeza, 2006; Yegiyan & Yonelidas, 2011). Similar to the effect emotions have on attentional guidance, not surprisingly, emotion affects the way information is remembered or the extent to which the information is remembered. Since the functional field of view for negative events is less extensive than for neutral events, it is rather intuitive to think that memory for central events is improved, while the memory for peripheral details is impoverished. This has been shown in many studies (e.g., Easterbrook, 1959; Libkuman, Stabler, & Otani, 2004; Yegiyan & Yonelidas, 2011). In general, memory for negative information is superior to memory for positive information, which is superior to neutral information. However, recent studies indicate that emotion does not have a uniform effect on memory. Particularly, scene memory is influenced by emotion differently based on the valence and arousal of the scene’s details. Those differential effects have been attributed to the specifics of the encoding stage, i.e., attention span during encoding (Christianson & Loftus, 1991; Easterbrook, 1959; Heuer & Reisberg, 1990), or to post-encoding effects (Waring & Kensinger, 2009).

51

4.3.1 The Arousal Model of Memory Narrowing As already mentioned, one prominent feature or dimension of emotion is arousal, which is believed to be the primary underlying mechanism in memory narrowing (Russell, 2003). In general, the greater the arousal level is, the greater memory narrowing occurs, regardless of valence, according to this account. The central/peripheral trade-off (Kensinger, Garoff-Eaton, & Schacter, 2007) or emotional memory narrowing (Heuer & Reisberg, 1990) pertains to the finding that memory for central details is superior for central events, but attenuated for peripheral events when the arousal level is high. At the physiological level, the arousal level is contingent on activation of the autonomic nervous system and fluctuations of levels of stress hormones (epinephrine and corticosteroids), resulting in changes in heart rate and skin conductance (Kaplan, van Damme, & Levine, 2012). For instance, administration of a cortisol hormone at the encoding phase enhances even long-term memory for emotional events in comparison to neutral ones (Buchanan & Lovallo, 2001). Arousal is particularly important for the encoding of emotional information in that it directs attention to the central information at the expense of peripheral information. The famous “weapon focus” has been shown in studies on eye-witness testimony on crimes, where information about the weapon is recalled in more detail than memory for the perpetrator or other background information (Christianson & Loftus, 1991). It seems that the emotional content of a scene enhances the ability to detect and categorize the gist of the scene (Pan, Sokolich, Lin, Abdel-Ghaffar, & Bishop, 2013). This might be attributed to the evolutionary need for a quick assessment and identification of the threatening environment necessary for successful and secure navigation through a visual space. Furthermore, the narrowing of information processing might be attributed to the evolutionary advantages of blocking out the visual cues irrelevant to survival (Dolcos et al., 2006). In addition, arousal not only enhances encoding, but also the consolidation of information in long-term memory and retrieval of information from memory (Kaplan et al., 2012). The uni-directional focus of attention and subsequent scene memory for central events has been attributed to the impact of arousal on the amygdala and its interaction with parts of the frontal and temporal regions, and those regions have been identified as the underlying cause of construction of persistent emotional memories (Dolcos et al., 2006). The importance of the amygdala in creating emotional memories has been also documented through studies of patients 52

with amygdala lesions whose attention is not as focused on central information as in the normal population (Adolphs, Traner, & Buchanan, 2005). According to this arousal model of memory narrowing, arousal is the primary determinant affecting all memory phases— encoding, consolidation and retrieval. Yet there is evidence that arousal alone cannot explain the narrowing of memory. For instance, through various recognition tasks, in a study by Yegiyan & Yonelinas (2011), subjects were supposed to remember details of a scene and were asked either about central or peripheral information. Both positive and negative arousal led to memory improvement for central events. However, negative arousal resulted in decreased memory for peripheral information, while positive arousal led to an increase in memory for peripheral details.

4.3.2 The Valence Model of Memory Narrowing Another model proposes the relationship between emotion and memory is not exclusive to arousal, but rather to emotional valence. Again, valence is a subjective experience of pleasantness or unpleasantness of a stimulus. This model posits that valence defines the processing of specific information and consequently, the memory for this information. As such, negative emotion enhances memory for specific, local information that is crucial in protecting the organism from threat, whereas positive emotion results in a widening of attention, and thus memory of the gist of an event (scene) (Kensinger, 2009). This is in accordance with Fredrickson’s (2001) Broaden-and-Build theory based on the idea that positive emotion broadens attention and subsequently allows the creation of detailed and robust memory representations, whereas negative emotion narrows attention and memory for those events. To further test the Broaden-and-Build Theory, Fredrickson & Branigan (2005) induced various emotions in participants (amusement, contentment, neutrality, anger, anxiety) through video films. Afterwards, subjects performed a global-local visual processing task. In this task, subjects see a triad of figures and are supposed to judge the similarity of the top figure with two other figures presented below. The similarity can be judged based on the overall shape of the figure (global feature) or based on the characters the each figure comprises of (local features). Participants showed a tendency toward global feature bias when positive emotion was induced in them, and a tendency toward local feature when negative emotion was elicited. 53

Interestingly, the relationship between emotion and cognitive processes seems to be bidirectional, as shown in a study of Srinivasan and Hanif (2010). Being primed with global features processing enhanced the identification of positive facial expressions, while priming with local features enhanced the identification of negative facial expressions.

4.4 Interindividual Differences and Attention Allocation

A clear prioritization of emotional events in information processing is apparent among all individuals regardless of culture or experience. However, the processing of emotional stimuli and attentional biases toward emotional content, do not seem to be determined solely by arousal or emotional valence of a presented stimulus. Other factors that seem to intervene are psychological and socio-cultural factors. For instance, culture determines attentional biases for either verbal or emotional content (Ishii, Reyes, & Kitayama, 2003; Kitayama & Ishii, 2002). Performing the Stroop interference task, Americans had greater problems disregarding verbal information in comparison with Asians who demonstrated trouble ignoring emotional content represented by a vocal tone. Individual differences in personality (especially considering tendencies toward anxiety) also have different effects on the cognitive processing of emotional stimuli. Sufferers from anxiety and other emotional disorders have a tendency toward attentional biases to threat, or critical stimuli. Those attentional biases can be observed for critical words related to panic in individuals with panic disorder, stimuli related to threat for anxious individuals, and phobic stimuli in patients suffering from phobias. For instance, the attentional blink (described above in the chapter) is attenuated for both negative and positive stimuli for every individual, but threatening stimuli like spiders are only detected more frequently by arachnophobes (spider phobes) (Trippe, Hewig, Heydel, Hecht, & Miltner, 2007). A similar effect has been found in inattentional blindness, when arachnophobic participants under limited attentional resources detect spider stimuli much more frequently than normal population (Brailsford, Catherwood, Tyson, & Edgar, in press). Also, fearful stimuli are more frequently perceived by highly anxious

54

participants (Fox, Russo, & Georgiou, 2005) under limited attentional processing (as when presented in attentional blink paradigm), suggesting that highly anxious individuals are unable to disengage attention from threatening stimuli (Yiend & Mathews, 2001). Individuals suffering from social anxiety exhibit attempts to disengage from emotional faces in the context of social evaluation in comparison with non-clinical population (Mansell, Clark, Ehlers, & Chen, 1999). Personality factors have also been shown to interact and influence the cognitive processing of emotional stimuli. As such, people with higher levels of extraversion have greater memory for autobiographical memories with positive valence in comparison with introverted people (Denkova, Dolcos, & Dolcos, 2012). In an eye tracking study, neuroticism was found to positively correlate with time spent looking at the eyes of faces which looked threatening (Perlman et al., 2009). In an inspiring study by Most el al. (2005), participants with higher scores regarding ‘Harm Avoidance’ (propensity to inhibition of aversive stimuli) were more vulnerable to emotion-induced blindness (r= -.59), or the inability to detect a target stimulus, when the preceding stimulus was of a negative emotional nature as opposed to a neutral stimulus. However, when participants were asked to look for a specific target (a horizontally presented building- specific attentional set condition) versus a non-specific target (a horizontally presented image- non-specific attentional set condition), participants with low Harm Avoidance scores showed a reduced magnitude of emotion-induced blindness, but participants with high Harm Avoidance scores seemed unaffected, and their performance was equally bad in both conditions. The results suggest that preferential processing or an attentional strategy used in processing of emotional stimuli is contingent on temperamental predispositions. However, it is questionable whether the manipulation of an attentional set reflects the level of volitional control over one’s attention. Furthermore, temperamental influences on the processing of emotional pictures were observed via measures of skin conductance (Mardaga, Laloyax, & Hansenne, 2006). Specifically, Harm Avoidance interacted with skin conductance responses (GSR) to emotional stimuli (longer recovery times were found for persons with higher scores of Harm Avoidance indicating prolonged emotion dissipation). Another personality dimension, ‘Novelty Seeking’, was associated with greater autonomous responses to emotional stimuli (Yoshino, Kimura, Yoshida, Takahashi, & Nomura, 2005). In accordance with the previous studies, low Harm Avoidance levels were found to be predictive of less attentional allocation to negative emotional stimuli and preferential attention to positive information (Mardaga & Hansenne, 2009). 55

Interestingly, the effect of temperament on the interaction between attention and emotion seems to develop in late childhood, as the correlation between temperament and attentional biases to emotional faces seen in adults were not observed for children between 3-9 years of age (Shade, 2010).

4.5 Summary

Although research demonstrates that the relationship between attention and emotion is bidirectional, the main goal of this chapter was to provide an introduction of how attention is modulated by either exposure to an emotional stimuli or the inducement of an emotional state or even by personality traits pertaining to predispositions to certain emotional reactions. All in all, both external and internal emotional cues influence attentional allocation such that emotional cues have a prioritized status in attentional processing. In the case of an emotional stimulus, its valence, as well as the arousal it elicits in an individual and motivation direction, determines subsequent attentional effects. Various models represent the importance of each of those factors in the effects emotion has on cognition. To conclude, the “pedestal-status” of emotion in the automatic grabbing of attention has an evolutionary value, since it resulted in greater survival and fitness of individual species.

56

References: Adolphs, R., Tranel, D., & Buchanan, T. W. (2005). Amygdala damage impairs emotional memory for gist but not details of complex stimuli. Nature Neuroscience, 8, 512–518. Aggleton, J. (1992). The amygdala: neurobiological aspects of emotion, memory, and mental dysfunction. New York: Wiley-Liss. Anderson, A. K. (2005). Affective influences on the attentional dynamics supporting awareness. Journal of Experimental Psychology: General, 134, 258 –281. Berman, M. G., Jonides, J., & Kaplan, S. (2008). The cognitive benefits of interacting with nature. Psychological Science, 19, 1207–1212. Brailsford, R., Catherwood, D., Tyson, P. J., & Edgar, G. (in press). Noticing spiders on the left: Evidence on attentional bias and spider fear in the inattentional blindness paradigm. Laterality: Asymmetries of Body, Brain, and Cognition. Brunye, T. T., Mahoney, C. R., Augustyn, J. S., & Taylor, H. A. (2009). Emotional state and local versus global spatial memory. Acta Psychologica, 130, 138-146. Buchanan, T. W., & Lovallo, W. R. (2001). Enhanced memory for emotional material following stresslevel cortisol treatment in humans. Psychoneuroendocrinology, 26(3), 307-317. Christianson, S. A., & Loftus, E. F. (1991). Remembering emotional events: The fate of detailed information. Cognition & Emotion, 5, 81-108. Cummins, D. D., & Cummins, R. (1999). Biological preparedness and evolutionary explanation. Cognition, 73, B37-B53. Davelaar, E. J., Goshen-Gottstein, Y., Ashkenazi, A., Haarmann, H. J., & Usher, M. (2005). The demise of short-term memory revisited: Empirical and computational investigations of recency effects. Psychological Review, 112, 3-42. Denkova, E., Dolcos, S., & Dolcos, F. (2012). Reliving emotional personal memories: Affective biases linked to personality and sex-related differences. Emotion, 12, 515–528. Derryberry, D., & Tucker, D. M. (1994). Motivating the focus of attention. In P. M. Niedenthal & S. Kitayama (Eds.), The heart’s eye: Emotional influences in perception and attention (pp. 167– 196). San Diego: Academic Press. Dolcos, F., LaBar, K. S., & Cabeza, R. (2006). The memory enhancing effect of emotion: Functional neuroimaging evidence. In B. Uttl, N. Ohta, & A. L. Siegenthaler (Eds.), Memory and emotion: Interdisciplinary perspectives (pp. 107-133). Malden, MA: Blackwell Publishing. Easterbrook, J. A. (1959). The effect of emotion on cue utilization and the organization of behavior. Psychology Review, 66, 183-201.

57

Eimer, M., & Kiss, M. (2008). Involuntary attentional capture is determined by task set: Evidence from event-related brain potentials. Journal of Cognitive Neuroscience, 20, 1423-1433. Elliot, A. J., & Covington, M. V. (2001). Approach and avoidance motivation. Educational Psychology Review, 13, 73-92. Feldman Barrett, L. (1998). Discrete emotions or dimensions? The role of valence focus and arousal focus. Cognition & Emotion, 12, 579–599. Fernandes, M. A., Koji, S., Dixon, M. J., & Aquino, J. M. (2011). Changing the focus of attention: the interacting effect of valence and arousal. Visual Cognition, 19 (9), 1191-1211. Fox, E., Russo, R., & Georgiou, G. A. (2005). Anxiety modulates the degree of attentive resources required to process emotional faces. Cognitive & Affective Behavioral Neuroscience, 5, 396–404. Fredrickson, B. L. (1998). What good are positive emotions? Review of General Psychology, 2, 300–319. Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56, 218–226. Fredrickson, B. L., & Branigan, C. (2005). Positive emotions broaden the scope of attention and thought– action repertoires. Cognition & Emotion, 19, 313–332. Frischen, A., Eastwood, J. D., & Smilek, D. (2008). Visual search for faces with emotional expressions. Psychological Bulletin, 134, 662–676. Gable, P. A., & Harmon-Jones, E. (2008). Approach-motivated positive affect reduces breadth of attention. Psychological Science, 19, 476–482. Gable, P. A., & Harmon-Jones, E. (2010). The blues broaden, but the nasty narrows: Attentional consequences of negative affects low and high in motivational intensity. Psychological Science, 21, 211-215. Heuer, F., & Reisberg, D. (1990). Vivid memories of emotional events: The accuracy of remembered minutiae. Memory & Cognition, 18, 496-506. Ishii, K., Reyes, J. A., & Kitayama, S. (2003) Spontaneous attention to word content versus emotional tone: Differences among three cultures. Psychological Science, 14, 39–46. Jiang, J., Scolaro, A. J., Bailey, K., & Chen, A. (2011). The effect of music-induced mood on attentional networks. International Journal of Psychology, 46(3), 214-222. Kaplan, R. L., Van Damme, I., & Levine, L. J. (2012). Motivation matters: Differing effects of pre-goal and post-goal emotions on attention and memory. Frontiers in Emotion Science, 3, 404. Kensinger, E. A. (2009). Remembering the details: Effects of emotion. Emotion Review, 1, 99–113. Kensinger, E. A., Garoff-Eaton, R. J., & Schacter, D. L. (2007). Effects of emotion on memory specificity: memory trade-offs elicited by negative visually arousing stimuli. Journal of Memory and Language, 56, 575–591.

58

Kitayama, S., & Ishii, K. (2002). Word and voice: Spontaneous attention to emotional speech in two cultures. Cognition & Emotion, 16, 29–59. Lang, P. J. (2000). Emotion, and motivation: Attention, perception, and action. Journal of Sport & Exercise Psychology, 20, S122-S140. Lazarus, R. S., (1991). Emotion and adaptation. New York: Oxford University Press LeDoux, J. E. (2000). Emotion circuits in the brain. Science, 286, 2504–2507. Libkuman, T. M., Stabler, C. L., & Otani, H. (2004). Arousal, valence, and memory for detail. Memory, 12(2), 237-247. Mansell, W., Clark, D. M., Ehlers, A., & Chen, Y. P. (1999). Social anxiety and attention away from emotional faces. Cognition & Emotion, 13(6), 673-690. Mardaga, S., & Hansenne, M. (2009). Personality modulation of P300 wave recorded within an emotional oddball protocol. Clinical Neurophysiology, 39, 41-48. Mardaga, S., Laloyaux, O., & Hansenne, M. (2006). Personality traits modulate skin conductance response to emotional pictures: An investigation with Cloninger's model of personality. Personality & Individual Differences, 40(8), 1603-1614. Martin, E. A., & Kerns, J. G. (2011). The influence of positive mood on different aspects of cognitive control. Cognition & Emotion, 25(2), 265-279. Mathews, A., & MacLeod, C. (2002). Induced processing biases have causal effects on anxiety. Cognition & Emotion, 16, 331–354. McRae, K., Misra, S., Prasad, A. K., Pereira, S. C., & Gross, J. J. (2011). Bottom-up and top-down emotion generation: Implications for emotion regulation. Social, Cognitive & Affective Neuroscience, 7(3), 253-262. Most, S. B., Chun, M. M., Widders, D. M., & Zald, D. H. (2005). Attentional rubbernecking: Cognitive control and personality in emotion-induced blindness. Psychonomic Bulletin & Review, 12, 654 – 661. Nummenmaa, L., Hyona, J., & Calvo, M. (2006). Eye movement assessment of selective attentional capture by emotional pictures. Emotion, 6, 257–268. Öhman, A., Flykt, A., & Esteves F. (2001). Emotion drives attention. Journal of Experimental Psychology: General, 130, 466–478. Pan, T. S, Sokolich, A. T., Lin, P., Abdel-Ghaffar, S., & Bishop, S. J. (2013). Rapid semantic categorization of scenes: an advantage for emotional images? Paper presented at Vision Sciences Society Conference. Naples, FI. Perlman, S. B., Morris, J. P., Vander Wyk, B. C., Green, S. R., Doyle, J. L., et al. (2009). Individual differences in personality predict how people look at faces. PLoS ONE, 4(6), e5952.

59

Prause, N., Janssen, E., & Hetrick, W. (2008). Attention and emotional responses to sexual stimuli and their relationship to sexual desire. Archives of Sexual Behavior, 37(6), 934-959. Raymond, J. E., Shapiro, K. L., & Arnell, K. M. (1992). Temporary suppression of visual processing in an RSVP task: An attentional blink? Journal of Experimental Psychology: Human Perception & Performance, 18(3), 849-860. Rowe, G., Hirsh, J. B., & Anderson, A. K. (2007). Positive affect increases the breadth of attentional selection. Proceedings of the National Academy of Sciences of the United States of America, 104, 383–388. Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172. Shade, S. K. (2010). Temperament and its effects on visual attention to emotional faces (Unpublished Master Thesis). Dietrich College of Humanities and Social Sciences at Carnegie Mellon University, Pittsburgh, PA. Srinivasan, N., & Hanif, A. (2010). Global-happy and local-sad: Perceptual processing affects emotion identification. Cognition & Emotion, 24(6), 1062-1069. Tooby, J., & Cosmides, L. (1990). The past explains the present: Emotional adaptations and the structure of ancestral environments. Ethology and Sociobiology, 11, 375-424. Trippe, R. H., Hewig, J., Heydel, C., Hecht, H., & Miltner, W. H. R. (2007). Attentional blink to emotional and threatening pictures in spider phobics: Electrophysiology and behavior. Brain Research, 1148, 149 – 160. Vuilleumier, P. (2005). How brains beware: Neural mechanisms of emotional attention. Trends in Cognitive Sciences, 9, 585–594. Vuilleumier, P., & Huang, Y. M. (2009). Emotional attention: Uncovering the mechanisms of affective biases in perception. Current Directions in Psychological Science, 18(3), 148152. Waring, J. D., & Kensinger, E. A. (2009). Effects of emotional valence and arousal upon memory tradeoffs with aging. Psychology & Aging, 24(2), 412– 422. Whalen, P. J. (1998). Fear, vigilance, and ambiguity: initial neuroimaging studies of the human amygdala. Current Directions in Psychological Science, 7, 177-188. Yegiyan, N. A., & Yonelinas, A. P. (2011). Encoding details: Positive emotion leads to memory broadening. Cognition & Emotion, 25 (7), 1255-1262. Yiend, J., & Mathews, A. (2001). Anxiety and attention to threatening pictures. Quarterly Journal of Experimental Psychology, 54, 665–681. Yoshino, A., Kimura, Y., Yoshida, T., Takahashi, Y., & Nomura, S. (2005). Relationships between temperament dimensions in personality and unconscious emotional responses. Biological Psychiatry, 57, 1–6.

60

Chapter 5: Introduction to the Experimental Part

5.1 Research Identification

Our perception of what we see is highly contingent on visual perception, and visual modality has remarkable significance for human perception and behavior. In the process of selection of visual information, visual attention has a key role in the processing of visual sensory input, as it allows conscious processing of information. Attending to specific visual information engenders a deeper processing of that information in comparison to visual information that is outside of the focus of visual attention. Multiple factors determine which information will be prioritized in visual processing, and thus attended to. For instance, purely physical attributes of visual information or low-level determinants may attract visual attention by having great visual saliency. Next to those low-level factors, attention allocation throughout the visual environment, as discussed in the previous chapters, is determined by several high-level factors. Both experimental studies aim at understanding three different highlevel determinants of visual attention: high-level scene factors (Experimental Study I), emotion, and personality (Experimental Study I). The main motivation for the study was the rather underexplored area of research that the role of high-level factors on visual attention represents.

5.2 Research Objectives: Experimental Study I

The determinants of visual attention allocation that we look at in Experimental Study I are high-level scene factors. An individual, through repeated encounters and exposure to certain types of information and occurrences of events or actions, learns and acquires knowledge about what constitutes the sensory world. Those expectations, or perceptual knowledge, produce 61

predictions that are employed in scene perception and guide attention to those locations with greatest relevance and are sources of the most relevant information. The first experimental study employed a change detection task to induce change blindness in order to observe the effect of high-level scene factors on the magnitude of change blindness. The main research questions included: Do high-level scene factors affect the magnitude of change blindness? What high-level scene factors are the most relevant in guiding visual attention? The high-level factors that were employed in our study were as following: probability of a change, relevance of the change to the scene context, location of the change (centrality versus marginality) within the scene, and location of the change within the figure or ground. The Experimental Study I is presented as Chapter 6.

5.3 Research Objectives: Experimental Study II An evolutionary account concerning emotion posits that valuable and survival-relevant environmental cues and information should yield preferential attentional processing. The role of emotion on cognition has only recently started enjoying increasing scientific interest. The cognitive response to emotional events may however be modulated by innate behavioral patterns, such as irritability or sensitivity to emotional events (temperament). Experimental Study II explored those rather neglected high-level determinants of visual attention, specifically emotion and personality. Chapter 4 introduced the role of emotion in visual attention and visual memory and highlighted the processing prioritization of emotional information. Furthermore, the contribution of inter-individual differences, invariant personality traits in particular, to visual attention allocation were discussed. So far, only few studies have explored those issues. Experimental Study II, or Chapter 7, employed change detection tasks involving images with varying levels of emotional arousal and valence to explore the effect of various emotional scene contexts on attentional allocation. Individuals’ current emotional states were measured via PANAS (Positive and Negative Affect Schedule), and personality traits were measured via TCI (Temperament and Character Inventory), in order to see the relationship between those two variables and attention allocation. 62

Chapter 6: Experimental Study I The Role of High Level Scene Factors in Change Blindness 6.1 Introduction: Research Rationale

When perceiving scenes in real life environments, we experience a feeling of having a rich visual representation of the world outside. A change in our visual field is accompanied by a motion signal or another transient event that automatically results in our attention being redirected to this change (Klein, Kingstone, & Pontefract, 1992) and the change is easily detected. What happens if a change occurs but this motion signal is absent? Many studies show that an observer in this situation has a difficulty detecting the change. The term ‘change blindness’ was coined to describe a failure in noticing even remarkable changes in a scene. Change blindness can be assured by a disruption of the visual continuity in a scene by presenting a change during eye movement (Bridgeman, Hendry, & Stark, 1975; Hollingsworth, Schrock & Henderson, 2001; Pashler, 1988), by inserting a brief blank in between scenes (Fernandez-Duque et al., 2003; McCarley et al., 2004; Rensink, O’Regan, & Clark, 1997, 2000; Richard et al., 2002; Scholl, 2000; Simons & Rensink, 2005), by using “mudsplashes” (O’Regan, Rensink, & Clark, 1999), by presenting a gradual change (Simons, Franconeri, & Reimer, 2000), or by ‘movie cuts’ (Levin & Simons, 1997; Simons, 1996). Over the last decade, the phenomenon of change blindness has helped generate a better understanding of visual perception, the role of attention in conscious perception, and the nature of representations and memory. Specifically, recent findings on change blindness have challenged the traditional view of the nature of visual representations, demonstrating that the belief in the richness of our visual experience is illusory or inaccurate (Blackmore et al., 1995; Levin et al., 2000). Many alternative hypotheses concerning the nature of visual representations 63

have been proposed based on the results of change blindness research. According to an extreme view, we do not form representations of the visual field at all (Noë, 2009; Noë et al., 2000). Similarly, a less extreme idea posits that we do not retain or represent the original attributes of the changed object (Dennett, 1991), or that only a very limited portion of visual information is retained through successive views of a scene made during saccades (Grimes, 1996; McConkie & Currie, 1996; McConkie & Zola, 1979) or in a saccade-simulating technique called the ‘flicker technique’ (Rensink et al., 1997). Other authors suggest that a representation of the visual scene is created, but change blindness occurs because the memory trace is very volatile (Wolfe, 1999), and it is replaced by a post-change presented stimuli (Rensink et al., 1997; Simons, 2000). Others posit that a representation of a change occurs but is not overly reportable; however, this implicit change detection can be manifested in subsequent behavior (Fernandez-Duque & Thorton, 2000, 2003; Laloyaux, Destrebecqz, & Cleeremans, 2006). In terms of change blindness research, rather little attention has been paid to stimulus properties and their effect on inducing change blindness (although there are studies that tried to explore this issue, e. g., Beck, Angelone, & Levin, 2004; Mazza, Turatto, & Umilta, 2005; O’Regan, Rensink, & Clark, 1999; Rensink et al., 1997; Turatto et al., 2002). Considering the great diversity in scenes and stimulus (change) properties, we assume that the inducement of change blindness for a particular kind of change is not random. Therefore, not all changing stimulus properties will result in the same effect on the inability to detect changes occurring in a scene. Research on eye movements has shown that changes tend to be observed when prechange and post-change regions are fixated (thus attended to) (Henderson & Hollingworth, 1999; Hollingworth & Henderson, 2002). In accordance with this view are the results of Rensink et al. (1997), which illustrate that changes occurring in regions of high interest (i.e., changes in objects listed in a pre-testing verbal description of the scene by participants) are detected faster than changes in regions of low interest. It means that regions of high interest were being attended to more frequently. Based on this research, it can be concluded that focused attention plays a tremendous role in successful (i.e., reportable and correct) change detection. Turatto et al. (2002) examined change detection for simple geometric figures using a ‘one-shot’ paradigm, where change detection was highly preferential for the figure. Mazza et al. (2005) found that changes in 64

the background were detected only when attention was focused on them, whereas changes in the figure were invariably detected. Not only is the localization of the change crucial for the speed and accuracy of change detection, but also the nature of change. For instance, Hollingsworth and Henderson (2000) showed that detection of semantically inconsistent objects in flicker task scenes is faster and more accurate than for semantically consistent objects. The results indicate that we do concoct some kind of visual representation when perceiving scenes, although it can be very poor and schematic. Supported by the current research on scene perception, the nature of scene representations, therefore, does not seem to be detailed and picture-like. However, the assumption that we do not create representations at all is somewhat misleading and oversimplified. Drawing from ample research evidence, we believe in a middle ground, meaning that out of all scene information, important scene information will be predominantly encoded and represented. The determination of which information will be considered important by the human visual system will depend both on high-level and low-level factors. It is clear that low-level scene factors such as hue, illumination, color, orientation, edges, and movement play crucial roles in attracting attention. In active tasks (such as change detection), high-level scene factors in particular influence attention allocation toward important parts of scenes (Henderson, 2003; Henderson & Hollingworth, 2002). The aim of the research study was to understand the high-level scene determinants which modulate attention in change detection tasks. For the purposes of our experiment, we included the following high-level change properties: (i) central/marginal changes, (ii) changes concerning the figure/background relationship, (iii) the relevance of the change to the scene context, and (iv) the probability of a change.

Centrality/marginality. The changes in this category were divided into central or marginal—dependent on the degree of interest of the scene region in which a change occurred (similar to Rensink et al., 1997). However, contrary to Rensink et al. (1997), who divided the changes based on the frequency of objects mentioned in the verbal description, we decided to have four independent researchers evaluate each change as being central or marginal. This

65

evaluation was based on the localization of a change, i.e. how centrally or marginally the change was located.

Relevance of a change to the scene context. When performing a specific goal-oriented task (i.e., being influenced by top-down processing), attention is guided toward relevant visual stimuli while ignoring any distracters presented (Engle, 2002; Engle, Kane, & Tuholski, 1999; Wolfe, 1994). Information irrelevant to the scene’s context should therefore be disregarded, and information relevant to the context will be prominent in the scanning of the scene. We hypothesize that this tendency will result in the faster detection of context-relevant changes than context-irrelevant changes (e.g., a phone being removed from a scene where a person is talking on the phone versus a scarf being removed from the talking person’s neck).

Probability of the change. Phenomena such as boundary extension and representational momentum demonstrated that perception is not momentary but is focused on a prediction of future (Intraub, 2002; Munger et al., 2005). We hypothesize that in change detection tasks one’s expectations of what kind of change could possibly occur are important for successful change detection. Therefore, attention would be allocated to those changes considered most probable. Based on this, we believe that when presented with a natural scene, a tendency to predict the future outcome plays a role in scene perception and probable changes would be detected faster than improbable ones for this reason.

Figure-ground localization. The localization of a change into a central or marginal region of a scene is not often always identical with figure-ground segmentation, whereas the latter consists in the formation of perceptual units based on the greater saliency for the figure regardless of its visual saliency properties (Rubin, 1921). However, both the figure and the central region of the scene are considered to be the initial points of scanning. We hypothesize that changes occurring in the figure will, for this reason, be detected faster than changes occurring in the background. In terms of the figure-ground localization category, we were inspired by a study by Turrato et al. (2002), and Mazza et al. (2005) who used the one-shot paradigm for simple stimuli. We, however, used naturalistic scenes, hypothesizing that change blindness will be more attenuated for background changes than figure changes. We formulated 66

one more subcategory within the figure-ground category. Intuitively, we believe that regions around a figure but considered to be a part of background should be especially interesting for attention allocation due to their ambiguous nature. They surround the most salient region, i.e., figure, but might be somewhat uninformative, since both figure and background provide us with more information, such as spatial relationships between the figure and other objects in the background. How might this affect change detection?

6.2 Method

6.2.1 Participants 8 (5 female and 3 male) volunteers with normal to corrected-to-normal vision participated in the study (mean age 21 years; range 19-26 years).

6.2.2 Procedure In the experiment, the original flicker paradigm designed by Rensink et al. (1997) was used. The method consists of the subject viewing an alternating sequence of two images (original and modified) and a blank screen (interjected between the two images), until the change between the two images is found. Each image was presented for 240 msec, with the blank field presented for 80 msec. The various image sets used were presented in a random order. The experiment was conducted using the experimental software PXLab (Irtel, 2007), and it was conducted in an adequately illuminated room. The images were displayed on a 17-in. desktop (Fujitsu Siemens) computer screen (resolution 800 x 600 pixels, 75 Hz) subtending visual angles of 24° (horizontal) and 18° (vertical) at a viewing distance of 50 cm. All participants were tested individually. Without revealing the gist of the study, the experimenter explained the nature of the experiment to the participants (a change detection experiment), and described how they would perform the task. The training consisted of two sample items (the sample items were not a part of the experimental set). After the completion of the training, the experimental portion was initiated. The participants were instructed to click on

67

the location of the change by means of mouse as soon as they perceived the change between the two alternating images. If a participant was not able to detect the change within the given time (the alternation between the two images continued for a minute), the images then disappeared and the presentation of new set of images was initiated. Reaction time as well as location and identification of the change were recorded. The only type of change presented was the disappearance of an object or a part of the scene from the image. The changes that occurred between the original and modified image could be described as obvious when not presented with the scene disruption (as in the flicker technique). In this case, the participants were able to notice the change immediately. In total, 30 images in each change category were presented. In each image a change classified according to a specific category (e.g., central-marginal) was presented. In each set, an equal number of changes from each subcategory was presented. The low-level properties of the changes, as determined by luminance, hue, and size, were equalized. The assignment of changes into each subcategory was executed by four experts and was included into the experiment only when all the experts assigned the change to the same category (see Table 1 for descriptions for various changes).

Table 1: Individual change categories used in the experiment.

Change

Centrality/

Category

Marginality

Context Relevance

Probability

Figure / Ground

Central Occurring within the central scene region

Relevant Change important for maintaining the scene context- i.e., change changes scene context

Probable Change that could possibly occur in a real world environment

Figure Change within a dominant element of the scene regardless of its localization/centrality

Marginal Occurring outside of the central scene region

Irrelevant Change does not change scene context

Improbable Change that could not occur in the real world

Ground Change outside of the dominant element of the scene

68

6.2.3 Stimuli 30 pairs of photographed outdoor and indoor naturalistic scenes were selected for the experiment. Each image could be characterized as possessing a certain amount of objects (comparable number) as well as a context. The two images in every image pair featured only a single change. All objects, as well as the context that the objects were part of, were typical for the scenes presented. The size of the images was 27° wide and 18° high (or vice versa). In all cases, the changes were quite large and easy to see once noticed. Also, all changes involved the disappearance of an object/ region of the scene.

6.3 Results

6.3.1 Percentage of Detected Trials 6.3.1.1 Descriptive Statistics Trials in which participants were not able to respond within 60 s were excluded from the study (as in the original experiment by Rensink et al. (1997)). Altogether, 6 trials were excluded. However, the following percentages of detected trials (detection within 60 s limit) in terms of change categories were observed: For the centrality/ marginality condition: 100% of central changes were detected; 93% of marginal changes were detected. For the relevance of the change condition: 99% of relevant changes were detected, whereas 94% of irrelevant changes were detected. For the probability condition: 100% of the probable changes were detected, in comparison to 95% of improbable changes. For the figure/background condition: 100% of changes occurring within the figure were detected within 60 s; 95% of changes within the background and 91% of changes within close proximity to the figure were detected.

69

Reaction time analysis: The reaction times on the detected trials were computed using Analysis of Variance.

Table 2: Values for individual subcategories of changes, M(SD) in seconds.

Mean

Standard Deviation

Central

6.39

3.97

Marginal

14.56

7.77

Relevant

4.09

2.65

Irrelevant

12.91

7.26

Probable

5.02

2.16

Improbable

13.35

7.81

Figure

4.33

2.60

Ground

12.09

7.27

Figure proximity

14.08

7.75

Category Centrality/ Marginality

Context Relevance

Probability

Figure / Ground

6.3.1.2 Change Detection Accuracy The central changes were detected much faster than the marginal: F(1,179) = 17,701, p < .001, M C = 6.39, M M = 14.56. The results are consistent with the report of Rensink et al. (1997), who found a preference for central changes in change detection tasks. This might be attributed to the fact that the central region is the most salient region, and it is also the primary 70

focus of attention. Relevant changes were identified in a shorter time period than irrelevant changes: F(1,179) = 16,635, p < .001, M R = 4.09, M Ir = 12.91. Hollingsworth and Henderson (2000) and Loftus and Mackworth (1967) demonstrated that inconsistent objects in scenes are detected sooner and remembered better than consistent ones. As in the case of inconsistent objects, information relevant to scene context is important for understanding the scene context. Therefore changes in this information were found to be detected earlier. Furthermore, probable changes were detected faster than improbable changes: F(1, 178) = 6.66, p < .001; M P = 5.02, M Im = 13.35. The result is consistent with other research (Beck et al., 2004) who found that probable changes are detected faster than improbable ones. Finally, changes in the figure were detected faster than changes in the background. However, changes occurring in near proximity of figure were detected, on average, even slower than changes in the background: F(2, 178) = 7,59, p < .001; M F = 4.33, M B = 12.09, M Pf = 14.08. The last two results will be further interpreted in the discussion (see Table 2 and Figure 3 for detailed results).

Figure 2: Change detection for various change categories (M, SE).

71

6.4 Discussion and Conclusion

Our experiment was motivated by several facts. First of all, the discovery of the phenomenon called change blindness illustrates that even important and remarkable visual changes can go unnoticed. Secondly, scene perception research shows that the human visual system has a tendency to attend predominantly to informative and important parts of a scene (Buswell, 1935; Henderson, 2003; Loftus & Mackworth, 1978; Yarbus, 1967). Lastly, in active tasks such as change detection, attending to information with high visual saliency (i.e., spatial frequency, color, and intensity) can be inhibited (Henderson, 2003; Henderson & Hollingworth, 2003; Turano, Geruschat, & Baker, 2003), therefore attention is rather dependent on high-level factors (Beck et al., 2004; Hollingsworth & Henderson, 2000; Rensink et al., 1997). Taken together, one could hypothesize that high-level scene factors determine which parts of scene are important and which are unimportant, and this distinction will influence to what extent the changes in those regions will be detected. The focus of the present study was to explore which high-level scene information is considered informative and important by the human visual system. We believe that real-world scene perception can provide us with a better understanding of visual system functioning due to their higher ecological validity, since coherent images (i.e., natural scenes) seem to be processed by the visual system more naturally (Biederman, 1972; Biederman, 1981; Henderson, 2003; Rensink, 2000). In the experiment, observers inspected various real-world scene images alternating in sequential order (i.e., flicker task) and were asked to click on the location where a change occurred once they perceived it. The results demonstrate that not all changes present in a scene are of the same likelihood to be detected. The changes that seem to be dominant in attracting attention were found to be changes occurring in the central region, context-related changes, probable changes and changes located within figure. Our results are consistent with the results of other studies. For instance, research on scene perception has shown that changes in central regions are detected faster than in marginal regions (Rensink et al., 1997), changes inconsistent with the scene context are detected faster than those consistent with the scene context

72

(Hollingsworth & Henderson, 2000), probable changes faster than improbable ones (Beck et al., 2004), changes in the figure are detected to a greater extent than ones in the background (Mazza et al., 2005; Turatto et al., 2002). Therefore, the individual results in our experiment are not very surprising. On the other hand, this study offers extensive comparison of various high-level scene factors in their influence of change detection. In terms of change blindness research, we consider the results in the category figure-background and probability-improbability especially interesting.

Figure-background. Each image is automatically segmented by the visual system into figure and background; where the figure is the most salient region of scene regardless of its visual saliency (Wertheimer, 1923), and therefore the observer’s initial attention is devoted to an inspection of the figure. Mazza et al. (2005) reported results of change blindness for stimuli in the foreground versus the background, which suggests that the figure is the primary target of attentional focus by default. It is therefore not very surprising that in our experiment, change detection showed preference for figure changes in comparison with background changes. On the contrary, rather surprising is the finding that participants needed, on average, even more time for the detection of changes presented in near-proximity to the figure. How could one interpret this finding? As has been already mentioned, the figure is the primary target of our attention. However, once one´s gaze is directed onto the figure and no change is found, top-down factors (i.e., expectations of an individual) become dominant in gaze guidance. As an individual might believe that the region of the figure was sufficiently inspected, she will deploy her attention outside of the figure, directing attention to other parts of the scene in background. However, in the case that no change is found in background, she will return to exploration of figure at last. Thus, we believe that this heuristic in gaze guidance in change detection task might help to explain this pattern of data.

Probability. We found that probable changes were detected faster than improbable changes. We suppose that this could be explained by the fact that when perceiving real-world images, expectations derived from knowledge play an important role in the encoding and memorizing of information. In change detection tasks, attention should, therefore, be deployed toward the stimuli that are believed to have properties that can or are able to be changed in 73

contrast with changes that do not occur in real-life environment (rf. Beck et al., 2004). For the change detection paradigm, we believe that this category is especially important. Change detection is deemed an active task in which top-down influence is particularly dominant (Henderson, 2003; Henderson & Hollingworth, 2003), therefore subjects’ expectations about possibly occurring changes can play an essential role in directing attention. It is important to mention that changes presented within this category were not so peculiar (so attention would be primarily attracted by this ‘peculiarity’ occurring in the scene). We believe that rather subjects’ expectations were the influencing factor of attentional guidance. In this aspect, the design of our experiment differed from Loftus and Mackworth (1978), and Hollingsworth and Henderson (2000) who showed that objects that are not consistent with the scene context or do not belong to the scene context are remembered much more that consistent ones.

6.5 Summary

In general, not all changes will induce change blindness (even remarkable ones). The inducement of change blindness is also contingent on several non-visual, semantic properties that the particular change possesses. Based on this idea, one can interpret many results of previous change (and inattentional) blindness studies. For instance, when watching a movie cut by Levin and Simons (1997), people failed to notice a change in a person’s identity as well as clothing. However, this change has certain attributes that decrease the likelihood that people will detect it. Is this change probable? Would this change occur in normal circumstances (i.e., in normal life)? How important is this change for the context presented in the movie cut? The actor can be identified as a tall, trim, good-looking, glasses-wearing, blond-headed man, and those attributes do not change remarkably. We suppose that if a more probable change occurred, for instance the man would discard his glasses, it would be more easily detected. The article discusses the significance of the categories under which a stimulus falls in terms of its non-visual or semantic properties. We agree with the idea of visual representations 74

being rather sparse, not detailed, but we also believe that certain properties are represented. The wealth of visual information is dependent on both visual and non-visual properties, but in realworld scene perception especially, attention is heavily biased by non-visual and semantic properties. Active tasks, such as change detection, are strategic and contingent on higher-level scene components properties (e.g., objects’ properties). Those properties will have priority in the process of being represented. It could be contested that the real-world environment does not engage the visual system in tasks such as intentional change detection. Regardless of this fact, we believe that studies on change blindness can reveal invaluable information about the way the visual system codes, utilizes, and retains information through successive fixations.

75

References: Beck, M. R., Angelone, B. L., & Levin, D. T. (2004). Knowledge about the probability of change affects change detection performance. Journal of Experimental Psychology: Human Perception and Performance, 30(4), 778–791. Biederman, I. (1972). Perceiving real world scenes. Science, 177, 77–80. Biedermann, I. (1981). On the semantics of a glance at a scene. In J. M. Kubovy & J. R. Pomerantz (Eds.), Perceptual organization (pp. 213-253). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Biederman, I., Mezzanotte, R. J., & Rabinowitz, J. C. (1982). Scene perception: Detecting and judging objects undergoing relational violations. Cognitive Psychology, 14, 143–177. Blackmore, S. J., Brelstaff, G., Nelson, K., & Troscianko, T. (1995). Is the richness of our visual world an illusion? Transsaccadic memory for complex scenes. Perception, 24, 1075–1081. Bridgeman, B., Hendry, D., & Stark, L. (1975). Failure to detect displacement of the visual world during saccadic eye movements. Vision Research, 15, 719-722. Buswell, G. T. (1935). How People Look at Pictures. Chicago: University of Chicago Press. Castelhano, M. S., & Henderson, J. M. (2008). The influence of color on the activation of scene gist. Journal of Experimental Psychology: Human Perception and Performance, 34, 660–675. Dennett, D. (1991). Consciousness explained. Boston: Little Brown. Engle, R. W. (2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11, 19-23. Engle, R. W., Kane, M. J., & Tuholski, S. W. (1999). Individual differences in working memory capacity and what they tell us about controlled attention, general fluid intelligence, and functions of the prefrontal cortex. In A. Miyake & P. Shah (Eds.), Models of working memory: Mechanisms of active maintenance and executive control (pp. 102–134). Cambridge, UK: Cambridge University Press. Fernandez-Duque, D., & Thornton, I. M. (2000). Change detection without awareness: Do explicit reports underestimate the representation of change in the visual system? Visual Cognition, 7(1-3), 323344. Fernandez-Duque, D., & Thornton, I. M. (2003). Explicit mechanisms do not account for implicit localization and identification of change: An empirical reply to Mitroff et al. (2002). Journal of Experimental Psychology-Human Perception and Performance, 29(5), 846-858. Fernandez-Duque, D., Grossi, G., Thornton, I. M., & Neville, H. J. (2003). Representation of change: Separate electrophysiological markers of attention, awareness, and implicit processing. Journal of Cognitive Neuroscience, 15(4), 491-507. Grimes, J. (1996). On the failure to detect changes in scenes across saccades. In K. Akins (Ed.), Perception (pp. 89–109). New York: Oxford University Press. 76

Henderson, J. M. (2003). Human gaze control in real-world scene perception. Trends in Cognitive Sciences, 7 (11), 498-504. Henderson, J. M., & Hollingworth, A. (1999). The role of fixation position in detecting scene changes across saccades. Psychological Science, 10, 438-443. Hollingworth, A., & Henderson, J. M. (2000). Semantic informativeness mediates the detection of changes in natural scenes. Visual Cognition (Special Issue on Change Blindness and Visual Memory), 7, 213-235. Hollingworth, A., & Henderson, J. M. (2002). Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception and Performance, 29, 388-403. Hollingworth, A., Schrock, G., & Henderson, J. M. (2001). Change detection in the flicker paradigm: The role of fixation position within the scene. Memory and Cognition, 29, 296–304. Intraub, H. (2002). Anticipatory spatial representation of natural scenes: momentum without movement? Visual Cognition, 9, 93-119. Irtel, H. (2007). PXLab: The Psychological Experiments Laboratory [online]. Version 2.1.11. Mannheim (Germany): University of Mannheim. Klein, R. M., Kingstone, A., & Pontefract, A. (1992). Orienting of visual attention. In K. Rayner (Ed.), Eye movements and visual cognition: Scene perception and reading (pp. 46-65). New York: Springer. Laloyaux, C., Destrebecqz, A., & Cleeremans, A. (2006). Implicit change identification: A replication of Fernandez-Duque and Thornton (2003). Journal of Experimental Psychology: Human Perception and Performance, 32(6), 1366-1379. Loftus, G. R., & Mackworth, N. H. (1978). Cognitive determinants of fixation location during picture viewing. Journal of Experimental Psychology: Human Perception and Performance, 4, 565-572. Levin, D. T., & Simons, D. J. (1997). Failure to detect changes to attended objects in motion pictures. Psychonomic Bulletin and Review, 4, 501–506. Levin, D. T., Momen, N., Drivdahl, S. B., & Simons, D. J. (2000) Change blindness blindness: The metacognitive error of overestimating change-detection ability. Visual Cognition, 7, 397-412. Mazza, V., Turatto, M., & Umilta, C. (2005). Foreground-background segmentation and attention: A change blindness study. Psychological Research, 69, 201-210. McCarley, J. S., Vais, M. J., Pringle, H. L., Kramer, A. F., Irwin, D. E., & Strayer, D. L. (2004). Conversation disrupts change detection in complex traffic scenes. Human Factors, 46, 424-436. McConkie, G. W., & Currie, C. B. (1996). Visual stability while viewing complex pictures. Journal of Experimental Psychology: Human Perception and Performance, 22, 563–581.

77

McConkie, G. W., & Zola, D. (1979). Is visual information integrated across successive fixations in reading? Perception and Psychophysics, 25, 221–224. Munger, M. P., Owens, T. R., & Conway, J. E. (2005). Are boundary extension and representational momentum related? Visual Cognition, 12, 1041-1056. Noë, A. (2009). Out of our heads: Why you are not your brain, and other lessons from biology of consciousness. New York: Hill & Wang. Noë, A., Pessoa, L., & Thompson, E. (2000). Beyond the grand illusion: what change blindness really teaches us about vision. Visual Cognition, 7 (1/2/3), 93-106. O’Regan, J. K. (1992). Solving the "real" mysteries of visual perception: The world as an outside memory. Canadian Journal of Psychology, 46(3), 461-488. O’Regan, J. K., & Noë, A. (2001). Acting out our sensory experience. Behavioral & Brain Sciences, 24, 955-975. O’Regan, J. K., Deubel, H., Clark, J. J., & Rensink, R. A. (2000). Picture changes during blinks: looking without seeing and seeing without looking. Visual Cognition, 7(1-3), 191-211. O’Regan, J. K., Rensink, R. A., & Clark, J. J. (1999). Change blindness as a result of “mudsplashes”. Nature, 398, 4. Pashler, H. (1988). Familiarity and visual change detection. Perception & Psychophvsics, 44, 369-378. Renninger, L. W., & Malik, J. (2004). When is scene identification just texture recognition. Vision Research, 44 (19), 2301-2311. Rensink, R. A., O’Regan, J. K., & Clark, J. J. (1997). To see or not to see: The need for attention to perceive changes in scenes. Psychological Science, 8, 368–373. Rensink, R. A., O’Regan, J. K., & Clark, J. J. (2000). On the failure to detect changes in scenes across brief interruptions. Visual Cognition, 7, 127–145. Rensink, R. A. (2000). Scene perception. In A. E. Kazdin (Ed.), Encyclopedia of Psychology (pp.151155). New York: Oxford University Press. Richard, C. M., Wright, R. D., Ee, C., Prime, S. L., Shimizu, Y., & Vavrik, J. (2002). Effect of a concurrent auditory task on visual search performance in a driving-related image-flicker task. Human Factors, 44, 108–119. Scholl, B. J. (2000). Attenuated change blindness for exogenously attended items in a flicker paradigm. Visual Cognition, 7, 377–396. Simons, D. J. (1996). In sight, out of mind: When object representations fail. Psychological Science, 7, 301-305. Simons, D. J. (2000). Current approaches to change blindness. Visual Cognition, 7, 1-15.

78

Simons, D. J., & Chabris, C. F. (1999). Gorillas in our midst: Sustained inattentional blindness for dynamic events. Perception, 28, 1059-1074. Simons, D. J., & Levin, D.T. (1997). Change blindness. Trends in Cognitive Sciences, 1, 261-267. Simons, D. J., & Rensink, R. A. (2005). Change blindness: Past, present, and future. Trends in Cognitive Sciences, 9(1), 16-20. Simons, D. J., Franconeri, S. L., & Reimer, R. L. (2000). Change blindness in the absence of a visual disruption. Perception, 29, 1143-1154. Turano, K. A., Geruschat, D. R., & Baker, F. H. (2003) Oculomotor strategies for the direction of gaze tested with a real-world activity. Vision Research, 43, 333–346 Turatto, M., Angrilli, A., Mazza, V., Umilta, C., & Driver, J. (2002). Looking without seeing the background change: Electrophysiological correlates of change detection versus change blindness. Cognition, 84, B1–B10. Varakin, D. A., Levin D. T., & Collins, K. M. (2007). Comparison and representation failures both cause real-world change blindness. Perception, 36(5), 737 – 749 Wertheimer, M. (1923). Untersuchungen zur Lehre von der Gestalt, II. Psychologische Forschung, 4, 301-350. Wolfe, J. M. (1999). Inattentional amnesia. In V. Coltheart (Ed.), Fleeting memories (pp.71–94). Cambridge, MA: MIT Press. Yarbus, A. F. (1967). Eye Movements and Vision. New York, Plenum Press.

79

Chapter 7: Experimental Study II Understanding the interplay between emotion and attention in scene perception

7.1 Introduction : Research Rationale

From an evolutionary perspective, instant selection and assessment of behavior-relevant information is essential for adaptive behavior. In this type of selection, attention plays an important role because it selects the relevant information (Egeth & Yantis, 1997). Following this line of thinking, attentional focus, or scope of attention, has been described by analogies such as attention as a spotlight (Posner, Snyder, & Davidson, 1980) or attention as a zoom lens of a camera (Eriksen & James, 1986), implying that attentional focus can be spatially restricted or widely allocated to simultaneously include peripheral stimuli. However, attention alone does not play an essential role in information selection. Emotions, through eliciting certain bodily reactions, can be deemed as a competitor for visual processing resources and can direct information processing (Gable & Harmon-Jones, 2012; Müller, Andersen, & Keil, 2008; Vuilleumier, 2005). This competition leads to a processing preference toward emotional stimuli in comparison to stimuli that do not possess emotional qualities (Keil & Ihssen, 2004; Kuhnbandner, Lichtenfeld, & Pekrun, 2011; Müller et al., 2008). Thus, it seems that the brain possesses mechanisms to allocate attention to both internal and external emotionally-significant sensory events (Vuilleumier & Huang, 2009). Even though it has been suggested that the processing of emotion-laden stimuli is rather automatic, i.e., independent of top-down factors such as attention, many studies provide contradictory evidence that even affective processing is governed by selective attention (Müller et al., 2008), and therefore the automaticity of processing of emotion-laden stimuli is still a subject of debate (Mitchell et al., 2007; Pessoa, Japee, & Underleider, 2005). The more traditional view of attention deems attention as being similar in different individuals, 80

disregarding inter-individual variances. However, recent research (Kaspar & Konig, 2012; Mardaga & Hansenne, 2009; Mardaga, Laloyax, & Hansenne, 2006; Matthews, Fox, Yiend, & Calder, 2003; Pessoa et al., 2005) shows evidence for interindividual differences (time-invariant personality traits) in attention for emotion-laden stimuli, as well as trait anxiety (Most, Chun, Widders, & Zald, 2005; Most, Chun, Johnson, & Kiehl, 2006). Automatic allocation of resources to emotionally charged stimuli has been found in many tasks, such as visual search (Öhman, Lundqvist, & Esteves, 2001), attentional blink paradigm (Anderson & Phelps, 2001; Anderson, 2005), rapid scene visual presentation (Most et al., 2005), the Sperling iconic memory task (Kuhnbandner et al., 2011), or when solving cognitive tasks (Schimmack, 2005). When searching for a particular stimulus (such as in Most´s et al. (2005)) in a rapid serial visual presentation (RSVP) paradigm, a preceding emotional prime can bias attention and result in a temporary inability to process the target. Most et al. (2005) coined the term ‘emotion-induced blindness’ for this particular phenomenon when emotion biases attention. Pessoa and Ungerleider (2004) and Schimmack (2005) showed that emotional stimuli bias attention and interfere with a task even when they are irrelevant to the ongoing task. Moreover, the processing of emotional stimulus influences the processing of subsequent stimuli. Presenting a task-irrelevant emotional face improves detection of an upcoming target (Phelps, Ling, & Carrasco, 2006) or improves reaction time to orienting to a stimulus occurring at the same spatial location while delaying orientation to stimuli at different locations (Pourtois, Thut, Grave de Perakta, Michel, & Vuilleumier, 2005). Much of the research in the area of this prioritized processing of emotional stimuli is still new. However, in general, increased sensory processing of emotional stimuli is believed to be the critical mechanism in processing prioritization (Schupp et al., 2007), as evidenced by a more extensive activation of visual cortex by emotional stimuli (Bradley et al., 2003). Consistent with the notion that external stimuli can cause attention to behave in a certain way, it has been illustrated that, for example, negative emotion narrows the attentional window by diminishing peripheral information processing (Gasper & Clore, 2002), while positive emotion broadens attention (Frederickson & Braningan, 2005). Frederickson (2001) proposed that the role of negative emotions in visual information processing is adaptive. By narrowing attentional scope, only relevant (i.e., threatening) information is processed. In contrast, positive emotions broaden

81

attentional and perceptual scope in order to encourage the seeking of novel and exploratory actions. In accordance with this theory, research studies suggest that emotions also influence global versus local visual information processing. Basso, Schefft, Ris, & Dember (1996) demonstrated that negative emotions or negative traits facilitate local features processing, whereas positive emotions or positive traits allow greater global configurations processing (Fredrickson & Branigan, 2005; Gasper & Clore, 2002; Kuhnbandner et al., 2011). Furthermore, Rowe, Hirsh, and Anderson (2007) showed that a positive mood contributes to broadened visualspatial processing using a modified flanker task. In their experiment, flankers (distractors) could occur at three spatial locations (near, medium, far) relative to the central target. They showed that positive moods increased the breath of attentional allocation, and thus participants in positive moods were less likely to be influenced by distant flankers. Taken together, executive control (attentional allocation) is clearly influenced by the emotional context of the external stimulus. The goal of the presented study focused on three areas of the effect of emotion on attention.

1. The role of emotional context in change detection In the study, standardized and validated images from IAPS (International Affective Picture System, Lang, Bradley, & Cuthbert, 2005) were used in order to examine the role of emotional context (created by real-world scenes with various degrees of valence and arousal) on the ability to detect changes within those images. As the emotional valence of visual stimuli have been found to affect field of view (FOV) (Kuhnbandner et al., 2011) Nobata, Hakoda, & Ninose, 2010; Rowe et al., 2007) and also eye-movement “behavior” in scene exploration (Kaspar, Hloucal, Kriz, Canzler, & Gameiro, 2013), it was believed that change detection would be affected by the emotional significance of the scenes via two mechanisms. The first mechanism is the modification of attentional scope (FOV), which would be greater for positive than negative images, allowing more relevant information to be processed simultaneously and resulting in faster change detection performance. For instance, research found a correlation between FOV and reaction time in an intentional change detection tasks (such as flicker) (Pringle, Irwin, Kramer, & Atchley 2001). Subjects with greater FOV were

82

faster at detecting changes in alternating scenes in comparison with subjects with smaller FOV. The second mechanism is related to the behavior characteristics of eye movement when emotional stimuli are presented as primes in natural scene perception. Viewing activity as measured by the number of fixations and saccadic eye movements is inhibited when negative images precede the subsequent free exploration of scenes (Kaspar et al., 2013). Considering the two mechanisms together, change detection in negative images should be slower than for positive and neutral images.

2. The role of emotional states in change detection The role of individuals’ affective states on attentional tasks has been explored by many studies. Results suggest that positive emotional states, when compared with neutral or negative states, increase field of view and reduce attentional biases in iconic memory tasks (Kuhnbandner et al., 2011), flanker tasks (Rowe et al., 2007), or Navon’s globallocal processing task (Gasper & Clore, 2002). In the current study, however, no experimental manipulation of emotional states took place; rather each participant assessed his/her own emotional state prior to the beginning of change detection task. It was hypothesized that the presence of negative emotions would lead to slower reaction times in change detection in comparison to positive emotional states.

3. The role of time-invariant personality traits in change detection Only recently, have studies investigating interindividual differences in attentional processes been discussed (Kaspar & Konig, 2012). In this respect, a particularly relevant factor is time-invariant personality traits (i.e., temperamental traits) as they predispose an individual to sensitivity and reactivity toward environmental stimuli. ‘Trait anxiety’ (Harm Avoidance in Cloninger’s model of personality) was found to be related to the processing of affective stimuli in temporal attention (Most et al., 2005) or change detection of unattended stimuli (Mardaga & Hansenne, 2009). Harm Avoidance is a temperament dimension that refers to sensitivity toward aversive stimuli and can be perceived as one’s trait anxiety, which has been shown to correlate negatively with executive control (Berggen & Derakshan, 2012; Stout, Shackman, & Larson, 2013). 83

Therefore, Harm Avoidance should be related to the performance of change detection in emotional stimuli, as subjects‘ control of attention might be modulated by their sensitivity to emotional stimuli (Most et al., 2005). The prediction was that individuals with higher levels of high trait anxiety (HA) would be slower at detecting changes especially in negative images, since a successful performance would require an inhibiting of sensitivity (i.e., looking toward) to arousing stimuli. To examine the role of temperament in the potential modulation of spatial attention, Cloninger’s Temperament and Character Inventory was used to assess participants’ personality traits. The relationship between Harm Avoidance and performance in attentional task was especially of interest to the current study.

7.2 Method

7.2.1 Participants 66 students (26 men, 40 women; age: M= 22.24, SD=2.05; range: 19-28) at Masaryk University took part in the study. The participants received one academic credit for participating in the experiment. Each participant was tested individually in a quiet room.

7.2.2 Procedure At the beginning of the experiment, participants were asked to fill out a brief questionnaire concerning demographics. Afterwards, they were administered the PANAS and Temperament and Character Inventory (TCI) tests, taking approximately 20 minutes. PANAS measures current mood, while TCI looks at personality characteristics. After the completion of the questionnaires, a brief training session comprising of 4 change blindness (flicker) trials was presented in order for them to be familiarized with the nature of the experiment. The experimental session was divided into 3 blocks with filler tasks in between them. Each block included 9 trials of flicker tasks with the random presentation of either positive,

84

negative, neutral (the total number of trials was 27: 9 pictures in each emotional valence category). In each block, a maximum of two images from the same emotional category was presented in order to eliminate the sequence effect (Flaisch, Junghofer, Bradley, Schupp, & Lang, 2008). The order of the blocks and the images presented in them was randomized. After each block, participants completed a short filler task that consisted of solving mathematic tasks. The idea behind administration of the filler task was to eliminate the effect of adaptation to emotional stimuli. The changes always consisted in the removal of a particular element of the scene. All changes were equalized in terms of luminance, contrast and pixel size.

7.2.3 Measures

7.2.3.1 Change Detection Method

The original flicker paradigm (Rensink et al., 1997) was used in the study. The flicker tasks consist of the consecutive presentation of two alternating images (with a blank screen inserted in between them), the first image original, the second featuring a change. The images alternate until the participant detected the change and clicked on the change using a mouse, or until a time limit was reached. The time limit was 40 s (approx. 58 image alternations). If no change was detected during this time period (i.e., participant did not click on the change in the image), the image disappeared and a new trial began. Two dependent variables were measured: reaction time (as measured in seconds) and correct identification versus misidentification of a change. The task was an explicit change detection method (i.e., participants were explicitly told to look for a change). The stimulus onset asynchrony lasted 250 ms (and the image was presented during the whole period of SOA), while interstimulus interval was 80 ms long. See figure below for illustration of the method as well as the example of images used.

85

Figure 3: Flicker paradigm used in the study. Alternation of two images (original and changed) with a blank screen in between them until the change is detected.

mask (80 ms)

mask (80 ms)

mask (80 ms)

Changed Stimulus (250 ms)

Original Stimulus (250 ms)

Changed Stimulus (250 ms)

Original Stimulus (250 ms)

7.2.3.2 Affective States Measures

The PANAS questionnaire (Positive and Negative Affect Scale, Watson, Clark, & Tellegen, 1988)

PANAS questionnaire was used prior to start of the change detection task in order to assess the current affective states of the participants. The original version of PANAS included 20 items pertaining to emotions or feelings (10 positive and 10 negative emotions), such as interested, upset, enthusiastic, or nervous, ashamed, alert etc. Participants are asked to indicate on a 5-point Likert Scale (ranging from 1 (Very Slightly or not at all) to 5 (Extremely)) to what extent they were feeling an emotion at that moment. In our study, the Czech translation of the adjectives pertaining to positive or negative emotions or feelings were used (5 experts in 86

psychology translated the adjectives). For the purpose of the study, only 6 positive and 6 negative emotions were selected (positive: active, alert, attentive, enthusiastic, interested, inspired; negative: afraid, nervous, jittery, irritable, upset, distressed). The final score was calculated for each dimension of the affect, positive affect or negative affect, by calculating the individual scores from either the positive or negative emotion’s adjectives. The scores could vary between 6 and 30. The reliability of both scales was quite high, indicating that the measures have good internal reliability (Positive Affect, Cronbach α = 0.75, Negative Affect, Cronbach α = 0.79).

7.2.3.3 Personality Measures Temperament and Character Inventory – TCI-R (Cloninger, Przybeck, Svrakic, & Wetzel, 1994)

Personality characteristics were measured by Cloninger’s Temperament and Character Inventory – TCI (Cloninger et al., 1994). A Czech version of the TCI was obtained from the Prague Psychiatric Centre. TCI describes personality by means of seven basic factors of the psychobiological model: the Temperament dimensions encompass Novelty Seeking (e.g., ‘‘It is hard for me to stay interested in the same things for a long time because my attention often gets distracted by other things’’), Harm Avoidance (e.g., ‘‘I often have to stop what I am doing because I start to get worried about what can go wrong’’), Reward Dependence (e.g., ‘‘I often give into the wishes of friends’’) and Persistence (e.g., ‘‘I often push myself to the point of exhaustion or try to do more than I really can’’). Character dimensions include the factors of Self-Directedness (e.g., ‘‘Each day I again try to get a step closer to my objectives’’), Cooperativeness (e.g., ‘‘I can generally accept other people for how they are, even if they are very different from me’’) and Self-Transcendence (e.g., ‘‘I am often called absent-minded, because I am so immersed in what I am doing that everything else passes me by’’). The TCI-R consists of 240 items to which respondents answered on a 5-point Likert scale from ‘‘definitely false’’ (1) to ‘‘definitely true’’ (5). The primary focus was, however, only on temperament dimensions, especially Novelty Seeking and Harm Avoidance, as they have been important in

87

the processing of emotional stimuli (Mardaga & Hansenne, 2009; Mardaga et al., 2006; Most et al., 2005; Yoshino et al., 2005).

7.2.4 Stimuli The stimuli for the change detection task included scenes of relatively equal complexity 1. The changes occurred in emotionally significant scenes (emotionally negative or positive) or neutral scenes. The changes took place in two different locations: changes in the central region, or changes in the marginal regions of the scenes. The changes themselves were never emotional (only the scenes were emotional). The pictures were selected from International Affective Picture System (IAPS). IAPS provides normative emotional items that are used in experiments concerning the interaction between emotion and attention. All images are standardized, color photographs and information about valence (ranging from pleasant to unpleasant) and arousal (ranging from calm to exited) of the images is provided in the IAPS Manual (Lang et al., 2005). The valence of the pictures was assessed as followed: 1) neutral (4.2 < Mvalence < 6.2); 2) positive (Mvalence > 6.2); and 3) negative (Mvalence < 4.2) (Grühn & Scheibe, 2008). Table 3 provides information about the images used in the study and their parameters (valence and arousal). The parameters of the emotional valence categories of scenes were defined as followed: Neutral images’ valence was moderate (M = 4.79, SD = 1.04; range = 3.26 - 6.34), while arousal was relatively low (M = 3.59, SD = 0.33; range = 2.93 - 3.95). Positive images’ valence was rather high on emotional valence (M = 7.48, SD = 0.47; range = 6.61 - 8.22), while also being high in arousal (M = 5.35, SD = 0.85, range = 3.73 - 6.73). Negative images were characterized by very low emotional valence (M = 2.46, SD = 0.43; range = 1.98 – 3.34) and higher levels of arousal (M = 5.80, SD = 0.47, range = 5.00 - 6.64). For illustration, see Table 3. Importantly, the emotionally significant scene categories differed from each other, both in terms of valence, F(2, 24) = 114.94, p < .001, and arousal F(2,24) = 34.73, p < .001. Post-hoc analyses using the Bonferroni post-hoc criterion revealed that, in terms of valence, negative images had significantly lower values than positive, and neutral at p < .001. In terms of arousal, post-hoc analyses showed significant differences in arousal between both emotional (negative and

1

This particular feature was assessed intuitively, i.e., approximate number of objects, and number of details in each scene accross individual emotional valence categories were equalized. 88

positive) and neutral images at significance level, p < .001. However, arousal did not differ between negative and positive images (for means and variance see Table 4).

Table 3: Parameters of images from IAPS used in the research study.

Image IAPS 2446 2383 2393 2396 2490 2515 2560 2590 2595 1340 2345 2389 2398 2598 5833 7502 8370 8420 6834 9252 9254 9428 9435 9495 9900 9903 9920

number

in Emotional Valence 4.70 4.72 4.87 4.91 3.32 6.09 6.34 3.26 4.88 7.13 7.41 6.61 7.48 7.19 8.22 7.75 7.77 7.76 2.91 1.98 2.03 2.31 2.27 3.34 2.46 2.36 2.50

Emotional Arousal 3.79 3.41 2.93 3.34 3.95 3.80 3.49 3.93 3.71 4.75 5.42 5.63 4.74 3.73 5.71 5.91 6.73 5.56 6.28 6.64 6.04 5.66 5.00 5.57 5.58 5.71 5.76

89

Emotional category neutral neutral neutral neutral neutral neutral neutral neutral neutral positive positive positive positive positive positive positive positive positive negative negative negative negative negative negative negative negative negative

Figure 4: Examples of the three different emotion categories of images used in the study.

Original Stimulus, Negative (OS)

Changed Stimulus, Negative (CS)

Original Stimulus, Neutral (OS)

Changed Stimulus, Neutral (CS)

Original Stimulus, Positive (OS)

Changed Stimulus, Positive (CS)

90

Table 4: Descriptive statistics for each emotional dimension (valence and arousal) of images used in our study.

Parameter Emotional Mean Category valence

arousal

Std. Std. Deviation Error

Minimum Maximum

neutral

4.79

1.04

0.35

3.26

6.34

positive

7.48

0.47

0.16

6.61

8.22

negative

2.46

0.43

0.14

1.98

3.34

neutral

3.59

0.33

0.11

2.93

3.95

positive

5.35

0.85

0.28

3.73

6.73

negative

5.80

0.47

0.16

5.00

6.64

7.3 Results

Firstly, general detection data will be reported in order to demonstrate measures of the subjects’ performance. Afterwards, data concerning the relationship between emotional scene valence and correlations between individuals’ emotional states, as well as relationship between personality determinants and change detection performance will be reported.

7.3.1 Change Detection 7.3.1.1 Descriptive Statistics Descriptive statistics for each emotional category as well as age are displayed in Table 5. All scores were computed by averaging scores for each emotional category for each participant. Normality tests (Kolmogorov-Smirnov) showed that data for those variables were normally distributed.

91

Table 5: Descriptive statistics for reaction times for different emotional scene categories.

Variable

Mean

Standard Deviation

Minimum

Maximum

age

22.24

2.05

19

28

RT (overall)

12.45

2.55

7.38

18.95

RT (neutral)

13.04

3.19

5.77

21.66

RT (positive)

12.33

3.48

5.73

22.42

RT (negative)

11.99

3.02

4.59

21.07

7.3.1.2 Change Detection Accuracy

Change detection accuracy for the whole sample Each participant’s correct-detection score for negative, positive, and neutral scenes was entered into a within-participants t-test. For the change trials, participants detected changes within the time limit, i.e., 40 s, in 90.97% of all the trials. In terms of emotion categories, participants detected 88.71% of changes on average in neutral scenes (SD = 10.22%); 91.75% of changes in the positive scenes (SD = 9.81%) and 92.42% (SD = 10.88%) of changes in the negative scenes. There was a significant difference in terms of the percentage of detected changes in each particular condition F(2, 130) = 3.13, p < .05. The main effect of emotional valence on detection accuracy was observed, specifically, as a significant difference between the detection accuracy of changes that occurred in neutral scenes and positive scenes, t(65) = -2.09, p < .05; as well as between neutral and negative scenes, t(65) = -2.32, p < .05. No significant difference between the detection accuracy of changes in negative and positive scenes was found, p > .05.

92

Change detection accuracy and gender differences

Addressing gender differences was an important and interesting part of the study as well. Even though no significant statistical differences were found between genders in terms of detection accuracy, one can see a trend in women’s lower percentage of change detection accuracy for both types of emotional scenes (negative and positive). See Table 6 for details.

7.3.1.3 Change Detection Latency Due to the fact that some of the data had missing values (if the participants did not detect the change within the time limit of 40 seconds), the Expectation Maximization Method was used to manage the missing data. Little’s MCAR test was not significant, indicating that data were missing in a random order (Chi-Square = 987.702, df = 951, p = .20). Detection latency (displayed in seconds) has been calculated for each trial in all conditions. Table 6 illustrates the descriptive statistics for individual variables as well as for men and women separately. When computing the repeated measures analysis of variance with the three different emotional categories (neutral, positive, negative), a main effect of emotional category was found, F(2,130) = 3.16, p .05. Furthermore, gender differences in reaction times for changes in individual emotional scene categories were calculated (see Table 6 for descriptive statistics). An independent t-test demonstrated that women were faster at detecting changes in positive-laden scenes, t(64) = 2.10, p < .05 as well as faster in overall reaction time performance, t(64) = 1.97, p = .05 (2-tailed). In general, a similar trend of women detecting changes faster than men was observed across all categories (see Table 6). 93

Table 6: Percentage correct and detection latency as a function of emotional valence of scenes.

Measure

Overall

Neutral

Positive

Negative

% Correct

90.97

88.72

91.75

92.42

Men Women

92.59 89.91

88.89 88.61

94.02 90.28

94.87 90.83

Latency (sec)

12.45 (2.55)

13.04 (3.19)

12.33(3.48)

11.99 (3.02)

Men Women

13.20 (2.86) 11.96 (2.23)

13.59 (3.04) 12.66 (3.27)

13.42 (4.09) 11.62 (2.85)

12.59 (3.66) 11.60 (2.50)

N = 66, N (men) = 26, N (women) = 40

Figure 5: Reaction times (means and standard errors) for each emotional scene category.

15 14.5

Reaction time for individual emotional categories

14 13.5

13 12.5 12 11.5 11 10.5 10 overall

neutral

positive

94

negative

7.3.2 Change Detection and Emotional States Emotional states were assessed via the 6 positive and 6 negative adjectives of the PANAS questionnaire. We correlated the emotional states with reaction time performance (values for the unattended trials were values obtained via the Expectation Maximization Method) for each emotion category along with overall reaction time performance. Moreover, separate overall positive and overall negative states were computed (sum of either positive or negative states scores). The descriptive statistics, as well as (two-tailed) correlations, are displayed below.

Table 7: Descriptive statistics for emotional states (based on self-reports).

Emotional state

Mean

Standard Deviation

sum positive

17.92

3.94

active

2.92

.85

alert

3.45

.81

attentive

3.27

.81

enthusiastic

2.83

.97

interested

2.94

1.08

inspired

2.50

1.06

sum negative

9.52

3.18

afraid

1.27

.60

nervous

1.62

.72

jittery

1.83

.95

irritable

1.36

.78

upset

1.68

.81

distressed

1.74

.85

95

Table 8: Correlations between emotional states and change detection performance in each emotional category (RT- reaction time in sec).

RT (neutral)

RT (positive)

RT(negative)

RT (overall)

sum positive

-

-

-

-

active

-

-

-

-

-

-

-

-.24

alert

x

attentive

-

-

-

-

enthusiastic

-

-

-

-

interested

-

-

-

-

inspired

-

-

-

-

sum negative

-

-

-

-

afraid

-

-

-

-

.27*

-

.25*

.29*

jittery

-

-

-

-

irritable

-

-

-

-

upset

-

-

-

-

distressed

-

-

-

-

nervous

N = 66; * p < .05, x p = .05

In general, self-reports of the participants demonstrated that the participants were in positive emotional states, with low levels of negative emotional states. Based on the results, overall emotional states did not correlate with the change detection latency performance to a great extent. The only self-reported emotional state that had a consistent positive correlation with performance in neutral and negative change detection conditions, as well as with overall performance, was feeling nervous, r = .27, r = .25, r = .29, p < .05, respectively. This indicates that great levels of nervousness results in longer change detection latencies in the above-mentioned conditions. Furthermore, alertedness correlated

96

negatively with performance only in the neural emotional condition, indicating that being alert augmented change detection speed. The results will be further debated in the discussion.

7.3.3 Change Detection and Personality Determinants The descriptive statistics for individual personality dimensions based on TCI (Temperament and Character Inventory) are displayed below. Each personality dimension was correlated with change detection performance in the emotionally significant images and overall change detection performance.

Table 8: Descriptive statistics for personality dimension.

Personality Dimension

Mean

Standard Deviation

Novelty Seeking

105.77

12.20

Harm Avoidance

94.05

17.30

Reward Dependence

93.48

15.39

Persistence

103.85

22.35

Self-Directedness

127.06

18.55

Cooperativeness

126.33

18.14

Self-Transcendence

74.50

20.50

97

Table 9: Correlations between personality dimensions and change detection performance in each emotional category.

RT(neutral)

RT (positive)

RT(negative)

RT (overall)

Novelty Seeking

-

-

-

-

Harm Avoidance

.26*

.23x

-

.24x

Anticipatory worry

-

.30*

-

.26*

.30*

.30*

-

.27*

Reward Dependence

-

-.25*

-

-

Openness to warm communication

-

-.31*

-

-

Attachment

-

-.33*

-

-

Persistence

-

-

-

-

Self-Directedness

-

-

-

-

Cooperativeness

-

-

-

-

Social acceptance

-.25*

-

-

-

Empathy

-.25*

-.29*

-.30*

-.36*

-

-

-

-

Shyness

Self-Transcendence

N=66; * p