STIMULUS REPETITION EFFECTS ON PICTURE ... - Google Sites

STIMULUS REPETITION EFFECTS ON PICTURE RECOGNITION AND EVENT RELATED POTENTIALS By J. L. Sanguinetti

A paper submitted in partial fulfillment of the requirements of the Honors Program in the Department of Psychology.

Examining Committee:

Approved By:

_________________________________

_________________________ Julian Keith, PhD Faculty Supervisor

_________________________________ _________________________________ _________________________________ Honors Council Representative

__________________________ Department Chair

_________________________________ Director of the Honors Scholars Program

University of North Carolina Wilmington Wilmington, North Carolina May 2007

2 In loving dedication to Mom, Dad, Jeanne, and Kramer

3 Contents 1

2

3

4

Introduction …………………………………………………………… 1.1 Electroencephalography……………………………………………. 1.1.1 Origins of the EEG………………………………………. 1.1.2 The Event-Related Potential Technique ………………… 1.1.3 Components …………………………………………. 1.1.4 Source Estimation …………………………………… 1.2 Recognition Memory …………………………………………... 1.2.1 Prior Occurrence …………………………………….. 1.2.2 Theoretical Framework ……………………………… 1.2.2.1 Animal Models …………………………….. 1.2.2.2 Lesion Studies ……………………………… 1.2.2.3 ERP and Dual Process Models …………….. 1.2.2.4 fMRI Evidence …………………………….. 1.3 Repetition and the Brain …………………………………….…. 1.3.2 ERP Repetition Effect ……………………………….. 1.3.3 ERP ‘Old/New’ Effect ……………………………….. 1.3.4 Repetition Suppression ………………………………. 1.3.5 The Current Study ……………………………………. Materials and Methods ………………………………………………….. 2.1 Subjects ………………………………………………………… 2.2 Equipment ……………………………………………………… 2.2.1 Electrode Layout ……………………………………… 2.3 Stimuli ………………………………………………………….. 2.4 Procedure ……………………………………………………….. 2.4.1 Experiment 1 ………………………………………….. 2.4.2 Experiment 2 ………………………………………….. Results …………………………………………………………………… 3.1 Experiment 2…………………………………………………….. 3.2 Visual Results …………………………………………………… 3.2.1 Topographic Maps …………………………………….. 3.2.2 Grand Averages ……………………………………….. 3.3 Source Estimation and Repetition Effects …………………………. 3.3.2 Amplitude Differences …………………………………. 3.3.3 Peak Latency Differences ……………………………… Discussion ……………………………………………………………….. 4.1 Findings …………………………………………………………... 4.2 Component Differences …………………………………………... 4.3 Problems ………………………………………………………….. 4.4 Future Research …………………………………………………… 4.5 Conclusion …………………………………………………………

6 6 8 11 14 21 21 21 23 26 27 29 30 31 32 33 34 36 38 38 40 42 44 46 46 52 53 53 56 56 59 71 71 77 79 79 80 81 82 82

References ………………………………………………………………..

83

4

Abstract In this study, we used state-of-the-art source estimation methods and event related potentials (ERP) to investigate the timing and amplitude of electrophysiological activity in four brain regions of interest (ROI) during a picture recognition memory task. When stimuli are repeated, subjects recognize them faster (as indexed by speeded reaction time). In the current experiment, subjects viewed pictures of animals in a study/test paradigm while continuous EEG was recorded. Pictures were repeated once, three times, or ten times in study. We believe that the same experimental factors that affect recognition will also affect the brain regions involved in recognition processes which will be revealed by a greater signal strength and decreased signal latency at regions implicated in recognition. This hypothesis was verified; source estimation analysis reveals that repetition of visual stimuli increases amplitude signal and amplitude latency from particular regions of interest within the brain.

5 Acknowledgements I would like to express my gratitude to my friends and family who have provided the social support on which I depend. Special thanks to my immediate family—Mom, Dad, Jeanne, Kramer, Grandmother, Grandfather, Bobby and Mimi—for providing your unconditional love. To Aneeka for having patience a monk would envy. To the Turrisi family (including the Cat) for providing intellectual stimulation, tea and home cooked meals. This work would never have been possible without the endless generosity of Lloyd Smith of Cortech Solutions, LLC. You not only provided the electroencephalogram and other equipment, but you also provided your time and energy and played the invisible hand in guiding this project to completion. I am indebted to you and only hope I have the opportunity to provide such generosity in the world. Many thanks to my committee for taking the time to read and critique my work, especially Dr. Cohen who allowed me to tinker in his lab when I was an EEG novice and for providing his ideas during the conception of this project. Finally, to my advisor Dr. Keith, who’s wit and humor made the hardest work I have ever done the most enjoyable. You have taken a naive college student and turned him into a naive adult, but with the skills and confidence to find his way. You stuck with me throughout our setbacks, and have been not only the best mentor imaginable, but also a true friend.

6

Chapter 1 Introduction This introduction is divided into three sections. The first section provides a theoretical overview of the electroencephalogram. This is done to explore the relationship between the extracranial electrical activity measured by scalp electrodes and the underlying bioelectrical generators in the brain. The second and third sections relate directly to the current study by summarizing what is known about recognition memory and the effects of stimulus repetition on recognition memory and the brain electrophysiology, respectively. 1.1

Electroencephalography The electroencephalogram (EEG) is a neurophysiological tool for measuring

electrical activity of the brain by means of electrodes placed on the scalp. These extracranial potentials are produced by electrical currents emanating from underlying neural populations in the brain. Unlike other imagining techniques (e.g. functional Magnetic Resonance Imaging [fMRI], Positron Emission Tomography), the EEG is continuous (i.e. in real time and ongoing). In its most simplistic form, when two electrodes are attached to the surface of the scalp and connected to an amplifier, the output will be variations in voltage over time (Rugg, 1995). Due to the spreading of electrical activity of the skull and underlying tissue, the EEG readout will always be the potential difference between at least two electrodes, with typical amplitude ranges of 100 to 100 mV, and meaningful frequencies between 0.1 and 55Hz (Rugg, 1995).

7

Figure 1. Two electrodes are detecting electrical activity on the scalp, taking the difference in the voltage at each electrode, and displaying the variation over time.

8 1.1.1

Origins of the EEG Information is transmitted throughout the brain and nervous system via the

propagation of electrochemical gradients along neuronal cell assemblies. This passage of currents within and between neurons naturally creates electrical signals that can be detected outside of the brain (by the EEG). Thus, monitoring extracellular potentials is a valid way of non-invasively measuring activity of the underlying tissue in the brain. It is widely accepted that the bioelectric potentials recorded at the scalp are generated within the brain; however, the relationship between what is going in the brain and what we observe at the scalp is not fully understood (Rugg, 1995; Nunez, 1981). These potentials are generated by the sum of the excitatory and inhibitory post-synaptic potentials (EPSPs and IPSPs) of neurons. These electrical fields are the sum of a sizable population of neurons (on the order of 10,000 neurons). Due to the nature of electromagnetism, the EEG can only measure potentials from groups of cells that are aligned perpendicular to the surface of the cortex (Zani and Proverbio, 2002). Neurons that have the necessary geometric configuration to produce fields that can be measured at the scalp do so because their individual electric fields summate to yield a net dipolar field (a field with positive and negative charges between which current flows)—known as an ‘open field’ (Rugg, 1995). It must not, however, be thought the electrical source is directly underneath the electrode. Even if the electrical activity originates at one point in the brain, the resulting EEG can be measured throughout the scalp. A specific neural generator in the brain produces a unique scalp voltage topography (the EEG), but an infinite number of different cortical source configurations

9 can produce the same scalp distribution—known as the inverse problem (discussed below) (Slotnick, 2005).

10

Figure 2. Continuous EEG from 16 electrodes. Figure 2. Continuous EEG as displayed on a typical computer screen. Show here are 16 electrodes.

11 1.1.2

The Event-Related Potential Technique Raw EEG data from individual subjects reflects “spontaneous” fluctuations of

electrical activity in the brain, and is meaningless when trying to understand underlying cognitive processes. However, researchers who study specific neural mechanisms involved in cognition (e.g. memory, attention, etc.) have devised methods to extract informative brain patterns from the raw data. Subjects are presented a series of stimuli that can be auditory or visual while the EEG is recording. In post analysis software, an epoch of the EEG that is time-locked to individual stimulus can be defined. For instance, if researchers are interested in semantic processing in the brain, they could present different words to the subject via a computer. In the software, an epoch of time can be defined around the presentation of each stimulus (e.g., 200 ms before to 1000 ms after a marker for the word in the raw EEG.) If there are voltage changes within this epoch that are specifically related to the brain’s response to the stimuli, then an event-related potential (ERP) has been found. It can be inferred that if there are similar ERPs across subjects, there is similar brain activity (and cognitive processes) across subjects. Relative to positron emission tomography (PET) and other imaging techniques, ERPs have a resolution down to the millisecond compared to seconds for the former. Equally important, ERP are reliably sensitive in detecting functional changes in brain activity. Thanks to these advantages, the event related potential technique is able to uncover steps in sensory-cognitive information processing that is occurring rapidly in the brain (Zani and Proverbio, 2002). This can help to uncover the functional organization,

12 and timing of activation, of the anatomically distributed functional systems that can be correlated with the findings of imaging studies.

13

Figure 3. A simplified model of the ERP technique for a visual word paradigm. Each time the subject views the word ‘dog’ a marker is placed in the raw EEG data. Later, the signals are averaged in the computer software and the ERP signal emerges from the background EEG.

14 1.1.3 Components In light of the discussion above, defining an ERP component might seem like an easy task. After extracting the signal using the technique described above, all one needs to do is to focus on the peak or trough of the resulting waveform and this becomes the component of interest. Measurement would be as straight forward as determining the amplitude and latency of the component in relation to some other feature of the waveform or to the baseline. There are at least two major problems to this simple approach to component definition (Donchin, et al. 1978). First, and probably the greatest impediment, is ‘component overlap’ (Picton & Struss, 1980). The waveform we observe by measuring voltage fluctuations at the scalp may be generated by several different sources in the brain. Since the brain is a conductive medium (as noted above), activity could be generated in spatial location and propagating to another where it is detected. In other words, the single voltage we measure at one location may very well be a reflection of the summation of many sources in the brain. One consequence of this is that a component (say at 200ms) may actually reflect the summation of two neural generates who were maximally active before and after that time independently, but whose open fields summated at the time recorded (Rugg, 1995). The second question is more theoretical. Some researchers such as Naatanen and Piction (1987) adopt what is called the ‘physiological approach’ in which they define the ERP component by its anatomical source within the brain. Accordingly, we must have a method of making the contributing sources unambiguous if we are going to measure particular ERP components. Other ERP researchers (see Donchin, 1979) argue that an ERP component should be defined in terms of the information processing operation with

15 which it is correlated. This is known as the ‘functional’ approach. The component is defined in terms of the cognitive function that is linked to the underlying brain system whose activity is recorded at the scalp. It is probably most realistic to adopt a mixture of both approaches (Rugg,1995). P100 The P100 is a typical sensory component found all sensory modalities. It is associated with a series of deflections in the ERP that are related to the transmission of sensory information from the peripheral sensory system to the cortex and/or the arrival of that information in the cortex (Luck, 2004). The P100 (or P1) reflects the processing of visual stimuli in the visual cortex. It is somewhat of a later sensory component than other modalities (e.g., auditory which can be on the order of 10s of ms) because the neurons in the sensory relay nuclei (e.g., the lateral geniculate nucleus) are configured in such a way they create a closed field that does not reach the scalp (Rugg, 1995). N200 About 200 ms after the presentation of auditory and visual stimuli, a negative component often becomes evident. The necessary condition to elicit this component is that the event must physically deviate in some way from the prevailing context; this is referred to as ‘mismatch negativity’ (Pritchard, 1991). In a series of now classical experiments Naatanen (1978) showed that improbable events (tones that deviated from a norm) elicited a negative component about 200 ms after presentation. P300 The P300 is the most studied ERP component. This is due in part to its side (5-20 mV) and because how easy it is to elicit the component. The standard paradigm for the P300 is

16 known as the ‘oddball paradigm.’ A series of events are presented to a subject, and the events are composed of two classes. One type of event is occurs less frequently than the other—hence the name oddball. Subjects are usually required to respond in some way to the rarer of the two events. The ERP components for the oddball stimuli usually consist of a positive deflection, that is maximal over the parietal/central area, that has a latency of 300 ms up to 900 ms (Rugg, 1995). Coles and Donchin (1988) have proposed that this effect reflects a process of context of memory updating by modifying the current model for the environment due to the incoming information.

17

Figure 4. P300 wave elicited in a typical “odd-ball paradigm.” The red line is the potential elicited from odd stimulus, the blue represents the background stimuli. (The colored circles are topographic head maps that represent the relative electrical activity on the scalp). Note: negative is up (image from Krigolson, Olav; http://web.uvic.ca/psyc/braincoglab/krigolson_res.html, with permission).

18 N400 The N400 component is sensitive to deviance to abstract attributes of the eliciting stimulus, such as meaning. Kutlas and Hillyard (1980) were the first to observe the N400 in a now classic paradigm. Subjects were required to read a series of sentences comprised of about seven words. Some sentences ended in semantically inappropriate but syntactically correct words, others with the final word printed in a larger font size than the preceding words, while other sentences were normal (they ended with appropriate words and the font was kept constant). Semantically deviant final words elicited a negative deflection with a latency of about 400 ms (the N400). Physically deviant words elicited the classical P300 (latency ~560 ms). Neither component showed up for the normal sentences.

19

Figure 5. Picture taken from Kutlas and Hillyard (1980). The black line represents the ERP for ‘normal’ sentences. The dotted line is the semantically inappropriate sentences. Note: negative is up.

20

Figure 6. Idealized waveforms for several components of the ERP (taken from Rugg 1995).

21 1.1.4 Source Estimation In section 1.1.1, the inverse problem was introduced. It was noted that any given scalp voltage topography could theoretically be produced by an infinite number of cortical source configurations. That is, the EEG activity we see at the scalp could have any of a number of sources in the brain. Since Helmholz (1853) pointed out the problem it has been understood that, mathematically, there was no way to solve the inverse problem (i.e. no one equation that could take the scalp data and give a specific generator in the brain). With the immense processing power of modern computer, a new technique has been developed, known as source estimation (sometimes called source localization). The strength of source estimation comes from the ability of the computer to model the conductive properties of the brain, dura, skull, etc, and the anatomical configurations of the neurons that create the open fields of the EEG, creating a head volume conductor model. This model is used to “localize” functional generators within the brain. This is known as the forward solution because the models are starting with known conductive properties of the underlying material (brain, skull, neurons etc.) and modifying the input parameters (EEG data) in such a way as to minimize the sum-ofsquares error between the data and model (Slotnick, 2005). A nonlinear model-fitting algorithm is used to iteratively modify the scalp data until a best-fit solution is found. 1.2

Recognition Memory

1.2.1 Prior Occurrence Recognition is the act of perceiving something as previously known (Mandler, 1980). It requires the capacity for a) the identification and b) judgment concerning prior occurrence of an object or event. Mandler (1980) defines recognizing as the process of

22 arriving at a decision about prior occurrence. This study is interested in the underlying neural processes that support recognition memory in humans. Our ability to recognize objects and events as previously encountered is a fundamental to our survival. Case studies of patients with localized damage illustrate this fact in a sad way. In a famous example, Oliver Sacks (1985) reports about a patient with temporal lobe damage who mistakes his wife for a hat! V.S. Ramachandran (2001) reports about patients with damage to the fusiform gyrus who recognize their close relatives but believe that they are imposters. They report a strange feeling that, although the imposters resemble their parents or siblings, they have a keen feeling they are fakes. Such reports are a reminder that our ability to accurately recognize people, objects, and events is a crucial skill for survival. Recognition memory is affected by many factors. Accuracy and performance increases with stimulus frequency, known as repetition priming (see Schacter et al. 1993 for a review, and below). Prior exposure facilitates recognition of items or events. Repetition priming is usually studied in speeded decision tasks (e.g., lexical decision) where priming is indexed by faster reaction times to repeated than unrepeated items, or under conditions that involve degraded viewing, where repetitions increase the probability of likelihood of correct identification. Depth of processing also plays a major role in affecting recognition. Instructing subjects to process items beyond their surface features (e.g., name the item vs. passively viewing it) increases the likelihood of correctly identifying the item in a later recognition memory test (Steward, 1972). Subjects who are asked to judge pictures of a person based

23 on personality score better on recognition tasks than subjects who judge the physical characteristics of pictures of other people (Cain, 2003). 1.2.2 Theoretical Framework Evidence from cognitive, neuropsychological, and neuroimaging studies increasingly indicates that separate memory processes contribute to recognition memory. Most researchers now agree that so called ‘dual process’ theories are needed to account for dissociations found in tests of recognition memory (Yonelinas, 2002). In contrast, ‘single process’ theories, which posit that recognition memory is dependant on only one form of memory, cannot be fully discounted at this time. A single process theorist would argue that dissociations found in the literature reflect quantitative differences in a single memory system, rather than qualitatively different memory systems. A. Imagine that you pass someone on the street and you know that you have met him or her before, but do not recall any specific details about that person (e.g., their name or where you met). Familiarity is usually associated with non-declarative or ‘unconscious’ memory (conceptual implicit memory but not perceptual implicit memory) and is context free, whereas recollection is usually associated with declarative or ‘conscious’ memory (similar but not identical to free recall), and is context dependent (Rugg & Yonelinas, 2003). Various methods have been developed to test a behavioral dissociation of the two forms of memory, and they indicate that familiarity and recollection rely on distinct memory systems. One such method looks at processing speed. It has been found subjects respond faster to judgments based on familiarity (e.g., such as distinguishing between items that were recently studied vs. non-studies items) than to judgments based

24 on recollection (e.g., recollecting specific information about the study event, like when an item was presented in a list of items) (Hintzman, 1998). Recollection depends on relational information about the study event, such as whether a word was initially paired with another context word, or sensory modality or spatial location in which an item was presented. Familiarity does not rely on such information. Jacoby (1991) developed the process-dissociation procedure under the above assumptions to derive quantitative estimates for the contribution of recollection and familiarity. The assumption of this method is that if a subject can recollect a given item, they should be able to determine when or where it was initially studied; familiarity will not reflect such discrimination. Subjects are presented with two classes of items for study. At test, they are either instructed to respond ‘old’ to one class of study stimuli (exclusion task), or they are instructed to respond ‘old’ to all items that have been studied (inclusion task). For the exclusion task, accurate performance reflects the ability of the subject to discriminate between the two classes of study items, which reflect recollection. Recognition based on familiarity is sufficient to discriminate old from new items in the inclusion task. With these assumptions, it is possible to calculate separate values for recollection and familiarity (Jacoby, 1991). Several studies using this technique have shown that recollection and familiarity behave independently (Curran & Hintzman, 1995). In the ‘remember/know’ procedure developed by Tulving (1985), subjects are required to introspect about the basis of their recognition and report whether their recognition stems from remembering (i.e. recollection of specific, episodic information about the study event) or knowing (i.e. having a feeling of experiencing the stimulus in

25 the past in the absence of recollecting any specific information about it). Because subjects are instructed to respond ‘remember’ whenever they recollect a test item, the probability of a ‘remember’ response can be used as an index of recollection, whereas the probability that an item is familiar is equal to the conditional probability that it received a ‘know’ response given it was not recollected (Yonelinas & Jacoby, 1995). The abovementioned behavioral tasks, plus a few others omitted for brevity, show that recognition memory probably relies on different memory processes, but this is stated with a caveat. Although each method is theoretically motivated and has been supported by empirical evidence, it is likely that at least some of the assumptions that each relies on are violated (Yonlias & Jacoby, 1995). Most theorists, when building a theory of recognition, do not rely on data from any one method; instead, they look for convergence of data across a variety of methods. When approached in such a fashion, these methods support the view that recognition memory can be understood as reflecting two distinct memory processes (Rugg, 2003). To better understand the nature of recognition memory, researchers have employed the methods of cognitive neuroscience, which attempts to understand the underlying neural correlates of cognitive processes. Studying the neural correlates of recognition memory would gives researchers a direct way to test ‘dual process’ theories, which predicts that recollection and familiarity should be reflected by different brain regions or neural mechanisms. If familiarity simply reflects a weak form of recollection—as the ‘single process’ theories suggest—then it should have similar neural correlates that differ only quantitatively from those related to recollection (Rugg and Yonelias, 2003).

26 1.2.2.1 Animal Models Distinctions similar to those in the dual process framework have been proposed in the context of research on animal memory. Models based on research with non-human primates propose that the hippocampus is critical for recollection, whereas familiarity is supported by adjacent medial temporal cortex (O’Reilly & Norman 2002; Brown & Aggleton, 2001) The medial temporal lobe (MTL), a region that includes the hippocampal formation and the parahippocampal, enthorhinal and perirhinal cortices, have dissociable mnemonic functions (Rugg & Yonlias 2003). ‘Delayed non-matching to sample’ is profoundly impaired by lesions confined to perirhinal cortex (Meunier et al, 1993). In a seminal study, Brown and Xiang (1998) found that a significant proportion of perirhinal neurons demonstrate lower firing rates for recently experienced than for experimentally novel objects. Evidence such as this has led to the proposal that the perirhinal region plays a crucial role in familiarity-based recognition (Rugg & Yonelias, 2003). Lesions of the hippocampus of non-human primates severely impairs memory for complex associations (e.g. those between a stimulus event and its context), which has led to the suggestion that the hippocampus plays a crucial role in memory for specific episodes, known as recollection here (Eichenbaum, 2000). Research on non-human primates has given researchers a jump start on the neural substrates of human recognition memory, and given them a framework in which to ask whether different regions of the human MTL contribute differently to recollection and familiarity (Rugg & Yonelias, 2003). It should be noted, however, that some models of human memory hold the view that the hippocampus supports both recognition and familiarity.

27 1.2.2.2 Amnesia Much of the neuropsychological evidence for recognition memory in humans comes from the study of patients with global amnesia (i.e. those who, following neurological insult, demonstrate abnormally low scores on test of memory such as the Weschler Memory Scale, but perform normaly on other cognitive functions). Early dual process models predicted these patients should suffer a selective deficit in recollection (Rugg & Yonelias, 2003). This is based on two findings. First, amnesiacs perform normally on implicit memory tasks (Moscovitch, 1993). If implicit memory and familiarity were equivalent, as was assumed, then familiarity should be not be impaired. Secondly, these patients confused recently presented items with frequently presented items, which should be the case if their decisions were based on a strength-like index such as familiarity. Current evidence does not support the view that familiarity is functionally equivalent to implicit memory or that familiarity is preserved in amnesic patients (Rugg & Yonelias, 2003). As expected if recollection is disproportionately affected, amnesiac patients do show greater impairment in relational recognition than item recognition (Downes et al., 2002; Pickering et al., 1989). Disproportionate deficits in free recall compared with recognition have also been reported in some studies, although others have found equal levels of deficits (Volpe, 1986; MacAndrew, 1994). Studies using the processdissociation, ‘remember-know,’ and behavioral testing indicate that memory deficits in amnesiacs are not confined to recollection (Verfaellie 1993; Knowlton 1995). However, the results do suggest that regardless of the fact that both recollection and other memory

28 processes like familiarity and implicit memory are affected, recollection is disrupted to a greater extent (see Yonelinas, 1998; Rugg & Yonelians, 2003, for review). As mentioned above, animal research on recognition memory has led to the proposal that the hippocampus and perirhinal cortex support processes akin to recollection and familiarity, respectively. Relevant evidence for this proposal comes form patients who exhibit damage to the hippocampus due to hypoxia (loss of oxygen). Postmortum studes indicate that such damage is restricted largely to the hippocampus (Cumming, et al., 1984; Zola, et al., 2000). Vargha-Khadem, et al. (1997) found normal or near-normal performance on an item recognition task but very poor recall in patients with hippocampal damage following hypoxia following early childhood, and Mayes et al. (2002) reported similar findings in adult onset selective hippocampal damage. Yet, other studies of post-hypoxic patients using MRI to localize the damage have failed to show a sparing of recognition memory (Verfaelli et al 2000). Several studies looking at relational memory in post-hypoxic patients also turns up similar conflicting evidence. These studies used ‘associative recognition’ tests which requires the subject to discriminate whether a pair of items were originally paired at study. Dissociations between item and associative recognition have been reported in both early onset and adult onset amnesic patients. However, other studies that have contrasted item recognition with associative recognition have found no evidence of relative sparing of item recognition (Stark & Squire, 2003;). In an important study on the matter, Yonelinas, et al. (2002) using coma duration as an index of severity of hypoxia, calculated the covariation between recall, recognition, and severity of hypoxia. The finding was that hypoxic severity predicted the degree to

29 which recollection, but not familiarity, was impaired. Further, estimates from remember/know and other procedures indicated a selective impairment in recollection. These findings give mixed evidence to the proposal that the hippocampus and perirhinal cortex underlie recollection and familiarity. As of late, the reasons for these divergent findings are unclear; several proposals have been put fourth (see Rugg & Yonelinas, 2003). Regardless, the study of amnesia has shown that at least some patients exhibit a selective loss of recollection, which means that recollection can be dissociated from familiarity. The question of whether the hippocampus plays a selective role in recollection awaits further evidence. 1.2.2.3 ERP and Dual Process Models The Event-Related Potentials (ERP) technique has also been used to study the nature of the neural correlates of recognition memory (see section 1.1.2 for a detailed discussion of the ERP technique). Since ERPs lack spatial resolution, they are unable to directly address questions about the neural substrates of recognition, but are still valuable because they can determine whether the neural correlates of recollection and familiarity differ qualitatively—as indexed by ERP effects that differ in scalp distribution rather than magnitude (Rugg & Yonelinas, 2003). These studies usually compare ERPs elicited by recollection and familiarity. Early reports looking at such comparisons offered limited evidence that recollection and familiarity have distinct neural correlates (for a review see Allen & Rugg, 2000), but later studies using a combination of functional, temporal and neuroanatomical grounds have found that the ERP correlates of familiarity and recollection can be dissociated. For example, Rugg et al. (1998) compared ERPs to a

30 task that involved a ‘shallow’ study task in which familiarity process are used with a ‘deep’ study task in which the probability of both familiarity and recollection process would be used. Both classes of recognition judgment elicited an early, frontal effect whereas the deep studied items elicited both the early effect and a late, left-lateralized positive component found over parietal electrode sites. Rugg et al. concluded that these findings show that the frontal effect is a correlate of familiarity, whereas the later parietal effect indexes recollection. Two studies by Curran (2000; 2003) support the findings of Rugg (1998). In a recognition memory test, Curran (2000) had subjects distinguish between ‘old’ (previously seen) and ‘new’ (unseen) words. Some of the words in the test were similar to, but not identical, with the study words (e.g. CAR CARS). These items elicit a high amount of false alarms, and are believed to give rise to strong familiarity in absence recollection (Hintzman, 1994). The ERPs associated with false alarms elicited the early, frontal positivity relative to items correctly judged new, but did not elicit the late parietal effect found in Rugg. On the other hand, ERPs elicited by recognized old items (indexed by correct responses) showed both the early frontal effect and the late parietal effect, leading to the conclusion that familiarity and recollection were dissociated. Similar findings were found in a similar experiment with pictures (Curran & Cleary 2003). 1.2.2.4 fMRI Evidence The ERP studies above suggest that familiarity and recollection depend on separate neural structures that are, at least, partially non-overlapping, but give no direct evidence as there where in the brain these structures might be. Such information can be supplied by fMRI studies. Two studies using the ‘remember/know’ paradigm found

31 increased left lateral parietal activity for old items endorsed as ‘remembered’ rather than ‘know’ (Henson et al., 1999; Eldridge et al., 2000). Together with ERP studies (above), which indicate parietal ERPs correlate with recollection, these fMRI results suggest that lateral parietal region plays a role in this form of memory (Rugg & Yonelinas, 2003). Eldridge et al. (2000) found greater activity in the hippocampus and adjacent MTL cortex for items subjects marked as ‘remembered’ relative to items marked ‘know’ or ‘new.’ In other studies, the level of hippocampal and perirhinal activity elicited by items during study has been a predictor of subsequent recognition memory performance (Davachi et al, 2003). In sum, the data from fMRI studies supports the view that the hippocampus is more active with recollection than when recognition is based on familiarity, but the possibility that the hippocampus plays a role in both recollection and familiarity cannot be ruled out without further evidence (Rugg & Yonelinas 2003). 1.3 Repetition and the Brain Objects are perceived more quickly and easily if you have been previously exposed to them, even in the absence of conscious awareness (Gauthier, 2003). It is thought that exposure of an item “primes” the representation of the object in the brain which makes for easier subsequent processing of the same or similar items (Rugg, 1995). As mentioned above, this phenomena has been studied by speeded decisions tasks in which priming is indexed by faster reaction times (RT) to repeated than unrepeated items, or tasks that require the subject to identify objects under degraded viewing conditions. For instance, Warren and Morton (1982) found that prior presentation of pictures serves to facilitate the recognition of that picture in a test phase up to 45 minutes later.

32 1.3.2 ERP Repetition Effect Almost universally, ERPs evoked by old items elicit more positive waveforms than new items. Based on the theoretical understanding of the ERP, it is assumed that these different patters reflect disparate brain activity. This effect has been studies in indirect memory tasks where performance is influenced by, but is not dependent, on memory for the study item. Indirect memory tasks are usually thought to engage priming effects and, related implicit memory systems. On the other hand, direct memory tasks have also been employed to study the effects of old items on the ERP. In a direct memory tasks make specific reference to a previous learning episode and encourage subjects to reflect on the content of their memory. Direct tasks are thought to engage explicit or conscious memory. There is quite a bit of literature on the modulate of ERPs by the repetition of words and other stimuli in indirect tests (Rugg, 1990, 1998). In a typical paradigm, the subject might be told to respond to occasional ‘target’ (e.g. non-word) items against a background of repeating ‘non-targets’ (e.g. words). In such studies, ERPs evoked by repeated items have been found to be more positive going than those to the first presentation (for a review, see Rugg, 1995). This has been labeled the ERP Repetition Effect. Modulation of at least two major components is normally found: 1) a reduction in the amplitude of the N400 component and 2) an increase in the amplitude of a late positive component usually associated with the P300 (Besson et al 1992; Joyce et al 1998;). Although this finding is consistent across most studies with words, the opposite effect has been found in studies concerned with the repetition of visual objects. For

33 instance, Rugg (1995) found decreased positively to repeated possible geometric objects (i.e., unlike Escher objects). One proposal for the reverse polarity may have to do with involvement of cortical cells with a different orientations from those activated when words or number stimuli are used, while an alternative explanation holds that it reflects reduced neural activity (Penny, 2001). 1.3.3 ERP ‘Old/New’ Effect In direct memory tests, ERPs can be directly related to subjects’ performance. The typical manipulation involves a study-test paradigm, in which items presented in the study phase are intermixed with new items in a test phase, and the subject is asked to respond ‘old’ or ‘new’ to items in the test (the Curran study discussed above is one such example). Numerous studies have described old/new effects in recognition memory and, like the ERP Repetition Effect, most find that correctly classified old items evoke a more positive-going ERPs than to new items (see Yonelinas, 2003 for a review). The ERP old/new effect has traditionally been broken down into three distinct components, probably reflecting different cognitive processes. The first effect, known as FN400, usually occurs bilaterally at frontal electrode sites between 300-500 ms post stimulus and is thought to reflect familiarity (Johnson, 1995; 1998 Friedman and Johnson, 2000). The second component occurs maximally at parietal electrode sites, particularly left, between 500 and 800 ms and is called the parietal effect. This effect is thought to reflect recollection because it is enhanced by items correctly identified as previously studied (Woodruff et al. 2006). Lastly is the late frontal component (LFC), which occurs later in the waveform over the frontal electrode sites, with right hemisphere

34 predominance. The LFC is thought to reflect post-retrieval verification processes (Ally and Budson, 2007). As can be seen, comparing across studies, the effects seen in repetition studies and old/new studies have similar findings (i.e. ERP components). This is most likely because the difference between the repetition paradigm and the old/new is only in the instructions that subjects receive. For instance, in a repetition task, they might be told to identify visually degraded stimuli or read for comprehension, but be told nothing about repetition. In an old/new study-test paradigm, they might be told respond differently to ‘old’ vs. ‘new’ stimuli.

Similar results are probably due, in part, to the same memory

underlying process used by the two approaches (Van Petten & Senkfor, 1996). 1.3.4 Repetition Suppression Contrary to what would be inferred from the above ERP literature, direct measurements of neural activity reveal that when stimuli are repeated, neural activity is usually reduced (for review see Grill-Spector 2006). This effect has been reported at multiple spatial scales—from the level of individual neurons in monkeys to the pooled activity of millions of neurons using fMRI in humans (Naccache & Dehaene 2001). The effect has also been found across multiple temporal scales—from milliseconds to minutes and days—and across many different brain regions, as well as different experimental conditions (Dale et al., 2000). This effect is commonly referred to as repetition suppression or neural priming, the latter because it is thought to use mnemonic processes related to priming and the former because the suppression in neural activity is thought to indicate a “sharpening’ in the neural networks involved in representing objects (Grill-Spector et al., 2006). In other

35 words, new objects are represented by many broadly tuned neurons and, over repetitions, the neurons carrying the least amount of information (presumably, those not necessary for proper representation) drop off. Concurrently, those neurons which are most informative increase their response and become more efficient—hence, the sharpening process (Gauthier, 2003). The ventral temporal cortex, medial temporal cortex and frontal cortex, as well as earlier sensory processing areas, like the primary visual cortices, have shown reductions (see review Kourtzi &Gill-Spector 2005). fMRI responses tend to decrease monotonically (i.e. sequentially) with the number of repetitions, reaching a ceiling effect at about six to eight presentations (Henson et al 2000; Sayres & Grill-Spector 2003). The effect is maximal when there are very few to no intervening stimuli, but it has been observed with tens of interveining stimuli and after multiple days between stimulus presentations (Grill-Sepector et al. 2006). In conjunction with the ERP data on repetition, most of the literature is in concurrence with these findings, but there are a few lone studies that have found the opposite effect. James et al. (2000), is a prime example. Subjects first passively viewed a sequence of 12 objects each appearing for 1 second and repeated ten times. The typical repetitions suppression was found, with decreasing activity as pictures were repeated in the fusiform gyrus (FG), posterior parietal (PP) and frontal lobes (FL). Next, a gradual unmasking paradigm was used in which six of these objects were gradually revealed. James et al. Data compared the effects of priming before and after object identification by taking images before and after subjects felt they could name the object. The masking

36 paradigm was used to slow recognition. Interestingly, repeated objects evoked more activity in the FG and PP than unfamiliar objects prior to identification. 1.3.5 The Current Study It is difficult to reconcile the disparate findings from the three major topics reviewed above. First, it is becoming clear that recognition memory is supported by multiple memory systems, as described by the dual process framework, and it is assume that these different systems are composed by (at least) partially non-overlapping neural mechanisms. Second, recognition performance is enhanced when one have repeatedly encountered an item. In the ERP literature, this enhancement is usually associated with an increase in amplitude of the waveform which is thought to reflect greater activity in the underlying neural generators. On the other hand, fMRI studies give a contradictory picture of what is happening in the brain when items are repeated. In an attempt to reconcile these findings, James et al. (2000) argues that in accordance with their study, the ERP Repetition Effect might reflect the pre-recognition processes, whereas the fMRI data might the reflect post-retrieval period. On the surface, this proposal makes sense, given that the temporal resolution of the ERP technique (on the order of tens of milliseconds) is much greater relative to fMRI (on the order of hundreds of milliseconds to seconds), but further investigation is needed to reconcile these findings. Regardless of the complications of this hypothesis, it does seem likely that ERP data will give a different picture of brain activity than fMRI. fMRI charts relative blood flow to different areas of the brain, time locked to events, which is on the order of seconds. The ERP technique, on the other hand, can detect brain activity in as little at

37 tens of milliseconds which makes it a much better tool for studying rapid, early stages of cognitive processing involved in recognition memory. Using sophisticated analysis software (EMSE, Source Signal Analysis, LLC), we will attempt to use the source estimation technique described in section 1.1.4 to “fit” the electrodynamic brain activity found at the scalp to sources within the brain. Given the high temporal resolution of the ERP technique, we will be able to reveal information about brain activity on time scales that the fMRI is unable to detect. We believe that experimental factors that affect recognition speed should likewise affect the ERP waveforms that are know to be influenced by recognition processes. Further, since these ERP waveforms found on the scalp are generated by the activity of underlying brain tissue, we believe that the same experimental factors that affect recognition will also affect the brain regions involved in recognition processes which will be revealed by a greater signal strength and decreased signal latency at regions implicated in recognition.

38 Chapter 2 Materials and Methods 2.1 Subjects The goal of this study is to investigate the electrical manifestations of the brain as repeated stimuli are presented to the subject. All subjects were undergraduates enrolled at the University of North Carolina Wilmington and were in normal health, right handed, and had normal or corrected-to-normal eyesight. Excluding three subjects who were personal acquaintances of the experimenter, subjects received participation credit for various psychology courses at the university. Subjects were acquired via a sign-up board in the main hall of the psychology building. All subjects signed an informed consent form approved by the UNCW Institutional Review Board. Subjects ranged between 19-42 years, mean = 21.5. Ten female (mean = 21.4 years) and six males (mean = 20.8) participated in this experiment.

39

Figure 7. Pictures of subject taken from the current experiment.

40 2.2 Equipment Data acquisition was made possible by a 64 channel, high resolution analogue to digital ActiveOne® amplifier system and ActiveView® real-time software. Both are developed by BioSemi® (www.biosemi.com). Post-analysis was completed in EMSE Suite—a modular biosignal data analysis program specifically designed to handle and integrate EEG/ERP and MRI data (http://www.sourcesignal.com). Stimuli were presented with Presentation® which is produced by Neurobehavioral Systems® (www.neuro-bs.com). “Active” electrodes are used, which are amplified through the electrode at the source and do not require abrasion of the skin. The ActiveTwo® head cap is a form-fitting cap designed by Dr. Peter Praamstra at the Behavioral Brain Sciences Center, University of Birmingham, United Kingdom. It is made of elastic and is fitted with custom made plastic electrode holders. The electrodes are not integrated into the cap—they must be physically inserted. Sixty four electrode holders are arranged according to the international 10/20 system (refer to figure 9 below). SignaGel® Electrode Gel is a highly conductive past used to form an electrical connection between the electrodes and the subjects scalp.

41

Figure 8. The second generation ActiveTwo® Active Electrode high density EEG system.

42 2.2.1 Electrode Layout Electrode layout follows the standard international 10/20 system (Horman, 1987). Electrodes are labeled according to a standardized scheme that is composed of two parts. The first part is composed of a letter or short string of letters that corresponds to the underlying brain area. For example, Occipital electrodes are designated ‘O,’ and CentralParental areas with ‘CP.’ The second part refers to the electrode placement relative to the midline. Electrodes found lying on the midline end in ‘Z;’ those left of the midline are assigned odd integers and those to the right have positive integers. The values increase the farther one moves away from the midline. Refer to figure 9 for more detail.

43

Figure 9. International 10/20 Electrode Layout (www.biosemi.com) for 64 electrodes.

44 2.3 Stimuli Real world pictures of animals were taken from Google Images. All pictures had their background removed or replaced with a white background in PhotoShop®. Pictures included a broad variety of animals and plants, ranging from coral reef and plant life to bugs to simple animals to apes. In total, 120 different pictures were presented.

45

Figure 10. Example stimuli from the current experiment.

46 2.4 Procedure 2.4.1 Experiment 1 Subjects were sat approximately 36’ in front of a 15” color CRT Apple ViewSmart® monitor. They were shown how to initiate the stimulus presentation program and told to remain seated and to move as little as possible, but to stay relaxed, and to blink normally. Once the program was started, subjects read a set of instructions which told them they would be presented a series of pictures and to simply fixate at the center of the screen. They were asked to concentrate on the pictures and to try to keep their attention directed towards the computer screen.

47

Figure 11. Example Stimulus on screen.

48 Experimental Paradigm The overall architecture of the experiment is a variation on a traditional ERP paradigm (Rugg, 1995). It was composed of a study and test phase, in which subjects encountered a series of pictures. Each picture was presented for one second, with a one second interval. The study phase was composed of three randomized conditions. In condition 1, pictures were presented once; in condition 2, pictures were presented three times; in condition three, pictures were presented ten times. Pictures were randomized across trials and conditions. Each condition contained 30 pictures, for a total 90 different pictures for the study phase, and 420 trials in all. In the test phase, subjects encountered each picture from the study phase once, plus 30 ‘novel’ pictures (i.e. pictures they have not seen before, or new pictures). Pictures were randomly displayed in the test phase for one second, with a one second interval. The 90 original pictures from the study phase were presented plus 30 novel pictures. This paradigm is similar to that used in typical ‘old/new’ ERP experiments except subjects were not instructed to respond to old versus new pictures (for a review of this paradigm see Rugg, 1995,). This omission was made in order to keep the EEG data as “clean” and free from movement artifacts as possible. Considering that clicking involved numerous muscular movements and task monitoring processes would involve numerous extra processes in the brain, it was assumed by the researchers that by omitting overt responses, event related potentials evoked from stimuli would be easier to detect. Several previous studies have shown that ERP waveforms elicited from false alarms are qualitatively different from those elicited by hits (Yonelias, 2003). Therefore,

49 it may be argued, by simply averaging the EEG data for all old pictures, we are conflating false alarm ERP data with ERPs elicited by hits. Although this may be true, the purpose of this experiment was to ascertain whether the brain is responding differently to repeated stimuli. Therefore, if we find significant differences between conditions, it can be argued that hits minus false alarms factored out. Having said that, we collected reaction time data in the exact same paradigm in Experiment 2 (without EEG) in order to test the hypothesis that subjects would respond faster to repeated stimuli.

50

Example of Study:

Seen 10x

Highly Familiar

Seen 3x

Familiar

Seen 1x

Less Familiar

Figure 12. Design of study and test phase of Experiment 1.

Novel

51 EEG/ERP Procedure EEG was recorded from a full array of 64 Ag–AgCl BioSemi (Amsterd, Netherlands) active electrodes embedded in the ActiveTwo electrode cap. EEG activity was amplified with a bandwidth of 0.03-35HZ (3 dB) and digitized at a sampling rate of 512 Hz. Reference in the recording was made to a common average reference (Dien, 1998). The sampling epoch for each trial started 250 ms before onset of stimulus and ended 1800 ms post-stimulus. Raw data files were imported into the EMSE Software Suite (Source Signal Imaging, San Diego, CA) and ERPs were averaged and corrected. Excessive EOG activity was correct for by using the empirical EMSE Ocular Artifact Correction Tool, methodology the investigator learned while attending two separate workshops hosted by Source Signal Imaging, LLC. and CortechSolutions, LLC. The Ocular Artifact Correction Tool allows the investigator to manually distinguish artifact data from clean data. Then, a logarithmic ratio of artifact data versus clean data is produced by EMSE to subtract artifact data where it is detected. Statistical Analysis The statistical analysis of this experiment using the source estimation method discussed in section 1.1.4. Beamformer analysis was performed on averaged files for each subject (four separate conditions for each subject) using four bilateral regions of interest (ROIs) selected from the review of recognition memory and repetition discussed in Chapter 1. The ROIs were the left and right anterior cingulated, left and right medial frontal gyrus, left and right hippocampus, and cuneus and lingual gyrus (primary occipital cortex).

52 Source files for each of the four conditions for each subject were saved. Next, peak detection analysis was performed on each source file. Peaks were defined from previous research using the ‘old/new’ paradigm (Ally & Budson, 2007). A text file was created by the peak detection analysis that contained information about peak amplitude, latency, and area under the curve for each ROI within each condition for each subject. ROIs were broken down into channels so that particular regions could be compared. 2.4.2 Experiment 2 Experiment 2 was identical to Experiment 1, except reaction time data was recorded from subjects instead of EEG. Subjects were given the same instructions from Experiment 1, except instead of being told to passively observe the set of pictures in the test phase, they were instructed to click one of two response buttons on a pad—left click for ‘old’ pictures and right click for ‘new’ pictures. This is the typical “old/new” studytest paradigm (discussed above).

53 Chapter 3 Results Results from experiment 2 are presented first to support the claim that repetition would increase recognition in the visual experiment. Next, visual results are given from the ERP experiment to show the basic ERP old/new effect is present in the data. Finally, statistical analysis from the source estimation technique is provided to support the claim that repetition would increase relative brain activity in selective areas that are involved with the increase in recognition seen in Experiment 2. 3.1 Experiment 2 Figure 13 shows the reaction times for each of the stimulus frequency conditions. Relative to novel pictures, a trend toward faster responding was observed for the pictures that were repeated 3 and 10 times during the study phase of training.

54

1050 1000 950 900 850 800

0

1

2

3

4

5

6

7

8

9

10

Picture Frequency Figure 13: Reaction times as a function of picture repetition frequency during the training phase. A repeated measures ANOVA revealed a significant effect of picture frequency; F(3,33)=12.37, p