Neuro-Cognitive Correlation of Memory Effectiveness and Emotional Arousal Khamis F. Alarabi and Abdul Wahab Department of Computer Science, International Islamic University Malaysia, P.O. Box 10, 50728 Kuala Lumpur, Malaysia
[email protected],
[email protected]
Studies have shown that memory effectiveness can be greatly influenced by emotional arousal. In this paper we have investigated the relationship between EEG-based emotional arousal and memory effectiveness. To scope our research in this work, we have considered information encoding and information retrieval operations as memory effectiveness. Previous studies indicated that, there is a strong correlation between emotions and memory consolidation. Other studies have reported that amygdala plays a crucial role for exerting the influence of emotional arousal. This role has been suggested to be greater during information encoding than retrieval. Therefore, four postgraduate students volunteered to participate in this memory test experiment, and their EEG brain wave signals were captured and recorded during test time. The results of processing their brain signals have provided evidence that the high level of emotional arousal is strongly correlated with memory consolidation. The results also support the notion that, the amygdala plays important role during information encoding rather than during information retrieval. effectiveness,
Universiti Teknologi Mara (UiTM), Kampus Jasin
77300 Merlimau, Jasin, Melaka, Malaysia
[email protected]
no difference for patients to remember emotionally arousal stimuli than neutral ones [9].
Abstract
Keywords: memory affective space model.
N. Kamaruddin Fakulti Sains Komputer dan Matematik,
EEG,
Arousal,
1. Introduction Several studies on human memory have investigated the relationship between emotions and memory effectiveness. Emotions have been suggested having a great impact on memory. Findings from laboratory studies have shown that emotional information can be better remembered than non-emotional ones. These findings suggested that emotional arousal supports memory effectiveness during information encoding and information retrieval [3]. The amygdala, a small structure in the medial temporal lobe of brain - has been suggested to play a crucial role in memory modulation [15][13]. Findings from neuroscience studies on brain support the notion that amygdala activity and adrenergic system influence memory [13]. Neuropsychological studies on patients with damage of amygdala reported that, there was
On the other hand, Neuroimaging studies reported that, the degree of amygdala activation is correlated to the emotionally arousing of the stimuli during encoding [1] and have shown that there is a high correlation between amygdala activity during encoding and the amount of memory recall of emotionally arousing stimuli[1][4]. Regarding these findings, and considering only the relationship between arousal and memory performance, it can be concluded that, the high level of arousal during information encoding, the high performance of the memory retrieval. This conclusion is corresponding to the arousal-performance curve, which was introduced by Yerkes and Dodson[2].
1.1 Memory test The arousal-performance curve of Yerkes and Dadson seems to indicate memory effectiveness is greatly influenced by emotional arousal. Although many studies have discussed how long-term memory can be tested, and to what extent the emotional arousal can influence it, Frank and Tomaz have investigated the enhancement of declarative memory that associated with emotional arousal. They have tested declarative memory through free recall and recognition tests. The findings of their work have supported the evidence that emotional arousal was associated with enhanced memory [3]. On the other hand, there was no standard scale to measure the level of arousal. Some studies have reported that arousal of stimuli can be measured by scaling the stimuli from sleepiness to active arousal using 1-9 points scale [10,17]. In this paper we makes use of the Encephalogram (EEG) device to monitor the brainwaves signals and extract the arousals based on the affective space model (ASM) during the memory testing. It has shown to be a potential ways to quantify arousal and valence effectively during memory testing.
2. Emotion modeling and recognition Emotions are mental and physiological states that affect cognition, perception and other psychological processes. There are many terms that can be used to describe our emotional states, such as Happy, Fear, excited and depressed. J. Russel (1980) suggested a circumplex model of affect as shown in figure 1 that helps describing the affect as a set of dimensions. Emotions then can be represented as coordinates on this two dimension affective space model (ASM) [17]. From figure 1 only two dimension of affect is considered based on the valence and arousal. As can be seen from figure 1the vertical axis, which represents the arousal, ranged from sleepiness to active arousal, and the horizontal axis which represents the valence ranged from pleasure to misery [12].
were two males and two females; their mean age was 29.5 years old. All participants were right-handed, and no one has experienced any neurological disease or brain injury. Before they participate in this study, all subjects have signed a printed consent form and have been informed with steps of experiment and that their memories will be later tested.
3.2 Materials To ensure the validity of the stimuli, we have chosen 40 words from the set of stimuli that has been suggested by Sondgrass and Vanderwart [14]. These 5-9-letter sets of words have been used as stimuli in many memory test studies, such as experiments of Rajaram and Roediger in 1993 [11].
3.3 Procedure Each participant needed to first signed a consent form that he/she was agreeing to participate in this experimental study which will include a recording of his/her brain signals using EEG device. Each participant had indicated that he/she was not in any drug and has not taken any medications that may affect his/her neurophysiological system. Participants were individually tested. Each of them was seated in a quiet room, in front of a 15-inch laptop monitor. The experimental protocol as shown in Figure 2, consists of several steps: a)
Figure 1: Affective space model.
2.1 Measuring emotions using EEG Brain activities including emotions can be recorded as electrical signals using EEG machines. Previous studies have reported the ability to recognize person’s emotions using EEG, where the signals have to be processed to recognize the emotion by calculating the values of its components, arousal and valence, and then plot them on the affective space model. As emotions are mental states, and they can be recorded within brain signals at any time of brain activities, we aimed in this paper to investigate the relationship between memory effectiveness and emotional arousal calculated from EEG brain signals.
Resting (eyes open/ eyes close) test, each test lasted for one minute. b) Emotional reference exposure, where participant were exposed to four groups of The Radboud Faces Database (RaFD) emotional stimuli [8]. These emotions were happy, fear, sad, and calm, and each one lasted for 1 minute. c) Memory stimuli were presented as a sequence of pictures; each picture lasted for 8 seconds.
d) Figure 2: Experimental protocol
3. Experiment methodology 3.1 Participants Four postgraduate students from International Islamic University of Malaysia participated in this study. They
Immediately after testing the emotions, participant now perform the first phase of the long-term memory test, which was the study phase. In this phase participant was exposed to twenty words from the materials mentioned earlier. Each word was presented on laptop monitor in black color against white background, and in 16 points
size “Lucida console” font for only 200 ms separated by blank frame of 1500 ms. As a distraction task to separate study (encoding) phase from the next one (memory test), participant was given a paper and pen, and was asked to write as much capitals of Asian countries as they can in 3- minute time. Just after the distraction task was completed, participant was exposed to 40 words, 20 of them were the same words which have been presented in encoding phase and the other 20 were new words. All words were chosen from the same pool and have been presented with the same way in encoding phase. Participant in this declarative memory test phase have to decide whether the presented word has been presented in study phase or not by pressing left arrow key or right arrow key on the laptop keyboard for yes and for no respectively. All experiment events were controlled by a C++ computer program, which was developed especially for this task. The program was also developed to record participant’s responses and their reaction time.
3.4 Data collection EEG brain signals were captured and recorded for each subject using eight-channel (BIMEC) EEG device during resting test, emotion test, encoding or study phase, and memory test phase and was time stamped for syncing with the test. The eight channels as shown in figure 2 were FP1, FP2, F3, F4, P3, P4, T3, and T4 in international 10-20 system. CZ was also used as a reference channel. Data were recorded at a frequency sampling of 250 Hz.
3.5 Data analysis In this work we have used Matlab® software to process the acquired signals, where EEG device recorded raw data which include brain signals and noise. Therefore, signals have been pre-processed by applying a digital filter to reduce the noise of AC power and artifact and to cut the high frequencies, because brain activities are low (< 100 Hz) frequency signals [16]. All steps that we have applied to analyze EEG signals for emotion recognition are shown in figure 3. The output of classifier was the components of emotion; valence and arousal. These values are easily plotted on affective space map to represent the emotion. Previous studies reported that emotions are mental states and they can be detected at any given time and for any neural activity. Therefore, we have analyzed subject’s emotions during all experiment events except that for distraction task. Results and their supporting details were discussed in next section.
EEG raw data
Steps direction ------------------------- PreFeature Classifier processing extraction Low pass KDE MLP filter 0.5-80 Hz
Valence and Arousal
Figure 3: Emotion recognition steps The output of classifier was the components of emotion; valence and arousal. These values are easily plotted on affective space map to represent the emotion. Previous studies reported that emotions are mental states and they can be detected at any given time and for any neural activity. Therefore, we have analyzed subject’s emotions during all experiment events except that for distraction task. Results and their supporting details were discussed in next section.
4. Results and discussion The main objective of this work was to investigate the relationships between memory effectiveness and emotions using EEG technique. Emotions have been suggested to play a crucial role during the operation of information encoding in long term memory and during information retrieval. To scope our research, we have focused in this paper on the role of emotional arousal on modulating declarative (or explicit) long-term memory. Neuroimaging studies have reported that amygdala activation during information encoding is correlated with the memory of emotional stimuli [1,6]. Thus, our initial hypothesis was that emotional arousal evaluated using EEG brain signals were correlated with the memory effectiveness. As shown in figure 4 we have compared the participants’ responses during word recognition (declarative) memory test when the calculated arousal was higher than the average arousal for each subject during encoding phase. The results explain the percentage of correct against false answers in the test for four subjects; S1, S2, S3, and S4. It can be seen that the percentage of correct answers was much higher than the percentage of false for subjects S1, S3, S4, while was opposite for S2. However, the average of correct answers for all subjects was consistent with results of S1, S3 and S4, and was consistent also with the notion suggested that the higher arousal during encoding can enhance memory retention [1,6,9,15]. On the other hand, we have tested the hypothesis that amygdala is more active at the earlier phases of memory operations, including information encoding [5,7]. We have found that the level of arousal during information encoding (study) phase for subjects S1, S2 and S3 were
significantly higher than its level during memory test (information retrieval) as shown in figure 5. The exception was for S4, where the level of arousal during encoding phase was slightly less than its level during retrieval. These findings are consistent with the findings of LaBar and Cabaza, which reported that the role of amygdala is emphasized during early stages of memory including the encoding and consolidation rather than during memory retrieval [7].
Acknowledgement This research was supported by FGRS research grant at the International Islamic University Malaysia (UIA).
3. References [1]
[2]
[3]
Figure 4: Percentage of responses’ for subjects
[4]
[5]
[6]
[7] Figure 5: level of arousal during study and test phases [8]
2. Conclusion To sum up, the aim of the current study was to investigate the relationship between memory effectiveness and the impact of emotional arousal through EEG brain signals. The results of this research demonstrated that there was a strong correlation between the high level of arousal and memory enhancement. Furthermore, the high level of arousal was associated with encoding phase of information in long term memory rather than with the retrieval operation in declarative memory. The both findings were consistent with the findings of previous studies of [1,6,9,15] and [5,7] respectively.
[9] [10]
[11]
[12]
L. Cahill, R. J. Haier, J. Fallon, M. T. Alkire, C. Tang, D. Keator, J. Wu, and J. L. McGaugh, “Amygdala activity at encoding correlated with long-term, free recall of emotional information.,” Proc. Natl. Acad. Sci. U.S. A., vol. 93, no. 15, pp. 8016–8021, Jul. 1996. R. E. Cochran, F. J. Lee, and E. Chown, “Modeling emotion: Arousal’s impact on memory,” in Proceedings of the 28th Annual Conference of the Cognitive Science Society, 2006, pp. 1133–1138. J. E. Frank and C. Tomaz, “Enhancement of declarative memory associated with emotional content in a Brazilian sample,” Braz. J. Med. Biol. Res., vol. 33, no. 12, pp. 1483–1489, Dec. 2000. E. A. Kensinger and D. L. Schacter, “Retrieving accurate and distorted memories: neuroimaging evidence for effects of emotion,” NeuroImage, vol. 27, no. 1, pp. 167–177, Aug. 2005. E. A. Kensinger, D. R. Addis, and R. K. Atapattu, “Amygdala activity at encoding corresponds with memory vividness and with memory for select episodic details,” Neuropsychologia, vol. 49, no. 4, pp. 663–673, 2011. M. Knight and M. Mather, “Reconciling findings of emotion-induced memory enhancement and impairment of preceding items.,” Emotion, vol. 9, no. 6, p. 763, 2009. K. S. LaBar and R. Cabeza, “Cognitive neuroscience of emotional memory,” Nat. Rev. Neurosci., vol. 7, no. 1, pp. 54–64, 2006. P. J. Lang, M. M. Bradley, and B. N. Cuthbert, International affective picture system (IAPS): Technical manual and affective ratings. Gainesville, FL: The Center for Research in Psychophysiology, University of Florida, 1999. M. Lewis, J. M. Haviland-Jones, and L. F. Barrett, Handbook of emotions. Guilford Press, 2010. K. R. Mickley Steinmetz, K. Schmidt, H. R. Zucker, and E. A. Kensinger, “The effect of emotional arousal and retention delay on subsequent-memory effects,” Cogn. Neurosci., vol. 3, no. 3–4, pp. 150–159, 2012. S. Rajaram and H. L. Roediger, “Direct comparison of four implicit memory tests.,” J. Exp. Psychol. Learn. Mem. Cogn., vol. 19, no. 4, p. 765, 1993. J. A. Russell, “A circumplex model of affect.,” J.
[13]
[14]
[15]
[16] [17]
Pers. Soc. Psychol., vol. 39, no. 6, p. 1161, 1980. T. Sharot and E. A. Phelps, “How arousal modulates memory: disentangling the effects of attention and retention,” Cogn. Affect. Behav. Neurosci., vol. 4, no. 3, pp. 294–306, Sep. 2004. J. G. Snodgrass and M. Vanderwart, “A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity.,” J. Exp. Psychol. [Hum. Learn.], vol. 6, no. 2, p. 174, 1980. S. Steidl, S. Mohi-uddin, and A. K. Anderson, “Effects of emotional arousal on multiple memory systems: Evidence from declarative and procedural learning,” Learn. Mem., vol. 13, no. 5, pp. 650– 658, Sep. 2006. M. Teplan, “Fundamentals of EEG measurement,” Meas. Sci. Rev., vol. 2, no. 2, pp. 1–11, 2002. N. Kamaruddin, A. Wahab, “Human Behavior state Profile Mapping based on Recalibrated Speech Affective Space Model”, 34th Annual International Conference of the IEEE Engineering in Medicine and Biology society, San Diego, California, USA, pp. 2021-2014, 28th August – 1st September 2012.
