NEURAL BASIS OF EMOTIONAL MODULATION OF ...

Seaman, S., Hsieh, L., Wu, L., & Young, R. (2010). Neural basis of emotional modulation while multitasking: ERP analysis. Poster at the Cognitive Neuroscience Society, Montreal, Quebec, Canada. 4/19/10.

NEURAL BASIS OF EMOTIONAL MODULATION OF SIMULATED DRIVING PERFORMANCE: AN fMRI MULTITASKING STUDY Li Hsieh1, Sean Seaman1, Quan Jiang2, Susan Bowyer2, John Moran2, Richard Young3 1

Wayne State University, Henry Ford Hospital, 3 School of Medicine, Wayne State University 2

This fMRI study investigated the role of emotion in multitasking using a multi-modal task designed to assess the effect of emotional speech on visual event detection during simulated driving. Ten participants were asked to respond to visual stimuli in a go/no go design while covertly answering spoken questions. Behavioral results showed longer visual reaction times during a concurrent speech task than with no speech; however this effect was moderated by presenting speech questions in an angry voice. fMRI analysis indicated increased activations (t > 3.2; p < 0.0014) associated with both neutral and angry speech tasks, compared to no speech, in the bilateral temporal lobes, the left inferior frontal gyrus, and the left middle frontal gyrus; and decreased activations in the right inferior parietal lobe and the right cuneus. Direct comparisons between angry and neutral speech tasks showed increased activations (t > 2.8; p < 0.0051) in the right prefrontal gyrus, the right middle frontal gyrus (BA10), the right insular, the right superior temporal gyrus, the right paracentral lobule (BA5), the right claustrum, and the right inferior parietal lobe (BA40). Decreased activations were found in the left frontal operculum, the left lingual gyrus (BA18), and the left parahippocamal gyrus (BA28). These results suggest that speech compared to no speech causes slightly longer behavioral reaction times and increased brain activation in language areas. Moreover, an angry emotional tone improves behavioral reaction time performance compared to a neutral tone, while eliciting the right frontoparietal networks and dampening the left frontal activity.

4/18/2010

Seaman, S., Hsieh, L., Wu, L., & Young, R. (2010). Neural basis of emotional modulation while multitasking: ERP analysis. Poster at the Cognitive Neuroscience Society, Montreal, Quebec, Canada. 4/19/10.

NEURAL BASIS OF EMOTIONAL MODULATION OF SIMULATED DRIVING PERFORMANCE: AN fMRI MULTITASKING STUDY Authors:

Li Hsieh1, Sean Seaman1, Quan Jiang2, Susan Bowyer2, John Moran2 and Richard Young3 Affiliations: 1Department

of Communication Sciences and Disorders, Wayne State y, Detroit,, MI USA University,

2Department

of Neurology, Henry Ford Hospital, Detroit, MI USA

3Department

of Psychiatry and Behavioral Neurosciences, School of Medicine, Wayne State University, Detroit, MI USA

1

Speech Emotion Attention Cognitive Memory Complexity

Overview – Driving as a Research Platform

• Driving is commonly referred to as a single task • But driving is really complex multitasking – It involves sensory, motor, and higher-level cognitive components – These play out over different levels of functional hierarchy, on different timescales, and often concurrently (Michon, 1989; 2 Groeger, 2000).

Emotional Modulation Driving: fMRI

1

4/18/2010

Overview: Multitasking & Language • Multitasking experiments give language q p platform for viewing g researchers a unique language use in one of its most natural forms – in a multisensory processing environment. • By seeing how the brain allocates esou ces for o co conversation e sat o while ee engaged gaged resources in a driving task, we can find what the shared anatomy and networks are for language, executive functions, and general perceptual processing. 3

Introduction • Many investigations into human performance involve the use of driving simulators • What is often lacking is a validation of these simulators using on on-road road data • Behavioral validation studies for predicting event detection on the road from lab data have recently had excellent results (Young et al., 2009) - Correlations of more than 0.9 for brake reaction times to visual events.

• Validation is especiallyy important g given: 1. The cognitive complexity of a real-world driving task and 2. The public policy implications of driver performance research.

• This study is the first we know of to apply a neuroscience measure (fMRI) to a validated driving paradigm. 4


2

4/18/2010

Previous Cognitive Neuroscience Studies of Conversation Effect on Driving • Project undertaken with Wayne State U i University, it U Univ. i off Mi Michigan, hi and dH Henry Ford Hospital to improve these predictions and understand underlying neural mechanisms. • Bowyer et al. (2009) for MEG measures of event detection while driving • Hsieh et al. (2009) for fMRI measures 5

Previous MEG Findings •

•

•

Right Superior Parietal Region activation appeared ~240 ms after red light that precedes a brake response, averaged across all subjects subjects. Right superior parietal region is an association area where higher functioning and integration of multiple sensory inputs occurs (i.e. visual and motor). Decreased correlation with reaction times in the right superior parietal region (Red Arrows) during conversation condition (E) compared to no conversation (D).

(D) No-Conversation

R

L

(E) Conversation

R

L

6

- Bowyer et al. (2009)


3

4/18/2010

Previous fMRI Findings: Red Light Responses

L

7

- Hsieh et al. (2009))

R

Goals 1. To evaluate conversation and emotional effects on visual event detection in a multitasking scenario (Enhanced Static Load Task (ESLT), Young et al., al 2009) in both the lab and MRI scanner. 2. To investigate fMRI brain activations and compare the behavioral performance between behavioral lab and fMRI sites. 3. To identify brain activation patterns mediating emotional changes during driving while engaged in cellular conversations.

8


4

4/18/2010

Methods • A battery of behavioral measures estimated the effects of a simulated cellular phone conversation on driving performance. • Collected fMRI data using a 3T GE MRI at Henry Ford Hospital. • Ten participants were engaged in simulated driving and responded to visual target events using light stimuli positioned around the driver’s field of view in the lab and the fMRI scanner. • In both environments, participants’ reaction times to g visual events were assessed,, and functional MRI signals were recorded to the visual events. • While in the fMRI scanner or in the lab, participants covertly answered questions from an experimenter in a hands-free cellular phone conversation. 9

Experiment • • • • • •

Neutral vs. angry conversations. Compared with no conversation baseline. Also did manual task of answering a phone ring. 10 participants performed hands-free phone conversations d i simulated during i l t dd driving i i iin th the llab. b 10 participants performed hands-free phone conversation in the fMRI scanner. Four different calls were received (each lasting 1 minute) in each 9 minute block. – Each participant completed 3 runs (4 conversations each), and a baseline run (without any conversation). Conversation + Driving Scene (repeated 4 times, 58.125 seconds)

Driving Scene alone (repeated 5 times, 36.325-58.120 seconds)

10


5

4/18/2010

Task Paradigm • In this fMRI study, ten participants were asked to respond to visual stimuli in a target/non-target design while covertly answering spoken questions. • This is essentially a Go/No Go task – participants are asked to respond to red circle stimuli, and not to respond to green circle stimuli. • The same driving paradigm was also conducted in the behavioral lab (below) and in the fMRI scanner.

11

Functional Imaging Preparation • Before imaging these results in fMRI, we assessed d th the iinfluence fl off one iimaging i restraint – the need for covert speech. • Multitasks of driving and conversations were replicated using covert speech – asking participants to respond without substantially moving their articulators.

12


6

4/18/2010

Overt vs. Covert - RT RT (msec) as a function of block, speech-type 1000

Conversation No Conversation Ring g

900 800 RT (msec)

700 600 500 400 300 200 100 0 Covert

Overt Speech-type

13

fMRI Adaptation of ESLT Mirror Participant

Screen Projector

• Due to the restraints of the scanner, the visual angle from the center light to the l ft light left li ht was reduced d d from f 20 degrees to 12 degrees. • Participants steered using a handheld controller with two buttons, and responded to red targets using a foot pedal.


14

7

4/18/2010

Conversation Effect on Simulated Driving in the Lab and fMRI: Behavioral Performance (RTs) 780 760

•Significant Conv. RT Effect in both sites: (F(9,2) = 5.24, p < .01) • 2.8; p < 0.0051) in the right prefrontal gyrus, the right middle frontal gyrus (BA10), the right insular, the right superior temporal gyrus, the right paracentral lobule (BA5), the right claustrum, and the right inferior parietal lobe (BA40). Decreased activations were found in the left frontal operculum, the left 18 lingual gyrus (BA18), and the left parahippocampal gyrus (BA28).


9

4/18/2010

Conclusions 1.

2. 3. 4. 5.

6.

Hands-free cellular phone conversations caused significant but small reaction time (RT) effects for visual event detection for simulated driving in the lab and MRI scanner. ANOVAs shown statistical significance of conversation effects on RT in both g settings. Consistently high accuracy for visual event detection was found regardless of emotionality of speech content during cellular phone conversations in the lab and MRI scanner. Angry conversations may increase the alertness for event detection. These fMRI results suggest that speech compared to no speech causes slightly longer behavioral reaction times and increased brain activation in language areas. Moreover, an angry emotional tone improves behavioral reaction time performance compared to a neutral tone, while eliciting the right frontoparietal networks and dampening the left frontal activity. In conclusion, this fMRI study suggests that a top-down emotional and attentional processing mediated by the frontalparietal network.

References •

•

•

• •

• • •

•

Angell L, Young R, Hankey J, Dingus T (2002). An evaluation of alternative methods for assessing driver workload in the early development of in-vehicle information systems. SAE Proceedings, 2002-01-1981, May. Angell L, Auflick J, Austria PA, Kocchar D, Tijerina L, Biever W, Diptiman T, Hogsett J, Kiger S (2006). Driver Workload Metrics Task 2 Final Report & Appendices. Final Research Report: DOT HS 810 635. NHTSA/USDOT, Washington, D.C. Bowyer, S.M., Hsieh, L., Moran, J.E., Young, R.A., Manoharan, A., Liao, C.-C., Malladi, K., Yu, Y.-J., g, Y.-R.,, Tepley, p y, N.,, 2009. Conversation effects on neural mechanisms underlying y g reaction time to Chiang, visual events while viewing a driving scene using MEG. Brain Research 1251, 151-161. Groeger, J.A. (2000). Understanding Driving: Applying Cognitive Psychology to a Complex Everyday Task. Psychology Press, New York, NY. Hsieh, L., Young, R., Bowyer, S.M., Moran, J.E., Genik, R.J. II, Green, C.C., Chiang, Y.R., Yu, Y.J., Liao, C.C., Seaman, S., 2009. Conversation effects on neural mechanisms underlying reaction time to visual events while viewing a driving scene: fMRI analysis and asynchrony model. Brain Research 1251, 162– 175. Michon, J.A., 1989. Explanatory pitfalls and rule-based driver models. Accident. Analysis and Prevention,. 21, 341–353. Schupp, H. T., Öhman, A., Junghöfer, J., Weike, A. I., Stockburger, J., & Hamm, A. O. (2004). The facilitated processing of threatening faces: An ERP analysis. Emotion, 4, 189-200. Young RA, RA Aryal BJ, BJ Muresan M, M Ding X, X Oja S, S Simpson SN (2005). (2005) Road Road-to-lab: to lab: Validation of the static load test for predicting on-road driving performance while using advanced in-vehicle information and communication devices. Proc. of the Third International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design. Young, R. A., Angell, L., Sullivan, J. M., Seaman, S., & Hsieh, L. (2009). Validation of the static load test for event detection during hands-free conversation Proceedings of the Fifth International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, 5, 268-275.

20


10