2011 Third International Conference on Computational Intelligence, Communication Systems and Networks
Intelligent System for Assessing Human Stress Using EEG Signals and Psychoanalysis Tests
Norizam Sulaiman1,2
Mohd Nasir Taib2,3, Sahrim Lias2,3, Zunairah Hj Murat2,3, Siti Armiza Mohd Aris2, Mahfuzah Mustafa2, Nazre Abdul Rashid2, Noor Hayatee Abdul Hamid2,3
1
Faculty of Electrical & Electronics Engineering Universiti Malaysia Pahang, Kuantan, Pahang, Malaysia
[email protected]
2
3
classifiers such as Artificial Neural Network (ANN), kNearest Neighbor (k-NN), Linear Discriminant Analysis (LDA), Support Vector Machines (SVM), Random Forests (RF), Neuro-Fuzzy System (NFS) and Bayesian to classify the stress level [4-5]. Thus, stress assessment can be implemented by both methods; self-report questionnaires and physiological signals.
Abstract—This paper presents a results of designing an intelligent system to evaluate human stress level using Electroencephalogram (EEG) signals and Psychoanalysis tests. The questionnaires for Psychoanalysis tests were created based on Cohen’s Perceived Stress Scale (PSS). EEG signals were captured using wireless EEG equipment. The Graphical User Interface (GUI) for the Psychoanalysis tests and EEG signals were created. The system was evaluated for 12 healthy subjects (7 females and 5 males). The results show that the intelligent system able to display the stress score, stress level and dominant index of EEG signals simultaneously. Thus, users can use the results of the system to take necessary action in order to improve their lifestyle.
II.
INTRODUCTION
Human stress can be assessed by either using self-report questionnaires or analyzing physiological signals. Stress is associated with negative psychological states [1]. Researchers had developed various types of questionnaires to evaluate human’s level of stress. Perceived Stress Scale (PSS), Stress Response Inventory (SRI), Life Event and Coping Inventory (LECI) and Hamilton Depression Rating Scale (HDRS) are among the most popular questionnaires used to determine human’s level of stress. Typically, stress are categorized into 3 levels; low, moderate and high. Among the physiological signals used by researchers to assess stress include Electrocardiogram (ECG), Heart Rate Variability (HRV), Electromyogram (EMG), Galvanic Skin Response (GSR), Skin Temperature (ST), Blood Pressure (BP), Blood Volume Pulse (BVP), Respiration and Salivary Cortisol [2]. These physiological signals reflect the imbalance in Autonomic Nervous System (ANS) due to stressors (stress factors) [3]. Feature extraction methods play vital role in selecting stress features from the physiological signals. Then, the extracted stress features are used by
978-0-7695-4482-3/11 $26.00 © 2011 IEEE DOI 10.1109/CICSyN.2011.82
REVIEW OF LITERATURE
Various integration systems were introduced by researchers in assessing human stress. Katsis et al. [6] had developed an integrated system based on physiological signals (BVP, HRV, GSR and Respiration) to assess affective state of patients with anxiety disorders. Wilson and Russell [7] had developed real-time assessment system to assess mental workload using physiological signals such as EEG, HRV and respiration. Online stress detection technique was developed in human-robot cooperation activities so that robot would recognize human psychological states and respond appropriately [8]. Healey and Picard, Haak et al. and Picot et al. [2, 9, 10] had developed a system to assess stress of car-drivers using physiological signals such as ECG, EMG, GSR and respiration. In addition, Jovanov et al. [11] had developed distributed wireless system using HRV data in order to monitor human stress.
Keywords-Intelligent system, Psychoanalysis tests, EEG signals, GUI, Stress Level, EEG dominant index
I.
Faculty of Electrical Engineering, Nondesctructive Biomedical and Pharmaceutical Research Centre, Universiti Teknologi MARA, Shah Alam, Selangor, Malaysia
[email protected]
III.
PROPOSED INTELLIGENT SYSTEM
Previous studies had introduced integration system to evaluate human stress either by using various physiological signals or self-report questionnaires. However, the studies had yet to come out with a system which combines the physiological signals with self-report questionnaires or psychoanalysis tests in assessing human stress. In addition, the previous studies had yet to come out with a good relationship between EEG dominant indexes with the stress questionnaires. Hence, the main goal of this paper is to develop an intelligent or user-interactive system which is able to assess 363
human stress level from both Psychoanalysis tests and EEG signals. The system involves the development of GUI and the brain dominant analysis. The developed intelligent system will enable users to know their stress level based on questionnaires and the brain dominant based on the EEG analysis. The novelty of the system is the combination of the stress questionnaires with the EEG dominant analysis which able to indicate the relationship between both methods in determining stress level. IV.
TABLE I.
The experiments were carried out involving 12 healthy subjects at Biomedical Research and Development Laboratory for Human Potential, University Teknologi MARA (UiTM), Shah Alam, Malaysia. The subjects consisted of 5 males and 7 females (age range from 21 to 40 years old). Prior to EEG measurement, subjects were instructed to register and sign the consent form. The study was conducted under approval of Local Ethics Committee. B. EEG Measurement and Protocol The EEG signal was recorded using EEG Data Acquisition instrument (g.MOBILab) employing bipolar EEG gold-plated electrodes which were placed at prefrontal area of brain region, Fp1 and Fp2 and references to earlobes A1, A2 and Fz as shown in Fig. 1. This montage followed the International 10-20 system [12]. The impedance for EEG electrodes was checked below 5 kΩ using Z-checker. The sampling rate for EEG measurement was set to 256 Hz. In addition, the EEG waveforms conditions were checked for any errors. Channel 2 (Left) Channel 1 (Right)
g-MOBILab
5 Seconds
3 Minutes
System Initialization
EEG recording
Within 3 minutes
Stress Test
The overall process flow of the experiment is shown in Fig. 2. The initialization process is done by clicking the test signal button as shown in Fig. 3. The condition of the signal will be checked in duration of 5 seconds. If no abnormality observed during 5 seconds of signal testing, the testing can be stopped by pressing reset/clear button and ready for registration process. Otherwise, need to call laboratory technician to check the condition of EEG equipment and electrodes. Next, registration is done by entering the subject’s name and identity card in the provided space in GUI window as shown in Fig. 4. Prior to performing psychoanalysis tests or answering stress questionnaires, the EEG signals recording are activated for 3 minutes by clicking stress/calmness button as shown in Fig. 3. During EEG recording, the subjects are asked to close their eyes and minimize their movement. Once EEG recording finished, stress questionnaires are answered by clicking start button as shown in Fig. 4. All 10 stress questionnaires as shown in Fig. 5 need to be answered within 3 minutes by selecting number 0 till 4 which represent the level of stress according to the type of questions. 0 represent never occurred and 4 represents very often occurred. Click on next button to go to another set of questions. After answering all type of questions, subjects can submit their answer to the system by clicking the next button. The scores and the brain dominant results will be displayed on the GUI as shown in Fig. 8. Next, user needs to click the reset button to restart the stress questionnaires. For analysis of EEG signals, the analysis was done in off-line manner. The analysis of the EEG signals is based on the dominant of the EEG signals. The analysis was focused merely on Alpha band which acts as an indicator to reflect the change in ANS due to stressors or stress factors. The increment or decrement in Alpha power was used as an indicator to indicate the existing of stress. The recorded EEG signals for stress evaluation were analysed when the subjects submit their answers. Then, the score of Psychoanalysis test, level of stress and EEG Dominant Index will be displayed in the GUI as shown in Fig. 6. The EEG Dominant Index is calculated using Asymmetry formula. If the next subjects are available, then subjects need to reset the system and then proceed to registration process. Otherwise, evaluation needs to be terminated.
METHODOLOGY
A. Subjects
10-20 Electrode connection
TIME FRAME PROTOCOL FOR EXPERIMENT
Computer
Figure 1. EEG measurement set-up.
The time frame protocol of the experiment is shown in Table 1. The overall duration of the stress evaluation is 6 minutes and 5 seconds. The experiment begins by 5 seconds of system initialization, followed by 3 minutes of EEG recording and within 3 minutes of stress questionnaires.
364
Figure 4. GUI for the system registration.
Figure 5. GUI for psychoanalysis tests of stress/calmness evaluation.
Figure 2. Process flow for stress/calmness assessment. Figure 6. Results of stress/calmness test evaluation.
C. GUI Development
The stress test scores are calculated based on psychoanalysis tests consisted of 10 questionnaires with the lowest score of zero until the highest score of 4 for each question. Thus, the total minimum score will be zero and the total maximum score will be 40. Then, the scores are divided into 3 sections to indicate the stress level as shown in Table 2. The questionnaires and scores for stress test are based on PSS.
The GUI for the system was developed using Excel Visual Basic and MATLAB. Here, one GUI for EEG analysis was designed using MATLAB as shown in Fig. 3. Meanwhile, three GUIs were developed using Excel Visual Basic as shown in Fig. 4, Fig. 5 and Fig. 6.
TABLE II.
Figure 3. MATLAB GUI for EEG signals testing & recording.
365
STRESS/C ALMNESS TESTS SCORES
Level of Stress/Calmness
Questionnaires Scores
Low
0 – 13
Moderate
14 – 26
High
27 – 40
D. EEG Dominant Analysis
Fig. 7 shows the distribution of the subjects based on the test scores for every test that they have undergone. Most of the subjects have moderate level of stress. Zero subjects have high level of stress.
The EEG signals were analysed in off-line manner. The EEG data were filtered at Alpha band only (8-13 Hz) and converted into power spectrum by using Fast Fourier Transform (FFT) with hamming window. The Energy Spectral Density (ESD) of the Alpha band was calculated by dividing the area of PSD curve with frequency range of each band. The dominant analysis was done by using (1) [14-15].
Distribution of Subjects based on Stress Test Scores 12 10
Subjects
Alpha Power (Right) – Alpha Power (Left) Dominant =
8 6 4
(1)
2
Alpha Power (Right) + Alpha Power (Left)
0 Low
Equation (1) was used to indicate the activity of the brain in term of Alpha power in the left and right side of brain hemisphere. If dominant is negative, it indicates that Alpha power in the left side of brain hemisphere is more dominant than the Alpha power in the right side of brain hemisphere. Conversely, if dominant is positive, it indicates that Alpha power in the right side of brain hemisphere is more dominant than the Alpha power in the left side of brain hemisphere. Moreover, if dominant yield zero, it indicates that the magnitude of Alpha power in the right side of brain hemisphere is same with magnitude of Alpha power in the left side of brain hemisphere. Hence, the Alpha activity in both side of brain hemisphere is balance. Thus, no brain dominant occurred. V.
The relationship between the average stress scores and stress level is shown in Fig. 8. The graph indicates that most of the subjects have moderate stress level and zero subjects have high stress level. Average Stress Scores Vs Stress Level 25
Scores
20
Low
Negative
Moderate
Left Dominant
2
Positive
Low
Right Dominant
3
Negative
Low
Left Dominant
4
Positive
Moderate
Right Dominant
High
Figure 8. Average stress scores versus stress level.
The relationship between EEG dominant and stress level is shown in Fig. 9. The graph indicates that subjects with low stress level have left dominant of EEG signals. Meanwhile, subjects with moderate and high stress level will have right dominant of EEG signals.
Brain Dominant
1
Moderate Stress Level
EXPERIMENTAL RESULTS Questionnaires
10
0
EEG Dominant Vs Stress Level 5 0 Dominant values
Asymmetry
15
5
The results of the experiment are shown in Table 3 which is based on Asymmetry, Questionnaires and Dominant. From Table 3, 50% of subjects have negative asymmetry which indicates that their brain activity is Left Dominant. Zero subjects have high stress where most of the subjects have moderate stress.
Subjects
High
Figure 7. Distribution of subjects based on stress test scores.
RESULTS AND DISCUSSION
TABLE III.
Moderate Level of Stress
-5
5
Negative
Moderate
Left Dominant
6
Positive
Moderate
Right Dominant
7
Positive
Moderate
Right Dominant
8
Negative
Moderate
Left Dominant
-20
9
Positive
Moderate
Right Dominant
-25
10
Negative
Moderate
Left Dominant
11
Negative
Moderate
Left Dominant
12
Positive
Moderate
Right Dominant
Low
Moderate
High
-10 -15
Stress Level
Figure 9. EEG average dominant versus stress level.
366
VI.
CONCLUSION
Sensor System”, Engineering in Medicine and Biology Magazine, IEEE, vol. 22, no. 3, pp. 49-55, 2003. [12] S. Sanei and J. A. Chambers, EEG Signal Processing, Wiley, 2007. [13] S. Cohen, T. Kamarck and R. Mermelstein, “A Global Measure of Perceived Stress”, Journal of Health and Social Behavior, vol. 24, pp. 385-396, 1983. [14] I.Papousek and G. Schulter, “Manipulation of frontal brain asymmetry by cognitive tasks”, Journal of Brain and Cognition, vol. 54, pp. 4351, 2004. [15] V. Knott, C. Mahoney, S. Kennedy and K. Evans, “EEG Power, frequency, asymmetry and coherence in male depression”, Journal of Psychiatry Research: Neuroimaging Section, vol. 106, pp. 123140, 2001.
The outcome of the project show that the developed GUI system able to assess and display human stress in term of stress level and brain’s EEG dominant. The system is capable to integrate the individual’s stress level with his or her brain dominant. The subjects with low stress level are likely to have Left EEG Dominant. Meanwhile, subjects with high stress level are likely to have Right EEG Dominant. ACKNOWLEDGMENT The authors would like to acknowledge the support given by the staffs, researchers and technicians at Advanced Signal Processing Research Group, Biomedical Research and Laboratory Development for Human Potential, Nondestructive Biomedical and Pharmaceutical Research Centre and Faculty of Electrical Engineering, Universiti Teknologi MARA (UiTM) Malaysia and Research Management Institute (RMI), Universiti Teknologi MARA, Malaysia. REFERENCES [1]
S. H. Seo and J. T. Lee, “Stress and EEG”, Convergence and Hybrid Information Technologies, Marius Crisan, InTech, pp. 413-426, 2010. [2] J.A. Healey and R. W. Picard, “Detecting Stress During Real-World Driving Tasks Using Physiological Sensors”, IEEE Transactions on Intelligent Transportation Systems”, HP Laboratories Cambridge, pp. 1-21, 2004 [3] E. Hoffmann, “Brain Training Against Stress: Theory, Methods and Results from an Outcome Study”, Stress Report, ver. 4.2, October, 2005. [4] G. F. Wilson and C. A. Russell, “Real-time Assessment of Mental Workload Using Psychophysiological Measures and Artifical Neural Networks”, The Journal of the Human Factors and Ergonomics Society, vol. 45, no. 4, pp. 635-643, 2003. [5] M. Senthimurugan, M. Latha and N. Malmurugan, “Classification in EEG-based Brain Computer Interfaces Using Inverse Model”, International Journal of Computer Theory and Engineering, vol. 3, no. 2, pp. 274-276, 2011. [6] C. D. Katsis, N. S. Katersidis and D. I. Fotiadis, “An Integrated System bssed on Physiological signals for the assessment of affective states in patients with anxiety disorders”, Journal of Biomedical Signal Processing and Control, pp.1-8, 2010. [7] G. F. Wilson and C. A. Russell, “Real-time Assessment of Mental Workload Using Psychophysiological Measures and Artifical Neural Networks”, The Journal of the Human Factors and Ergonomics Society, vol. 45, no. 4, pp. 635-643, 2003. [8] R. Pramila, S. Jared, B. Robert and S. Nilanjan, “Online Stress Detection using Psychophysiological Signal for Implicit HumanRobot Cooperation,” Robotica, vol. 20, no. 6, pp. 673-686, 2002. [9] P. V. D. Haak, R. V. Lon, J. V. D. Meer and L. Rothkrantz, “Stress Assessment of Car-Drivers using EEG-analysis”, International Conference on Computer Systems and Technologies, pp. 473-477, 2010. [10] A. Picot, S. Charbonnier and A. Caplier, “On-line Automatic Detection of Driver Drowsiness using a single Electroencephalographic Channel”, International Conference of IEEE Engineering in Medicine and Biology Society (EMBC), Vancouver, Canada, 2008. [11] E. Jovanov, A. O. Lords, D. Raskovic, P. Cox, R. Adhami and F. Andrasik, “Stress Monitoring Using a Distributed Wireless Intelligent
367
