Reactive Rhythm Activities and Offline Classification of Imagined Speeds of Finger Movements Yunfa Fu1, 2, Baolei Xu1, 2, Lili Pei1, 2, Hongyi Li1 1. State Key Laboratory of Robotics Shenyang Institute of Automation, Chinese Academy of Sciences Shenyang, China 2. Graduate School of the Chinese Academy of Sciences Beijing, China
[email protected] Abstract—Reactive rhythm bands to imagined speeds of index finger movement and offline classification of imagined speeds were explored in the paper. EEG was recorded when 4 subjects executed motor imagery of two tasks at first-person perspective that involved left index fingers at two speeds (4Hz and 1Hz, trained and paced by metronome). Relatively prominent rhythmic activities around 10~13 Hz and 24~26 Hz over C3, 12~13 Hz over Cz, and 11~13 Hz, 8~10Hz, and 25~26 Hz over C4 were found by spectrum analysis. The most significant band 913Hz was used to construct the feature space. The Fisher discriminant analysis method (FDA), multi-layer perceptron neural network (MLP), and distinction sensitive learning vector quantization (DSLVQ) were applied in offline identification of imagined speeds respectively. The best average misclassification rates between fast and slow movement imagination by only three derivations C3, Cz, and C4 with FDA, MLP, and DSLVQ were 27.5±1%, 31.2±3.7%, and 34.6% at 3260 ms respectively. The results showed that FDA in this study was better than MLP and DSLVQ. This study demonstrated that discrimination for imagined speeds of finger movement was possible and feasible. Classification of imagined movement speeds with high accuracy based on EEG is also possible through improving methods in the paper. The study may provide a strategy to realize fine control of robots by brain-controlled robot interface. Keywords: Imagined movement speeds; Fisher discriminant analysis (FDA); Multi-layer perceptron (MLP); Distinction sensitive learning vector quantization (DSLVQ); Brain-controlled robot interface (BCRI)
I.
INTRODUCTION
A new human-machine fusion technology is direct brainmachine interface based on cerebral cognition. One of its potential application fields is brain-robot fusion. Braincontrolled robot interface (BCRI) is a brain-robot fusion technology which combines cognitive neuroscience and robotics. It is expected that direct control of robots by only thinking but no muscle activity involvement will be achieved by developing BCRI [1-3]. BCRI may not only help the motion-disabled persons to improve their life quality but also assist healthy persons to accomplish specific tasks [4-12]. In addition, the study for BCRI is not only a way of using brain and developing brain but also a window of understanding brain and researching artificial intelligence [1]. A type of BCRI is based on EEG evoked by motor
imagery [13-16]. The critical problems to be conquered for this kind of BCRI are identification of variation of speed or force-related parameters involved in imagined limbs movements with high accuracy and speed based on EEG measured by non-invasive method that has low spatial resolution. Recognition results are used to control the robot's motion parameters: speed and force. More studies were focused on discriminating different parts of body involved in motor imagery than identifying variation of speed or force parameters related to imagined movements [17-19]. Identification fine parameters involved in movement imagination or subtle other thinking activity details based on neuron firing spike acquired by invasive methods with high spatial resolution may have higher recognition accuracy than EEG from non-invasive methods with low spatial resolution [5]. However, some studies have proved the latter was feasible [17-19]. Ying Gu etc. [17] shown that the task parameter of speed can be discriminated in single-trial EEG traces with greater accuracy than the type of movement when tasks are executed at the same joint in healthy volunteers. Ying Gu etc. [18] also showed the possibility of identifying the speeds of an imagined movement from EEG recordings in amyotrophic lateral sclerosis (ALS) patients. Han Yuan etc. [19] demonstrated the continuous decoding of dynamic hand and speed information of the imagined clenching and their findings suggested an application to providing continuous and complex control of noninvasive brain-computer interface for movement-impaired paralytics. Robot-oriented control tasks, we attempted to discriminate the speed of the imagined tapping with left index finger in the study. The power of frequency bands which were reactive to the speed of the task was selected to construct feature space. We expect that classification of speeds will be performed by a small amount of electrodes and real-time feature extraction and classification methods. II.
METHODS
A. EEG data acquisition Four healthy subjects (three males and one female) were recruited to participate in the data acquisition (age range, 2340 years; mean ± SD, 29.5±7.1years). All subjects were right-handed and were previously unfamiliar to EEG and BMI. None of them had known sensory-motor diseases or any
The work was supported by the National Natural Science Foundation of China (Grant No. 60705021) and the research project of State Key Laboratory of Robotics of Shenyang Institute of Automation (SIA), Chinese Academy of Science (CAS) (Grant No. 08A120C101).
978-1-4244-5089-3/11/$26.00 ©2011 IEEE
history of psychological disorders. Each subject gave informed consent for the study approved by Shenyang Institute of Automation (SIA), Chinese Academy of Science.
electrodes i and the reference. From (1), CAR gives a reference independent recording.
Subjects were instructed to imagine themselves tapping on the mouse key or table at two different speeds (4Hz and 1Hz) with their left index fingers, paced by metronome. Three sessions were recorded and each session was comprised of 4 runs separated by 5 minutes breaks. Each run consists of 20 trials (10 for each motor imagery task, yielding a total of 80 trials per session and 240 trials for each subject). Before being asked to recall and feel the kinesthetic experience of movement at a first-person perspective rather than mentally watching them or another person executing the task, subjects were trained to execute actual movements to get real experience of each speed [19]. The tasks were presented randomly to subjects. During motor imagination, no visual feedback which indicated effect of task was provided to subjects who were asked to keep relaxation and avoid muscle activation, blink, slow eye movement, and facial muscle tension. Timing f a single trial is referenced in [20], [21].
C. Spectrum analysis for rhythmic activities related to imaged speeds of left index finger movement In order to identify frequency components which were reactive to imagined speeds, the spectrum of the reference interval of 2s before motor imagination on-set and of the action interval of movement imagination were calculated by FFT and compared at C3, Cz, and C4 channels based on the gBSanalyze of g.tec GUGER TECHNOLOGY [22, 23]. The difference between these two power spectra can be used to find reactive frequency bands which show a power increase or decrease related to the reference period [22]. In the study, the length of the reference and action interval to be analyzed was 2000 ms. Reference interval and action interval started at 0 ms, 2000 ms respectively. Data window type was Hanning window.
EEG signals were acquired with a 64-channel digital DC EEG amplifier (Neuro Scan Labs, synAmps 2) at sampling frequency of 500Hz and a 24-bit A/D converter. Signals were filtered with a band pass 0.05-100 Hz and a 50 Hz notch filter .The Ag-AgCl electrodes layout (extended 10-20 system) were used in data acquisition ..
D. Power calculation of reactive frequency band to imaged speeds for feature space The power of frequency band which was reactive to imagined speeds at three CAR derivations recorded over C3, Cz, and C4 was calculated by filtering the data with fourthorder Butterworth filter band pass, squaring, and averaging over consecutive samples. The window length was 252ms with overlap of 248 ms.
B. EEG Data Preprocessing After reference electrode was converted from vertex to bilateral mastoid reference (M1, M2), EEG signals were down-sampled at 250 Hz. In order to reject artifact, EEG signals that amplitude value exceeded 75% of max will be excluded from further analyses. Then, the moment of the third seconds after cued event was acted as a new event marker used as a trigger to split continuous data into trials: 2000ms before and 4000ms after the trigger. The trials were DCcorrected and de-trended [22].
E. Discrimination of imagined speeds of left index finger movement In this study, imagined left index fast movement class 1 LHF (4Hz) and slow movement class 2 LHS (1Hz) will be identified. Fisher discriminant analysis, multi-layer perception (MLP), and distinction sensitive learning vector quantization (DSLVQ) were used to discriminate imagined speeds respectively. Fisher proposed linear discriminative analysis as follows [24, 25]:
Although a lot of electrodes were used for data acquisition, it may be necessary for BCRI research to reduce the number of electrodes down for practical application. On the other hand, not only actual voluntary motor is controlled by motor cortex, some famous studies have shown that motor imagery can modify the neuronal activity in the primary sensorimotor areas in a very similar way as observable with a real executed movement [16]. Therefore, in the study, the channels C3, CZ, and C4 which overlap motor cortex area were selected to explore identification of imagined speeds of index finger movement. In the study, common average reference (CAR) derivations recorded over C3, Cz, and C4 during imagined speeds of left index finger were calculated as follows [ 22]:
VCAR (i )
1 N = V(i ) − ∑V( j ) N j =1
∑ (x
n
− m1 )( xn − m1 )T +
n∈LHF
∑(x
n
− m2 )( xn − m2 )T
(2)
n∈LHS
w ∝ S w −1 (m2 − m1 ) T
y=w x
(3) (4)
m1 and m2 were the mean value vector of class 1 LHF and class 2 LHS respectively and S w was the total within class covariance matrix and w was an adjustable weight where
(1)
where N is the total number of electrodes used for the recording in motor cortex area and around it and i and j were the channel number.
Sw =
V(i ) is the potential between the active
vector that was designed to maximize the class separability between fast and slow imagined movement. Input data x were reduced dimensionally into one dimension y with w .Decision-making rules of discrimination as follows:
y > y0 → x ∈ class 1 LHF
Cz 5
(5)
y < y0 → x ∈ class 2 LHS
0 -5
where y 0 was a selected threshold. Classification interval started at 2000 ms, ended at 6000 ms, and stepped to 252ms. Features were band power of 9-13 Hz at three LAR derivations C3, Cz, and C4. The performance of the classifier was evaluated by 10 ×10 fold cross validation to randomly mix the training and testing data. The discriminant function also can be approximated by multi-layer perceptron (MLP) with changing the connection strength between the units. In the study, MLP had one input layer with three input nodes corresponding to three feature channels, one hidden layer with four hidden nodes, and one output layer with two output nodes corresponding to two classes. In addition to the above two methods, distinction sensitive learning vector quantization (DSLVQ) was executed in the paper with codebooks per class: 2, alpha for learning speed: 0.05 and epochs for the number of iterations: 1000.
0
10
20
30
40
30
40
frequency [Hz] 100
1
μV
2
0.01
0.0001
0
10
20
frequency [Hz]
Figure 2.
Reactive rhythmic activity to imagined speeds of left index finger movement at Cz
III. RESULTS AND DISCUSSION A. Reactive rhythmic activity to imagined speeds of left index finger movement Fig. 1 to 3 displayed rhythmic activity reactive to imagined speeds of left index finger movement at C3, Cz, and C4 respectively. The calculated power spectra in the action interval and in the reference interval were indicated by the green line and the blue one respectively. The significance level at the top of the graphs was showed by the magenta line. The dotted lines represented the 99% significance level for the power differences. Relatively prominent rhythmic activities around 10~13 Hz and 24~26 Hz over C3, 12~13Hz over Cz, and 11~13 Hz ,8~10Hz ,and 25~26 Hz over C4 were found at 0 to 40 Hz range during imagined speeds of left index finger movement. C3
C4 5 0 -5
20
30
40
30
40
100
1
μV
2
0.01
0
10
20
frequency [Hz]
0 -5
10
frequency [Hz]
0.0001
5
0
0
10
20
30
40
frequency [Hz]
100
Figure 3.
Reactive rhythmic activity to imagined speeds of left index finger movement at C4
1
μV
2
0.01
0.0001
0
10
20
30
40
frequency [Hz]
Figure 1.
Reactive rhythmic activity to imagined speeds of left index finger movement at C3
B. The result of discrimination of imagined speeds involved left index finger movement. Fig.4 showed the identification result with FDA for imagined speeds of left index finger movement. The identification error time course decreased from about 52.5% at the beginning of the trial to a minimum 27.5±1% at 3260 ms. This meant that the data set could be classified with an accuracy of about 72.5 ± 1% and the time for the best classification was at 1260ms after the event.
65
60
60
55
55
50 45
50
[%]
[%]
40
45
35
40
30 25
35
2
Figure 4.
2.5
3
3.5 4 time [s]
4.5
5
5.5
Identification error of imagined speeds of left index finger movement with FDA
Fig.5 showed the identification result with MLP for imagined speeds of left index finger movement. The identification error time course decreased from about 55.5 % at the beginning of the trial. It reached 33.0±3.79% at 3008 ms and 31.2±3.7% at 3260 ms which was the least error. The error rose up after 3260 ms.
30
2
Figure 6.
2.5
3
3.5
4 time [s]
4.5
5
5.5
Identification error of imagined speeds of left index finger movement with DSLVQ
40
C. Discussion In this study, compared with MLP and DSLVQ, FDA achieved a relative better identification result for imagined speeds of left index finger movement. Identification error curves with MLP and DSLVQ fluctuated more intensively than that with FDA. But from Fig.4, Fig.5, and Fig.6, the optimal time of the least error occurred at 3260 ms. This might confirm the frequency component reactive to imagined speeds. Although the present study had achieved the best error rate 27.5±1%, the further reduction of error rate depended on the more significant frequency bands reactive to imagined speeds , and the more frequency domain and time domain features related to imagined speeds which were induced by good training strategies and found by better analysis methods [2638]. In addition, in order to reduce error rate, it was also crucial to establish an appropriate mathematical model describing the relationship between EEG and imagined speeds and to apply better identification methods.
35
IV. CONCLUSION
65 60 55 50 [%] 45
30
2
Figure 5.
2.5
3
3.5 4 time [s]
4.5
5
5.5
Identification error of imagined speeds of left index finger movement with MLP
Fig.6 showed the identification result with DSLVQ for imagined speeds of left index finger movement. The identification error course fluctuated over time. Errors in nine times were less than 50% and six times were less than 40%. It reached minimum 34.6% at 3260 ms and 4268 ms respectively.
The best error rate 27.5±1% has been achieved by only three derivations C3, Cz, and C4 for discrimination of imagined speeds involved in left index finger movement in the study. The results were mainly based on a frequency band 913Hz related to imagined speeds and FDA with fast training and differentiating effect. This also demonstrated that identification for imagined speeds of forefinger movement was possible and feasible. Our research in the future lies in finding more frequency domain and time domain features reactive to imagine speeds with online feedback trained subjects, establishing the appropriate mathematical model between EEG and imagined movement speeds, introducing advanced feature extraction and recognition methods to further improve the accuracy and speed of recognition, and constructing BCRI system for verification.
ACKNOWLEDGMENT
[17] Ying Gu, Kim Dremstrup.Dario Farina. Single-trial discrimination of type
The authors are grateful to Dr. Lun Zhao, Yuxuan Wang, Yongcheng Li for helpful discussions. Also the authors would like to thank Lijie Dang and Tongran Liu for assistance in acquiring the experiment data.
[18] Ying Gu, Dario Farina1, Ander Ramos MurguiaFLDAy , Kim Dremstrup1,
[1]
Mitsuo Kawato.Brain controlled robots. HFSP JournalVol. 2, No. 3, June 2008, 136–142.
[2]
Christian J Bell, Pradeep Shenoy, Rawichote Chalodhornand Rajesh P N Rao. Control of a humanoid robot by a noninvasive brain–computer interface in humans. J. Neural Eng. 5 (2008) 214–220.
[3]
J.d.R. Millán, F. Renkens, J. Mouriño, and W. Gerstner, “Non-invasive brainactuated control of a mobile robot by human EEG,” IEEE Trans Biomed Eng, vol.51, pp. 1026–1033, 2004.
[4]
Dennis J. McFarland and Jonathan R. Wolpaw. Brain-Computer Interface Operation of Robotic and Prosthetic Devices. Computer, October 2008,5256. www.computer.org/join/grades.htm
[5]
Hochberg L R, Serruya M D, Friehs G M, et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia [J]. Nature, 2006,442 (13 ) :164-171.
[6]
F. Gala′n,M. Nuttin, E. Lew, P.W. Ferrez, G. Vanacker, J. Philips, J. del R. Milla′n. A brain-actuated wheelchair: Asynchronous and non-invasive Brain– computer interfaces for continuous control of robots. Clinical Neurophysiology 119 (2008) 2159–2169
[8]
[9]
J. del R. Mill′an, F. Gal′an, D. Vanhooydonck, E. Lew, J. Philips and M. Nuttin. Asynchronous Non-Invasive Brain-Actuated Control of an Intelligent Wheelchair. 31st Annual International Conference of the IEEE EMBS Minneapolis, Minnesota, USA, September 2-6, 2009. 3361-3364 Eduardo I′a˜nez, M. Clara Furi′o, Jos′e M. Azor′ın, Jos′e Alejandro Huizzi, and Eduardo Fern′andez. Brain-Robot Interface for Controlling a Remote Robot. ArmJ. Mira et al. (Eds.): IWINAC 2009, Part II, LNCS 5602, pp. 353–361. Alexandre O. G. Barbosa, David R. Achanccaray, and Marco A. Meggiolaro. .Activation of a Mobile Robot through a Brain Computer Interface. 2010 IEEE International Conference on Robotics and Automation.Anchorage Convention DistrictMay 3-8, 2010, Anchorage, Alaska, USA,4815-4825.
[10] Wolpaw J R and McFarland D J 2004 Control of a two-dimensional movement signal by a noninvasive brain–computer interface in humans Proc. Natl. Acad. Sci. USA 101 17849–54
[11] G. Vanacker, J.d.R. Millán, E. Lew, P.W. Ferrez, F. Galán, J. Philips, H. Van Brussel, and M. Nuttin, “Context-based filtering for assisted brain-actuated wheelchair driving,” Comp Intell Neurosci, 2007.
[12] Hyun K. Kim , S. James Biggs,
David W. Schloerb, Jose M. Carmena,,Mikhail A. Lebedev, Miguel A. L. Nicolelis, and Mandayam A. Srinivasan. Continuous Shared Control for Stabilizing Reaching and Grasping With Brain-Machine Interfaces. IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 53, NO. 6, JUNE 2006.1164-1173.
[13] Muhammad Nabeel Anwar, Vittorio Sanguineti, Pietro Giovanni Morasso, Koji Ito. Motor imagery in robot-assistive rehabilitation: A study with healthy subjects. 2009 IEEE 11th International Conference on Rehabilitation Robotics Kyoto International Conference Center, Japan, June 23-26, 2009. 337-342
[14] Mufti Mahmud , David Hawellek, Aleksander Valjamae. A Brain-Machine Interface Based on EEG: Extracted Alpha Waves Applied to Mobile Robot. 2009 Advanced Technologies for Enhanced Quality of Life,28-31.
[15] Tao Geng and John Q. Gan Motor Prediction in Brain-Computer Interfaces for Controlling Mobile Robots 30th Annual International IEEE EMBS Conference, Vancouver, British Columbia, Canada, August 20-24, 2008,634637.
[16] GERT PFURTSCHELLER AND CHRISTA NEUPER. Motor Imagery and Direct Brain-Computer Communication. Paper,IEEE,VOL.89.NO.7,JULY 2001,1123-1134.
EEG
recordings.
Clinical
Pedro Montoya and Niels Birbaumer. Off line identifi cation of imagined speed of wrist movements in paralyzed ALS patients from single-trial EEG .Frontiers in Neuroprosthetics. August 2009 .Volume 1 ,Article 3,1-7.
[19] Han Yuan, Christopher Perdoni and Bin He Relationship between speed and
REFERENCES
[7]
and speed of wrist movements from Neurophysiology 120 (2009) 1596–1600.
Invited
EEG activity during imagined and executed hand movements J. Neural Eng. 7 (2010) 026001 1-10
[20] Yunfa Fu, Baolei Xu, Lili Pei, Hongyi Li. Event-related Perturbation in Spectral Power and in Potentials during Periodic Fast and Slow Motor Imagination for Brain-controlled Robots Interface. Proceedings of the 2010 IEEE International Conference on Robotics and Biomimetics December 1418, 2010, Tianjin, China,1299-1304.
[21] Yunfa Fu etc.Time Domain Features for Relationship between Speed and Slow Potentials Activity during Periodic Movement and Motor Imagery at Fast and Slow for BCRI. 2011 3rd International Conference on Bioinformatics and Biomedical Technology,Submitted [22] g.tec GUGER TECHNOLOGIES: gBSanalyze Toolbox [23] C. Guger, A. Schlögl, C. Neuper, D. Walterspacher, T. Strein, and G. Pfurtscheller,Rapid Prototyping of an EEG-based brain-computer interface (BCI),“ IEEE Trans. Rehab. Engng., vol 9, 2001
[24] Bishop,C.M.,Neural
Networks
for
Pattern
Recognition.
Clarendon
Press,Oxford,1995
[25] g.tec GUGER TECHNOLOGIES: g.CLASSIFYtoolbox [26] G. Pfurtscheller, "Quantification of ERD and ERS in the time domain". Handbook of Electroencephalography and Clinical Neurophysiology, vol. 6, G. Pfurtscheller and F.H. Lopes da Silva, Elsevier, 1999.
[27] Vos, J.E., "Representation in the frequency domain of nonstationary EEGs. In CEAN - Computerized EEG Analysis, Ed., G. Dolce and H. Künkel, pp. 41-50. Stuttgart: Fischer, 1975
[28] Tallon-Baudry, C.& Bertrand, O. (1999),Oscillatory gamma activity in humans and its role in object representation. Trends in Cognitive Science,3,151-162..
[29] Pfurtscheller G and Lopes da Silva F H 1999 Event-related EEG/MEG synchronization and desynchronization: basic principles Clin. Neurophysiol. 110 1842–57.
[30] Neuper C, Wörtz M, Pfurtscheller G. ERD/ERS patterns reflecting sensorimotor activation and deactivation. Prog Brain Res. 2006;159:211-22.
[31] Eleanor A. Curran and Maria J. Stokes. Learning to control brain activity: A review of the production and control of EEG components for driving brain– computer interface (BCI)systems. Brain and Cognition 51 (2003)326–336
[32] Schloegl A., Neuper C. , Pfurtscheller G. SUBJECT SPECIFIC EEG PATTERNS DURING MOTOR IMAGINARY. Proceedings - 19th International Conference – IEEE/EMBS Oct. 30 - Nov. 2, 1997 Chicago, IL. USA1530-1532
[33] Aleksandra Vuckovic, Francisco Sepulveda. Delta band contribution in cue based single trial classification of real and imaginary wrist movements. Med Biol Eng Comput (2008) 46:529–539
[34] Mehrdad Fatourechi , Gary E. Birch , Rabab K.Ward. A self-paced brain interface system that uses movement related potentials and changes in the power of brain rhythms. J Comput Neurosci (2007) 23:21–37
[35] Stancák A Jr, Pfurtscheller G
Desynchronization and recovery of beta rhythms during brisk and slow self-paced finger movements in man. Neurosci Lett. 1995 Aug 18;196(1-2):21-4
[36] Andrej Stancák Jr and Gert Pfurtscheller Event-related desynchronisation of central beta-rhythms during brisk and slow self-paced finger movements of dominant and nondominant hand Cognitive Brain Research Volume 4, Issue 3, 1 October 1996, Pages 171-183 Research report
[37] Stancák A Jr, Wackermann J Spatial EEG synchronisation over sensorimotor hand areas in brisk and slow self-paced index finger movements. Brain Topogr. 1998 Fall;11(1):23-31
[38] Stancák A Jr, Pfurtscheller G Mu-rhythm changes in brisk and slow selfpaced finger movements. . Neuroreport. 1996 Apr 26;7(6):1161-4.
