Brain Computer Interface (BCI), Empirical Mode Decomposition. (EMD), Multivariate Synchronization Index (MSI), Canonical. Correlation Analysis (CCA), Ruler ...
Comparison of New Techniques Based on EMD for Control of a SSVEP-BCI Richard M. G. Tello*, Sandra Mara Torres MtiUert, Teodiano Bastos-Filho*, Andre Ferreira* *Post-Graduate Program of Electrical Engineering (PPGEE). UFES. Av. Fernando Ferrari 514. Vitoria, Brazil tDepartment of Computer and Electronics (CEVNES). Rodovia BR 101 Norte, Km. 60, Sao Mateus, Brazil
Abstract-This paper presents the comparation of three differ ent feature extraction techniques based on the Empirical Mode Decomposition (EMD) for a SSVEP-BCI. This approach based
a type de BCI named of Independent BCI, which would be the future of interfaces based on SSVEP.
on the characterization of the signal by EMD, is proposed as a novel alternative to other techniques and it was demonstrated that it exceeds both in accuracy rate and Information Transfer Rate (ITR). The experiments were performed in an offline way, and seven volunteers participated of the study. The stimulis were generated both by LCD and LEDs. The frequencies used were
8, 11, 13 and 15 Hz. The results here reported such represent
the average of the seven participants, achieving a success rate of
81 % and ITR of 23.32 bits/min of the total set of cases analyzed.
It is further confirmed that the highest success rates and ITRs were obtained for stimulation by LEDs.
Index Terms-Steady-Vtate Visual Evoked Potential (SSVEP),
Brain Computer Interface (BCI), Empirical Mode Decomposition (EMD), Multivariate Synchronization Index (MSI), Canonical Correlation Analysis (CCA), Ruler Based Classifier, Information Transfer Rate (ITR).
I.
INTRODUCTION
About 40 years ago, Regan (1966) started experimenting with long visual stimulus trains, consisting of sinusoidally modulated monochromatic light. These stimuli produced a stable Visual Evoked Potentials (VEP) of small amplitude, which could be extracted by averaging over multiple trials. These EEG waves were termed as "Steady-State" Visually Evoked Potentials (SSVEPs) of the human visual system. The amplitude and phase of the SSVEP are highly sensitive to stimulus parameters, such as repetition rate, color, contrast or modulation depth, and spatial frequency [1], [2]. Many authors agree that the SSVEP is a response to the stimulus that has a complex amplitude and phase topography across the posterior scalp, with considerable inter-subject variability [3], [4], [5], [6], [7]. Since the beginning of its conception, a BCI was defined as a mean of helping people with neuromotor complications. Nowadays, this concept is extended to applications that im prove the life of any person. The success of the great expansion and related study of SSVEP-BCI has led to the study of many techniques, seeking to enhance the SSVEP. SSVEP-based BCI system has the advantage of having higher accuracy and higher Information Transfer Rate (ITR) [S], [9]. In addition, short/no training time and few EEG channels are required [5], [10]. The SSVEP-BCI main idea is activating commands through gaze control. Nevertheless, some SSVEP-based BCI ap proaches may not depend on gaze control [11], [12], defining
978-1-4799-2399-1/14/$31.00 ©2014 IEEE
II.
MET HODS
A. Subjects and EEG preparation
Seven subjects (five males and two females), ages from 26 to 32 years old, were recruited to participate in this study. The mean and standard deviation of the ages was 27.29 and 3.59, respectively. The experiments were performed according to the rules of the ethics commitee of the UFESlBrazil, under registration number CEP-04S/0S. All measurements were noninvasive and the subjects were free to withdraw at any time without any penalty. Previously, a selection of volunteers was performed and topics related with the precaution as visual problems, headaches, family history with epilepsy and problems related to brain damage were consulted. B.
System architecture and visual stimulus
For the development of the BCI, 12 channels of EEG signal with the reference at the left ear lobe were recorded at 600 samples/s, with 1 to 100 Hz pass-band. The GND was placed on the forehead. Using the extended international 1020 system, the electrode positions were P7, P07, P05, P03, POz, P04, P06, POS, PS, 01, 02 and Oz (Fig. 1).
Fig. 1.
Electrode placement on the scalp during the experiments.
The equipment used for EEG signal recording was the BrainNet-36, manufactured by Lynx Tecnologia Ltd. The volunteers sat on a comfortable chair, in front of a 17-in LCD display, 70 cm far from it. Due to the fact of evaluating two types of stimulation (via LCD and LEDs), a coupling structure
992
of small boxes (4cm x 4cm x 4cm) containing LEDs in the four sides of the LCD was mounted. The participants were asked to watch a stimulation screen generated by an FPGA based subsystem. Such stimulation screen consists of four stripes (checkboard), presented simulta neously to the user, controlled by an FPGA Xilinx Spartan3E. On the other hand, the timing of the four LED flickers was precisely controlled by a microcontroller (PIC18F4550, Microchip Technology Inc., USA) programmed using MPLAB IDE software (MPLAB C30 compiler), with 50/50% on-off duties for all four frequencies. Four Light-Emitting Diodes (LED) (part number: NCM85414022; luminous intensity from 8 to 10 mcd) of white color, covered with thin papers diffusers, were used. For both types of stimulation, the flickering frequencies were 8.0 Hz (top), 11.0 Hz (right), l3.0 Hz (bottom) and 15.0 Hz (left). Fig. 2 shows the location of the stimuli and their respective frequency of stimulation.
Cue beep
Cross Fixation
�
D/� i.
; 1/
1//
o
5
35
Fig. 3.
45
75
85
115
125
155
t(s)
Sequence of events during EEG signal recording.
(PSD), called throughout of the text as Traditional-EMD; (ii) Canonical Correlation Analysis (CCA), called EMD-CCA; and (iii) Multivariate Synchronization Index, called EMD-MSI. The evoked potentials were recorded from the visual cortex on the occipital lobe (01, 02, Oz) from the twelve electrodes. Fig. 4 shows a volunteer, during experiment, and stimulation devices.
=8 Hz 1
=15 Hz
=11 Hz
4
2
=13 Hz 3
Fig. 2. Representation of the LCD display with the four target LED sets and four stripes (checkboard). Fig. 4.
C.
Acquisition system with the flickers stimuli.
Experimental Tasks
The experiments were performed in an offline way, follow ing the protocol shown in Fig. 3. During the first five seconds a cross fixed on the screen is shown to the volunteers. Before finishing it the five seconds, a beep is issued and the volunteer has to fix his/her attention on the stimulus located on the top side. Then the volunteer takes ten seconds for a break, and in the next thirty seconds, he/she fixes his/her attention to the right side. This protocol is repeated following in clockwise for the others stimuli, ending in 155 seconds. III.
DATA ANALY SIS
The data from the twelve channels were segmented and windowed (10, 5, 4, 2 and 1 s each one with overlapping of 50%). Subsequently, a spatial filtering was applied using a Common Average Reference (CAR) filter, and a band-pass filter between 3 - 60 Hz was also applied for the twelve electrodes. Three feature extractor were proposed. All of them use the EMD method as a first step. The second part of the hybrid system for each extractor are: (i) Power Spectral Density
The techniques mentioned above are described as follows: A. Traditional Empirical Mode Decomposition (Traditional EMD)
Huang [13] introduced the concept of Empirical Mode Decomposition and application of Hilbert transform, which was collectively called Hilbert-Huang Transform (HHT) to extract time-frequency information from a nonlinear and non stationary signal. It was regarded as an important progress since the Fast Fourier Transform (FFT). The first step of the HHT method is the signal pre processing. In this step, original data are transformed into n order Intrinsic Mode Function (IMF), satisfying the re quirements of the Hilbert Transform through the method of Empirical Mode Decomposition (EMD) [14]. The EMD approach attempts to sequentially decompose a signal into a finite number of intrinsic mode functions (IMFs) by iteratively conducting a sifting process [8]. An IMF is an analytical, self constructed, well-defined, data-driven function whose ampli tudes and frequencies vary with time [13].
993
[0 I[ EEG epoch x
+-
�
r------------------------ ------�::�!:-���- �::!.-1-------------------I
I
1
SIFTING PROCESS
C=
:--------- ---------2 IMF
./
(./. h+1 ,! h(O)� i,/ ./ . /! / ,...��:�: -..L.----i/· ��:f��� :��� � . •.. .
k=J
c
t
c=
O MFS,
d
a
Ic/ c/ ... c/IT
••
••
••
••
••
••
• ••
i
1"' ......I,,
••
: :
. ..
..
!
2
[02[ EEG epochi
··
··
··
··
· ··
·
··
.. ..
..
.. ..
-
. ..
Electrode
•
• •• ••
••
• .•
. ..
0I
..
..
..
..
..
. .. ..
..
..
..
..
..
..
..
.
(2)
where D is the number of selected data for each epoch. Subsequently, the number of IMF to be used is defined (all with the same number for the three channels). In this study, the best results were obtained for IMF of order 2 (IMF 2). Then, the Power Spectral Density (PSD) of these signals in decomposition was calculated, and finally a classifier based on rules finds the maximum values and performs a decision by majority vote for the selection of the class. Fig. 6 shows the structure of the rule-based classifier. A
............................................... ....................................................�
A, S, C, D represents the values of amplitude of signals from the feature extraction and that meet the limits defmed for its
Description:
(8 Hz), Class B (11 Hz), (13 Hz), ClassD (15 Hz) andClass Unknown.
categorization in classes, i.e., Class A ClassC
o A
Circles : Choice nooes Triangles: Tenninal nooes
Determinate the class A (mean A
Fig. 5. Flowchart of the analysis and classification using Traditional-EMD technique (adapted from [8]).
A
if > meanB) v ((meanA: meanB) 1\ (maxA: maxB) )
Example: if A=B
Example: if A=B=C
> meanB) v (meanA > meanC) v ((meanA: meanBJ
Determinate the class A if (meanA
1\ (maxA :
J
j=l
x(t) =
meanC)
1\ (max A:
maxC) )
Values not found inA, B, C orD
J
L Cj(t) + r(t)
v ((meanA:
Unknown
The IMFs are denoted by L Cj(t) and into residue r(t), such that [15]:
maxB) )
[if A=B=C, if A=B=D, if A=C=D. ifB=C=Dl
(1)
Fig. 6, Structure of the rule-based classifier implemented for the Traditional EMD technique.
j=l
where J represents the number IMFs extracted and r(t) is the residual function representing the trend within the signal x(t). The sifting process is sUlmnarized as follows: 1) Identify all extremes (minima and maxima) of x(t). 2) Using cubic spline interpolation, construct upper envelop eu(t) and lower envelop el(t), defined by the local maxima and minima, respectively. 3) The mean envelop m(t) is determinated of the upper and lower envelops, i.e., m(t) = (eu(t) + el(t))/2. 4) Subtract m(t) from the original data x(t) to acquire IMF component, such that: c(t) = x(t) - m(t). 5) If c(t) satisfies all IMF conditions then stop sifting process and calculate the residue r(t) = x(t) - c(t) otherwise repeat whole process by setting x(t) = c(t).
B.
EMD-Canonical Correlation Analysis (EMD-CCA)
Lin et al. [16] first proposed the use of CCA for multichan nel SSVEP detection, which was an array signal processing technique for EEG signals, and extracted CCA coefficients for all stimulus frequencies, assuming the frequency with the largest coefficient as the SSVEP's frequency [17], Describing it mathematically, this method assumes that X is a multichannel EEG signal, and Y consists of a "Fourier series" of simulated stimulus signals [18], assuming that there are K targets with stimulus frequencies iI, h, ... ,fb respectively. A pair of linear combinations x = XT Wx and T y = y Wy, called canonical variables, is found by using CCA between the two sets, such that the correlation is maximized. The reference signals Y is set as:
The calculation of IMFs is made for the three occipital electrodes (01, 02 and Oz). The algorithm was applied to the three EEG channels instead of one. This algorithm is used as a input of an hybrid system as shown in Fig. 5. The IMFs can be arranged in a J x D matrix, C, where each row Ck represents the kth IMF:
994
y=
sin (27rfkt) cos (27rfkt) sin (27rNhfkt) cos (27rNhfkt)
)
t=
1
2
T
Fs) Fs) ...) Fs
(3)
where !k is the stimulus frequency, Nh is the number of harmonics, T is the number of sampling points (NFFT), and Fs is the sampling rate. We used Nh = 3 harmonics in the analysis. CCA method needs to find the weight vectors, Wx and Wy, that maximize the correlation between x and y. That is, it constrains and limits conditions established by Equations 4 and 5.
Fig. 7 shows the use of CCA for frequency recognition.
EMD
IMF
IMF
IMF
Oz
2 (01)
2 (02)
2 (Oz)
C.
EMD - Multivariate Synchronization Index (EMD - MSJ)
This approach results from the combination of two types of techniques (EMD-MSI) to form a hybrid technique. MSI is a method to estimate the synchronization between the actual mixed signals and the reference signals as a potential index for recognizing the stimulus frequency. Besides, in [21] was proposed the use of a S-estimator as the index. This S estimator aims to measure the amount of synchronization over a single or two regions of the cortex. The S-estimator is based on the entropy of the normalized eigenvalues of the correlation matrix of multivariate signals. Thus, the MSI technique creates a reference signal from the stimulus frequencies used in an SSVEP-based BCI system, similarly to CCA. EEG signals are then denoted as a matrix X of size N x M and reference signals by a matriz Y of size 2Nh x M, where, N is the number of channels, M is the number of samples, and Nh is the number of harmonics for the sine and cosine components. In our case, the input signals X were the signals from the IMFs (for the three channels). Then, a correlation matrix is calculated as:
c= where,
[ C11 C21
C12 C22
]
1 XXT M C12 = xyT
C11 =
r y
-l
5in(2rr/kt) c05(2rrj,t) sin (2rrN,,/kt) cos (2rrN,J,t)
Fig. 7.
1
j
�
-J-+--+... -+---..
� , ()cO..-+-+--f IVVVVVVV' - -
(8)
� C21 = � yXT C22 = � yyT
x
.·
VVVWJVVVIfV\M/\ --_.()cO ../
(7)
(9) (10) (11)
To reduce the influence in the measure of the synchroniza tion of the autocorrelation of X and y, it was adopted the following linear transformation:
Block diagram for classification using EMD-CCA technique.
The CCA technique measures the linear association between two sets of variables using its autocorrelation and crosscorre lation [19], i.e., in mathematical terms, the total correlation is calculated as the ratio between the autocorrelation and crosscorrelation of the input and output vectors, as shown in Equation 6.
(6)
y'E[WI XXTWxJE[W;rYYTWy]
u
=
[ C11�(1/2)
o
C22-(1/2)
(12)
Then, the transformed correlation matrix is:
R= UCUT
(13)
Let AI, A2, . . . , Ap be the eigenvalues of matrix R. Then, the normalized eigenvalues are calculated as follows:
Ai � (14) AI' tr (R) 2 L:f=1 Ai where P = N +2Nh. Then, the synchronization index between
two sets of signals can be calculated as follows:
In the same way as in the previous method, we used the IMF of order 2 for the three channels (01, 02 and Oz). Finally, the process of correlation of these signals was executed, which determines the channel and the class through a criterion of maxima. This kind of approaches were discussed in [20], [15], due to its efficient methodology for elimination of noise signals (for example, eye blinks) compared to other techniques.
(15) After, we have created the reference signals in the same way as was referenced for the case of CCA (Equation 3). Next, we calculate the synchronization index between the signals from the IMFs and each reference signal Y, and then obtain k indices (SI, S2,,,,,Sk)'
995
IV.
90,00
EXPERIMENTAL RESULTS
80,00
The most common measure to assess the performance of a BCI system is the Shannon's Information Transfer Rate (ITR) [10], which is defined by Equation 16 [8], [22]. co
log2 N + and=
��s
P log2 P + (1 - P) log2
70,00
�
_
60,00
� i3
.'J.
30,00
20,00
•
���' ¥. ,
"" , A> ..�..;,#I', " ' ,,' ... :....." , " ,� ...,.:' .
