Evaluation of an SSVEP based Brain-Computer ... - IEEE Xplore

FrE1.4

Proceedings of the 4th International IEEE EMBS Conference on Neural Engineering Antalya, Turkey, April 29 - May 2, 2009

Evaluation of an SSVEP based Brain-Computer Interface on the command and application levels Hubert Cecotti, Ivan Volosyak and Axel Gr¨aser Institute of Automation, University of Bremen, Otto-Hahn-Allee 1, 28359 Bremen, Germany Email: {cecotti, volosyak, ag}@iat.uni-bremen.de, Telephone: +49-421-218-3344, Fax: +49-421-218-4596

Abstract—A Brain-Computer Interface (BCI) provides the possibility to translate brain neural activity patterns into control commands for computers without user’s movement. The brain activity is most commonly measured non-invasively via standard electroencephalographic (EEG) electrodes placed on the surface of the scalp. We propose the evaluation of the Bremen-BCI system based on Steady-State Visual Evoked potentials (SSVEPs), which was evaluated with 37 BCI-naive subjects, including eight handicapped persons, on the international rehabilitation fair RehaCare2008. In spite of the noisy environment during the fair, the spelling tasks were successfully completed. We propose two evaluation methods, one based on the main task to achieve and the second, on the commands that are needed to achieve the task. In the command level, the mean accuracy of the command detection is 92.84%, with an average information transfer rate of 22.6bpm (bits per minute). In the speller level, the average information transfer rate is 17.4bpm (equivalent to 3.5 letters per minute with 30 possible letters). These results highlight the differences between two evaluation methods. Differences can emerge between the raw BCI and its connection to an application.

I. I NTRODUCTION The main goal of Brain-Computer Interface (BCI) research is to provide communication to persons with motor disabilities [1], [2]. These severe disabilities like spinal cord injuries prevent these persons from using other communication means. The initial challenge was showing that BCIs can work. One current challenge to address is the use of a BCI in a real environment, outside of the laboratory, beyond simple demonstration purposes. For a typical user the goal is not to send bits to a computer. The real goal is to communicate with the real world, with devices and other persons. Although the notion of Brain-Computer Interface limits the communication scope to the computer stage, the true BCI meaning is indeed BrainReal world Interface. BCIs must also take into account the ergonomic aspect of the application. For instance, Blankertz et al. proposed a new speller called hex-o-spell that uses only two mental states [3]. In such case, the information transfer rate between the brain and the real world (the speller) depends on the graphical user interface, on how the two basic commands are combined. A recent study on BCI-naive subjects have been described in [4] with a BCI system based on sensorimotor rhythms. In this paper, we propose to evaluate the Bremen-BCI, which is based on the detection of Steady-States Visual Evoked Potentials (SSVEP). The proposed evaluation aims at comparing both the information transfer rate between the user and the

978-1-4244-2073-5/09/$25.00 ©2009 IEEE

computer, and between the user and the speller. For an SSVEPBCI, the system reflects the user attention to an oscillating visual stimulus. The usually used stimuli are flickering lights at different frequencies. Their responses in the visual cortex correspond to SSVEP at the same frequencies and higher harmonics [5]. Besides, it has been shown that even the frequency of vertical refresh rate of a cathode ray tube monitor (60Hz) can evoke SSVEP responses [6]. The amplitude and the phase that define an SSVEP response depend on the frequency, intensity and the structure of the repetitive visual pattern [7]. For instance, SSVEP based BCIs have been used for neuroprosthetic devices control, for the restoration of the grasp function in spinal cord injured persons [8] and video games [9]. They are described as reliable in the literature [10]– [12]. Contrary to the P300 paradigm, where each cell of the matrix can correspond directly to a command, the low number of commands of an SSVEP-BCI involves a particular strategy for creating the graphical user interface. The paper is organized as follows: the system is presented in the second section; the protocol experiment is defined in the third section. Finally, the results and their analysis are detailed in the last section. II. S YSTEM OVERVIEW A. Materials and hardware The EEG signal was recorded from the surface of the scalp via 8 standard EEG electrodes. PZ , P O3 , P O4 , OZ , O9 , O10 as the input electrodes, AFZ and CZ are used for the ground and the reference respectively [13]. The standard abrasive electrolytic electrode gel was applied between the electrodes and the skin to bring impedances below 15kΩ. An EEG amplifier g.USBamp (Guger Technologies, Graz, Austria) has been used for the experiments. The sampling frequency was 128Hz. During the EEG acquisition, an analog bandpass filter between 2 and 30Hz, and a notch filter around 50Hz (mains frequency in Europe) were applied directly in the amplifier. The number of frequencies is chosen in relation to a classical two dimensional control: four classes are dedicated to the directions (up, down, left and right) and one class for an action. The frequencies are determined in relation to the refresh rate of the LCD screen that produces the stimuli: 60Hz. As the quality of the SSVEP response depends on the frequency stability, the frequencies that are used in this experiment are: 7.50Hz (“left”), 8.57Hz (“right”), 10.00Hz (“up”), 12.00Hz (“down”) and 6.66Hz (“select”). The chosen

474

frequencies correspond respectively to periods equivalent to 9, 8, 7, 6 and 5 frames on the LCD screen. A fixed number of frames within one period of the visual signal assures the frequency stability. B. Signal processing The detection of the SSVEP responses is based on the method presented in [12]. It uses the minimum energy combination (MEC) for creating different channels. It is worth mentioning that the system does not require any training. A channel represents a weighted combination of the electrode signals. Once the channel are created, the frequency power of each of the five possible frequencies and their first harmonics is computed. The estimated power of the SSVEP response corresponding to the stimulus i is estimated by: P ˆ(i)

=

j=N s k=N h 1 P (j, f(i) ∗ k) Ns Nh j=1

(1)

k=1

where Ns is the number of channels created by the MEC, Nh is the number of harmonics taken into account (Nh = 2), f(i) is the frequency of the flickering light of the stimulus i. P (j, f ) is the power of the frequency f in the channel j. The command C is produced if: C

=

argmax P ˆ(i) and P ˆ(i) > α i

(2)

If the frequency power is superior to a threshold α then the command C corresponding to the frequency f(i) is produced. The time segment length used for the signal analysis is 2s, with an idle time of 2s between two consecutive commands. C. Speller The graphical user interface (GUI) of the Bremen-BCI speller is represented by a virtual keyboard containing 32 symbols (30 characters and two special commands: delete and clear) [14]. It is located in the middle of the screen. The visual stimuli are parts of the GUI on the same LCD screen. This solution is more convenient for the subjects as they don’t need to shift completely their gaze between the stimuli and the application. The five white boxes at outer sides of the screen are flickering with different frequencies as described in section II-A. They correspond to the commands “left”, “right”, “up”, “down”, and “select” at the upper left corner of the screen. At the beginning of each spelling trial, a cursor is located in the middle of the virtual keyboard over the ‘E’ character. By focusing on one of the five oscillating boxes, the subject moves the actual cursor selection to the desired letter and selects the letter by activating the command “select”. After every selection the cursor moves back to the center of the layout, i.e. to the initial letter ‘E’. In case of spelling errors the last letter or the whole spelled text can be erased by selecting the special commands “Del” or “Clr”. The current status of the spelling, i.e the text that the subject has already spelled, is shown in the line at the bottom of the screen. The special design of letters

arrangement takes into account the probabilities of occurrence of each letter in English. III. P ROTOCOL EXPERIMENT The experiments were carried out at the booth of the Institute of Automation from the University of Bremen, on the 19th international rehabilitation fair RehaCare 2008 in D¨usseldorf, Germany. The condition of the experiments were not optimal for achieving the best results. The environment was noisy and the surrounding lights may have had an effect on the visual stimuli quality. Besides, the experiments were performed on 37 volunteer BCI-naive subjects recruited from visitors at the booth. The subjects were composed of 22 females, 15 males, with a mean age of 35.03 years and a standard deviation of 11.86. Among these subjects, eight persons were handicapped. A short run was first carried out in order to introduce and explain the experimental procedure to the subjects. During this run, the threshold was manually adjusted for every subject. The task was to spell five messages with the Bremen-BCI speller. Three words were the same for every subject (copy spelling) and two words were chosen by the subject (free spelling). The copy spelling words were BCI, BRAIN, and GEHIRN. The first free spelling word is chosen in order to be composed of 5 letters. Each subject verbally told to the experimenters the message that s/he intended to spell before each free spelling run to avoid a change in the goal during the trial. The task to achieve was well defined: in any case of misspelling the participants were forced to correct all kinds of errors by using special characters: “Del”, to suppress the last written character and “Clr”, to erase all characters. IV. E VALUATION AND RESULTS Contrary to other BCIs like those based on P300, the commands given from an SSVEP-BCI are often combined for creating commands of higher level in relation to the application [15]. A character is usually written by using a sequence of several commands with an SSVEP-BCI, contrary to spellers based on P300-BCI, where each cell represents a character. Although it could be possible to assign an SSVEP response for each character, it remains a challenge [11]. For such strategy and in our speller, the display should present 32 flickering boxes with 32 different oscillating signals, with different frequencies, phases, etc. As the spelling task depends on the speller interface, it is possible to evaluate the speller with two kinds of methods. The first method is the evaluation of only the global task that the subject has to complete. In the speller case, the subject has to write a word. This evaluation can be achieved by comparing the expected word and the output word. This kind of evaluation translates the efficiency of the overall program and may not translate the efficiency of the raw BCI. For writing words, the speed will largely depends on the layout. Other features like the language model could be added to speed up the spelling process. However, such improvements are strictly dependent on the application and not the BCI part. Such evaluation remains local to the speller and the

475

outcome cannot be extended to other applications using the same BCI part. Furthermore, for a correct evaluation of the speller, subjects have to write many words to represent a real use of the program. For instance, if the words to spell contain only letters close to the center of the speller layout, the average time for writing a word will be lower. A large number of written words is also needed for providing statistically relevant data. The second method is the commonly used evaluation of the basic BCI commands that the subject has to perform for achieving the desired task. This measurement evaluates mainly the efficiency of the SSVEP response detection. This BCI measurement does not depend on the application. The efficiency of one particular command for moving the cursor can be extended to the efficiency of the same command for other problems like for moving other devices. Furthermore, the number of commands for achieving one letter increases the pertinence of the results. For instance, a minimum of nine commands is needed for writing the word “BCI”. For the same task, the accuracy of nine commands is statistically more representative of the BCI than the spelling accuracy of three letters. However, the drawback of this measurement is that several sequences of commands can lead to the same goal. Although only the main task was dictated to the subject, we propose to evaluate the accuracy of the BCI commands based on one main hypothesis. This hypothesis assumes that the commands given by the user are optimal. It means that the user does not play around with the cursor, each command is a step toward the next character to spell. We note C = (xc , yc ) and T = (xt , yt ) the actual position of the cursor and the target in the layout. The cursor represents the current letter that can be selected on the screen. The target represents the position where the subject has to move the cursor and select the character. In other word, the target represents the next letter to write for completing the word. Several paths are possible to reach the target. The path chosen by the subject is free. Nevertheless, the subject is always supposed to reach the target by using the shortest path. Several cases exist where several commands correspond to an optimal decision. For instance, from the letter ‘E’ to the letter ‘B’, it is possible to go right then down, or down then right. In such situation, if the first command is down or right then the command is considered as correct. We acknowledge that the user may want to produce the command down and a misclassification producing the command “right” is classified as correct. This scenario occurs only when the cursor and the target are not aligned vertically or horizontally. In case of error from the user or from the SSVEP response detection, the procedure continues as the target is the same. In addition, we add the following constraint that deals with the respect to the layout in relation to the commands. The layout restricts the type of command to use. For instance, when the cursor is on ‘Z’, it is impossible to go down or right. Thus, if a “down or “left command is produced, it is obviously an error. For such situation, we define the function M ove(com) that defines if the command is possible or not, independently

of the character to spell; com is one of the four commands for moving the cursor. 

0 1

M ove(com) =

com leads outside of the layout otherwise

(3)

We define Δx = xt − xc and Δy = yt − yc . The following rules are applied for determining the accuracy (Acc) of each command:  Acc(lef t) =  Acc(right) =  Acc(up) =  Acc(down) =

1 0

if Δx < 0 and M ove(lef t) otherwise

(4)

1 0

if Δx > 0 and M ove(right) otherwise

(5)

1 if Δy < 0 and M ove(up) 0 otherwise

(6)

1 if Δy > 0 and M ove(down) 0 otherwise

(7)

 Acc(select) =

1 if Δx = 0 and Δy = 0 0 otherwise

(8)

For one task, the accuracy is given in % : Acc =

100 #com

i=#com 

Acc(com(i))

(9)

i=1

where #com is the number of commands performed during the task, com(i) is the ith command, com(i) being one of the five possible commands. For the free spelling tasks, the accuracy was computed once the task was finished as the word to spell was unknown. The information transfer rate (IT RBCI ) in bits per minute (bpm) is defined by [16]:   p log2 (p) + (1 − p) log2 ( N1−p −1 ) + log2 (N ) IT RBCI = (10) T where N is the number of possible commands, T (in minutes) is the time needed for achieving the complete task, p is the probability to detect a command correctly. IT RBCI represents the information transfer rate between the subject and the basic commands of the application. In this case, N = 5, it corresponds to the five flickering boxes, and p = Acc/100. The information transfer rate (IT RApp ) in bpm is defined by: IT RApp = T ∗ #letters ∗

log(N ) log(2)

(11)

where #letters is the number of written letters. When the evaluation concerns the main task, i.e. the word to write, N = 30, it corresponds to the number of possible characters in the application; the command “Del” and “Clr”

476

are not taken into account as real characters. These commands are just used to correct the errors from the user or the command detection. As each task should be completed due to the protocol, p = 1 when the task is achieved, p = 0 if not. In term of letters per minute (lpm), IT RApp can be reformulated as follows: IT RApp (lpm)

=

IT RApp (bpm) log(N )/ log(2)

(12)

TABLE I R ESULTS OVER 32 SUBJECTS . BCI level Accuracy (%) ITR (bpm) Mean S.D Max Min

92.84 6.80 100.00 75.00

22.60 9.67 40.68 3.64

ACKNOWLEDGMENT This research was supported by a Marie Curie European Transfer of Knowledge project BrainRobot, MTKD-CT-2004014211, within the 6th European Community Framework Program and by a Marie Curie European Re-Integration Grant RehaBCI, PERG02-GA-2007-224753, within the 7th European Community Framework Program. The authors would like to thank Thorsten L¨uth, Aavo Moltsaar, Karsten Stenzel and Diana Valbuena for their help during the experiments. R EFERENCES

Application level ITR (bpm) ITR (lpm) 17.40 7.94 31.44 1.89

with improving the graphical user interface while keeping a convenient access to BCI-naive and new users.

3.46 1.69 6.41 0.39

Among the 37 persons, five subjects were not able to perform any task. It corresponds to a BCI literacy of 86.5% over 37 subjects during their first BCI session. Table I presents the mean, the standard deviation (S.D.), the maximum and the minimum ITR at the BCI level and at the application level for the 32 remaining subjects who were able to use the speller. The mean accuracy over subject for the command detection is 92.84%. The maximum ITR for both evaluations is achieved by the same subject. The number of letters per minute presented in the table corresponds to a perfect accuracy. As we can observe, there is an information loss between the BCI and the speller. This difference can be explained by the choice of the words to write. One mistake in the word involves a correction. It means that for a false classification of the select button, the user must produce five commands to delete the erroneous character. The cost of a “select” error has a high cost in the GUI. Subjects may also have some problems to navigate in the layout and pay attention to both the visual stimuli and the current location of the cursor. V. C ONCLUSION A Brain-Computer Interface based on Steady-State Visual Evoked Potentials has been tested in real conditions during an international fair. The system has been proven successful and robust. The presented study has proved that it is possible to operate an SSVEP-BCI with a high accuracy and a high literacy during the first session. The performance are promising and are one step toward the incorporation of SSVEP based BCI into industrial products. Several ways have been proposed for analyzing the results. The differences observed between the two evaluations highlighted the strong link between the BCI and its application. While a fast system impairs with a fast detection of the commands, the personal BCI experience is strongly related to the output of the application: a slow command that achieves a big step remains usually more impressive than a fast command that produces a little step toward the final goal of the user. Further works will deal

[1] B. Z. Allison, E. W. Wolpaw, and J. R. Wolpaw, “Brain-computer interface systems: progress and prospects,” Expert Review of Medical Devices, vol. 4, no. 4, pp. 463–474, 2007. [2] N. Birbaumer and L. G. Cohen, “Brain-computer interfaces: communication and restoration of movement in paralysis,” Journal of PhysiologyLondon, vol. 579, no. 3, pp. 621–636, 2007. [3] B. Blankertz, G. Dornhege, M. Krauledat, M. Schr¨oder, J. Williamson, R. Murray-Smith, and K.-R. M¨uller, “The berlin brain-computer interface presents the novel mental typewriter hex-o-spell,” In Proc. of the 3rd International Brain-Computer Interface Workshop and Training Course 2006, pp. 108–109, 2006. [4] B. Blankertz, F. Losch, M. Krauledat, G. Dornhege, G. Curio, and K.R. M¨uller, “The berlin brain-computer interface: Accurate performance from first-session in bci-naive subjects,” IEEE Trans. Biomed. Eng., vol. 55, no. 10, pp. 2452–2462, 2008. [5] G. R. M¨uller-Putz, R. Scherer, C. Brauneis, and G. Pfurtscheller, “Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components,” Journal of Neural Engineering, vol. 2, no. 1, pp. 123–130, 2008. [6] E. Lyskov, V. Ponomarev, M. Sandstrom, K. Mild, and S. Medvedev, “Steady-state visual evoked potential to computer monitor flicker,” Int. J. Psychophysiol, vol. 28, pp. 285–290, 1998. [7] Z. Wu, Y. Lai, D. Wu, and D. Yao, “Stimulator selection in SSVEPbased BCI,” Int. Jour. Medical Engineering and Physics, 2008. [8] G. R. M¨uller-Putz and G. Pfurtscheller, “Control of an electrical prosthesis with an SSVEP-based BCI,” IEEE Trans Biomed Eng., vol. 55, pp. 361–362, 2008. [9] E. C. Lalor, S. P. Kelly, C. Finucane, R. Burke, R. Smith, R. B. Reilly, and G. McDarby, “Steady-state VEP-based brain-computer interface control in an immersive 3d gaming environment,” EURASIP Journal on Applied Signal Processing, vol. 19, pp. 3156–3164, 2006. [10] H. Cecotti and A. Gr¨aser, “Convolutional neural network with embedded Fourier transform for EEG classification,” In Proc. of the 19th International Conference on Pattern Recognition, 2008. [11] X. R. Gao, D. F. Xu, M. Cheng, and S. K. Gao, “A BCI based environmental controller for the motion-disabled,” IEEE Trans. Rehab. Eng., vol. 11, no. 2, p. 137140, 2003. [12] O. Friman, I. Volosyak, and A. Gr¨aser, “Multiple channel detection of steady-state visual evoked potentials for brain-computer interfaces,” IEEE Trans. Biomed. Eng., vol. 54, no. 4, pp. 742–750, 2007. [13] G. E. Chatrian, E. Lettich, and P. L. Nelson, “Ten percent electrode system for topographic studies of spontaneous and evoked eeg activity,” Am J EEG Technol, vol. 25, pp. 83–92, 1985. [14] D. Valbuena, I. Sugiarto, and A. Gr¨aser, “Spelling with the bremen braincomputer interface and the integrated SSVEP stimulator,” Proceedings of the 4th International Brain-Computer Interface Workshop and Training Course, pp. 291–296, 2008. [15] T. L¨uth, D. Ojdanic, O. Friman, O. Prenzel, and A. Gr¨aser, “Low level control in a semi-autonomous rehabilitation robotic system via a braincomputer interface,” ICORR 2007. IEEE 10th International Conference on Rehabilitation Robotics, pp. 721–728, 2007. [16] J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain-computer interfaces for communication and control,” Clin Neurophysiol, vol. 113, pp. 767–791, 2002.

477