(EEG) give us an opportunity to observe brain activities related ... Recently, a new technology known as brain-computer interface. (BCI) or ..... namics Associate.
2011 4th International Conference on Biomedical Engineering and Informatics (BMEI)
A Collaborative Brain-Computer Interface Yijun Wang, Yu-Te Wang, Tzyy-Ping Jung*
Xiaorong Gao, Shangkai Gao
Swartz Center for Computational Neuroscience University of California San Diego San Diego, USA
Dept. Biomedical Engineering, School of Medicine Tsinghua University Beijing, China
Abstract—Electroencephalogram (EEG) based brain-computer interfaces (BCI) have been studied for several decades since the 1970s. Current BCI research mainly aims to provide a new communication channel to patients with motor disabilities to improve their quality of life. The BCI technology can also benefit normal healthy users; however, little progress has been made in real-world practices due to low BCI performance caused by technical limits of EEG. To overcome this bottleneck, this study uses a collaborative BCI to improve overall performance through integrating information from multiple users. A dataset involving 15 subjects participating in a Go/NoGo decision-making experiment was used to evaluate the collaborative method. Using collaborative computing techniques, the classification accuracy for predicting a Go/NoGo decision was enhanced substantially from 75.8% to 91.4%, 97.6%, and 99.1% as the number of subjects increased from 1 to 5, 10, and 15, respectively. These results suggest that a collaborative BCI can effectively fuse brain activities of a group of people to improve human behavior. Keywords-Brain-computer interface (BCI); collaborative computing; Electroencephalogram (EEG); human performance; Go/NoGo decision making
I.
INTRODUCTION
The human brain is the most complex system in the world. The functional brain imaging technologies such as functional magnetic resonance imaging (fMRI) and Electroencephalogram (EEG) give us an opportunity to observe brain activities related to thoughts, emotions, and behavior, and therefore, help us understand the relationship between the brain and behavior. Recently, a new technology known as brain-computer interface (BCI) or brain-machine interface (BMI) has made a significant progress in brain science [1]. The BCI study covers the three aspects in exploring the human brain: understanding the brain, protecting the brain, and creating the brain. During the past two decades, the BCI technology has become a hot research topic in the areas of neuroscience, neural engineering, medicine, and rehabilitation [2][3]. In essence, a BCI is a communication channel that bypasses the traditional pathway of peripheral nerves and muscles, and creates a direct link between the human brain and an output device [1]. Currently, the main focus of BCI research lies in the clinical use which aims to provide a new communication channel to patients with motor disabilities to improve their quality of life. In current BCI systems, commonly used neural recording technologies include EEG, Magnetoencephalogram (MEG), Electrocorticogram (ECoG), fMRI, near infrared spectroscopy (NIRS), and neuronal
978-1-4244-9350-0/11/$26.00 ©2011 IEEE
recording. Among these methods, EEG is the most widely used modality in current BCI studies due to its advantages such as simple and inexpensive equipment, flexibility and mobility, and short time constants. In present-day BCIs, the following EEG signals have been paid much attention: visual evoked potential (VEP), sensorimotor mu/beta rhythms, P300 evoked potential, slow cortical potential (SCP), and movement-related cortical potential (MRCP) [1]. Although the EEG-based BCI technology has achieved great successes, moving a BCI system from a laboratory demonstration to a real-life application still poses severe challenges to the BCI community. Applications of the BCI technology are very limited due to bottleneck problems including high system cost, low communication speed, low recognition accuracy, and easy user fatigue [4]. To overcome these problems, a practical solution is to develop a multi-user collaborative BCI system, which can utilize collective intelligence from a group of users. Recently, we first proposed the framework for a collaborative BCI system and further investigated the feasibility and practicality of the system [5]. The development of group-synchronized neural recording systems and group collaborative cognitive computing methods will open a totally new direction for BCI research. In this study, we propose to study the feasibility of using a collaborative BCI system to improve human decision making in a Go/NoGo decision-making task. In the Go/NoGo task, the N2 event-related potential (ERP) component, which reflects the processing of motor inhibition, will be used as a feature for identifying the NoGo condition. To evaluate the performance of the collaborative BCI, EEG-based prediction of a Go/NoGo decision will be executed using a single-trial classification paradigm and a collaborative classification paradigm respectively.
II.
METHODS
A. System diagram Figure 1 shows the system diagram of a collaborative BCI. Similar to a single-user BCI, a collaborative BCI consists of three major parts: a data acquisition module, a signal processing module, and a command translation module. Consequently, there are three major procedures in system operations: 1) Brain signals from a group of users are acquired by multiple EEG recording devices, and then are synchronized with common environmental events.
583
2) Integrated EE EG and event data d are processsed for extractiing features for decoding d users’’ intentions.
serrver requires a low-latency, hhigh bandwidthh, and reliable wirreless infrastruucture, which m might be very costly.
3) Extracted features from a group of ussers are direcctly translated to operation com mmands, which can also be ussed to give sensory feedback to the users.
2) Coomputational ccost: Advanceed signal proocessing and maachine learningg techniques hhave been widely used in currrent BCI stuudies [6] [7]. These approaaches always reqquire large am mount of com mputational ressources. In a colllaborative BC CI where a larrge amount off subjects are invvolved, the computationall cost will significantly inccrease. Becauuse real-time data proceessing in a colllaborative BC CI will leadd to a large amount of com mputation, the data server haas to be equipped with highperrformance CPU Us and large am mounts of mem mory.
mpared to a sin ngle-user BCI, the complexity y of system inp put Com from m multiple users will lead to t technical ch hallenges in bo oth dataa recording an nd signal-processing procedurres. For examp ple, new w algorithms for implemen nting collaborrative computiing havve to be develo oped to perform m the proceduree of collaboratiive EEG G analysis, wh hich plays the most importan nt role in the daata proocessing modulle.
3) Sysstem robustnesss: A collaboraative BCI systeem inevitably connsists of muultiple EEG rrecording andd processing devvices. To assurre the stability of the system,, the software shoould have the aability to keepp the whole sysstem working eveen when subssets of the whhole system faail (e.g., data connnection loss)). In other w words, the ovverall system perrformance shouuld not be serioously affected by the failure of a subsystem orr subsystems.
Figure 1. System diagram m of a collaborativee BCI.
B. System implem mentation The implemeentation of a collaborative BCI has possed sevveral specific requirements for hardwarre and softwaare dessigns: 1) Multiple EE EG recording g systems independently y and simultan neously.
need n
to
wo ork
2) Multiple-subjject data need n to be received and a synchronized d with respect to the commo on environmen ntal events. 3) Multiple-subjject data reccording and data processiing procedures haave to be perfo ormed in real tim me. Ideeally, the systeem can be im mplemented usiing a centralizzed parradigm similar to a conventio onal BCI (Figu ure 2(a)). In th his parradigm, EEG data d from multtiple subjects are received and a recoorded, then th hrown into a conventional BCI module for f signnal processing and command d translation usiing a data serv ver. A ccentralized paraadigm is optim mal for designin ng a collaboratiive BC CI system; how wever, practicaality of system m implementatiion mayy be limited for the following g reasons: 1) Data transmiission: When wired w EEG systtems are used, all systems need d to be conneccted to the datta server for daata sending/receiiving in real time. Thereforee, the data sev ver requires high h capacities of data communiication, memory, and storage. Because userrs’ natural beehaviors such as standing and walking are allways limited when w using wirred EEG systemss, portable and mobile EEG recording r devicces are more preferable in natu ural environmeents. Then, a daata
Figure 22. (a) A centralizeed paradigm and (b) a distributed paradigm for a collaboraative BCI.
To solve the problems existing iin the centralizzed paradigm, this stuudy proposes a distributed paradigm to facilitate the implem mentation of a collaborative BCI. As show wn in Figure 2(b), thhe whole systtem consists oof multiple disstributed BCI subsysttems and a sim mplified data server. For eaach subject, a BCI suubsystem workks independenttly, each subsyystem has its capabillity in EEG ddata acquisitioon and processsing. In this paradiggm, the amounnt of data transsmitted betweenn subsystems and thee data server, aas well as the computational cost for data processsing, are signifficantly reduceed. The single-user BCI has been w well studied in pprevious studiees. Therefore thhis distributed paradiggm is a moree practical sollution for impplementing a collaboorative BCI. T The only disaddvantage of thhe distributed paradiggm is that the ooverall costs off the system harrdware might
584
incrrease due to th he employmen nt of a data-pro ocessing platforrm for each user. In practice, porttable data-proccessing platform ms cann be integrated into the EEG recording dev vice to reduce the t system cost, and improve i system m practicality as well [8]. C. Go/NoGo decision-making experiment e Following a Go/NoGo paaradigm, 15 human subjects perrformed in alternation an "animal" categorization task and da singgle-photograph h recognition n task. Details of the t expperimental setu up and the images used in thee experiment can c be found in [9]. During the t experimen nt, target (G Go) phootographs were randomly mixed m with no on-target (NoG Go) imaages and flasheed for 20 ms on o a computer screen. For eaach targget, subjects had h to lift theeir finger from m the button as quiickly and accu urately as posssible. When no on-target imag ges apppeared, subjects had to withhold their button n press. For eaach subbject, 32-chann nel EEG data of o 10 blocks (100 ( trials each), werre recorded to ogether with sttimulus/respon nse related eveent coddes at a 1000 Hz H sampling rate and downsaampled to 200 Hz H for offline analysiis, totally resullting in 500 triaals per conditio on. D. Single-trial EE EG classificatiion This study peerformed a sin ngle-trial EEG classification on eacch subject using g a standard maachine-learning g paradigm. Firrst, inddependent com mponent analy ysis (ICA) waas employed to rem move eye-moveement and musscle artifacts [1 10]. Second, ER RP seggments in a prredefined timee window were extracted affter rem moving the ERP P baseline calcculated within [-100 [ ms - 0 ms]. m Thiird, the interccepted ERPs from all 32 electrodes weere conncatenated, and d then inputted d to a supportt vector machiine (SV VM)-based classifier to prediict the Go/NoG Go decision. For F eacch subject, a 10x10-fold cross c validatio on was used to estiimate the classiification perforrmance. E. Collaborativee classification Using the disstributed system m paradigm (F Figure 2(b)), the t collaborative dataa analysis wass performed with w an ensemb ble classsifier [11], wh hich consists of o multiple sub-classifiers and da votting system. In n the case of a binary b classificcation where tw wo classses are labeled d as +1 and -1 respectively, the t procedure for f aw weighted voting g can be describ bed as follows:
m
sign w(i) y (i)
(1) ( i 1 where m is the nu umber of subjeects, w(i) is thee subject speciific weiight and y(i) is the output of a sub-classsifier. An SV VM classsifier was traiined as a sub-classifier for each e subject, and a the training accurracy was used as a the voting weight. w III.
componnent also largeely differed undder two condittions over the medial frontal and paarietal areas. T The distinct spaatio-temporal pattern s of N2 and P P3 componentts under the G Go and NoGo conditioons provide the basis foor predicting a Go/NoGo decisioon using EEG [[12].
Figure 3.. Scalp ERP wavee forms and differrence waves undeer Go and NoGo conditionns at all electrode ppositions.
B. Singgle-trial classif ification Thee single-trial EEG classifiication achievved accuracy significcantly higher tthan the channce level (50% %) using data prior too mean responnse time (RT) of the Go triaals across all subjectts (377±48 mss). Figure 4 sshows the accuuracy for all subjectts when using the time winddow of [0 RT T] (mean±std: 75.8±6 .7%, range: 644.4% - 85.2%)). Consistent w with the time coursess of the N2 andd P3 componennts, the predicttion accuracy was enhhanced from 661.2±4.1% to 668.1±4.3%, 70.9±4.5%, and 74.4±6 .3% as the lenngth of time w window increassed from 200 ms to 2250 ms, 300 m ms, and 350 ms,, respectively. These results suggestted that a Go/N /NoGo decisionn can be reliabbly predicted by singgle-trial EEG cllassification.
RESULTS
A. Event-related potentials As shown in Figure 3, during 180 ms - 250 ms after an imaage onset, the N2 ERP comp ponent, which located over the t medial frontal co ortex (MFC), showed s a signiificant differen nce betw ween the Go and a NoGo cond ditions (paired d t-test, p
