A Review of Brain-Computer Interface Technology

2015 International Siberian Conference on Control and Communications (SIBCON)

A Review of Brain-Computer Interface Technology Philipp Stankevich Vladimir Spitsyn Institute of Cybernetics National Research Tomsk Polytechnic University Tomsk, Russia [email protected] Abstract—Since the invention of electroencephalography people have started to think how to develop a new communication channel based on brain’s signals analysis. It led to the development of Brain-Computer Interface (BCI). The last decades the number of Brain-Computer Interface research is significantly increased all over the world. This work aims to provide an overview of currently used technologies in the field of Brain-Computer Interface.

I. INTRUDUCTION

B. Tasks and problems There are many tasks which Brain-Computer Interface aims to solve. Firstly BCI aims to provide an alternative communication channel for people with disabilities e.g. text writing [3][4], command selection from a list [5] or control a prosthesis [6]. Secondly BCI can potentially provide general computer control for a wide range of users, but until now there is no significant success in this task, because to solve it the high quality (speed and accuracy) of BCI system is required. The next task is the control in video games. This application of BCI became widely popular last years [7].The last task but not the least is to control various technical systems like robots, drones [8] or automated home [9].

Last decades the technologies of Brain-Computer Interface (BCI) became a popular issue. Brain-Computer Interface aims to provide an alternative communication channel to people. It seems to be feasible to analyze brain‟s activity and transform it to the commands. A lot of scientists and research groups all over the world work in this field. Due to this now we have a number of approaches to implement BCI system. In this work we aim to analyze these approaches and describe the most significant of them. Mainly this work concerns non-invasive BCI approaches.

Until recently there was small number of works in the field of Brain-Computer Interface because of low spatial resolution of data acquisition devices (such EEG) and their high costs. One more reason is low performance of computers (and also high prices), and thus inability to process brain‟s activity signals in real time [10]. Last decades the computers‟ performance has been sufficiently increased (and their costs were reduced) which increase scientific interest in the area of BCI. However, today there are still a number of problems associated with processing of brain‟s signals such as:

In Section 2 firstly we are going to provide a definition of BCI system then discuss current tasks and problems in the BCI field. In Section 3 we are describing several approaches to the classification of BCI systems. In Section 3 we are concerning signals processing in the application of BCI. We describe the most popular features of the signal, their processing and classification. Finally in this section we are discussing general principals to build BCI system. In last Section 4 we draw the conclusion about current BCI technologies and we also concern some prospective directions in the field of BCI.

 Low signal-to-noise ratio. Skull prevents the propagation of brain electric waves and furthermore other electronic devices or power line itself can on the captured signal. These entire factors reduce the signalto noise-ratio.

Keywords—electroencephalography (EEG); Brain-Computer Interface (BCI); signal processing; feature extraction; feature classification

II. BRAIN-COMPUTER INTERFACE A. Definition Brain-Computer Interface term was firstly used by Jacques Vidal [1] to define any system which contains computer and is able to get the information about brain. Now we have more precise definition of BCI. In 1999 on the 1st International Meeting on Brain-Computer Interface it was defined that the BCI must be independent from peripheral nerves (which transfer the information to the brain or from it) and muscles [2].

 Artifacts. These are components of the signal which was not produced by brain. They were produced externally e.g. by muscles activity. When muscles work they produce electrical activity which also captured with brain‟s activities. The most typical artifacts are eye movement, head movement and so forth.  Difference in the brain’s activity patterns. Each person has his own unique brain‟s activity even his brain‟s activity may be different depends on his brain‟s state despite the fact that he does the same actions. This also makes difficult to analyze and classify brain‟s activity.  Low information transfer rate. All mentioned problems cause reducing in Information Transfer Rate i.e. reducing the quality of the BCI in terms of accuracy and speed.

978-1-4799-7103-9/15/$31.00 ©2015 IEEE

2015 International Siberian Conference on Control and Communications (SIBCON) III. CLASSIFICATION OF THE SYSTEMS A. Invasive and non-invasive systems We can classify all BCI systems into invasive and noninvasive systems. Invasive methods use direct connection to the gray matter (invasively). Mainly there are two invasive methods: electrocorticography (ECoG) and microelectrodes method. ECoG implies imposing electrodes on the exposed surface of the brain to record brain‟s activity. Microelectrodes method uses array of needle electrodes placed into gray matter to record electrical activity from the brain. These invasive methods allow us to get higher spatial resolution and better signal-to-noise ratio. However invasive methods have some technical and ethic problems. Due to these problems non-invasive methods are considered as the most prospective methods for BCI systems development. The most popular and most studied method is electroencephalography (EEG). EGG records brain‟s activity through a number of electrodes placed on the scalp. EEG is the first and most studied method for BCI. EEG-based BCI are portable, easy to use and low cost. However, it suffers from noise and low spatial resolution. The EEG signal represents summary activity of brain‟s neurons. Other non-invasive methods are functional magnetic resonance imagining (fMRI), magneto encephalography (MEG) and positron emission tomography (PET). FMRI is based on detecting brain‟s activity by associated blood flow which captures through nuclear magnetic resonance. MEG maps brain‟s activity by recording magnetic fields which produces by electrical activity of brain‟s neurons. PET reflects brain‟s activity by detecting pairs of gamma rays emitted indirectly by a positron-emitting tracer which was introduced into the human‟s body in form of biologically active molecule. Although these technologies can provide higher spatial resolution their usage is limited only to laboratory terms due to large size, costs and powerful magnetic fields [11]. The last and recently discovered method is near-infrared spectroscopy (NIRS). NIRS reflects brain‟s activity by detecting blood flow in cerebral cortex. NIRS has two probes: emission and detection. The emission probe emits near-infrared light into the blood vessels of brain and detection probe receives the reflection of the light from the cerebral blood. Unlike previously discussed methods it can be portable due to smaller size, but NIRS provides information of brain‟s activity indirectly (through blood cells) that is why the accuracy of this method can insufficient for NIRS-based BCI. B. Dependent and independent systems Another way to classify BCI systems is by their dependency [10]. A dependent BCI system use brain‟s normal output pathways (e.g. muscles) to generate brain‟s activity with a certain EEG pattern. One example of dependent system is BCI based on visual evoked potentials (VEPs) [6]. In case of VEPs we analyze brain‟s EEG but this EEG is generated by gaze direction (eye muscle). In such systems we do not capture the muscles activity directly we get this activity indirectly through the EEG.

Fig. 1. Design of brain-computer interface

An independent BCI system produces a certain brain‟s activity independently without using of brain‟s normal output pathways. Examples of this type of systems are P300 and motor imagery BCIs. In these systems the EEG pattern depends only on a user‟s intent not on peripheral nerves or muscles. C. Adaptive systems Wolpaw and others [10] postulate three levels of BCI adaptation. First is user specific adaptation when new user first accesses the BCI system the algorithm adapts to the user specific features. The second level of adaptation is adaptation to the user‟s brain state. The user‟s brain state may change during the usage of BCI due to such factors as fatigue, illness and etc. Thus, it can affect the accuracy of translation algorithm. That is why we need to identify these changes in brain‟s state and adapt BCI system to them. The third level is adaptation that encourages user‟s adaptation (training). It can be achieved through feedback. When BCI user sees the feedback it encourages him to produce the signals which lead to successful operations. It was noted [12] that the user can managed to control even the amplitude of alpha and beta rhythms which doesn‟t occur in everyday life. D. Hybrid systems One of the ways to improve the quality of BCI system is usage of several data acquisition approaches like combining EGG device with NIRS system [13]. In this case we can get more information about brain‟s processes thus it can positively affect the accuracy of translation algorithm. We also can analyze EEG waves together with non-brain activity like gaze direction through using eye tracking system [14]. However this approach does not refer to pure BCI system by definition, but it can help to improve the speed and accuracy of the BCI algorithm. IV. SIGNALS PROCESSING The BCI system likes others systems which deal with signal processing and classification falls into the steps which

2015 International Siberian Conference on Control and Communications (SIBCON) are preprocessing, feature extraction, feature analysis (can be optional) and feature classification. The design of BCI system can be seen on Fig. 1. In this section we mainly concern about EEG signals as the most widely used approach to data acquisition but some of the methods like feature analysis and classification can be applied for others types of signals. Let us consider each step of building a BCI system. A. Preprocessing EEG has low signal-to-noise ratio and contains various artifacts. Preliminary processing of the signal is used to improve its quality. This preprocessing is usually done by applying different filters in frequency and time domains. Frequency filtering is needed to suppress signal in a certain frequency domain. The most used filter is notch filter which suppress signal in a narrow domain. Typically it is used to filter 50 or 60 Hz frequency from power lines. Bandwidth filter is used to suppress signal from a certain low to a certain high frequency. It may be useful to remove eye‟s blinking artifacts from the source signal. Another way to improve quality of the signal is spatial filtering. This filtering derives the signal from multiple electrodes. The simplest filtering uses two electrodes. In this case we subtract signal of one electrode from another. More complex way of filtering is Laplacian filtering. In this filter we typically use five electrodes to derive the signal (Fig. 2). We also can derive the signal using all the electrodes. This filter calls Common average reference (CAR). In filters which use more than two electrodes we subtract average value of all the used electrodes from target electrode to derive the signal value.

B. Features of the signal We can extract features from EEG signal based on specific properties of it. Usually EGG signal divides into several domains by its frequency. They are alpha (8-13 Hz), beta (1330 Hz), gamma (30-70 Hz), delta (0.5-4 Hz) and theta (4-8 Hz) [16]. Each frequency domain is usually associated with certain human‟s activity e.g. alpha reflects visual activity, beta is connected with sensory-motor activity, delta is presented in the signal during sleep [16]. There are also some specific rhythms like mu-rhythm (10-15 Hz) which is between alpha and beta rhythms. Most EEG features are somehow connected with these rhythms. Let us consider the features of EEG signal. Beta and mu rhythms are associated with brain‟s motor output channels and they can be considered as features for BCI. Movements and preparation for the movements are accompanied by decreasing in these rhythms. This decrease calls event-related desynchronization (ERD). After movement the increase of these rhythms which calls event-related synchronization (ERS) is occur. ERD and ERS do not require actual movements. They can be replaced by motor imagery [10]. Feature extraction for BCI can be based on slow cortical potentials (SCP) [17]. The period of such potentials is 0.5-10 s. Negative SCPs are associated with cortical activation while positive SCPs are associated with reducing in cortical activity. We can also see increasing in SCPs while preparation for the movements [18]. Visually evoked potential is one of the widely used features. These potentials are evoked responses elicited by visual stimuli. VEPs are usually embedded in the ongoing EEG [19]. We can capture them in the area of visual cortex. There are several types of VEPs e.g. steady-state visually evoked potential (SSVEP), code book visually evoked potential (cVEP). The evoked response potential (ERP) can be elicited not only by visual stimuli but also by audio or sensory stimuli. One of the specific evoked response potential is P300 [4]. We can observe this potential as a response for target stimulus among other non-target stimuli. This potential has length of 300 ms that is why it is called P300. The many BCI systems were built based on this phenomenon. A popular P300 BCI system is the speller which use matrix of letters. In this matrix the columns and rows are flashed thus the target letter may be identified as crossing of column and row. C. Feature extraction and analysis After the features of the signal were identified we need to transform the signal to a feature space. The signal usually contains some redundant information which is not necessary (even not needed) for future classification. Some techniques can be applied to remove redundant information and transform the signal to the features for classification. Usually signal can be analyzed in frequency and time domain. Let us consider signal analysis in frequency domain.

Fig. 2. Small and large Laplacian filter. The location of the electrodes is shown above and the r2 measure is shown below [15]

Frequency analysis like Fourier analysis derives features of the signal in frequency domain. These features can be for

2015 International Siberian Conference on Control and Communications (SIBCON) example power of the signal in specific frequency bands. We can also use wavelet analysis as an alternative approach to Fourier analysis. This analysis uses signal decomposing which is not based on sinus functions. It is based on special functions of specific form are called wavelets like Haar, Daubechi and others. In [20] the wavelet analysis was used for BCI application. Auto regressive model assumes that the signal linearly depends on previous value. We can also apply this model to EEG signal for features deriving to build BCI system. It was done in [21]. Principle component analysis (PCA) [22] allows us to make features uncorrelated. This method is based on eigenvectors and eigenvalues of correlation matrix. The features are transformed and represented in order of their significance. We can leave only a number of the most significant features and thus remove the redundant information from the signal. The native EEG is the sum of different sources in the cortex. To get the source signals we can use Independent component analysis (ICA) [23]. This approach can be useful for splitting one activity from another. It also can be applied for artifacts removal like eyes blinking. The independent components of EGG signal are shown on Fig. 3. In raw EEG we can see that the source signal may be represented as the sum of independent components (for example, see the first two signals).

D. Feature classification The main idea of Brain-Computer Interface is to translate user‟s thoughts to computer commands. So after processing of the signal and feature extraction we need to classify the certain set of features to a certain command. In a simplest BCI we can use a threshold to derive the command, but this approach cannot provide the needed quality of the system to use it in real-time conditions. To build a high quality BCI we have to apply more complex techniques to make a decision about the command. That is why we consider machine learning techniques as approach to build a robust classifier. Let us consider different types of classifiers which are currently used in the area of Brain-Computer Interface. The first approach to consider is supervised learning. The K-Nearest Neighbors algorithm is the one of the simplest machine learning algorithms. It first stores the features of training examples and then it finds a set of k nearest objects in feature space to classify an object. This approach was used for example in [20] to classify motor imagery tasks. Linear Discriminant Analysis (LDA) is a well-known classification algorithm which reduces the feature space dimension with concerning about discriminatory information. The LDA projects the data into low-dimensional feature space with maximization of inner and intra class distances to maximize discrimination between classes. This is a popular method to classify features extracted from EEG signal [25][26]. Support Vector Machine (SVM) is a popular robust classifier proposed by Vapnik and Chervonenkis [27]. SVM separates features of different classes linearly by a clear gap that is as wide as possible. If classes cannot be separated linearly in this feature space SVM makes a non-leaner transformation onto high dimensional feature space where these classes can be linearly separated. For instance, in [5] SVM was employed as a classifier for BCI application. Artificial Neural Networks (ANNs) are the most popular method in machine learning. ANN is a mathematical model which is inspired by biological neural networks [22]. There are a lot of types of neural networks and they are widely used for BCI application. An interesting type of ANNs is Convolutional Neural Networks (CNN). This network models human‟s visual system and it is the best classifier for computer vision. There are several attempts to apply CNN in the BCI field e.g. [28]. A deep learning paradigm is a recently developed technology for deep layers training of ANN. Such deep networks can also be applied for BCI features classification and can potentially bring good results.

Fig. 3. Indpependent components of EGG signal. These components was calculated through Independent Component Analysis (ICA) [24]

There are many other machine learning techniques such Naïve Bayesian Classifier, Bayesian Network, Hidden Markov Models, Decision Trees and etc which also can be applied as classifiers for Brain-Computer Interface. Furthermore classifiers can be joined into ensembles to provide better classification accuracy. Such methods like Boosting and Bagging e.g. AdaBoost, RandomForest can be used to build ensembles of classifiers.

2015 International Siberian Conference on Control and Communications (SIBCON) Unsupervised learning like clustering can also be applied for feature processing of EGG signal. The simplest clustering algorithm is k-means. For instance it has been used with SVM for discrimination between two motor imagery tasks [29]. Selforganized Kohonen maps (SOM) is a two dimensional neural network which is trained according to principle „winner takes all‟. This approach allows us to activate the same neurons by similar data, thus transform the multidimensional feature space into two dimensional. SOM was used with auto regression (AR) to classify motor imagery tasks [21]. E. Online & offline analysis The many research tried to do offline analysis of EEG signal. Most of them use averaging of the signal and it allows them to achieve better result. However the BCI methods should be tested online (in real time), because single trial classification is the most challenging task that should be solved for real use of BCI systems. F. Evaluation of BCI system The evaluation of BCI system is a complex task. We should evaluate the accuracy as well as the speed of BCI in real-time conditions (online). The rate which composes both of these measures is Information Transfer Rate (ITR) introduced by Shannon [30]. This rate shows the amount of information which can go through the system per time unit. We should not only do the experimental evaluation of BCI system but also do the theoretical evaluation of ITR. The experimental evaluation of BCI must follow the next principals to provide adequate evaluation [31]:  The training and test sets should be independent. This means that we should record training and test data separately at least at different sessions otherwise the data sets will be correlated.

nerves. That is why we need to do more research for invasive BCI materials. What translation algorithm can provide the necessary speed and accuracy to build a high quality BCI? The deep learning is an emerging trend in machine learning. The deep networks win competitions on pattern recognition for last several years [34]. It seems to be feasible to employ such networks in the field of Brain-Computer Interface. Thereby we should continue working on Brain-Computer Interface while the aforementioned question is still unanswered. While doing research we should think about getting high quality results taking into account the principles of BCI system evaluation. The gained result should be disseminated through preparing high quality papers to inform the scientific society. ACKNOWLEDGMENT We thank management staff of Cybernetics Institute and Computer Engineering Department of National Research Tomsk Polytechnic University personally Dr. Alena A. Zakharova and Prof. Nikolay G. Markov for the provided facilities and supporting of this work. REFERENCES [1] [2]

[3]

[4]

 The test data should not be used for selection of method (translation algorithm) parameters.

[5]

There a lot of works were published about BCI topic but unfortunately only small number of them follows above principles.

[6]

[7]

V. CONCLUSION AND FUTURE RESEARCH The development of the high quality BCI system is a tough issue. Unlit now there are no BCI systems which can be used in everyday life. Thus we have a number of questions to answer: Does EGG contain enough information to build high quality BCI or we have to use invasive methods? Some recent studies say „yes‟ for example Nathan Intrator shows that multiple features can be extracted from only two forehead dry electrodes and these features can well describe the different brain‟s states [32]. What materials should we use to capture brain signals invasively? There was a partial success in connecting directly to brain‟s nerves [33] but after some time the connection of the electrodes fell down due to their abruption by brain‟s

[8]

[9]

[10] [11]

[12]

[13]

J.J. Vidal, “Toward direct brain-computer communication,” Annual review of biophysics and bioengineering Vol. 2, pp. 157-180, 1973 JR Wolpaw et al., “Brain-computer interface technology: a review of the first international meeting”, IEEE Trans. on rehabilitation eng Vol. 8 No.2, pp. 164-173, 2000 A. Lenhardt, M. Kaper, and H.J. Ritter, “An adaptive P300-based online brain-computer interface,” IEEE Trans. on Neural Systems and Rehabilitation Eng, vol.16, No 2, pp.121-130, 2008 A. Rakotomamonjy, V. Guigue, “BCI Competition III: Dataset IIEnsemble of SVMs for BCI P300 Speller,” IEEE Trans. on Biomedical Eng., vol.55, issue 3, pp.1147-1154, 2008 H. Riechmann, A. Finke, and H. Ritter, “Hierarchical Codebook Visually Evoked Potentials for fast and flexible BCIs,” 35th Annual International Conference of the IEEE EMBS, pp. 2776-2779, 2013 G.R. Müller-Putz, G. Pfurtscheller, “Control of an Electrical Prosthesis With an SSVEP-Based BCI,” IEEE Trans. on Biomedical Eng., vol.55, no.1, 2008 D. Marshall, D. Coyle, S. Wilson, and M. Callaghan, “Games, gameplay, and BCI: the state of the art,” IEEE Trans. on computational intellegence and AI in games. Vol. 5 No.2, pp. 82-99, 2013 K. LaFleur et al. “Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface,” Journal of Neural Engineering, 2013 C.T. Lin, B.S. Lin, F.C. Lin, and C.J. Chang, “Brain computer interfacebased smart living environmental auto-ajustment control system in UPnP home networking,” IEEE Systems Journal Vol.8 No.2, pp.363370, 2014 J.R. Wolpaw “Brain-compurer interfaces for communication and control,” Clinical Neurophysiology, pp.767-791, 2002 B. Zhang, J. Wang, and T. Fuhlbrigge, “A review of the commercial brain-computer interface technology from perspective of industrial robotics,” Proc. of the 2010 IEEE Int. Conf. on automation and Logictics, 2010 E. Nedermeyer, F. Lopes da Silva, Electroencephalograhy: basic principles, clinical applications and related fields, 5 th ed, Lippincott Williams & Wilkins, pp. 168-192, 2005 Y. Tomiya at el., “Bimodal BCI Using Simultaneously NIRS and EEG,” IEEE Trans. on Biomedical Eng., vol.61, no.4, 2014

2015 International Siberian Conference on Control and Communications (SIBCON) [14] P. Cipresso “The combined use of Brain Computer Interface and EyeTracking technology for cognitive assessment in Amyotrophic Lateral Sclerosis,” 2011 5th Int. Conf. on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops, pp.320-324, 2011 [15] D.J. McFarland, LM McCane, SV David and JR. Wolpaw “Spatial filter selection for EEG-based communication,” Electroencephalography and clinical Neurophysiology, pp.386-394, 1997 [16] V.N. Kiroy Brain-computer interface, Publishing house of Southern Federal Uviversity, p. 240, 2011 [17] G. Dornhege, B. Blankertz1, G. Curio, “Speeding up classification of multi-channel brain-computer interfaces: common spatial patterns for slow cortical potential pdf,” Proc. of the 1st Int. IEEE EMBS Conf. on Neural Engineering, 2003 [18] H. Shibasaki, M. Hallett, “What is the Bereitschaftspotential?,” Clinical Neurophysiology, vol. 117, pp.2341-2356, 2006 [19] Q. He, B. Wu, H. Wang, and L. Zhu, “VEP Feature Extraction and Classification for Brain-Computer Interface,” Proc. of the Int. Conf. on Sign. Procesing, 2006 [20] R. Aldea, M. Fira, and A. Lazar, “Classification of motor imagery tasks uning k-nearest neighbors,” 2014 12th Symposium on Neural Networks Application in Electrical Engineeriong, 2014 [21] T. Yamagutchi at el. “Pattern recognition of EEG signal during motor imagery by using SOM,” the 2nd Int. Conf. on Innovative Computing, Information and Control, p. 121, 2007 [22] S.O. Haykin, Neural networks: a compehensive foundation, 2d ed, Person Education, 1999 [23] A. Hyvarinen, E. Oja, “Independent Component Analysis: Algorithms and Application,” Neural Networks, vol. 13, Issue 4-5, pp.411-430, 2000 [24] J.D. Kropotov, Quantitative EEG, Event-Ralated Potentials and Neurotherapy, Elsevier Press, 2009

[25] C.Y. Chan, C.W. Wu, and C.T. Lin, “A novel classification method for motor imagery based on Brain-Computer Interface,” 2014 Int. Conf. on Neural Networks, pp. 4099-4102, pp.1046-1051, 2014 [26] S. Bhattacharyya at al., “Performance analysis of LDA, QDA and KNN algorithms in left-right limb movement classification from EEG data,” Proc. of 2010 Int. Conf. on Systems in Medcine and Biology, pp. 126131, 2010 [27] C. Cortes, V. Vapnik, “Support vector networks,” Machine Learning, Vol. 20, Issue 3, pp.273-297, 1995 [28] H. Cecotti, A. Graser, “Convolutional Neural Networks for P300 Detection with Application to Brain-Computer Interfaces,” IEEE Tran. On Pattern Analysis and Machine Intellegence, vol. 3, No.2, pp.433-455, 2011 [29] A. Banitalebi, S.K. Setarehdan, G.A. Hossein-Zadeh, “A Technique Based on Chaos for Brain Computer Interfacing,” Proc. of the 14th Int. CSI Computer Conference, pp.464-469, 2009 [30] C.E. Shannon and W Weaver, Mathematical Theory of Communication. Champaign, IL: Univ. Illinois Press, 1964 [31] K.R. Müller at el. “Machine learning and signal processing tools for Brain computer interfacing,” Presentation, TU Berlin, 2011 [32] G. Castellani, N Intrator, and D Remondini “Systems biology and brain activity in neuronal pathways by smart device and advanced signal processing,” Fronriers in Genetics, vol. 5, 2014 [33] W. H. Dobelle, “Artificial vision for the blind by connecting a television camera to the visual cortex,” ASAIO Journal, vol.46. issue 1, pp.3-9, 2000 [34] O. Russakovsky, J. Deng et al. “ImageNet Large Scale Visual Recognition Challenge,” arXiv:1409.0575, 2014