On Transferring Spatial Filters in a Brain Reading Scenario Jan Hendrik Metzen?, Su Kyoung Kim?,†, Timo Duchrow†, Elsa Andrea Kirchner?,†, Frank Kirchner?,† ? † University Bremen, Robotics Group, Robert-Hooke-Str. 5, 28359 Bremen, Germany DFKI - Robotics Innovation Center, Robert-Hooke-Str. 5, 28359 Bremen, Germany
1 Introduction
3 Methods Dataflow Segmentation
Window [0ms - 1000ms] after stimulus
Detection of the P300: Perception of an important, task-relevant information is accompanied by a particular event-related potential (ERP), the “P300”. This ERP is to be detected using machine learning techniques in the single-trial.
Cz
CSP filter 3
CP3 AF7
CPz
CP4
F7 FT9
Labyrinth Oddball Scenario P3
AF4
F8
C5
C3
C1
Cz
C2
C4
C6
P4
POz
T8
P7
P5 P3 P1 Pz P2 P4 P6 PO3 O1
POz Oz
PO4 O2
P8
0.03
Pseudo-channel 4
Pseudo-channel 5
0.06
Pseudo-channel 6
Pseudo-channel 7
0.09
The Figure shows for 8 exemplary channels and 8 pseudo-channels obtained using CSP filtering the median (bold) and the 10 and 90 percentiles of the single-trials for the ’target’ (red) and ’standard’ condition (blue). The middle column shows the weights of an exemplary CSP filter.
Targets (n = 120)
0.7 0.6 0.5 0.4 0.3 0.2 0.1
Channels: m=16
Channels: m=32
Channels: m=62
InterSubject LeaveOneSessionOut NoFilter FromScratch
200 400 600 800 200 400 600 800 200 400 600 800 Size calibration dataset (t)
Best approach for a given number of retained channels and calibration dataset size
PO8 PO10
Channels: m=8
Pseudo-channel 3
TP10
PO7
2
Pseudo-channel 2
FT10
0.00
Channels: m=4
Average F1-measure on target class for the four spatial filtering settings for different sizes of the calibration dataset and different numbers of retained channels after spatial filtering.
0.03
FT8 FC2 FC4 FC6
Channels: m=2
Pseudo-channel 1
0.06
AF8
F5 F3 F1 Fz F2 F4 F6
PO9
ms
AF3
Pseudo-channel 0
TP9
-2
600
Fp2
CP5 CP3 CP1 CPz CP2 CP4 CP6
Standards (n = 720)
300
Fp1
FT7 FC5 FC3 FC1 T7
Pz
Targets
!V ERP: Standards Averaged
0
(SVM)
0.7 0.6 0.5 0.4 0.3 0.2 0.1
Retained channels (m)
Operator Monitoring: Ensure that a telemanipulation operator has recognized an important message that is displayed to him. Avoid that a message is displayed too obtrusively (distracting!). Instead, display message unobtrusively and check in operator’s brain activity whether message was perceived. If not: Display message again.
4
Classification
CSP maps the data onto axes such that the variance for instances of the first class is maximized (c) and the variance for the second class is minimized (or vice versa). With Xi ∈ Rn×t being the i-th of the nc examples of band-pass filtered and centered EEG segments with t samples for class c, this is achieved by a simultaneous diagonalization of the two empirical intra-class covariance Pnc (c) (c) T −1 matrices Σc = nc X i=1 i (Xi ) , i.e. by solving Σ1W = ΛΣ2W where Λ is the vector of generalized eigenvalues and W is the corresponding matrix of generalized eigenvectors whose columns correspond to the learned spatial filters.
Filter Weights
Brain Reading: The decoding of a user’s mental state and intent based on detecting patterns in brain activity by external observation, e.g. by means of electroencephalography.
Pz
Feature Normalization [2.5,97.5] percentile [0,1]
Common Spatial Patterns (CSP)
2 Brain Reading
-4
4Hz
1000Hz to 25Hz
Time Domain features
CSP
Lowpass-Filter
Decimation
Detrend Mean
Feature Extraction
Spatial Filtering
Performance of different spatial filtering settings F-Measure (Targets) F-Measure (Targets)
Machine learning approaches are increasingly used in brain-machine-interfaces to allow automatic adaptation to user-specific brain patterns. One of the most crucial factors for the practical success of these systems is that this adaptation can be achieved with a minimum amount of training data since training data needs to be recorded during a calibration procedure prior to the actual usage session. To this end, one promising approach is to reuse models based on data recorded in preceding sessions of the same or of other users. We investigate under which conditions it is favorable to reuse models (more specifically spatial filters) trained on data from historic sessions compared to learning new spatial filters on the current session’s calibration data. We present an empirical study in a scenario in which Brain Reading is used to support robotic telemanipulation.
5 Results
26
LeaveOneSessionOut NoFilter FromScratch
25 24 23 22 21 0
100
Averaged ERP: Targets !V -4
Pz
2 P300
4 0
300
600
ms
R The subject plays a physical simulation of the BRIO labyrinth and has to respond to rare ’target’ stimuli by pressing a buzzer. Average event-related potentials (ERPs) evoked by ’target’ and more frequent ’standard’ stimuli are depicted.
Dataset: The dataset used in this paper consists of the labeled EEG data recorded in ten sessions from five (male) subjects; each subject performed two sessions and each session consisted of five repetitions (called “runs”) of the Labyrinth Oddball paradigm. Sessions have been recorded on different days and thus, the positioning of the EEG cap can vary between sessions of the same user. EEGs were recorded continuously from 64 electrodes (extended 10-20 system with reference at electrode FCz). Two of the 64 channels (replacing the electrodes TP7 and TP8) were used to record electromyography signals of muscles of the lower arm and have been discarded in this study.
• Historic spatial filters: One CSP filter matrix (consisting of 62 CSP filters) was computed for each of the 10 sessions. Each of these matrices was trained on the merged data of all 5 runs of the respective session. • Classification task: The dataflow was trained on a subset of the data recorded during the first run of a session (the calibration data) and was tested on each of the remaining 4 runs. Each of the 10 sessions has been considered once as ’current’ session with the 9 remaining sessions acting as ’historic’ sessions. • 4 Spatial Filtering Setups have been considered: no spatial filtering (’NoFilter’), learning a CSP filter matrix on the calibration data from scratch (’FromScratch’), reusing a CSP matrix from any of the historic sessions (’LeaveOneSessionOut’), and reusing a CSP matrix from any of the historic sessions excluding those from the current subject (’InterSubject’). • Filter selection: Which historic spatial filter is selected for a session was determined—along with the classifier’s meta-parameter–using five-fold cross-validation on the calibration data. • Performance measure: Because of the considerable class skew of 6 : 1, the F1 measure (the harmonic mean of precision and recall) on the ’target’ class was used as performance measure.
300 400 500 600 Size calibration dataset (t)
700
800
The Figure shows which method achieves the highest F1-measure for a given combination of retained channels and size of calibration dataset. Filled symbols denote settings where the “winner” method is significantly better than each of the other two methods (p < 0.05). The “InterSubject” setting is omitted since it is mostly on par with the “LeaveOneSessionOut” setting.
4 Setup
-2 2
200
6 Conclusion 1
2
3
4
Reusing spatial filters trained on historic sessions achieves superior results compared to learning a session-specific filter anew (’FromScratch’) in situations when the calibration dataset is small or when the number of channels must be reduced. Reusing spatial filters for novel users (’InterSubject’), for which no historic spatial filters exist and thus filters of other users need to be reused, is only slightly worse than for known users (’LeaveOneSessionOut’). If the calibration dataset is small but many channels can be retained, using no spatial filter (’NoFilter’) is preferable. If the calibration dataset becomes larger, learning a session-specific spatial filters is usually the best choice.
Contact: Jan Hendrik Metzen University Bremen, Robotics Group Director: Prof. Dr. Frank Kirchner www.dfki.de/robotics
[email protected] Robotics Innovation Center
