IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 21, NO. 6, NOVEMBER 2013
897
The Component Structure of Event-Related Potentials in the P300 Speller Paradigm Siri-Maria Kamp, Anthony R. Murphy, and Emanuel Donchin
Abstract—We investigated the componential structure of eventrelated potentials elicited while participants use the P300 BCI. Six healthy participants “typed” all characters in a 6 6 matrix twice in a random sequence. A principal component analysis indicated that in addition to the P300, target flashes elicited an earlier frontal positivity, possibly a Novelty P3. The amplitudes of both P300 and the Novelty P3 varied with the matrix row in which the target character was located. However, the P300 elicited by row flashes was largest for targets in the lower part of the matrix, whereas the Novelty P3 elicited by column flashes was largest in the top part. Classification accuracy using stepwise linear discriminant analysis mirrored the pattern in the Novelty P3 (an accuracy difference of 0.1 between rows 1 and 6). When separate classifiers were generated to rely solely on the P300 or solely on the Novelty P3, the latter function led to higher accuracy (a mean accuracy difference of about 0.2 between classifiers). A possible explanation is that some nontarget flashes elicit a P300, leading to lower selection accuracy of the respective classifier. In an additional set of data from six different participants we replicated the ERP structure of the initial analyses and characterized the spatial distributions more closely by using a dense electrode array. Overall, our findings provide new insights in the componential structure of ERPs elicited in the P300 speller paradigm and have important implications for optimizing the speller’s selection accuracy. Index Terms—Brain–computer interface, Novelty P3, P300.
I. INTRODUCTION RAIN–COMPUTER interfaces (BCIs) allow paralyzed patients who retain cognitive capacities—such as patients suffering from amyotrophic lateral sclerosis (ALS)—to communicate without using muscle activity (e.g., [1]). The P300 speller [2] is a BCI that capitalizes on the fact that rare, task-relevant, stimuli elicit the P300 component in the event-related potentials (ERPs). The P300 speller thus implements the so-called “oddball paradigm,” in which two classes of events (with the term “event” referring, for example, to the presentation of a visual or auditory stimulus) are presented in succession, in a random sequence, with one of the two classes presented rarely. The participant’s task requires the classification of the events. Events in the rare class, whether or not they are the “targets,” elicit a P300 [3], [4]. In the original version of the P300 speller
B
Manuscript received March 09, 2013; revised May 21, 2013; accepted September 22, 2013. Date of current version November 04, 2013. S.-M. Kamp was with the Department of Psychology, University of South Florida, Tampa, FL 33620,USA. She is now with the Department of Psychology, IRTG “Adaptive Minds,” Saarland University, 66123 Saarbrücken, Germany (e-mail:
[email protected]). A. R. Murphy and E. Donchin are with the Department of Psychology, University of South Florida, Tampa, FL 33620 USA (e-mail:
[email protected]. edu;
[email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNSRE.2013.2285398
[2], the user views a matrix with six rows and six columns containing the letters of the alphabet and several additional symbols. The users select one character that they wish to communicate (“target” ) and focus attention on this character. Then, the rows and columns of the matrix are each intensified in a random sequence. This creates an oddball paradigm, in which 12 events constitute the sequence. The flashes of the row and the column containing the target comprise the “rare” class and thus will elicit a P300, and the 10 other flashes, which do not contain the target, are the “frequent” events that do not elicit a P300. A classification function is developed which can detect which flashes elicit a P300. Thus, by detecting which of the six rows, and which of the six columns, elicits the P300, the classification function identifies the target character, which lies at the intersection of the target row and target column. As Farwell and Donchin showed in 1988 [2], using this paradigm the participants can successfully “type” text into a computer using their P300, rather than their fingers. As ERPs are embedded in the ongoing electro-encephalographic activity (EEG), which for the purpose of the P300 BCI is considered noise, extracting the ERP from the background EEG requires signal averaging. For this reason, several repetitions of the flash sequence are necessary for communicating any given character in order to increase the signal-to-noise ratio. Consequently, the system works at the relatively slow speed of a few characters per minute. For example, in an offline bootstrap analysis of classification performance, Donchin et al. [5] reported that the rate of communication was 7.8 characters/min at an accuracy of 80%. Although more recent studies have reported faster typing speeds, these studies have typically included healthy participants; ALS patients often communicate more slowly [6]. Typing, even at such a slow speed, can dramatically improve quality of life for a patient who has no alternative means of communication [7]. However, to achieve the highest possible utility it is important to identify the conditions under which the system’s performance can be improved. Previous studies examined the optimal number of scalp electrodes to use [8], [9] and the most effective classification algorithm to detect the P300 [10]. Furthermore, parameters affecting the optimal user interface design have been investigated, such as the number, size or color of the characters included in the matrix [11]–[13], the inter-stimulus interval between flashes [2], [11], [14], [15], and whether the flashes are induced by changes in luminance, color, or a combination of both [16], or by overlaying the characters with famous 1In the context of the P300 speller, the infrequent events that elicit the P300 are also targets. Note however that, in general, an item need not be a target in order to elicit a P300; it merely needs to belong to the class of infrequent events in a task-relevant sequence, for a review see [4].
1534-4320 © 2013 IEEE
898
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 21, NO. 6, NOVEMBER 2013
faces [17]. Another recent study modified the P300 speller so that each flash included pseudo-randomly selected groups of characters not organized in rows or columns [18]. Although these and other studies provided suggestions for improvements in P300 speller performance, optimizing the system remains a challenge (for a general review see [7]). A key assumption of the P300 speller is that it is the P300 component that best distinguishes between target- and nontarget flashes; an assumption that is natural since the BCI has been developed as an implementation of the oddball paradigm. However, the P300 is not the only component elicited by rare events in variations of the oddball paradigm. Thus, for example, when the oddball sequence is ignored, the rare events elicit a frontal component labeled the P3a [19]. Another component of interest is the Novelty P3 [20], which is largest for novel, irrelevant events that are inserted in the oddball sequence. Since there is evidence that the P3a and the Novelty P3 are actually instances of the same ERP component [21], we will henceforth refer to it as the Novelty P3. The Novelty P3 partly overlaps spatially and temporally with the P300 but they have quite different scalp distributions. Utilizing principal component analysis (PCA) on the data from 128 electrodes, Donchin and his colleagues were able to separate the two components from each other and from other overlapping ERPs and thus elucidated the detailed pattern of relationships that characterize the various components elicited in the oddball paradigm [22]–[24] (see also [25], for similar conclusions drawn from independent component analysis). It is possible that a Novelty P3 and other components besides the P300 are elicited in the P300 speller paradigm as well and that they can provide the classifier with complementary information. That is, the different ERP components may provide independent information that improves the discriminability of ERPs elicited by target- and nontarget flashes. In this context it is important to examine the degree to which either of these two components will serve best as a classifier in the BCI, or perhaps a combination of the two components may provide for improved classification. It will also be fruitful in a next step to outline whether the ERP components respond in a different way to experimental parameters, such as the location of the target character within the matrix, which we investigate in the present study. Further analyses could also investigate the specific task given to the participant (merely “attend” to target flashes, or count the target flashes), the number of characters in the matrix, or whether the participant is fixating the target character or merely attending to it covertly. In conclusion, information on the nature of the ERP components that distinguish between target and standard flashes in the P300 speller may provide directions for the development of classification functions and provide suggestions for the optimal interface design. Using an array of 16 electrodes we investigated the componential structure of the ERPs elicited in the P300 speller paradigm. We acquired data in which each participant “typed” each of 36 matrix characters twice, and then analyzed the ERPs using a spatio-temporal PCA [22]. PCA provides a tool to disentangle components that overlap in time and space. This approach to the analysis of ERP data in the temporal domain has been described in Donchin [26] (see also [27]) and has been in active use as a method to parse the ERP waveform into its components. With
the introduction of dense electrode arrays, the use of spatio-temporal PCA has been introduced, which has expanded the ability to examine the contribution of multiple components to the ERP data. As described by Spencer et al. [2] the “factor scores” associated with the spatial and temporal components serve as measures of ERP component amplitude, and can therefore be used as the dependent variables assessed in the study. The use of PCA for the analysis of ERPs in BCI research is not common (but see for example [28]) and is therefore expected to contribute substantially to an understanding of the componential structure elicited in the P300 speller paradigm. In a supplemental analysis to the main experiment, we applied PCA to the data from a different set of six participants in order to 1) determine whether the componential structure elicited in the main experiment could be generalized to a different sample and 2) examine the spatial distributions of the elicited components with a dense electrode array. An additional goal of the present study was to examine the degree to which the effectiveness of the P300 BCI varies with the specific matrix cell in which the target is located. Virtually all applications of the P300 BCI assume that there are no systematic differences between different target locations within the matrix. Participants’ self-report from our laboratory, however, indicated that the difficulty of focusing attention on a character varies with its location (see also [12]). We therefore examined the degree to which ERP amplitudes and classification accuracy vary with target location. II. METHODS A. Participants In the main experiment, seven healthy college students completed a 1.5-h-long session in exchange for partial course credit. Data from one participant were excluded due to noncompliance with the instructions. The remaining participants were between 18 and 25 years old. Five participants were male and five participants were right handed. The dataset for the additional analysis was from a different set of six participants, who participated in two experimental sessions that occurred three days apart and that were each 2.5 h long. None of them had participated in the original experiment. We here report only data from the second session (in which the conditions matched the conditions used in the first sample). Participants in the second sample were between 21 and 29 years old, five were female, and all were right handed. All participants gave informed consent and all procedures were approved by the Institutional Review Board at the University of South Florida. B. BCI System and Stimuli We used the BCI2000 system [29] with a 6 6 matrix displaying the letters of the alphabet, nine digits and a space bar icon. The characters were displayed in light grey on a black background on a computer screen. Each “typed” character (henceforth referred to as a “trial”) included 15 random sequences of flashes of each row and each column of the matrix. Thus, every trial included a total of 180 flashes (15 sequences of 12 flashes). The duration of each flash was 100 ms and the interval between the onsets of two successive flashes was
KAMP et al.: THE COMPONENT STRUCTURE OF EVENT-RELATED POTENTIALS IN THE P300 SPELLER PARADIGM
175 ms. After each trial there was a pause of about 3 s for the feedback to be displayed before the next trial began. The character the participant was instructed to “type” in a given trial was indicated above the character matrix, and feedback on which character the BCI had selected as the target was displayed below. The online classification function was generated from data previously acquired from different participants using the same matrix layout, number of flashes and interval between flashes. We used this pre-generated classification rule rather than classification rules based on each individual’s data to reduce the duration of each participant run, because toward the end of pilot runs participants tended to show signs of fatigue. Online classification accuracy ranged between 0.35 and 0.76. For the purpose of our offline analysis, we generated individual classification rules for each participant. C. Procedure and Task During the preparations for the EEG recording, the participant was given written instructions to focus attention on the character indicated on top of the screen and to count the total number of flashes of the row and column that include the target. They were told that most of the time the target would flash 30 times, but sometimes this number would deviate from 30, in which case they were to report this to the experimenter during the next break. The purpose of this task was to ensure that participants focused their attention on the target, but in reality the correct number was always 30. After the completion of 4 practice trials, each participant “typed” each character as indicated on top of the screen. The order of characters was generated randomly for each participant and consisted of two complete cycles through the matrix, so that each participant “typed” each character twice. Breaks were allowed after each block of 4 letters. At the end of the session the participant was debriefed and questioned about differences in task difficulty with target location. It is important to emphasize that the subject was not instructed to fixate the character. We assumed, following Farwell and Donchin who followed Posner [30], that subjects can shift attention across elements of a display without necessarily shifting their gaze. D. Data Acquisition The EEG was recorded with an elastic cap including 16 tin electrodes (F3, Fz, F4, C3, Cz, C4, T3, T4, CP3, CP4, P3, Pz, P4, PO7, Oz, and PO8; Electro-Cap International, Inc.), placed according to the 10–20 electrode system. The data were referenced to linked mastoids and digitized at 160 Hz. E. Classification Accuracy Offline, we estimated the performance of a stepwise linear discriminant analysis (SWLDA)-based classifier generated individually for each participant (e.g., [2]). We generated the SWLDA weights for each participant based on the EEG epochs of only the first sequence of “typed” characters (recall that each participant completed two complete cycles of all characters), using features from all 16 electrodes and a time window of the first 800 ms after the flash. After applying an 8-point moving
899
average filter, the SWLDA generated weights for the 60 features (microvolt measurements at combinations of electrodes and time points) that best distinguished between target and nontarget flashes. The individual SWLDA functions were applied to both the first and the second sequence of “typed” characters to estimate classification accuracy. Thus, we used a bootstrap algorithm similar to [5], which picked a random sample of (between 1 and 15) sets of 12 flashes including two target- and 10 standard flashes (each set thus simulates one sequence of all row- and column flashes). We submitted this sample to the classification rule and the row and the column with the highest SWLDA score were selected as targets. This was repeated 1000 times and the percent of correct selections was calculated as an estimate of classification performance. Overall accuracy was first computed for sets of 1–15 flashes. The second step complemented the analysis of target character position on ERP amplitudes described below, such that classification accuracy was estimated for each of the 36 target character positions separately. F. Principal Component Analysis (PCA) To analyze the componential structure of the ERPs, we conducted a spatio-temporal PCA [22] using Dien’s toolbox ([31], v. 1.23). Beforehand, the data were band pass filtered using cutoff frequencies of 0.1 and 13 Hz, and then segmented into epochs of 200 ms before to 600 ms after the flash (note that in order to determine whether the filtering settings affected the outcome of the PCA we repeated the analysis on the unfiltered data; the result was practically identical). We computed subject averages separately for each target location, for row- and column flashes, and for target- and nontarget flashes. In an initial analysis step, we conducted individual PCA’s for each participant. Both the spatial distributions of the resulting spatial factors (SFs), as well as visual inspection of the subject ERPs revealed strong similarities in the ERPs across subjects. That is, each participant showed a frontally distributed- and a parietally distributed component. Due to these similarities, all subject ERPs were submitted to a single, overall PCA. Thus, a total of 864 ERP segments, including ERPs from six participants, 36 matrix positions, two flash types (nontarget or target) and two flash types (row or column) from 16 electrodes and 128 time points were submitted to the PCA. Both PCA steps were conducted on covariance matrices without Kaiser normalization. We applied an Infomax rotation in the spatial step and Varimax rotations in the temporal step (for a comparison of rotation methods, see [32]). The number of factors to rotate was determined by a scree test. G. Analysis of Target Character Location We analyzed the variance of ERP amplitudes and classification accuracy for one, three, and six flashes with the location of the target character within the matrix. We operationalized target location in three ways: a) grouping according to edges, corners, and the middle of the matrix (three levels); b) grouping according to the matrix row of the target character (six levels); and c) grouping according to the matrix column of the target (six levels). These groupings were used as independent variables in ANOVAs, which also included participant as a random
900
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 21, NO. 6, NOVEMBER 2013
effect. A Bonferroni correction resulted in a significance level of (note, however, that no additional effects were significant at ). Main effects were investigated with Tukey-Kramer post-hoc tests. We did not find any significant effects for comparisons (a) and (c), and therefore we only report results on comparison (b). H. Classification Based On Only P300 or Only Novelty P3 For each SF, the PCA results in “factor score coefficients” that can be used to calculate “virtual ERPs” (factor scores over time; [22]). In effect, the “loadings” on each SF are used as a filter that when applied to the single trial data isolates the EEG that is associated with the specific factor. The averaged output of this process yields what have come to be known at the “virtual ERPs” which can be conceptualized as a display of the time course of brain activity associated with the specific SF over time and can be calculated for individual EEG trials. Thus, we next developed classification rules, extracting SWLDA weights from the virtual ERPs from only the P300 factor or only the Novelty P3. Note that the SWLDA step was done separately for each participant to ensure that potential individual differences in the temporal morphology of each component did not affect the result. We then calculated the percent of trials in a bootstrap analysis in which each classification function made the correct target selection. To identify why each classifier made incorrect decisions, we investigated the virtual ERPs elicited in trials in which each of these classifiers identified the character that the participant was actually focusing on, as compared to trials in which it did not. The goal was to determine whether nontarget flashes in some instances elicit a P300 or a Novelty P3, which in turn leads to a classification of a nontarget as a target. Finally, we calculated separate virtual ERPs for target flashes, nontarget flashes adjacent to the target, and standard flashes not adjacent to the target to determine whether flashes neighboring the target can elicit a Novelty P3 and a P300. I. Supplemental Analysis To investigate whether the component structure revealed by our original analysis generalized beyond our initial sample, we conducted an analysis of the data from six additional participants. The additional data stem from the second session of a two-session study. Participants were presented with the P300 speller paradigm with a 6 6, a 4 4, and a 2 2 matrix in random order. Only data from the 6 6 matrix are reported here. Participants either received identical instructions as in the main experiment or were instructed to “focus their attention” on the target character and to note every time it flashed . Otherwise the parameters of the P300 speller paradigm were analogous to the main experiment. The EEG was recorded from 128 electrodes with an EGI system (Electrodesics, Inc.), which was filtered at 0.1–20 Hz, segmented, and offline rereferenced to linked mastoids. We again performed a spatio-temporal PCA on the ERPs. III. RESULTS Grand average ERPs across all matrix positions and all participants for row-, column-, target-, and nontarget flashes at the
Fig. 1. Grand average ERPs for row-, column-, target-, and nontarget (standard) flashes across all participants and target locations for frontal (Fz) and parietal (Pz) recording sites.
Fig. 2. SWLDA-based classification accuracy for each participant as a function of the number of flashes included in the analysis.
frontal (Fz) and parietal (Pz) electrodes are shown in Fig. 1. Two positive peaks are pronounced in the ERPs of the target flashes, the first peak with a frontal maximum at 250 ms, and the second one with a parietal maximum about 420 ms after the flash. These characteristics suggest that the earlier peak may be an instance of the Novelty P3 and the later peak represents the P300. Therefore, our data support the idea that components other than the P300 are elicited in the P300 speller paradigm, which distinguish between target and standard flashes. The PCA (see below) was applied to detect whether the two components are indeed independent, to quantify their amplitudes, and to investigate whether additional components were elicited that were not as clearly visible in the average ERPs. A. SWLDA and Overall Classification Accuracy In order to determine whether the BCI was able to accurately identify target flashes in a traditional SWLDA analysis, we calculated individual classification functions for the raw EEG data of each participant offline. Classification accuracy for each participant and for 1–15 flashes included in the analysis is shown in Fig. 2. For all but one participant the correct target character was identified with a probability of at least 0.8 when four flashes were included. For participant 4, the BCI reached an accuracy of about 0.7 at five flashes. The mean overall classification accuracies for 1, 3, and 6 flashes, as estimated by the bootstrap analysis, were 0.48, 0.78, and 0.91, respectively. These results, in combination with the online accuracies based on the pregenerated SWLDA rule, ranging between 0.35 and 0.76, show that the BCI worked reliably for all participants using a traditional SWLDA. In order to determine whether each individual classification function was based on the P300, the Novelty P3, or both ERP components, we also examined the weights generated by the SWLDA. All functions contained features with positive weights
KAMP et al.: THE COMPONENT STRUCTURE OF EVENT-RELATED POTENTIALS IN THE P300 SPELLER PARADIGM
901
Fig. 3. PCA results. (a) Spatial factor loadings for all four spatial factors. (b) Factor scores plotted over time: “virtual ERPs” for row-, column-, target-, and nontarget (standard) flashes. (c) Loadings of the first four temporal factors for SF1 and SF2. Temporal factor 3 of SF1 was identified as P300 and temporal factor 2 of SF2 was identified as the frontal positivity.
from at least one fronto-central electrode at a latency around 250 ms as well as at least one parietal electrode at a latency around 400 ms. Therefore, classification rules for all participants were based on both the P300 and the Novelty P3. B. PCA We next conducted a spatio-temporal PCA to analyze the component structure of the ERPs, and to investigate whether the P300 and the Novelty P3 explained different portions of the variance. We rotated four SFs with 10 temporal factors (TFs) each. The PCA solution accounted for a total of 87% of the variance in the data. Fig. 3(a) and (b) shows the SF loadings and the associated virtual ERPs. SF1 was parietally distributed and showed a positive peak in the virtual ERPs for target flashes around 430 ms, which is consistent with the P300. SF2 was frontally distributed and peaked
at 250 ms, thus representing the frontal positivity. Since these SFs accounted for the largest part of the variance (SF1: 46%; SF2: 29%), since they corresponded to two components typically elicited in oddball paradigms (P300 and Novelty P3), and because the other SFs did not exhibit a morphology that was easily interpretable as specific ERP components, the other SFs were not further analyzed. The first four TFs for SFs 1 and 2 are plotted in Fig. 3(c). Since their time course matched the respective peaks, TF3 of SF1 was identified as the P300, and TF2 of SF2 as the Novelty P3. We next investigated whether the spatio-temporal factor scores (as a measure of component amplitude) corresponding to the P300 and the Novelty P3 differed between target- and nontarget flashes [Fig. 4(a)]. Since variance with experimental manipulations is a defining characteristic of ERP components [33], this analysis aimed to confirm that the factors could
902
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 21, NO. 6, NOVEMBER 2013
Fig. 4. Factor scores for the P300 and the Novelty P3 by participant and flash type. (a) Target versus standard flashes. Factor scores are collapsed across rowand column flashes and all target positions. (b) Row- versus column flashes. Factor scores are for target flashes only.
indeed be interpreted as P300 and Novelty P3, respectively. The effect for flash type (target versus standard) was significant for all participants for the Novelty P3 and for all but one participant for the P300. For the sake of simplicity we restricted all further analysis to the target flash ERPs. In order to determine whether subsequently, it was justified to collapse ERPs elicited by row- and column flashes, we also compared the factor scores between row- and column flashes [Fig. 4(b)]. The 6 (participant) by 2 (flash type: row versus column) ANOVAs on the P300 and Novelty P3 scores revealed main effects for participant [P300: , ; Novelty P3: , ], main effects for flash type [P300: , ; Novelty P3: , ], and significant interactions [P300: , ; Novelty P3: , ]. Thus, factor scores tended to be larger for row- than for column flashes. Due to the significant interactions, all further analyses were conducted separately for ERPs elicited by row- and column flashes. The fact that the PCA identified the P300 and the Novelty P3 as different SFs indicates that the two components explain different portions of the variance in the data. To further investigate whether the two components are independent of each other, we computed a partial correlation between their amplitudes (i.e., their spatio-temporal factor scores) across target locations, when between-participant variance and variance due to row- versus column flashes was accounted for. There was a significant, but only modest partial correlation, , . C. Effects of Target Location on ERP Amplitudes Next, we tested whether ERP amplitudes varied with the location of the target character within the matrix, and if so, whether P300 and the Novelty P3 did so in the same qualitative fashion. Fig. 5(a) displays the spatio-temporal factor scores for the P300 and the Novelty P3 elicited by row- and column flashes for all target locations. The three groups of locations outlined in the methods section were used as independent
Fig. 5. Variance of ERPs and classification accuracy with the location of the target character. A: Factor scores of P300 and Novelty P3 elicited by row- and column flashes, by target character location. The color of each square indicates the magnitude of the factor score in the target flash ERPs when the respective character is the target. B: Proportion of accurate selections by the number of flashes analyzed (1 or 3), and by target character location. Note: Each panel is displayed in a different scale.
variables in ANOVAs, which also included participant as a random effect. Since the only significant variance with target location was due to the matrix row, we only report results from this comparison here. The ANOVA testing for effects of the target row on the P300 elicited by row flashes revealed a main effect for target row . A post-hoc test indicated that the P300 was smaller for targets in the first row compared to all other rows, and that row 2 differed from row . The analogous ANOVA on the factor scores of the Novelty P3 elicited by column flashes also revealed a main effect for target row , confirming the visual impression that factor scores were larger when the target was located in the top rows compared to the bottom rows. Post-hoc tests showed that row 1 differed from rows 5 and 6; and that row 3 differed from row 6. The variance of P300 and Novelty P3 with the target row was therefore in opposite directions. The P300 elicited by column flashes also showed larger amplitudes when the target was in the bottom row, but this effect was not significant , and there was only a statistical trend within the frontal positivity elicited by row flashes . Since ERP amplitudes varied with the target row, we next investigated whether the accuracy of the SWLDA based classification function, using one, three, or six flashes, mirrored this pattern [Fig. 5(b)]. The ANOVA on the matrix row of the target resulted in a main effect when one flash was included in the analysis , indicating that accuracies for targets in rows and were higher than for targets in row . The same trend was apparent when 3 flashes were included . Mean accuracy was 0.83 for row 1 and 0.72 for row 6.
KAMP et al.: THE COMPONENT STRUCTURE OF EVENT-RELATED POTENTIALS IN THE P300 SPELLER PARADIGM
903
TABLE I ERP AMPLITUDES AND CLASSIFICATION ACCURACY FOR TARGETS IN THE FIRST ROW, COMPARED TO TARGETS IN THE LAST ROW
Fig. 6. Classification accuracy, by participant, when the selection is based on the P300 rule, the Novelty P3 rule, and a rule that combines information from participant , etc. both SFs. Note:
When six flashes were analyzed the effect for matrix row was not significant , possibly due to a ceiling effect in accuracy. Therefore, SWLDA-based classification accuracy mirrored the pattern of the Novelty P3. The main effects of matrix row were present for all participants (Table I). D. Classification Based on Only P300 or Only Novelty P3 We identified two major ERP components elicited in the P300 speller paradigm, the P300 and the Novelty P3, which were represented by separate PCA factors and which showed only a small correlation. A natural next step is to examine which of the components is more powerful in discriminating between target- and standard flashes. Therefore, we generated classification functions based on the EEG that was filtered through the SF coefficients of each component. Thereby, we ensured that each classification function was based on only the respective component. Fig. 6 shows the accuracy of the classification functions based on the P300 and the Novelty P3, compared to a classification function that combined scores from both, when 15 flashes were included in the analysis. The Novelty P3-based function generally yielded better accuracies than the P300-based function. Only for participant 1 classification accuracy was superior for the P300 function. Not surprisingly, there was a trend for the P300 based function to yield better accuracies for targets in the bottom row compared to the top row , while this pattern was reversed for the Novelty P3 (row 1: ; row 6: ). Thus, there were notable differences in how well a classification function was able to identify target flashes depending on which component the function was based on. The finding that the Novelty P3-based function was generally superior to the P300-based function was somewhat surprising. In order to ex-
Fig. 7. Virtual ERPs of correct selections and row- and column errors by (a) the P300-based classification function and (b) the Novelty P3-based classification function. Note that the baseline period (200 ms before the stimulus) is not displayed because these time points were not included in the classification function.
amine possible reasons for the accuracy differences, we investigated the virtual ERPs for trials in which each function misclassified a nontarget character as the target. Fig. 7 shows the virtual ERPs of each SF, elicited in trials in which the respective function made incorrect selections (since row- and column selections are independent of each other we conducted this analysis separately for row- and column flashes). Not surprisingly, when the target character was correctly identified by either classification function, the respective target flashes elicited a large ERP component (P300 or Novelty P3, respectively; Fig. 7(a) and (b), top panels). For trials where the P300 classifier made erroneous selections, flashes of the row- or column that were wrongfully selected as targets showed a larger P300 than the actual target flashes (Fig. 7(a), middle/bottom panels). For the Novelty P3-based classifier the pattern was different: Both the wrongfully selected flashes and the actual target flashes elicited a Novelty P3 with about equal amplitudes (Fig. 7(b), middle/bottom panels). This suggests that nontarget flashes sometimes elicit a P300, which led our P300 based classifiers to make erroneous selections more often than the Novelty P3 classifier. Errors made by the Novelty P3 classifier might be primarily caused by other factors, such as high levels of noise. One possible pattern that may lead nontarget flashes to elicit a P300 is that flashes neighboring the target may attract involuntary attention, thus eliciting a P300. To test this, we calculated P300 and Novelty P3 virtual ERPs elicited by flashes neighboring the target, as compared to actual target flashes and standards that did not neighbor the target. There was no evidence for a P300 or Novelty P3 being elicited by flashes neighboring the target (Fig. 8). This suggests that the P300 elicited by some nontarget flashes that led to erroneous selections was due to factors other than the spatial relationship between the flash and the target character.
904
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 21, NO. 6, NOVEMBER 2013
Fig. 10. PCA from the supplemental analysis: Spatial factors 1 and 2. (a) Spatial factor loadings and (b) virtual ERPs. The spatio-temporal characteristics of the first two factors closely resemble the Novelty P3 and the P300 obtained in the main experiment. Fig. 8. Virtual ERPs of (a) the P300 (SF1) and (b) the Novelty P3 (SF2) for target flashes, standard flashes that are not adjacent to the target (“standard flash”) and standard flashes adjacent to the target.
Fig. 9. Grand average ERPs from the supplemental analysis. Shown are ERPs elicited by target- and standard flashes at the frontal (Fz) and parietal (Pz) electrodes.
E. Supplemental Analysis The goals of our supplemental analysis were twofold: First, we wanted ensure that the ERP structure of the first experiment generalized beyond our original sample. Second, the data were recorded with a 128 channel system, so we were able to characterize more precisely the spatial distributions. The ERPs elicited by target- and standard flashes at Fz and Pz, similar to the main experiment, were characterized by an early frontal positivity and a later parietal positivity (Fig. 9). In line with this, the first two SFs obtained in the PCA (Fig. 10) showed 1) a fronto-central distribution and an earlier positive peak; the same pattern as obtained previously for the Novelty P3 and 2) a parietal distribution and a later positive peak, corresponding to the P300. These data thus confirm that in our paradigm the two ERP components that capture the most variance in the data are the Novelty P3 and the P300. IV. DISCUSSION Our results indicate that two ERP components with distinct spatial and temporal characteristics—the P300 and an additional, frontal positivity—distinguish the ERPs elicited by target events from the nontarget events in the P300 speller paradigm. All the classification rules, for each subject, generated offline on the raw EEG by a SWLDA contained features
from both the P300 and the Novelty P3. The two components were only modestly correlated, and shared only 10.4% of their variance between trials. It is likely that their correlation represents the influence of third variables, such as alertness and attentional resources allocated to the task, rather than a direct link between the components. The components varied in opposite directions with the location of the target within the matrix: The P300 elicited by row flashes was largest for targets in the lower part of the matrix, while the Novelty P3 elicited by column flashes was largest for targets in the upper rows. Classification accuracy paralleled the pattern of the Novelty P3. Additional analyses suggested that the Novelty P3 may be a more useful basis for classification than the P300; this difference might be due to some nontarget flashes eliciting a P300. Finally, it is noteworthy that we obtained a comparable component structure in a separate sample of six participants, whose ERP data were recorded from a dense electrode net. Prior P300 speller studies have seldom distinguished between the frontal positivity and the parietal P300, possibly since their overlap in time and space [22]–[24] required a componential ERP analysis to characterize them as separate components. Its spatio-temporal characteristics and its response to experimental variables suggest that the frontal positivity is an instance of the Novelty P3 [20]. However, prior research has suggested that the Novelty P3 and the P300 respond to different experimental manipulations. Thus, the P300 is elicited by rare, task-relevant events as well as by novel, task-irrelevant events. The Novelty P3, in turn, is most pronounced in response to novel, task-irrelevant events and is smaller when elicited by rare, task-relevant events [20], [22]–[24]. In our study the frontal positivity was large although the eliciting event (the target flash) was task-relevant. Note, however, that some prior studies have indicated that task irrelevance is not necessary for the elicitation of a Novelty P3 [25], [34], supporting the idea that the frontal component in our data is indeed a Novelty P3. One possibility is that the flashes of the target character are perceptually more salient than nontarget flashes. Since the Novelty P3 has been shown to be more sensitive to saliency than task-irrelevance [24], [34], this
KAMP et al.: THE COMPONENT STRUCTURE OF EVENT-RELATED POTENTIALS IN THE P300 SPELLER PARADIGM
could explain why a Novelty P3 is elicited in the P300 speller paradigm. In our view, the morphology and eliciting conditions of the frontal component thus suggest that it is an instance of the Novelty P3, and for the purpose of clarity we have referred to this component as the Novelty P3 throughout the paper. However, we cannot rule out the possibility that this component is a paradigm specific component not elicited in traditional oddball paradigms. A relevant issue when discussing the P300 and the Novelty P3 is that the two components have been reported to show different patterns of habituation with repeated stimulus presentations—the Novelty P3 tends to show reduced amplitudes with repeated presentations [20], while the P300 does not [35]. We analyzed patterns of habituation both within the flash sequence of each typed character, and across the entire experimental session. However, we did not find consistent patterns of habituation, either for the P300 or for the Novelty P3. While this null result needs corroboration from further studies, it is possible that the fact that the target flashes were task-relevant prevented the typical habituation to occur for the Novelty P3. This would be in line with a study [36] that found that when novel stimuli were to be memorized and were therefore task-relevant, the Novelty P3 did not habituate. In the present dataset, classification based on the Novelty P3 was generally more effective than classification based on the P300. However, one limitation of our study is that our results were based on an offline bootstrap method and we did not test the online performance of each classifier. Doing so is a fruitful direction for future studies. It is worth noting, however that although we observed good classification accuracies for our PCA based classifiers, the traditional SWLDA based on the raw data was superior in all cases. Hence, while our data provide evidence for the usefulness of considering the component structure in developing classifiers, we do not suggest that the precise method of first conducting a spatial PCA and then conducting a SWLDA on the virtual ERPs is necessarily the best method for obtaining optimal online accuracy. It is worth noting that our analysis of the componential structure of the ERPs did not reveal any early components, such as the N1 or N2, that differentiated between target- and standard flashes (such early components also did not emerge in a reanalysis of the ERPs without the band-pass filter applied in our original analysis). This is in contrast to studies reporting that target flashes elicit a strong occipital N1 and/or N2 component in the P300 speller paradigm [37]–[39]. One possible explanation for this discrepancy may be the limited number of occipital electrodes included in our electrode setup, however, our supplemental analysis, using 128 electrodes, also identified the P300 and the Novelty P3 as the main components elicited in our paradigm. One possible explanation for the discrepancy between our data and some previous studies that have reported “exogenous” ERP components to drive P300 speller accuracy may be differences in the specific task parameters of our study, including the participants’ task. For example, Treder and Blankerz [38] found that early ERP components only distinguished target- from standard ERPs in an “overt,” but not a “covert” attention condition. Although there are no obvious reasons why our task parameters should be closer to their “covert” condition, it is a possible ex-
905
planation for the discrepancy. However, overall our data do not support the view that early “exogenous” ERP components critically contribute to P300 speller performance. One possible explanation for the fact that the effects of target location on the P300 and the Novelty P3 were in opposite directions could be a systematic variance of task difficulty with target location, and that the Novelty P3 and the P300 respond differently to this variable. In support of this idea, previous studies have indicated that the P300 and the Novelty P3 are sensitive to task difficulty. For example, when the difficulty of a primary task is increased, the amplitude of the P300 elicited in the primary task increases, while the P300 associated with a secondary task decreases [40]. Other studies have reported a decrease in P300 amplitude, but an increased amplitude of the Novelty P3 as perceptual discrimination difficulty in an oddball task increases [24], [41]. We investigated whether task difficulty differences contributed to our results by analyzing participant reports from the debriefing, and additionally by comparing the number of times participants reported numbers of flashes deviating from 30 (i.e., “counting errors”) across locations. We did not find any evidence for systematic task difficulty variations with target location. While it is possible that our measures were not sensitive enough to detect small task difficulty differences, future studies should investigate alternative explanations for the effects of target location on ERP amplitudes. Further research is also necessary to determine why effects of target location on ERP amplitudes were restricted to either rowor column flashes. Well-established determinants of P300 amplitude are the probability of the infrequent event [4], [42] and task difficulty [40]. However, since all flashes are embedded in the same sequence in the P300 speller, and the target row and the target column are highlighted with the same probability, there is no theoretical reason to expect a difference between the ERPs of row- and column flashes. One possibility is that depending on the precise location of the target within the matrix, row- and column flashes differ in their perceptual salience and thus their effectiveness to draw attention. Alternatively, our data may indicate that the cognitive processes engaged while using the BCI speller are not equivalent to those of classic two-stimulus oddball tasks. Practically, our finding suggests that it might be beneficial to use distinct classification functions for row- and column flashes in the P300 speller. Classification accuracy of a traditional SWLDA for targets in the top row was about 10% points higher than for targets in the bottom row when one or three flashes were analyzed. Thus, amplitudes of the Novelty P3, but not the P300 positively co-varied with accuracy, although information from both components was included in the classification functions of every participant. Allison and Pineda [12] found that P300 amplitude was affected by matrix size, but that this effect did not translate into classification accuracy differences. Variations in ERP amplitudes therefore do not necessarily affect discrimination between targets and nontargets. The covariance of accuracy with the Novelty P3 may be due to smaller amplitudes elicited by column flashes, resulting in the precise amplitudes to be more critical than for row flashes. While our study provides first evidence for differences in classification accuracy based on target character location within
906
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 21, NO. 6, NOVEMBER 2013
the matrix, future work must determine the generalizability of the results. First, it is crucial to determine whether differences in the target character positions within the matrix generalize to ALS patients and other potential end users of the P300 speller. Second, it is unclear whether our results generalize to P300 BCIs with different parameters. In our matrix all digits were located in the bottom rows, while all letters were in the upper rows. Therefore, the identity of the target character was confounded with its location. Although target identity should not affect the error rate, we cannot rule out the possibility that the variance with target location is due to specific characters located in specific positions. Furthermore, it is unclear whether our findings generalize to setups in which the flashes include pseudo-random groups of characters not aligned in rows or columns. This recently developed “checkerboard paradigm” appears to be superior to the row/column paradigm in its communication speed as well as ALS patients’ preferences [18]. Finally, our data were based on an offline bootstrap analysis; whether the same patterns are found with an individually generated online classification function remains to be established. If our findings generalize beyond our sample and paradigm, they have several practical implications. First, the matrix may be reorganized such that commonly used characters are placed in the most advantageous positions, leading to a decrease in overall error rate. Similarly, items for which it is most critical to avoid erroneous selections could be placed in locations associated with the best classification accuracy. The latter idea may not be relevant for the P300 speller, because the significance of spelling errors should not vary with letter identity. However, recent projects have adapted the P300 BCI to control robotic arms [43], wheelchairs [44], [45], or web browsers [46], for which some selections may be more sensitive to errors than others. It is worth noting that reorganizing the matrix is only advisable if the benefit in increased accuracy outweighs a potential cost in the system’s usability. For example, in the P300 speller matrix the letters are organized in alphabetic order. Changing this order might make each letter harder and less intuitive to find. The present results may also help speed up the classification process of the P300 BCI. Thus, the execution of a selection could depend on the suspected target location. If the suspected location is associated with high accuracy, the selection could be executed after a few flashes, but if the associated accuracy is low, more flashes could be added to increase selection confidence. One difficulty with this idea is a potential covariance of the probability of a wrongful selection of a character with its location. For example, if the true target was located in the bottom row (which is associated with low accuracy), but the BCI “suspects” the target to be in the top row (associated with high accuracy), this wrongful selection would be executed after only a few flash sequences. It is important, therefore, that future studies explore the potential of this idea. It is also possible that future classifiers may utilize the relative amplitudes of P300 and Novelty P3 to decide whether a flash is a target or a nontarget. Thus, in our study the relative amplitude of the Novelty P3 compared to the P300 was largest in the top rows while the reverse was true for the bottom rows. Future classification rules may utilize such patterns to optimize performance.
In conclusion, we found that (at least) two ERP components are elicited in the P300 speller paradigm, the P300 and the Novelty P3. The components are independent of each other and contribute to classification performance in different ways and to different degrees. ACKNOWLEDGMENT We would like to thank M. P. Miller and S. C. Colbert for support with the analysis scripts. Furthermore, we are grateful to T. Brumback and G. R. Forester, who provided helpful comments on an earlier version of this manuscript. REFERENCES [1] J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain-computer interfaces for communication and control,” Clin. Neurophysiol., vol. 113, pp. 767–791, 2002. [2] L. A. Farwell and E. Donchin, “Talking off the top of your head: toward a mental prosthesis utilizing event-related potentials,” Electroencephalogr. Clin. Neurophysiol., vol. 70, pp. 510–523, 1988. [3] S. Sutton, M. Braren, J. Zubin, and E. R. John, “Evoked-potential correlates of stimulus uncertainty,” Science, vol. 150, pp. 1187–1188, 1965. [4] E. Donchin, “Surprise!… surprise?,” Psychophysiology, vol. 18, pp. 493–513, 1981. [5] E. Donchin, K. M. Spencer, and R. Wijesinghe, “The mental prosthesis: Assessing the speed of a P300-based brain-computer interface,” IEEE Trans. Rehabil. Eng., vol. 8, no. 2, pp. 174–179, 2000. [6] E. W. Sellers and E. Donchin, “A P300-based brain-computer interface: Initial tests by ALS patients,” Clin. Neurophysiol., vol. 117, pp. 538–548, 2006. [7] E. W. Sellers, Y. Arbel, and E. Donchin, “Bcis that use event-related potentials,” in Brain-Computer Interfaces: Principles and Practice, J. R. Wolpaw and E. W. Wolpaw, Eds. Oxford, U.K.: Oxford Univ. Press, 2012, p. 300. [8] D. J. Krusienski, E. W. Sellers, T. M. McFarland, T. M. Vaughan, and J. R. Wolpaw, “Toward enhanced P300 speller performance,” J. Neurosci. Meth., vol. 167, pp. 15–21, 2008. [9] M. Thulasidas and C. Guan, “Optimization of BCI speller based on P300 potential,” in Proc. IEEE Eng. Med. Biol., Shanghai, China, 2005, pp. 5396–5399. [10] D. J. Krusienski, E. Sellers, F. Cabestaing, S. Bayoudh, D. J. McFarland, T. M. Vaughan, and J. R. Wolpaw, “A comparison of classification techniques for the P300 speller,” J. Neural Eng., vol. 3, pp. 299–305, 2006. [11] E. W. Sellers, D. J. Krusienski, D. J. McFarland, T. M. Vaughan, and J. R. Wolpaw, “A P300 event-related potential brain-computer interface (BCI): the effects of matrix size and inter stimulus interval on performance,” Biol. Psych., vol. 73, pp. 242–252, 2006. [12] B. Z. Allison and J. A. Pineda, “ERPs evoked by different matrix sizes: Implications for a brain computer interface (BCI) system,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 11, no. 2, pp. 110–113, Jun. 2003. [13] M. Salvaris and F. Sepulveda, “Visual modifications of the P300 speller BCI paradigm,” J. Neural Eng., vol. 6, p. 046011, 2009. [14] B. Z. Allison and J. A. Pineda, “Effects of SOA and flash pattern manipulations on ERPS, performance, and preference: Implications for a BCI system,” Int. J. Psychophysiol., vol. 59, pp. 127–140, 2006. [15] D. J. McFarland, W. A. Sarnacki, G. Townsend, T. Vaughan, and J. Wolpaw, “The P300-based brain-computer interface (BCI): Effects of stimulus rate,” Clin. Neurophysiol., vol. 122, pp. 731–737, 2011. [16] K. Takano, T. Komatsu, N. Hata, Y. Nakajima, and K. Kansaku, “Visual stimuli for the P300 brain-computer interface: A comparison of white/gray and green/blue flicker matrices,” Clin. Neurophysiol., vol. 120, pp. 1562–1566, 2009. [17] T. Kaufmann, S. M. Schulz, C. Gruenzinger, and A. Kuebler, “Flashing characters with famous faces improves ERP-based brain-computer interface performance,” J. Neural Eng., vol. 8, p. 056016, 2011. [18] G. Townsend, B. K. LaPallo, C. B. Boulay, D. J. Krusienski, G. E. Frye, C. K. Hauser, N. E. Schwartz, T. M. Vaughan, J. R. Wolpaw, and E. W. Sellers, “A novel P300-based brain-computer interface stimulus presentation paradigm: Moving beyond rows and columns,” Clin. Neurophysiol., vol. 121, pp. 1109–1120, 2010.
KAMP et al.: THE COMPONENT STRUCTURE OF EVENT-RELATED POTENTIALS IN THE P300 SPELLER PARADIGM
[19] N. K. Squires, K. C. Squires, and S. A. Hillyard, “Two varieties of longlatency positive waves evoked by unpredictable auditory stimuli in man,” Electroencephalogr. Clin. Neurophysiol., vol. 38, pp. 387–401, 1975. [20] E. Courchesne, S. A. Hillyard, and R. Galambos, “Stimulus novelty, task relevance, and the visual evoked potential in man,” Electroencephalogr. Clin. Neurophysiol., vol. 45, pp. 754–766, 1975. [21] R. F. Simons, F. K. Graham, M. A. Miles, and X. Chen, “On the relationship OFA and the novelty-P3,” Biol. Psych., vol. 56, pp. 207–218, 2001. [22] K. M. Spencer, J. Dien, and E. Donchin, “A componential analysis of the ERP elicited by novel events using a dense electrode array,” Psychophysiology, vol. 36, pp. 409–414, 1999. [23] J. Dien, K. M. Spencer, and E. Donchin, “Parsing the ’Late positive complex’: mental chronometry and the ERP components that inhabit the neighborhood of the P300,” Psychophysiology, vol. 41, pp. 665–678, 2004. [24] A. Goldstein, K. M. Spencer, and E. Donchin, “The influence of stimulus deviance and novelty on the P300 and novelty P3,” Psychophysiology, vol. 39, pp. 781–790, 2003. [25] S. Debener, S. Makeig, A. Delorme, and A. K. Engel, “What is novel in the novelty oddball paradigm? Functional significance of the novelty P3 event-related potential as revealed by independent component analysis,” Cogn. Brain Res., vol. 22, pp. 309–321, 2005. [26] E. Donchin, “A multivariate approach to the analysis of average evoked potentials,” IEEE Trans. Biomed. Eng., vol. 13, no. 3, pp. 131–139, Jul. 1966. [27] E. Donchin and E. Heffley, “Multivariate analysis of event-related potential data: A tutorial review,” in Multidisciplinary perspectives in event-related potential research, D. Otto, Ed. Washington, DC: U.S. Gov. Print. Office, 1978, pp. 555–572. [28] N. A. Gates, C. K. Hauser, and E. W. Sellers, “A longitudinal study of brain-computer interface and progression of amyotrophic lateral sclerosis,” in Found. of Augmented Cognition. Directing the Future of Adaptive Syst. Berlin, Germany: Springer, 2011, p. 300. [29] G. Schalk, D. McFarland, T. Hinterberger, N. Birbaumer, and J. Wolpaw, “BCI2000: A general-purpose brain-computer interface (BCI) system,” IEEE Trans. Biomed. Eng., vol. 51, no. 6, pp. 1034–1043, Jun. 2004. [30] M. I. Posner, “Orienting of attention,” Q. J. Exp. Psychol., vol. 32, pp. 3–25, 1980. [31] J. Dien, “The ERP PCA toolkit: An open source program for advanced statistical analysis of event related potential data,” J. Neurosci. Meth., vol. 187, pp. 138–145, 2010. [32] J. Dien, “Evaluating two-step PCA of ERP data with Geomin, Infomax, Oblimin, Promax, and Varimax rotations,” Psychophysiology, vol. 47, pp. 170–183, 2010. [33] E. Donchin, W. Ritter, and W. McCallum, “Cognitive psychophysiology: The endogenous components of the ERP,” in Event-Related Brain Potentials in man, E. Callaway, P. Tueting, and S. H. Koslow, Eds. New York: Academic, 1978. [34] Y. M. Cycowicz and D. Friedman, “The old switcheroo: when target environmental sounds elicit a novelty P3,” Clin. Neurophysiol., vol. 115, pp. 1359–1367, 2004. [35] E. Donchin, E. Heffley, S. A. Hillyard, N. Loveless, I. Malzman, A. Oehman, F. Roesler, D. Ruchkin, and D. Siddle, “Cognition and eventrelated potentials II. The orienting reflex and P300,” Ann. NY Acad. Sci., vol. 425, pp. 39–57, 1984. [36] Y. M. Cycowicz and D. Friedman, “The effect of intention to learn novel, environmental sounds on the novelty P3 and old/new recognition memory,” Biol. Psych., vol. 50, pp. 35–60, 1999. [37] S. L. Shishkin, I. P. Ganin, I. A. Basyul, A. Y. Zhigalov, and A. Y. Kaplan, “N1 wave in the P300 BCI is not sensitive to the physical characteristics of stimuli,” J. Integr. Neurosci., vol. 8, pp. 471–85, 2009. [38] M. S. Treder and B. Blankertz, “Covert attention and visual speller design in an ERP-based brain-computer interface,” Behav. Brain Funct., vol. 6, pp. 28–41, 2010. [39] L. Bianchi, S. Sami, A. Hillebrandt, I. P. Fawcett, L. R. Quitadamo, and S. Seri, “Which physiological components are more suitable for visual ERP based brain-computer interface? A preliminary MEG/EEG study,” Brain Topogr., vol. 23, pp. 180–185, 2010.
907
[40] C. Wickens, A. Kramer, L. Vanasse, and E. Donchin, “Performance of concurrent tasks: A psychophysiological analysis of the reciprocity of information-processing resources,” Science, vol. 221, pp. 1080–1082, 1983. [41] G. F. Hagen, J. R. Gatherwright, B. A. Lopez, and J. Polich, “P3A from visual stimuli: Task difficulty effects,” Int. J. Psychophysiol., vol. 59, pp. 8–14, 2006. [42] C. C. Duncan-Johnson and E. Donchin, “On quantifying surprise: The variation of event-related potentials with subjective probability,” Psychophysiology, vol. 14, pp. 456–67, 1977. [43] Y. Arbel, R. Alqasemi, R. Dubey, and E. Donchin, “Adapting the P300brain computer interface (BCI) for the control of a wheelchair-mounted robotic ARM system,” Psychophysiology, vol. 44, pp. S82–3, 2007. [44] I. Iturrate, J. M. Antelis, A. Kuebler, and J. Minguez, “A noninvasive brain-actuated wheelchair based on a P300 neurophysiological protocol and automated navigation,” IEEE Trans. Robotics, vol. 25, no. 3, pp. 614–627, Jun. 2009. [45] B. Rebsamen, C. L. Teo, Q. Zeng, M. H. Ang, Jr., E. Burdet, C. Guan, H. Zhang, and C. Laugier, “Controlling a wheelchair indoors using thought,” IEEE Intell. Syst., vol. 22, no. 2, pp. 18–24, Mar./Apr. 2007. [46] E. Mugler, C. A. Ruf, S. Halder, M. Bensch, and A. Kubler, “Design and implementation of a P300-based brain-computer interface for controlling an internet browser,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 18, no. 6, pp. 599–609, Dec. 2010.
Siri-Maria Kamp received the Ph.D. degree from the University of South Florida, Tampa, FL, USA, in 2013. She is a post-doctoral researcher in the international research training group (IRTG) “Adaptive Minds” at Saarland University, Saarbrücken, Germany. Her current research utilizes event-related potentials to study the neural changes underlying age differences in human memory. The research project presented in this paper was conducted at the Cognitive Psychophysiology Laboratory at the University of South Florida, Tampa, FL, USA, where she conducted her pre-doctoral research.
Anthony R. Murphy is a graduate student in the Cognitive Psychophysiology Laboratory, University of South Florida, Tampa, FL, USA.
Emanuel Donchin received the Ph.D. degree from the University of California, Los Angeles, CA, USA, in 1965. Between 1965 and 1968, he was a Research Associate at the Department of Neurology, Stanford University and at the Neurobiology Branch at NASAAmes Research Center. In 1968, he joined the Department of Psychology at the University of Illinois, Urbana-Champaign (UIUC), IL, USA, as an Associate Professor. He remained at the UIUC until 2001, serving as Head of the department between 1980 and 1994. He is currently a Professor Emeritus at the UIUC and a Professor at the Department of Psychology, University of South Florida, serving as Chair from July 2001 until August 2008. His field of professional interest is cognitive psychophysiology.
