Accepted Manuscript Title: Brain computer interface with the P300 speller: usability for disabled people with amyotrophic lateral sclerosisUsability of BCI in ALS patients–> Author: Violaine guy Marie-H´el`ene soriani Mariane bruno Th´eodore papadopoulo Claude desnuelle Maureen clerc PII: DOI: Reference:
S1877-0657(17)30410-4 https://doi.org/doi:10.1016/j.rehab.2017.09.004 REHAB 1135
To appear in: Received date: Revised date: Accepted date:
29-5-2017 19-9-2017 19-9-2017
Please cite this article as: guy V, soriani M-H, bruno M, papadopoulo T, desnuelle C, clerc M, Brain computer interface with the P300 speller: usability for disabled people with amyotrophic lateral sclerosisUsability of BCI in ALS patients–>, Annals of Physical and Rehabilitation Medicine (2017), https://doi.org/10.1016/j.rehab.2017.09.004 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Brain computer interface with the P300 speller: usability for disabled people with amyotrophic lateral sclerosis
BRUNOa,c,
Théodore
ip t
Violaine GUYa,c, Marie-Hélène SORIANIa,c, Mariane b,c a,c PAPADOPOULO , Claude DESNUELLE , Maureen CLERC b,c a
cr
Centre de référence SLA, CHU de Nice - Hôpital Pasteur 2, 30 avenue Voie Romaine, 06001 Nice, France b
Université Côte d’Azur, France
an
c
us
Equipe projet Athena, Inria Sophia Antipolis-Méditerranée, 2004 Route des Lucioles, 06902 Sophia Antipolis, France
M
Corresponding author: Pr. Claude Desnuelle, Centre de Référence SLA, Pôle Neurosciences Cliniques, Hôpital Pasteur 2 – CHU de Nice, 30 Voie Romaine, CS 51069, 06001 NICE Cedex 1 Tel: +33 492 03 84 14, Fax: +33 492 03 83 26 e-mail:
[email protected]
d
Running title: Usability of BCI in ALS patients
Ac ce pt e
Brain computer interface with the P300 speller: usability for disabled people with amyotrophic lateral sclerosis Abstract
Objectives. Amyotrophic lateral sclerosis (ALS), a progressive neurodegenerative disease, restricts patients’ communication capacity a few years after onset. A proof-of-concept of brain– computer interface (BCI) has shown promise in ALS and “locked-in” patients, mostly in preclinical studies or with only a few patients, but performance was estimated not high enough to support adoption by people with physical limitation of speech. Here, we evaluated a visual BCI device in a clinical study to determine whether disabled people with multiple deficiencies related to ALS would be able to use BCI to communicate in a daily environment. Methods. After clinical evaluation of physical, cognitive and language capacities, 20 patients with ALS were included. The P300-speller BCI system consisted of electroencephalography acquisition connected to real-time processing software and separate keyboard-display control
Page 1 of 23
software. It was equipped with original features such as optimal stopping of flashes and word prediction. The study consisted of two 3-block sessions (copy spelling, free spelling and free use) with the system in several modes of operation to evaluate its usability in terms of effectiveness, efficiency and satisfaction.
ip t
Results. The system was effective in that all participants successfully achieved all spelling tasks and was efficient in that 65% of participants selected more than 95% of the correct symbols. The mean number of correct symbols selected per minute ranged from 3.6 (without word prediction)
cr
to 5.04 (with word prediction). Participants expressed satisfaction: the mean score was 8.7 on a
us
10-point visual analog scale assessing comfort, ease of use and utility. Patients quickly learned how to operate the system, which did not require much learning effort.
Conclusion. With its word prediction and optimal stopping of flashes, which improves
an
information transfer rate, the BCI system may be competitive with alternative communication systems such as eye trackers. Remaining requirements to improve the device for suitable
M
ergonomic use are in progress.
and alternative communication
Ac ce pt e
Introduction
d
Keywords: brain–computer interface, amyotrophic lateral sclerosis, P300 speller, augmentative
Amyotrophic lateral sclerosis (ALS) is a rare, rapidly progressive and devastating fatal neurodegenerative motor neuron disease affecting mostly older people; the disease is clinically characterized by a combination of lower and upper motor-neuron degeneration symptoms, with widespread distribution in bulbar, cervical, thoracic, and lumbosacral regions. It commonly starts with focal onset and spreads inexorably to other anatomical areas [1], leading in a few years to a very severe condition of muscle weakness including limbs, thoracic and bulbar functions. The disease incidence is about 2 to 3/100 000 in Western countries and prevalence about 4 to 6/100 000 (2). The creation of multidisciplinary ALS centres has greatly improved clinical care in the past decade and enhanced the survival and quality of life of patients.
Page 2 of 23
In 25% to 30% of affected individuals in the early stage of ALS, dysarthria occurs as a first or predominant sign [2]. It affects up to 70% of patients with limb-onset disease, who gradually lose the ability to communicate orally or by writing, as do all patients with bulbar-
ip t
onset disease when limbs eventually become affected. We have no hard evidence (only grade IV evidence) regarding speech and language management strategies for patients with ALS. The
cr
European Federation of the Neurological Societies–ALS guidelines [3] suggest, as a good
us
practice point, assessing communication every 3 to 6 months and the use of appropriate communication support systems. Various oral communication devices and non-verbal strategies
an
used consist of low-tech and high-tech augmentative and alternative communication (AAC) systems.
M
In this study, we investigated the usability of a brain-computer interface (BCI) system for typing text for people with ALS. The principle of a BCI is to interpret the electric signals of the
d
brain and translate them into commands. The feasibility of BCI communication has been
Ac ce pt e
reported in the past few years for individuals with ALS [4, 5, 6, 7, 8, 9 and 10]. We evaluated BCI communication in a population of 20 severely disabled patients followed in the ALS Centre of Nice University Hospital. The BCI system consisted of a virtual keyboard called the P300 speller [11, 12] equipped with optimal stopping of flashes and word prediction, which are expected to improve performance in terms of information transfer rate (ITR). All patients underwent 2 sessions including 3 operating modes of progressively increasing complexity to investigate the usability of the system in terms of effectiveness, efficiency and satisfaction as recommended by the International Organization for Standardization (ISO 9241-1998) [13, 14].
Materials and methods
Page 3 of 23
Population studied This was a non-invasive non-randomized prospective single-center study promoted by Nice University Hospital in accordance with the legal national regulations (approval by the local
ip t
ethics committee CPP Sud Méditerranée III (ref.2013.01.03 ter) and registered at ClinicalTrials.gov (NCT01897818).
cr
After detailed information about the P300 speller and study, 20 patients who were
us
routinely followed from disease onset in our centre were included. Participants meeting the inclusion criteria and not the non-inclusion criteria in Appendix A were included. Oral
an
communication disability was not considered critical to select patients because the goal was not to test BCI as an AAC in a target population but to test whether disabled people with multiple
M
deficiencies related to ALS would be able, in a non-specific environment, to use BCI to
Ac ce pt e
Experimental design
d
communicate according to the concept of usability defined previously.
At inclusion, patients underwent assessment that included general neurological examination, ALS Functional Rating Scale–Revised (FRS-R) [15] and modified Norris bulbar scale [16]. Global cognitive impairment was measured with the Mini Mental State Examination (MMSE) and the Frontal Assessment Battery. Specific psychometric tests were administered to evaluate executive functions (Wisconsin card sorting test, phonemic verbal fluency, Symbol Digit Modalities test and Trail Making Test A and B), attention (Symbol Digit Modalities Test) and language (French naming test DO80). Mood was evaluated by the State Trait Anxiety Inventory scale and depression by the Montgomery-Asberg Depression Rating Scale.
Page 4 of 23
The initial use of the BCI device took place within 2 weeks after the initial assessment. An occupational therapist set up the system and provided explanations to the patient. All stages of the study were performed in a standard room. Participants sat in a comfortable chair or in their
ip t
own wheelchair 90 cm from the LCD monitor. Each patient participated in two identical P300-speller sessions 2 to 4 weeks apart. Each
cr
session lasted 60 to 90 min and consisted of 3 blocks: copy spelling (block 1), free spelling
us
(block 2), and free use (block 3). At the beginning of each session, participants viewed a short audiovisual explanation about the subsequent experiments while they were wearing an EEG cap
an
(ANT Neuro WaveguardTM, with active electrodes), from which 12 electrodes (Fz, Cz, C3, C4, Pz, P3, P4, Oz, O1, O2, P7, P8, grounded to AFz, average reference) were connected to a Refa8
M
amplifier (256-Hz sampling rate). Conductive gel was applied to each electrode, with impedances < 10 kOhms. Ability to stare at a screen and execute the instructions to use the
d
device and evocation of a reliable visual P300 response were tested at that time. During
Ac ce pt e
calibration, participants successively focused on 10 letters, flashing 20 times. The recorded calibration data was used to train spatial filters and the linear discriminant analysis (LDA) classifier [17].
During the copy-spelling task (block 1 of each session), participants used the P300 speller to type two 10-letter words they overtly chose from a list (Appendix B, Table A1); while typing, participants were provided with cues (each keyboard letter to type briefly highlighted in blue) and feedback (letter highlighted in green if correctly selected by the P300 speller and in red otherwise). During typing, the word was progressively displayed under the keyboard, including possible typing errors. In this task, participants were instructed not to correct possible errors, so that the accuracy metrics were homogeneous.
Page 5 of 23
During the free-spelling task (block 2 of each session), participants used the P300 speller to type two 5-letter words (or one 10-letter word) they covertly chose from a list (Appendix B, Table A2); they did not receive any cues or feedback and were again instructed not to correct
ip t
possible errors. The free-use task (block 3) was optional, depending on the patient’s fatigue and
cr
motivation. Participants who chose to perform this block could use the P300 speller freely to
us
type words and sentences of their choice, including punctuation marks, and were also offered the possibility to use word prediction. In word prediction mode, participants could spell character-
an
by-character or select full words suggested on the screen. Error correction was possible by using a backspace key.
M
After each session, patients were asked to answer a questionnaire about satisfaction with
Ac ce pt e
P300-speller system
d
the usefulness, comfort, and ease of use of the system on a 10-point visual analog scale (VAS).
The P300-speller system was designed at Inria Sophia Antipolis [17]. It consisted of an EEG acquisition and real-time processing software using the OpenViBE platform [18] and a separate keyboard-display control software, both software running on a Windows 7 laptop with a One Intel Quad Core processor i7-3740QM (2.70 GHz, 6 MB cache). The laptop monitor was used to monitor the EEG signal quality, and a separate 22” 1680x1050 LCD monitor was used to display the keyboard. The keyboard had 43 symbols including punctuation marks and a backspace key. A choice of AZERTY (French layout) or ABCDE layout was available. As for all P300-speller keyboards, characters flashed in order to elicit a P300 response for the attended character. Here, the flashing consisted of briefly covering the character with a
Page 6 of 23
“smiley face”, as this has been shown to elicit stronger P300 responses than simply highlighting the character [19]. The flash duration was 116.7 ms, and the inter-stimulus interval was 183.3 ms. Instead of row-column flashing, characters were flashed in pseudo-random groups, designed
ip t
to minimize the consecutive flashing of the same characters, and the simultaneous flashing of neighbor characters [20].
cr
The timing of each flash was digitally conveyed to the EEG amplifier via a hardware
us
connection. The stream of EEG data was segmented into epochs starting at the flash onset and lasting 500 ms. A real-time signal processing pipeline allowed to detect whether each EEG
an
epoch contained a P300 response (target epoch) or not (non-target epoch). For this, a spatial filter was used (X-DAWN) followed by an LDA classifier [21]. During calibration and copy-spelling
M
modes only, the target and non-target epochs were averaged across trials and displayed as 2 curves on the laptop monitor, because a sanity check of target and non-target epochs had
d
different averaged time courses. The channels selected for display were Pz during calibration and
Ac ce pt e
the output of the spatial filter during copy spelling. Figure 1 depicts the P300 speller system. In our system, the number of flashes per selection varied across sessions, contrary to previous P300-speller studies in ALS, in which the number of flashes per symbol was fixed [22] or adapted to the patient after calibration but fixed across the BCI sessions [8, 23 and 24].
The system had an early stopping procedure [25] based on symbol-based evidence accumulation, in which the keyboard stopped flashing and the selected character was displayed as soon as its evidence exceeded all others by at least 90% (100% evidence indicating that according to its observations, the classifier has 100% confidence that this character is the one being attended).
Page 7 of 23
The P300-speller application also had word prediction ability based on the Presage library (http://presage.sourceforge.net/). This feature allowed for word completion and next word prediction by displaying a list of the 10 most probable words on the right side of the keyboard
ip t
(Fig. 1c). The same flashing principle was used for full words as for characters, to allow their selection via the P300 detection: “smiley faces” were briefly overlayed on each word. Each full
cr
word also included a following blank space.
us
Statistical analysis
Assuming a 75%-point estimate for efficiency and effectiveness scores, for the cohort of more
an
than 250 ALS patients routinely followed in the ALS Center of Nice University Hospital, we deemed it sufficient to include 20 patients in this study.
M
Effectiveness was calculated as the proportion of patients who completed the copy and free spelling (blocks 1 and 2) in both sessions, whatever the performance. The 95% confidence
d
interval (CI) of this score was calculated by the Poisson distribution (fitted to rare events).
Ac ce pt e
Efficiency was assessed with data from the second session. The Wilcoxon signed rank test was used for paired comparisons. The ITR was calculated with the Wolpaw formula [26]. Results
Among 21 participants selected, results for 20 (10 males) were analyzed: one was fatigued during the initial test. Table 1 gives a description of the participants. The mean age was 62, mean ALS FRS-R score 25/48 (range 8–37), and mean modified Norris bulbar score 22/39 (range 1– 39). The mean elapsed time since symptoms onset was 6 years (range 1–15). All participants had normal gaze control on clinical examination. At the time of the study, 8 patients were using AAC: an alphabet board or a voice synthesizer with a keyboard or linked to a computer. Five patients had experienced an eye-tracker AAC device before the study, but it was considered
Page 8 of 23
inappropriate because of lack of ability to maintain a correct position. For 7 participants, all neuropsychological test results were in the normal ranges. The remaining 13 were unable to perform the entire neuropsychological evaluation because of severe dysarthria and/or writing
ip t
disability. Despite this, for 10 of these 13 participants, the uncompleted results of the performed tests were assessed as normal. One patient showed altered mental flexibility according to the
cr
Trail Making Test score, combined with impaired working memory (MMSE), which suggested
us
dysexecutive syndrome. The 17 participants who underwent the naming test (DO80) showed no semantic memory impairment. According to the tests described, no chronic depression was
an
evidenced.
During the sessions, the mean installation time for the system was 13 min (range 6–30,
phase time was 7 min (range 6–9).
M
variability in terms of the time needed to achieve correct impedance), and the mean calibration
d
In assessing usability, all 20 participants succeeded in completing blocks 1 and 2 for both
Ac ce pt e
sessions, for an effectiveness score of 100% (95% CI 82–100). No significant change was observed between the 2 sessions, 2 weeks apart (p=0.27 for block 1, and p=0.74 for block 2), so the system was mastered during the first session. In block 1 (“copy spelling”), more than 90% of symbols were correctly selected by all participants (Fig. 2). Table 2 gives individual accuracies for each of the 3 blocks.
Regarding efficiency, the 20 participants showed a mean of 10 or fewer flashes per selection and for 14, a mean of 5 or fewer (Fig. 3). The spelling accuracy was lower in block 2 (“free-spelling”) than block 1 (“copy spelling”) probably because in block 2, the participants had to locate on their own the letter/symbol position on the keyboard, no longer guided by the cue received during copy spelling. However, 95% of the participants still achieved > 75% correct
Page 9 of 23
symbol selection (Fig. 4), and 65% of the participants, > 95% correct symbol selection. Two participants showed poor free spelling accuracy (50% or below) (Table 2): for patient 9, because of technical problems with the display, which was not flashing correctly, the system had to be
ip t
reset and recalibrated after block 2, and for patient 16, who became confused when looking for the letters on the keyboard screen.
cr
All 20 participants chose to perform the optional block 3 (“free use”), in which they
us
could type any sentence of their choice. This block was particularly motivating because they could experience that the P300 speller indeed elicited free communication. The mean number of
an
correct symbols per minute was 5.04 (SD 1.66) and 3.58 (SD 1.64) with and without word prediction. Among the 20 participants, 12 typed the same text with and without word prediction
M
(see Appendix C). The median improvement with word prediction for these 12 patients was 1.18 (SD 0.84) additional correct symbols per minute (Wilcoxon signed-rank test p=0.016). Individual
d
improvement per participant is reported in Figure 5. Note that the two participants with low
Ac ce pt e
spelling accuracy in block 2 (patients 9 and 16) nevertheless showed 100% spelling accuracy in block 3. As an explanation for this improvement, for patient 9, the system was reset, and patient 16 figured out how to find the letters or full words to select on the screen. Patient satisfaction was rated on a VAS for 3 criteria. The mean score for comfort of use, from very uncomfortable to very comfortable (0 to 10) was 8.1 (SD 1.4); ease of use, from very difficult to very easy (0 to 10), 8.4 (SD 1.0); and appreciation of the utility of the system, from totally useless to very useful (0 to 10), 9.75 (SD 0.5). We could not compare the system to use of other high-tech AAC devices because no patient was routinely equipped with such a tool.
Discussion
Page 10 of 23
Altogether, the results of our study demonstrate the usability of the P300 speller by disabled patients with ALS. The effectiveness of the system was 100% because all patients accomplished blocks 1 and 2 in the 2 sessions; for efficiency, 95% of participants achieved up to 75% correct
ip t
symbol selection (up to 95% for 65% of participants), and the mean satisfaction score was 8.7/10 for usefulness, ease of use and comfort of use.
cr
The study was conducted with a relatively large group of 20 patients with ALS. Most
us
existing studies involved only a few patients or similar cohorts of 20 to 25 patients [5, 23, 24 and 27]. The study by Huggins et al. [28], with 61 patients, was only a survey of the opinion of
an
people with ALS regarding the BCI design, which did not include testing the system. As reported in a meta-analysis [29], few studies have tested BCI in semi-realistic settings. The present study
M
departs from a laboratory setting because the system was set up by an occupational therapist with no prior experience in EEG or BCI, in a regular office area of the hospital facilities. Moreover,
d
the patients used a realistic keyboard, which allowed for free spelling with use of a backspace
Ac ce pt e
key to correct errors, and word prediction to accelerate throughput. Behavioral performances have been reported to affect P300 detection [23, 30], which could be a source of concern for using the P300 speller, especially because for some patients, speech or writing deficiencies prevented a complete cognitive evaluation. However, a P300 wave was present in all patients, and the system was therefore usable for all of them. BCI was proposed as alternative cognitive assessment tool for non-verbal patients [31]. To favour high spelling throughput, our system included optimal stopping of flashes and word prediction. Optimal stopping of flashes has been proposed previously in a research context, and in a clinical context, it has been studied offline, from data for which the number of flashes was fixed beforehand [32]. In this study, investigating optimal stopping of flashes in the core of
Page 11 of 23
the system, the optimal stopping procedure increased the throughput for all patients. In some other studies, an adaptation stage was used to select the number of flashes. For instance, in the protocol used by McCane et al. [23], symbols flashed between 10 and 30 times before selection
ip t
(which they estimated offline could have been reduced to between 4 and 22 without detriment to the accuracy). Pasqualotto et al. [24] set the number of flashes to between 5 and 8. Our online
cr
optimal stopping procedure was able to determine the optimal number of flashes online, at each
us
individual selection, which helps increase motivation [33]. The average number of flashes per selection ranged from 3 to 10 among our participants (Fig. 3).
an
All participants decided to try word prediction in block 3, and for the 12 who typed out the same text with and without word prediction, the spelling rate (in symbols per minute) was
M
improved with word prediction. For a few patients, word prediction decreased the spelling rate, which could be attributed to their difficulty in scanning the proposed words fast enough and in
d
switching between typing strategies, as can be observed in non-BCI settings with word
Ac ce pt e
prediction [34]. Globally, word prediction brings a benefit but may require some extra training to be fully effective. However, the word prediction engine can be trained, and its dictionary and grammar rules updated with the users' writing, which would increase performance. The participants were all able to operate the system under conditions that compared favorably with the literature.
However, our experimental study raised a number of technical and practical issues to be addressed for regular daily use of the system. Our system was designed to be set up by caregivers (and not by EEG or computer science experts), so it should be further simplified to allow anyone to set it up easily [35]. Especially, the installation time — 13 min, on average — should be reduced to < 5 min to be practical. With the current electrode technology, reducing the number of
Page 12 of 23
electrodes (we used 12 + 1 ground) would contribute to this goal. Relevant electrode positions should be selected to provide both good signals and good comfort even for patients who need a support for holding their head [8, 36]. Alternatively, use of gel-free electrodes would both
ip t
improve setup time and be viable for long-term daily use, seven days a week. Hence, comfortable and aesthetic headsets with gel-free electrodes should be designed. Wires were used
cr
to connect the amplifier to the EEG cap and to a computer running the P300-speller software.
us
For a daily ambulatory use, the EEG system should be lightweight and portable and wirelessly connected to the computer.
an
A comparison by Pasqualotto et al. [24] of BCI and an eye-tracker AAC in ALS favoured the eye-tracker because the BCI demanded more effort and was more time-consuming. Their
M
reported ITR was 8.67 and 12.87 bits/min for BCI and the eye-tracker. Our study produced a mean ITR of 17.72 bits/min in free use without word prediction (Fig. 5, table on the right). The
d
ITR formula is impossible to apply with word prediction, where letters or entire words of
Ac ce pt e
different lengths can be selected. Note that our keyboard has fewer symbols (43 symbols vs 49 symbols in [24]), which mechanically lowers the ITR. Despite its ergonomic limitations, BCI has the advantage of not imposing strict positioning in front of the screen as with eye-tracker devices, a point of great importance in the context of severely disabled patients with ALS. Regarding the P300 speller, each new user session started with a calibration stage lasting about 7 min. This calibration should be much faster or should occur in the background, without needing the user's cooperation. There are many ways to further adapt the system to the user and improve the speed (natural language statistics, online error correction). The tested P300 speller is a standalone keyboard. To be fully useful, it should interact with other applications (Web browsing, gaming, robotics or neuroprosthetics), taking into account the ergonomic constraints
Page 13 of 23
involved in the P300-speller paradigm [13,14]. By replacing letters with pictograms, the P300speller principle can be used for environmental control. The keyboard could also include features such as a “pause” and voice synthesis.
ip t
Future development will include addressing the above software-related issues, and developing a commercial low-cost EEG headset.
cr
Conclusion
us
In the present study, involving 20 patients with ALS, the P300-speller BCI was effective. It may be considered a reliable input method allowing patients with ALS to communicate with their
an
environment. Compared to previous studies, the key features of our system, such as word prediction and optimal stopping of flashes, allowed for significant gains in efficiency; hence,
M
visual P300-based systems are a possible communication alternative for disabled patients, provided that some practical aspects of the EEG system setup are simplified to improve the
d
satisfaction in use. Additional developments can be made at the hardware and software level,
Ac ce pt e
such as in the electrode detection and wire-free transmission of signals, to further enhance performance and usability.
Acknowledgements This work received financial support from the French ALS Association (ARSLA), ANR COADAPT (ANR09 EMER002) and assistance from the Research Department of Nice University Hospital. The authors thank all the patients who participated in the study and their caregivers for their involvement. Helpful advice from E. Maby and J. Mattout is acknowledged, as is technical help from D. Devlaminck and L. Mahé. The authors thank A. Metelkina and C. Sakarovitch for statistical analysis and the staff of the Nice ALS Centre for technical help in clinical evaluations. Conflict of interest. None declared. Legends Figure 1: P300-speller system overview. In our P300-speller system (a), the user was provided with a keyboard displayed on a monitor connected to a single laptop running EEG signal acquisition, processing, classification and keyboard management. The laptop monitor was used
Page 14 of 23
to control the EEG signals and (b) the separability of target and non-target evoked responses. (c) The keyboard displayed to the user was equipped with a backspace character for correcting mistakes in free use mode. (c) Example of a sentence typed during free use.
ip t
Figure 2: Copy-spelling efficiency. Spelling accuracy histogram for block 1 (“copy spelling”). In this task the participants typed two 10-letter words out of a list (Appendix B), and were cued on the letters to select. The proportion of correct letters was above 90% for all participants, and
us
cr
reached 100% for 60% of participants.
Figure 3: Copy-spelling efficiency. Number of flashes per selection during copy spelling (block 1) determined online by an optimal stopping procedure. All participants experienced between 3
an
and 10 flashes, and 15 of 20 experienced 5 flashes or less per selection.
M
Figure 4: Free-spelling efficiency. Spelling accuracy in free spelling (block 2). In this task the participants typed four 5-letter words they covertly chose from a list, without any cue on the letters to select. For 90% of the participants, the proportion of correct letters was over 75% and
Ac ce pt e
d
over 90% for 65% of the participants.
Figure 5: Free use effectiveness and efficiency. Spelling rate in block 3 (“free use”), for 12 participants who typed the same text with and without word prediction displaying the number of correct symbols per minute and showing the effect of word prediction. The Information Transfer Rate (ITR), reported for individual subjects, without word prediction, achieved a mean value of 17.72 bits/min. References
[1] Pradat PF, Kabashi E, Desnuelle C. Deciphering spreading mechanisms in amyotrophic lateral sclerosis: clinical evidence and potential molecular processes. Curr Opin Neurol. 2015;28:455-61 [2] Traynor BJ, Codd MB, Corr B, Forde C, Frost E, Hardiman O. Clinical features of amyotrophic lateral sclerosis according to the El Escorial and Airlie House diagnostic criteria: a population-based study. Arch Neurol. 2000; 57: 1171-6. [3] Andersen PM, Borasio GD, Dengler R, Hardiman O, Kollewe K, Leigh PN, et al., EFNS task force on management of amyotrophic lateral sclerosis: guidelines for diagnosing and clinical care of patients and relatives. Eur J Neurol. 2005; 12:921-38.
Page 15 of 23
Ac ce pt e
d
M
an
us
cr
ip t
[4] Nijboer F, Sellers EW, Mellinger J, Jordan MA, Matuz T, Furdea A, et al. A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clin Neurophysiol. 2008;119:1909-16 [5] Mak JN, McFarland DJ, Vaughan TM, McCane LM, Tsui PZ, Zeitlin DJ,et al. EEG correlates of P300-based brain–computer interface (BCI) performance in people with amyotrophic lateral sclerosis, J Neural Eng. 2012; 9:026014 [6] Cipresso P, Carelli L, Solca F, Meazzi D, Meriggi P, Poletti B et al. The use of P300-based BCIs in amyotrophic lateral sclerosis: from augmentative and alternative communication to cognitive assessment. Brain Behav. 2012;4:479-98. [7] Silvoni S, Cavinato M, Volpato C, Ruf CA, Birbaumer N, Piccione F. Amyotrophic lateral sclerosis progression and stability of brain computer interface communication. Amyotroph Lateral Scler Frontotemporal Degener. 2013;5-6:390-6 [8] McCane LM, Heckman SM, McFarland DJ, Townsend G, Mak JN, Sellers EW, et al. P300based brain-computer interface (BCI) event-related potentials (ERPs): People with amyotrophic lateral sclerosis (ALS) vs. age-matched controls. Clin Neurophysiol. 2015:126: 2124-2131. [9] Geronimo A, Stephens HE, Schiff SJ, Simmons Z. Acceptance of brain-computer interfaces in amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener. 2015;34:258-64 [10] Riccio A, Holz EM, Arico P, Leotta F, Aloise F, Desideri L et al. Hybrid P300-based braincomputer interface to improve usability for people with severe motor disability: electromyographic signals for error correction during a spelling task. Arch Phys Med Rehabil. 2015;96S3:S54-61 [11] Farwell LA, Donchin E. Talking off the top of your head: A mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin Neurophysiol. 1988;70:510-523, [12] Clerc M, Mattout J, Devlaminck D, Papadopoulo T, Guy V, Desnuelle C Verbal communication through Brain Computer Interfaces. Interspeech. 2013. 14th Annual Conference of the International Speech Communication Association. [13] Schettini F, Riccio A, Simione L, Liberati G, Caruso M, Frasca V et al. Assistive device with conventional, alternative and Brain-Computer Interface inputs to enhance interaction with the environment for people with amyotrophic lateral sclerosis: a feasibility and usability study. Arch Phys Med Rehab 2015; 96(3 Suppl 1): S46-53. [14] Zickler C, Riccio A, Leotta F, Hillian-Tress S, Halder S, Holz E, et al.. A Brain-Computer Interface as Input Channel for a Standard Assistive Technology Interface. Clin EEG Neurosci, 2011;42:236-244. [15] Cedarbaum JM, Stambler N, Malta E, Fuller C, Hilt D, Thurmond B et al. .The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function. BDNF ALS Study Group (Phase III). J Neurol Sci. 1999;169:13-21. [16] Norris FH, Jr, Calenchini PR, Fallat RJ, Panchari S, Jewett B The administration of guanidine in amyotrophic lateral sclerosis. Neurology 1974;24:721-8. [17] Thomas E, Daucé E, Devlaminck D et al, CoAdapt P300 speller: optimized flashing sequences and online learning, 6th International Brain Computer Interface Conference, September 2014. [18] Renard Y, Lotte F, Gibert G, Congedo M, Maby E, Delannoy V, et al. OpenViBE: an opensource software platform to design, test and use brain-computer interfaces in real and virtual environments, Presence: teleoperators and virtual environments, MIT Press. 2010;19 :1:35-53.
Page 16 of 23
Ac ce pt e
d
M
an
us
cr
ip t
[19] Jin J, Allison BZ, Kofmann T, Kübler A, Zhang Y, Wang X et al. The Changing Face of P300 BCIs: A comparison of stimulus changes in a P300 BCI involving faces, emotion and movement. PloS One. 2012;7:e49688. [20] Thomas E, Clerc M, Carpentier A, Munos R. Optimizing P300-speller sequences by RIPping groups apart. International IEEE/EMBS Conference on Neural Engineering. 2013. [21] Rivet B, Souloumiac A, Attina V, Gibert, G. xDAWN algorithm to enhance evoked potentials: applications to Brain-Computer Interface. IEEE Trans Biomed Engin. 2009;56:203543. [22] Geronimo A, Simmons Z, Schiff SJ. Performance predictors of brain-computer interfaces in patients with amyotrophic lateral sclerosis. J Neural Eng. 2016;13:026002 [23] McCane L, Sellers EW, McFarland DJ, Mak JN, Carmack CS, Zeltin D, et al. Braincomputer interface (BCI) evaluation in people with amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener. 2014;15:207-15. [24] Pasqualotto E, Matuz T, Federici S, Ruf CA, Bartl M, Belardinelli MO et al. Usability and workload of access technology for people with severe motor impairment: A comparison of braincomputer interfacing and eye tracking. Neurorehabil Neural Repair. 2015;10:950-7. [25] Mainsah BO, Collins LM, Colwell KA, Sellers EW, Ryan DB, Caves K et al. Increasing BCI rates with dynamic stopping towards more practical use: an ALS study. J Neurol Eng. 2015;12:016013. [26] Wolpaw JR, Ramoser H, McFarland D, Pfurtscheller G. EEG-based communication: improved accuracy by response verification. IEEE Trans Rehabil Eng.1998;3:326-333. [27] Silvoni S, Volpato C, Cavinato M, Merchetti M, Priftis K, Merico A et al. P300-based brain-computer interface communication: Evaluation and follow-up in amyotrophic lateral sclerosis. Front Neurosci. 2009;3:60-72 [28] Huggins JE, Wren PA, Gruis KL. What would brain-computer interface users want? Opinions priorities of potential users with amyotrophic lateral sclerosis. Amyotroph Lateral Scler. 2011;5:318-24 [29] Marchetti M, Priftis K. Brain-Computer interfaces in amyotrophic lateral sclerosis: a metanalysis. Clin Neurophysiol 2015;126:1255-1263. [30] Seer C, Fürkotter S, Vogts MB, Lange F, Abdulla S, Dengler R et al. Executive dysfunctions and event-related brain potential in patients with amyotrophic lateral sclerosis. Front Aging Neurosci. 2015;7:225-42. [31] Poletti B, Carelli L, Solca F, Lafronza A, Pedroli E, Faini A et al. Cognitive assessment in amyotrophic lateral sclerosis by means of P300-brain computer interface: a preliminary study. Amyotroph lateral Scler Frontotemporal Degener. 2016;12:1-9. [32] Aloise F, Schettini F, Aricò P, Salinari S, Guger C, Rinsma J et al. Asynchronous P300based Brain-Computer Interface to control a virtual environment: initial tests on end users. Clin EEG Neurosci., 2011;4:219-24. [33] Mattout J, Perrin M, Bertrand O, Maby E. Improving BCI performance through coadaptation: applications to the P300-speller. Ann Phys Rehabil Med, 2015;58:23-28. [34] Koster HH, Levine S. Learning and performance of able-bodied individuals using scanning systems with and without word prediction. Assistive Technology. 1994;6:42-53. [35] Liberati G, Pizzimenti A, Simione L, Riccio A, Schettini F et al. Developing brain-computer interfaces from a user-centered perspective: Assessing the needs of persons with amyotrophic lateral sclerosis, caregivers, and professionals. Applied Ergonomics. 2015;50:139-146.
Page 17 of 23
[36] Krusienski DJ, Sellers EW, McFarland DJ, Vaughan TM, Wolpaw JR. Toward enhanced P300 speller performance. J. Neurosci. Methods. 2008;167:15-21.
VS N N N N N VS VS N AB VS N N N VS N N N VS VS
ip t
Y Y N Y N N N N N Y N Y N N Y N N N N N
cr
4 23 39 22 35 23 18 13 33 8 1 36 29 29 16 21 39 34 20 17
Trunk Alternative hypotonia communication device
us
12 16 27 21 26 22 35 11 27 8 28 31 34 28 21 32 30 25 29 37
an
4 9 9 8 9 15 4 7 5 11 2 4 4 8 13 1 3 2 2 1
El Escorial-R diagnostic criteria at inclusion Definite Definite Probable Definte Definite Definite Definite Definite Definite Definite Definite Definite Probable Probable Definite Definite Probable Definite Definite Definite
M
40 63 63 59 42 55 62 37 71 54 65 62 53 89 50 90 66 53 74 66
Ac ce pt e
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
ALS FRS-R Norris score/ score/ 48 39
d
Patient Age Disease no. (years) duration (years)
Table 1. Demographic and clinical characteristics of participants with amyotrophic lateral sclerosis (ALS). Trunk hypotonia: present = Y, absent = N. Augmentative and alternative communication (AAC) device: AB = alphabet board, N = no AAC device used, VS = voice synthesizer. FRS-R, Functional Rating Scale–Revised
Page 18 of 23
Block 1 (copy spelling)
Block 2 (free spelling)
Block 3 (free use — no word prediction)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
100% 95% 95% 50% 95% 90% 100% 100% 100% 100% 100% 100% 100% 90% 95% 95% 95% 100% 100% 100%
80% 100% 95% 100% 100% 75% 95% 75% 45% 100% 90% 90% 80% 95% 90% 50% 100% 100% 85% 100%
75.8% 74.5% 59.3% 94.6% 90.9% 56.5% 93.1% 63.2% 100.0% 100.0% 72.1% 84.2% 100.0% 85.7% 100.0% 94.6% 77.8% 45.5% 85.4% 51.3%
Table 2. Participant accuracy (correct symbol selection percentage) in the 3 block sessions.
Ac ce pt e
d
M
an
us
cr
ip t
Patient no.
Page 19 of 23
cr us
Usability of BCI in ALS patients, Guy V et al
Ac
ce
pt
ed
M
an
Figure 1
Page 20 of 23
Usability of BCI in ALS patients, Guy V et al
Ac ce pt e
d
M
an
us
cr
ip t
Figure 2
Page 21 of 23
Usability of BCI in ALS patients, Guy V et al
Ac ce pt e
d
M
an
us
cr
ip t
Figure 4
Page 22 of 23
Usability of BCI in ALS patients
Ac ce pt e
d
M
an
us
cr
ip t
Figure 5
Page 23 of 23
