Usability of EEG Systems: User Experience Study Krzysztof Izdebski
Anderson Souza Oliveira
Bryan R. Schlink
Institute of Cognitive Science University of Osnabrück 49076 Osnabrück, Germany
Human Neuromechanics Laboratory, School of Kinesiology, University of Michigan,
Human Neuromechanics Laboratory, School of Kinesiology, University of Michigan,
[email protected]
[email protected]
[email protected]
Petr Legkov Institute of Cognitive Science University of Osnabrück 49076 Osnabrück, Germany
[email protected]
Silke Kärcher
W. David Hairston
Institute of Cognitive Science University of Osnabrück 49076 Osnabrück, Germany
US Army Research Laboratory, Human Research and Engineering Directorate, Aberdeen Proving Ground, MD, USA
[email protected]
[email protected]
Daniel P. Ferris
Peter König
Human Neuromechanics Laboratory, School of Kinesiology, University of Michigan, Ann Arbor, MI, USA
Institute of Cognitive Science University of Osnabrück 49076 Osnabrück, Germany
[email protected]
[email protected]
ABSTRACT In recent years there was a change in EEG experimental designs from simple behavior in the lab to complex behavior outside. That change required also an adjustment of EEG systems – from being static and sensitive to mobile and noise-resistant. The rapid technological development has to balance performance (e.g. number of channels, low impedance contact) with usability (e.g. comfort for the participant, contact pressure, wet/dry electrodes) and mobility (e.g. wiring, weight). This has led to wide variety of designs which differ widely in properties. Here we compare 7 EEG systems with respect to the participant’s user experience. Results demonstrate that from perspective of user experience of participants, mobile wet system (Cwet) had the highest score.
CCS Concepts • Development of standards; 500 • Intelligent biomedical devices for assistive environments; 100 • Method and tools for biosignal data analysis; 300 • Non-invasive sensing devices for biosignal data acquisition; 100 • Usability and HCI sues, multimodal interfaces; 300.
Keywords Usability; EEG; user experience; UX; interferences; internal validity; comparability; data quality Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]. PETRA '16, June 29-July 01, 2016, Corfu Island, Greece Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM 978-1-4503-4337-4/16/06…$15.00 DOI: http://dx.doi.org/10.1145/2910674.2910714
Acknowledgements This project is funded Cognition and Neuroergonomics / Collaborative Technology Alliance #W911NF-10-2-0022.
1.INTRODUCTION Performing EEG experiments where subjects are allowed to move and perform complex behavior is a challenging task [4]. Experimental paradigms involving mobile EEG, where subjects move with their head or whole body, typically induce many different types of motion artifacts. However, with the increase of computational power, more advanced signal processing and artifact rejection in fully mobile EEG recording is possible today [5, 19]. Furthermore, technological advances of hardware fostered the development of mobile EEG systems. Thus, EEG studies can take a more active form and we are now able to explore behavior in motion, more closely resembling real life conditions. A prominent example for this change is the MoBI lab [13]: “a wearable mobile brain/body imaging system that continuously capture the wearer’s high-density electrical brain and muscle signals, three-dimensional body movements, audiovisual scene and point of regard, plus new data-driven analysis methods to model their interrelationships”. Other ambitious approaches aim to measure phenomena such as fatigue during real world driving [15] or EEG activity during sport [21]. These new types of active approaches further fuel the advancement of all aspects of EEG recording systems, but also raise several questions. Over the last few years, the focus of development has been mostly on features connected to signal quality, e.g. wireless amplifiers. Even though there are several studies measuring comparability of effects between systems [7, 22], there is no comprehensive study looking at features influencing EEG data quality beyond those quantifiable solely from EEG signals. Specifically, the influence of participants’ subjective user experience (UX) is rarely addressed and has been largely overlooked. Yet numerous studies classify level of fatigue from EEG data (e.g. [1, 2, 8]) and emotions (e.g. [9]), both of which can change performance in a given task (e.g. [8,
11]). Thus, if participants experience fatigue during the experiment, such as from use of a heavy system, or if the emotional state of the participant changes due to discomfort, the EEG results may be influenced. Here, using data recorded from seven EEG systems in two similar experiments, we compare the user experience of the participants with each system and discuss possible interferences due to phenomena facilitated or even caused by the system design.
2.METHODS This study uses data recorded during two experiments performed at University of Michigan (UM) and University of Osnabrück (UOS) with close collaboration on UX factors measurement methodology. Both paradigms have been designed and performed as part of a project geared towards development of standardized benchmarking methodologies of EEG systems [18].
2.1.Systems Seven EEG systems were used, ranging from wearable mass market technology to those developed specifically for research purposes. Table 1. Summary of EEG systems; gN – gNautilus; Emo – Emotiv; Cdry – Cognionics Dry; aC – actiCap; BSM – Biosemi; Cwet – Cognionics Wet; target use
Electrode type
Availability
gN
mobile scientific
dry
in stock
Emo
mass market
dry
in stock
Cdry
mobile scientific
dry
in stock
ANT
ambulatory scientific
wet
upgraded
aC
stationary scientific
wet
in stock
BSM
stationary scientific
wet
in stock
Cwet
mobile scientific
wet
in stock
Name
Picture
dry, which do not need special conductive gel, since they rely on direct connection of electrode to the skull.
2.2. Differences between systems There are a number of interesting features that differed between systems used in this study that have to be considered from a data quality perspective. First, the number of electrodes ranges from 14 (Emo), through 32 (gN) and 64 (Cdry, aC, Cwet), up to 128 (ANT, BSM) channels. While MoBI lab approaches usually speak about high-density recording[13, 17], Lau et al [10] shows that a minimum of 35 channels is required for mobile brain imagining with EEG. Another interesting feature to consider is sampling rate. The higher the sampling rate, the more detailed the recorded signal, meaning that ERPs with smaller time difference from each other can be analyzed. Moreover, the sampling rate determines the Nyquist frequency, the highest usable frequency in the recording. An inadequate sampling rate causes distortion of the input waveforms. In order to avoid this, a low pass has to be set at (or below) the Nyquist frequency to block higher frequencies. Otherwise, these lead to aliasing and erroneous interpretations. There are few paradigms that require higher sampling rate than 1000 Hz. The sampling rate for the systems used in this study range from as low as 128 Hz (Emo), through 512 (gN, Cry, BSM, Cwet) and 1000 (aC), up to 2048 Hz (ANT). Last, but not least, Analog to Digital Conversion (ADC) resolution describes the discrimination accuracy of voltage when the analog signal is digitized. The higher ADC resolution, the more precisely the analog input is represented in the recording. Because there is a lot of power in the low frequency ranges of EEG, there is general interest in using a wide dynamic range, thus requiring a larger ADC, thereby decreasing the distance between levels, increasing resolution. It should be noted that there is a direct link between ADC rate and power consumption [6, 23], as a result, smaller, consumer-oriented systems tend to use lower bit rates [7]. Among systems used in this study, 4 have ADC resolution close or equal to the optimal value: gN (24 bits/sec), Cdry (24 bits/sec), Cwet (24 bits/sec), and ANT (22 bits/sec).
2.3.Setup
Considering the state of modern technology and use cases of EEG, we categorized systems by 2 features: Target use – what is the system designed for: (1) mobile scientific refers to real world paradigms done outside of the lab, such as fatigue during driving [16]; (2) ambulatory scientific refers to active paradigms, but performed in lab environment, such as walking on treadmill [8]; (3) stationary scientific refers to systems designed to be used as seated setup with a limited behavioral response; (4) mass market, aiming to be available not only in scientific setups but also for private use; electrode type – what is the type of electrodes used – either (1) wet, requiring gel to provide good quality of connection to skull, or (2)
This study compares the above mentioned 7 EEG systems used in 2 similar experiments. In the study performed at University of Osnabrück (UOS), there were 4 participants (22-25 y, 1f), who performed 3 sessions with each of 4 EEG systems (gN, Emo, ANT, aC). Experimental recording took between 2 and 3 hours. In the study performed at University of Michigan (UM), there were 9 participants (21-36 y, 3f), who did 1 session with each of the 3 EEG systems (BSM, Cdry, Cwet). Experimental recording took between 1 and 2 hours. All participants were informed about the procedure and purpose of the study and had signed an informed consent. Experimental procedures conformed to the Declaration of Helsinski and national guidelines. The comparison described here focuses on factors and features related to the EEG systems and not related to the experimental stimuli. Experimental procedure and paradigm will be published elsewhere ([18], Melnik et. al, in preparation), Details are provided on request.
2.4.Measurements After each session participants filled out a questionnaire measuring their UX and possible interferences (such as sleep duration), which was designed as part of standardized benchmarking methodologies of EEG systems at UOS and UM. Analyses here focuses on the subjects’ ratings of discomfort, cap fit and induced emotions. All questions used a common 5-point Likert scale rating from worst (1) to best (5).
3. RESULTS
Emo
2
4
1
3
2
4
2.67
3.1.1.Discomfort
Cdry
2
4
1.5
3
4
4
3.08
ANT
2
4
3
4
2
2
2.83
aC
3
4
5
4
3
2
3.50
BSM
4
4
4
4
4
4
4.00
Cwet
5
5
5
4
4
5
4.67
While the questionnaire contained more questions asking about discomfort, three were specific to EEG systems. First, participants evaluated their overall general discomfort. Second, they were asked to consider the unpleasant feeling under the cap caused by sweating and hotness. Third, subjects evaluated the discomfort caused by pinching or pressure of electrodes onto the skin. It is worth noting that even though participants do not report unpleasant feeling under cap nor high discomfort due to pressure of electrodes, ANT and aC still induced general discomfort.
3.1.2.Cap fit Cap fit represents the participant’s perspective as to how well the EEG cap fit their head. This question relates to the flexibility of caps – some had few sizes (ANT, BSM, aC), other had adjustment methods (Cdry), and the rest had only one size (gN, Emo, Cwet). Even though BSM did have multiple cap sizes (so that one could choose which was the closest fit to their head), participants still rated it very low. Systems that had neither adjustment method nor multiple sizes also received low scores.
3.1.3.Negative – positive axis (n/p) Negative–positive axis describes the mood of participants at the end of the experiment. The more negative a participant was, the lower was the score for this feature. Emo and gN, 2 out of the 3 dry systems, and ANT, a wet system, show low scores, meaning participants felt negative at the end of experiment. aC showed a little higher score. Cdry, BSM and Cwet shown high score, meaning participants felt positive at the end of experiment.
3.1.4.Movement restrictions Participants were asked to evaluate the following statement: I felt discomfort because the system restricted my movement. Naturally, wired systems – ANT and aC received lower scores than wireless system. It is important to notice that, while being wired system, BSM did not show low score for movement restriction.
3.1.5.Results overview By taking a mean over all UX features per system, we calculated a general UX score for each system. Table 2 shows the medium answer across all participants to questions that measure subjective differences between systems. From the perspective of user experience, Cwet is the best system. It had high score on all of the investigated features. Runner-up, with slightly higher discomfort scores was BSM. The other 5 measured systems present some shortcomings, either by causing significant discomfort, not having enough cap fit adjustment alternatives or restricting movement of participants in significant ways. Table 2. User experience (UX) of participant features and grades; color coding indicates the UX of participant from worse (1, red) to best (5, dark green)
gN
1
Disc: Gener al
Disc: under Cap
Disc: pinch
cap fit
n/p
mR
Mean
4
4
2
4
2
4.5
3.42
http://www.biosemi.com/faq/shielding%20vs%20active%20elect rodes.htm
4. DISCUSSION Fatigue modulates alpha, theta and beta bands of the EEG and the signal to noise ratio [1] and decreases efficiency of processing [8]. That means fatigue has significant effects on signal quality and task performance. Thus, differences of comfort level that may in turn be a factor inducing fatigue, render systems less comparable due to uncontrolled interferences. Therefore, it is important to consider participants comfort as part of experimental design. The experimental design, with multiple sessions and only a few subjects comparing many systems, provided a low number of data points per system. Nevertheless, using 7 EEG systems and multiple sessions per participant and system, this study is among the most comprehensive performed to date and shows interesting tendencies and observations, pointing out issues little discussed before. One concern in this study is comparability of data recorded at different labs. There might be confounds induced by the length of recording, lab environment, experimenters, cultural difference between participants, experimental. However, most of the data used in this study are independent of all of those differences, since they were based on data provided by manufacturer and objective observation. Thus, while the two experiments contained different stimuli, the study results are still valid as the same questionnaires and data assembling measures were used. This study provides a number of interesting implications and clues as to the source of interferences and comparability issues rarely considered before. On the one hand, technological advancements and differences in construction of cap may render some of the motion artifacts not significant anymore. One of such improvements is shielded cable used in ANT system. It has been shown that it greatly decreases cable sway during movement [3], thus removing one of the most common motion artifacts in mobile paradigms. In 1996, Biosemi presented the idea of active electrodes [14]: To reduce noise caused by movement of the cables, the distance to the first amplifier has been decreased to a very minimum by placing it on every electrode. This way, cables did not have to be shielded, making the whole setup smaller and cheaper 1 . Alternatively, in other systems, active shielding is used with the intention to make cable sway irrelevant (e.g. ANT, ant-neuro.com). Another technological advancement that received a lot of attention are dry electrodes [7, 12, 16, 22]. They were developed to decrease set up time, since electrodes do not have to be gelled anymore. In addition, dry electrodes can potentially solve the problem of contact quality decreasing over time due to drying up of conductive substance. Another step to deal with motion artefacts was to add additional sensors, such as accelerometer or EOG [20] to the systems to use as a reference for automatic artefact removal. However, this study points out that technical signal quality is not the only factor for data quality over all, thus, it would be interesting
to investigate the relation between mobility and user experience of participant features and signal features per se. In other words, instead of comparing just signal quality, focus rather on precise measurement of subjective and objective factors related to EEG experiment, similarly to this study. Since fatigue may cause significant changes in the data [8, 15] one of such features could be the experience of experimenters on participant’s fatigue. Here the subfeatures could be length of setup or activities that participant was doing while waiting for setup to complete. It would be also interesting to see the correlation between fatigue induced specifically by system. This is a difficult task, since there is a number of sources influencing both fatigue and mood, however it could provide an invaluable clue as to the effect size and comparability of effect sizes recorded with different systems. And most of all, it would be interesting to see more usability studies focusing on user experience of participants – specifically comfort, distraction, influence of system on emotions and motivation – with more systems, especially the modern mobile systems or possibly even consumer grade EEG wearables. It is also important that most of the systems, regardless of how mobile they are, still do not provide good enough user experience to participants, which in turn may induce additional interferences, rendering recordings with different systems less comparable and maybe even obscuring some effects.
[10]
[11]
[12]
[13]
[14]
[15]
5. REFERENCES [1] Cao, T., Wan, F., Wong, C.M., Cruz, J.N. da and Hu, Y. 2014. Objective evaluation of fatigue by EEG spectral analysis in steady-state visual evoked potential-based braincomputer interfaces. Biomedical engineering online. 13, 1 (2014), 28. [2] Ebrahimi, T. 2005. Estimating Driving Performance Based on EEG Spectrum Analysis. EURASIP Journal on Applied Signal Processing. (2005), 85–94. [3] Ehinger, B.V., Fischer, P., Gert, A.L., Kaufhold, L., Weber, F., Pipa, G. and König, P. 2014. Kinesthetic and vestibular information modulate alpha activity during spatial navigation: a mobile EEG study. Frontiers in human neuroscience. 8, February (2014), 71. [4] Gramann, K., Gwin, J.T., Ferris, D.P., Oie, K., Jung, T.P., Lin, C.T., Liao, L.D. and Makeig, S. 2011. Cognition in action: Imaging brain/body dynamics in mobile humans. Reviews in the Neurosciences. 22, 6 (2011), 593–608. [5] Gwin, J.T., Gramann, K., Makeig, S. and Ferris, D.P. 2011. Electrocortical activity is coupled to gait cycle phase during treadmill walking. NeuroImage. 54, 2 (2011), 1289–1296. [6] Hairston, D.W., Proie, R., Conroy, J. and Nothwang, W. 2015. Batteryless Electroencephalography (EEG): Subthreshold Voltage System-on-a-Chip (SoC) Design for Neurophysiological Measurement. ARMY RESEARCH LAB ABERDEEN PROVING GROUND MD HUMAN RESEARCH AND ENGINEERING DIRECTORATE. [7] Hairston, W.D., Whitaker, K.W., Ries, A.J., Vettel, J.M., Bradford, C.J., Kerick, S.E. and McDowell, K. 2014. Usability of four commercially-oriented EEG systems. Journal of neural engineering. 11, 4 (2014). [8] Hopstaken, J.F., Linden, D.V.D., Bakker, A.B., Kompier, M.A.J. and Leung, Y.K. 2016. Shifts in Attention During Mental Fatigue : Evidence From Subjective, Behavioral, Physiological, and Eye-Tracking Data. Journal of Experimental Psychology: Human Perception and Performance. (2016). [9] Jatupaiboon, N., Pan-ngum, S. and Israsena, P. 2013. Emotion classification using minimal EEG channels and
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
frequency bands. Computer Science and Software Engineering (JCSSE), 2013 10th International Joint Conference on. (2013), 21–24. Lau, T.M. 2012. How Many Electrodes Are Really Needed for EEG-Based Mobile Brain Imaging? Journal of Behavioral and Brain Science. 02, 03 (2012), 387–393. Li, W., Jiang, Z., Liu, Y., Wu, Q., Zhou, Z., Jorgensen, N., Li, X. and Li, C. 2014. Positive and negative emotions modulate attention allocation in color-flanker task processing: Evidence from event related potentials. Motivation and Emotion. 38, 3 (2014), 451–461. Lopez-Gordo, M.A., Sanchez-Morillo, D. and Pelayo Valle, F. 2014. Dry EEG electrodes. Sensors (Basel, Switzerland). 14, 7 (2014), 12847–12870. Makeig, S., Gramann, K., Jung, T.-P., Sejnowski, T.J. and Poiznerb, H. 2009. Linking Brain, Mind and Behavior. International Journal. 73, 2 (2009), 95–100. MettingVanRijn, A.C., Kuiper, A.P., Dankers, T.E. and Grimbergen, C. a. 1996. Low-cost active electrode improves the resolution in biopotential\nrecordings. Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 1, (1996), 101–102. Michail, E., Kokonozi, A., Chouvarda, I. and Maglaveras, N. 2008. EEG and HRV markers of sleepiness and loss of control during car driving. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. July 2015 (2008), 2566–2569. Mihajlovi, V., Alexandra-Maria, T., Chen, Y.-H., Grundlehner, B., Penders, J. and Serdijn, W. 2014. Signal Quality in Dry Electrode EEG and the Relation to Skinelectrode Contact Impedance Magnitude. International Conference on Biomedical Electronics and Devices. (2014), 12–22. Ojeda, A., Bigdely-Shamlo, N. and Makeig, S. 2014. MoBILAB: an open source toolbox for analysis and visualization of mobile brain/body imaging data. Frontiers in human neuroscience. 8, March (2014), 121. Oliveira, A.S., Schlink, B.R., Hairston, D.W., König, P. and Ferris, D.P. 2016. Induction and separation of motion artifacts in EEG data using a mobile phantom head device. J Neural Eng epub ahead of print. (2016). Plöchl, M., Ossandón, J.P. and König, P. 2012. Combining EEG and eye tracking: identification, characterization, and correction of eye movement artifacts in electroencephalographic data. Frontiers in human neuroscience. 6, October (2012), 278. Sweeney, K.T., Ward, T.E. and McLoone, S.F. 2012. Artifact Removal in Physiological Signals - Practices and Possibilities. IEEE Transactions on Information Technology in Biomedicine. 16, 3 (2012), 488–500. Thompson, T., Steffert, T., Ros, T., Leach, J. and Gruzelier, J. 2008. EEG applications for sport and performance. Methods. 45, 4 (2008), 279–288. Wyckoff, S.N., Sherlin, L.H., Ford, N.L. and Dalke, D. 2015. Validation of a wireless dry electrode system for electroencephalography. Journal of NeuroEngineering and Rehabilitation. 12, 1 (2015), 95. Zhang, D., Svensson, C. and Alvandpour, A. 2011. Power consumption bounds for SAR ADCs. 2011 20th European Conference on Circuit Theory and Design, ECCTD 2011. c (2011), 556–559.
