Delineating the Operational Envelope of Mobile and Conventional EDA Sensing on Key Body Locations Panagiotis Tsiamyrtzis Malcolm Dcosta Dvijesh Shastri Dept. of Statistics Computational Physiology Lab Computer Science Athens Univ. of Econ. & Bus. Univ. Of Houston Univ. of Houston-Downtown
[email protected] [email protected] [email protected] Eswar Prasad Ioannis Pavlidis Computational Physiology Lab Computational Physiology Lab Univ. of Houston Univ. of Houston
[email protected] [email protected] ABSTRACT
Electrodermal activity (EDA) is an important affective indicator, measured conventionally on the fingers with desktop sensing instruments. Recently, a new generation of wearable, battery-powered EDA devices came into being, encouraging the migration of EDA sensing to other body locations. To investigate the implications of such sensor/location shifts in psychophysiological studies we performed a validation experiment. In this experiment we used startle stimuli to instantaneously arouse the sympathetic system of n = 23 subjects while sitting. Startle stimuli are standard but minimal stressors, and thus ideal for determining the sensor and location resolution limit. The experiment revealed that precise measurement of small EDA responses on the fingers and palm is feasible either with conventional or mobile EDA sensors. By contrast, precise measurement of small EDA responses on the sole is challenging, while on the wrist even detection of such responses is problematic for both EDA modalities. Given that affective wristbands have emerged as the dominant form of EDA sensing, researchers should beware of these limitations. Author Keywords
Electrodermal Activity (EDA); skin conductance; affective sensors; wearable sensors; startle ACM Classification Keywords
H.5.2 Information Interfaces and Presentation (e.g. HCI): User Interfaces; Input devices and strategies INTRODUCTION
Sympathetic signals are fundamental indicators of emotional state in affective human studies [1]. Peripheral sensing of sympathetic signals primarily targets transient perspiratory responses, also known as electrodermal activity (EDA) [2]. Conventionally, EDA is measured with clinical instruments Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]. CHI 2016, May 7–12, 2016, San Jose, California, USA. Copyright © 2016 ACM ISBN 978-1-4503-3362-7/16/05 ...$15.00. http://dx.doi.org/10.1145/2858036.2858536
that are grid-powered, have limited Internet connectivity, and feature tethered probes. These probes are attached to the subject’s fingers. Such conventional sensing schemes restrict mobility, and in some cases interfere with the subject’s task (e.g., dexterous action studies [10]). The availability of more autonomous and unobtrusive forms of EDA sensing has increased with the proliferation of mobile computing [11]. There are several mobile EDA sensors in wristband or other wearable forms. These include the Q Sensor (Affectiva, Waltham, Massachusetts), the Shimmer3 GSR + Optical Pulse Development Kit (Shimmer, Dublin, Ireland), and the E4 Wristband (Empatica, Milan, Italy). There are also several conventional EDA instruments built around desktop data acquisition modules. These include the Powerlab (ADInstruments, Bella Vista, New South Wales), the MP 150 (Biopac, Goleta, California), and others. The technology within each family of EDA instruments (conventional vs. mobile) is largely the same, but there are differences across families. Specifically, both instrument families use Ag/AgCl disc electrodes with contact areas of 1.0 cm2 for their recordings, as recommended in the literature [4]; they differ, however, in terms of power mode (AC vs. DC), packaging (large vs. small form factor), standard communication capabilities (wired vs. wireless), and wearable options. The power mode has some implications in measurement accuracy (AC is better [2]), while the wearable options in the mobile family of sensors offer flexibility but also present a challenge. The challenge stems from the fact that sympathetically induced EDA responses differ in their strength among various body locations [2, 18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body locations. The combination of lesser accuracy with lesser responses may bias results in mobile studies of subject affect. Such biases have not been sufficiently appreciated or studied in the literature. M. van Dooren et al. investigated the relative strength of EDA responses on 16 body locations [18]. They aroused subjects via emotional film segments, measuring responses with a conventional EDA device. The study concluded that EDA measurements at the foot were most similar with those of the finger. In contrast, arm, back, and armpit traces differed most
from the finger trace. Ranogajec and Geršak also found that EDA finger responses are comparable to EDA foot responses [14]. Poh et al. [13] compared the measurements of mobile EDA sensors on subjects’ wrists against measurements taken with conventional EDA sensors on subjects’ fingers; the stimuli included physical activity, mental arithmetic, Stroop, and horror movies. Then, Poh and other researchers used mobile EDA sensors to collect affective signals from daily activities including reading, walking, and sleeping [13, 16]. They also used mobile EDA sensors in an epilepsy study [12]. The range of applications kept expanding. Chaspari et al. [3] used mobile EDA sensing to model verbal response latencies in autistic children. McDuff et al. [8] used mobile EDA sensing to quantify user emotions in a reflection study of past events. Overwhelmingly, in all these studies researchers used the wristband form of mobile EDA sensing and reported interesting findings. To summarize, some researchers investigated the strength of EDA responses at several body locations using a conventional EDA device for the measurements; thus, they studied the location effect but not the sensor effect [18]. Other researchers attempted to investigate the sensor effect by using conventional EDA sensors on the subjects’ fingers and mobile EDA sensors on the subjects’ wrists; in doing so, they confounded sensor effect with location effect [13]. Importantly, the stimuli used in all these studies were emotionally complex and thus, non-ideal for inter-subject comparison. This leaves the investigation of mobile EDA sensing incomplete. We aim to fill this gap by answering two questions: (a) What is the EDA sensor effect at body locations with wearability potential? (b) What are the body’s EDA responses in these locations when sympathetic arousal takes place in its most primal form? Stimuli used in prior studies were capable of generating complex emotions, which might depend on the subject’s genotype and phenotype. In contrast, we use auditory startle stimuli in our study. Auditory startle is known to invoke a threshold response on the fingers of healthy subjects [6]. This is the least common denominator when it comes to subject arousal, allowing us to establish a firm basis for comparisons. An EDA device that successfully captures startle responses on the fingers, meets the ‘gold standard’, as it has the capability of measuring a ubiquitous sympathetic arousal of low intensity and minimal duration at a prime neurophysiological site. Conventional EDA devices belong to this category. Even a gold standard device, however, may not capture startle responses on a different body location (e.g., wrist), where there may be lower concentration of perspiration glands and sparser innervation. Hence, an EDA device should be validated for every intended body location. Using EDA sensors on body locations that have not been validated against the gold standard may introduce bias into sympathetic measurements in the form of underestimation. With the proliferation of mobile affective studies, this is becoming a pressing question.
METHODS
Ethics Statement. The experimental procedures were approved by the University of Houston’s Institutional Review Board (IRB), and were performed in accordance with the approved guidelines. Informed consent was obtained from each subject before conducting the experiments. Subjects. We recruited subjects through email solicitations and flyer posting in the university campus community (population about 35, 000). We excluded children (< 18), subjects with hearing impairments, and subjects on medications. Age brings psychological and physiological changes that are especially prominent during developmental and late years [2]; for this reason, we did not include children and older adults (> 59) in the subject pool. The use of auditory startle stimuli in the experiment necessitated the exclusion of subjects with hearing problems. Certain medications affect sympathetic responses [5]; to simplify screening and minimize confounding factors, we excluded all medication cases.
Experimental Design. After each subject consented, s/he filled out a biographic questionnaire, the Trait Anxiety Inventory (TAI), and the State Anxiety Inventory (SAI) [17]. The last two meant to check if any pathological or extraordinary conditions were biasing responses. Next, we asked each subject to wash her/his hands and feet prior to sensor attachment. We waited 10 min for normal moisture levels to reestablish on their skin, before attaching the sensors. This preparation ensured optimal and uniform skin conditions in the attachment areas for all subjects. We used Galvanic Skin Response probes connected to an ADInstruments PowerLab data acquisition unit (ADInstruments, Bella Vista, Australia) as conventional EDA sensors (C EDA); we used Q Sensor sets (Affectiva,Waltham, Massachusetts) [15] as mobile EDA sensors (M EDA). This was a validation experiment that aimed to isolate the sensor effect from the location effect in EDA measurements. Ideally, for the sensor validation part we needed to test if the C EDA sensor and the M EDA sensor give the same measurement on the same body location at the same time. The recommended test locations are the fingers and palm, where the sympathetic responses are known to be strong. Because it is impossible to attach the C and M sensors on the same location, however, we used the bilateral symmetry of the human body as a workaround. The results show that the C sensor measurements on the fingers and palm of the right hand were in agreement with the M sensor measurements on the fingers and palm of the left hand, respectively (see RESULTS section). For this to happen both of the following conditions had to be true: (a) The C and M sensors had to have similar capability and (b) the subject pool had to have bilateral symmetry. Hence, our subject pool had bilateral symmetry and this is not surprising, because the great majority of the human population has bilateral symmetry in its functional responses under non-extreme conditions [7]. A total of n = 25 subjects fulfilling the inclusion/exclusion criteria volunteered for the experiment. We attached seven
C Wrist
Headset
C Palm
M Wrist
M Palm
C Fingers
B
4.30
1.87
0.66
180
M Fingers
S003 C Fingers - Raw Signal
3.09
µS
A
C
M Sole
4.30
240
300
Time [s]
360
420
S003 C Fingers - Noise-filtered Signal
µS
3.09
1.87
A 0.66
180
tOn tP
tOff
300
Time [s]
360
420
Figure 1. Experiment and sample outcome. A, Experimental setup, demonstrating all seven nodes. B, Raw EDA signal on the fingers of subject S003, captured via the conventional sensing device. The dotted lines mark the occurrence of the three stimuli. Only the last minute of the baseline period is depicted to economize space. C, The signal after noise filtering. There are multiple arousals after each stimulus; circles mark onsets, triangles mark peaks, and crosses mark offsets; ton , denotes the time of Onset occurrence, t p the time of Peak occurrence, and to f f the time of Offset occurrence; A stands for the arousal’s amplitude.
EDA sensors to each subject: three sets of a conventional EDA sensor (C EDA) on the subject’s right half and four sets of a mobile EDA sensor (M EDA) on the subject’s left half. Specifically, we had the following sensor-location pairs (aka nodes - Fig. 1A): • Conventional EDA on the right hand fingers (C Fingers)
served as the auditory startle stimulus. The first stimulus was delivered in the fourth minute. After that, two more stimuli were delivered at one-minute interval each. All sensor measurements were synchronized and recorded throughout the experiment. Hence, our usable data bank accrued 161 EDA signals (23 subjects × 7 EDA signals per subject - one for each node).
• Conventional EDA on the right palm (C Palm) • Conventional EDA on the right wrist (C Wrist) • Mobile EDA on the left hand fingers (M Fingers) • Mobile EDA on the left palm (M Palm) • Mobile EDA on the left wrist (M Wrist) • Mobile EDA on the left sole (M Sole) One of the 25 subjects had missing conventional signals due to a technical problem. For another subject the recorded EDA responses across body locations had a low signal to noise ratio, rendering analysis impossible. Hence, we excluded these two subjects from any further consideration. The usable data set included n = 23 subjects (11 males / 12 females; age: 23.91 ± 8.12). Figure 2, Fig. 3, and Fig. 4 depict the analyzed data set for the finger, wrist, and sole locations, respectively, facilitating cross-sensor and cross-location comparisons. The main experiment lasted seven minutes for each subject. During these seven minutes, the subject listened to soothing music via a headset in a near dark room. This soothing music was interrupted three times with a glass breaking sound, which
Signal Processing. The C sensor samples at the rate of 25 points per second, while the M sensor samples at the rate of 32 points per second. For this reason, we resampled the M signals down to 25 points per second to establish uniformity across the signal bank. Then, we applied a moving window filter twice to reduce noise. We set the window size at W = 125 points. We arrived at this selection by performing sensitivity analysis with window sizes W = 25 × k, k = 1, 2, 3, . . . We chose the size of the increment to be 25 points because it matches the minimum resolution (i.e., rate of sampling). At k = 5 we attained optimal performance by substantially reducing noise without destructing signal information; at k > 5 the signals exhibited over-smoothing, which affected the performance of the Peak and Onset detection algorithms. Even under the best circumstances, EDA signals are not only noisy (Fig. 1B) but also variable. An electrodermal response may feature more than one Peak. Such peaks correspond to multiple neurophysiological firings provoked by a single stimulus [2]. Hence, for each stimulus a subject can experience none, one, or several peaks (Fig. 1C). For this reason, signal abstraction is quintessential to fair comparisons; noise filtering
1.95
1.67
1.39
0.57
8.29
µ S
180
240
300
360
S006
2.81
420
2.21
180
240
300
360
S007
8.72
2.31
2.82
2.99
3.06
0.14
2.23
1.89
0.94
2.67
−2.02
1.64
0.79
0.87
420
180
4.79
3.05
240
300
360
S008
420
180
6.15
5.29
4.16
2.08
5.76
−0.44
1.59
1.86
3.52
1.11
5.36
−1.74
−0.45
1.28
−1.57
2.89
0.15
4.97
−3.04
µ S
1.90
0.64
240
300
360
S010
420
180
6.93
8.33
6.78
5.74
3.00
−0.77
µ S
10.55
180
240
300
360
S014
360
S019
µ S
15.82
µ S
13.78
300
360
S015
2.21
2.77
−2.22
1.66
5.97
1.55
420
180
3.03
0.83
S020
300
360
S016
420
180
240
300
360
S017
420
10.84
21.71
4.89
9.13
3.89
2.63
15.97
3.76
7.43
3.28
2.37
10.22
2.63
5.73
2.67
420
180
12.41
2.35
0.73
6.58
240
6.02
0.81
3.14
3.53 2.87
1.47
7.78
S013
8.29
3.63
360
420
1.97
2.89
300
300
−0.76
2.11
240
240
3.85
27.46
180
180
10.61
3.15
420
S012
420
360
S009
2.28
−1.65 240
360
420
0.70
3.38
180
300
360
12.93
1.68
420
240
300
2.59
4.56
7.25 300
180
4.93
9.12
240
S011
420
5.00
4.23
180
360
2.15
2.76
10.99
300
6.01
4.83
12.86
240
240
0.85
1.72
180
1.01
µ S
10.67
1.09
4.08
240
300
360
S021
420
180
240
300
4.49
360
S022
420
0.92
5.38
5.51
6.57
4.27
5.04
0.88
11.74
5.38
1.57
0.65
0.73
3.16
4.58
0.85
9.70
4.19
0.78
0.58
−5.11
2.05
4.12
0.82
420
180
240
300
360
420
S025
5.81
3.00
5.55
2.31
0.83
3.74
2.49
4.11
2.14
420
180
240
300
360
420
Time [s]
180
2.49
240
300
240
300
360
420
2.66
0.92
5.20
180
Time [s]
6.99
6.54
360
180
3.51
1.00
300
S024
420
7.88
7.89
Time [s]
360
8.42
9.95
240
300
4.02
1.09
180
240
360
M Fingers
360
S023
9.23
µ S
300
C Fingers
240
µ S
180
µ S
3.46
S005
3.41
µ S
2.77
2.51
S004
µ S
4.47
3.87
13.05
S003
3.85
3.06
µ S
S002
15.43
420
Time [s]
Figure 2. C and M EDA responses on the fingers per subject. From the four minutes of baseline only the last minute is depicted in the graphs to economize space. Vertical dotted lines identify stimuli times and inverted triangles denote peaks, facilitating arousal comparison. The M EDA signal for subject S009 was not collected due to technical reasons. The graphs confirm qualitatively the high responsiveness of the fingers location and the agreement between the two sensor modalities.
alone is not sufficient. What is of interest here is the ability of the sensor to measure the essence of the stimulus’ response at the specific site. A normal neurophysiological response (arousal) can be reconstructed to a good approximation from three key points in the corresponding EDA signal [2]: Onset, Peak, and Offset. The Onset point represents the start of the EDA activation; the Peak point represents the culmination of the activation; and, the Offset point represents the ebbing of the activation. On the noise-filtered signals, we focused on the non-baseline period from t = 4 to 7 min. We divided this period into three segments: S 1 = [4, 5) min, S 2 = [5, 6) min, and S 3 = [6, 7] min; each segment included the respective stimulus S 1 , S 2 , S 3 and the period up to the delivery of the next stimulus or the end of the experiment. Within each segment we sought
arousals (firings), each characterized by a Peak and an Onset. An Offset might or might not have existed, depending on the recovery rate and the timing of the next arousal. To facilitate curve analysis we performed averaging on the noise-filtered signals reducing the resolution down to two points per second. Then, we executed the following algorithmic steps: 1. Peak Detection: We applied a Peak detection algorithm on the non-baseline portion of each signal - S 1 ∪ S 2 ∪ S 3 . The template we used for Peak detection was a 5-tuple structure fulfilling the following condition: P(t − 2) < P(t − 1) < P(t) > P(t + 1) > P(t + 2), where P(t) is the signal intensity P recorded at time t. In essence, this template is the discrete definition of the derivative test for locating curve maxima. Figure 2, Fig. 3A, and Fig. 4 depict the peaks located by the template matching algorithm for all legitimate signals in
180
360
420
240
300
360
420
180
240
300
360
S019
420
180
240
300
360
420
S014
2.40 1.78
0.22
10.16
420
180
420
300
360
S020
180
420
180
240
300
360
420
180
240
180
240
300
300
360
420
S003
360
S004
S005
2.42
0.03
0.10
3.05
0.10
0.02
0.03
0.03
2.80
0.09
0.02
3.56
360
420
180
240
S008
300
0.11
360
S009
µ S 180
240
300
360
S012
420
180
240
300
360
420
180
240
S013
300
360
420
180
2.64
0.09
0.14
2.06
0.08
420
180
S014
2.27
0.07
13.76
2.21
1.87
0.06
12.84 180
300
360
420
S019
180
240
300
360
420
180
S020
8.94
360
S023
0.59
420
0.07
180
240
300
360
420
S024
0.80
180
240
300
360
420
240
300
360
420
0.06
−0.53
0.34
3.59 420
420
180
180
240
Time [s]
300
180
240
300
360
240
300
360
420
S022
µ S
0.08
4.07
0.08
3.94
0.15 0.14
420
180
0.16
0.13 240
300
360
420
0.38 0.37
Time [s]
420
Time [s]
S025
5.22
360
360
3.68
0.35
300
300
S021
3.81
4.13
240
240
0.07
4.68
180
360
S016
0.07
0.36
0.05 420
180
3.65
−0.09
360
420
3.54
0.06
300
360
8.06
0.07
240
300
S015
8.36
0.57
180
300
S011
0.11
3.75
0.54 0.52
240
3.85
360
µ S
µ S
300
240
0.10
2.33
8.65
240
180
2.14 240
S017
180
420
0.14
2.01
0.06 420
360
3.80
14.67
360
300
S010
3.22
15.59
300
240
0.14
0.07
240
420
S006
0.03
0.15
0.07
180
360
0.03
2.15
2.29
300
µ S
300
240
420
2.37
240
420
Time [s]
0.10
180
360
Time [s]
3.31
420
180
Time [s]
S025
0.17
360
300
S022
0.24
300
240
S021
0.73
0.03
240
180
M Wrist
µ S
360
0.03
S007
µ S
300
2.46
180
µ S
240
0.60
Time [s]
S002
1.16
11.77
420
µ S
µ S
Time [s]
420
21.40
0.39 240
0.27
360
3.02
14.99
0.14 300
420
0.63
0.18
240
360
11.84
0.23
180
300
S016
0.30
0.47
420
240
18.20
0.34
360
180
0.88
0.42 300
420
13.52
0.76
S024
360
0.37
1.12
0.53
240
300
S015
15.20
3.69 180
240
0.14
0.45
4.98
420
180
0.16
µ S
180
3.82
µ S
420
0.87
2.50
420
0.12
S023
B
360
0.10
1.41
360
300
S011
1.74
5.15
300
240
2.67
6.52
240
180
0.12
0.65
180
420
4.94
6.27
360
360
0.18
7.56
300
300
2.43
1.96
240
240
2.77
2.51
180
180
S010
7.89
9.27
µ S
300
S009
0.15
µ S
S017
240
6.22
S013
360
180
0.21
S012
300
420
3.13
1.16
240
0.14
2.88
1.33
180
0.69
0.17
0.25
360
0.47
7.50
1.50
300
360
0.20
0.07
0.26
2.98
8.79
240
300
S008
0.91
0.20
0.28
180
240
0.69
µ S
420
0.08
µ S
360
0.32
1.14
C Wrist
µ S
300
S007
S006
1.36
0.90
µ S
240
S005
1.11
0.09
M Wrist
180
1.66
S004
0.09
µ S
S003
µ S
S002
C Wrist
A
420
Time [s]
Figure 3. C and M EDA responses on the wrist per subject. From the four minutes of baseline only the last minute is depicted in the graphs to economize space. Vertical dotted lines identify stimuli times and inverted triangles denote peaks, facilitating arousal comparison. A, Signals manifesting sympathetic responses on the wrist. B, Signals manifesting the absence of sympathetic responses on the wrist (complement to panel A), which were excluded from further processing and analysis. The graphs confirm qualitatively the low responsiveness of the wrist location, a result that was documented quantitatively in the text.
the finger, wrist, and sole nodes. For the palm node, signals and results were quite similar to the finger node. The Peak detection algorithm accurately located almost all legitimate peaks in our signal bank. It failed to locate peaks only in a
couple of very weak wrist signals, where there is inherently ambiguity as to what constitutes a Peak - see S003 and S013 in Figure 3A.
180
240
300
240
300
360
240
300
360
240
300
360
240
300
Time [s]
0.47
420
180
240
360
360
S007
420
180
240
300
360
S008
0.98
420
180
240
300
360
S009
1.91
420
0.69
1.74
0.93
1.64
1.39
0.87
1.36
0.54
1.03
0.82
1.09
0.47
420
180
240
300
360
S011
420
180
240
300
360
S012
1.65
0.62
420
180
240
300
360
S013
2.03
420
2.15
1.82
1.35
1.71
1.67
1.40
1.05
1.40
1.18
0.99
0.75
1.08
0.69
420
180
240
300
360
S015
420
180
240
300
360
S016
1.03
420
180
240
300
360
S017
3.80
420
3.32
1.14
0.99
2.97
2.86
0.83
0.94
2.13
2.41
0.52
0.90
1.30
1.95
420
180
240
300
360
S020
420
180
240
300
360
S021
2.73
420
180
240
300
360
S022
2.42
420
0.62
1.83
2.40
1.86
0.59
1.43
2.08
1.31
0.56
1.03
1.76
0.76
0.53
420
180
240
300
360
S024
2.41
360
300
420
180
240
300
360
420
0.92
1.55
0.84
2.70
1.13
0.76
2.39
180
240
300
360
420
Time [s]
3.01
180
240
300
240
300
360
420
3.33
1.98
420
180
Time [s]
S025
1.00
µ S
1.63
2.24
S023
180
3.01
1.45
S019
180
0.50
1.71
2.23
S014
180
2.05
2.10
S010
180
3.34
µ S
S006
0.54
2.03
µ S
360
0.57
2.47
µ S
300
S005
2.88
3.67
µ S
240
S004
4.00
2.34
360
M Sole
180
S003
2.66
µ S
S002
420
Time [s]
Figure 4. M EDA responses on the sole per subject. From the four minutes of baseline only the last minute is depicted in the graphs to economize space. Vertical dotted lines identify stimuli times and inverted triangles denote peaks, facilitating arousal comparison. The graphs confirm qualitatively the responsiveness of the sole location, a result that was documented quantitatively in the text. However, amplitude correlations with the finger and palm locations are not strong, suggesting that precise EDA measurements on the sole may be challenging.
2. Detection of First Onset: Between each stimulus and the first Peak that followed it within the respective segment, we determined the corresponding Onset; this was the Onset of the first arousal in response to the specific stimulus. We computed the intensity of this first Onset as PS + 10%(Peak1 − PS ), where PS denotes the signal intensity at the time of the stimulus and Peak1 denotes the first Peak. 3. Detection of Additional Onsets: When multiple firings took place in response to a stimulus, then more than one Peak points existed within the respective segment S i , i = 1, 2, 3. We located the Onset point corresponding to such an additional Peak in the valley between this additional Peak and the preceding Peak. The template we used for this
was a 3-tuple structure fulfilling the following condition: P(t − 1) > P(t) < P(t + 1), where P(t) is the signal intensity P recorded at time t. 4. Offset Detection: For each Peak we determined the matching Offset as the 50% × (Peak - Onset) drop-off point. If this point occurred after the next Onset or stimulus, then it was rejected and the Offset was treated as a missing value; this is an indication that the subject had not recovered at the time a new arousal set in. RESULTS
Peaks are the most characteristic points of an electrodermal response. Hence, our analysis proceeds at two levels:
C Fingers
Detection level We pay attention to the occurrence of peaks, as proxies of neurophysiological responses; the absence of peaks signifies the absence of a response at a node.
To ensure that the subjects’ trait and state anxiety levels did not affect manifestation of sympathetic activation [9], we obtained their trait (TAI) and state (SAI) anxiety psychometrics [17]. Then, we examined in each node the correlations between the number of peaks detected per subject versus the corresponding TAI and SAI scores. None of these correlations was statistically significant (p > 0.05). Detection Level Analysis The number of peaks denote responsiveness to stimuli, with zero peaks signifying a non-responsive subject in the specific node. Different subjects exhibited different levels of responsiveness at different nodes. As we observe in Fig. 5, the wrist locations had the highest number of non-responsive subjects irrespective of the sensor type. This indicates that wrists responded poorly to stimuli. The rest of the locations had minimal numbers of non-responsive subjects, indicating the regular presence of arousals in response to stimuli. We are interested to examine if indeed the proportion of nonresponsive subjects differed significantly among the nodes. To do so we use the Binomial distribution. In each node we sampled 23 subjects, where each subject presented a binary outcome: failure, if s/he was non-responsive or success, if s/he had at least one arousal. Hence, for each node if we call Yi the random variable that denotes the outcome of subject i and θ the unknown probability of success, we have Yi |θ ∼ Bernoulli(θ), giving: ( 0 if no peaks in the entire experiment Yi = (1) 1 if at least one peak in the entire experiment We count the total number of successes in the 23 subjects, forming the random variable X|θ ∼ Binomial(n, θ), n = 23: X=
n X
Yi
(2)
i=1
As we observe in Fig. 5, the maximum likelihood estimates (MLE) at the wrists are quite smaller compared to the other locations. In fact, running a seven-sample test for equality of proportions shows that there are significant differences among the seven nodes (p < 0.01). Next, we exclude the two wrist nodes that appear to be the culprit and we compare the proportions in the remaining five nodes. This time the test returns a non-significant number (p > 0.05), which indicates that with the exception of the wrist nodes, all the other nodes are statistically equivalent in terms of peak presence.
# Subjects
1.00
C Palm 14
0.96
C Wrist 14
12
12
12
10
10
10
8
8
8
6
6
6
4
4
4
2
2
2
0
0
0 2 4 6 8 10 12
M Fingers 14
# Subjects
Measurement level We pay attention to the parameters of the neurophysiological responses recorded at the various nodes. These parameters include the times of occurrence for Onset (ton ), Peak (t p ), and Offset (to f f ) as well as the amplitude of the recorded response (A = Peak - Onset). The times ton , t p , and to f f quantify the arousal’s evolution, while the amplitude quantifies the arousal’s intensity (Fig. 1C).
14
0.96
0 2 4 6 8 10 12
0
M Palm 14
0.83
0.44
0 2 4 6 8 10 12
M Wrist 14
0.35
M Sole 14
12
12
12
12
10
10
10
10
8
8
8
8
6
6
6
6
4
4
4
4
2
2
2
2
0
0
0
0 2 4 6 8 10 12
# Peaks
0 2 4 6 8 10 12
# Peaks
0 2 4 6 8 10 12
# Peaks
0
0.91
0 2 4 6 8 10 12
# Peaks
Figure 5. Responsiveness of subjects per node. Each bar chart depicts the distribution of the number of subjects over different levels of response (i.e., number of recorded peaks in the node). Red bars indicate the number of completely non responsive subjects for the specific node. The maximum likelihood estimates (MLE, θˆ = X/n ) for getting at least one peak appear on the upper right corner of the node’s panel.
Measurement Level Analysis We start the measurement level analysis by studying the relationship of arousal timing between each pair of nodes, taking into account all three stimuli. Figure 6 shows a matrix that is split along its diagonal. The matrix portion below the diagonal shows the scatterplots of Onset (ton ), Peak (t p ), and Offset (to f f ) times for all node pairs. The matrix portion above the diagonal shows the correlation coefficients for the corresponding scatterplots. As indicated both visually and numerically, the timing agreement between nodes is exceptionally high. Hence, the arousal’s evolution is captured accurately at the locations of interest (fingers, palm, wrist, sole), irrespective of the type of sensor used. One has to note, however, that in the case of wrist nodes, the number of points is small. This is consistent with the finding in Fig. 5. Arousal detection at the wrist is rare, but when it occurs, both the C and M sensors track equally well its evolution. Next, we study the relationship of arousal intensity between each pair of nodes, taking into account all three stimuli. Figure 7 also shows a matrix that is split along its diagonal. The matrix portion below the diagonal shows the amplitude (A) scatterplots for all node pairs. The matrix portion above the diagonal shows the correlation coefficients for the corresponding scatterplots. As indicated both visually and numerically, the agreement is poor for pairs involving one wrist node, irrespective of the sensor type attached to this wrist node. Agreement is also poor for pairs involving the mobile sole node. Agreement gets at least moderately strong for pairs involving fingers, palms, or finger-palm, irrespective of the sensing modality. This brings to the fore a problem with EDA sensing: while the timing of the phenomenon is accurately captured across
250
250
toff
350
250
350
0.991
0.983
0.995
0.985
0.987
0.993
C Palm
0.983
0.992
0.991
0.986
0.989
C Wrist
0.983
0.978
0.984
0.984
M Finger
0.991
0.985
0.993
M Palm
0.977
0.988
M Wrist
0.988
350 250
350
250
350
250
350
250
250
M Sole
350
250
350
250
350
250
Time [s]
250
350
250
350
250
C Finger
350
tp
350
ton
350
Time [s] Figure 6. Arousal timing relationships between nodes. BELOW THE DIAGONAL: Scatter-plots of Onset (ton ), Peak, (t p ), and Offset (to f f ) times for the various node pairs. ABOVE THE DIAGONAL: The Pearson correlation coefficients that correspond to the strength of the linear relationships depicted in the scatter-plots below the diagonal (all are significant, p < 0.01).
locations and sensor types, the magnitude of the phenomenon poses a challenge. This challenge is bigger for the sole and insurmountable for the wrist location, irrespective of the sensor type. DISCUSSION
We recognize the importance of quantifiable and objective information the EDA responses can provide to studies of subject affect. Traditionally, EDA sensing is performed on the fingers with conventional EDA devices. Although, this has serious
usability problems, it captures well minimal bursts of sympathetic activation. The new mobile EDA sensors, attached on various body locations, have obvious usability advantages. However, one has to be careful not to compromise measurement accuracy or if s/he does, s/he should at least be aware of it. In this study we found that in response to minimal standardized sympathetic stimuli, conventional EDA devices are in moderate agreement with mobile EDA devices on the fingers and palm. At the same time, we found that both conventional and
A 0.0 0.2 0.4 15
0.0 1.5 3.0
-0.064
0.739
0.559
0.245
0.169
C Palm
-0.049
0.828
0.757
-0.079
0.141
C Wrist
0.067
0.254
0.624
0.394
M Finger
0.6
-0.054
0.171
M Palm
-0.328
0.043
M Wrist
0.096
0.0 1.5 3.0
0.0 1.0 2.0
0 5
0.815
3.0
0.0 0.2 0.4
μS
0 5 10
C Finger
5 10
0.0 0.4 0.8
0
0.0
1.5
M Sole 0 5
15
0.0
1.0
2.0
0.0 0.4 0.8
0.0
1.5
3.0
μS Figure 7. Arousal intensity relationships between nodes. BELOW THE DIAGONAL: Scatter-plots of amplitude (A) for the various node pairs. ABOVE THE DIAGONAL: The Pearson correlation coefficients that correspond to the strength of the linear relationships depicted in the scatter-plots below the diagonal (coefficients in bold are significant, p < 0.01).
mobile EDA devices give significantly inferior measurements when attached to the wrist. Actually, not only measurement but also mere detection of sympathetic responses on the wrist is quite challenging. Given the proliferation of mobile EDA devices, often in the form of affective wristbands, and the accompanying marketing hype, these results are a waking call for a more careful examination of operational limitations; other-
wise, the introduction of measurement bias in affective studies appears likely. A simple way for researchers to guard against this, is to screen their subjects through a startle experiment, ensuring that they produce sympathetic responses on the wrist before embarking on their study.
Another body location - the sole - that is a candidate site for wearable EDA sensing provides strong detection capability, but relatively poor measurement capability.
In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10). ACM, Austin, TX, 849–858.
This study tested sensors and locations in a stationary context. While this was necessary to minimize confounding factors and establish a clear first-level comparison, it certainly does not account for additional effects that are present in practice. The main such effect is ambulatory motion that needs to be studied in the future.
9. Richard S Neary and Marvin Zuckerman. 1976. Sensation seeking, trait and state anxiety, and the electrodermal orienting response. Psychophysiology 13, 3 (1976), 205–211.
ACKNOWLEDGMENTS
This material is based upon work supported by the National Science Foundation via grant # IIS-1249208, entitled ‘EAGER: The Effect of Stress and the Role of Computer Mediation on Exam Performance’. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the funding agency. REFERENCES
1. John L Andreassi. 2000. Psychophysiology: Human behavior & physiological response. Psychology Press, New York, New York. 2. Wolfram Boucsein. 2012. Electrodermal activity. Springer Science & Business Media, New York, New York. 3. Theodora Chaspari, Daniel Bone, James Gibson, Chi-Chun Lee, and Shrikanth Narayanan. May 26-31, 2013. Using physiology and language cues for modeling verbal response latencies of children with ASD. In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, Vancouver, BC, Canada, 3702–3706. 4. Don C Fowles, Margaret J Christie, Robert Edelberg, William W Grings, David T Lykken, and Peter H Venables. 1981. Publication recommendations for electrodermal measurements. Psychophysiology 18, 3 (1981), 232–239. 5. Alan S Horn, Joseph T Coyle, and Solomon H Snyder. 1971. Catecholamine uptake by synaptosomes from rat brain structure-activity relationships of drugs with differential effects on dopamine and norepinephrine neurons. Mol. Pharmacol. 7, 1 (1971), 66–80. 6. Peter J Lang, Margaret M Bradley, and Bruce N Cuthbert. 1990. Emotion, attention, and the startle reflex. Psychol. Rev. 97, 3 (1990), 377–395. 7. AL MacMillan and JM Spalding. 1969. Human sweating response to electrophoresed acetylcholine: A test of postganglionic sympathetic function. J Neurol Neurosurg Psychiatry 32, 2 (1969), 155–160. 8. Daniel McDuff, Amy Karlson, Ashish Kapoor, Asta Roseway, and Mary Czerwinski. May 5-10, 2012. AffectAura: An intelligent system for emotional memory.
10. I Pavlidis, P Tsiamyrtzis, D Shastri, A Wesley, Y Zhou, P Lindner, P Buddharaju, R Joseph, A Mandapati, B Dunkin, and others. 2012. Fast by nature-how stress patterns define human experience and performance in dexterous tasks. Sci. Rep. 2 (2012), 305; doi: 10.1038/srep00305. 11. Rosalind W Picard. 2010. Emotion research by the people, for the people. Emot Rev 2, 3 (2010), 250–254. 12. Ming-Zher Poh, Tobias Loddenkemper, Nicholas C Swenson, Shubhi Goyal, Joseph R Madsen, and Rosalind W Picard. September 1-4, 2010a. Continuous monitoring of electrodermal activity during epileptic seizures using a wearable sensor. In Proceedings of the 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2010). IEEE, Buenos Aires, Argentina, 4415–4418. 13. Ming-Zher Poh, Nicholas C Swenson, and Rosalind W Picard. 2010b. A wearable sensor for unobtrusive, long-term assessment of electrodermal activity. IEEE Trans. Biomed. Eng. 57, 5 (2010), 1243–1252. 14. Slaven Ranogajec and Gregor Geršak. September 22-24, 2014. Measuring site dependency when measuring skin conductance. In Proceedings of the Twenty-Third International Electrotechnical and Computer Science Conference ERK 2014, Vol. 23. IEEE Region 8, Portorož, Slovenia, 155–158. 15. Rozalind W Picard et al., inventors; Massachusetts Institute of Technology, assignee. 2012 Mar 20. Washable wearable biosensor. (March 20 2012 Mar 20). United States patent US 8,140,143. 16. Akane Sano and Rosalind W Picard. August 30-September 3, 2011. Toward a taxonomy of autonomic sleep patterns with electrodermal activity. In Proceedings of the 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2011). IEEE, Boston, MA, 777–780. 17. Charles D Spielberger. 2010. State-Trait Anxiety Inventory. In The Corsini Encyclopedia of Psychology, Irving B Weiner and Edward W Craighead (Eds.). John Wiley & Sons, Inc., Hoboken, New Jersey. 18. Marieke van Dooren, Joris H Janssen, and others. 2012. Emotional sweating across the body: Comparing 16 different skin conductance measurement locations. Physiol Behav 106, 2 (2012), 298–304.
