S2.MEDICON Konstantinidis et al Chalkidiki Greece

1

WADEDA: A Wearable Affective Device with On-Chip Signal Processing Capabilities for measuring ElectroDermal Activity 2

E.I. Konstantinidis1, C.A. Frantzidis1, C. Papadelis , C. Pappas1 and P.D. Bamidis1 1

Lab of Medical Informatics, Medical School, Aristotle University of Thessaloniki, Greece 2

Center for Mind/Brain (CIMEC), University of Trento, Mattarello, (TN) Italy

Abstract— In this paper a miniaturized and wearable skin conductance sensor equipped with a micro-processor is proposed. The device facilitates the acquisition of long-term monitoring of the electrodermal activity under real-world situations. Its generic and flexible design permits the data storage during daily activities, while the prototype equipped with onchip firmware, performs real-time signal filtering and feature extraction. The system’s architecture based on the recent hardware advances, aims to enhance the robustness of previous skin conductance sensors. Emerging applications under non-laboratory experiments are introduced in order to highlight the applicability of the proposed device. The results obtained from preliminary experiments are described. Keywords— Affective Computing, Emotional Processing, Microprocessor, Skin Conductance, Wearable Sensors

I. INTRODUCTION

The skin is the largest organ of the human body. It is responsible for material exchange, thermoregulation, prevention of foreign matter entrance, etc. [1] Its function is controlled through signals emitted by the central nervous system [2]. As a consequence of the signals' arrival, columns of sweat fill the ducts resulting to increased conductivity in the corneum [1]. Thus, sweat alterations (sweating) cause measurable electrical changes in the skin surface. These changes in the electrodermal activity (EDA) may be attributed either to thermoregulatory processes or to emotional sweating which is sensitive to mental changes. The emotional sweating is consisted of a low-frequency component which affects the skin conductance level (SCL) and a rapid stimulus-specific, phasic skin conductance response (SCR) appearing as a wave superimposed to the SCL [3]. The EDA is a reliable index of the limbic system’s function which is related with evolutionary processes promoting species survival [2]. SCRs appear as a response to novel and highly arousing events [4]. Arousal is an emotional dimension reflecting the activation level. Increases in the activation level are correlated with increases in attention and better memory performance [5]. Therefore, the skin conductance has been widely used to psychological experiments. Such experiments mainly involved short-term simultaneous

Σ2.MEDICON Konstantinidis et al Chalkidiki Greece

recordings from both the central and the autonomic nervous system and artificially elicited emotions under specific laboratory conditions [6]. Despite their great impact, the affecting computing community has mentioned the need for development of a new form of wearable, computerized system able to gather and unobtrusively process physiological signals while the user performs his real-world activities. Such systems are expected to provide a deeper understanding of the user’s personalized needs [7]. Towards the introduction of an affective wearable, there are certain design specifications that should be taken into consideration. More specifically, as revealed by its name, such a system should be equipped not only with sensing abilities but it should also be capable of employing pattern recognition techniques in order to detect the user’s affective state [7]. Requirements for using an affective wearable in real time activities are the small size and weight of the device in order to permit the physical contact over long time without obtrusiveness or causing user disturbance. Moreover, they should be able to gather huge amount of data in everyday settings being independent from other devices. So, their power supply should allow them to function for hours while they are equipped with sufficient memory in order to store huge amount of data and having the computational efficiency to perform real-time feature extraction. Another important aspect which is often neglected is that wearable systems are in close linkage with biomedical telemonitoring systems since both of them record neurophysiological signals [6]. Therefore, special care should be given to the proper design in order to eliminate noise interference. Previous attempts lack to provide a solution able to fulfill the aforementioned specifications. One of the first attempts [8] resulted in a recording device which was consisted of a wheatstone bridge for the detection of conductivity changes and a low-pass filter for noise removal. The recordings took place inside a clinical magnetic scanner and the data were then transmitted to a computer located outside the room. A later study [9], used the same circuitry and added a fiberoptic skin conductance transducer in order to acquire artifact free SCRs and fMRI data. These systems were proposed mainly for the acquisition of short-term data during experimental procedures. So, they are focused on the im-

2

provement of the SCR acquisition, while the issue of recognizing the user’s affective state is completely outside from their scope. Moreover, they should be connected with a personal computer in order to store the SCR data. The MIT group introduced devices able to acquire longterm data able to model the affective profile of people while they participate in social activities. Their first prototype [5] was a glove sensing EDA changes which then were mapped in a bright LED display. Despite being a pioneering approach, there still was a need for connection with a host computer for data transmission, while no sophisticated recognition techniques other than a simple value mapping were adopted. So, only raw analysis of conductivity alterations based on the LED brightness is feasible, since the device was not designed for scientific use. A later work [10] contained a microcontroller for EDA acquisition with adjustable gain and then data transmission to a computer by means of Bluetooth technology. This device was technologically innovative since it could gather data in a range of 100 m from the computer while it was reliable and causing minimum obtrusion. However, its great power consumption limited the available operation time while was not fully independent from the computer. Moreover, the device could not store data or process them in order to extract features that could be used for affective recognition. Aiming to enhance the arsenal of affective wearables, we introduce an extremely small and lightweight prototype for reliable acquisition of long-term data without the need of transmitting them to a host computer. However, connection with other recording devices or host computers is feasible through a fiber optic. The device is also equipped with a modern microprocessor for keeping constant sampling rate, adjusting the amplification gain and performing initial filtering. Furthermore, software implementation on chip was adopted in order to perform feature extraction. Raw data or SCR features [6], [11] (amplitude, latency, rise time) can be stored on the memory card inside the device. Therefore, independency is achieved since it can perform acquisition and real-time affective computations without the assistance of any other equipment. So, we aim to provide a device fulfilling the specifications outlined above serving both as an affective wearable system able to sense and understand the user’s affective state while it could also be used in neurophysiological experimental procedures.

today’s data acquisition systems are powered from a single low voltage supply. Following this trend, the EDA system is consisted of the measurement circuit, the microcontroller, the memory card and the output circuit. The measurement circuit is based on the Wheatstone bridge principle. The unknown resistance Rx, which represents the human’s finger resistance, can be calculated by the values of the rest known 3 resistors. The bridge balance is achieved by means of a potentiometer. Most of the previous approaches related to EDA used an analog potentiometer. The proposed measurement circuit employs a digital potentiometer which value can be selected by the microcontroller. Moreover, the amplifier’s gain, ranging from 28 to 1300, is digitally programmable by the microcontroller. Thus, the proposed system is advanced by being able to gain the acquired signal based on limitations stemming from environmental conditions and personalized specifications. Moreover, the bridge balancing takes place continuously during the experiment. The analog to digital converter (ADC) converts the amplified signal to a 16bit word which is then acquired by the microcontroller. The selected DSP 16bit microcontroller possesses 256Kbytes total memory and 32Kbytes RAM. So, it is able to perform low pass filtering (LPF). This capability eliminates the external filter circuit which is presented to previously introduced EDA systems. Apart from that, the high clock frequency provides a layer for more demanding filtering algorithms when it is desirable. As depicted in Fig.1, the microcontroller acts as the master controller of the sampling function. It is responsible for data storage (raw signal or extracted features) to files in the memory card, operating the Led indicators and/or output the signal through a fiber optic connection (digital output). The code execution of the miniaturized device (4.9cm x 2.6cm) is explained to the next section.

Fig. 1 Visualization of the Hardware and Software implemented blocks involving data acquisition, amplification and signal processing stages. II.

MATERIALS & METHODS

A. Hardware Implementation Single-supply operation has become an increasingly desirable characteristic of modern sensor amplifiers. Many of


B. Code Execution The system is designed to fulfill different experiment protocol requirements. The operating modes support led indicators (color and intension depends on the acquired

3

signal), memory card file storage and digital output. As it is depicted in Table 1, a set of parameters customizes the execution of functionalities. These parameters are provided to the system by the user modifying a “settings” file in the memory card. In case of memory card absence the system load the default values. The first step of the data acquisition procedure is the establishment of a constant and user-selected sampling frequency, which is generated by the microcontroller’s internal timer interrupt functionality. During the interrupt function a sample is acquired. Moreover, the interrupt routine implements and executes the two filters (LPF and custom filter) according to the parameters’ values. Finally, it stores the filtered data to a buffer array. This acts as the communication data layer between the sampling routine and the main program. Its length ensures the unobtrusive execution of time-consuming functionalities, like memory card storage and feature extraction.

available for synchronization purposes by annotating the data.

Table 1 System Parameters Parameter Operation Mode Sampling Frequency LPF Custom Filter Digital Output Mode Feature Extraction Algorithm For Feature Extraction Led Indicators Mode Memory Card Organization External Inputs Functionality

Description Led Indicator and/or Memory Card and/or Digital Output Sampling Frequency in Hz Cut-off frequency for the codeimplemented low pass filter Factors for a custom internal filter Pulse Width Modulation or Digital Words Creating a file with features or raw data to memory card Selection of predefined internal algorithms for feature extraction Arousal Level Depiction Type and Name of Files to be written Two external Synchronization inputs functionality

The main program either executes a single operation mode (see Table 1) or concurrently performs more than one. During the led function, the led’s color and intensity depends on the subject’s arousal level. The digital output transmits the filtered data through a fiber optic. The output format could be either a pulse of modulated width (PWM) or a sequence of digital words. Both of them can be easily integrated to an EEG recording system (PWM to voltage) or to a PC (digital words). The aforementioned tasks are performed in real-time. Finally, in case of using the device as a long-term wearable, the memory card mode is selected. Moreover, it supports experiments in which an external device (EEG recorder) is not advisable. Besides this, external inputs are


Fig. 2 Flow Chart Diagram of the code execution depicting the functional blocks followed according to the operating mode.

III.

RESULTS

Towards evaluating the device’s applicability, two approaches were followed. Initially, the measurement of a known electronic resistor took place in order to evaluate whether the device reliably measures voltage proportional to a resistance (skin resistance). The calculated (by the device) value of the known resistance had a negligible error in comparison to the real value. Then, skin conductance was recorded during the presentation of auditory stimuli. The stimuli were noxious sounds (white noise or intense tones) and unpleasant stimuli. The sounds duration ranged from 15 seconds. The experimental procedure consisted of 5 successive sounds. The inter-stimulus interval was set at 10 seconds. The overall experimental procedure was lasted for 1 minute. The device evaluation employed 15 healthy volunteers (13 males and 2 female subjects). The averaged electrodermal activity recorded as a response to the noxious stimuli was extracted. Figure 3 in-

4

dicatively illustrates the normalized EDA changes due to two stimuli are indicatively. The skin conductance signal is depicted with red line, while the onset and sound duration with the blue pulse. The sound stimulation evokes phasic SCRs while the SCL is also altered.

also introduces the notion of on-chip processing data processing and feature extraction by means of algorithmic steps executed by the microcontroller as firmware. Empowering an EDA sensor with such capabilities may extend its applicability to a plethora of applications ranging from emotion aware computing to neurophysuological experiments and healthcare systems.

1.

2. 3. 4. Fig. 3 EDA activity elicited by novel and noxious stimuli 5. IV.

DISCUSSION 6.

The proposed system was designed for facilitating a variety of applications. Among them, virtual gaming [12] and smart human-computer interfaces employ affective sensors for sensing the user’s state over short temporal windows. Response delays may cause user irritation and application’s doom. Therefore, the approach of on-chip processing and feature extraction is proposed in this study. Pre-processing is performed inside the micro-controller, which then employ pattern recognition techniques for sensing autonomic arousal alterations (both phasic and tonic). Several healthcare applications have been recently proposed for the detection of stress, anxiety, depression. Longterm recordings are required for obtaining reliable neurophysiological markers. Unobtrusiveness is a key issue for such applications, since the user should perform his daily activities without feeling frustration. So, the affective wearable might be miniaturized and light-weight. Our approach facilitates data acquisition over a long period by embedding a memory card. The storage of EDA features instead of raw data further extends the device’s independency and limits power consumption. Biomedical devices have been used during experimental procedures. Simultaneous recordings of various systems (respiratory, nervous, etc.) result in the adoption of several devices [13]. Their parallel function causes noise interference and artifact rejection. Data transmission to a host computer through cables enhances the signal contamination. The proposed device offers the possibility of noise-free transmission through a fiber-optic circuit performing data digitization and transfer through frequency modulation. This study proposed a miniaturized, light-weight skin conductance sensor able to store either raw EDA signals or SCR/SCL features without the need of a host computer. It


7. 8.

9.

10.

11.

12.

13.

REFERENCES Dawson M E, Schell A M and Filion D L (2001) The electrodermal system. In: Cacioppo J T, Tassinary L G, Berntson G G, editors. Handbook of psychophysiology. Cambridge. University Press: 53-84 Edelberg R (1972) Electrical activity of the skin. In Greenfield N S, Sternbach R A, editors. Handbook of psychophysiology:376-418 Boucsein W (1992) Electrodermal activity. New York. Plenum Press. Lang P J, Bradley M M and Cuthbert B. N. (1998) Emotion, motivation, and anxiety: brain mechanisms and psychophysiology. Biological Psychiatry, Vol. 44, Issue 12:1248–1263 Picard R W and Scheirer J (2001) The Galvactivator: A Glove that Senses and Communicates Skin Conductivity. Proceedings 9th Int. Conf. on HCI, 2001, New Orleans, USA, 2001 Frantzidis C, Bratsas C, Klados M, Konstantinidis E, Lithari C, Vivas A, Papadelis C, Kaldoudi E, Pappas C and Bamidis P (2010) On the classification of emotional biosignals evoked while viewing affective pictures: an integrated data mining based approach for healthcare applications. IEEE transactions on Information Technology in Biomedicine DOI 10.1109/TITB.2009.2038481 Picard R W and Healey J (1997) Affective Wearables. Personal technologies, Vol 1(4):231-240 Shastri A, Lomarev M P, Nelson S J, George M S, Holzwarth M R and Bohning D E (2001) A low-cost system for monitoring skin conductance during functional MRI. Journal of Magnetic Resonance Imaging, Vol. 14:187-193 Lagopoulos J, Malhi G S and Shnier R C (2005) A fiber-optic system for recording skin conductance in the MRI scanner. Behavior Research Methods, Vol. 37, Issue 4:657-664 Strauss M, Reynolds C, Hughes S, Park K, McDarby G and Picard R W (2005) The handwave Bluetooth skin conductance sensor. In Tao J, Tan T and Picard R W, editors. ACII, Lecture Notes in Computer Science, Springer, volume 3784:699-706, 2005 Frantzidis C A, Konstantinidis E I, Pappas C and Bamidis P D (2009) An Automated System for Processing Electrodermal Activity. Studies in Health Technology and Informatics, Vol. 150, ISBN 978-1-60750044-5, 2009 Konstantinidis E I, Luneski A, Frantzidis C A, Pappas C and Bamidis P D (2009) A Proposed Framework of an Interactive Semi-Virtual Environment for Enhanced Education of Children with Autism Spectrum Disorders, The 22nd IEEE International Symposium on Computer-Based Medical Systems, CBMS 2009, 3-4 August, Albuquerque, New Mexico, USA Konstantinidis E I, Bamidis P D and Koufogiannis D (2008) Development of a Generic and Flexible Human Body Wireless Sensor Network, in Proceedings of the 6th European Symposium on Biomedical Engineering (ESBME 2008) Author: Evdokimos Konstantinidis Institute: Lab of Medical Informatics, Aristotle University Street: Medical School, Aristotle University, PO Box 323, 54124 City: Thessaloniki Country: Greece E-mail:[email protected]