An SVM Classifier for Fatigue-Detection using Skin ...

An SVM Classifier for Fatigue-Detection using Skin Conductance for Use in the BITS-Lifeguard Wearable Computing System Mahesh M. Bundele

Rahul Banerjee

Department of Computer Science & Engineering Babasaheb Naik College of Engineering Pusad, India [email protected]

Computer Science Group Birla Institute of Technology & Science Pilani, India [email protected]

Abstract—Monitoring driver fatigue, inattention, drowsiness and alertness is very important in order to prevent vehicular accidents. The system detecting and monitoring should be noninvasive type and non-distracting to the driver. The physiological parameters such as skin conductance, oximetry pulse, respiration, SPO2 and BVP can lead to the acceptable solution to the problem. The author is working on the subset of the project ‘BITS Life Guard system’ and trying to correlate the fatigue of a driver with the set of physiological parameters so as to fulfill the requirements. This paper is an attempt towards finding the correlation of skin conductance with the fatigue of a driver. Artificial Neural Network approach is used to design the system by taking actual body parameters of the drivers under different state of work & environment. Multilayer Perceptron (MLP) Neural Network (NN) and the Support Vector Machine (SVM) are used to correlate the driver’s fatigue level with skin conductance. Two state classifiers were designed and tested with 18 input features for 2392 total data rows and found that SVM gives a better Classification Accuracy. The performance measures used for designing are Percentage Classification Accuracy (PCLA), Mean Square Error (MSE) and Receiver Operating Characteristics (ROC). Keywords-Fatigue Detection; Skin Conductance; Multilayer Perceptron Neural Networks; Support Vector Machine; Wearable Computing;

I.

INTRODUCTION

All Fatigue is a complex phenomenon, widely used term, related to a physiological and psychological state of human beings. It is used as a synonym for drowsiness and tiredness and often mistakenly associated with sleepiness. Both, fatigue and sleepiness are characterized by a decrease in memory, reaction time, information processing, decision making and vigilance, but one can be fatigued without being sleepy [1]. Fatigue is discriminated as sensory, muscular and cognitive fatigue. Some researchers have categorized the influence of fatigue to physiological states as endogenous and exogenous factors. The endogenous factors are associated with long term fluctuations of alertness that emanate from within the organisms and affect state of an individual while performing a specific task like driving. Long time working, late night or mid afternoon driving and sleep deficits are the examples of endogenous precursors that are harmful to the driving performance. The investigations regarding the relation of fatigue and driving performance are mainly oriented towards the endogenous

fluctuation of alertness and sleep research whereas the exogenous factors and the relationship of driver and environment are mostly discounted. The physiological parameters of the body like EEG, ECG, EOG, EMG, Heart rate, BVP, Skin Conductance, SPO2 level, Respiration and Respiration rate etc. are directly correlated to endogenous factors. The fatigued driving can also be correlated with driver’s physiological behavior and vehicle behavior. The systems have been built up by using physiological behavioral analysis of the driver such as eye blinking rate, eye gaze, yawning frequency, head nodding, driver posture analysis, facial features and the vehicular behavioral analysis like steering wheel movement, lane departure, pressure on the accelerating pedal, the accelerating and decelerating behavior etc. JAIO Feng uses an active remote infrared illumination for real time tracking of eyes and nostrils. The features are extracted from 3D face model of a driver with the help of image processing [3]. Hu Shuyan [6] has used SVM for the detection of drowsiness of a driver by measuring eyelid related parameter. The eyelid parameters were extracted from EOG data collected in a driving simulator. Muhammed B. Kurt [7] has developed a method for automatic estimation of vigilance level by using EEG, EMG, EOG recorded during transition from wakefulness to sleep. Wavelet transform was used to separate EEG sub band frequency signals. Artificial Neural Network ANN is used for three state classification as awake, drowsy and sleep. Mervyn V M has worked on automatic detection of drowsiness using SVM from EEG of the driver [8]. One of the requirements of the BITS Life Guard system is to obtain an optimal set of physiological parameters which does not disturb the driver’s comfort and hence the parameters like skin conductance, oximetry pulse, respiration and respiration rate, SPO2 are taken as primary parameters to work with [2]. This paper presents comparative performance analysis of MLP NN and SVM NN for correlating the fatigue of a driver with skin conductance as a body parameter. II.

DATA COLLECTION AND PROCESSING

A. Data Collection After making a thorough study of existing systems and their limitations, it has been decided to measure the Physiological body parameters such as Skin Conductance,

Respiratory, SPO2, Pulse Oximetry of the drivers driving different vehicles such as State Transport Bus, Truck, Taxi Cars etc of varying age groups and working in different environments. The skin conductance signal was recorded for 3-4 minutes for pre-driving and post driving states of a driver by using Nexus-10, a biofeedback signal recording and processing equipment, B. V. Mind Media Schepersweg, The Netherlands. It is a 10-channel physiological monitoring and biofeedback platform that utilizes Bluetooth 1.1 class 2 wireless communication and flash memory technologies. It uses a Bio-trace software for recording & processing various body parameters. Fig. 1 shows the skin conductance sample recorded for pre-driving and postdriving states of a taxi driver from the city of Nagpur, prior to his start and after completion of return trip of 550 Kms. on single a day. The measurements of these parameters were taken for the drivers at different places like Pilani, Pusad and Amravati. The drivers were aged between 25-55 years and drove trucks, buses and taxi cars. Pre-driving measurements were taken when a driver was fresh and was about to begin driving. The post-driving measurements were taken immediately after the driver completed return trip of at least 100-600 km. B. Feature extraction As the focus of this work is to prevent road accidents, detection of fatigue / drowsiness level in the smallest possible time was desirable. Therefore signals recorded were divided into two-second frames and the feature extraction was carried out using Matlab. These features included: Mean of Signal (MOS), Standard Deviation of Signal (STDEVS), Frame Energy (FE), Maximum Frequency (MAXF), Standard Deviation of Frequency Spectrum (STDFS) and Mean of Frequency Spectrum (MOFS). In addition to these six basic features, twelve more features were extracted by computing gradients the slope of the each of these features. These features were extracted from skin conductance signal recorded at a sampling frequency of 32..

Fig.1 Skin Conductance Sample of a driver from Nagpur City under Pre and Post Driving Conditions

For this purpose 10 subjects data was selected for predriving and post driving state and the Eighteen features were extracted as discussed before. Total 1201 rows for predriving features and 1182 rows for post driving features were obtained. The scatter plots for 18 features were plotted. It was found that most of the features were overlapping and inseparable through conventional approaches. Many of the features possess distinct features for predriving and post-driving state of a driver, but they are not linearly separable. This has led to choose the Neural Network Approach for separating the data using Two State Classifier. The scatter plot shown in Fig. 2 illustrates the nonlinearity of separation and the need for the neural network analysis. It is showing the relationship between slope of FE and gradient of MOFS for pre and post-driving clusters which are linearly non-separable. C. Preparation of Training, Cross validation, Test Data Partitions Sets of features of skin conductance signal from predriving and post-driving states were collected in a file for 2383 records. The 18 features of each record were extracted. These 2383 sets of features are then used as inputs to the neural network for the classification of the fatigue level. Three different data partitions were used with different tagging orders. In the first case, the first 50 % samples (1:1191) are used for training, the second 25 % samples (1192:1788) for cross validation, and the third 25 % samples (1789:2383) for testing of the classifier. In the second case, the last 50 % samples (1193:2383) are used for training, the second 25 % (597:1192) for cross validation, and first 25 % samples (1:596) for testing of the classifier. The number of output classes are two. This implies that the number of PEs required in the output layer of the neural network based classifier is two.

Fig.2 Scatter Plot of Features of Skin Conductance

III.

EXPERIMENTATION

The features extracted from skin conductance signal were used as input to various neural networks. The details of MLP NN and SVM NN has been discussed below. A. Design of MLP NN Classifier The primary advantage of using the MLP NN for approximation of the mapping from input to the output of the system is its simplicity and suitability for real time applications [12], [13]. The choice of the number of hidden layers and the number of units in each of the hidden layers is critical [16]. It has been established that a MLP NN that has only one hidden layer, with a sufficient number of neurons, acts as universal approximators of non-linear mappings [10]. Experimentally, it can be verified that the addition of extra hidden layer can enhance the discriminating ability of the NN model. However, it does so at the cost of the added computational complexity. The trade-off between accuracy and complexity of the model should be resolved carefully. The single and two hidden layer MLP NN were designed by adopting independent validation method and ROC analysis method. The network parameters were varied as depicted in Table 1. The learning and generalization ability of the estimated Neural Network based classifier is assessed on the basis of certain performance measures such as MSE, NMSE, the rate of correct classification and the area under ROC curve. B. Single Hidden layer MLP NN Classifier Single hidden layer MLP NN is designed to give optimal performance on the basis of the best PCLA. The dataset of 2383 records was divided into three parts in the ratio 2:1:1, first part of data was used for training the network, second part of the data was used for cross validation and the third part of the data was used for testing the network. The network with 18 inputs and 2 outputs PEs with TANH transfer function (TF) and Momentum Learning Algorithm (LR) was trained three times and tested for classification accuracy for test, cross validation, and training datasets. The process was repeated by varying hidden layer PEs from 2 to 100 for number of epochs set to maximum of 5000. TABLE I. VARIABLE PARAMETERS USED FOR DESIGNING MLP NN Parameter Number of hidden layers Number of hidden neurons Learning-rate parameter Momentum constant TF in Hidden & output layers Learning Rules Number of Epochs

The variation in MSE and PCLA were recorded for Test, Cross Validation, and Training datasets. Further for the maximum value of PCLA, the number of neurons in hidden layer was freeze and the network was trained and tested by varying Learning Algorithms and the transfer functions as indicated in Table 1. The single hidden layer MLP NN with 20 hidden PEs, TANH TF and LMQ LR has performed the best amongst all the networks and the Table 2 shows the performance measures obtained for optimal single hidden layer MLP NN classifier. C. Two Hidden layer MLP NN Classifier A two hidden layer MLP NN was designed for the best classification accuracy and therefore design process followed for single hidden layer has been adopted. The MLP NN having 2 hidden layers, TANH TF and MOM LR at hidden layers and output layer with 18 inputs and 2 outputs was trained for three times by giving pre-driving and post-driving feature matrix derived from skin conductance as input to the network. Total dataset of size 2383 x 18 was divided into three parts in the ratio 2:1:1, first part used as training dataset, second as cross validation and third as testing dataset. As the hidden layers in a Neural network increases, the complexity of computation also increases. Here the network is designed by keeping L1 PE fix to 5 and varying L2 PE from 5-100 in steps of 5. Then step by step the L1 PE was also varied from 5-100 in steps of 5 varying simultaneously the L2 PE. Separate networks were designed to compare the performance of MLP NN while L1 & L2 PE are varied. After training the network three times with each set of PEs, the network was tested for test dataset, cross validation dataset and training dataset. The performance of the network was recorded as percentage classification accuracy (PCLA) and MSE for various datasets. Each time maximum number of epochs set for training was 5000. It is found that the PCLA for two hidden layer MLP NN is improved to 87.04 % as compared to 85.54 % for one hidden layer MLP NN. The Fig.3 shows PCLA for test dataset, CV dataset and training dataset versus hidden PE variation for L1 & L2 from 2-100. It is clear that the best classification accuracy 87.04 % for test dataset is obtained for L1 PE 30 & L2 PE 50 with TANH TF & MOM LR. Fig. 4 shows MSE versus hidden PE variation for L1 & L2.

Typical Range (1 to 3) (2 to 100)

TABLE II. PERFORMANCE MEASURES OF ONE HIDDEN LAYER MLP NN  ON TESTING INSTANCES  Performance Set 1: Forward Tagging Set 2 : Reverse Tagging Measures

(0 to 1) (0 to 1) Tanh, LinTanh, Sigmoidal, Linear Sigmoidal, Bias Axon,Linear Axon, Softmax Axon, Axon Momentum, conjugate-gradient, Levenberg Marquardt, step, quick-propagation, Delta-Bar-Delta Maximum 5000

Status(Post)

Status(Pre)

Status(Post)

Status(Pre)

MSE

0.08898149

0.088487

0.10404063

0.10415

NMSE

0.3575364

0.35555

0.41683846

0.417275

PCLA

91.509434

87.41007

89.8601399

85.16129

AROC

0.933029 and 0.93616 for convex hull

0.930251 and 0.934062 for convex hull

TABLE IV.  PERFORMANCE MEASURES OF TWO HIDDEN LAYER MLP  NN ON TESTING INSTANCES  Performance Set 1: Forward Tagging Set 2 : Reverse Tagging Measures Status(Post) Status(Pre) Status(Post) Status(Pre)

MSE

0.08898149

0.088487

0.10404063

0.10415

NMSE

0.3575364

0.35555

0.41683846 0.417275

PCLA

91.509434

87.41007

89.8601399 85.16129

AROC

0.944533 and 0.946728 for convex hull

0.934412 and 0.935771 for convex hull

Fig.3 Two Hidden Layer MLP NN: PCLA vs Hidden Layer L1 and L2 PE

Fig.5 Two Hidden Layer MLP NN: MSE vs Epochs Fig.4 Two Hidden Layer MLP NN: MSE vs L1 and L2 PEs

TABLE III. OPTIMAL PARAMETERS OF TWO HIDDEN LAYER MLP NN CLASSIFIER Parameter Hidden Layer-1 Hidden Layer-2 Processing Elements Transfer Function Learning Rule

Output Layer

30

50

2

TANH

TANH

TANH

LevenbergMarquardt

LevenbergMarquardt

LevenbergMarquardt

The network was further refined by changing the Learning Rule Algorithms such as MOM, CG, LMQ, QP, DBD and Step. Fig. 5 shows the comparative analysis of Learning Algorithms. The two hidden layer MLP NN with LMQ LR was trained and tested by varying number of Epochs from 50 to 600 and the L1 PE from 5 to 30 with L2 PE at 50 and the performance was recorded for test dataset, CV dataset and training dataset. It is found that the PCLA is further increased to 89.45, 91.24 and 94.06 % for three datasets respectively while trained with TANH TF, LMQ LR, L1 PE 10 & L2 PE 50 in 400 Epochs. The network was further refined by changing the transfer functions LMQ LR and the best performing TF-LR was found as TANH-LMQ.

Table 3 shows the set of optimal which has fetched maximum PCLA and shows the performance measures.

parameters Table 4

D. Design of SVM Neural Network Classifier The support vector machine (SVM) is a new kind of classifier that is motivated by two concepts. First, transforming data into a high-dimensional space can transform complex problems (with complex decision surfaces) into simpler problems that can use linear discriminant functions [20]. Second, SVMs are motivated by the concept of training and using only those inputs that are near the decision surface since they provide the most information about the classification. The first step in a SVM is transforming the data into a high dimensional space. This is done using a Radial Basis Function (RBF) network that places a Gaussian at each data sample. Thus, the feature space becomes as large as the number of samples. The RBF, however, uses back-propagation to train a linear combination of the Gaussians to produce the final result. The SVM, however, uses the idea of large margin classifiers for training. This decouples the capacity of the classifier from the input space and at the same time provides good generalization. This is an ideal combination for classification. The learning algorithm is based on the Adatron algorithm extended to the RBF network. Here the classifier is designed by using the same datasets as per

Table 1 training the network three times with training dataset by varying number of epochs and recording the ROC values corresponding to test data set and train dataset and computed AROC for each epoch value. Fig. 6 shows the PCLA variation for test, CV and training datasets with respect to number of epochs and the Fig. 7 shows corresponding variation in MSE. It was observed that maximum testing PCLA is obtained at 50 epochs with a lower value of MSE. The plot of area under ROC curve with reference to variation in number of epochs is shown in Fig. 8. It was observed that AROC value for 50 epochs is maximum and near unity. The ROC and its convex hull for test datasets are shown in Fig. 9 and 10. Table 5 indicates the performance measures of the SVM NN trained and tested for 50 epochs. Table 6 displays the confusion matrix for testing instances of datasets 1and 2.

Fig.7 SVM NN Classifier: MSE vs Epochs

TABLE V.  PERFORMANCE MEASURES OF SVM NN CLASSIFIER ON TESTING  INSTANCES  Performance Set 1: Forward Tagging Set 2 : Reverse Tagging Measures Status(Post)

MSE NMSE PCLA AROC

Status(Pre)

0.084243633 0.084236255 0.33698971 0.3369602 93.3333333 92.56756757 0.968671 and 0.971453 for convex hull

Status(Post)

Status(Pre)

0.089498621 0.089500964 0.358321316 0.358330698 88.59934853 91.69550173 0.960664 and 0.963781 for convex hull

Fig.8 SVM NN Classifier: AROC vs Epochs

ROC and its Convex hull for SVM Classifier on Test dataset (Case 1) 1 0.9

Fig.6 SVM NN Classifier: PCLA vs Epochs

TABLE VI. CONFUSION MATRIX FOR SVM NN ON TESTING INSTANCES (  NUMBER OF EPOCHS=50)  Output/Desired Set 1: Forward Tagging Set 2: Reverse Tagging Status (Post) Status (Pre) Status (Post)

Status (Pre)

Detections (Sensitivity)

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1

Status (Post) Status (Pre)

280 20

22 274

272 35

24 265

0

0

0.2 0.4 0.6 0.8 False Alarms (1-Specificity)

Fig.9 ROC Curve for SVM NN ( Case 1 )

1

Fig.10 ROC Curve for SVM NN (Case 2)

IV.

CONCLUSION

The comparative analysis of MLP NN and SVM NN for two state classification problem of fatigue of a driver using individual validation and ROC analysis method, it can be concluded that SVM NN performs better than MLP NN, supports the classification of pre-driving and post driving states of a driver more accurately. Out of 300 post driving skin conductance signals, SVM NN has identified 280 as post-driving whereas 20 signals identified as pre driving and out of 296 pre-driving skin conductance signals, 274 were identified as pre driving while 22 were identified as postdriving for dataset 1 giving average PCLA 92.95 % as compared to 89.45 % testing accuracy of MLP NN. Similarly out of 307 post-driving skin conductance signals, 272 were identified as post-driving whereas 35 were identified as pre-driving and 289 pre-driving skin conductance signals, 265 were identified as pre-driving while 24 were identified as post-driving for dataset 2 giving average PCLA of 90.147 % as compared to 87.51 of MLP NN showing the consistency and robustness of the network. Further AROC for ROC curve and convex hull, obtained for SVM NN are 0.968671 and 0.971453 respectively are near unity value as compared to 0.944533 and 0.946728 of MLP NN. Hence, the skin conductance can be used as one of the supporting parameters for the detection of fatigue of a driver. ACKNOWLEDGEMENT The authors are thankful to Nadir N. Charniya, Department of Electronics & Telecommunication, Babasaheb Naik College of Engineering Pusad, 445215 District-Yavatmal, MS for the technical support. REFERENCES [1] T. Brandt, R. Stemmer, B. Mertsching, and A. Rakotonirainy, “Affordable visual driver monitoring system for fatigue and monotony,” IEEE, International Conference on Systems, Man and Cybernetics, The Hague, 2004

[2] Mahesh M. Bundele, “Identification of body parameters for changes in reflexes of a vehicular driver under drowsiness / fatigue / stress conditions,” Published in the Proceeding of FRONTIER 2008, A National Conference organized by Babasaheb Naik College of Engineering Pusad, MS India, Pp 123-131. [3] JIAO Feng, TU Jin, LI Ming-biao, and HE Gui-ming, “Locating eye and nose features precisely in IR video sequences for predicting driver fatigue,” Journal of Communication and Computer, USA, vol. 4, No.1 (Serial No.26), January 2007. [4] Rahul Banerjee, “From research to classroom: A course in pervasive computing,” IEEE Pervasive Computing, July-September 2005, Vol. 4, No. 3, pp.83-86. [5] J. Principe, N. Euliano, and C. Lefebvre, Neural and adaptive systems: fundamentals through simulations, John Wiley and Sons, 1999. [6] Hu Shuyan, and Zheng Gangtie, “Driver drowsiness detection with eyelid related parameters by support vector machine,” Expert Systems with Applications, Elsevier 36 (2009) pp.7651–7658. [7] Muhammed B. Kurt a, Necmettin Sezgin a, Mehmet Akin a, Gokhan Kirbas b, and Muhittin Bayram a, “The ANN based computing of drowsy level,” Journal of Elsevier Expert Systems and Applications, 2009 pp.2534-2552. [8] Mervyn V.M. Yeo a, Xiaoping Li , Kaiquan Shen , Einar P.V., and Wilder-Smith, “Can SVM be used for automatic EEG detection of drowsiness during car driving?,” Journal of Safety Science, Elsevier 47 (2009) pp.115–124. [9] Abdulhamit Subasi, “Automatic recognition of alertness level from EEG by using neural network and wavelet coefficients,”, Expert Systems with Applications 28 (2005) pp.701–711, Elsevier. [10] Nadir N. Charniya, and S. V. Dudul, “Neural network based sensor for classification of material type and its surface properties,” Proceeding of International Joint Conference on Neural networks, pp. 424-429, Orlando, USA, August 2007. [11] Nexus-10 Hardware & Software Manual, Published by Mind Media B. V. [12] R. P. Lippmann, “An introduction to computing with neural nets,” IEEE ASSP Magazine, pp.4-22, April 1987. [13] E. B. Baum, “On the capabilities of multilayer perceptions, Journal of Complexity,”, vol. 4, pp.193-215, 1988. [14] P. Arpaia, P. Daponte, D. Grimaldi, and L. Michaeli, “ANN – based error reduction for experimentally modeled sensors,”, IEEE Trans. on Instrumentation and Measurement, vol. 51, no. 1, pp.23-29, Feb. 2002. [15] S. Haykin, “Neural network: A comprehensive foundation,” McMillan, New York, 1994. [16] I.V. Turchenko, “Simulation modelling of multi-parameter sensor signal identification using neural networks,” Proceedings of the 2nd IEEE international conference on Intelligent Systems, Sofia, Bulgaria, pp.48-53, June 2004. [17] P.M. Gopych, “ROC curves within the framework of neural network assembly memory model: some analytic results,” International Journal of information theories & applications Vol.10 pp. 189-197. [18] James M. DeLeo, “The receiver operating characteristic function as a tool for uncertainty management in artificial neural network decision-making,” US Government work not protected by U.S. copyright. [19] Kevin Woods, and Kevin W. Bowyer, “Generating ROC curves for artificial neural networks,” IEEE Transactions on medical Imaging, Vol. 16, No. 3, June 1997 pp.329-337. [20] Suresh S. Salankar, and Balasaheb M. Patre, “SVM based model as an optimal classifier for the classification of sonar signals,” International Journal of Computer, Information, and Systems Science, and Engineering 1;1 Winter 2007 pp.68-76.