Classification of Hand Motions Using Linear ...

Classification of Hand Motions Using Linear Discriminant Analysis and Support Vector Machine Haibin Zeng1, Ke Li1,*

Xincheng Tian2, Na Wei3,4, Rui Song2, Lelai Zhou2

1. Laboratory of Motor Control and Rehabilitation, Department of Biomedical Engineering, School of Control Science and Engineering, Shandong University, Jinan, China * Corresponding author. Email: [email protected]

2. School of Control Science and Engineering, Shandong University, Jinan, China 3. Department of Geriatrics, Qilu Hospital, Shandong University, Jinan, China 4. Suzhou Institute of Shandong University

Abstract—In this study, we aimed to recognize six hand motions using surface electromyogram (sEMG) signals recorded from eight muscles of the right hand. Twenty-five healthy subjects participated in this experiment. Data were segmented using windows of 250-ms length with a 150-ms overlapping. In this paper, we extracted 24 features per muscle. Three feature sets -the original features, the features produced by a discriminant analysis (DA) and those selected by a multiple regression analysis (MRA) entered into one of the following classifiers: linear discriminant analysis (LDA) or support vector machine (SVM). The results showed that the original features classified by the SVM reached an average accuracy of 91.2 s 0.383 %, significantly higher than the other approaches. The index finger extension (IFE) had higher classification accuracy than the other hand motions. The probability of the thumb opposition (TO) falsely classified as key pinch (KP) was 1.1 %, that of the hand grasp (HG) falsely classified as four fingers flexion (FFF) was 1.0 %. Keywords—surface electromyogram; classification; hand motions; discriminant analysis; multiple regression analysis; linear discriminant analysis; support vector machine.

I. INTRODUCTION The human hand is an amazingly dexterous tool for a variety of complex actions. It is a challenge issue that how to control a robotic hand with a similar dexterity of natural human hand. To precisely predict the movement intention, the surface electromyogram (sEMG) were considered as reliably biosignals. A question is how to extract the pertinent features and to accurately predict hand motions using sEMG signals. Many papers addressed the classification of individual finger motions and wrist motions. The purposes of these studies were to recognize different actions using classifier and features extracted from the recorded electromyogram (EMG) signals. The EMG signals recoded from the intrinsic muscles cannot provide sufficient information to predict hand motions, whereas the EMG signals recorded by the extrinsic muscles showed better classification outcomes [1]. Regarding the features, except those common time-domain and frequency-domain features, in many studies the time-frequency domain features and nonlinear features are extracted [2, 3]. There are a number of classifiers available for hand motion recognition, such as the deep convolutional network [4], the linear discriminant analysis

978-1-5386-3524-7/17/$31.00 ©2017 IEEE

(LDA) [5], common spatial patterns proportional estimator [5], the support vector machine (SVM) [2], etc. These classifiers typically show different performance in respect of different datasets. The current study aimed to recognize six hand motions using sEMG signals recorded from the eight hand muscles. The LDA and SVM were used to classify the hand motions. The performances of the two approaches were compared in order to find out the most suitable features and classifier. II. METHODS A. Subjects Twenty-five healthy subjects participated in this experiment (13 males, 12 females, mean age = 23.2 ± 1.7 y) without any visual disorders or any musculoskeletal or neurological diseases of the upper-limbs. All subjects were right-handed. The handedness of each subject was determined by Edinburgh Handedness Inventory. The purpose and procedure were fully informed. All subjects gave their written consent. B. Experimental Set-Up The experiment selected 16 muscles of both hands: brachioradialis (BRA), flexor carpi ulnaris (FCU), flexor carpi radialis (FCR), extensor digitorum communis (EDC), flexor digitorum superficialis (FDS), abductor pollicis brevis (APB), first dorsal interosseous (FDI) and abductor digiti minimi (ADM). The sEMG signals of the 16 muscles were recorded by TrignoTM Wireless EMG System (Delsys Incorporated). Signals of 16 channels were simultaneously collected at a sampling frequency of 1000 Hz. The program for signal recording was designed in Labview (National instrument, Austin, TX). C. Procedure Each subject sat on a chair which was in the middle line 10 cm away from the lower edge of the testing table with their both hands placed on the start location. Moreover, all subjects should make sure their radial styloid was placed upon wooden mats below. Subjects were prompted to perform six individual hand motions, including hand grasp (HG), four fingers flexion (FFF),

2353

Fig. 2. A sEMG signal of the HG motion recorded from the muscle APB with two arrows pointing the start of hand grasp and hand open. Fig. 1. The experimental set-up and the locations of the sEMG sensors. The motion of this figure was the holding peroid of the KP motion.

index finger extension (IFE), thumb internal rotation (TIR), thumb opposition (TO), key pinch (KP). For the HG, FFF, TIR, TO and KP, the ready motion is the same: all five fingers open, four fingers together and holding an angle of 30 degrees between thumb and index finger. For the IFE, the ready motion is performed with middle finger, ring finger and little finger closed to palm, holding index finger open and a 30 degrees angle between thumb and index finger. Except for thumb opposition, each hand motion was held for 3-4 s with no force and repeated 10 times. Thumb internal rotation was repeated 10 times and contained 3 thumb internal rotation every time. Subjects rested for 7-10 s between each repetition. To avoid fatigue, subjects were allowed 1-2 minutes of rest between trials, where a trial consisted of one hand motion. Fig. 1 shows the experimental setup and the locations of the sEMG sensors. D. Feature Extraction Data analysis were performed using MATLAB 2015b software (The Mathworks, Natick, MA, USA). Fig. 2 shows a sEMG signal of the HG motion recorded from the muscle APB with two arrows pointing the start of hand grasp and hand open. Data were segmented using windows of 250-ms length with a 150-ms overlapping. In this paper, we extracted the following 24 features: • Time-domain features: mean value (MEAN), root mean square (RMS), variance (VAR), standard value (STD), mean absolute value (MAV), zero-crossings (ZC), waveform length (WL), slope sign change (SSC) Willison amplitude (WAMP), 6th-order autoregressive coefficients (AR1, AR2, AR3, AR4, AR5, AR6); • Frequency-domain features: mean frequency (MNF), mean power frequency (MPF), median frequency (MF); • Time-frequency domain features: level-3 discrete onedimensional wavelet decomposition, the maximum value of approximate coefficients of the last level (MAX_A3) and the maximum value of detail coefficients of level three (MAX_D3), level two (MAX_D2), level one (MAX_D1); • Nonlinear features: Lempel-Ziv complexity (LZC), Detrended fluctuation analysis (DFA).

TABLE I.

FUNCTIONS AT GROUP CENTROIDS Functiona

Hand motion

1

2

3

4

5

HG

-1.019

1.698

.679

.835

-.526

FFF

-2.680

.953

-.776

-1.077

.437

IFE

-2.939

-1.871

-.261

.643

-.162

TIR

3.224

.146

-1.495

.717

.174

TO

2.103

-.600

.298

-1.253

-.633

KP

1.311

-.325

1.555

.136

.709

a.

Unstandardized canonical discriminant functions evaluated at group means

E. Dimensionality Reduction and Features Selection Considering there are totally eight channels for one hand and 24 features in each channel, the feature sets for one hand contained 192 (24 * 8) features. For this huge feature sets, we employed two ways to reduce the numbers of the features: dimensionality reduction and features selection. Here, we used discriminant analysis (DA) to reduce the dimensionality of the feature set. In the DA, the feature sets were reduced and projected to c-1 features (rule discriminant functions) where c is the number of classes (hand motions). As for features selection, a multiple regression analysis (MRA) was employed. F. Classification To avoid the difference between right and left hand influence the classification result, we only chose the data of the right hand (8 channels) to classify the six hand motions. For the classification, we selected two classifiers: the LDA classifier, coarse gaussian SVM classifier. Coarse gaussian SVM is a support vector machine that makes coarse distinctions between classes, using the Gaussian kernel with kernel scale set to P × 4 where P is the number of predictors. Data were divided evenly into training data sets and testing data sets and each classifier was evaluated using two-fold cross validation. Statistical analysis was performed using SPSS 20.0 (SPSS Inc., Chicago, IL) to analyze the classification accuracy. A pvalue less than 0.05 was considered statistically significant.

2354

Fig. 3. A representative scatter plot of the discriminant result with group centroids using function 1 and function 2.

Fig. 4. Average classification accuracy of two classifiers (LDA and SVM) using three feature sets (original, DA and MRA) using two-fold cross validation. (*) represents p < 0.05 between the LDA and SVM, (**) represents p < 0.05 between two of the three feature sets. Error bars represent standard error. The significant differences between accuracy of the SVM using original and accuracy of all other approaches was found (p < 0.05).

III. RESULTS A. Dimensionality Reduction and Features Selection By employing the DA, 192 features of eight channels were reduced into five features. The functions at group centroids of the five canonical discriminant functions are shown in TABLE I. Fig. 3 shows the discriminant result with group centroids that using function 1 and function 2. The MRA that used the backward method to remove the features produced a feature set that contained 95 features, it removed 97 features. B. Classification Three feature sets --the original features, the features produced by the DA and those selected by the MRA entered into one of the following classifiers: the LDA or the SVM. Three feature sets produced by original feature sets, DA and MRA were put into two classifiers: LDA and SVM. Fig. 4 shows the average classification accuracy of two classifiers (LDA and SVM) using three feature sets (original, DA and MRA). The results showed that the original features classified by the SVM reached an average accuracy of 91.2 ± 0.383 %, significantly higher than the other approaches. For the original feature sets and the MRA, the SVM had a better classification accuracy than LDA. However, the LDA and SVM had the similar classification accuracy for the feature sets produced by the DA. Fig. 5 shows a representative classification accuracy of six hand motions through six classification methods that formed by one kind of classifier and data. It indicated that the IFE had a higher accuracy than the other hand motions.

Fig. 5. A representative classification accuracy of six hand motions through six classification methods that formed by one kind of classifier and data. The total accuracy are as followed: 64.9 % for LDA-Original, 84.1 % for LDA-DA, 80.5 % for LDA-MRA, 91 % for SVM-Original, 83.8 % for SVM-DA, 90.7 % for SVM-MRA.

that of the HG falsely classified as FFF was 1.0 %, and that of the FFF falsely classified as HG was 0.8 %. IV. DISCUSSION In this study, we recognized six hand motions using sEMG signals recorded from eight muscles of the right hand. We segmented data into 250 ms windows for feature extraction, dimensionality reduction and features selection. The LDA and SVM were chosen as the classifiers. The result demonstrated: • For the SVM, the classification that using original features has higher average accuracy of 91.2 % than the other approaches, the MRA can achieve the similar accuracy (90.7 %);

Fig. 6 and Fig. 7 shows the ROC plot and confusion matrix of one classification for the six motions with an accuracy of 91.1 %. They indicated that the IFE was classified with an accuracy of 96.2 % and area under curve (AUC) of 0.9718, and the TO has lower classification accuracy and AUC than the other hand motions. The probability of the TIR falsely classified as TO was 0.9 %, that of the TO falsely classified as KP was 1.1 %,

• For the SVM, the classification that using original features has higher average accuracy of 91.2 % than the other approaches, the MRA can achieve the similar

2355

Fig. 6. A representative ROC plot of one classification for the six motions with an accuracy of 91.1 %. The area under curve (AUC) of the HG is 0.9462, that of the FFF is 0.9454, that of the IFE is 0.9718, that of the TIR is 0.948, that of the TO is 0.901, and that of the KP is 0.936.

accuracy (90.7 %); • For the LDA, DA and especially MRA can both improve the classification accuracy; • For the six hand motions, the IFE has higher accuracy and ACU than the other hand motions. Maybe, the reason is that the IFE is the simplest motion that only uses the muscle FDI. The probability of the TIR falsely classified as TO, that of the TO falsely classified as KP, that of the HG falsely classified as FFF, and that of the FFF falsely classified as HG were higher than the others. It is possible that the sEMG signal of the APB always makes great significance in the hand motion TO, KP and TIR, which leads to the false predictions between TO, KP and TIR.

Fig. 7. A representative confusion matrix of one classification for the six motions with an accuracy of 91.1 %.

This study was supported by National Natural Science Foundation of China (31200744), Key Research & Development Programs of Shandong Province (2015GSF118127), China Postdoctoral Science Foundation (2014M560558, 2015T80723), Postdoctoral Innovation Foundation of Shandong Province (201401012), Young Scholars Program of Shandong University, Natural Science Foundation of Jiangsu Province (BK20170398). The authors give thanks to all the subjects for their participation in the experiment.

REFERENCES [1]

[2]

V. CONCLUSION In this study, six hand motions were recognized using sEMG signals of eight muscles. The original feature sets based on the 24 features per muscle together with the SVM classifier was shown to be effective to recognize six hand motions. An average classification accuracy of 91.2 % was achieved. In future, more features should be analyzed and more advanced classifiers need to be examined.

[3]

[4] [5]

ACKNOWLEDGMENT

2356

Adenike, A., Levi, J. & Todd, A. An Analysis of Intrinsic and Extrinsic Hand Muscle EMG for Improved Pattern Recognition Control[J]. IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, 24: 485-494, 2016. Khushaba, R. N., Al-Timemy, A., Kodagoda, S. & Nazarpour, K. Combined influence of forearm orientation and muscular contraction on EMG pattern recognition[J]. Expert Systems with Applications, 61: 154161, 2016. Ganesh, R., Ali, H. & Hung, T. Transradial Amputee Gesture Classification Using an Optimal Number of sEMG Sensors_ An Approach Using ICA Clustering[J]. IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, 24: 837-846, 2016. Geng, W. et al. Gesture recognition by instantaneous surface EMG images[J]. Scientific reports, 6: 36571, 2016. Celadon, N., Dosen, S., Binder, I., Ariano, P. & Farina, D. Proportional estimation of finger movements from high-density surface electromyography[J]. Journal of neuroengineering and rehabilitation, 13: 73, 2016.