Study on Online Gesture sEMG Recognition

Study on Online Gesture sEMG Recognition Zhangyan Zhao1, Xiang Chen1, Xu Zhang1, Jihai Yang1, Youqiang Tu1, Vuokko Lantz2, and Kongqiao Wang3 1

Department of Electronics Science & Technology University of Science & Technology of China, Hefei, China, 230027 [email protected] 2 Nokia Research Center, Interaction CTC, Interacting in Smart Environments P.O. Box 407, FI-00045 Nokia Group, Finland [email protected] 3 Nokia Research Center, NOKIA (CHINA) Investment CO., LTD., Beijing, 100013

Abstract. We have realized an online gesture recognition platform for hand gestures using 2-channel surface EMG signals acquired from the forearm. Several features, such as AMV, AMV ratio and fourth-order AR model coefficients are extracted from the sEMG signal and the gesture segments are recognized with a Weighted Euclidean Distance Classifier. An above 90% recognition rate has been achieved with only a 400 μs recognition time. The methods developed in this study are aimed to be applied in a fast-response sEMG control system and be transplanted into an embedded microprocessor system. Keywords: surface EMG, online gesture recognition.

1 Introduction At present, there are many alternative methods suitable for the capture and recognition of hand gestures, such as video signal and image analysis techniques, data gloves [1] with position sensors, or surface electromyogram (sEMG) sensors [2] and various pattern recognition algorithms. Recognition approaches which are based image analysis are agile in the sense that they can recognize many different kinds of gestures but they can adapt poorly to the changes in the environment. Gesture recognition with data gloves do not suffer from such adaptability problems and thus provides steadier recognition performance but the manufacturing price is much higher. Surface EMG -based gesture capture is easy to set up and the signals do not depend on the environmental factors. However, the gesture recognition is based on indirect information as the sEMG sensors measure muscle activities and not the movements of the hand and fingers themselves. Therefore, in order to achieve acceptable recognition performance with sEMG sensors, the feature extraction and classification methods need to be designed carefully. A sEMG-based hand gesture recognition system can be used for controlling other devices via a computer interface. There are various applications areas in the realm of HCI: artificial limbs, sport electronic devices, virtual reality games, to name a few. online gesture recognition is a prerequisite for the practical application of sEMG-based D.-S. Huang, L. Heutte, and M. Loog (Eds.): ICIC 2007, LNCS 4681, pp. 1257–1265, 2007. © Springer-Verlag Berlin Heidelberg 2007

1258

Z. Zhao et al.

control. There exits promising prior research work on this: Jun-UK Chu have achieved 97% recognition accuracy with 100 ms recognition delay for 8 gestures using 4 electrodes [2]; Jong-Sung Kim and Mina Agarabi have realized PC mouse control by sEMG online recognition [3][4]. In this study, we have realized an online (real-time) sEMG recognition system with 2 electrodes. The system includes the sensor electrodes, signal amplifiers and preprocessing filters, and data acquisition, gesture action detection, feature extraction and gesture classification units, as shown in Fig.1. sEMG Electrodes

Data acquisition & caching

Feature Extraction

Amplifiers & filters

Action Detection

Learning or Classification

Fig. 1. Digram of the online sEMG gesture recognition system

2 Methods 2.1 Electrodes, Data Acquisition and Data Caching Generally, the dominant energy of sEMG signal is in the range of 20–500Hz, and the sEMG’s amplitude is limited to 0–10mV. Therefore, we used a 2-active-surfaceelectrodes system with inbuilt 20-1000Hz band-pass filters and 60dB-gain amplifiers

Fig. 2. DELSYS sEMG electrode [8]

Study on Online Gesture sEMG Recognition

1259

(BAGNOLI-2 by DELSYS). The sEMG sensors consist of a pair of line-shaped differential electrodes with 1mm x 10mm contact area and 10mm electrode-toelectrode spacing (Fig.2). We have evaluated the basic signal preprocessing characteristics of the DELSYS system, the analysis results are illustrated in Fig.3. Obviously, the system has built-in filtering with a 20-500Hz bandwidth and the signal to noise ratio (SNR) is above 30dB. Thus no additional filtering or noise reduction is needed for the sEMG measurements prior to gesture recognition.

Fig. 3. SEMG signals characteristics of the DELSYS electrodes

The hand gestures considered in this work consist of different finger actions (see Fig. 6 for the illustrations). Therefore, the selection of electrode positions in this study is based on the anatomy of the muscles related to finger action. We placed the two electrodes on the anterior aspect of the forearm as shown Fig. 4. Electrode 1 is sensitive to sEMG signals from muscles M.extensor indecis, M.extensor pollicis brevis, M.extensor pollicis longus and M.abductor pollicis longus; Electrode 2 is sensitive to sEMG signals from the muscles M.extensor digitiquinti proprius, M.digitorum communis and M.extensor carpi ulnaris. These muscles have important roles in various finger and hand actions. The 2-channel sEMG signal was digitized by a PCI analog-to-digital card (PCI6010 by NI) with 16bit resolution and 1-2 kHz sampling rate. SEMG data can be read continuously from the NI card but a continuous read-data-loop will occupy too much CPU time. Thus we set a timer for interruptions in every 10-50ms. During the timer interruptions, our customized data acquisition program reads a fix-sized data block to a cache array. The timer-interrupt mode reduces the CPU occupation significantly.

1260

Z. Zhao et al.

Fig. 4. Electrode positions on the forearm

2.2 Gesture Action Detection During a gesture action, the amplitude of the sEMG signal is much higher than in relaxation state. In order to detect gesture action segments, we calculated the 64-point moving average of the squared sums of the two sEMG signal values (see Eq. 1). In Equation (1), the variables Electrode1n and Electrode2n denote the nth sample values of the two sEMG data channels. MA( n ) =

1 63 ∑ ( Electrode1n−i + Electrode2n−i ) 2 64 i = 0

(1)

When the MA value reaches up to a certain threshold, the sEMG signal will be considered as gesture action signal. We designed a round queue structure for the MA

Fig. 5. Action Detection Queue


1261

operation to simplify its time complexity and the program structure. As shown in Fig.5, for each data sample point, the squared 2-channel-data sum updates the auxiliary variable Q[i] and the comparator gives a detection results after only an addition, a subtraction and a comparison operation. 2.3 Feature Extraction Features such as AR model coefficients, cepstrum, absolute integral, DFT and wavelet transformation coefficients [5] have produces good recognition performance for sEMG gesture activation signal. Not aiming only for a high recognition rate but also for a fast calculation, we extracted 3 kinds of features of from the sEMG signal to make 9-dimensional feature vectors as following: Absolute Mean Values Ratio (AMVR) of the 2 channels (1 feature); Absolute Mean Values (AMV) of both channels (2 features); 3 coefficients of fourth-order AR model for each channel (6 features). The AR model coefficients were calculated with Burg algorithm. 2.4 Learning and Classification Many pattern recognition methods have been applied to the sEMG gesture classification problem. Generally speaking, linear classifiers require only simple arithmetic operations and have steady classification performance whereas non-linear classifiers, such as neural networks and SVMs, provide higher classification accuracy but involve much more complex calculation operations [6][7]. In order to reach a high classification speed with a satisfying classification rate, a Weighted Euclidean Distance Classifier was adopted in our research work. Euclidean Distance Classifier is defined by the following parameters: the mean value and the standard deviation of each of the sEMG signal features. For the estimation of these statistical parameters, several tens of gesture action samples are needed for each gesture class. Typically, 10-50 samples per class are sufficient. The class-specific parameters are saved as gesture feature files. Euclidean Distance Classifier is computationally simple and thus its operation is very fast. However, it treats all the features equally and cannot identify the gesture-class characteristic features that are most effective in the gesture classification. Therefore, after all the gesture feature files were formed, we calculated the standard deviation wf (Eg. 2) of each feature f between the different gesture action classes and used them as feature weights.

w f = σ [ E ( x f ,act1 ), E ( x f , act 2 ),..., E ( x f ,actn )]

(2)

After gesture action sEMG data arrives and the 9 features (xf,actr) are extracted, the gesture classification process begins. The distances between the observed gesture action and each of the reference actions, defined by the parameters saved in gesture feature files, are calculated according to Eg. 3. Finally, we classify the observed gesture into the gesture class corresponding to the minimum Weighted Euclidean Distance.

Dactr ,actn =

9

∑[w f =1

f

(

x f ,actr − E ( x f ,actn )

σ ( x f ,actn )

)2 ]

(3)

1262

Z. Zhao et al.

3 Experiments and Results 3.1 Introduction to the Experiments We have designed an online sEMG gesture recognition platform (Fig. 6) and used it for performing recognition experiments with 4 subjects (2 men and 2 women, including 2 novice users). The sampling rate of the sEMG signal was set to 1kHz while the amplifier gain was 60dB. The platform supports four gesture classes, and we chose to experiment with two groups of gestures: 4 single finger actions and 4 combined finger actions. Illustrations of these gestures are shown in Fig. 7. As can be seen from Fig. 8, the sEMG signals of different gestures actions show some differences in their waveform and amplitude. The classifier parameters were learned from 20 repetitions of each gesture action. The recognition performance of the platform was tested with the electrode in the same positions as in the gesture learning phase.

Fig. 6. Illustrations of the sEMG gesture recognition platform. The window on the top is the interface for classifier parameter learning, and the window on the bottom shows the gesture classification interface.


1263

Fig. 7. Two groups of studied gesture action. The four gestures in the top row are single finger actions and the four gestures in the bottom combined finger actions.

Fig. 8. Examples of the sEMG waveforms and moving averages of the 8 gestures

3.2 Recognition Speed CPU occupancy: The CPU occupancy factor is less than 1% when the platform works in the PC environment of AMD Athlon 3200+, 1GB RAM and at the sample rate within 30kHz.

1264

Z. Zhao et al.

Action detection speed: Gesture action detection is performed for each data sample. However, the effective sEMG data comes 32ms later than the actual signal, due to the 64-point online frontward moving averaging. Feature extraction and recognition speed: Feature extraction and classification form the core of the sEMG gesture recognition. We measured the feature extraction and classification time with the 3.579545MHz PC system clock. As shown in Fig. 9, the calculation time can be approximated with a liner function of the action duration. The feature extraction and classification time is within 300μs and 450μs whereas the gesture duration varies typically between 0.25s and 1.5s.

Fig. 9. Feature extraction and classification time, showing a linear relation to the action time Table 1. Recognition results for 4 subjects and for two gesture groups, four gestures in each group


1265

3.3 Recognition Results The recognition result obtained for the 4 subjects were similar to each other. Tab.1 shows the recognition results for all of them. Each line shows an action done while each row shows an action recognized as. The recognition results of the 4 subjects are listed separately. As we can see, the recognition accuracy is fairly high, about 92.4%.

4 Conclusion We have designed a system for the recognition of 4 kinds of gestures. Gesture actions are captured with sEMG sensors and the recognition platform is realized with a series of fast arithmetics for data caching, action detection, feature extraction and classification. The devised system can achieve a 400μs recognition time and approximately 90% recognition accuracy for two different sets of four finger gesture actions. Other researches usually use tens even hundreds of microseconds to finish the recognitions, so the system in this paper is much faster than usual. However, it can only recognize 4 gestures at a time compared with about 6 as usual. Gesture recognition within 1ms is important for many practical applications. For example, EMG controlled precise robot arms, PC games and micro HCI systems based on microprocessors require high recognition speeds. Compared to other offline and online sEMG gesture recognition methods, our system utilizes fewer sEMG electrodes and is based on simpler arithmetic operations. It gained a high recognition speed but brought a certain limitations to the recognition accuracy. So in future, we are going to place more research effort on the optimization of the placing of the sEMG electrodes and on the development of more effective but still fast recognition algorithms.

References 1. Wu, J.Q., Gao, W.: Chinese Hand-language Recognition System Based on DGMM. Journal of Computer Research and Development, Vol.37 No.5, May (2000) 2. Chu, J.U., Moon, I., Mun, M.S.: A Real-Time EMG Pattern Recognition System Based on Linear-Nonlinear Feature Projection for a Multifunction Myoelectric Hand. IEEE Trans. Biomed. Eng., Vol. 53, (2006) 2232-2239 3. Kim, J.S., Jeong, H., Son, W.: A New Means of HCI EMG-MOUSE. IEEE Conf. Systems,Man & Cybernetics (2004) 100-104 4. Agarabi, M., Bonato, P., De Luca, C.J.: A sEMG-based Method for Assessing the Design of Computer Mice. IEEE Conf. EMBS (2004) 2450-2453 5. Lei, M., Wang, Z.Z.: Development and Expectations of Researches on sEMG signal in EMG artificial limb applications. Chinese Journal of Medical Instrumentation, 15 (2001) 156-160 6. Yang, F.S., Gao, S.K.: Biomedical Signal Process. Higher Education Press, Beijing (1995) 7. Bian, Z.Q., Zhang G.X.: Pattern Recognition (2nd Edition). Tsinghua University Press, Beijing (2002) 8. http://www.delsys.com