Comparison of statistical and non-statistical classifiers for thumb motion classification Jan Havlík, Zdeněk Horčík Czech Technical University in Prague, Faculty of Electrical Engineering, Prague Summary This paper deals with a comparison of statistical and non-statistical classifiers for thumb motion classification. Presented work is a part of research of relation between brain and muscle activity. The thumb motion is represented by trajectory coordinates of special mark placed on the thumb. Motions are classified using k-Means classifier (non-statistical classification, no prior information is needed) and Bayes classifier (statistical models of classes are needed, classifier training is necessary). The efficiency of classifiers is evaluated using the standard HTK parameters. Real testing data includes more than 900 stationary states which are used for classifier testing. The classification results are compared for both methods. Keywords motion analysis, motion classification, k-Means classification, Bayes classification Introduction Presented paper deals with a comparison of two classification methods used for thumb motion classification – k-Means classifier and Bayes classifier. The k-Means classifier represents the non–statistical classifications methods which do not require a training process, the Bayes classifier represents the statistical classification algorithms required a training phase. Thumb motion classification is one of the tasks solved within the finding of correlation between human body motions and human brain activity. The body motions are represented by the parameters of thumb trajectory, the brain activity is represented by the electro– encephalograph (EEG) signals. The thumb trajectory is acquired as a trajectory of special mark placed on the thumb [1]. The goal of presented research is to assign the typical changes of EEG signals to the type of muscle activity. Experiment and Parameterization The thumb is marked by special mark – the target composed from black and white concentric circles. The sensed person moves thumb between three positions. Each motion is triggered with optical synchronization pulse. The period of pulses is 6 ± 1 s. The motions are sensed using a pair of standard DV camcorders. The sensed motions are recorded to a tape and stored to a PC after the experiment. Both video–sequences are processed by a 2–D parameterization process. The coordinates of the mark projections to the camcorders scan planes are obtained. After the 2–D projection the space coordinates of the thumb mark are obtained using the 3–D parameterization of both sequences together. The coordinates X, Y and Z of the mark are computed for each half–frame of the video–sequences solving goniometric equations described the space arrangement of the experiment. The obtained coordinates are combined to positional vectors χ, one vector for each half–frame of the video–sequence. As is known from kinematics the sequence of positional vectors χ fully represents the sensed motions [2] The positional vectors χ are used as feature vectors for classifiers.
k-Means Classification The goal of k-Means classification is to divide the feature vectors χ into K clusters ω1, ω2, …, ωK such that the Euclidian distance relative to the cluster centres µ1, µ2, …, µK is minimized. The clustering algorithm could be described in steps below [3]. Step 1 Set the number of clusters K and generate cluster centres µ1, µ2, …, µK randomly. Step 2 Assign each input vector χ to the cluster ω*, for which is distance between vector χ and appropriate cluster center µk is minimal. Step 3 Recalculate the cluster centres µk using the vectors χ which have been assigned into the cluster k (M is the number of vectors χ assigned into the cluster k). 1 M μk = ∑ χ m (1) M m =1 Step 4 Repeat steps 2 and 3 until the centres µk no longer change. Bayes Classification The goal of Bayes classification is to assign the feature vectors χ to R classes with classification labels ω1, ω2, …, ωR. The probabilities that the vector χ belongs to the class ωr (the posterior probabilities) are p(χ ω r )P(ω r ) P(ω r χ ) = R . ∑ p(χ ωr )P(ωr )
(2)
r =1
The equation (2) has been called the Bayes theorem. The feature vector χ belongs to class ω*, for which is the posterior probability P(ωr|χ) maximal [4]. Based on the experiment definition, the prior probabilities equals for all classes. That means the feature vector χ could be classified to the class ω*, for which is p(χ|ωr) maximal. Let's assume that the likelihood functions p(χ|ωr) have the normal distribution. The probability density functions p(χ|ωr) could be written as 1 1 ′ p(χ ω r ) = exp − (χ − μ r ) Σ −r 1 (χ − μ r ) , (3) N 2 (2π ) Σ r where the N is the number of entries in the feature vector χ, the µr is the mean of the normal distribution and the Σr is the covariance matrix of the distribution [5]. The parameters µr and Σr have to be determined for each class ωr before the classification. These parameters could be determined from the set of M known vectors χ belonging to the class ωr as 1 M μ= (4) ∑χ m M m=1 and 1 M ′ Σ = ∑ (χ m − μ )(χ m − μ ) . (5) M m =1 The equations (4) and (5) have been called Maximum Likelihood Estimation of normal distribution [6]. Results Presented classifiers have been compared using a pair of evaluation parameters defined in HTK Toolkit [7] – parameters %Correct and Accuracy. Let assign Total the number of all
stationary states in reference file which would be classified, Dels the number of deleted stationary states, Subs the number of substituted stationary states and Ins the number of inserted stationary states. Than the evaluation parameters could be defined as Total − Dels − Subs %Correct = × 100 (6) Total and Total − Dels − Subs − Ins Accuracy = × 100 . (7) Total The classifiers have been tested using the motion database includes more than 900 stationary states. More than 550 realizations of stationary states derived from basic experiments with well separable classes (experiments A and B). Next 160 stationary states were recorded in experiment with varying realizations of the same states (experiment C). Rest states were derived from experiment with very short distances between stationary states models (experiment D). The results of k-Means classifier testing are shown in Table 1, the results of Bayes classification are shown in Table 2. Experiment %Correct Accuracy Total Dels Subs A 95,1 % 95,1 % 243 11 1 B 99,4 % 99,4 % 319 2 0 C 68,1 % 68,1 % 160 43 8 D 75,9 % 68,0 % 203 38 11 Tab. 1: Results of k-Means classification
Ins 0 0 0 16
Experiment %Correct Accuracy Total Dels A 94,2 % 94,2 % 243 12 B 99,4 % 99,4 % 319 2 C 66,3 % 66,3 % 160 47 D 93,6 % 93,6 % 203 9 Tab. 2: Results of Bayes classification
Ins 0 0 0 0
Subs 2 0 7 4
Conclusion The results for motions with well separable stationary states (experiments A and B) are fully sufficient for both compared classifiers. The results from experiment with varying realizations of stationary states (experiment C) are almost the same for both classification methods. Not a single classifier gives appropriate result for this experiment. The problem are the changes of stationary states coordinates during the experiment. Neither k-Means classifier nor Bayes classifier can adapt to the new arrange of experiment. The results from experiment with hard separable stationary states (experiment D) are different for evaluated classifiers. k-Means classification algorithm could not classify some stationary states. There is a big number of deletions, substitutions and especially insertions in the classified states. The results of Bayes classification from this experiment are very good, there are a minimal deletions and substitutions and there are no insertions. The Bayes classifier can better classify the hard separable sets of feature vectors than k-Means classifier. It could be supposed that the well trained Bayes classifier will be more sensitive than the kMeans classifier for the thumb motion classification. Acknowledgement
This work has been supported by the research program No. MSM 6840770012 of the Czech Technical University in Prague (sponsored by the Ministry of Education, Youth and Sports of the Czech Republic). References [1] Allard, P., Strokes, I., Blanchi, J.–P.: Three Dimensional Analysis of Human Movement. Human Kinetics, 1995. [2] Havlík, J., Horčík, Z.: Three-Dimensional Thumb Motion Parameterization. In Applied Electronics 2005, pages 127 – 130. University of West Bohemia in Pilsen, Pisles, 2005. [3] Moore, A. W.: K-means and Hierarchical Clustering [online]. URL: http://www.autonlab.org/tutorials/kmeans.html, [2005-09-14]. [4] Kotek, Z. et al: Recognition Methods and Their Applications (in Czech). Academia, Prague, 1993. [5] Rektorys, K. et al.: Overview of Applied Mathematics II (in Czech). Prometheus, Prague, 2000. [6] Moon, T. K., Stirling, W. C.: Mathematical Methods and Algorithms for Signal Processing. Prentice Hall, 2000. [7] Young, S. et al.: The HTK Book. Cambridge University Engineering Department, 2002. Ing. Jan Havlík Czech Technical University in Prague Faculty of Electrical Engineering Department of Circuit Theory Technická 2 CZ – 166 27 Praha 6 tel: +420 224 352 048 e-mail:
[email protected]
