A New Approach for Gender Classification Based on Gait Analysis Maodi Hu, Yunhong Wang School of Computer Science and Engineer Beihang University, Beijing, 100191
[email protected],
[email protected]
Abstract—In this paper, we propose a novel pattern to represent spatio-temporal information of gait appearance which is called Gait Principal Component Image (GPCI). GPCI is a grey-level image which compresses the spatiotemporal information by amplifying the dynamic variation of different body part. The detection of gait period is based on LLE coefficients and it is also a new attempt. KNN classifier is employed for gender classification. The framework can be applied in real-time setting because of its rapidity and robustness. The experimental results on IRIP Gait Database (32 males, 28 females) show that the proposed approach achieves a high accuracy in automatic gender classification. Keywords-gait; gender classification; principal component; LLE coefficients;
I. I NTRODUCTION Gait, referring to the pattern of walking or locomotion, has been used as an efficient biometric feature in human identification [1]. Recently, with the growing demands in recognition and classification of the side and longrange surveillance video, gait has become a hot research topic. Gait information has more advantages than face in these cases. X.Li et al.[2] summarized the limitations of other biometric features, like face, fingerprint, iris, and handwriting. Distance between camera / scanner and people, people cooperation, People’s attention impede the acquisition of the traditional features to make identification or classification. Gait, is considered to contain gender information as face does. X.Li et al. [2] described their analysis in effectiveness of the seven human gait components for gender recognition. We can see that all body segments (head, arm, trunk, thigh, front-leg, back-leg, and feet) have their contribution to gender classification. In [3] [4], L.Lee et al. proposed a gait representation by fitting each ellipse to seven parts of silhouette, and features are the parameters of these seven ellipses. In [5], J.H.Yoo et al. used a sequential set of 2D stick figure to represent the gait signature, then SVM classifier was employed to classify gender on a considerably large database, but the number of males is much larger than that of females. As many large gait database always have more males than females. We have collected a gait database with similar number of men and women which named IRIP Gait Database [6], and used for doing research on recognition and classification. S.Lee et al. [7] defines one gait cycle as the period starting from a double support stance frame with left foot forward to the next. As most of feature extraction methods used in gait recognition and gender classification
are based on period detection, accurate gait period has become one of the most fundamental demands in preprocessing procedure. S.Sarkar et al. [8] use the variation of foreground pixels number to estimate the state that two legs are farthest apart or overlap, but it does not work well when the silhouettes are not integrated. The false period estimation and detection may fatally block the automated process based on known period. In this paper, we propose a new gait period detection approach based on LLE coefficients to solve this problem. The comparison of the two approaches will be presented in the section 3. The result of this period extraction method is very effective in our experiment. The spatio-temporal information was extracted from the image sequence through various ways. Some approaches use shape cues, like body height, width [9], angular [10], Radon Transform [11], frieze patterns[12], ellipses fit [3][4], etc. Some refer to image differences and self similarity cues, such as the shape variation-based frieze patterns [7]. And Nonlinear dimensionality reduction technique [13], Manifold Learning and HMM [14] have been used to extract spatio-temporal information recently. But most of these approaches are not suitable for gender classification in real-time monitoring. We put forward a new gait representation for gender classification, called Gait Principal Component Image (GPCI), to depict the physical changes in one gait period. Experimental results for gender classification by GPCI are encouraging. The rest of the paper is organized as follows: In Section 2, we summarize the gender classification framework. In Section 3, a comparison of the proposed period extraction methods with existing algorithms is made. Section 4 describes the proposed gait presentation GPCI and the performance of our algorithm in IRIP Gait Database is shown in Section 5. II. OVERVIEW OF THE FRAMEWORK The proposed gender classification algorithm could be divided into the following four steps: 1.Background subtraction and Image sequences preprocessing 2.Gait period extraction 3.Generate GPCI 4.Gender Classification. To get silhouettes, we employed a simple subtraction method to separate foreground and morphological filtering to reduce noise after binarization. Then the foreground and
(a) 2200
Numer of Pixels in Lower Portion
2100 2000 1900 1800 1700 1600 1500 1400 1300
0
10
20
30
40 Frame Index
50
60
70
80
40 50 Frame Index
60
70
80
(b) 1.5 1
Figure 1.
Flow diagram of proposed algorithm
LLE Coeffcient
0.5 0 −0.5 −1 −1.5
background can be separated by following equation 1. ||fi (x, y) − f¯k (x, y)|| ≤ μk , background (1) ||fi (x, y) − f¯k (x, y)|| > μk , f oreground fk (x, y) is the i-th frame of a given video sequence and f¯k (x, y) is the mean value of first k frames in the given sequence is. μk denotes the square deviation of the first k frames. To reduce noise, some morphology methods, such as erosion and dilation were used to erase the small spots on the binarilized image and to fix discontinuous point on the contour. the last step of generating silhouette is to align center of each silhouette to the middle of the Image. Since we get the silhouettes, the proposed period detection method is employed to extract gait sequences over each period. Then GPCI are generated to represent the spatio-temporal information. Finally, KNN will be used to classify the gender. The steps are shown by Figure 1. III. G AIT PERIOD DETECTION A. Period extraction based on pixels number Gait period can be extracted from the silhouette sequence by counting the number of foreground pixels in each frame over time. S. Sarkar et al. [8] use the following approach to detect the period. They demonstrated that the number of foreground pixels will reach a maximum in full stride stance and drop to a minimum in heels together stance, and the variation of the pixels number mainly affected the legs, which are selected by considering only the bottom half of the silhouette. But in practice, the period can’t always be estimated by this approach because the silhouette is not perfect, especially for the real-time monitoring applications.
−2
0
10
20
30
(c) Figure 2. Silhouette frames and corresponding number of pixels (6th person in IRIP Gait Database), LLE coefficients (a) 1st, 9th, 17th, 25th, 33rd, 41st, 49th, 57th and 65th silhouette frame (b)number of pixels in Lower 53% portion (c)one dimensional LLE coefficient
The Figure 2(b) shows the number of pixels in lower 53% portion of body (body part below the waist accounts for the entire proportion is about 53% based on anatomical knowledge), which is calculated from the 6th person in IRIP Gait Database, the variation curve of the number of whole body’s pixels is very similar. We can hardly extract the authentic period from it. If the gait cycle can’t be extracted, these methods will only depend on manual period extraction, and it will be hard to do the gait recognition or gender classification automatically. B. Period extraction based on LLE LLE [15] is a nonlinear dimensionality reduction method to map high dimensional image sequence to low dimensional manifold. A linear approximation of LLE to solve linearization problem has been proposed in [16], LEA can finds the corresponding low-dimensional point on manifold via the linear projection. T.Ding [13] has demonstrated that LLE is capable of extracting the spatiotemporal variation information of gait silhouette. The one dimensional LLE coefficient sequences of the gait sequence of Figure 2(a) shown in Figure 2(c) is more
1
cross−correlation sequence
0.8 0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8
0
10
20
30
40 lag + 35
50
60
70
80
Figure 3. Normalized auto-correlation over the range of lags -35 to 35
convenient to observe the period than Figure 2 (b). So the period can be extracted from the LLE coefficient sequences. As [15] described, LLE recovers global nonlinear structure from locally linear fits. Firstly, assign K nearest neighbors to each data point; Secondly, Compute the weights W by solving a least-squares problem represented by equation 2. arg min W
Mi
|Xiα −
α=1
K
2
Wαβ Xiβ |
(2)
β=1
Xiα is the original α-th frame in the i-th gait sequence. Mi is frames number of the sequence. The value of K has great impact on the computing time. Through our experiments, 8 is enough in this application. Thirdly, compute the best low dimensional representing vectors Y by minimizing the following equation 3. Φ(W ) =
Mi α=1
|Yiα −
K
Wαβ Yiβ |2
(3)
β=1
Figure 3. is the auto-correlation sequence of LLE coefficients Yi in Figure 2.(c), from which we can figure out the period clearly from the local maxima points. Because the left leg forwarding half gait cycle and right one forwarding half cycle are almost the same in the database, the span between two local maxima is the number of frames over half a period. So the second local maxima point in the right half of the peak is the frames number in one period. To test this algorithm, we apply it on IRIP Gait Database, in which the number of frames in one period ranges from 18 to 34. We test it on the 0 degree, 30 degree, 60 degree, 90 degree-1, 90 degree-2, 120 degree, 150 degree, 180 degree gait silhouette, and the experimental results will be shown in the section 5. The proposed gait representation method does not need to assign full stride stance or heels together stance to be the starting point. For some other gait representation needs exactly the same starting and ending stance, maxima or
Figure 4. Gait silhouette image and GPCI: the first row, 1st 5th 9th 13th frame; the second row, 17th 21st 25th frame and GPCI of 28 frames in a period
minima of LLE coefficients can be used. The following feature extraction process will depend upon these automatic extracted periods. IV. G AIT P RINCIPAL C OMPONENT I MAGE As described before, the gait silhouette is used for feature extraction and gender classification. We propose a new spatio-temporal representation of gait, called Gait Principal Component Image, which considers gait as a sequence of silhouette frames, and represents the class information of one gait period in a single image. Unlike GEI [17], which calculates the average image, our new approach is to concentrate on using PCA to amplify the dynamic variation of different body part. Dimensionality reduction algorithms are applicable to map data into a space of much lower dimensionality. PCA(Principal Components Analysis), as a stable unsupervised linear dimensionality reduction approach to discover projection direction which achieves the smallest loss, has been widely used to decorrelate data for classification. P. S. Huang et al. [18] have applied PCA to reduce gait silhouette Image dimensionality by projecting all the silhouette of different persons into a low dimensional space, so it can optimize the class separability of different gait classes. Our approach is also based on PCA in the same purpose, but in a very different way. Suppose there are L periods of Gait Silhouette, and the resolution of them is m*n. Ni is frames number in the i-th period. Xij (1-by-mn vector) denotes the j-th silhouette frame of the i-th period (such as row by row, column by column). So Xi is an Ni-by-mn matrix, of which each row is the values of pixels in the silhouettes over a period. Let mXi (Ni -by-1 vector) to be the average pixel value in each silhouette frame of the i-th period. 1Ni denotes the 1-by-mn vector of ones. Then we use PCA as the following sequence: Firstly, calculate the covariance matrix Ri represented by the following equation 4.
Ri = (Xi − mXi ∗ 1Ni )(Xi − mXi ∗ 1Ni )T mn 1 where, mXi T = { mn Xijk }j=1...Ni
(4)
k=1
Ri is an Ni -by-Ni symmetric matrix. In IRIP Gait Database, Ni is between 18 and 34. So the eigenvectors of Ri can be calculated quickly with little calculating amounts. Then get the eigenvector ui corresponding to the largest eigen value of Ri by equation 5. ui = EigenV ecteri,max(eigenvaluei )
(5)
Now we have got the base of 1 dimensional space which shows the largest change between each pixel over one period. The last step, equation 6 is to compute the vector representation Yi in the 1 dimensional space. Yi = ui T (Xi − mXi ∗ 1Ni )
(6)
The Yi should be normalized to 0-255 by equation 7, then it can be displayed as an Image. Zi = 255
Yi − min(Yi ) max(Yi ) − min(Yi )
Figure 5. Gait Principal Component Image in different angle of view: The first row, C1: 0 degree, C2: 30 degree, C3: 60 degree, C4: 90 degree1; The second row, C5: 90 degree-2, C6: 120 degree, C7: 150 degree, C8: 180 degree
(7)
Zi is our proposed gait representation of one period sequences: GPCI. That’s the whole process to calculate. It may need more calculating amounts than GEI, but it is also efficient. It just costs less than 1 second to compute a GPCI for one period in my personal computer (CPU: PD3.4G, Memory: 1.5G). The last region of Figure 4 shows us a GPCI in 90 degree angle of view computed from one period sequences. Figure 5 demonstrate the GPCIs in 0 to 180 degree, which were captured by C1 to C8. GPCI has stronger representation than GEI for gait changes in sequence. For example, the sequence of one binary pixel in the Image is ”01101100”; another one is ”10010011”. They have the same numbers of ”1” and ”0”, so their averages are the same. It means the two pixels make no difference in GEI. But if we treat every digit as an axis in the space, then we can find the direction to make all pixels have the greatest degree of distinction. And this is how GPCI works. In order to testify the usefulness of our proposed approach and reduce the computing complexity, KNN classifier based on Euclidian Distance was employed to the classification step. In next section, there will be some experimental results to show its efficiency. V. E XPERIMENTAL RESULTS In IRIP Gait Database, there are gait data collected from 60 volunteers, including 32 male subjects and 28 female subjects aged between 22 and 28. Eight cameras were placed at different angles recording the movement of a person. These cameras were divided into two groups, each of which consists of four cameras and forms a 1/4 circle. The face of the person is captured by another camera from the front view. The arrangement of these eight cameras is illustrated in Figure 6. Cameras from C1 to C8 are used
Figure 6.
Cameras setup for data acquisition
to record human gait. Camera C9 records human face. We don’t use the data captured by C9 in this paper. Automatic extraction of gait period by using LLE coefficients have been applied on the 60 persons. For each participant, there are five sequences recorded the walking pattern from left to right and right to left. In each sequence, the gait silhouette captured from 8 cameras of different degrees were all in the experiment (C1: 0 degree, C2: 30 degree, C3: 60 degree, C4: 90 degree-1, C5: 90 degree-2, C6: 120 degree, C7 150 degree, C8: 180 degree). We find that the same algorithm on left to right walking and right to left walking have the very similar correct rate in nearly all the gait research. So we only apply our approach on the left to right part. In the 300 tested sequences (60 persons * 5 sequences / person), we think the following period length results are incorrect: 1.Being smaller than 18 or larger than 34; 2.Difference from actual length of the period by visual observation; 3.Do not exist a local maximum point in autocorrelation sequence except the middle point. The correct rates in all the 8 angles of view are shown
Camera No. C1 C2 C3 C4 C5 C6 C7 C8
Correct Rate 82% 99.33% 99.67% 100% 100% 100% 100% 91%
Incorrect Number 54 2 1 0 0 0 0 27
Period det. LLE* LLE* Pixels num Pixels num
Feature GPCI* GEI Ellipse Fit H&V Proj.
Classifier 7NN 7NN PCA+SVM PCA+SVM
Correct rate 92.33% 92.00% 90.00%[6] 90.3%[19]
Table III T EST RESULTS BY DIFFERENT GENDER CLASSIFICATION ALGORITHMS ON IRIP G AIT DATABASE
Table I S UMMARY OF THE PROPOSED LLE BASED PERIOD EXTRACTION ALGORITHM
VI. C ONCLUSIONS AND F UTURE W ORK K value of KNN GEI GPCI*
K=7 92.00% 92.33%
K=9 90.00% 92.33%
K=11 90.67% 91.00%
Table II P ERFORMANCE OF THE PROPOSED GENDER CLASSIFICATION ALGORITHM
in the Table VI. We can see the periods extracted from 30 to 150 degree are almost right, but in 0 degree and 180 degree, its accuracy is not so high. However, people can’t point out the period in the silhouette of walking in front side or back side. We apply the proposed representation GPCI on IRIP Gait Database for gender Classification. In the classification process, we divided the sequences into two sets, training sequences (gallery) and test sequences (probe). There are 60 subjects in the database, including 32 men and 28 women. As we mainly consider the 90 degree angle of view, the walking sequence captured by C5 was used in this experiment. We use Leave-one-out method to partition gallery data and probe data. In the experiment, One GPCI of each person (5 sequence / person) has been chosen as probe data in turn and the remaining GPCIs (59 persons * 5 sequences / person) were used as gallery data. In the end, the KNN classification based on Euclidian Distance was employed to judge the gender of the test subject. Because there are 5 sequences for each person, the top 5 nearest neighbors may be the same person. So, the values of K we tested are more than 5. The GEI and GPCI are both computed on the same period sequences extracted by our proposed LLE based period extraction automatically. Proper period can improve classification accuracy. We can see the correct classification rate is so encouraging in Table VI. On the same database, D.Zhang [6] et al achieved 90% gender classification rate with 7 ellipses fitted feature and PCA+SVM classifier. Furthermore, another approach [19] which used horizontal and vertical projection feature of silhouettes and PCA+SVM classifier, had 90.3% gender classification rate. We test our proposed GPCI with KNN classifier in this paper to improve this classification rate to 92.33%. The performances of these algorithms on IRIP Gait Database are shown in TableVI.
In this paper, we propose a gait period extraction approach and a new representation for gender classification. LLE and PCA were employed as the main method in the process. We used Gait Principal Component Image (GPCI) as the gait appearance feature for gender classification. The experimental results show that GPCI is capable of capturing the spatio-temporal information, and the proposed method has high classification accuracy. In the future work, we will try to contrive other gait representation for gender classification. And other dimension reduction techniques and classifiers will be employed to improve classification accuracy. VII. ACKNOWLEDGEMENT This work was supported by the opening funding of the State Key Laboratory of Virtual Reality Technology and Systems(Beihang University). R EFERENCES [1] A. Kale, A. Sundaresan, A. N. Rajagopalan, N. P. Cuntoor, A. K. Roy-Chowdhury, V. Kruger, and R. Chellappa, “Identification of humans using gait,” IEEE Transactions on image processing, vol. 13, no. 9, September 2004. [2] X. Li, S. J. Maybank, S. Yan, D. Tao, and D. Xu, “Gait components and their application to gender recognition,” IEEE Transactions On Systems, Man, And Cybernetics-Part C, vol. 38, no. 2, March 2008. [3] L.Lee and W.E.L.Grimson, “Gait analysis for recognition and classification,” In Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, 2002. [4] L.Lee, “Gait dynamics for recognition and classification,” Technical Report AIM-2001-019, September 2001. [5] J. H. Yoo, D. Hwang, and M. S. Nixon, “Gender classification in human gait with support vector machine,” Advanced Concepts for Intelligent Vision Systems, vol. 3708, pp. 138– 145, 2005. [6] D. Zhang and Y. Wang, “Gender recognition based on fusion of face and gait information,” In IEEE Proceeding of International Conference on Machine Learning and Cybernetics, 2008. [7] S. Lee, Y. Liu, and R. Collins, “Shape variation-based frieze pattern for robust gait recognition,” In IEEE Proceedings of CVPR, 2007.
[8] S. Sarkar, P.J.Phillips, Z.Liu, I.R.Vega, P. Grother, and K.W.Bowyer, “The human id gait challenge problem: Data sets, performance, and analysis,” IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 27, no. 2, pp. 162–177, February 2005. [9] A. Kale, N. Cuntoor, B. Yegnanarayana, A. Rajagopalan, and R. Chellappa, “Gait analysis for human identification,” In Proceedings of 4th Int.Conf. Audio and Video-Based Person Authentication, pp. 706–714, 2003. [10] N. V. Boulgouris, K. N. Plataniotis, and D. Hatzinakos, “An angular transform of gait sequences for gait assisted recognition,” In IEEE Int. Conference on Image Processing, pp. 857–860, October 2004. [11] N. V. Boulgouris and Z. X. Chi, “Gait recognition using radon transform and linear discriminant analysis,” IEEE Transactions On Image Processing, vol. 16, no. 3, pp. 857– 860, March 2007. [12] Y.Liu, R.Collins, and Y.Tsin, “Gait sequence analysis using frieze patterns,” European Conference on Computer Vision, pp. 657–671, May 2002. [13] T. Ding, “A robust identification approach to gait recognition,” In IEEE Proceedings of CVPR, 2008. [14] M. H. Cheng, M. F. Ho, and C.-L. Huang, “Gait analysis for human identification through manifold learning and hmm,” Pattern Recognition, vol. 41, no. 8, pp. 2541–2553, August 2008. [15] S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science, vol. 290 22, December 2000. [16] T. H. Y Fu, “Locally linear embedded eigenspace analysis,” IFP-TR, Univ. of Illinois at Urbana-Champaign, Jan 2005. [17] J. Han and B. Bhanu, “Individual recognition using gait energy image,” IEEE Transactions on pattern analysis and machine intelligence, vol. 28, no. 2, February 2006. [18] P. S. Huang, C. J. Harris, and M. S. Nixon, “Recognising humans by gait via parametric canonical space,” Artificial Intelligence in Engineering, vol. 13, no. 4, pp. 359–366, October 1999. [19] D. Zhang and Y. Wang, “Investigating the separability of features from different views for gait based gender classification,” In Proceedings of ICPR, 2008.
