Gait Recognition using Periodic Temporal Super Resolution for Low ...

Gait Recognition using Periodic Temporal Super Resolution for Low Frame-rate Videos Naoki Akae, Yasushi Makihara, and Yasushi Yagi Osaka University {akae,makihara,yagi}@am.sanken.osaka-u.ac.jp

Abstract

period significantly degrades the existing gait recognition algorithms. Despite its importance, only a few studies have tackled gait recognition with low frame-rates. Mori et al. [14] proposed a matching method between a low frame-rate probe (input/query) sequence and a normal frame-rate gallery (enrollment) sequence based on period-based phase synchronization. Although the problem setting was effective for several cases of criminal investigation, it was naturally not applicable to matching between dual low frame-rate videos such as cross-camera person matching. In such dual low frame-rate problem settings, temporal interpolation is one possible approach. Al-Huseiny et al. [1] proposed the level-set approach to temporal interpolation and applied it to gait recognition for low frame-rate videos. The temporal interpolation works well for videos with a middle range of frame-rates (e.g., more than 6 fps), it fails, however, for quite low frame-rate videos (e.g., 1 or 2 fps). Another possible approach is temporal super resolution. Shechtman et al. [22] proposed a space-time super resolution by combining information from multiple lowresolution video sequences of the same dynamic scene. The multiple camera setting is, however, not always available in actual scenes. Furthermore, methods of temporal super resolution from a single video were also proposed. Shimano et al. [23] proposed an example-based temporal super resolution using self similarity and Makihara et al. [9] proposed a reconstruction-based method using multiple-period observation and phase registration within a single video. The initial contribution of this paper is to improve gait recognition performance for low frame-rate videos by introducing periodic temporal super resolution [9]. Once an input to a low frame-rate video has been subjected to temporal super resolution, existing gait features [8][5][10][21] are successfully applied. On the other hand, it was reported that periodic temporal super resolution suffers from errors of initial phase estimation using linear phase evolution prior, which typically oc-

This paper describes a method of gait recognition where both a gallery and a probe are based on low frame-rate videos. The sparsity of phases (stances) per gait period makes it much harder to match the gait using existing gait recognition algorithms. Consequently, we introduce a super resolution technique to generate a high frame-rate periodic image sequence as a preprocess to matching. First, the initial phase for each frame is estimated based on an exemplar of a high frame-rate gait image sequence. Images between a pair of adjacent frames sorted by the estimated phases are then filled using a morphing technique to avoid ghosting effects. Next, a manifold of the periodic gait image sequence is reconstructed based on the estimated phase and morphed images. Finally, the phase estimation and manifold reconstruction are iterated to generate better high frame-rate images in the energy minimization framework. Experiments with real data on 100 subjects demonstrate the effectiveness of the proposed method particularly for low frame-rate videos of less than 5 fps.

1. Introduction Gait recognition is one of the behavioral biometrics which has attracted much attention because it can be applied to uncooperative subjects and/or subjects at a distance from cameras. In fact, a burglar’s gait captured by a CCTV camera was accepted as evidence in a UK court [27]. Although gait recognition has promising properties, it still suffers from many types of intra-class variations such as observation views [10][2][28][7], walking speeds [11][25], clothing [6], and elapsed times [21][13]. In addition to these difficulties, gait recognition for low frame-rate videos has remained an important and challenging problem because (1) digital videos captured by CCTV cameras are often recorded at quite low frame-rates (e.g., 1 to 6 fps) due to limitations of the communication bandwidth and/or storage size, and (2) the sparsity of phases (stances) per one gait 1 978-1-4577-1359-0/11/$26.00 ©2011 IEEE

curred due to the wagon wheel effect in quite low frame-rate cases (e.g., 1 or 2 fps) [9]. Therefore, the second contribution of this paper is the improvement of initial phase estimation using an exemplar high frame-rate gait image sequence. Although the subject of the exemplar gait image sequence is different from those of a probe and a gallery, the exemplar sequence still makes the initial phase estimation much more robust than only linear phase evolution prior. Moreover, if sufficient images are not available as inputs and hence the observed gait stances are still sparse after temporal super resolution, the reconstructed periodic image sequence often contains alpha blending-like ghosts because the method [9] interpolates it in image intensity based eigen space. Therefore, the third contribution of this paper is to improve the quality of the reconstructed periodic image sequence by incorporating morphing techniques to fill the gait stances between a pair of adjacent images sorted by estimated phases. Note that the proposed method fills images between a pair of adjacent frames of relatively similar gait stances after a sorting phase, while the existing method [1] fills images between a pair of adjacent frames of the largely different gait stances in low frame-rate videos. The remainder of this paper is organized as follows. Section 2 addresses related work on gait recognition and temporal super resolution, and Section 3 describes the preprocessing for silhouette-based gait recognition. Section 4 addresses the proposed method of periodic temporal super resolution. Section 5 describes experiments using real gait image sequences and Section 6 discusses the limitations of the proposed method. Finally, Section 7 gives the conclusions and identifies future research.

Figure 1. Normalized gait silhouette sequence

low frame-rate videos. They suffer, however, from a phase synchronization step in the case where both a gallery and a probe are low frame-rate videos because corresponding phases (stances) between the gallery and a probe are often missing due to the sparsity of the frame-based gait features per gait period. Temporal super resolution: Approaches to temporal super resolution fall into two groups: example-based approaches and reconstruction-based approaches. The example-based methods typically try to generate a high frame-rate video from a single low frame-rate video based on a number of exemplar pairs of spatio-temporal patches in both the high frame-rate and low frame-rate videos [23]. The reconstruction-based method typically tries to generate a high frame-rate video from multiple low frame-rate videos based on spatio-temporal registration [22]. On the other hand, in the case of a periodic or quasi-periodic image sequence, multiple periods of image sub-sequences with subframe-order differences are available for reconstructionbased periodic temporal super resolution from a single low frame-rate video [9]. In such periodic temporal super resolution, stable phase estimation for each frame is key to the generation of accurate high frame-rate periodic image sequences.

2. Related work

3. Preprocessing

Period-based gait feature: Because a gait is a type of periodic motion, period-based gait features have been widely used for gait recognition. The most well-known one is the averaged silhouette [8] (also called GEI [5]) which is obtained by simply averaging gait silhouette images over one gait period. In addition, gait features based on frequency analysis are also proposed such as amplitude spectra [10][29], Fourier descriptors [15][30], and Gabor features [26]. These period-based gait features are computed from multiple images within one gait period, and hence they deteriorate significantly when the number of images within one gait period reduces to low levels, as in low frame-rate videos. Frame-based gait feature: A frame-based gait feature such as a raw gait silhouette image is used in conjunction with phase synchronization using phase shift and/or stretching in the linear domain [21][17][14] or using Dynamic Time Warping (DTW) or Hidden Markov Model (HMM) in the non-linear domain [4][24]. The frame-based gait features themselves do not deteriorate even in the case of

In this section, a preprocessing part of our silhouettebased gait recognition is briefly addressed. First, given a gait image sequence, a silhouette for each frame is extracted as a foreground region by background subtraction-based graph-cut segmentation [12]. Next, a normalized gait silhouette sequence of pre-determined size is generated by image size-normalization and registration based on the height and the center of gravity of each silhouette as shown in Fig. 1. Principal Component Analysis (PCA) is then applied to the normalized gait silhouette sequence and each normalized gait silhouette is projected into PCA space for dimension reduction. The dimension of the PCA space is decided so that information loss is less than 1%, in other words, the cumulative contribution rate is over 99%. Note that the normalized gait silhouette sequence is expressed as a manifold parameterized by phase in parametric eigen space [16]. The gait silhouette and gait silhouette sequence are expressed as a point and a trajectory in the PCA space respectively, as illustrated in Fig. 2.

4.2. Manifold representation 6000 4000 4000 3rd PC

2000 2000 0

0 -6000

-2000 -4000

-2000

0

1st PC

2nd PC

-4000 2000

4000

Figure 2. Gait silhouette sequence in PCA space

4. Periodic temporal super resolution 4.1. Definition of quasi-periodic image sequence In this section, we define an input gait silhouette sequence as a quasi-periodic image sequence. An image drawn from the periodic image sequence at time t is denoted by the vector-form x (t), which satisfies x (t + P ) = x (t) ∀t,

(1)

t P

s˜ = s − ⌊s⌋,

(2) (3)

where sP (·) is a phase evolution function and ⌊·⌋ is a floor function. The periodic image sequence is now represented in the phase domain as x s (s) = x (s−1 P (s)).

(4)

Note that the periodic image sequence constructs a manifold with respect to the relative phase s˜ ∈ [0, 1] and which satisfies x s (1) = x s (0). On the other hand, an input gait silhouette sequence is composed of N in discretely observed images X in = in {x in − 1) which are denoted as i }(i = 0, . . . , N x in i sP,i

= x s (sP,i ) = s0 + f1P ,

T 4N ×M [Asp , . . . , Asp , respectively. An in0 N cp −1 ] ∈ R terpolated point y ˆ (˜ s) in the eigenspace for a relative phase s˜ at the jth interval is then expressed as T

T

y ˆ (˜ s) =

cp

Asp T w (˜ s)

(5) (6)

where s P = {sP,i } is a phase sequence for the input image sequence, s0 is the phase for the first frame, and f is the frame-rate. Moreover, the input image sequence is often dein graded to a quasi-periodic image sequence X in Q = {x Q,i } due to fluctuations of the sampling rate of the network camera and/or gait motion itself and it is defined as x in Q,i

=

x s (sQ,i )

(7)

sQ,i

=

sP,i + ∆si ,

(8)

where s Q = {sQ,i } is a quasi-periodic phase sequence. In summary, the problem setting can be stated as a simultaneous estimation problem of a periodic manifold x s and a phase sequence s Q from an input quasi-periodic image sequence X in Q.

w

(9) 2

w (˜ s) =

where P is a period. Two non-dimensional time parameters, phase s and relative phase s˜, are then introduced as s = sP (t) =

The periodic manifold x s is represented by a parametric eigenspace method [16]. More specifically, a cubic N-spline function parameterized by the phase in the eigenspace is exploited. Let us consider N cp control points {y cp j } in the M dimensional eigenspace accompanied by correspondcp cp − 1). ing phases {scp j (= j/N )}, (j = 0, . . . , N Next, a spline parameter vector for a k power-term coefcp sp ficient at the jth interval [scp j , sj+1 ] is denoted as a j,k ∈ RM (k = 0, 1, 2, 3). Subsequently, a submatrix Asp j at the jth interval and a total spline matrix Asp are desp T fined as Asp = [a sp ∈ R4×M and Asp = j j,0 , . . . , a j,3 ]

3

[0, . . . , 0, 1, w, w , w , 0, . . . , 0] s˜ − scp j cp = ˜ ≤ scp cp , (sj ≤ s j+1 ) scp − s j+1 j

T

(10) (11)

where w (˜ s) is an interpolation coefficient vector whose components from 4j to (4j + 3) are [1, w, w2 , w3 ], and w is the interpolation ratio between the control points. On the other hand, it is well known that the relationship cp T between a control points matrix Y cp = [y cp 0 , . . . , y N cp −1 ] sp and a spline parameter matrix A is derived from the C2continuous boundary conditions as [9] Asp = DY cp ,

(12)

4N cp ×N cp

where D ∈ R is a coefficient matrix. Hence, once the control points Y cp have been given, interpolation y ˆ (˜ s) for a relative phase s˜ is obtained using Eqs. (9) and (12). The reconstruction problem of the periodic manifold x s (s; Y cp ) can then be replaced by an estimation problem of the control points Y cp as discussed in the following sections.

4.3. Energy minimization framework Similar to the other temporal super resolution approaches [22][23], we adopt an energy minimization approach. First, suppose that an input quasi-periodic image in sequence in the eigenspace is expressed as Y in Q = {y Q,i } and recall that the accompanying phase sequence s Q = {sQ,i } is unknown. Subsequently, the interpolation coefficient vector for the ith phase sQ,i is defined as w (sQ,i ), in the same way as Eq. (10), and then approximation by the periodic manifold with the ith phase sQ,i is y ˆ (Y cp , sQ,i ) = Asp T w (sQ,i ) = Y cp T DT w (sQ,i ). (13) The energy function is then constructed by considering the following three aspects: (1) data fitness between the interpolation y ˆ (Y cp , sQ,i ) and the input y in Q,i , (2) smoothness

of the periodic manifold y s (s; Y cp ) in eigenspace, and (3) smoothness of the phase evolution s Q based on the linear phase evolution prior. The actual form of the function is N −1 1 ∑ 2 ||Y cp T DT w (sQ,i ) − y in Q,i || N in i=0 in

E(Y

cp

, s Q)

=

1 ||BY cp ||2 N cp )2 N in −1( 1 ∑ 1 + λs in sQ,i −sQ,i−1 − ′ , (14) N i=1 P + λm

where the first, second, and third terms are the data term, the smoothness terms for the periodic manifold and the phase evolution. B is a matrix for the manifold curvature, and P ′ (= f P ) is a global period in the frame domain derived from the linear phase evolution prior. We can see that the objective function E(Y cp , s Q ) is a quadratic form with respect to the manifold control points Y cp and, therefore, a linear solution of the manifold control points Y cp is provided under the fixed phase s Q . On the other hand, the phase s Q is a complex form, since the spline curves are switched piecewise based on phase s Q and the interpolation ratio w appears as a sixth-order polynomial in the data term. To solve this highly nonlinear optimization problem, iterative solutions are employed in the following sections.

4.4. Initial phase estimation In the initial phase estimation step, instead of using linear phase evolution prior [9], continuous Dynamic Programming (DP) [18] is applied directly between an input low frame-rate image sequence YQin and an exemplar high frame-rate image sequence which is expressed as a set of cp manifold control points Y ex = {y ex j }(j = 0, . . . , N −1). Note that the only data term in the energy function (14) is considered in this step because the manifold smoothness term is independent of phase and the phase smoothness term based on linear phase evolution prior is negligible. First, the phase space is quantized by the same interval as that of the manifold control points (1/N cp ). This means that the jth quantized phase coincides with the jth control point y ex j . Next, a set of phases which can be transited from a previous step to jth quantized phase is defined as Rj by considering the frame-rate of the input image sequence and gait period range. A cumulative cost and the optimal transition path from a previous step at the ith input frame and jth exemplar control point are then denoted as c(i, j) and p(i, j), respectively, and the optimal phase is provided by the following DP steps. 1. Initialize cost matrix in 2 c(0, j) = ||y ex j − y Q,0 ||

(15)

2. Update cumulative cost and transition path p(i, j) = arg minj {c(i − 1, k)} k∈R

(16)

in 2 c(i, j) = c(i−1, p(i, j)) + ||y ex j −y Q,i || (17)

3. Optimize the terminal phase p∗ (N in − 1) = arg min c(N in − 1, j) j

(18)

4. Back track p∗ (i − 1) = p(i, p∗ (i))

(19)

After the initial phase is estimated as s init Q as a result of the above DP procedure, the global period P ′ is computed from averaged phase evolution speed.

4.5. Interpolation between sorted phases Once an initial phase sequence s init = {sinit Q Q,i } is estiin mated, the input image sequence X Q = {x in Q,i } is sorted based on the initial phase sequence s init as X sort = Q Q sort {x Q,i }. The intermediate images between each pair of sort adjacent sorted images x sort Q,i and x Q,i+1 are then interpolated by the phase quantization interval (1/N cp ) using the morphing technique [3]. An advantage of morphingbased interpolation is to avoid ghosts occurring in the case of interpolation in eigen space. In addition, note that the proposed method interpolates images between a pair of adjacent sorted images, while the existing morphing-based gait recognition approach [1] interpolates images between a pair of adjacent original images whose the elapsed phase is much larger, or in short, whose gait stances are quite different from each other. Finally, the interpolated image seint int quence X int − 1) associated Q = {x Q,i }(i = 0, · · · , N with the interpolated sorted phase sequence s 0Q = {s0Q,i } is fed into the energy minimization framework, and the initial manifold control points Y cp,0 under the fixed phase sequence s 0Q are then easily calculated using a linear solution as described in Section 4.3.

4.6. Phase estimation by quadratic approximation After the initial solution is obtained, the phase sequence and manifold control points are alternately updated based on the previous solution. The first step is phase estimation based on the quadratic approximation of the data term with respect to phase. The search domain of the phase at the rth iteration is set based tol on the previous solution as Rir = {s|sr−1 ≤ s ≤ Q,i − s r−1 tol sQ,i + s }. The minimum data term for the ith phase sQ,i is found by the Newton method within the search domain. After phase minimizing, the data term has been obtained as

∗

sdata,r , and the ith data term in Eq. (14) is approximated Q,i by a Taylor expansion up to the second-order term as ˆ data,r (sQ,i ) = E data,r (sdata,r ∗ ) E i i Q,i data,r ∗ dEi (sQ,i − sdata,r ) + Q,i dsQ,i ∗ sQ,i =sdata,r Q,i 2 data,r ∗ 2 1 d Ei + (sQ,i − sdata,r ) .(20) Q,i 2 ds2Q,i data,r ∗ sQ,i =sQ,i

Now, the total energy function is a quadratic form with respect to the phase s Q and thus the optimal phase s Q is given as  int −1  1 N∑ r ∗ ˆ data,r (sQ,i ) s Q = arg min int E i s Q N i=0  int ( ) N∑ −1 1 1 + λs int sQ,i+1 −sQ,i − ′ 2 (21)  N P

Figure 3. Results of temporal super resolution. The first row is an input low frame-rate image sequence (2 fps), and rows two to four are temporal interpolation or super resolution results by Morph [1], TSR (LPEP) [9], and TSR (Ex + Morph), respectively (every 2 steps or control point). Note that the red and green box images are key shapes for level-set morphing. The bottom row is an original frame-rate (60 fps) image sequence as a reference (every 2 steps).

5.2. Result of periodic temporal super resolution

5. Experiments

In this section, we showed several results of the periodic temporal super resolution with linear phase evolution prior (TSR (LPEP)), with the exemplar image sequence (TSR (Ex)), and with the exemplar image sequence and morphing (TSR (Ex + Morph)), compared with temporal interpolation using the level-set morphing approach (Morph) [1]. Figure 3 shows results for a 2-fps image sequence. We can see that Morph (the second row of Fig. 3) cannot successfully recover a high frame-rate image sequence because the two key gait stances are too far apart (a single support phase and the next single support phase) due to the quite low frame-rate for interpolating intermediate gait stances (double support phases). Although TSR (LPEP) (the third row of Fig. 3) reconstructs a smooth gait silhouette sequence, it still suffers from backward play, that is, the wagon wheel effect, and also ghosts. On the other hand, TSR (Ex + Morph), (the fourth row of Fig. 3), successfully reconstructs a natural gait silhouette sequence with neither a wagon wheel effect nor ghosting, and hence looks quite similar to the original normal frame-rate image sequence (the bottom row of Fig. 3).

5.1. Dataset

5.3. Result of gait recognition

We evaluated the proposed method with real data from the OU-ISIR Gait Database [19]. A total of 200 real gait sequences from 100 subjects were used in the experiments. Each subject was requested to walk on a speed-controlled treadmill and two gait sequences were captured: one for the probe, and the other for the gallery. The normalized gait silhouette size, frame rate, and recording time for each sequence were 88 × 128 pixels, 60 fps, and 6 sec. Low frame-rate gait sequences were constructed by selecting the gait silhouettes at a specified interval, that is, by down sampling.

In this section, we evaluate the proposed methods TSR (LPEP), TSR (Ex), and TSR (Ex + Morph) in terms of gait recognition performance compared with the approach without temporal super resolution (NoTSR) and with temporal interpolation Morph [1]. Frame-based gait feature: In these experiments, the method of period-based phase synchronization [14] is employed for matching low frame-rate probe image sequence and super-resoluted or temporally interpolated high framerate gallery image sequence, where the periodic gallery image sequences are directly matched to the probe sequence

i=1

s.t.

tol tol sr−1 ≤ sQ,i ≤ sr−1 Q,i − s Q,i + s

(22)

sQ,i+1 ≥ sQ,i ,

(23)

where Eqs. (22) and (23) are the lower and upper limit constraints and the monotonically increasing constraints, respectively. As a result, the problem is formulated as a convex quadratic programming problem and is solved by the active set method. The next step is the update of the manifold control points. The manifold control points Y cp,r at the rth iteration are computed based on the phase sequence s rQ ∗ at the rth iteration via linear solution. These procedures are iterated until convergence by gradually relaxing the manifold smoothness constraint so that the manifold fits the data.

(a) Input image sequence

(b) Temporal super resolution result (every 5 control points) Figure 6. Stroboscopic effect (a) CMC curve for 1-fps image (b) Rank-1 identification rate sequences for each frame-rate Figure 4. CMC curve and rank-1 identification rate for framebased gait feature

quence appears to be almost standing still because the observed images are always the same even though the object is actually moving periodically as shown in Fig. 6. Any reconstruction-based approach from a single video cannot solve the stroboscopic effect on its own. Therefore a combined framework of the reconstruction-based and examplebased temporal super resolution techniques needs to be studied in the future.

7. Conclusions

(a) Averaged silhouette (b) Frequency-domain feature Figure 5. Rank-1 identification rate for each period-based gait feature

so as to synchronize the phase by shifting the frame. The performance measure used is Cumulative Matching Characteristics (CMC) curve [20] and rank-1 identification rate. The result is shown in Fig. 4. While all the approaches are degraded as the frame-rate decreases, either of the proposed temporal super resolution methods still achieves the best performance for less than 5 fps. Period-based gait feature: The gait features used are the most popular averaged silhouette [8][5] and the frequencydomain feature [10]. The performance measure used is the rank-1 identification rate. As for NoTSR, the frequencydomain feature cannot be extracted at less than 4 fps because of the sampling theorem. The results for each gait feature are shown in Fig. 5. While all the approaches achieved a 100 % identification rate at more than 5 fps, either of the periodic temporal super resolution approaches achieved a better performance at less than 5 fps

6. Limitations While the proposed method solved the wagon wheel effect which the linear phase evolution prior-based method [9] suffers from, the troublesome stroboscopic effect (temporal aliasing) still remains. The stroboscopic effect typically occurs when the sampling interval coincides with the period of a moving object. In such cases, the observed image se-

This paper described gait recognition for low frame-rate videos. We first improved gait recognition for low framerate videos by incorporating periodic temporal super resolution from a single image sequence. Second, we solved the troublesome wagon wheel problem which the existing periodic temporal super resolution approach suffered from by introducing stable initial phase estimation based on an exemplar high frame-rate video. Third, images between a pair of adjacent images sorted by estimated phases are interpolated by a morphing technique to avoid alpha blending-like ghosting. Experimental results using a real gait image sequence showed the effectiveness of the proposed method in terms of not only recognition performance but also the visual quality of temporal super resolution. In the future we propose to address the problem of the stroboscopic effect by combining the example-based and reconstruction-based approaches.

Acknowledgement This work was partially supported by Grant-in-Aid for Scientific Research (S) 21220003 and ”R&D Program for Implementation of Anti-Crime and Anti-Terrorism Technologies for a Safe and Secure Society ”, Strategic Funds for the Promotion of Science and Technology of the Ministry of Education, Culture, Sports, Science and Technology, the Japanese Government.

References [1] M. S. Al-Huseiny, S. Mahmoodi, and M. S. Nixon. Gait learning-based regenerative model: A level set approach. In The 20th Int. Conf. on Pattern Recognition, pages 2644– 2647, Istanbul, Turkey, Aug. 2010. 1, 2, 4, 5

[2] C. BenAbdelkader. Gait as a Biometric For Person Identification in Video. PhD thesis, Maryland Univ., 2002. 1 [3] D. Cohen-Or, D. Levin, and A. Solomivici. Threedimensional distance field metamorphosis. ACM Trans. Graphics, 17(2):116–141, 1998. 4 [4] N. Cuntoor, A. Kale, and R. Chellappa. Combining multiple evidences for gait recognition. In Proc. of IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, volume 3, pages 33–36, 2003. 2 [5] J. Han and B. Bhanu. Individual recognition using gait energy image. Trans. on Pattern Analysis and Machine Intelligence, 28(2):316– 322, 2006. 1, 2, 6 [6] M. A. Hossain, Y. Makihara, J. Wang, and Y. Yagi. Clothinginvariant gait identification using part-based clothing categorization and adaptive weight control. Pattern Recognition, 43(6):2281–2291, Jun. 2010. 1 [7] A. Kale, A. Roy-Chowdhury, and R. Chellappa. Towards a view invariant gait recognition algorithm. In Proc. of IEEE Conf. on Advanced Video and Signal Based Surveillance, pages 143–150, 2003. 1 [8] Z. Liu and S. Sarkar. Simplest representation yet for gait recognition: Averaged silhouette. In Proc. of the 17th Int. Conf. on Pattern Recognition, volume 1, pages 211–214, Aug. 2004. 1, 2, 6 [9] Y. Makihara, A. Mori, and Y. Yagi. Temporal super resolution from a single quasi-periodic image sequence based on phase registration. In Proc. of the 10th Asian Conf. on Computer Vision, pages 107–120, Queenstown, New Zealand, Nov. 2010. 1, 2, 3, 4, 5, 6 [10] Y. Makihara, R. Sagawa, Y. Mukaigawa, T. Echigo, and Y. Yagi. Gait recognition using a view transformation model in the frequency domain. In Proc. of the 9th European Conf. on Computer Vision, pages 151–163, Graz, Austria, May 2006. 1, 2, 6 [11] Y. Makihara, A. Tsuji, and Y. Yagi. Silhouette transformation based on walking speed for gait identification. In Proc. of the 23rd IEEE Conf. on Computer Vision and Pattern Recognition, San Francisco, CA, USA, Jun 2010. 1 [12] Y. Makihara and Y. Yagi. Silhouette extraction based on iterative spatio-temporal local color transformation and graphcut segmentation. In Proc. of the 19th Int. Conf. on Pattern Recognition, Tampa, Florida USA, Dec. 2008. 2 [13] D. Matovski, M. Nixon, S. Mahmoodi, and J. Carter. The effect of time on the performance of gait biometrics. In Proc. of the 4th IEEE Int. Conf. on Biometrics: Theory Applications and Systems, pages 1–6, Washington D.C., USA, Sep. 2010. 1 [14] A. Mori, Y. Makihara, and Y. Yagi. Gait recognition using period-based phase synchronization for low frame-rate videos. In Proc. of the 20th Int. Conf. on Pattern Recognition, pages 2194–2197, Istanbul, Turkey, Aug. 2010. 1, 2, 5 [15] S. Mowbray and M. Nixon. Automatic gait recognition via fourier descriptors of deformable objects. In Proc. of IEEE Conf. on Advanced Video and Signal Based Surveillance, pages 566–573, 2003. 2

[16] H. Murase and S. K. Nayar. Parametric eigenspace representation for visual learning and recognition. In Proc. of SPIE, 2031, 1993. 2, 3 [17] H. Murase and R. Sakai. Moving object recognition in eigenspace representation: Gait analysis and lip reading. Pattern Recognition Letters, 17:155–162, 1996. 2 [18] R. Oka. Spotting method for classification of real world data. Computer Journal, 41(8):559–565, 1998. 4 [19] Ou-isir gait database. http://www.am.sanken.osakau.ac.jp/GaitDB/index.html. 5 [20] P. Phillips, H. Moon, S. Rizvi, and P. Rauss. The feret evaluation methodology for face-recognition algorithms. Trans. of Pattern Analysis and Machine Intelligence, 22(10):1090– 1104, 2000. 6 [21] S. Sarkar, J. Phillips, Z. Liu, I. Vega, P. Grother, and K. Bowyer. The humanid gait challenge problem: Data sets, performance, and analysis. Trans. of Pattern Analysis and Machine Intelligence, 27(2):162–177, 2005. 1, 2 [22] E. Shechtman, Y. Caspi, and M. Irani. Space-time superresolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(4):531–545, Apr. 2005. 1, 2, 3 [23] M. Shimano, T. Okabe, I. Sato, and Y. Sato. Video temporal super-resolution based on self-similarity. In Proc. of the 10th Asian Conf. on Computer Vision, volume 1, pages 93–106, Queeenstown, New zealand, Nov. 2010. 1, 2, 3 [24] A. Sunderesan, A. Chowdhury, and R. Chellappa. A hidden markov model based framework for recognition of humans from gait sequences. In Proc. IEEE Int’l Conf. on Image Processing 2003, volume 2, pages 93–96, 2003. 2 [25] R. Tanawongsuwan and A. Bobick. A study of human gaits across different speeds. Technical report, Georgia Tech, 2003. 1 [26] D. Tao, X. Li, X. Wu, and S. Maybank. Human carrying status in visual surveillance. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, volume 2, pages 1670–1677, New York, USA, Jun. 2006. 2 [27] How biometrics could change security. http://news.bbc.co.uk/2/hi/programmes/click online/ 7702065.stm. 1 [28] Y. Wang, S. Yu, Y. Wang, and T. Tan. Gait recognition based on fusion of multi-view gait sequences. In Proc. of the IAPR Int. Conf. on Biometrics 2006, pages 605–611, Jan. 2006. 1 [29] G. Zhao, R. Chen, G. Chen, and H. Li. Amplitude spectrumbased gait recognition. In Proc. of the 6th IEEE Int. Conf. on Automatic Face and Gesture Recognition, pages 23–30, 2004. 2 [30] G. Zhao, R. Chen, G. Chen, and H. Li. Recognition of human periodic movements from unstructured information using a motion-based frequency domain approach. Image and Vision Computing, 24:795–809, 2006. 2