sparse representation and nonlinear resampling is proposed in this paper. First ... of particle diversity impoverishment caused by traditional resampling methods.
Journal of Beijing Institute of Technology, 2018, Vol. 27, No. 1
Particle Filter Object Tracking Algorithm Based on Sparse Representation and Nonlinear Resampling Zheyi Fan 苣 , 摇 Shuqin Weng, 摇 Jiao Jiang, 摇 Yixuan Zhu and Zhiwen Liu ( School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China)
Abstract: Object tracking with abrupt motion is an important research topic and has attracted wide attention. To obtain accurate tracking results, an improved particle filter tracking algorithm based on sparse representation and nonlinear resampling is proposed in this paper. First, the sparse represen鄄 tation is used to compute particle weights by considering the fact that the weights are sparse when the object moves abruptly, so the potential object region can be predicted more precisely. Then, a nonlinear resampling process is proposed by utilizing the nonlinear sorting strategy, which can solve the problem of particle diversity impoverishment caused by traditional resampling methods. Experi鄄 mental results based on videos containing objects with various abrupt motions have demonstrated the effectiveness of the proposed algorithm. Key words: object tracking; abrupt motion; particle filter; sparse representation; nonlinear resam鄄 pling CLC number: TP 391郾 4摇 摇 Document code: A摇 摇 Article ID: 1004鄄 0579(2018)01鄄 0051鄄 07
摇 摇 As an active research topic in the field of im鄄
Therefore, some novel methods have been pro鄄
age processing and computer vision, video real鄄
posed to handle the abrupt motion tracking. On
time object tracking
the one hand, Kwon et al. [4] combined the Wang鄄
[1]
is important in establishing
spatial and temporal coherent relationships of ob鄄 ject motion states between consecutive frames. Yet it is still challenging to guarantee the stability and accuracy of tracking in complex real鄄world scenarios due to occlusion, illumination changes and abrupt motion
[2 - 3]
. Abrupt motions of ob鄄
jects, such as uncertain and fast motions, fast and high dynamic changing directions range, are ubiquitous in the scenes like sport events as well as in low frame rate videos, so tracking these kinds of objects have attracted wide attention. However, most traditional tracking methods cannot accurately track objects with abrupt mo鄄
Landau sampling with Markov Chain Monte Carlo ( MCMC) to propose the WLMC鄄based tracking.
Nguyen et al. [5] utilized sparse estimates of mo鄄 tion direction derived from local features to gener鄄 ate particles by MCMC, which can effectively re鄄 duce the search space and handle abrupt mo鄄 tions. Zhou et al. [6] proposed an adaptive sto鄄 chastic approximation Monte Carlo sampling to solve the problem of abrupt motion tracking. On the other hand, considering that particle鄄 filtering based tracking methods [7] can be effec鄄 tively applied to estimate the object motion states of nonlinear and non鄄Gaussian system, research鄄
tions due to their smooth motion assumptions.
ers have proposed abrupt motion tracking algo鄄
Received摇 2017鄄 03鄄 27 Supported by the National Natural Science Foundation of Chi鄄 na(61701029) 苣 Author for correspondence, lecturer, Ph. D. E鄄mail: funye@ bit. edu. cn DOI: 10. 15918 / j. jbit1004鄄0579. 201827. 0107
od [8 - 10] . Su et al. [8] detect the regions with visual
rithms based on the particle filter ( PF ) meth鄄
saliency as the global proposal distribution and then sample particles from it to avoid suffering from local maxima. Morimitsu et al. [9] combined
— 51 —
Journal of Beijing Institute of Technology, 2018, Vol. 27, No. 1
frame description with attributed relational graphs
ges.
motions in structured sports videos. These PF鄄
state can be predicted by previous observation re鄄
ever, most of them cause the problem of particle
p( x k | z1:k - 1 ) =
with PF, to track multiple objects with abrupt based methods can handle abrupt motions, how鄄 diversity impoverishment. The traditional resam鄄
sults
pling process of PF duplicates particles with large
weights and removes those with small weights, which leaves many repetitive particles in the sam鄄
乙 p( x | x k
k -1
) p( x k - 1 | z1:k - 1 ) dx k - 1
(1)
In the update stage, a posteriori distribution
can be updated by the current observation result
ple set. Thus, the posteriori distribution of object states cannot be accurately represented by these
In the prediction stage, the current object
p( x k | z1:k ) =
p( z k | x k ) p( x k | z1:k - 1 ) p( z k | z1:k - 1 )
(2)
samples. Aiming at this problem, Choi et al. [10]
where p( x k - 1 | z1:k - 1 ) is the posteriori density of
pling particles based on the Gaussian distribution.
p( z k | x k ) is the observation model, and p ( z k |
retained the diversity of particles through resam鄄 To better handle the problem of abrupt mo鄄
tions and particle diversity impoverishment in ex鄄
isting object tracking algorithms, an improved PF object tracking algorithm based on sparse repre鄄 sentation and nonlinear resampling is proposed.
frame k - 1, p( x k | x k - 1 ) is the transition model, z1:k - 1 ) is a normalization constant.
Based on the Monte Carlo sampling, a weigh鄄
ted particle set S = { x ik ,棕 ik } i = 1,…,N is propagated in the state space to approximate the posterior dis鄄 tribution
N
p( x k | z1:k ) 抑 移 棕 ik 啄( x k - x ik )
First, considering the fact that particle weights
i =1
are sparse when object moves abruptly, the
棕 ik 邑 棕 ik -1
sparse representation is used to compute particle weights, which can reconstruct the object of in鄄
terest effectively and further predict the potential
p( z k | x ik ) p( x ik | x ik -1 ) q( x ik | x ik -1 ,z k )
(3) (4)
where 啄(·) is the Dirac function, 棕 ik is the N
object region more accurately. Then, a nonlinear
weight of x ik and meets 移 棕 ik = 1. The particle
strategy is proposed to reserve more kinds of val鄄
and it is in proportion to the observation model p
resampling process based on the nonlinear sorting
i =1
weight can be computed by Eq. (4 ) recursively,
id particles, so the problem of particle diversity
( z k | x k ) when the proposal distribution q ( x ik |
1摇 PF Tracking Method
2摇 Proposed Tracking Method
摇 摇 Based on Monte Carlo importance sampling,
2郾 1摇 Motion model
work to express a posteriori probability of object
tion model describes the transition process be鄄
the empirical conditional probability distribution
model can effectively capture the motion state of
crete particles. The weights and locations of par鄄
accurately gained, thereby making it suitable for
object state by minimum variance. Assume x k and
tion model is
vation result of the kth frame. The tracking
where X k is the predicted state of the interested
impoverishment can be alleviated.
PF uses Bayesian estimation as the main frame鄄
x ik - 1 ,z k ) is replaced by p( x k | x k - 1 ) .
As a basic element of PF tracking, the mo鄄
state. The core of PF tracking method is applying
tween consecutive frames. The random motion
of state system to generate a set of weighted dis鄄
object, whose motion features are difficult to be
ticles are updated in each frame to estimate the
abrupt motions. The definition of the random mo鄄
z k respectively denote the object state and obser鄄
Xk = Xk - 1 + Rk + Uk
process includes the prediction and update sta鄄
(5)
object at time k, U k is white Gaussian noise with
— 52 —
Zheyi Fan et al. / Particle Filter Object Tracking Algorithm Based on Sparse Representation and Nonlinear Resampling
zero mean, R k is the spread radius of particles,
weights of others are roughly zeros, as shown in
ject states changing in the previous t frames
cle weights have sparsity under the situation of
which is proportional to the mean value of the ob鄄 Rk = C
k -1
移 | Xn - Xn - 1 | / t n = k -t
where C is a scaling factor.
(6)
The object is usually denoted by a rectangle,
whose state can be defined as
X = ( x,v x ,y,v y ) T
Fig. 1. Thus, considering the fact that the parti鄄 abrupt motions, the observation model can be re鄄 presented by the linear combination of the fea鄄
tures of all particles, and the coefficients can be calculated by constrained l1 norm minimization.
(7)
where ( x, y) is the coordinate of object region center; ( v x , v y ) denotes the velocity of object in the x and y directions, respectively.
2郾 2摇 Observation model
The observation model describes the object
appearance. A suitable observation can effectively
differentiate the object from the background, which is crucial to the tracking accuracy. The color feature can be easily calculated and is insen鄄
Fig. 1摇 Particle weights for a single object
sitive to the changes of image sizes and viewing angles, so we adopt blocked color histogram [11]
with abrupt motion
Assuming that M denotes the observation
as the observation model. The object region is
model extracted from the object template at initial
space, and then the color histogram is extracted
tracted from the ith particle region, when the
firstly partitioned into 4 sub鄄regions in HSV color
frame, y i ( i = 1,2,…,N) denotes the feature ex鄄
from each sub鄄region. Finally, all 4 histograms
background is invariant as the object moves ab鄄
2郾 3摇 Weights calculation based on sparse rep鄄
other particle weights vanish. So the object model
are concatenated to form a 512鄄bins color feature. resentation
The core idea of the sparse representation
classification ( SRC) method is to reach the spar鄄
ruptly, only a few particles match the object and can be described by the linear representation of all particle feature vectors, namely
M = 棕 1 y1 + 棕 2 y2 + 棕3 y3 + … + 棕 N y N
(8)
sest representation of the coefficient matrix when
where 棕 = ( 棕1 ,棕2 ,棕3 ,…,棕 N ) is the weight vec鄄
method can reduce the importance of feature
Transform Eq. (8) to a l1 norm problem and
the reconstruction error is minimum. Since this
T
tor, and N is the number of particles.
choice and is robust to occlusion, it has been
set each element value of the weight vector be鄄
tracking can be considered as a binary classifica鄄
be described as
widely applied in pattern recognition.
Object
tion problem, which recognizes object region
tween 0 and 1. The optimization problem can then 棕 = arg min椰棕椰1 ïìï ís. t 摇 0臆w i 臆1,i = 1,2,…,N ï î椰M - Y棕椰22 臆着
from background and then tracks the interested object by classification approach.
In the PF tracking framework, a weighted
particle set is used to approximate a posteriori distribution of the object state. When the object moves abruptly, only a few particles close to the object have relatively large weights, while the
(9)
where 着 is the error term, which is a user鄄defined small positive鄄valued parameter. 2郾 4摇 Nonlinear resampling
The resampling process can effectively solve
the problem of particle degradation by duplicating
— 53 —
Journal of Beijing Institute of Technology, 2018, Vol. 27, No. 1
the particles with large weights and removing the
its corresponding weight and avoids the situation
tional resampling method duplicates or removes
approximately zero. The validity and diversity of
ones carrying small weights. However, the tradi鄄 particles depending on their weights only, resul鄄
ting in many repetitive particles in the sample set and causing particle diversity impoverishment. This can reduce the kinds of particles to a great extent and seriously influence the representation
where most particle reservation probabilities are particles are guaranteed.
2郾 5摇 Tracking algorithm
A robust tracking algorithm for the object
with abrupt motions is proposed in this paper. The main procedures are summarized as follows. Step 1摇 Initialization
ability of object state probability distribution when the object moves abruptly. Aiming at this prob鄄
淤 In the initial frame, the initial center posi鄄
lem, this paper proposes an improved resampling
tion of the object is obtained by manual selection
The details are as follows.
(x0 , y0 ). Because the initial motion trend is un鄄
ticle set { x ,i = 1,2,…,N} is sorted in descend鄄
Thus, the initial state of object is denoted as X0 =
于 The sorted indices of each particle are
N} are sampled from the prior distribution p(X0 ).
algorithm based on a nonlinear sorting strategy.
淤 Based on the weights of particles, the par鄄
ing order.
i k
mapped to the reservation probability through a nonlinear function.
and the corresponding coordinate is denoted by known generally, the velocity is set to be zero.
[x0 , 0, y0 , 0]. Then N particles { X0i ,i = 1,2,…, 于 The motion model of object is established
And then the reservation
by Eq. (5) .
1 - r( x ) / C0 r( x ik ) + C1
region is established by extracting its color fea鄄
probability is normalized by p( x ik ) =
寛 x ik ) = p( x ik ) p(
i k
(10)
N
p( x ik ) 移 i =1
(11)
盂 The observation model of the initial object
tures.
Step 2摇 Object tracking
淤 The new particle set in the tth frame is
where C0 and C1 are constants, r( x ) is the sor鄄
predicted by Eq. (5) .
寛 x ik ) is the normalized p( x ik ) . ity of x ik , p(
extracted and the corresponding weight of each
i k
ted index of x , p( x ) is the reservation probabil鄄 i k
i k
盂 The duplicated number of each particle is
determined by its corresponding reservation prob鄄
于 The color features of particle regions are
particle is calculated by Eq. (9) .
盂 The weights are normalized by
ability and the number of all particles, namely 寛 x )夜 n = 骔 N p( i k
i k
(12)
where 骔 x 夜 rounds x to the nearest integer to鄄 wards minus infinity, N is the number of all parti鄄
estimated by
寛 = X k
ber of x ik .
(12) , the lacked particles are complemented by
residual resampling when R < N. Then the new particle set { x jk ,j = 1,2,…,N} can be obtained.
The proposed resampling algorithm based on
the nonlinear sorting strategy allocates the reser鄄
vation probability to each particle depending on
(13)
榆 The motion state of the tracked object is
cles in the sample set, n ik is the duplicated num鄄 榆 After R particles generated by Eqs. (11 )
N
棕 ik 移 i =1
棕 ik = 棕 ik
N
寛i 棕 ik X 移 k i =1
(14)
虞 Particles are resampled based on our pro鄄
posed algorithm in section 2郾 4. 愚 t = t + 1, turn to 淤.
3摇 Experiments and Analysis 摇
摇
In this section, two experiments are de鄄
scribed to prove the effectiveness of the proposed method. First, a single moving point tracking
— 54 —
Zheyi Fan et al. / Particle Filter Object Tracking Algorithm Based on Sparse Representation and Nonlinear Resampling
program is designed to compare the proposed
Gaussian distribution [10] resampling and partial
rithms. Then, the tracking experiments are con鄄
cles is N = 500 and the state vector is
nonlinear resampling with other resampling algo鄄
ducted on videos containing objects with different
systematic [12] resampling. The number of parti鄄 x k + 1 = 0郾 5x k + 25
kinds of abrupt motions, and the tracking results are compared with other approaches. All experi鄄
mented are performed by MATLAB R2014a on a
3郾 10 GHz Intel Core computer with 4 GB of RAM.
3郾 1摇 Performance of the nonlinear resampling
xk + 8cos(1郾 2k) + n k 1 + x2k
y k = 0郾 05x2k + v k
(15) (16)
where x0 ~ N(0,5) , n k ~ N(0,10) and v k ~ N(0,
1) are white Gaussian noises. The tracking errors
First, we design a tracking program based on
are compared in Fig. 2. Since our resampling
posed nonlinear resampling with some typical re鄄
cles during the tracking process, it can obtain
tinomial
other methods.
a one鄄dimensional system to compare the pro鄄
strategy effectively ensures the diversity of parti鄄
sampling methods, like residual resampling, mul鄄
better tracking results with smaller errors than
resampling,
systematic
resampling,
Fig. 2摇 Comparisons of the tracking errors with different resampling methods
摇 摇 The root mean square error ( RMSE) is cal鄄
Tab. 1摇 RMSE results of 6 resampling methods
culated to quantitatively evaluate the performance of each resampling method. The RMSE results
obtained by 6 kinds of resampling strategies are
listed in Tab. 1. The results demonstrate that our nonlinear resampling has the smallest RMSE val鄄 ue.
3郾 2摇 Video object tracking results Tracking experiments
are
conducted
on
several videos including various abrupt motions ,
Resampling method
RMSE results
Residual resampling
17郾 7 伊 10 - 3
Multinomial resampling
43郾 3 伊 10 - 3
Systematic resampling
27郾 1 伊 10 - 3
Gaussian resampling
8郾 1 伊 10 - 3
Partial systematic resampling
10郾 5 伊 10 - 3
Nonlinear resampling
7郾 2 伊 10 - 3
such as low frame rate videos, sudden dynamic
— 55 —
Journal of Beijing Institute of Technology, 2018, Vol. 27, No. 1
changes and multi鄄cameras switching. The track鄄
tion. Although WLMC searches the object in the
PF, WLMC
tracking process and usually deviates to other
ing results are compared among the traditional [4]
and our proposed method. The
number of particles is N = 500 and the observation model is blocked color histogram feature.
The tracking results of each method are
shown in Fig. 3. Fig. 3a and Fig. 3b are Face
[13]
whole state space, it is unstable throughout the wrong locations.
To quantitatively analyze the results of differ鄄
ent methods mentioned above, success rate is
used to evaluate the performance. If the center of
and Animal scenarios for sudden dynamic chan鄄
the ground truth is in the estimated rectangle, the
face that moves left and right rapidly, and the tar鄄
frame [6] . The success rate is represented by the
tive two frames. Fig. 3c is a Boxing sport event
frames and the number of total frames, and is
a low frame rate video of Tennis, constructed
rupt motion is difficult to track successfully, so
this experiment, so a large shift of object position
rithm shows better tracking performance handling
sults reveal that our proposed method can predict
of each algorithm for the test videos. The time
ges. The tracked object in video Face is a human
object is considered accurately tracked at that
get in video Animal jumps fast between consecu鄄
ratio between the number of accurately tracked
in which the camera switches 8 times. Fig. 3d is
shown in Tab. 2. Obviously, the object with ab鄄
manually by keeping one in every 35 frames in
the success rates are relatively low. But our algo鄄
between adjacent frames exists. The tracking re鄄
this challenge. Tab. 3 shows the average run time
and track the object of interest more successfully
cost of our algorithm is longer than the traditional
owing to the effectiveness of the proposed weight calculation method and nonlinear resampling. The traditional PF, by contrast, cannot accurately track the object due to its smooth motion assump鄄
PF method for the introduction of sparse repre鄄
sentation. However, the WLMC needs longer time due to its global search strategy. Thus, our meth鄄 od can obtain better tracking results and has a rel鄄
Fig. 3摇 Tracking results of groundtruth ( solid line) , our algorithm ( dotted line) , the traditional PF ( dash dot line) and WLMC ( dashed line) on video sequences with various kinds of abrupt motions
— 56 —
Zheyi Fan et al. / Particle Filter Object Tracking Algorithm Based on Sparse Representation and Nonlinear Resampling Pattern Analysis and Machine Intelligence, 2014, 36
atively high efficiency.
(7) : 1428 - 1441.
Tab. 2摇 Success rate of 3 methods on test videos %
Method
Face
Animal
Boxing
Tennis
WLMC
74
73
64
23
PF
26
Ours
20
87
80
46 75
Face
Animal
Boxing
Transactions on Pattern Analysis and Machine Intel鄄 ligence, 2013,35(4) : 1011 - 1024.
[5] 摇 Nguyen T, Pridmore T P. Tracking using multiple linear searches and motion direction sampling[ C] 椅
45
Tennis
2014 22nd International Conference on Pattern Rec鄄 s
PF
0郾 239 7
0郾 311 3
0郾 161 1
0郾 211 2
Ours
0郾 556 8
0郾 672 6
0郾 398 8
0郾 468 4
WLMC
0郾 502 1
1郾 052 4
0郾 655 3
tracking methods for abrupt motions [ J ] . IEEE
6
Tab. 3摇 Run time of 3 algorithms on test videos Method
[4] 摇 Kwon J, Lee K M. Wang鄄Landau Monte Carlo鄄based
ognition, Stockholm, Sweden, 2014: 2191 - 2196.
[6] 摇 Zhou X, Lu Y. Abrupt motion tracking via adaptive stochastic approximation Monte Carlo sampling[ C]
椅2010 IEEE Conference on Computer Vision and
Pattern Recognition, San Francisco, CA, USA,
0郾 624 7
2010: 1847 - 1854.
[7] 摇 Doucet A, Gordon N J. Sequential monte carlo methods in practice[ M] . Berlin,Germany: Spring鄄
4摇 Conclusion To track the objects with abrupt motions ac鄄
curately, sparse representation is introduced to calculate the particle weights by l1 norm minimi鄄 zation, which can reconstruct the interested ob鄄
ject better. Moreover, a nonlinear resampling
er, 2001.
[8] 摇 Su Y, Zhao Q, Zhao L, et al. Abrupt motion track鄄
ing using a visual saliency embedded particle filter [ J] . Pattern Recognition, 2014, 47 ( 5 ) : 1826 -
1834.
[9] 摇 Morimitsu H, Bloch I. Attributed graphs for track鄄
ing multiple objects in structured sports videos[ C]
strategy is proposed to improve the traditional re鄄
椅Proceedings of the IEEE International Conference
sampling process. This method gains the duplica鄄 ted number of each particle depending on its cor鄄
on Computer Vision Workshops, Santiago, Chile, 2015: 34 - 42.
responding reservation probability and the number
[10] 摇 Choi H D, Ahn C K, Lim M T. Gaussian distribu鄄
tains the diversity of particles. Experiments show
ceedings of the International Conference on Con鄄
of all particles in the set, so it effectively main鄄
tion resampling algorithm of particle filter[ C] 椅Pro鄄 trol, Automation and Systems, Gwangju, Korea,
that the proposed resampling algorithm has im鄄 proved performance compared to previous algo鄄 rithms, and our tracking method is robust to ab鄄
2013.
[11] 摇 Li Mo, Yin Licheng, Yan Tianyi. Particle filter
tracking algorithm based on integral histogram and
rupt motions.
improved resampling [ J ] . Optics and Electronic Technology, 2013, 11(3) : 45 - 48. ( in Chinese)
References: [1] 摇 Li M, Fan Z Y, Liu Z W. An improved visual track鄄
[12] 摇 Yu J X, Liu W J, Tang Y L. Improved particle filter
algorithms based on partial systematic resampling [ C] 椅 2010 IEEE International Conference on Intel鄄
ing algorithm based on particle filter [ C ] 椅 2013
ligent Computing and Intelligent Systems, Xiamen,
IEEE International Conference of IEEE Region 10, Xi蒺an, Shaanxi, China, 2013:1 - 4.
China, 2010: 483 - 487.
[2] 摇 Cai Z W, Wen L Y, Lei Z, et al. Robust deformable
[13] 摇 Birchfield S. Elliptical head tracking using intensity
[ J] . IEEE Transaction on Image Processing, 2014,
Computer Society Conferenceon Computer Vision
and occluded object tracking with dynamic graph
23(12) : 5497 - 5509.
[3] 摇 Kwon J, Lee K M. Tracking by sampling and integra鄄
ting multiple trackers [ J ] . IEEE Transactions on
— 57 —
gradients andcolor histograms [ C ] 椅 1998 IEEE and Pattern Recognition, Santa Barbara, CA, USA,
1998: 232 - 237.
( Edited by Jianying Cai)
