2011 Canadian Conference on Computer and Robot Vision
A PSO Accelerated Immune Particle Filter for Dynamic State Estimation S. Akhtar, A.R. Ahmad, E. M. Abdel-Rahman, T. Naqvi Systems Design Engineering Dept. University of Waterloo Waterloo, Canada sakhtar;eihab;
[email protected],
[email protected]
may not be optimal. Consequently, the performance of such filters greatly depends upon the degree of nonlinearity in the dynamic process [2]. For nonlinear systems with non Gaussian noise statistics, Particle Filter (PF) is a very effective framework [3]. A PF approximates the Bayes’ filter using Sequential Monte Carlo (SMC) sampling. The SMC based algorithms approximate the posterior densities of the hidden states with a set of particles. Particles are updated and weighted over time and propagated to make the prediction for the next time step. Generally, the performance of the PF is superior to other non-linear filters [2], [3]. However, PF suffers due to ‘curse of dimensionality’ and ‘degeneracy or particle impoverishment’. An infinite number of particles can exactly represent any given pdf. However, this large requirement of the number of particles will add the computational cost. Moreover, the requirement for the number of particles grows exponentially with dimension. Degeneracy occurs, when there are only a few particles in the vicinity of the true state. Only these particles will have significant weights. During resampling step, particles having large weights are statistically selected many times and the particles having negligible weights are discarded. This will lead to many identical samples that do not remain the true representative of the posterior distribution. This is called particle impoverishment. As a result, the generic PF loses the diversity of exploring the whole solution space to find the global solution. Due to degeneracy, the solution obtained may not be a global maxima/minima or the solution may even diverge. In recent years, a lot of research interest has been seen in the use of population based optimization algorithms in the fileds of computer science and engineering [4], [5], [6]. These methods are used to move the solution toward the more probable regions of search space, so are geared to alleviate the degeneracy issue of the generic PF. Recently, a trend towards the use of hybrid optimization methods by combining Particle Swarm Optimization (PSO) and Artificial Immune System (AIS) is seen [7], [8], [9], [10]. In [7], an Immune Particle Swarm Optimization based on receptor editing is proposed. The method works by editing the low affinity particles after each generation. In [8], the antibodies are improved by using PSO before the cloning operation. In
Abstract—Particle Filter (PF) is a flexible and powerful Sequential Monte Carlo (SMC) technique to solve the nonlinear state/parameter estimation problems. The generic PF suffers due to degeneracy or sample impoverishment, which adversely affects its performance. In order to overcome this issue of the generic PF, a Particle Swarm Optimization accelerated Immune Particle Filter (PSO-acc-IPF) is proposed in this work. It combines the robustness and the diversified search capability of the Immune Algorithm (IA) and the speed and the computational efficiency of the Particle Swarm Optimization (PSO) in pursuing the global optimal solution. Mutation plays the key role in the proposed algorithm to help avoid the local optima and search for a global best solution. A two stage mutation operation is proposed. The first stage, with a high mutation rate, helps in exploring a larger solution space and the second stage, with a smaller mutation rate, helps in local optimal search. Later on, PSO is employed to accelerate the convergence speed. To validate the effectiveness of the proposed algorithm, its performance is compared with the generic PF and PSO Particle Filter (PSO-PF). The simulation results have demonstrated better robustness in state estimation for switching dynamic systems. Keywords-Particle filter, dynamic state estimation, soft computing, particle swarm optimization, artificial immune system.
I. I NTRODUCTION The estimation of the hidden states of a process has widespread applications in a variety of work domains such as engineering, economics, physics, chemistry, geology etc. It is widely known as filtering and is extensively used in engineering applications, e.g., navigation, traffic management, robotics, automatic control and modeling, biomedical signal processing, noise removal in a radio signal, fault detection and diagnosis etc. A Bayesian framework is a powerful theoretical tool to help solve these estimation problems. In solving linear dynamical estimation problems, Kalman Filter (KF) [1] has shown a lot of success both in terms of efficiency and efficacy. The KF is based on recursive Bayesian estimation also know as Bayes’ filter. In addition, in solving nonlinear dynamical estimation problems, the Extended Kalman Filter (EKF) [2] and the Unscented Kalman Filter (UKF) [2] has shown success. The EKF and UKF make use of a linear approximation of the nonlinear dynamics around an operating point. This approximation 978-0-7695-4362-8/11 $26.00 © 2011 IEEE DOI 10.1109/CRV.2011.17
72
After obtaining the measurements zk at time step k, the posterior pdf of the state xk is calculated by Bayes’ rule.
[9], information diffusion and clonal selection are combined to achieve a diversified search mechanism and avoid the local optimal solution. In [10], PSO is employed to improve the mutation scheme. Here, the mutation step is replaced by a single iteration of PSO. In order to overcome the effect of particle impoverishment in generic PF, we propose a PSO accelerated Immune Particle Filter (PSO-acc-IPF). The proposed algorithm involves a novel two stage mutation operation to improve the efficacy of the generic PF algorithm. The first stage of mutation is designed to improve the global search capability, while the second stage is designed to be biased towards local search. Furthermore, the convergence toward the optimal solution is accelerated by the use of PSO so as to improve the efficiency of the algorithm. The rest of the paper is organized as follows: In Section II, a brief introduction of the PF, AIS and PSO is given. In Section III, the proposed PSO-acc-IPF is described and explained with implementation details. In Section IV, some simulation results are presented to show the effectiveness of the proposed algorithm. In Section V, some concluding remarks as well as some interesting future research directions are provided.
p(xk |z1:k ) =
Z p(zk |xk ) p(xk |z1:k ) = p(xk |xk−1 ) p(zk |z1:k−1 ) ×p(xk−1 |z1:k−1 )dxk−1
In this section, we briefly describe some of the most popular algorithmic paradigms for solving the nonlinear optimization problems related to this research work. A. Non-linear Bayesian Filtering and Particle Filter Consider the following non-linear system dynamical equation and measurement equation: (1)
zk = hk (xk , νk )
(2)
(4)
(5)
In the update step in Equation 4, the measurement zk is used to modify the prior density to obtain the required posterior density of the current state xk . Whereas the Bayesian solution in Equation 5 recursively calculates the exact posterior density, this recursive propagation of the posterior density is only a conceptual solution [3]. It can not be analytically determined for the general case. Still, the solution is available for a restrictive set of cases, like KF, which assumes that the posterior density at every time step is Gaussian. Nevertheless, if the assumption does not hold then the analytical computation of the integral in Equation 5 may become intractable. In such a situation, approximate solutions such as PF are applied. The PF is a Monte Carlo1 (MC) based simulation technique for implementing a recursive Bayesian filter. The idea is to use samples to represent the posterior density. A generic PF algorithm consists of four steps: initialization, prediction, update and resampling. During the Initialization step, N-samples are drawn from the initial i.i.d distribution. Afterward, the values of the particles are Predicted for the next time step according to the dynamics of the process. During the Update step, each predicted particle is weighted by the likelihood function, which is determined by comparing the predictions with the observations. In the Resampling step, the particles are selected on the bases of their likelihood. It gives birth to some new particles at the expense of less likely particles.
II. O PTIMIZATION PARADIGMS
xk = fk (xk−1 , µk )
p(zk |xk ) × p(xk |z1:k−1 ) p(zk |z1:k−1 )
B. Clonal Selection Theory
where fk and hk are non-linear functions of the states, µk and νk are independent and identically distributed (i.i.d) process and measurement noises, xk and zk are the system states and measurements and k and k − 1 are the current and previous time steps respectively. A Bayesian tracking framework recursively calculates some degree of belief in the state xk at time k based on the given data z1:k (i.e. all the observations up to current time step k) [3]. In other words, it is required to construct the probability density function (pdf) p(xk |z1:k ) (conditional probability density of states xk given all the observations up to current time step k). If the posterior pdf, p(xk−1 |z1:k−1 ), at time k − 1 is available, then the prior pdf of the states at time k can be predicted using the system model as: Z p(xk |z1:k−1 ) = p(xk , xk−1 |z1:k−1 )dxk−1 (3)
The clonal selection theory, proposed by Burnet [11], is an accepted theory for modeling the human immune system. This theory is based on cloning and affinity maturation concept. When the body is exposed to an exogenous agent (antigen), the B-cells with the best binding or affinity with the antigen proliferate by cloning. These cloned cells are called antibodies. To introduce the genetic diversity, a high rate mutation is applied to the cloned cells. The cells with higher affinity to antigen survive. Some of the cells with higher affinity become memory cells. In an Artificial Immune System (AIS), insipired by the clonal selection theory [12], a population of N antibodies 1 Monte Carlo methods are a class of computational algorithms that rely on random samples to compute the results. The technique provides an approximate solution of a complex physical or mathematical problem. They are used in mathematics for the evaluation of complex integrals.
73
III. T HE P ROPOSED PSO ACCELERATED IPF
(solutions) is randomly generated. On the basis of antigen affinity, n antibodies are selected using a selection criteria (elitism, roulette wheel selection etc.). These selected antibodies go through cloning operation. The antibodies with higher affinity generate more clones. Then these cloned antibodies undergo hyper mutation or multiple mutation and receptor editing or diversification operations. These operations lend them the ability to escape local optima. The mutation rate is kept inversely proportional to the antigen affinity. Some of the best members are added to the original population. A percentage of the worst members of the previous population of antibodies is replaced with some new randomly generated solutions which will add diversity to the population. AIS is highly robust, adaptive, self-organized and has an inherently parallel structure [4]. It has powerful learning capabilities and depicts an evolutionary response. It has the ability to escape the local optimum region through mutation operation. It also adds diversification by replacing the worst performing individuals in the population.
This section explains the proposed PSO-acc-IPF in detail. The proposed algorithm has two stages of mutation. At the first stage, the selected antibodies go through a mutation operation, which is inversely proportional to their affinity to antigens. This stage adds global search ability to the antibodies. The second stage of mutation occurs after the cloning process. Here, the selected, antibodies go through a small mutation, which helps in searching for the local best solution. Later on, the PSO is employed to move the antibodies towards the more desired region in the search space. This two stage mutation helps achieving higher diversity to improve the global search as well as better local search for more feasible local regions. The block diagram of the proposed algorithm is given in Figure 1. The step-by-step description of the algorithm is given below.
C. Particle Swarm Optimization The PSO method is inspired by swarm behavior such as a bird flocking [13], where position and direction of motion of each individual in the swarm is influenced by the current direction of motion, the best position in the past and the best position of the entire swarm. Mathematically, each particle updates its velocity and position according to the folowing equations. vk (i) = w · vk−1 (i) + c1 · rand( ) · (gk − xk (i)) +c2 · rand( ) · (pk (i) − xk (i)), xk+1 (i) = xk (i) + vk (i)
∀i
(6) (7)
where xk (i) and vk (i) represent the position and velocity of the ith individual at the k th generation/iteration, pk (i) is the best position of the ith individual until the current generation, gk is the global best position of the whole swarm until the current generation, w is an inertial weight, rand( ) is a uniformly distributed random number in the range [0, 1], c1 and c2 are constant positive integers and are usually selected as c1 = c2 = 2 [13], [14]. PSO has shown considerable success in solving nonlinear, non-differentiable, multimodal optimization problems [13]. The computation of PSO is easy and adds only a slight computational load when it is incorporated into the generic PF framework. It also has the flexibility to control the balance between local and global exploration of the problem space which helps to enhance the search ability. In addition, by deriving the particles towards high likelihood region, it helps to overcome the degeneracy issue of the generic PF.
Figure 1.
Block diagram of PSO-acc-IPF
A. Initilization The basic idea of the Immune System based PF is to consider the particles as the antibodies. Each antibody represents the solution of the target state estimation problem. The initial population of the antibodies are generated using the
74
process model of the system. These antibodies in the initial population later undergo cloning and mutation operations.
is selected for mutation. The selected particle is altered by applying the following mutation operation. xmut,i = xik + β · u k
B. Affinity Computation Affinity defines the binding between the antibodies and the antigens. Each antibody generates an estimate of the states of a dynamical system from where the output can be estimated. This estimated output is compared with the desired output (process output) to find the affinity. In the proposed algorithm, the Euclidean distance between the estimated output and the desired output, smoothened by an exponential kernel, is used as the measure of affinity. q af f inity(i) = δ · e− (ydes − yest )T (ydes − yest ) (8)
(10)
where u ∈ Rn is a uniformly distributed random variable with predefined range and β is the strength of mutation (inversely proportional to the affinity value for the first stage of mutation and very small for the second stage of mutation). E. PSO Convergence After the second stage of mutation, the best child is selected to replace the respective parent. Now the selected antibody (best child) and the antibodies resulting from the first stage of mutation are combined to get a new population. The PSO is applied to this population to move all the antibodies towards the global optima. In this way, degeneracy can be avoided.
where ydes is the desired output, yest is the estimated output and δ is a constant multiplier. C. Cloning
F. Promotion and Suppression of Antibodies
In the cloning process, some antibodies are selected and several neighbors or clones are generated around each selected antibody. The antibodies having more affinity to antigen have better chances of selection for cloning. The Roulette wheel selection method is used where the affinity values are converted to probabilities. The more probable antibodies have more chance of selection. The selected antibodies are regarded as parents and the clones are considered as children, i.e., the children are the neighborhood of their parents. xchild,i−l = xparent,i +α (9) k k
An elitism based selection scheme is used to preserve the best antibodies of the current generation. They will replace some of the antibodies with low affinity in the population. This updated population of antibodies will then be used to generate the estimates of the next iteration of the particle filter. IV. S IMULATION AND E XPERIMENTAL R ESULTS In order to evaluate the performance of the proposed PSOacc-IPF, a number of simulations are performed, as discussed here. The simulation results are compared with other Particle filters in terms of tracking accuracy. In the first example, a non-stationary univariate growth model is considered. In the second example, the tracking model of a vehicle is considered. In the third example, the video sequence of a ball bouncing on the surface of the table is considered.
where xparent,i represents the ith parent at time step k, k child,i−l and xk represents the lth child of the ith parent at time step k. The parameter α, controls the dispersion of the children from their parents. D. Mutation The mutation operator produces spontaneous random changes in the selected antibodies. In the proposed PSOacc-IPF, the mutation step is carried out in two stages . At the first stage, some of the estimates (antibodies) from the system model (PF estimates) are selected for mutation. The mutation is considered to be inversely proportional to the affinity value of the antibody at this stage, i.e., the antibodies with low affinity will undergo more changes and vice versa. This enables the search procedure to surf the whole solution space and lends the ability to avoid being trapped in local optima. At the second stage, mutation operator is applied after cloning. A minor mutation is used here in order to preserve the attributes of the parents in the child antibodies while introducing some diversity in the population. To select an antibody for mutation, first the rate of mutation Pm is selected. For each antibody in the initial population, a random number is drawn from a uniform distribution Um ∼ U [0 − 1]. If Um < Pm , then that particle
A. Univariate Growth Model The univariate model is popular in economics, especially in the evaluation of the performance of estimation methods. It is a non-stationary, highly non-linear model that is also bimodal in nature. It is given by the following set of equations: xk−1 +c3 ·cos(1.2(k−1))+wk (11) xk = c1 ·xk−1 +c2 · 1 − x2k−1 x2k + vk (12) 20 2 where wk ∼ N (0, σw ) and vk ∼ N (0, σv2 ) are mutually independent Gaussian noises, c1 , c2 and c3 are the constants. In our simulation studies, the following values for the constants and the standard deviation of the noises are considered: c1 = 1, c2 = 12, c3 = 7 yk =
75
σw = σv = 2 The process is started with an initial state x0 = 0. A constant velocity model is considered for the PF predictions. Figure 2 shows the tracking performance of a generic PF, Particle Swarm Optimized Particle Filter (PSO-PF) and PSO-acc-IPF. In case of generic PF a total of 300 particles are considered. For PSO-PF, 30 particles and 10 PSO iterations are considered. For PSO-acc-IPF, 30 antibodies are considered in the initial population, 10-antibodies are selected for the first stage of mutation and 10-antibodies are selected for cloning and 170 clones are produced and 5 PSO iterations with 20 particles is carried out to make an even comparison in terms of number of particles utilized. The simulation is run for 100-time steps. The simulation results in Figure 2 show that the tracking performance of the three particle filters is very similar for the considered non-linear system dynamics for the first 50-steps. An abrupt state jump occurs at the time step k = 50. The generic PF loses the track just after the state jump and it is eventually lost completely. The reason for this is that after the state jump, there are no particles in the proximity of the jumped state and all the particles have negligible likelihood. Whereas, in case of PSO-PF, the track is initially lost after the state jump but the system is able to track it back after a few dozen time steps. In contrast, in the case of PSO-accIPF, the tracking performance remains good even after the state jump.
velocity respectively, and ωk is the turn rate of the vehicle at time step k, wk ∼ N (0, Q) and vk ∼ N (0, R) are mutually independent Gaussian noises, uk in a control input to the turning rate, T is the discrete time interval when the observations are taken and zk is a measurement vector. This model produces the effect of missing measurements and during this stage the vehicle moves faster. Q and R are the noise covariance matrices with diagonal covariance coefficients of 12 and 0.12 respectively.
B. Dynamic Vehicle System
The simulation results of a particular run are given in Figure 3 and Figure 4. Here the Figure 3 shows the x-axis position and the Figure 4 gives the y-axis position of the target vehicle. It is clear from the plots that in the case of genetic PF, the target vehicle is completely lost when a sudden speed change is seen. Whereas, the PSO-PF has taken some time to converge. In contrast, the PSO-acc-IPF has shown good tracking performance even after this speed change.
Figure 2.
In this simulation study, an almost constant speed turning vehicle is considered [15]. A constant speed turn is a turn with a constant angular speed along a road of constant radius of curvature. Here, the angular rate is allowed to vary. The discrete time model is given by the following set of equations. sin ωk−1 T ωk−1 T 1 0 1−cos 0 ωk−1 ωk−1 0 cos ωk−1 T 0 − sin ωk−1 T 0 1−cos ωk−1 T sin ωk−1 T xk = 0 1 0 xk−1 + ωk−1 ωk−1 0 sin ωk−1 T 0 cos ωk−1 T 0 0 0 0 0 1 T2 0 0 2 T 0 0 0 T 2 wk + 0 uk 2 0 0 T 1 0 0 (13) 1 0 0 0 0 zk = xk + vk (14) 0 0 1 0 0
Estimation performance with state jump
C. Ball Tracking In this study, the video sequence of a ball bouncing on the surface of the table is considered. The video is captured at a frame rate of 15 frames per second and total of 60 frames are considered. The frame size is 240x320 pixels. The background subtraction is used to extract the moving ball. The centroid of the detected ball region is then computed and is used as the desired ball position. The dynamics of the bouncing ball are described by a set of states xk . A constant velocity model is employed for PF predictions. The position and velocities in x and y direction are the states of the system. A generic PF with 500 particles and the PSO-PF with 50 particles and 10-iterations of PSO run is used. For PSO-acc-IPF, 50 antibodies are used in the initial population, 10-antibodies are selected for first stage of mutation and 20-antibodies are selected for cloning and 200 clones are
where xk = {xk x˙ k yk y˙ k ωk }T represents the state vector. xk and x˙ k represent the x-axis distance and velocity respectively, yk and y˙ k represent the y-axis distance and
76
and the PSO improves the convergence speed towards the global/near-global optimal solution. The algorithm is based on a two stage mutation operation to have a more through exploration of the solution space. This sort of strategy results in improved performance when the target states vary fast or jump abruptly. The simulation results have demonstrated an improved performance of the proposed PSO-acc-IPF for abruptly varying state estimation problems. The proposed algorithm can be made more computationally efficient by adapting the number of particles while maintaining the tracking performance. This will make it more suitable for real time applications. ACKNOWLEDGMENT Figure 3.
The authors would like to thank University of Waterloo and Dalhousie University to support this research work.
x-axis position estimation
R EFERENCES [1] R. E. Kalman, “A new approach to linear filtering and prediction problems,” in Transactions of the ASME–Journal of Basic Engineering, vol. 82, no. Series D, 1960, pp. 35–45. [2] D. Simon, Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches. New Jersey, USA: John Wiley & Sons Inc., 2006. [3] M. S. Arulampalam, S. Maskell, N. Gordom, and T. Clapp, “A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking,” in IEEE Transactions Signal Processing, vol. 50, No. 2, Feb 2002, pp. 8–15. Figure 4.
[4] E.-G. Talbi, Ed., Metaheuristics. From Design to Implementation. Hoboken, New Jersey: John Wiley & Sons Inc., 2009.
y-axis position estimation
[5] A. Ahmad, O. Basir, K. Hassanein, and M. Imam, “An effective module placement strategy for genetic algorithms based layout design,” in Intl Journal of Production Research, vol. 44, No. 8, 2006, pp. 1545–1567.
produced and 5 PSO iterations with 30 particles is used later on. Some selected frames are shown in Figure 5. These frames are selected because there is a change in direction of motion, such as when the ball is bouncing off the ground or when it is coming from the top of its bounce. Here the red circles represent the observed ball position, obtained by background subtraction, and green circles represent the predicted ball position. It is evident from the results that the PSO-acc-IPF has performed the best in tracking the bouncing ball in comparison to both the generic PF and the PSO-PF.
[6] A. Ahmad, “An Intelligent Expert System for Decision Analysis & Support in Multi-Attribute Layout Optimization,” Ph.D. dissertation, University of Waterloo, 2005. [7] H.-W. Ge and Y.-C. Liang, “A hidden markov model and immune particle swarm optimization-based algorithm for multiple sequence alignment,” in Australian Conference on Artificial Intelligence, 2005, pp. 756–765.
V. C ONCLUSION [8] Q. Wang, C. Wang, and X. Z. Gao, “A hybrid optimization algorithm based on clonal selection principle and particle swarm intelligence,” in Proceedings of the Sixth International Conference on Intelligent Systems Design and Applications Volume 02, ser. ISDA ’06, 2006, pp. 975–979.
In this paper, a hybrid PF based on Clonal Selection theory of immune system and PSO is proposed to avoid the degeneracy issue of the generic PF. By incorporating the IA, the diversity to search the whole solution space is added
77
Generic PF
PSO-PF
PSO-acc-IPF
Figure 5. Tracking a bouncing ball, Frame# 14, 19, 27, 31 and 38
78
[9] Y. Lv, S. Li, S. Chen, Q. Jiang, and W. Guo, “Particle swarm optimization based on information diffusion and clonal selection,” in SEAL, 2006, pp. 521–528.
[13] J. Kennedy and R. C. Eberhart, The particle swarm: social adaptation in information-processing systems. Maidenhead, UK, England: McGraw-Hill Ltd., UK, 1999, pp. 379–388. [Online]. Available: http://portal.acm.org/citation.cfm?id=329055.329090
[10] C.-J. Lin, C.-H. Chen, and C.-Y. Lee, “Efficient immunebased particle swarm optimization learning for neuro-fuzzy networks design,” in J. Inf. Sci. Eng., vol. 24, no. 5, 2008, pp. 1505–1520.
[14] T. Guofeng, F. Zheng, and X. Xu, “A particle swarm optimized particle filter for nonlinear system state estimation,” in 2006 IEEE Congress on Evolutionary Computation, Vancouver, BC, Canada, July 16-21, 2006, pp. 438–442.
[11] F. M. Burnet, The clonal selection theory of acquired immunity. Cambridge, UK: Vanderbilt University Press, 1959.
[15] Y. S. Kim and K. S. Hong, “An IMM algorithm for tracking maneuvering vehicles in an adaptive cruise control environment,” Int. J. Control Autom. and Syst., vol. 2, no. 3, pp. 310–318, Sep. 2004.
[12] L. N. de Castro and F. J. V. Zuben, “The clonal selection algorithm with engineering applications,” In GECCO’00, Workshop on Artificial Immune Systems and Their Applications, pp. 36–37, 2000.
79
