Covariance Matrix Adaptation Particle Filter S. Mostapha Kalami Heris
Hamid Khaloozadeh
Control Engineering Department, Faculty of Electrical and Computer Engineering, K. N. Toosi University of Technology, Tehran, Iran e-Mail:
[email protected]
Control Engineering Department, Faculty of Electrical and Computer Engineering, K. N. Toosi University of Technology, Tehran, Iran e-Mail:
[email protected]
Abstract— Based on Covariance Matrix Adaptation Evolution Strategy (CMA-ES) and Particle Filter (PF), an intelligent particle filter, namely Covariance Matrix Adaptation Particle Filter (CMA-PF), is proposed in this paper. Search abilities of CMA-ES are utilized within proposed method to perform Prior Regularization, which helps the particle filter to generate particles with higher importance weights. This helps the CMAPF to operate efficiently and prevents degeneracy and sample impoverishment. According to simulation results, efficiency and applicability of CMA-PF is confirmed. Keywords— Particle Filter; CMA-ES; Intelligent Filtering; Evolutionary Filtering; Nonlinear Filtering; State Estimation.
I.
INTRODUCTION
Problem of state estimation has many applications in various fields of engineering and science, e.g. maneuvering target tracking, state estimation of power systems, weather forecasting, econometrics and robotics. About 40 years ago, study in the field of nonlinear filtering has been started [1]. Using a sequence of noisy measurements, to find the actual values of states of a dynamical system, is the goal of filtering or state estimation problem. Bayesian recursive estimation is the basis for most of filtering methods. Grid-based methods [1,2] for dynamical systems with finite state space, and Kalman Filter (KF) [3] for linear dynamical systems are examples of optimal methods based on Bayesian framework. In addition, there exist suboptimal approaches for general nonlinear state estimation problem, e.g. Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF), and Particle Filter (PF) [1,2]. Since its introduction by Gordon et al. [4], many modifications have been made on particle filter, and variants like Bootstrap Particle Filter (BPF), Auxiliary Particle Filter (APF), Regularized Particle Filter (RPF), Unscented Particle Filter (UPF), and Gaussian Particle Filter (GPF), are proposed [1,2,4,5]. Particle filters perform relatively better than other nonlinear filtering methods. However, there are some problems with particle filters in practice, e.g. degeneracy and sample impoverishment. Intelligent search and optimization methods, can be incorporated into particle filter to enhance the performance of the classic particle filter. According to the literature, several metaheuristics have been used to create intelligent particle filters, e.g. Genetic Algorithm (GA) [6-8], Evolution Strategy (ES) [9,10], Particle Swarm Optimization (PSO) [11,12], Artificial Fish Swarm (AFS) [13], Ant Colony
978-1-4799-3351-8/14/$31.00 ©2014 IEEE
Optimization (ACO) [14,15], Immune Genetic Algorithm (IGA) [16], and Invasive Weed Optimization (IWO) [17]. In this paper, a new intelligent particle filter based on Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is proposed. The proposed intelligent filter is called Covariance Matrix Adaptation Particle Filter or CMA-PF. The intelligent search mechanism is used in sampling phase of proposed method. CMA-PF implicitly uses measurement information to create an adaptive proposal density to achieve better sampling. As a numerical example, state estimation of a nonlinear uncertain HIV infection model is studied and results are presented in this paper. Based on the simulation results, proposed methods, the CMA-PF, outperforms other classic (non-intelligent) particle filters, i.e. SIR, BPF, APF, and RPF, and the Evolutionary Particle Filter (EPF) [8]. The rest of this paper is organized as follows. A brief review on generic particle filter is given in Section II. Section III includes an overview of Covariance Matrix Adaptation Evolution Strategy (CMA-ES). In Section IV, Covariance Matrix Adaptation Particle Filter (CMA-PF) is proposed and explained in detail, and Section V presents simulation results. Finally, Section VI concludes the paper. II.
GENERIC PARTICLE FILTER
General form of discrete-time nonlinear dynamical system is represented by x k +1 = f k (x k , d k ) y k = h k (x k , e k )
(1)
where k is index of discrete time. x k , y k , d k and ek denote state, output, process noise, measurement noise, respectively. The noise signals are assumed to be independent and with known Probability Density Function (PDF). Also initial state of the dynamical system is a random variable with known PDF, p ( x0 ) . Nonlinear filtering problem is the estimation of state x k according to information of measurements up to time k . Functions f k and h k , PDF of process and measurement noises, and PDF for initial state x 0 are assumed to be completely known, in the framework of nonlinear filtering. As mentioned in [1], [2], and [4], nonlinear filtering problem could be treated from a Bayesian perspective. Particle Filter (PF) [1,2,4] is an approximation for general Bayesian recursive
filter, in which statistical approximation is utilized. In the framework of PF, samples are referred to as particles, and the posterior PDF is approximated as a weighted sum of singleton probability density functions. The approximated posterior PDF is given by N
p (x k |Y k ) ≈ pˆ (x k |Y k ) = ∑w ki δ (x k − xik )
(2)
i =1
where N is the number of particles; δ (·) indicates Dirac's delta function; and xik and w ki define position and weight (importance) of the i th particle, respectively. Y k is the set of measurements up to time k . According to (2) and the fact that
pˆ (x k |Y k ) is a PDF, it is needed that
N
∑w
i k
= 1.
i =1
Steps of generic particle filter are listed below: Step 1) Sampling. Positions of particles are sampled from a proposal density function, q (xik | xik −1 ,Y k ) . The most common choice for proposal density function is the state transition probability, p (xik | xik −1 ) . Selection of the proposal PDF is very critical and performance of the particle filter very tightly depends on proposal PDF. Step 2) Importance Update. Importance weights of particles must satisfy w ki ∝
p (xik | x ik −1 ,Y k ) , q (xik | xik −1 ,Y k )
(3)
due to Monte Carlo integration [1], used in the structure of PF. Because of Markovian property of the dynamical system given by (1), update rule of particle weights is as follows [1]. w ki ∝
p (y k | x ik ) p (xik | xik −1 ) i w k −1 q (xik | xik −1 , y k )
(4)
Step 3) Resampling. Resampling is performed to avoid degeneracy in particle filter. In the resampling phase, particles with low importance are removed and replaced with more probable new particles. A good indicator of degeneracy is the number of effective particles. The more the effective particles, the less the degeneracy. A commonly used definition for the number of effective particles is given by N keff
1 N
∑ (w ki )2
.
(5)
i =1
Variants of particle filter use various methods of resampling. For example, in Sampling Importance Resampling (SIR) algorithm, whenever the number of effective particles falls under a pre-specified threshold, the resampling is performed.
III.
COVARIANCE MATRIX ADAPTATION EVOLUTION STRATEGY
Evolution Strategy (ES) was introduced by Rechenberg in early 1960's and its variants were developed further by Rechenberg and Schwefel in 1970's. The concept of adaptation is extensively explored in the ESs, and many variants of classic ES have been developed. The state of the art in the field of ESs is represented by Covariance Matrix Adaptation Evolution Strategy (CMA-ES), introduced by Hansen and Muller [18] as Completely Derandomized Self-adaptation Evolution Strategy, in 2001. CMA-ES is an Estimation of Distribution Algorithm (EDA), which uses a group of dominant and promising solutions to estimate the probability distribution of probable good solutions for an optimization problem. An EDA generates samples from the estimated distribution to create the next generation of candidate solutions, dominant members of which are used to estimate a new distribution. This cycle is repeated until some termination criteria are satisfied. Estimated probability distribution in CMA-ES is assumed to be multivariate Gaussian normal with general form of covariance matrix. Due to limitations on the length of paper, mathematical background of CMA-ES is not described here. Interested readers can read further descriptions in main reference [18], or search for CMA-ES Tutorial in Google, to find the Hansen's introductory tutorial. IV.
COVARIANCE MATRIX ADAPTATION PARTICLE FILTER
As mentioned before, filtering approaches based on general Bayesian estimator work in a recursive manner and calculate posterior PDF according to the prior PDF. In recursion of particle filter, information of particles at time k − 1 , which is contained in {xik −1}iN=1 and {w ki −1 }iN=1 , and measurement data up to time k , which is contained in Y k , are used to create the
new set of particles, denoted by {xik }iN=1 and {w ki }iN=1 . Newly created particles are used to estimate the posterior PDF according to (2). A similar recursion scheme is used for Covariance Matrix Adaptation Particle Filter (CMA-PF), in which a new operation called Prior Regularization is introduced. This operation searches for prior particles, which could lead to better current particles. In the prior regularization phase, CMA-ES is used to solve an optimization problem, solutions of which are corrected prior particles. The new prior particles are advanced in time, using transition probability of the dynamical system, to find the set of current particles. According to (4), it is necessary for any particle in current time to be linked to a particle in the previous time step. Prior regularization is added to CMA-PF, because the CMA-ES does not provide a direct connection between individuals of two successive iterations (generations), and evolution is performed on a PDF, which describes the good and promising individuals. So the CMA-ES could not be incorporated in particle filter, like genetic algorithm which is used within EPF [8].
In the rest of this section, description of CMA-PF steps follows. Inputs of this procedure are set of prior particles {xik −1}iN=1 and information of dynamical system. After termination of a CMA-PF cycle, the set of current particle {xik }iN=1 is returned. Step 1) Selection. Prior regularization is applied to particles with a probability of π pr ∈ [0,1] . To perform prior regularization, the set of all prior particles is partitioned into two parts, namely A and B , which their cardinalities are defined by B = ⎡⎢π pr N ⎤⎥ A =N − B
(6)
After termination of optimization, members of B are replaced by solutions, provided by CMA-ES at iteration T (last iteration). Step 3) Sampling. The set of prior particles is recreated by unification of particles in sets A and B . Current particles {xik }iN=1 , are sampled from transition probability, i.e. xik ~ p (x k | xik −1 ) .
Step 4) Importance Update. Prior particles, which are used to create current particles in previous step, come from two sources. Particles of set A are deterministic, whereas particles of set B are samples of the normal distribution provided by CMA-ES. So, it could be shown that
⎧⎪ p (xik | xik −1 ), i ∈A q (xik ) = ⎨ i i ⎪⎩E{ p (x k | x k −1 )}, i ∈ B
where ⎡⎢ x ⎤⎥ indicates the smallest integer which is greater than or equal to x (ceiling of x ). Step 2) Prior Regularization. The particles of set B , are improved using CMA-ES. The members of set B , are used as initial population of CMA-ES. Some of particles in B , may be changed randomly or replaced by new random particles to achieve more diversity. The CMA-ES is executed for T iterations. The population size and number of parents in CMAES ar both set to be equal to the number of elements in B , i.e. λ = μ = B . The aim of prior regularization is to find prior particles, which could lead to better current particles after a simple state transition. A possible definition for the fitness of a prior particle, located at xik −1 , is as follows: F (xik −1 ) E{ p (y k | xik )}
of a current (new) particle located at xik . In other words, q (xik ) is proposal density, the PDF from which the set of current particles are generated. If i th particle, located at xik −1 , is a member of B , we have xik −1 ~ N (m k −1 , σ k2 −1Ck −1 )
⎧⎪ p (x ik | xik −1 ), i ∈A q (xik ) = ⎨ i i i i ⎪⎩ ∫ p (x k | x k −1 ) N (x k −1 ) dx k −1 , i ∈ B
Pr{i ∈ A } = 1 − π pr
F ( x ik −1 ) = ∫ p (y k | xik ) p ( x ik | x ik −1 ) dxik
Pr{i ∈ B } = π pr
(8)
(13)
(14)
Hence, (13) can be written in the compact form:
The integrand term in (8) is the numerator of fraction in the weight update rule of generic particle filter, denoted by (4). According to Monte Carlo integration rule [1], approximate value of fitness is given by ∀i ∈ B
(12)
where N (xik −1 ) abbreviates N (xik −1 ; m k −1 , σ k2 −1Ck −1 ) . According to the selection step of CMA-PF and (6), it could be said that:
fitness function (7) could be rewritten as
xik −1 ~ N (m k −1 , σ k2 −1Ck −1 )
∀i ∈ B ,
where m k −1 , σ k −1 , and Ck −1 , are mean value, step size, and covariance matrix, returned by CMA-ES at the end of prior regularization phase. Submission of (12) into (11) yields:
where E{X } indicates the expected value of random variable
xik ~ p (x k | xik −1 ) ,
(11)
according to (10), where q (xik ) is the probability of generation
(7)
X , and xik is the next state of the system starting from xik −1 . According to that
(10)
q (xik ) = π pr q cma (xik | m k −1 , σ k2 −1Ck −1 )
CMA-basedPDF
+ (1 − π pr ) p (xik | xik −1 )
Transitional Prior
(9)
where M samples, {xik, j }Mj =1 , are sampled from p (x k | xik −1 ) . Note that, definition of fitness function F (xik −1 ) , in the form of (7), is just one possible case and could be replaced by another problem-dependent function.
(15)
where q cma (xik | m k −1 , σ k2 −1Ck −1 )
∫ p (x
i k
| x ik −1 ) N (x ik −1 ; m k −1 , σ k2 −1Ck −1 ) dxik −1
(16)
According to (4) and (15), weights update rule in CMA-PF is given by
w ki ∝
p (y k | xik ) p (xik | xik −1 ) wi π pr q cma (xik ) + (1 − π pr ) p (xik | xik −1 ) k −1
(17)
The weights are normalized to sum up to unity. Step 5) Resampling. Resampling phase of CMA-PF is same as the standard particle filter. When the number of effective particles, which is defined by (5), falls under a pre-specified threshold, then resampling is performed. As mentioned in [1], by means of Cumulated Sum of Weights (CSW), a discretedomain probability distribution is created over particles set. Sampling from this discrete PDF, results in resampled particles set. This operation is known as Roulette Wheel Selection rule in the field of Evolutionary Computation. The selection process is carried out for N times (number of particles), to produce a new set of particles. Then weights of all resampled particles, are reset to N1 . V. SIMULATION RESULTS In this section, the problem of state estimation of the HIV infection model, assuming parametric uncertainty, is solved using CMA-PF and other variants of particle filter (SIR, BPF, APF, RPF, and EPF), and the algorithms are compared according to the obtained results. Various models are proposed to mathematically describe the Human Immunodeficiency Virus (HIV) infection and its interaction with human immune system [19,20], and optimal control of this system has been addressed in several works [21,22]. The simplest one is the three-state dynamical model [20] for HIV infection, which is presented and studied in several references.
x = λ − dx − β (1 − ηu )xv y = β (1 − ηu )xv − ay v = ky − γ v
(18)
where x is the concentration of healthy CD4+ T-cells; y is the concentration of HIV-infected CD4+ T-cells; and v is the concentration of free HIV copies. In the first state equation, λ is the constant production rate of healthy CD4+ T-cells, d is relative death rate of healthy CD4+ T-cells, and β is aggressiveness of HIV. u is the control input and indicates the usage of antiviral drugs. In general, at time t , the value of input signal u (t ) is in the interval [0,1]. Constant parameter η ∈ [0,1] is efficacy of anti-HIV therapy. In the second state equation, a is death rate of infected CD4+ T-cells. In the third state equation, free viruses are produced by a factor k , proportional to the concentration of infected CD4+ T-cells, and γ is death rate of HIVs. Unit of time in the model (18) is one day. Numerical values for parameters of model (18) are as follows [22]: λ = 1 , d = 0.1 , β = 0.02 , η = 0.98 , a = 0.2 ,
k = 25 , γ = 1 . Initial state of the system (18) is defined as x 0 = 0.4 , y 0 = 4.8 , and v 0 = 119.9 [22]. Parameters of the model are arbitrary and of no clinical significance. Hence units of parameters and states are ignored.
It is possible to measure the concentration of free HIV copies in blood plasma, via counting RNA instances of viruses. Another usual measurement is determination of the concentration of healthy CD4+ T-Cells. Unfortunately, it is not possible to distinguish between healthy and infected CD4+ TCells. So the measurement equation, related to system is given by:
⎡x k ⎡1 1 0 ⎤ ⎢ yk = ⎢ ⎥ ⎢y k ⎣0 0 1 ⎦ ⎢ ⎣v k
⎤ ⎥ + ⎡e1, k ⎤ ⎥ ⎢e 2, k ⎥ ⎦ ⎥⎦ ⎣
(19)
where e1, k ~ N (0, σ 12 ) and e 2, k ~ N (0, σ 22 ) are independent zero-mean Gaussian noises. In this paper, it is assumed that σ 12 = 0.05 and σ 22 = 1 . It should be noticed that, every-time measurement of outputs is not practically possible, and our scenario in this paper is quite artificial from clinical point of view. Parameters of nominal HIV infection model (18) have different values for various individuals. This parametric uncertainty is addressed as follows. It is assumed that constant production rate of healthy CD4+ T-Cells, λ , and aggressiveness of HIV, β , are changes to 0.8 and 0.03, respectively; while estimation algorithms are not informed about the actual parameter values, and they use nominal ones. Estimation time-span is set to 90 days, and a bi-weekly (15 days) periodic input is applied to the system. The control signal is defined by: u (t ) = mod( ⎢⎡
t +1 15
⎧ ⎡t + 1⎤ ⎪1, ⎢ ⎥ is odd ⎢ 15 ⎥ ⎥⎤ , 2) = ⎨ ⎪0, otherwise ⎩
(20)
All of algorithms use 400 particles, except CMA-PF which uses 100 particles. Parameters of EPF are as follows: crossover probability π c = 0.2 , and mutation probability π m = 0.1 . For CMA-PF, prior regularization probability is set to π pr = 0.2 , and maximum number of CMA-ES iterations is T = 5 . Parameters of CMA-ES are set according to [23]. One hundred independent simulations are performed for all of estimation algorithms. Simulation results show that, CMAPF efficiently estimates the states of the modified model, and outperforms all other estimators. In addition, RPF has relatively good performance, compared to other algorithms (except CMA-PF). Actual states of the system and estimated values are presented in Figs. 1, 2 and 3. Mean estimation RMSE for all states is given in Table I.
CONCLUSION
VI.
Concentration of Healthy CD4+ T-Cells and its Estimated Values
An intelligent particle filter based on CMA-ES, namely CMA-PF, is proposed in this paper. Suggested method utilizes search abilities of CMA-ES to perform the so-called Prior Regularization operation, which results in the generation of particles with higher importance. In other words, CMA-PF implements an adaptive structure for proposal PDF, which helps CMA-PF to be safe from degeneracy and sample impoverishment and track abruptly and highly changing states, efficiently. Also according to the obtained results, CMA-PF is able to estimate the state of nonlinear systems which have parametric uncertainties. CMA-PF and some other particle filters are applied to solve state estimation problem of HIV infection model, with uncertainty. Obtained results show applicability and efficiency of the CMA-PF, which outperforms all other approaches. Efficient parameter setting rules for CMA-PF may be one of possible future works.
Utilization of prior regularization with other meta-heuristics, may be another research direction in the field of intelligent filtering. Also application of proposed method in realistic fields like visual tracking and robotics are samples of potential future works. TABLE I.
NUMERICAL INFORMATION ABOUT STATE ESTIMATION RMSE OF HIV INFECTION MODEL
SIR BPF APF RPF EPF CMA-PF
x
y
v
0.7545 (3) 0.7556 (4) 0.7582 (5) 0.6619 (2) 0.7612 (6) 0.2648 (1)
0.8109 (3) 0.8109 (4) 0.8112 (5) 0.6327 (2) 0.8129 (6) 0.2409 (1)
19.9325 (3) 19.9357 (4) 19.9424 (5) 15.3190 (2) 19.9868 (6) 0.9473 (1)
Actual Trajectory
4
SIR BPF APF
2
RPF EPF CMA-PF
0 0
10
20
30
40
Time
50
60
70
80
90
Concentration of Infected CD4+ T-Cells and its Estimated Values
Figure 1. Actual and estimated values for concentration of healthy CD4+ T-Cells of modified HIV infection model
Actual Trajectory
6
SIR BPF APF
4
RPF EPF CMA-PF
2 0
10
20
30
40
Time
50
60
70
80
90
Figure 2. Actual and estimated values for concentration of infected CD4+ T-Cells of modified HIV infection model
Viral Load and its Estimated Values
150
Actual Trajectory SIR BPF APF
100
RPF EPF 50 0
CMA-PF 10
20
30
40
Time
50
60
70
80
90
Figure 3. Actual and estimated values of viral load for modified HIV infection model
REFERENCES [1]
B. Ristic, S. Arulampalam, and N. Gordon, Beyond the Kalman Filter: Particle Filters for Tracking Applications. Artech House, 2004. [2] M. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking,” Signal Processing, IEEE Transactions on, vol. 50, no. 2, pp. 174–188, Feb. 2002. [3] R. E. Kalman, “A new approach to linear filtering and prediction problems,” Transactions of the ASME–Journal of Basic Engineering, vol. 82, no. Series D, pp. 35–45, 1960. [4] N. Gordon, D. Salmond, and A. Smith, “Novel approach to nonlinear/non-gaussian bayesian state estimation,” Radar and Signal Processing, IEE Proceedings F, vol. 140, no. 2, pp. 107–113, Apr. 1993. [5] J. Kotecha and P. Djuric, “Gaussian particle filtering,” Signal Processing, IEEE Transactions on, vol. 51, no. 10, pp. 2592–2601, Oct. 2003. [6] T. Higuchi, “Monte carlo filter using the genetic algorithm operators,” Journal of Statistical Computation and Simulation, vol. 59, pp. 1–23, 1997. [7] F. Pernkopf, “Tracking of multiple targets using online learning for reference model adaptation,” Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 38, no. 6, pp. 1465 –1475, 2008. [8] S. Park, J. P. Hwang, E. Kim, and H.-J. Kang, “A new evolutionary particle filter for the prevention of sample impoverishment,” Evolutionary Computation, IEEE Transactions on, vol. 13, no. 4, pp. 801–809, Aug. 2009. [9] K. Uosaki, Y. Kimura, and T. Hatanaka, “Nonlinear state estimation by evolution strategies based particle filters,” in Evolutionary Computation, 2003. CEC ’03. The 2003 Congress on, vol. 3, Dec. 2003, pp. 2102– 2109. [10] K. Uosaki and T. Hatanaka, “Evolution strategies based particle filters for fault detection,” in Computational Intelligence in Image and Signal Processing, 2007. CIISP 2007. IEEE Symposium on, Apr. 2007, pp. 58– 65. [11] G. Tong, Z. Fang, and X. Xu, “A particle swarm optimized particle filter for nonlinear system state estimation,” in Evolutionary Computation, 2006. CEC 2006. IEEE Congress on, 2006, pp. 438–442. [12] Y. Zheng and Y. Meng, “Swarming particles with multi-feature model for free-selected object tracking,” in Intelligent Robots and Systems,
[13]
[14]
[15]
[16]
[17]
[18]
[19] [20] [21]
[22]
[23]
2008. IROS 2008. IEEE/RSJ International Conference on, Sep. 2008, pp. 2553–2558. L. Xiaolong, F. Jinfu, L. Qian, L. Taorong, and L. Bingjie, “A swarm intelligence optimization for particle filter,” in Intelligent Control and Automation, 2008. WCICA 2008. 7th World Congress on, Jun. 2008, pp. 1986–1991. B. Xu, Q. Chen, X. Wang, and J. Zhu, “A novel estimator with moving ants,” Simulation Modelling Practice and Theory, vol. 17, no. 10, pp. 1663–1677, 2009. J. Zhu, B. Xu, F. Wang, and Z. Wang, “A real-time moving ant estimator for bearings-only tracking,” in Advances in Swarm Intelligence, Y. Tan, Y. Shi, and K. Tan, Eds. Springer Berlin / Heidelberg, 2010, vol. 6145, pp. 273–280. H. Han, Y.-S. Ding, K.-R. Hao, and X. Liang, “An evolutionary particle filter with the immune genetic algorithm for intelligent video target tracking,” Computers and Mathematics with Applications, vol. 62, no. 7, pp. 2685 – 2695, 2011. M. Ahmadi, H. Mojallali, and R. Izadi-Zamanabadi, “State estimation of nonlinear stochastic systems using a novel metaheuristic particle filter,” Swarm and Evolutionary Computation, vol. 4, no. 0, pp. 44 – 53, 2012. N. Hansen and A. Ostermeier, “Completely derandomized selfadaptation in evolution strategies,” Evolutionary Computation, vol. 9, no. 2, pp. 159–195, 2001. M. A. Nowak and R. May, Virus Dynamics: Mathematical principles of immunology and virology. Oxford University Press, USA, 2001. A. Landi et al., “Modelling and control of HIV dynamics,” Comput. Methods Prog. Biomed., vol. 89, pp. 162–168, Feb. 2008. S. M. Kalami Heris and H. Khaloozadeh, “Open- and closed-loop multiobjective optimal strategies for hiv therapy using NSGA-II,” Biomedical Engineering, IEEE Transactions on, vol. 58, no. 6, pp. 1678 –1685, June 2011. J. H. Ko, W. H. Kim, and C. C. Chung, “Optimized structured treatment interruption for HIV therapy and its performance analysis on controllability,” Biomed. Eng., IEEE Trans., vol. 53, no. 3, pp. 380–386, Mar. 2006. N. Hansen, “The CMA evolution strategy: A comparing review,” in Towards a new evolutionary computation: Advances on estimation of distribution algorithms, J. Lozano, P. Larranaga, I. Inza, and E. Bengoetxea, Eds. Springer, 2006, pp. 75–102.
