A novel suboptimal algorithm for state estimation of ... - Springer Link

J Control Theory Appl 2011 9 (2) 148–154 DOI 10.1007/s11768-010-9228-2

A novel suboptimal algorithm for state estimation of Markov jump linear systems Wei LIU, Huaguang ZHANG, Jie FU, Zhanshan WANG School of Information Science and Engineering, Northeastern University, Shenyang Liaoning 110004, China

Abstract: This paper is concerned with state estimation problem for Markov jump linear systems where the disturbances involved in the systems equations and measurement equations are assumed to be Gaussian noise sequences. Based on two properties of conditional expectation, orthogonal projective theorem is applied to the state estimation problem of the considered systems so that a novel suboptimal algorithm is obtained. The novelty of the algorithm lies in using orthogonal projective theorem instead of Kalman filters to estimate the state. A numerical comparison of the algorithm with the interacting multiple model algorithm is given to illustrate the effectiveness of the proposed algorithm. Keywords: State estimation; Markov jump; Linear systems; Orthogonal projective theorem

1

Introduction

In recent years, state estimation problem for Markov jump linear systems (MJLSs), which are linear systems whose parameters switch according to a finite-state Markov chain, has received great attention (see, e.g., [1∼18]) due to its application backgrounds such as tracking of maneuvered objects, fault detection, image processing, and speech recognition. Optimal algorithms for estimating the state of MJLSs have been given in the literature (see, e.g., [1, 2]), which need to exponentially increase computation and memory load with time. Since a prohibitive computational and memory load is needed for optimal estimate of state, it is necessary to consider suboptimal estimation algorithm in practice. A variety of suboptimal algorithms have been proposed (see, e.g., [3∼6, 10, 13∼17]). In [3], the stochastic sample algorithm was proposed, which randomly explores the space of all possible discrete trajectories from an obtainable distribution. In [4], the interacting multiple model (IMM) algorithm was presented by mixing previous hypotheses and considering all of the modes over the most recent sampling periods. In [5], the interacting multiplemodel-extended Viterbi (IMM-EV) algorithm was proposed by incorporating the extended Viterbi algorithm into the IMM algorithm. In [6], a suboptimal algorithm was derived by maintaining a fixed number of candidate paths in a history where each path is identified by an optimal subset of estimated mode probabilities. Among these algorithms, the IMM algorithm is the most popular, which obtains good performance at modest computational load. It is effective in applications such as target tracking [19]. In the IMM algorithm, state estimation is derived by computing model-conditional estimates and weights. Based on a lot of approximations, suboptimal model-conditional estimates and weights are obtained by using Kalman filters and Bayes’ rule. However, excessive approximations are used in the IMM algorithm, which seriously affect the performance and lead to the divergence of the IMM algorithm in many situations.

The above discussion motivates our research. Forty-nine years ago, orthogonal projective theorem was applied to the state estimation problem of linear systems without jump, which led to the famous Kalman filter. Recently, orthogonal projective theorem has been applied to the state estimation problem of MJLSs [17]. In this paper, we will use orthogonal projective theorem and two properties of conditional expectation to obtain a suboptimal algorithm for the state estimation problem of MJLSs in the sense of least mean square error estimation. Just like the IMM algorithm, the proposed algorithm consists of modelconditional estimates and weights. However, compared with the IMM algorithm, the proposed algorithm requires fewer approximations and achieves a better performance. In addition, the proposed algorithm does not increase the computational and storage load as the length of the noise observation sequence increases. This paper is organized as follows. In Section 2, the problem under consideration is formulated and auxiliary lemmas are presented. A recursive suboptimal algorithm is proposed in Section 3. In Section 4, the proposed algorithm is compared with the IMM algorithm. The numerical results indicate that the proposed algorithm outperforms the IMM algorithm. The conclusion is provided in Section 5.

2 Problem formulation and preliminaries Consider the following MJLS: xk+1 = A(θk+1 )xk + B(θk+1 )wk+1 , yk = C(θk )xk + D(θk )vk , k = 0, 1 · · · ,

(1) (2)

where xk+1 ∈ Rn is the unknown state; θk is a known discrete-time Markov chain with finite state space {1, 2, · · · , N } and transition probabilities pij  P (θk+1 = j|θk = i), i, j = 1, 2, · · · , N, s

(3)

wk ∈ R is the state random disturbance; yk ∈ Rp is the noise observation; vk ∈ Rq is the observation random disturbance; A(θk ), B(θk ), C(θk ), and D(θk ) are time-varying matrices of appropriate dimensions; the ini-

Received 17 October 2009; revised 1 February 2010. This work was supported by the National Natural Science Foundation of China (No. 50977008, 60521003, 60774048). c South China University of Technology and Academy of Mathematics and Systems Science, CAS and Springer-Verlag Berlin Heidelberg 2011 

149

W. LIU et al. / J Control Theory Appl 2011 9 (2) 148–154

¯0 tial state x0 is normally distributed with expectation x and covariance matrix P0 , that is, x0 ∼ N (¯ x0 , P0 ); wk and vk are zero-mean normalized white Gaussian noise sequences; and x0 , θk , wi , and vj are mutually independent with i, j = 0, 1, · · · . Given a noise observation sequence k

(y0T , y1T , · · ·

, ykT )T ,

y  optimal state estimate of MJLSs in the sense of least mean square error estimation is E[xk |y k ]. However, exact calculation E[xk |y k ] involves a prohibitive computational and memory load. Hence, in this paper, our aim is to design a computationally feasible suboptimal algorithm that approximately calculates E[xk |y k ]. Remark 1 Since optimal estimate of state involves a prohibitive computational and memory load, suboptimal algorithm has to be considered to limit the computational requirements. Furthermore, we always hope that the approximations in a suboptimal algorithm are as few as possible. In order to obtain the suboptimal algorithm, the following lemmas are considered. Lemma 1 (Orthogonal projective theorem) [20] Suppose that X, Y1 , and 2 are random vectors with second  Y Y1 . Then, it holds that moment. Let Y  Y2 E[X|Y ] = E[X|Y1 ] + Kk (Y2 − E[Y2 |Y1 ]),

(4)

where Kk  E[(X − E[X|Y1 ])(Y2 − E[Y2 |Y1 ])T ] ×(E[(Y2 − E[Y2 |Y1 ])(Y2 − E[Y2 |Y1 ])T ])−1 . Lemma 2 Let X = (Xij )m×n where Xij is random variable, and let Y be random vector. Then, it holds that E[X] = E[E[X|Y ]]. (5) Proof From [20, equation (3.3), p. 106], it follows that E[Xij ] = E[E[Xij |Y ]]. Then, noting that E[X] is defined as (E[Xij ])m×n , we have E[X] = (E[E[Xij |Y ]])m×n . Then, we obtain Lemma 2 by noting that E[E[X|Y ]] is defined as (E[E[Xij |Y ]])m×n . Lemma 3 Let Z be a discrete random vector. Then, it holds that E[X|Y = y]  = P (Z = z|Y = y)E[X|Y = y, Z = z]. (6) z

Proof Using [21, p. 135], we get E[Xij |Y = y]  = P (Z = z|Y = y)E[Xij |Y = y, Z = z]. z

Then, noting that E[X|Y = y] is defined as (E[Xij |Y = y])m×n , we see that E[X|Y = y]  = P (Z = z|Y = y)(E[Xij |Y = y, Z = z])m×n . z

Then, we obtain Lemma 3 by noting that E[X|Y = y, Z = z] is defined as (E[Xij |Y = y, Z = z])m×n .

3

Main result

In this paper, we will propose a novel suboptimal algorithm that approximately calculates optimal state estimate E[xk |y k ]. Before deriving the suboptimal algorithm, we briefly in-

troduce the main idea for the formation of the suboptimal algorithm. From Lemma 3, E[xk |y k ] can be expressed as E[xk |y k ] =

N  i=1

P (θk = i|y k )E[xk |y k , θk = i],

(7)

where P (θk = i|y k ) is called weight and E[xk |y k , θk = i] is called model-conditional estimate. Obviously, P (θk = i|y k ) and E[xk |y k , θk = i] are required to obtain E[xk |y k ]. To calculate P (θk = i|y k ), we use the equation presented in the IMM algorithm. The equation is P (θk = i|y k ) f (yk |y k−1 , θk = i)P (θk = i|y k−1 ) = N  f (yk |y k−1 , θk = j)P (θk = j|y k−1 )

(8)

j=1

with P (θk = j|y k−1 ) N  P (θk−1 = i|y k−1 )P (θk = j|θk−1 = i), = i=1

where f ( · | · ) denotes the conditional density function. In this paper, f (yk |y k−1 , θk = i) will be approximately obtained in the assumption that f (yk |y k−1 , θk = i) is normally distributed, and the concrete derivation for it is given in the following suboptimal algorithm. Using Lemma 1, that is, orthogonal projective theorem, E[xk |y k , θk = i] is rewritten as E[xk |y k , θk = i] = E[xk |y k−1 , θk = i] + E[(xk − E[xk |y k−1 , θk = i]) ×(yk −E[yk |y k−1, θk = i])T ]{E[(yk −E[yk |y k−1, θk = i]) ×(yk − E[yk |y k−1 , θk = i])T ]}−1 ×(yk − E[yk |y k−1 , θk = i]). Obviously, we can obtain E[xk |y k , θk = i] by calculating the following four terms: E[xk |y k−1 , θk = i], E[(xk −E[xk |y k−1 , θk = i])(yk −E[yk |y k−1 , θk = i])T ], E[(yk −E[yk |y k−1 , θk = i])(yk −E[yk |y k−1 , θk = i])T ], E[yk |y k−1 , θk = i]. Based on the above discussion, we will present a novel suboptimal algorithm that approximately calculates E[xk |y k ] below. For notational simplicity, let (i)

x ˆk  E[xk |y k , θk = i],

(k,i)

ˆk+1  E[xk+1 |y k , θk+1 = i], Pi (k)  P (θk = i|y k ), x (k,i)

(i)

¯k  E[xk |θk = i], yˆk+1  E[yk+1 |y k , θk+1 = i], x (i) (¯ xk x ¯T  E[xk xT k) k |θk = i], πi (k)  P (θk = i), i = 1, 2, · · · , N. (i)

(i)

(i) Starting with Pi (k − 1), x ˆk−1 , x ¯k−1 , (¯ xk−1 x ¯T k−1 ) , k = 1, 2, · · · , the suboptimal algorithm is given by the following four steps. Step 1 Compute Pl (k), l = 1, 2, · · · , N , according to

150

W. LIU et al. / J Control Theory Appl 2011 9 (2) 148–154

the following equations: Pi (k − 1)pil = i|y k−1 , θk = l) = N ,  Pj (k − 1)pjl

P (θk−1

(9)

pendix. (l) (l) Step 3 x ¯k and (¯ xk x ¯T are updated from the followk) ing equations (17) and (18), respectively.

j=1

(k−1,l)

yˆk

= C(l)A(l)

N 

i=1

(i)

P (θk−1= i|y k−1, θk = l)ˆ xk−1 , (10)

(k−1,l)

(l) x ¯k

+

N  j=1

−[

j=1

πj (k)C(j)B(j)B(j)T C(j)T

pij πi (k −

i=1 j=1

(k−1,l)

(ˆ yk

P (θk = l|y k−1 ) = Pl (k) =

−

(11) (12)

, θk = l)P (θk = l|y k−1 ) f (yk |y , N  f (yk |y k−1 , θk = i)P (θk = i|y k−1 )

(13)

i=1

where f (yk |y k−1 , θk = i) (k−1,i) T 0.5

)(yk − yˆk

) ]|

(k−1,i) T

)

(k−1,i) (E[(yk − yˆk )(yk (k−1,i) (yk − yˆk )}}.

(k−1,i) T

− yˆk

) ])−1

× The derivation of equations (9)∼(13) is left to the appendix. (l) Step 2 Compute x ˆk from the following equations: N  (k−1,l) (i) x ˆk = A(l) P (θk−1 = i|y k−1 , θk = l)ˆ xk−1 , (14) i=1 (k−1,l)

(k−1,l)

ˆk )(yk − yˆk )T ] E[(xk − x N N   (i) T T pij πi (k − 1)A(j)(¯ xk−1 x ¯T = k−1 ) A(j) C(j) i=1 j=1

+

N  j=1

−[

πj (k)B(j)B(j)T C(j)T

N N  

i=1 j=1 (k−1,l)

−ˆ xk

(i)

(k−1,l) T

pij πi (k − 1)A(j)¯ xk−1 ](ˆ yk [

N  N 

i=1 j=1

i=1

(i)

Pi (k)ˆ xk .

(19)

ˆ0 are extensively used The initial conditions Pl (0) and x by the existing suboptimal algorithms [3∼5, 10, 14] and the derivation for them is omitted. For the rest initial condi(l) (l) x0 x ¯T tions x ¯0 and (¯ 0 ) , since x0 is independent of θ0 , it (l) is easy to obtain from x0 ∼ N (¯ x0 , P0 ) that x ¯0 = x ¯0 T (l) T and (¯ x0 x ¯0 ) = P0 + x ¯0 (¯ x0 ) . Pl (k) and conditional ex(l) (l) (l) ¯k , and (¯ xk x ¯T pectations x ˆk , x k ) . Conditional expecta(l) T (l) xk x ¯k ) are used to calculate E[(xk+1 − tions x ¯k and (¯ (k,i) (k,i) T (k,i) x ˆk+1 )(yk+1 − yˆk+1 ) ] and E[(yk+1 − yˆk+1 )(yk+1 − (k,i)

× exp{−0.5(yk − yˆk ×

N 

(i)

ˆk+1 (see equation yˆk+1 )T ], which are essential to obtain x

(k−1,i)

 1/{|E[(yk − yˆk

(18)

(l)

Pi (k − 1)pil ,

i=1 k−1

πl (k)

The derivation of equation (19) is left to the appendix.

) ,

N 

i=1

x ˆk =

(i) 1)C(j)A(j)¯ xk−1 ]T

(k−1,l) T

+ˆ yk

, (17) πl (k) N  (i) T A(l)[ πi (k − 1)pil (¯ xk−1 x ¯T k−1 ) ]A(l)

The derivation of equations (17) and (18) is left to the appendix. Step 4 Finally, x ˆk is computed according to

(i) (k−1,l) T 1)C(j)A(j)¯ xk−1 ](ˆ yk )

N N (k−1,l)   −ˆ yk [ pij πi (k i=1 j=1

(i)

πi (k − 1)pil x ¯k−1

+B(l)B(l)T .

πj (k)D(j)D(j)T

N N  

i=1

(l) ¯T = (¯ xk x k)

i=1 j=1

N 

=

N 

(k−1,l)

)(yk − yˆk )T ] E[(yk − yˆk N  N  (i) T pij πi (k − 1)C(j)A(j)(¯ xk−1 x ¯T = k−1 ) A(j) ×C(j)T +

A(l)

)

(i)

pij πi (k − 1)C(j)A(j)¯ xk−1 ]T

(k−1,l) (k−1,l) T +ˆ xk (ˆ yk ) , (15) (l) (k−1,l) (k−1,l) (k−1,l) T x ˆk = x ˆk + E[(xk − x ˆk )(yk − yˆk ) ] (k−1,l) (k−1,l) T −1 )(yk − yˆk ) ]} × {E[(yk − yˆk (k−1,l) × (yk − yˆk ). (16)

The derivation of equations (14)∼(16) is left to the ap-

(k,i)

(k,i)

(16)), where E[(yk+1 − yˆk+1 )(yk+1 − yˆk+1 )T ] is also essential to calculate f (yk+1 |y k , θk+1 = j), which is used to calculate Pi (k + 1) (see equation (13)). In this algorithm, (l) ˆk can be obtained Pl (k) can be obtained from (9)∼(13); x (l) (l) from (9)∼(11), (14)∼(16); x ¯k and (¯ xk x ¯T can be obk) (l) (l) tained from (17) and (18), respectively. Pl (k), x ˆk , x ¯k and (i) (i) (l) ¯T can be obtained from Pi (k − 1), x ˆk−1 , x ¯k−1 (¯ xk x k) (i) and (¯ xk−1 x ¯T in a fixed computation load. Hence, the k−1 ) proposed algorithm is finite-dimensionally computable, and does not increase computation and storage load as the length of the noise observation sequence increases. Remark 2 The IMM algorithm and the proposed algorithm are all based on calculating weight Pi (k) and mode(i) conditional estimate x ˆk . For calculating Pi (k), these two algorithms are all based on equation (8), while these two algorithms use different approximate methods for the cal(i) ˆk , culation of f (yk |y k−1 , θk = i) in (8). For calculating x (i) these two algorithms are totally different. x ˆk in the proposed algorithm is calculated by using orthogonal projective theorem (see equations (9)∼(11) and (14)∼(16)) and (i) the calculation of x ˆk does not require any approximation. (i) x ˆk in the IMM algorithm is calculated by using Kalman fil(i) ters and the calculation of x ˆk uses some approximations. From the above discussion, we see that the calculation of (i) ˆk in the IMM algorithm uses apf (yk |y k−1 , θk = i) and x proximations, while the proposed algorithm only requires using approximations in calculating f (yk |y k−1 , θk = i).

W. LIU et al. / J Control Theory Appl 2011 9 (2) 148–154

Hence, the proposed algorithm requires less approximations in contrast to the IMM algorithm. Remark 3 A main method to improve the performance of the suboptimal algorithm is to improve the precision of the approximations used in this algorithm. Since the proposed algorithm uses fewer approximations than the IMM algorithm, the proposed algorithm is easier and more promising to improve performance than the IMM algorithm. Remark 4 It is worth noting that the proposed algorithm becomes an optimal algorithm if f (yk |y k−1 , θk = i) is exact. However, the IMM algorithm does not have such feature.

4

Numerical example

In this section, an example is presented to compare the performance of the proposed algorithm with that of the IMM algorithm. We consider the one-dimensional model of (1) and (2) where θk ∈ {1, 2, 3}; A(1) = 0.2, A(2) = 0.8, A(3) = −0.4; B(1) = 11, B(2) = 22, B(3) = 1;

151

C(1) = 1, C(2) = 1, C(3) = 1; D(1) = 1, D(2) = 2, D(3) = 4; x ¯0 = 0, P0 = 1; π1 (0) = 0.5, π2 (0) = 0.3, π3 (0) = 0.2. Moreover, the transition probability matrix is given by ⎛ ⎞ 0.95 0.025 0.025 P = ⎝ 0.05 0.025 0.925 ⎠ . 0.7 0.2 0.1 For the above one-dimensional model, we run 100 Monte Carlo simulations from k = 0 to 100 on the artificially generated measurements. For an estimation x ˆk of state xk , we call N 1  (ˆ xk − xk )2 (20) N k=0 as the RMS errors of x ˆk − xk from k = 0 to k = N . We select RMS errors as a performance evaluation criteria of these two algorithms. A sample trajectory of state and the corresponding errors are given in Fig. 1. The RMS errors are given in Table 1.

Fig. 1 Trajectory and errors of the IMM algorithm and the proposed algorithm.

It is presented in Table 1 that the RMS errors for the IMM algorithm and the proposed algorithm are 1.652 and 1.233, respectively. From Table 1, it is seen that, compared with the IMM algorithm, the proposed algorithm has the lesser RMS errors. Fig.1 shows that the maximum error of the proposed algorithm is less than that of the IMM algorithm. Hence, the presented results show that the proposed algorithm outperforms the IMM algorithm. Table 1 RMS errors of the IMM algorithm and the proposed algorithm. IMM algorithm

Proposed algorithm

1.652

1.233

RMS error

5

Conclusions

In this paper, a suboptimal algorithm for estimating the state of MJLSs has been proposed. The novelty of the proposed algorithm in contrast to the existing algorithms is using the orthogonal projective theorem instead of Kalman filters to estimate the state. The advantage of the proposed algorithm over the IMM algorithm is that the proposed algorithm requires fewer approximations. Numerical comparison between the IMM algorithm and the proposed algorithm indicates that the proposed algorithm outperforms the IMM

algorithm. References [1] I. Matei, N. C. Martins, J. S. Baras. Optimal state estimation for discrete-time Markovian jump linear systems, in the presence of delayed mode observations[C]//Proceedings of the 2008 American Control Conference. New York: IEEE, 2008: 3560 – 3565. [2] G. A. Ackerson, K. Fu. On state estimation in switching environments[J]. IEEE Transactions on Automatic Control, 1970, 15(1): 10 – 17. [3] A. Doucet, A. Logothetis, V. Krishnamurthy. Stochastic sampling algorithms for state estimation of jump Markov linear systems[J]. IEEE Transactions on Automatic Control, 2000, 45(2): 188 – 201. [4] H. A. P. Blom, Y. Bar-Shalom. The interacting multiple model algorithm for systems with Markovian switching coefficients[J]. IEEE Transactions on Automatic Control, 1988, 33(8): 780 – 783. [5] T. Ho, B. Chen. Novel extended viterbi-based multiple-model algorithms for state estimation of discrete-time systems with Markov jump parameters[J]. IEEE Transactions on Signal Processing, 2006, 54(2): 393 – 404. [6] R. J. Elliott, F. Dufour, W. P. Malcolm. State and mode estimation for discrete-time jump Markov systems[J]. SIAM Journal of Optimization and Control, 2005, 44(3): 1081 – 1104. [7] R. J. Elliott, F. Dufour, W. P. Malcolm. On the performance of Gaussian mixture estimation techniques for dicrete-time jump Markov linear systems[C]//Proceedings of the 45th IEEE Conference on Decision and Control. New York: IEEE, 2006: 314 – 319. [8] L. B. Senthooran, S. Rajsmanoharan, B. C. Williams. Active

152

W. LIU et al. / J Control Theory Appl 2011 9 (2) 148–154 estimation for jump Markov linear systems[J]. IEEE Transactions on Automatic Control, 2008, 53(10): 2223 – 2236.

[9] T. R. Kronhamn. Reduced mode-tree expansion rates in jump Markov estimators[C]//Computational Advances in Multi-sensor Adaptive Processing. New York: IEEE, 2007: 157 – 160.

Derivation of equation (10) Using (1), (2), Lemma 3, and noting that wk and vk are independent of θk , we get (k−1,l)

yˆk

+D(θk )vk |y k−1 , θk = l]

[10] A. Doucet, N. J. Gordon, V. Krishnamurthy. Particle filters for state estimation of jump Markov linear systems[J]. IEEE Transactions on Signal Processing, 2001, 49(3): 613 – 624.

= E[C(θk )A(θk )xk−1 |y k−1 , θk = l] N P = P (θk−1 = i|y k−1 , θk = l)

[11] M. H. Terra, J. Y. Ishihara, A. P. Junior. Array algorithm for filtering of discrete-time Markovian jump linear systems[J]. IEEE Transactions on Automatic Control, 2007, 52(7): 1293 – 1296.

i=1

×E[C(θk )A(θk )xk−1 |y k−1 , θk−1 = i, θk = l] N P = P (θk−1 = i|y k−1 , θk = l)

[12] M. H. Terra, J. Y. Ishihara, G. Jesus. Information filtering and array algorithms for discrete-time Markovian jump linear systems[J]. IEEE Transactions on Automatic Control, 2009, 54(1): 158 – 162.

i=1

×E[C(l)A(l)xk−1 |y k−1 , θk−1 = i, θk = l] N P = P (θk−1 = i|y k−1 , θk = l)

[13] O. L. V. Costa. Linear minimum mean square error estimation for discrete-time Markovian jump linear systems[J]. IEEE Transactions on Automatic Control, 1994, 39(8): 1685 – 1689.

i=1

[14] U. Orguner, M. Demirekler. Risk-sensitive filtering for jump Markov linear systems[J]. Automatica, 2008, 44(1): 109 – 118. [15] I. Rapoport, Y. Oshman. Efficient fault tolerant estimation using the IMM methodology[J]. IEEE Transactions on Aerospace and Electronic Systems, 2007, 43(2): 492 – 508. [16] O. L. V. Costa, S. Guerra. Stationary filter for linear minimum mean square error estimator of discrete-time Markovian jump systems[J]. IEEE Transactions on Automatic Control, 2002, 47(8): 1351 – 1356. [17] W. Liu, H. Zhang, V. Krishnamurthy. On state estimation of discretetime Markov jump linear systems[C]//Proceedings of the 2009 Chinese Control and Decision Conference. New York: IEEE, 2009: 1110 – 1115. [18] P. L. Ainsleigh. Performance of multiple-model filters and parametersensitivity analysis for likelihood evaluation with shock variance models[J]. IEEE Transactions on Aerospace and Electronic Systems, 2007, 43(1): 135 – 149. [19] W. J. Farrell. Interacting multiple model filter for tactical ballistic missile tracking[J]. IEEE Transactions on Aerospace and Electronic Systems, 2008, 44(2): 418 – 426.

×C(l)A(l)E[xk−1 |y k−1 , θk−1 = i, θk = l] N P P (θk−1 = i|y k−1 , θk = l) = i=1

×C(l)A(l)E[xk−1 |y k−1 , θk−1 = i], (a1) where the last equality follows from the Markov property. Then, (i) ˆk−1 , we immediately obreplacing E[xk−1 |y k−1 , θk−1 = i] by x tain (10). Derivation of equation (11) Noticing that wk is independent of xk−1 , θk and vk , and noticing that vk is independent of xk−1 and θk , we obtain from (1) and (2) that =

= E[(C(θk )(A(θk )xk−1 + B(θk )wk ) + D(θk )vk ) ×(C(θk )(A(θk )xk−1 + B(θk )wk ) + D(θk )vk )T ] −E[C(θk )(A(θk )xk−1 + B(θk )wk ) + D(θk )vk ] (k−1,l) T

P (θk−1 = i|y k−1 , θk = l) P (yk−1 , θk = l, θk−1 = i) = P (yk−1 , θk = l) P (yk−1 , θk = l, θk−1 = i) = N P P (yk−1 , θk = l, θk−1 = j) j=1

=

P (yk−1 , θk−1 = i)P (θk = l|yk−1 , θk−1 = i) N P P (yk−1 , θk−1 = j)P (θk = l|yk−1 , θk−1 = j)

j=1

P (yk−1 , θk−1 = i)P (θk = l |θ k−1 = i) = N P P (yk−1 , θk−1 = j)P (θk = l|θk−1 = j) j=1

=

P ( θk−1 = i| y k−1 )P (θk = l|θk−1 = i) N P P (θk−1 = j|y k−1 )P (θk = l|θk−1 = j)

j=1

=

Pi (k − 1)pil . N P Pj (k − 1)pjl

j=1

This completes the derivation of equation (9).

(E[C(θk )(A(θk )xk−1

(k−1,l) (k−1,l) T (ˆ yk ) T T E[C(θk )A(θk )xk−1 xk−1 A(θk ) C(θk )T ]+E[C(θk )B(θk ) ×wk wkT B(θk )T C(θk )T ] + E[D(θk )vk vkT D(θk )T ] (k−1,l) T (k−1,l) yk ) − yˆk (E[C(θk ) −E[C(θk )A(θk )xk−1 ](ˆ (k−1,l) (k−1,l) T T ×A(θk )xk−1 ]) + yˆk (ˆ yk ) . (a2)

+B(θk )wk ) + D(θk )vk ])T + yˆk

= Using Bayes’ rule and applying

(k−1,l)

) − yˆk

×(ˆ yk

[21] S. M. Rossm. Introduction to Probability Models[M]. 9th ed. Oxford: Elsevier, 2007.

Derivation of equation (9) Markov property, we have

(k−1,l) (k−1,l) T )(yk − yˆk ) ] (k−1,l) T (k−1,l) T E[yk yk ] − E[yk ](ˆ yk ) − yˆk E[ykT ] (k−1,l) (k−1,l) T +ˆ yk (ˆ yk )

E[(yk − yˆk

[20] S. Haykin. Adaptive Filter Theory[M]. 4th ed. Upper Saddle River: Prentice Hall, 2002.

Appendix

= E [yk |y k−1 , θk = l] = E[C(θk )(A(θk )xk−1 + B(θk )wk )

From (a2) and Lemma 2, it follows that (k−1,l)

E[(yk − yˆk

(k−1,l) T

)(yk − yˆk

) ]

T T = E[E[C(θk )A(θk )xk−1 xT k−1 A(θk ) C(θk ) |θk−1 , θk ]] N N P P P (θk = j)C(j)B(j)B(j)T C(j)T + P (θk = j) + j=1

j=1

×D(j)D(j)T − E[E[C(θk )A(θk )xk−1 |θk−1 , θk ]] (k−1,l) T

×(ˆ yk

(k−1,l)

) − yˆk

(E[E[C(θk ) (k−1,l)

(k−1,l)

×A(θk )xk−1 |θk−1 , θk ]])T + yˆk (ˆ yk )T N P N P = P (θk−1 = i, θk = j)E[C(θk )A(θk )xk−1 xT k−1 i=1 j=1

×A(θk )T C(θk )T |θk−1 = i, θk = j] + ×B(j)B(j)T C(j)T + −(

N N P P

i=1 j=1

N P j=1

N P j=1

P (θk = j)C(j)

P (θk = j)D(j)D(j)T

P (θk−1 = i, θk = j)E[C(θk )A(θk ) (k−1,l) T

yk ×xk−1 |θk−1 = i, θk = j])(ˆ

(k−1,l)

) − yˆk

153

W. LIU et al. / J Control Theory Appl 2011 9 (2) 148–154 ×(

N N P P

i=1 j=1

P (θk−1 = i, θk = j)E[C(θk )A(θk ) (k−1,l)

(k−1,l)

(ˆ yk )T ×xk−1 |θk−1 = i, θk = j])T + yˆk N N P P = P (θk−1 = i, θk = j)E[C(j)A(j)xk−1 xT k−1 i=1 j=1

×A(j)T C(j)T |θk−1 = i, θk = j] + T

T

×B(j)B(j) C(j) + −(

N N P P

i=1 j=1

N P j=1

N P j=1

P (θk = j)C(j)

P (θk = j)D(j)D(j)T

(k−1,i)

P (θk−1 = i, θk = j)E[C(j)A(j) (k−1,l)

∼ N (ˆ yk

(k−1,l)

(k−1,l)

(k−1,l)

(ˆ yk )T ×xk−1 |θk−1 = i, θk = j])T + yˆk N N P P = P (θk = j|θk−1 = i)P (θk−1 = i)C(j)A(j) T

T

= i, θk = j]A(j) C(j) N P P (θk = j)C(j)B(j)B(j)T C(j)T + P (θk = j) j=1

T

×D(j)D(j) − (

N P N P

P (θk = j|θk−1 = i)

i=1 j=1

=

×P (θk−1 = i)C(j)A(j)E[xk−1 |θk−1 = i, θk = j])T =

i=1 j=1

(k−1,l) T

(ˆ yk

)

P (θk = j|θk−1 = i)P (θk−1 = i)C(j)A(j)

T T ×E[xk−1 xT k−1 |θk−1 = i]A(j) C(j) +

×C(j)B(j)B(j)T C(j)T + −(

N N P P

i=1 j=1

N P j=1

N P j=1

P (θk = j)

P (θk = j)D(j)D(j)T

(k−1,l)

) .

(i) x ¯k−1

(i) (¯ xk−1 x ¯T k−1 ) ,

by pij , πi (k − 1), and equation (11). Derivation of equation (12)

= i]

respectively, yields

By Bayes’ rule, we have that

k−1

) P (θk = l|y N P = P (θk−1 = i|y k−1 )P (θk = l|y k−1 , θk−1 = i) N P i=1

P (θk−1 = i|y k−1 )P (θk = l|θk−1 = i). k−1

and using Bayes’ rule, we obtain that P (θk−1 = i, θk = l) P (θk−1 = i|θk = l) = P (θk = l) P (θk−1 = i)P (θk = l|θk−1 = i) = P (θk = l) πi (k − 1)pil . (a6) = πl (k)

T T = E[A(l)xk−1 xT k−1 A(l) |θk = l] + B(l)B(l) N P = P (θk−1 = i|θk = l) i=1

T ×E[A(l)xk−1 xT k−1 A(l) |θk−1 = i, θk = l]

+B(l)B(l)T N P = P (θk−1 = i|θk = l)A(l) i=1

i=1

=

(a5)

+E[B(θk )wk wkT B(θk )T |θk = l]

P (θk = j|θk−1 = i), P (θk−1 = i), E[xk−1 |θk−1 = i],

P (θk−1 = i|θk = l)A(l)E[xk−1 |θk−1 = i],

T = E[A(θk )xk−1 xT k−1 A(θk ) |θk = l]

Then, replacing E[xk−1 xT k−1 |θk−1

i=1

×(A(θk )xk−1 + B(θk )wk )T |θk = l]

(k−1,l) T

(ˆ yk

N P

P (θk−1 = i|θk = l)A(l)E[xk−1 |θk−1 = i, θk = l]

(l) ¯T = E[xk xT (¯ xk x k) k |θk = l] = E[(A(θk )xk−1 + B(θk )wk )

i=1 j=1

(k−1,l)

i=1

(i)

yk )T − yˆk ×E[xk−1 |θk−1 = i])(ˆ N N P P ×( P (θk = j|θk−1 = i)P (θk−1 = i)C(j)A(j) ×E[xk−1 |θk−1 = i])T + yˆk

N P

¯k−1 , we Using (a5), (a6), and replacing E[xk−1 |θk−1 = i] by x get (17). Derivation of equation (18) Using (1) and Lemma 3, and noting that wk is independent of θk , we have that

P (θk = j|θk−1 = i)P (θk−1 = i)C(j)A(j) (k−1,l)

) ]). (a4)

x ¯k = E[xk |θk = l] = E[A(θk )xk−1 + B(θk )wk |θk = l] = E[A(θk )xk−1 |θk = l] = E[A(l)xk−1 |θk = l] N P P (θk−1 = i|θk = l)E[A(l)xk−1 |θk−1 = i, θk = l] = =

i=1 j=1

(k−1,l)

(k−1,i) T

)(yk − yˆk

i=1

×P (θk−1 = i)C(j)A(j)E[xk−1 |θk−1 = i, θk = j]) N N (k−1,l) T (k−1,l) P P ×(ˆ yk ) − yˆk ( P (θk = j|θk−1 = i) +ˆ yk N N P P

(k−1,i)

, E[(yk − yˆk

(l)

i=1 j=1

×E[xk−1 xT k−1 |θk−1 N P

(k−1,i)

From (8) and (a4), (13) is immediately derived. The derivation of equations (14) and (15) are similar to that of equations (10) and (11), respectively, and is omitted here. From Lemma 1 (orthogonal projective theorem), equation (16) immediately follows. Derivation of equation (17) Since wk is independent of θk , we see from (1) and Lemma 3 that

i=1 j=1

j=1

(k−1,i)

) (yk − yˆk )T ] as an approximation We use E[(yk − yˆk k−1 of Var(yk |y , θk = i). Then, we have f (yk |y k−1 , θk = i)

yk )T − yˆk ×xk−1 |θk−1 = i, θk = j])(ˆ N N P P ×( P (θk−1 = i, θk = j)E[C(j)A(j)

+

Derivation of equation (13) Equation (13) is approximately derived in the assumption that f (yk |y k−1 , θk = i) is normally distributed, which is different from the existing approximation approaches such as hypotheses merging used in many suboptimal algorithms [4, 14]. By the preceding assumption, it follows that ` (k−1,i) ´ , Var(yk |y k−1 , θk = i) . f (yk |y k−1 , θk = i) ∼ N yˆk

(a3)

Replacing P (θk−1 = i|y ) and P (θk = l|θk−1 = i) by Pi (k − 1) and pil , respectively, we immediately obtain (12).

T T ×E[xk−1 xT k−1 |θk−1 = i]A(l) + B(l)B(l) . (a7)

Using (a6), (a7), and replacing E[xk−1 xT k−1 |θk−1 = i] by (i) (¯ xk−1 x ¯T ) , we obtain equation (18). k−1 From Lemma 3, (19) is immediately proved.

154

W. LIU et al. / J Control Theory Appl 2011 9 (2) 148–154 Wei LIU received his M.S. degree from University of Science and Technology Liaoning, China, in 2006, and Ph.D. degree from Northeastern University, China, in 2011, both in Control Theory and Control Engineering. He is currently a lecturer in Henan Polytechnic University. His research interests include multiple-model estimation, and positive definiteness and stability of interval matrices. E-mail: [email protected].

Huaguang ZHANG received his B.S. and M.S. degrees in Control Engineering from Northeastern Dianli University of China, Jilin, China, in 1982 and 1985, respectively, and Ph.D. degree in Thermal Power Engineering and Automation from Southeast University, Nanjing, China, in 1991. He is currently a professor in Northeastern University. His main research interests include dynamics of recurrent neural network, fuzzy control, stochastic system control, nonlinear control, and their applications. E-mail: hg [email protected].

Jie FU received her Ph.D. degree in Control Theory and Control Engineering from Northeastern University, Shenyang, China, in 2009. She is currently a postdoctoral fellow with College of Automation, Chongqing University, Chongqing, China. Her research interests include stability analysis of stochastic neural networks, and nonlinear control. E-mail: [email protected]. Zhanshan WANG received his M.S. degree in Control Theory and Control Engineering from Fushun Petroleum Institute (now Liaoning Shihua University), Fushun, China, in 2001. He received his Ph.D. degree in Control Theory and Control Engineering from Northeastern University, Shenyang, China, in 2006. He is currently an associate professor in Northeastern University. His research interests include stability analysis of recurrent neural networks, fault diagnosis, fault tolerant control, and nonlinear control. E-mail: [email protected].