On the Filtering Problem for Continuous-Time Markov Jump Linear ...

European Journal of Control (2011)4:339–354 © 2011 EUCA DOI:10.3166/EJC.17.339–354

On the Filtering Problem for Continuous-Time Markov Jump Linear Systems with no Observation of the Markov Chaing Oswaldo Luiz do Valle Costa1,∗ , Marcelo Dutra Fragoso2,∗∗ , Marcos Garcia Todorov2 1 2

Depto. Eng. Telecomunicações e Controle, Escola Politécnica da Universidade de São Paulo, CEP 05508-900, São Paulo, S.P., Brazil; National Laboratory for Scientific Computing, LNCC/CNPq, Av. Getulio Vargas 333, Quitandinha, 25651-070, Petrópolis, RJ, Brazil

We consider in this paper the optimal stationary dynamic linear filtering problem for continuous-time linear systems subject to Markovian jumps in the parameters (LSMJP) and additive noise (Wiener process). It is assumed that only an output of the system is available and therefore the values of the jump parameter are not accessible. It is a well known fact that in this setting the optimal nonlinear filter is infinite dimensional, which makes the linear filtering a natural numerically treatable choice. The goal is to design a dynamic linear filter such that the closed loop system is mean square stable and minimizes the stationary expected value of the mean square estimation error. It is shown that an explicit analytical solution to this optimal filtering problem is obtained from the stationary solution associated to a certain Riccati equation. It is also shown that the problem can be formulated using a linear matrix inequalities (LMI) approach, which can be extended to consider convex polytopic uncertainties on the parameters of the possible modes of operation of the system and on the transition rate matrix of the Markov process. As far as the authors are aware of, this is the first time that this stationary filtering problem (exact and robust versions) for LSMJP with no knowledge of the Markov jump parameters is considered in the literature. Finally, we illustrate the results with an example.

∗ Correspondence to: O.L. do Valle Costa, E-mail: [email protected] ∗∗ E-mail: [email protected]

Keywords: Hybrid systems, robust linear estimation, jump parameter, continuous-time linear systems.

1. Introduction There has been by now a steadily rising level of activity with linear systems which are subject to abrupt changes in their structures. This is particularly true for the case in which the abrupt changes are modelled by a Markov chain. Within this context we have the so-called linear systems with Markovian jump parameters (LSMJP) for which a considerable body of techniques have been developed. This class has been the subject of intensive research over the last few years and the associated literature is now fairly extensive (see, e.g., [3], [5], [6], [14], [27], [34], [35], [36], [26], [30] and the references therein). This, in turn, has led to a litany of applications in a variety of fields (see, e.g., [6], [26], and references therein). The application of these models includes, for instance, safetycritical and high-integrity systems (e.g., aircraft, chemical plants, nuclear power station, robotic manipulator systems, large scale flexible structures for space stations such as antenna, solar arrays and more recently wireless and network systems). An enormous impetus to the theory of filtering was given with the appearance of the seminal papers [20], [22]

Received 6 January 2010; Accepted 23 August 2010 Recommended by D. Arzelier, E.F. Camacho

340

and [31], which have been widely celebrated as a great achievement in stochastic systems theory and of fundamental importance in applications. One of the challenging questions that remains in this area is that the description of the optimal nonlinear filter can rarely be given in terms of a closed finite system of stochastic differential equations, i.e., the so-called finite filters (the exceptions are, for instance, the classical Kalman filter, those described, for instance, in [28], the Benes class [1], and also the class studied in [33]). Unfortunately, this is what happens with the optimal nonlinear filter for the LSMJP model when the jump and state variables are not accessible, i.e., in the general setting the filter is not finite. Under some simplifications in the MJLS model, some optimal finite filters have been derived in the literature. For instance, in [2], a finite filter is derived under the simplification hypothesis that the observation process is not fed by the state. Another related finite dimensional filter is derived in [31], but there the variable to be estimated is just the Markov chain (the state variable is assumed to be accessible). In the general setting, this problem has been studied in [37] for the small noise observation scenario, without requiring knowledge of the generator of the Markov chain, with asymptotic optimality proved. In [24], a particular structure for the forward generator of the Markov chain is considered (a lower triangular matrix). Finite filters are also derived in the simplest case in which the Markov chain is accessible and in this setting the specialized literature abounds in results, which includes the H∞ and robust approach ( see, e.g., [4], [25], [29], [32], and references therein ). See also [9], [10], [11], [13], [12], [30], and references therein, for some related results. Considering the general setting of MJLS, in which the optimal nonlinear filter is infinite dimensional, it was derived in [16] the best linear mean square estimator (finite horizon) for such systems (see also, [18]). The approach adopted there produced a filter which bears those desirable properties of the Kalman filter: a recursive scheme suitable for computer implementation which allows some offline computation that alleviates the computational burden. This filter has dimension Nn, with n denoting the dimension of the state vector and N the number of states of the Markov chain. As far as the authors are aware, this was the first attempt to derive an optimal linear filter for the class of continuous-time MJLS. In [19] it was proved that the covariance matrix of the error of this filter converges to a stationary value, which coincides with the unique positive semi-definite solution of a certain algebraic Riccati filter equation (ARE). In view of this, and within the same spirit as in the standard linear case, it was conjectured in [19] that an optimal stationary filter could be derived via the solution of the ARE instead of the solution of the Riccati differential equation associated to the best linear filter. The idea of deriving the best

O. L.V. Costa et al.

linear filter was subsequently used in [8] to analyze the H∞ -filter case, via an LMI approach (see also in [23]). In this case, differently from the case considered in this paper, the optimization criterion was the H∞ norm and the filter parameters were not obtained explicitly in an analytical way. The present paper innovates in the following directions: (i) We derive a dynamic linear filter independent of the jump parameter such that the closed loop system (that is, the filter and state system) is mean square stable and minimizes the stationary expected value of the square estimation error. An explicit analytical solution to this optimal filtering problem is obtained from the stationary solution associated to a certain Riccati equation analyzed in [19]. As far as the authors are aware of, this is the first time that this stationary mean square filtering problem for LSMJP with no knowledge of the Markov jump parameters is considered in the literature. (ii) It is shown that an equivalent formulation based on an LMI approach can be derived for the problem. This formulation allows to extend the problem to handle the case in which there are convex polytopic uncertainties on the parameters of the possible modes of operation of the system and on the transition rate matrix of the Markov process (the robust version). From our knowledge this is the first time that a robust linear stationary mean square filtering problem of LSMJP with no knowledge on the jump parameter has been analyzed in the literature. (iii) It is also shown how approximations of a certain algebraic Riccati equation can yield to a solution of the LMI problem. It should be mentioned that some key assumptions of the formulation proposed here is that the LSMJP must be mean square stable and the Markov chain must be irreducible. Since the Markov jump parameter is not observable we need to deal directly with the second moment matrix of the state variable. Mean square stability and irreducibility of the Markov chain are needed to guarantee the convergence of the second moment matrix of the state variable and of the covariance matrix of the error of the filter to stationary values. A brief outline of the content of this paper is as follows. In section 2 it is presented the problem formulation and assumptions, two key theorems for the LMI approach, and the definition of the stationary filtering problem. A solution for the stationary filter problem based on the ARE approach is presented in section 3. The LMI approach is presented in section 4, including the LMI filter formulation for the case in which there are convex polytopic uncertainties on the parameters of the possible modes of operation of the system as well as on the transition rate matrix

341

Filtering with no Observation of the Markov Chain

(subsection 4.2) and how approximations of the filtering ARE derived in section 3 can yield to a solution of the LMI optimization problem (subsection 4.3). We conclude the paper with a numerical example and final remarks. In the appendix we present the proof of several results. We conclude this section with some basic notation that will be adopted throughout the paper. We shall denote by Rn the n-dimensional Euclidean space and by B(Rn , Rm ) the normed bounded linear space of all m × n matrices with B(Rn ) := B(Rn , Rn ). The n × n identity matrix will be denoted by In or simply by I whenever the dimension is clearly defined. For L ∈ B(Rn ), we denote σ (L) the spectrum of L, L will indicate the transpose of L, and tr(L) the trace of L. As usual, L ≥ 0 (L > 0) will mean that the symmetric matrix L ∈ B(Rn ) is positive semi-definite (positive respectively. we set B(Rn )+ := In addition, definite), n + L ∈ B(R ); L = L ≥ 0 . We use R to denote the interval [0, ∞) and define by L ⊗ K ∈ B(Rsn , Rrm ) the Kronecker product for any L ∈ B(Rs , Rr ) and K ∈ B(Rn , Rm ). Recall also that for L ∈ B(Rn ) and K ∈ B(Rm ) the Kronecker sum is defined as L ⊕ K := L⊗Im + In ⊗K ∈ B(Rnm ). For Di ∈ B(Rn ), i = 1, . . . , N, diag(Di ) stands for an Nn × Nn matrix where the matrices Di , i = 1, . . . , N, are put together corner-to-corner diagonally, with all other entries being zero, and dg(Dj ) stands for an Nn × Nn matrix where only Dj is put together corner-to-corner diagonally with all other entries being zero. In addition, set Hn,m the linear space made up of all N-sequences of matrices V = (V1 , . . . , VN ) with Vi ∈ B(Rn , Rm ), i = 1, . . . , N and, for simplicity, set Hn := Hn,n . For V = (V1 , . . . , VN ) ∈ Hn , U = (U1 , . . . , UN ) ∈ Hn , we write V ≥ U (respectively V > U) if for each i ∈ S, Vi ≥ Ui (respectively Vi > Ui ). We denote by IRe {λ} the real part of a complex number λ. For L ∈ B(Hn , Hn ) we set IRe {λ(L)} := sup {IRe {λ}; λ ∈ σ (L)}. Finally 1{.} stands for the Dirac measure.

IP(θt+h = j|θt = i) =

λij h + o(h), i = j , 1 + λii h + o(h), i = j (4)

where [ λij ] is the stationary N × N transition rate matrix of{θ } with λij ≥ 0, i = j and λi = −λii = j: j =i λij < ∞. We define pij (t):=IP(θt+s = j|θs = i), i, j = 1, . . . , N and denote pi (t):=IP(θt = i), for any i ∈ S. Notice that, in this setting, Pt := (p1 (t), . . . , pN (t)) , satisfies the Kolmogorov forward differential equation dPt /dt = Pt ; P0 = P, t ∈ R+ , where :=[ λij ] . We set pi = limt→∞ pi (t) > 0. A.3) x0 and {θt } are independent of {w(t)} . We recall now the following definition of mean square stability for system (1) (other equivalent versions for mean square stability can be found in [17]). Definition 2.1: System (1) is mean square stable (MSS), if with w(t) = 0 we have that E(x(t)2 ) → 0 whenever t → ∞ for any initial condition x0 and θ0 . Our final assumption is the following. A.4) System (1) is MSS. It is desired to design a dynamic estimator v(t) for v(t) given in (3) of the following form:

2. Problem Formulation and Auxiliary Results

d z(t) = Af z(t)dt + Bf dy(t),

Let (, F, IP) be a complete probability space carrying its natural filtration Ft , t ∈ R+ , as usual augmented by all null sets in the IP–completion of F , and consider the class of hybrid dynamical systems modelled by the following stochastic Markovian jump linear system. For t ∈ R+ : dx(t) = Aθt x(t)dt + Cθt dw(t), x(0) = x0 ,

(1)

dy(t) = Hθt x(t)dt + Gθt dw(t),

(2)

v(t) = Lθt x(t),

{v(t)} denotes the vector in Rr that it is desired to estimate. The jump parameter θt is not observable. Furthermore, we assume that: A.1) W = (w(t), Ft ) , t ∈ R+ is a standard Wiener prop cess in R . A.2) θ = (θt , Ft ) , t ∈ R+ is a homogeneous irreducible Markov process with right continuous trajectories and taking values on the finite set S := {1, 2, . . . , N}. In addition

(3)

where {x(t)} denotes the state vector in Rn (signal process), {y(t)} the output process in Rm , which generates the observational information that is available at time t, and

(5)

z(t), v(t) = Lf

(6)

e(t) = v(t) − v(t),

(7)

where Af ∈ B(Rnf , Rnf ), Bf ∈ B(Rm , Rnf ), Lf ∈ the estimation error. Defining B(Rnf , R r ), and e(t) denotes

z(t) , we have from (1), (2) and (5)-(7) xe (t) = x(t) that 0 Aθt Cθt x (t)dt + dw(t) (8) dxe (t) = Bf Gθt Bf Hθt Af e e(t) = [Lθt − Lf ]xe (t) which is a continuous-time Markovian jump linear system.

342


We define zi (t) := x(t)1{θt =i} ∈ Rn , z(t) := (z1 (t) , . . . , zN (t) ) ∈ RNn , Zi (t) := E zi (t)zi (t) ∈ B(Rn )+ + and Z(t) := E z(t)z(t) = diag(Zi (t)) ∈ B RNn . Set z(t) z(t) , (9) Z(t) := E ⎤ ⎡ U1 (t) ⎥ ⎢ z(t) , (10) U(t) := ⎣ ... ⎦ , Ui (t) := E zi (t) UN (t) z(t)

Z(t) U(t) z(t) = z(t) P(t) := E . (11) z(t) U(t) Z(t) The next definitions are needed to derive a matricial differential equation for the evolution of Z(t) and P(t) as defined in (11), which will be convenient for the LMI formulation of the filter problem. Moreover we will show that P(t) converges to a stationary value P ≥ 0 when t → ∞ provided that Af is a stable matrix. We define (12) A := ⊗ In + diag(Ai ) ∈ B RNn , Gt :=

√ p1 (t)G1

√ G := p1 G1

...

√

pN (t)GN ∈ B RNp , Rm ,

√ ... pN GN ∈ B RNp , Rm ,

Ct := diag( pi (t)Ci ) ∈ B RNp , RNn ,

√

C := diag( pi Ci ) ∈ B R , R

H := H1 L := L1

. . . HN

Np

Nn

(13) (14) (15)

Define now

A 0 ∈ B RNp+nf , RNp+nf , Bf H Af C C:= ∈ B RNp , RNn+nf , Bf G Ct Ct := ∈ B RNp , RNn+nf , Bf Gt

L := L −Lf ∈ B RNn+nf , Rr := A

,

∈ B RNn , Rm ,

. . . LN ∈ B RNn , Rr ,

(16)

(23) (24)

⎧ |λi | ⎪ ⎪ ⎪ ⎨ −λi ϕ,i,j = ⎪ −λj ⎪ ⎪ ⎩ 0

i = j, i = j, j = , . i = j, i = , otherwise

(25)

As shown in Proposition 6.4 in the appendix, ≥ 0. We

1/2 set for ∈ S, = ⊗ In and ϒ := . The next 0 theorem presents a necessary and sufficient condition for MSS of system (8) based on an LMI representation. Theorem 2.1: System (8) is MSS if and only if there exists P > 0, with diag(Qi ) U P= U Z such that N =1

ϒ dg(Q )ϒ < 0.

(26)

Proof: See the appendix. (17)

(18)

and, for matrices Zi ∈ B(Rn ), i ∈ S, set Z = diag(Zi ) ∈ B(RNn ) and define the operator V(Z) as follows: N λij Zi − ( ⊗ In )Z + Z( ⊗ In ) . V(Z) := diag

We have the following theorem establishing the evolution and convergence of P(t) as well as some properties of the associated Lyapunov like equation. Theorem 2.2: P(t) given by (11) satisfies the equation + ˙ = AP(t) P(t) + P(t)A

N =1

ϒ dg(Z (t))ϒ + Ct Ct . (27)

Moreover under assumptions A.1)-A.4) and that Af is sta-

i=1

(19) From (81) and (82) in the appendix we have the following matricial equation for Z(t): ˙ = AZ(t) + Z(t)A + V(Z(t)) + Ct Ct . Z(t)

(22)

and := ϕ,i,j ∈ B RN , ∈ S, where

+ PA + AP

(21)

(20)

t→∞

ble we have that P(t) → P ≥ 0, with P of the following form: Z U P= , Z = diag(Zi ) ≥ 0, U Z

343


and P is the only solution of the equation in V + VA + 0 = AV

X V= R

N =1

R , X

ϒ dg(X )ϒ + CC ,

X = diag(Xi ).

(28)

N =1

−1 d zop (t) = A zop (t)dt + Z(t)H Gt Gt (dy(t) (35)

zop (0) = E(z(0)),

(36)

zop (t) ), zop (t) = z(t) − zop (t), given with Z(t) = E( zop (t) by

Proof: See the appendix. In what follows recall that v(t) = Lz(t) and that e(t) = v(t) − v(t) = Lz(t) − Lf z(t). The stationary filtering problem is posed as follows. Definition 2.2: [The Stationary Filtering Problem] Find (Af , Bf , Lf ) in (5), (6), such that Af is stable and minimizes limt→∞ E(e(t)2 ), that is, minimizes 2 lim E(e(t) ) = tr lim E e(t)e(t) t→∞

t→∞

− H zop (t)dt)

(30)

then V ≥ P (respectively V > P).

= tr L lim P(t)L = tr LPL .

(34)

Theorem 3.1: (Theorem 4.1 in [18] and Theorem 4.1 in [19]) The best linear mean square estimator x(t) for (1) is given by the following filter: xop (t) = N z i=1 op,i (t) where

ϒ dg(X )ϒ + CC ≤ 0

(respectively < 0)

t→∞

T (S) = SH (GG )−1 .

(29)

Furthermore if V satisfies + VA + AV

assumption could be easily removed). We shall assume from now on that for each i ∈ S, Gi Gi > 0. From this hypothesis we get that Gt Gt = N i=1 pi (t)Gi Gi > 0 and N similarly GG = i=1 pi Gi Gi > 0. Define

−1 ˙ = A Z(t)H Gt Gt H Z(t) Z(t) Z(t) + Z(t)A (t) − + Ct Ct + V(Z(t)), and Z(t) = diag(Zi (t)) as in (20). Moreover Z(t) → Z as t → ∞ where Z is the unique positive semi-definite solution of the algebraic Riccati equation −1 ZH GG H Z + CC + V(Z), 0 = A Z + ZA − and Z the unique solution of the linear algebraic equation

(31)

Thus, from Theorem 2.2, we have that the stationary filtering problem can be re-written as the problem offinding

AZ + ZA + V(Z) + CC = 0. Moreover A − T ( Z)H is stable.

LP L (Af , Bf , Lf ) such that Af is stable and minimizes tr

The next result exhibits explicitly the dynamic equation for the optimal stationary filter.

where, according to Theorem 2.2, Z U , Z = diag(Zi ) P= Z U

Theorem 3.2: An optimal filter for the stationary filtering problem is:

(32)

d zop (t) = Af ,op zop (t)dt + Bf ,op dy(t) −1 = A zop (t)dt + ZH GG (dy(t) − H zop (t)dt),

is the unique solution of the equation + PA + AP

N =1

ϒ dg(Z )ϒ + CC = 0.

(33)

zop (t), vop (t) = Lf ,op zop (0) = E(z(0)), with

3. The Algebraic Riccati Equation Formulation

Af ,op = A − T ( Z)H,

Bf ,op = T ( Z),

Lf ,op = L (37)

In this section we present a solution for the stationary filtering problem based on the associated filtering ARE derived in [19]. We first recall some results derived in [18] and [19]. To keep the same notation as in those papers we assume here that Ci Gi = 0 for each i ∈ S (although this

and Z the unique solution of the ARE −1 ZH GG H Z + CC + V(Z) = 0, A Z+ ZA − (38)

344


and Z the unique solution of the linear algebraic equation AZ + ZA + V(Z) + CC = 0.

Notice that

(39)

0 = AZ + ZA + V(Z) + CC .

Moreover the minimal stationary expected value of the square estimation error is

and from (45) and (38) we have that

ZL ). J ∗ := tr(L

(40)

Proof: Consider any Af , Bf , Lf such that Af is stable and write d z(t) = Af z(t) + Bf dy(t),

(45)

−1 ZH GG H Z = 0. A(Z − Z) + (Z − Z)A + (46) For (43) with Uop = Z = Z − Z we have from (46) that H Bf ,op + Zop + Uop Bf ,op HUop + Af ,op

Zop Af ,op + Bf ,op GG Bf ,op =

z(t). v(t) = Lf Consider vop (t) = L zop (t) where zop (t) is given by (35), (36), and e(t) = v(t) − v(t). Noticing that e(t) = Lz(t) − Lf z(t) = Lz(t) − L zop (t) + L zop (t) − Lf z(t) z(t) = L zop (t) + zop (t) − Lf vop (t) − v(t) = L zop (t) + L and from the orthogonality (see [18]) between zop (t) and v(t), we have that vop (t) − vop (t) − v(t) E e(t)2 = tr E L zop (t) +

vop (t) − v(t) × L zop (t) +

Z) T ( Z)H Z + (A − T ( Z)H) Z + ZH T ( + Z(A − T ( Z)H) +T ( Z)GG T ( Z) = −1 ZH GG H Z = 0, A Z + ZA + and similarly for (44), Bf ,op + Uop Af ,op AUop + Zop Hop

= A Z + ZH T ( Z) + Z(A − T ( Z)H) −1 = A Z + ZA + (Z − Z)H GG H Z −1 = A Z + ZA + Z GG H Z = 0. This shows that for the choice of Af ,op , Bf ,op , Lf ,op as in (37) the unique solution Pop of (33) is given in (42) by Z = Z − Z. It is easy to see that in this Zop = Z, Uop = ZL , which shows from (41) that (37) case LPop L = L yields the minimal cost tr(L ZL ).

vop (t) − = tr L Z(t)L + E v(t)2 t→∞ ZL ≥ tr L Z(t)L −→ tr L t→∞ since that, as stated in Theorem 3.1, Z(t) → Z. Combining this with (31) yields that 2 lim E(e(t) ) = tr LPL ≥ tr L (41) ZL . t→∞

On the other hand, consider Af = Af ,op , Bf = Bf ,op , Lf = Lf ,op as in (37). We want to show that in this case the unique solution of (33), denoted by Zop Uop , Zop = diag(Zop,i ), (42) Pop = Uop Zop is given in (42) by Zop = Z, Uop = Zop = Z − Z, that is, we have the following equations satisfied 0 = Bf ,op HUop + Af ,op H Bf ,op Zop + Uop

+ Zop Af ,op + Bf ,op GG Bf ,op ,

(43)

0 = AUop + Zop Hop Bf ,op + Uop Af ,op .

(44)

Remark 3.1: As far as the authors are aware of, the filter in Theorem 3.2 is the first stationary finite optimal filter derived for the setting considered here. Among the favorable features of this filter, vis-à-vis the optimal nonlinear filter, we can mention, for instance, the fact that it is finitedimensional and the filter gain can be calculated off-line, making it more suitable for applications. Another interesting feature of this filter, which distinguishes it from the filter for the case with complete observations of the system operation modes (Markov chain), is that the filter matrix, which is given by A defined in (12), combines the systems modes {Ai ; i = 1, ...., N} with the transition rate matrix in a greater complexity degree than just Ai , as in the complete observations case. A way to flesh out on the rationale behind this is to bear in mind that the transition rate plays also an important role in the estimation problem, as it does in the stability case (the spectral criteria is given in terms of A = ⊗ In2 + diag(Ai ⊕ Ai )). In addition, notice that the Riccati-like-equation (38) depends upon a certain Lyapunov-like-equation (39), i.e., we have to solve

345


first equation (39) and plug it in equation (38) (via V(Z)). This, in turn, adds another degree of complexity for the filter. Finally, it is worth noticing here that the explicit analytical expression of the stationary filter allows us to infer, via Proposition 6.1, that it is devised in such a way that its mean square stability is guaranteed provided the system (1) is MSS. Remark 3.2: Notice that, contrary to [8] and [23], we have in Theorem 3.2 an explicit analytical equation for the stationary filter.

4. The LMI Formulation and Robust Filter In this section we shall formulate the filter problem defined in section 2 as an LMI optimization problem. In addition, we present the robust version of the filter based on this LMI formulation. 4.1. The LMI Formulation We start by making the following definition. Definition 4.1: We say that (P, W , Af , Bf , Lf ) ∈ R1 if it satisfies Q U > 0, Q = diag(Qi ), (47) P= Z U P P L >0 (48) LP W ⎤ ⎡ + PA ϒ1 dg(Q1 ) . . . ϒN dg(QN ) C AP ⎢ dg(Q1 )ϒ −dg(Q1 ) . . . 0 0⎥ 1 ⎢ ⎥ ⎢ .. .. .. .. .. ⎥ < 0. ⎥ ⎢ . . . . . ⎥ ⎢ ⎦ ⎣dg(QN )ϒ 0 . . . −dg(Q ) 0 N N C 0 ... 0 −I (49) From the results of section 2 we have the following result: Proposition 4.1: We have that inf{tr(W ); (P, W , Af , Bf , Lf ) ∈ R1 } = J ∗ .

(50)

Proof: As seen in section 3, there exists an optimal solution (Af ,op , Bf ,op , Lf ,op ) for the stationary filtering problem posed in Definition 2.2 and Pop is as in (42) satisfying (33). ¯ W ¯ , Af , Bf , Lf ) satisfying (47), Let us show first that any (P, ¯ ) > tr(L (48) and (49) will be such that tr(W ZL). From Schur’s complement we have that (48) and (49) are equiv P¯ + P¯ A + N ϒ dg(Q ¯ )ϒ + ¯ > alent to W L P¯ L and A =1 C C < 0. From Theorem 2.1 we have that system (8) is

MSS and, for P(t) as in (11), we have from Theorem 2.2 that P(t) → P as t → ∞ and (31), (33) hold. Furthermore from Theorem 2.2 we have that P¯ > P and from (41) we get ¯ ) > tr( LP L ) ≥ tr(L ZL ) = J ∗ . Let that tr(W L P¯ L ) ≥ tr( us show now that we have a sequence (P , W , Af , Bf , Lf ) that satisfies (47), (48), (49) and that converges to the optimal solution of the minimization problem posed in (50) as ↓ 0. Fix Af = Af ,op , Bf = Bf ,op , Lf = L, and Q U P = , Q = diag(Q ,i ) the unique solution of U Z + N ϒ dg(Q , )ϒ + CC + I = 0, and + P A AP =1 W = LP L + I. Then from Theorem 2.2, P ↓ Pop as ↓ 0, P > 0 for each > 0, and clearly W ↓ L as ↓ 0. LPop N ¯ Since 0 > AP + P A + =1 ϒ dg(Q , )ϒ + CC we have from Schur’s complement that (47), (48) and (49) are satisfied. Taking the limit as ↓ 0 we obtain that (50) holds. We consider from now on that nf = Nn. The next theorem re-writes the problem defined in (50) as an LMI optimization problem. First we need the following definition. Definition 4.2: We say that (Xi , i = 1, . . . , N, Y , W , R, F, J) ∈ R2 if it satisfies the following LMI: X = diag(Xi ), ⎤ ⎡ X X L − J ⎣ X Y L ⎦ > 0, L−J L W ⎡ P(X, Y , F, R, H, A) S(X, Y ) ⎣ S(X, Y ) −D(X) 0 T (X, Y , C, G)

(51) (52) ⎤ T (X, Y , C, G) ⎦ < 0, 0 −I (53)

where P(X, Y , F, R, H, A) = XA + A X A X + Y A + FH + R

XA + A Y + H F + R , A Y + Y A + FH + H F (54)

X 1 S(X, Y ) = Y 1

. . . X N , . . . Y N

(55)

XC T (X, Y , C, G) = , Y C + FG ⎡

dg(X1 ) ⎢ .. D(X) = ⎣ . 0

... .. .

0 .. .

(56) ⎤

⎥ ⎦. . . . dg(XN )

(57)

Theorem 4.1: There exists (P, W , Af , Bf , Lf ) ∈ R1 if and only if there exists (Xi , i = 1, . . . , N, Y , W , R, F, J) ∈ R2 .

346


Moreover if (Xi , i = 1, . . . , N, Y , W , R, F, J) ∈ R2 then by choosing an arbitrary non-singular (Nn × Nn) matrix U, setting P as in (47) with Q = X −1 = diag(Xi−1 ) = diag(Qi ),

(58)

Z = U (X −1 − Y −1 )−1 U,

(59)

(notice that from (52), X > XY −1 X ⇒ X −1 > Y −1 and therefore X −1 − Y −1 > 0), and by making V = Y (Y −1 − X −1 )(U )−1 ,

(60)

Af = V −1 R(U X)−1 ,

(61)

Bf = V −1 F,

(62)

Lf = J(U X)

−1

(63)

we have that (P, W , Af , Bf , Lf ) ∈ R1 . Furthermore

Set X = Q−1 = diag(Xi ), Xi = Qi−1 , i = 1, . . . , N, J = Lf U Q−1 = Lf U X, F = VBf and R = VAf U Q−1 = VAf U X. With these definitions we have that ⎤ ⎡ X X L − J P P L Y L ⎦ , (67) T1 T1 = ⎣ X LP W L−J L W and

⎡

⎤ + PA ϒ1 dg(Q1 ) . . . ϒN dg(QN ) C AP ⎢ dg(Q1 )ϒ −dg(Q1 ) . . . 0 0⎥ 1 ⎢ ⎥ . . . . .. ⎥ ⎢ .. .. .. .. T2 ⎢ . ⎥ ⎢ ⎥ ⎣dg(QN )ϒ 0 . . . −dg(QN ) 0⎦ N 0 ... 0 −I C ⎤ ⎡ P(X, Y , F, R, H, A) S(X, Y ) T (X, Y , C, G) ⎦, S(X, Y ) −D(X) 0 T2 = ⎣ 0 −I T (X, Y , C, G) (68)

inf{tr(W ); (Xi , i = 1, . . . , N, Y , W , R, F, J) ∈ R2 } = J ∗ . (64) Proof: For (Af , Bf , Lf ) fixed, consider P, W satisfying (47)-(49). Without loss of generality, suppose further that U is non-singular (if not, re-define U as U + I so that it is non-singular and > 0 is small enough so that (47)–(49) still hold). As in [21] define Y V P−1 = > 0, V Y where Y > 0 and Y > 0 are Nn × Nn. We have that

and thus from (48) and (49) we have that (52) and (53) are satisfied. Consider now X, Y , W , R, F, J satisfying (51), (52), (53) and P, Af , Bf , Lf as in (58), (59), (61), (62), (63). From (58) and (59) we get that Q − U ZU = Y −1 > 0 and thus from Schur’s complement, (47) holds. From (59) and (60) we get that (65) and (66) hold and therefore we have again that (67) and (68) are satisfied. Pre and pos multiplying (67) by (T1−1 ) and (T1−1 ) respectively and pre and pos multiplying (68) by (T2−1 ) and (T2−1 ) respectively we get that (52) and (53) imply (48) and (49), showing that (P, W , Af , Bf , Lf ) satisfy (47)–(49). Finally from (50) we obtain (64).

QY + UV = I,

(65)

4.2. Robust Filter

U Y + ZV = 0,

(66)

Assume now that A = (A1 , . . . , AN ) ∈ Hn , C = (C1 , . . . , CN ) ∈ Hp,n , H = (H1 , . . . , HN ) ∈ Hn,m , G = (G1 , . . . , GN ) ∈ Hp,m and are not exactly known but instead there are known matrices Aκ = (Aκ1 , . . . , AκN ) ∈ Hn , Cκ = (C1κ , . . . , CNκ ) ∈ Hp,n , Hκ = (H1κ , . . . , HNκ ) ∈ Hn,m , Gκ = (Gκ1 , . . . , GκN ) ∈ Hp,m and transition rate matrices κ , κ κ = 1, . . . , M such that for 0 ≤ ε κ ≤ 1, M κ=1 ε = 1, we have that

and thus Y −1 = Q + UV Y −1 = Q − U Z −1 U < Q, V = −1 −1 −1 −1 U −U QY = U (Y −Q)Y implying that V is nonsingular. Define the non-singular 2Nn × 2Nn matrix −1 Q Y T= , 0 V and the non-singular matrices T1 , T2 as follows: T 0 T1 = , 0 I ⎡ ⎤ T 0 0 0 0 ⎢ 0 dg(Q−1 ) 0 0 0⎥ 1 ⎢ ⎥ ⎢ ⎥ . . T2 = ⎢ 0 . . 0 0 0⎥ ⎢ ⎥ −1 ⎣0 ⎦ 0 0 dg(QN ) 0 0 0 0 0 I

A=

G=

M

εκ Aκ ,

C=

M

ε κ Cκ ,

κ=1

κ=1

M

M

κ=1

ε κ Gκ ,

=

H=

M

ε κ Hκ ,

κ=1

εκ κ .

(69)

κ=1

We assume that each transition rate matrix κ , κ = 1, . . . , M, is irreducible. The following result is proved in the appendix.

347


Proposition 4.2: Let be as in (69). Then is an irreducible transition rate matrix.

M

κ=1 ε

= V(Q) we get, after taking the sum of (72) multiplied by εκ , over κ from 1 to M, that + P(A) + V(Q) AP 0

Proof: See the appendix. Define S κ (X, Y ) as in (55) replacing by κ (defined as in (25) with λκij instead of λij ) and Aκ , Crκ , Grκ as respec√ √ tively in (12), (14), (16) replacing , Ai , pi Gi , pi Ci κ and Cκ by respectively κ , Aκi , Gκi and Ciκ . Set also A r as in (21) and (22) replacing A, C and G by respectively Aκ , Crκ and Grκ . Our next result presents the robust linear filter for system (1)–(3): Theorem 4.2: Suppose that the following LMI optimiza¯ Y¯ , W ¯ , R, ¯ F, ¯ J: ¯ tion problem has an ( −) optimal solution X, inf tr(W ) subject to (51), (52) and for κ = 1, . . . , M, ⎡ ⎤ P(X, Y , F, R, H κ , Aκ ) S κ (X, Y ) T (X, Y , Crκ , Grκ ) ⎣ ⎦ < 0. S κ (X, Y ) −D(X) 0 κ κ T (X, Y , Cr , Gr ) 0 −I (70)

Then for the filter given as in (61)-(63) we have that system ¯ ). (8) is MSS, and limt→∞ E(e(t)2 ) < tr(W Proof: A, H, C, G, as in (69) for some 0 ≤ ε κ ≤ M Consider κ and C as respectively 1, κ=1 ε = 1, and set A, C, G, A in (12), (14), (16), (21) and (22), with pi > 0, i = 1, . . . , N the stationary distribution associated to . Notice that, from Proposition 4.2, is irreducible, which guarantees the existence of the stationary distribution pi > 0. Since (70) holds for each κ = 1, . . . , M, we have from Theorem ¯ , Af , Bf , Lf ) (which, according to 4.1 that there exists (P, W ¯ Y¯ , R, ¯ F, ¯ J) ¯ (58), (59), (61), (62) and (63) depend only on X, ¯ ) > tr( such that tr(W LP L ) and for each κ = 1, . . . , M κ P + P(A κ ) + A

N =1

κ V κ (Q)

ϒκ dg(Q )(ϒκ ) + Crκ (Crκ ) < 0. (71)

From (19) and (88) we have that (71) can be re-written as κ V (Q) 0 κ κ A P + P(A ) + + Crκ (Crκ ) < 0. 0 0 (72) where V κ is as in (19) replacing λij and by respec κ κ tively λκij and κ . Noticing that M κ=1 ε A = A and

M 0 + ε κ Crκ (Crκ ) < 0. 0 κ=1

(73) M κ κ κ κ Cκ − Set Cr = M r κ=1 ε Cr . Since 0 ≤ κ=1 ε Cr − Cr M κ κ κ Cr = κ=1 ε Cr (Cr ) − Cr Cr we conclude that Cr Cr ≤ M κ κ κ ε C (C ) and from (73), κ=1

r

r

V(Q) AP + P(A) + 0

0 + Cr (Cr ) < 0. 0

(74)

Let us show now that CC ≤ Cr (Cr ) . For any vector x = (x1 · · · xN ) and y of appropriate dimensions we have that

x

N

x y CC p i xi = y

Ci Bf G i i=1 ! x i × Ci Gi Bf y N

Ci Ci y x ≤ i Bf G i i=1

x = x y Cr (Cr ) y y

Gi Bf

! x i

y

showing that indeed CC ≤ Cr (Cr ) . Combining this with (72) and (74) and the identity (88) we get that + PA + AP

N

ϒ dg(Q )(ϒ ) + CC < 0.

(75)

=1

From (75) and Theorems 2.1 and 2.2 we get the result. 4.3. ARE Approximations for the LMI Problem In this subsection we show how approximations of the filtering ARE derived in section 3 can yield to a solution of the LMI optimization problem presented in section 4, that is, how they can yield to a sequence of feasible solutions for the LMI (51), (52), (53) which converges to the optimization problem posed in (64). Consider > 0. We have the following theorem (recall the definition of T (.) in (34)). Theorem 4.3: Consider Z the unique positive definite solution of the following ARE −1 0 = A Z + Z A − Z H GG H Z + CC + V(Z ) + I,

(76)

348


where Z = diag(Z ,i ) > 0 is the unique solution of 0 = AZ + Z A + CC + V(Z ) + 2 I.

(77)

−1 , i ∈ S, X = diag(Xi ), Y = Z −1 , F = Set Xi = Z ,i −1 −1 −Z T (Z ), R = −(Y A + FH)(I − Y X), J = L(I − Y −1 X), and W > L Z L . Then with this choice of X, Y , W , R, F, J we have that (51), (52), (53) are satisfied and moreover, by taking U = (X −1 − Y −1 ), we have Z )H, Bf = T ( Z ), from (61), (62), (63) that Af = A − T ( Lf = L, P as in (47) is given by Z − Z Z P = , (78) Z − Z Z − Z

Z L ). L ) = tr(L and tr( LP Proof: See the appendix. Remark 4.1: As ↓ 0 it is easy to see that Z ↓ Z and Z, where Z and Z are as in Theorem 3.1. Therefore the Z ↓ approximating ARE solutions in Theorem 4.3 indeed lead to an optimal solution for the LMI optimization problem posed in (64).

5. A Numerical Example In this section we carry out a numerical example in order to illustrate the performance of the filters derived here. We confine our discussion performing the comparison vis-àvis the optimal filter for the case with complete observation of the regime (from now on denoted by COF), which can be found in [26]. In order to give just a glimpse of the performance, we focus our attention on the Riccati and LMI versions of the stationary filter. Consider the two-mode scalar system given by: ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ −1 0.1 0 A2 C2 A1 C 1 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 0 0.2 ⎦ , ⎣ H2 G2 ⎦ ⎣H1 G1 ⎦ = ⎣ 0 L1 1 − − L2 − ⎤ ⎡ 0.1 0.15 0 ⎥ ⎢ 0 0.3 ⎦ , =⎣ 1 1

−

−

with the transition rate matrix −4 4 = . 2 −2 Suppose also that x0 is independent of θ0 , so that i = E(zi (0)) = E(x0 1{θ0 =i} ) = E(x0 )IP(θ0 = i), 1, 2, with x0 gaussian and with xˆ 0 := E(x0 ) = 0.2, E[(x0 − xˆ 0 )2 ] = 0.01. Finally, let us assume that the distribution of θ0 is identical to the invariant distribution

of θ: p1 := IP(θ0 = 1) = 1/3, p2 := IP(θ0 = 2) = 2/3. A solution obtained for the stationary filtering problem of Theorem 4.2 (Riccati equation formulation) is given by "

Af ,op

Bf ,op

Lf ,op

−

#

⎡

−5.0000 = ⎣ 4.0000 1.0000

2.0489 −2.2652 1.0000

⎤ −0.0489 0.3652 ⎦ −

with zôp (0) = (0.0667, 0.1333). As for the filter corresponding to the LMI formulation, we found that "

Af

Bf

Lf

−

#

⎡

3.2638 ⎢ = ⎣ −1.3719 0.0052

61.7401 −23.2150 0.0142

⎤ −1.7448 × 103 0.6665 × 103 ⎥ ⎦ −

is a feasible solution to (5.7)–(5.9), with U = I, zôp (0) = (0.0667, 0.1333), and inf tr(W ) = 0.0311 ≈ lim E(e(t)2 ). t→∞

(79)

For this example, we performed a Monte Carlo simulation of 100 randomly generated trajectories of the system, and compared the obtained results for each stationary filter (Riccati and LMI versions) with those of the COF (see Section 5.3 of [26]) over 50 time units. The mean values for the corresponding quadratic errors (that is, the mean of e(t)2 for 100 simulations) on different time instants is shown in Table 1. As one can see, all values are close to inf tr(W ) in (79). A plot of the quadratic estimation errors for each simulation is depicted in Figs. 1–3, where thick lines correspond to the average trajectories, and the dashed lines represent inf tr(W ) in (79). Also, a comparison of the average estimation errors between each of the filters is presented in Fig. 4. Finally, a sample trajectory of the estimation problem is shown in figure 5. From the simulations we can see that, as expected, the COF performs better, although the estimation errors are quite similar. Perhaps this similarity has to do with the required stability for the system, although we have not considered an exhaustive number of examples. The simulations seem, prima facie, strongly support the theoretical results. Table 1. Average squared estimation error over time (100 simulations). Time units

Riccati

LMI

COF

5 10 15 25 50

0.0353 0.0301 0.0301 0.0268 0.0304

0.0357 0.0302 0.0301 0.0268 0.0303

0.0348 0.0263 0.0265 0.0252 0.0265

349


Fig. 1. Squared estimation errors for the Riccati filter.

Fig. 2. Squared estimation errors for the LMI filter.

Fig. 3. Squared estimation errors for the COF.

Fig. 4. Comparison of average squared errors via Monte Carlo (100 simulations).

6. Conclusion In this paper we have considered the linear stationary minimum expected square estimation error filtering problem for LSMJP in the case in which, both, the base-state {x(t)} and the regime {θt } are not accessible. Although much work has been done in this theme, the problem still remains a challenging one, since the optimal nonlinear filter is not

finite. In view of this, either it is considered a particular structure for the Markov processes or some approximated filters. This paper is, to some extent, a continuation of previous papers by the authors (see, e.g., [18] and [19]). It is shown that an explicit analytical solution to the optimal linear filtering problem is obtained from the stationary solution associated to a certain Riccati equation analyzed in a

350


Fig. 5. Sample trajectory of the estimation problem.

preceding paper [19]. The result is tailored in such a way that it requires just to solve off-line a set of algebraic equations. Moreover we formulate the LMI and robust versions for the optimal stationary linear filtering problem and show how approximations of a certain algebraic Riccati equation can yield to a solution of the LMI problem. It is perhaps noteworthy here that, when compared with the Kalman filter and the optimal filter for the case in which the regime is known, our filter has an additional complexity in the sense that the innovation gain is given in terms of two algebraic equations (see equations (38) and (39)). Notice that our filter by-pass the estimation of the regime {θt }, although in some applications we do need to identify it. Following an idea in [30], we could use a certainty equivalence approach for this, via the Wonham filter, i.e., we use Wonham’s filter with xˆ (t) substituting x(t). In [30], due to the particular interest in identifying the regime, a certain dual-sensor measurement architecture is used and an approximation to the modal estimator dynamics is derived. This is then used, in the spirit of certainty equivalence, to estimate {x(t)} (this is what is named in [30] of polymorphic estimator -PME). These ideas within the stationary filter developed here are being examined by the authors at the moment.

Acknowledgments The authors would like to thank the editor and the referees for their very helpful suggestions which greatly improved the quality of the paper. Research supported in part by the Brazilian National Research Council-CNPq, under the Grants No. 301067/ 2009-0, 301740/2007-0, USP project MaCLinC, and by FAPERJ under the Grants No. E-26/100-579/2007, E-26/170-008/2008.

References 1. Benes VE. Exact finite dimensional filters for certain diffusions with non-linear drift. Stochastics 1981; 5: 65–92 2. Bjork T. Finite dimensional optimal filters for a class of ito processes with jumping parameters. Stochastics 1980; 4: 167–183

3. Blom HAP. Bar-Shalom Y. The interacting multiple model algorithm for systems with Markovian switching coefficients. IEEE Trans Autom Control 1988; 33: 780–783 4. Boukas EK, Liu ZK. Robust H∞ filtering for polytopic uncertain time-delay systems with Markov jumps. Comput Electr Eng 2002; 28: 171–193 5. Costa OLV. Linear minimum mean square error estimation for discrete-time Markovian jump linear systems. IEEE Trans Autom Control 1994; 39: 1685–1689 6. Costa OLV, Fragoso MD, Marques RP. Discrete-Time Markov Jump Linear Systems. Springer-Verlag, 2004 7. Cox DR, Miller HD. The Theory of Stochastic Processes. J. Wiley, New York, 1965 8. de Souza CE, Trofino A, Barbosa KA. Mode-independent H∞ filters for Markovian jump linear systems. IEEE Trans Autom Control 2006; 51: 1837–1841 9. Doucet A, Andrieu C. Iterative algorithms for state estimation of jump Markov linear systems. IEEE Trans Autom Control 2000; 49: 1216–1227 10. Doucet A, Logothetis A, Krishnamurthy V. Stochastic sampling algorithms for state estimation of jump Markov linear systems. IEEE Trans Autom Control 2000; 45: 188–202 11. Dufour F, Bertrand P. An image based filter for discretetime Markovian jump linear systems. Automatica 1996; 32: 241–247 12. Elliott R, Dufour F, Sworder F. Exact hybrid filters in discrete-time. IEEE Trans Autom Control 1996; 41: 1807– 1810 13. Elliott RJ, Aggoun L, Moore J-B. Hidden Markov Models: Estimation and Control. Springer-Verlag, NY, 1995 14. Jun-E F, James L, Zhan S. Stabilization of Markovian systems via probability rate synthesis and output feedback. IEEE Trans Autom Control 2010; 55: 773–777 15. Fragoso MD, Baczynski J. Lyapunov coupled equations for continuous-time infinite Markov jump linear systems. J Math Anal Appl 2002; 274: 319–335 16. Fragoso MD, Costa OLV, Baczynski J. The minimum linear mean square filter for a class of hybrid systems. In Proceedings of the European Control Conference - ECC2001; pages 319-322, Porto, Portugal, 2001 17. Fragoso MD, Costa OLV. A unified approach for stochastic and mean square stability of continuous-time linear systems with Markovian jumping parameters and additive disturbance. SIAM J Control Optim 2005; 44: 1165–1191 18. Fragoso MD, Costa OLV, Baczynski J, Rocha N. Optimal linear mean square filter for continuous-time jump linear systems. IEEE Trans Autom Control 2005; 50 19. Fragoso MD, Rocha N. Stationary filter for continuous-time Markovian jump linear systems. SIAM J Control Optim 2005; 44

351


20. Fujisaki M, Kallianpur G, Kunita H. Stochastic differential equations for the nonlinear filtering problem. Osaka J Math 1972; 9: 19–40 21. Geromel JC. Optimal linear filtering under parameter uncertainty. IEEE Trans Signal Process 1999; 47: 168–175 22. Kushner HJ. Dynamical equations for nonlinear filtering. J Diff Equ 1967; 3: 179–190 23. Liu H, Ho DWC, Sun F. Design of H∞ filter for Markov jumping linear systems with non-accessible mode information. Automatica 2008; 44: 2655–2660 24. Loparo KA, Roth Z, Eckert SJ. Nonlinear filtering for systems with random strucuture. IEEE Trans Autom Control 1986; 31: 1064–1068 25. Mahmoud MS, Shi P. Robust Kalman filtering for continuous time-lag systems with Markovian jump parameters. IEEE Trans Circuits Syst I, Fundam Theory Appl 2003; 50: 98–105 26. Mariton M. Jump linear systems in automatic control. Marcel Dekker 1990 27. Raouf J, Boukas EK. Stabilization of singular Markovian jump systems with discontinuities and saturating inputs. J Optim Theory Appl 2009; 143: 369–389 28. Van Schuppen J. Stochastic filtering theory: A discussion of concepts methods and results. J Differ Equ 1967; 3: 179– 190 29. Shi P, Karan M, Kaya CY. Robust Kalman filter design for Markovian jump linear systems with norm-bounded unknown nonlinearities. Circuits Syst Signal Process 2005; 24: 135–150 30. Sworder DD, Boyd JE. Estimation Problems in Hybrid Systems. Cambridge University Press, London, 1999 31. Wonham WH. Some applications of stochastic differential equations to optimal nonlinear filtering. SIAM J Control Optim 1965; 3: 347–369 32. Xiong JL, Lam J. Fixed-order robust H∞ filter design for Markovian jump systems with uncertain switching probabilities. IEEE Trans Signal Process 2006; 24: 1421–1430 33. Yau SS-T. Finite dimensional filters with nonlinear drift I: A class of filters including both kalman-bucy filters and benes filters. J Math Systems Estim Control 1994; 2: 181–203 34. Lixian Z, Boukas EK. Stability and stabilization of Markovian jump linear systems with partly unknown transition probabilities. Automatica 2009; 45: 463–468 35. Lixian Z, Boukas EK, Peng S. H∞ model reduction for discrete-time Markov jump linear systems with partially known transition probabilities. Int J Control 2009; 82: 343– 351 36. Lixian Z, Peng S. Stability, L2 -gain and asynchronous H∞ control of discretetime switched systems with average dwell time. IEEE Trans Autom Control 2009; 54: 2193–2200 37. Zhang Q. Nonlinear filtering and control of a switching diffusion with small observation noise. SIAM J Control Optim 1998; 36: 1638–1668

system (1) and present the evolution and convergence of the second moment matrix for system (1), as well as some properties of the associated Lyapunov like equation. Set Z(t) = (Z1 (t), . . . , ZN (t)) ∈ Hn+ , R(t) = (R1 (t), . . . , RN (t)) ∈ Hn+ , Ri (t) = pi (t)Ci Ci and R = (R1 , . . . , RN ) ∈ Hn+ , Ri = pi Ci Ci and the operator L ∈ B(Hn , Hn ) as follows: for V = (V1 , . . . , VN ) ∈ Hn ,

Lj (V) := Aj Vj + Vj Aj +

N

i=1 λij Vi .

(80)

Writing V = diag(Vi ) ∈ B(RNn ) the following identity can be easily established: diag(Li (V)) = AV + V A + V (V ).

(81)

The next result presents a necessary and sufficient condition for system (1) to be MSS. Proposition 6.1: System (1) is MSS if and only if IRe {λ(L)} < 0. Moreover if IRe {λ(L)} < 0 then A is stable. Proof: See Propositions 4.8, 4.9 and 5.5 of [17]. Next we present the evolution and convergence results for the second moment matrix for system (1), and properties of the associated Lyapunov like equation. Proposition 6.2: For t ∈ R+ we have that ˙ = L(Z(t)) + R(t). Z(t)

(82)

If IRe {λ(L)} < 0 then Z(t) → −L−1 (R) ≥ 0 as t → ∞ and −L−1 (R) is the unique solution of the equation in Z ∈ Hn ,

L(Z) + R = 0.

(83)

Moreover IRe {λ(L)} < 0 implies that for any V = (V1 , . . . , VN ) ∈ Hn there exists a unique U = (U1 , . . . , UN ) ∈ Hn such that

L(U) + V = 0

(84)

and if V ≥ 0 (respectively V > 0) then U ≥ 0 (respectively U > 0).

Appendix

Proof: See Propositions 4.10, 5.3, 5.4 and Theorem 5.6 in [17] and Theorem 8 in [15].

In this appendix we present the proof of Theorems 2.1, 2.2 and 4.3, and Proposition 4.2.

The next result presents a differential matricial equation for the evolution of P(t) as defined in (11).

6.1. Proof of Theorems 2.1 and 2.2

Proposition 6.3: For t ∈ R+ , P(t) + P(t)A + V (Z(t)) ˙P(t) = A 0

The proof of these theorems will need several intermediate results. First we recall some mean square stability results for

0 + Ct Ct . 0

(85)

352


Proof: By Ito’s calculus and noting that Bf Hθt x(t) = Bf Hz(t), we have from (8) that: ˙ = B HU(t) + A Z(t) f f Z(t) + U(t) H Bf + Z(t)Af + Bf Gt Gt Bf .

(86)

Similarly by Ito’s calculus, ˙ U(t) = AU(t) + Z(t)H Bf + U(t)Af + Ct Gt Bf .

(87)

By combining (20), (86) and (87) we get (85). We want now to re-write (20) so that the term V (Z(t)) can be decomposed as a sum of matrices. After some straightforward calculations we have that (19) can be re-written, for Z = diag(Zi ), as (recall the definition of in (25)):

V (Z) =

N

⊗ In dg(Z ).

=1

We have the following result:

Proof: For any vector v = v1 N N i=1 j=1

− v

λj vj −

j =

i =

=

i =

λi v2 +

j = λj ,

λi vi2 − 2

i =

i =

λi vi v

i =

=1

=

N =1

1/2

1/2

⊗ In )(

$

t

z(t) = eAf t z(0) +

≤ e

Af t

$

λi v − vi

2

eAf (t−s) Bf Hθs x(s)ds.

1/2 E z(0)2 + Bf Hmax t

×

1/2 eAf (t−s) E x(s)2 ds

0

≥0

i =

It follows from Proposition 6.4 that for Z = diag(Zi ) ≥ 0, we have that (

and thus

0

we get that

showing the desired result.

N

t→∞ t→∞ that is, E x(t)2 −→ 0 and E z(t)2 −→ 0. From [17], IR {λ(L)} < 0. Consider now an initial condition xe (0) = e t→∞ z(0) . Then clearly 0 z(t) = eAf t z(0) and since z(t) → 0 for any initial condition z(0), it follows that Af is a stable matrix. (⇐=) Suppose now that IRe {λ(L)} < 0 and Af is a stable matrix. From [17], IRe {λ(L)} < 0 implies that eLt ≤ ae−bt for some a > 0, b > 0 and t ∈ R+. From (8) with w(t) = 0 and according to [17], E x(t)2 ≤ ce−bt x0 2 for some c > 0. Moreover,

From the triangular inequality, 1/2 1/2 z(0)2 ≤ E eAf t E z(t)2 $ t 1/2 + E eAf (t−s) Bf Hθs x(s)2 ds

vi λi v .

λi v2 + vi2 − 2vi v =

V (Z) =

Proof: (=⇒) If (8) is MSS then, considering w(t) = 0 in (8), we have for any initial condition xe (0), θ0 , that t→∞ z(t)2 −→ 0, E xe (t)2 = E x(t)2 + E

0

λi vi2

i =

But recalling that −λ = v v =

vN , we have that

···

ϕ,i,j vi vj = |λ |v2 +

Proposition 6.5: System (8) is MSS if and only if IRe {λ(L)} < 0 and Af is a stable matrix.

d z(t) = Af z(t)dt + Bf Hθt x(t)dt

Proposition 6.4: ≥ 0.

v v =

We present next a necessary and sufficient condition for MSS of (8) based on the spectrum of the operator L and the matrix Af .

⊗ In )dg(Z )

where Hmax = max{Hi , i = 1, . . . , N}. Since Af is stable, we can find a > 0, b > 0, such that eAf t ≤ a e−b t . Then, for some a¯ > 0, b¯ > 0, 1/2 ¯ E z(t)2 ≤ a¯ e−b t z(0) + e−bt tx0 t→∞ z(t)2 → 0 showing that E xe (t)2 = E x(t)2 +E for any initial condition xe (0), θ0 , so that, from [17], system (8) is MSS. We prove now Theorem 2.1.

1/2

(

1/2

⊗ In )dg(Z )(

⊗ In ) ≥ 0. (88)

∈ Proof of Theorem 2.1: (=⇒) Consider the operator L n+n n+n n+n f f B(H ,H ) as follows: for V = (V1 , . . . , VN ) ∈ H f , N (V)) is given by: L(V) = (L1 (V), . . . , L

1/2

Therefore, recalling that for ∈ S , = ⊗ In and

we have from (85) and (88) that (27) holds. ϒ := 0

j (V) := L Aj V j + V j Aj +

N i=1

λij Vi ,

(89)

353


where Ai =

Ai Bf Hi

t→∞ Pj ≥ 0, with P = ( P1 , . . . , PN ) satisfying and Pj (t) −→

0 . Af

Consider model (8) with w(t) = 0, and for t ∈ R+ , E(zi (t) z(t) ) E(zi (t)zi (t) ) , Pi (t) = z(t) z(t) 1{θt =i} ) E( z(t)zi (t) ) E(

" j ( 0=L P) + pi

(90)

and P(t) = ( P1 (t), . . . , PN (t)) ∈ Hn+nf . From Proposition ˙ 6.2, P(t)), and from [17] system (8) is MSS if P(t) = L( PN ) ∈ Hn+nf , Pi > 0, and only if there exists P = ( P1 , . . . , i = 1, . . . , N, such that j ( L P) < 0,

j ∈ S.

where Qj ∈ B(Rn ), and Zj ∈ B(Rnf ), and define Z = N , Z = diag(Q ), and Z j i j=1 ⎡ ⎤ U1 Z U ⎢ .. ⎥ . P= U = ⎣ . ⎦, Z U UN Notice that P > 0. Indeed since for each j ∈ S , Pj > 0, it follows from Schur’s complement that Zj > Uj Qj−1 Uj , for j ∈ S , so that Z=

N j=1

Zj >

N

Zi (t) Ui (t) t→∞ Zi −→ Note that Pi (t) = Qi (t) Ui Ui (t) Qi (t) = E( z(t) z(t) 1{θt =i} ). Moreover Z(t) =

N

Uj Qj−1 Uj = U Z −1 U.

j=1

From Schur’s complement again it follows that Z U = P > 0. U Z Re-organizing equations (91) and using (88) we obtain that (26) holds. (⇐=) From (26) and (88), it follows that L(Q) < 0 where Q = (Q1 , . . . , QN ) ∈ Hn which means, from [17], that IRe {λ(L)} < 0. From the Lyapunov equation (26), it is easy is stable, and thus in particular Af is stato see that A ble. Since IRe {λ(L)} < 0 and Af is stable it follows from Proposition 6.5 that system (8) is MSS.

Defining P =

Z U

N

Ui Qi

(92) where

Qi = Z.

i=1

⎡

⎤ U1 U ⎢ ⎥ , Z = diag(Zi ), U = ⎣ ... ⎦ it fol Z UN

t→∞

lows that P(t) → P. Furthermore from (92) we have that P satisfies (28), (29). Suppose that V also satisfies (28), (29) and set Z = (Z1 , . . . , ZN ), X = (X1 , . . . , XN ). Then L(X) = 0, L(Z) = 0 and since IRe {λ(L)} < 0 we have from Proposition 6.2 that Zj = Xj , j ∈ S . This yields . From Proposition 6.1, (V − P) + (V − P)A that 0 = A IRe {λ(L)} < 0 implies that A is stable and thus that the is also stable, yielding that V = P. block diagonal matrix A Finally suppose that V is such that (29), (30) are satisfied. Then L(X) ≤ 0 (respectively L(X) < 0) and it follows that L(X − Z) ≤ 0 (respectively L(X − Z) < 0). This implies from Proposition 6.2 that X ≥ Z (respectively X > Z). Using (V − P) + (V − P)A ≤ 0 this fact, we conclude that A (respectively A(V − P) + (V − P)A < 0) and again from it follows that V −P ≥ 0 (respectively V −P > stability of A 0).

6.2. Proof of Proposition 4.2 and Theorem 4.3 Proof of Proposition 4.2: It is easy to see that is a transition rate matrix since −λii = −

M κ=1

ε κ λκii =

M

εκ

κ=1

j =i

λκij =

M

ε κ λκij =

j =i κ=1

λij .

j =i

Suppose by contradiction that is not irreducible. Then, according to [7], section 3.10, by relabelling the states appropriately, and κ can be written as

Next we prove Theorem 2.2. Proof of Theorem 2.2: As previously shown, we have from (·) as (85) and (88) that (27) holds. Consider the operator L in (89), Pi (t) as in (90). From Proposition 6.2, # " Cj Gj Bf Cj Cj ˙P (t) = L j j (P(t)) + pj (t) Bf Gj Cj Bf Gj Gj Bf

t→∞ Qi (t) →

i=1

(91)

Partitionate Pj as follows Qj Uj , j∈S Pj = Uj Zj

# Cj Gj Bf . Bf Gj Gj Bf

Cj Cj Bf Gj Cj

=

11 21

κ M M 11 0 = ε κ κ = εκ 22 κ21 κ=1

κ=1

κ12 . κ22

where 11 is a square matrix. But since ε κ ≥ 0 and each element of the matrix κ12 is non-negative, we must have for κ such that ε κ > 0 that κ12 = 0, in contradiction with the hypothesis that κ is irreducible.

354


Proof of Theorem 4.3: First of all notice that, by Schur’s complement, (52) is equivalent to (I − XY −1 )L − J X − XY −1 X > 0. (93) L(I − Y −1 X) − J W − LY −1 L − Y −1 X)

we have that (93) is equivBy choosing J = L(I alent to X −1 > Y −1 and W > LY −1 L . Consider now the following matrices: −1 X −1 X , T3 = 0 −Y −1 ⎤ ⎡ ⎡ ⎤ dg(X1−1 ) . . . 0 T3 0 0 ⎥ ⎢ .. .. .. T4 = ⎣ ⎦ , T5 = ⎣ 0 T4 0⎦ . . . . 0 0 I −1 0 . . . dg(XN ) Pre and pos multiplying (53) by T5 and T5 and considering R = −(Y A + FH)(I − Y −1 X) yields ⎤ ⎡ 11 12 13 ⎣ 0 ⎦ < 0, (94) 12 22 0 −I 13 where (* below means the transpose of the element on the opposite diagonal) 11 = ⎡

⎤

AX −1 + X −1 A ∗ ⎦, ⎣A(X −1 − Y −1 ) + (X −1 − Y −1 ) A(X −1 − Y −1 ) −1 −1 −1 −1 A − Y FHY + (X − Y )A

1 dg(X1−1 ) . . . N dg(XN−1 ) , 12 = 0 ... 0 C 13 = , −Y −1 F G ⎡ ⎤ dg(X1−1 ) . . . 0 ⎢ ⎥ .. .. .. 22 = − ⎣ ⎦. . . . 0

...

dg(XN−1 )

From Schur’s complement (94) is equivalent to 11 21

21 = 11 − 12 22

13

− −1 22 0

0 I

12 > 0. 13 (95)

After some algebraic manipulations we get that 11 = −(AX −1 + X −1 A +

N

i dg(Xi−1 ) i + CC ),

i=1

21 = −(A(X −1 − Y −1 ) + (X −1 − Y −1 )A − Y −1 FHY −1 ),

(96) (97)

22 = −(A(X −1 − Y −1 ) + (X −1 − Y −1 )A + Y −1 F GG F Y −1 ).

(98)

From (76) and (77) we have that −1 0 = A(Z − Z ) + (Z − Z )A + Z H GG H Z + I. (99) From the assumption that IRe {λ(L)} < 0 and Proposition 6.1 Z . it follows that A is stable and from (99) we have that Z > Z −1 , F = − Z −1 T ( Z ), Thus with the choice X = Z −1 , Y = we have that X −1 > Y −1 and from (98), (99), 22 = −(A(X −1 − Y −1 ) + (X −1 − Y −1 )A + Y −1 F GG F Y −1 ) = I. From (88), (77) and (96), 11 = −(A(X −1 − Y −1 ) + (X −1 − Y −1 )A + Y −1 F GG F Y −1 ) = 2 I. Z H (GG )−1 H Z we Finally noticing that Y −1 FHY −1 = − get from (97), (99), 21 = −(A(X −1 − Y −1 ) + (X −1 − Y −1 )A − Y −1 FHY −1 = −(A(Z − Z ) + (Z − Z )A −1 + Z H GG H Z ) = I. Thus we have that (95) is indeed verified. It remains to Z )H, Bf = T ( Z ), Lf = L and P show that Af = A − T ( is as in (78). By making R = −(Y A + FH)(I − Y −1 X), J = L(I − Y −1 X), U = (X −1 − Y −1 ) we have that V = −Y and from (61), (62), (63) that Af = A + Y −1 FH = A − T ( Z )H, Bf = −Y −1 F = T ( Z ), Lf = L and P is as in (78). Finally it is easy to see that LP L = L Z L, completing the proof.