linear quadratic optimization problems for some discrete-time

ˆ Dedicated to Dr. Constantin VARSAN on the occasion of his 70th birthday

LINEAR QUADRATIC OPTIMIZATION PROBLEMS FOR SOME DISCRETE-TIME STOCHASTIC LINEAR SYSTEMS VASILE DRAGAN and TOADER MOROZAN

We investigate two problems of optimization of quadratic cost functions along the trajectories of a discrete-time linear system affected by Markov jump perturbations and independent random perturbations. Depending upon the class of admissible controls, the corresponding optimal control is obtained either as the minimal solution or as the maximal and stabilizing solution of a system of discrete-time Riccati type equations. AMS 2000 Subject Classification: 93C55, 93E15, 93E20, 93D15. Key words: linear quadratic problem, discrete-time stochastic system, Markov chain, independent random perturbations, discrete-time Riccati equation. 1. INTRODUCTION

The discrete-time linear control systems have been intensively considered in the control literature in both the deterministic and the stochastic framework. This interest is wholly motivated by the wide area of applications including engineering, economics, and biology. The state space approach for the problem of minimization of a quadratic cost functional along the trajectories of a linear controlled system has a long history. Such an optimization problem is usually known as the linear quadratic optimization problem. In the discrete-time stochastic framework, the linear quadratic optimization problem was separately investigated for systems with independent random perturbations and systems with Markov perturbations, respectively. For the case of discrete-time systems with independent random perturbations we refer to [24, 22, 26] while for discrete-time systems with Markov switching we mention [1]–[8], [15]–[21], [23, 25]. In [11] and [12] the problem of the optimization of a quadratic cost functional along the trajectories of a discrete-time linear system subject to Markov and independent random perturbations was MATH. REPORTS 11(61), 4 (2009), 307–319

308

Vasile Dragan and Toader Morozan

2

investigated and was used to solve the problem of the tracking of a given reference signal. In this paper two problems of optimization of quadratic cost functions along the trajectories of a discrete-time linear system affected by Markov jump perturbations and independent random perturbations are investigated. Depending upon the class of admissible controls, the corresponding optimal control is obtained either as the minimal solution or as the maximal and stabilizing solution of a system of discrete-time Riccati type equations.

2. PROBLEM FORMULATION

(1)

Consider the discrete-time controlled system r h i X x(t + 1) = A0 (t, ηt ) + wk (t)Ak (t, ηt ) x(t)+ k=1 r i h X wk (t)Bk (t, ηt ) u(t), + B0 (t, ηt ) + k=1

where x(t) ∈ Rn is the state vector, u(t) ∈ Rm is the vector of the control parameters, ηt , t ∈ Z+ , is a Markov chain on a given probability space {Ω, F, P} with transition matrices Pt = [pt (i, j)], t ≥ 0, and state space the finite set D = {1, 2, . . . , N } and {w(t)}t≥0 , w(t) = (w1 (t), . . . , wr (t))T , is a sequence of independent random vectors. The superscript T stands for the transpose of a matrix or a vector. We introduce the σ-algebras Ft = σ[w(s), 0 ≤ s ≤ t], Gt = σ[ηs , 0 ≤ s ≤ et = Gt ∨ Ft−1 if t ≥ 1 and H e0 = σ[η0 ]. t], Ht = Ft ∨ Gt , H Throughout the paper we assume that Ft is independent of Gt for each t ∈ Z+ , E[w(t)] = 0, E[w(t)wT (t)] = Ir , t ≥ 0. We associate with system (1) the two cost functionals ∞ X   (2) J1 (t0 , x0 , u) = E |C(t, ηt )xu (t, t0 , x0 )|2 + |D(t, ηt )u(t)|2 , t=t0

(3)

J2 (t0 , x0 , u) =

∞ X

h

E xT u (t, t0 , x0 )M (t, ηt )xu (t, t0 , x0 )+

t=t0

+2xT u (t, t0 , x0 )L(t, ηt )u(t)

i + uT (t)R(t, ηt )u(t) ,

where xu (t, t0 , x0 ) is the solution of (1) corresponding to the control u, with x(t0 , t0 , x0 ) = x0 , t0 ∈ Z+ , x0 ∈ Rn , M (t, i) = M T (t, i), R(t, i) = RT (t, i). Throughout the paper we assume that the sequences {Ak (t, i)}t≥0 , {Bk (t, i)}t≥0 ,

3

Linear quadratic optimization problems

309

{C(t, i)}t≥0 , {D(t, i)}t≥0 , {M (t, i)}t≥0 , {R(t, i)}t≥0 , {L(t, i)}t≥0 , i ∈ D, 0 ≤ k ≤ r, are bounded. Also, we assume that DT (t, i)D(t, i)  0. This means that there exists δ > 0 such that DT (t, i)D(t, i) ≥ δIm for all t ∈ Z+ , i ∈ D. Two classes of admissible controls will be considered in the paper, namely, • U1 (t0 , x0 ) is the set of all sequences u = {u(t)}t≥t0 of m-dimensional et -measurable, E|u(t)|2 < ∞ and the random vectors u(t) such that u(t) is H series (2) is convergent. • U2 (t0 , x0 ) is the set of all sequences u = {u(t)}t≥t0 of m-dimensional et -measurable, E|u(t)|2 < ∞, the series random vectors u(t) such that u(t) is H (3) is convergent and lim E|xu (t, t0 , x0 )|2 = 0.

(4)

t→∞

Now, we are in a position to formulate the optimization problems which are solved in this paper: OP1. Given t0 ∈ Z+ and x0 ∈ Rn , find u e ∈ U1 (t0 , x0 ) such that J1 (t0 , x0 , u e) ≤ J1 (t0 , x0 , u) for all u ∈ U1 (t0 , x0 ). e OP2. Given t0 ∈ Z+ and x0 ∈ Rn , find u e ∈ U2 (t0 , x0 ) such that e J2 (t0 , x0 , u e) ≤ J2 (t0 , x0 , u) for all u ∈ U2 (t0 , x0 ). 3. THE SOLUTION OF OP1

Let Sn be the space of n × n symmetric matrices and SnN = Sn ⊕ Sn ⊕ · · · ⊕ Sn , that is a real Hilbert space with the inner product hX, Y i =

N X

Tr[X(i)Y (i)]; X = (X(1), . . . , X(N )), Y = (Y (1), . . . , Y (N )) ∈ SnN .

i=1

Given F (t) = (F (t, 1), . . . , F (t, N )), F (t, i) ∈ Rm×n , t ∈ Z+ , i ∈ D, we define the Lyapunov type operator LF (t) on SnN , as (5) =

(LF (t)X)(i) = N X r X

pt (j, i)[Ak (t, j) + Bk (t, j)F (t, j)]X(j)[Ak (t, j) + Bk (t, j)F (t, j)]T

j=1 k=0

for X ∈ SnN , i ∈ D. Set TF (t, s) = LF (t − 1) · · · LF (s) if t > s ≥ 0 and TF (t, s) = ISnN if t = s, where ISnN is the identity operator on SnN . Definition 1. We say that the system (1) is stochastic stabilizable if there exists a bounded sequence {F (t)}t≥0 such that kTF (t, s)k ≤ βq t−s , t ≥ s ≥ 0, for some β ≥ 1, q ∈ (0, 1).

310

Vasile Dragan and Toader Morozan

4

If the sequence {F (t)}t≥0 has the above property, then we shall say that it is a stabilizing feedback gain for system (1). Remark 1. It follows from Theorem 3.6 in [10] that if F (t) is a stabilizing feedback gain for system (1), then there exist β ≥ 1 and q ∈ (0, 1) such that E[|xF (t, t0 , x0 )|2 ] ≤ βq t−t0 |x0 |2 for all t ≥ t0 ≥ 0, x0 ∈ Rn , where xF (t, t0 , x0 ) is the solution of system (1) corresponding to the control u(t) = F (t, ηt )xF (t), t ≥ t0 . Therefore, if the system (1) is stochastic stabilizable, then the set U2 (t0 , x0 ) is not empty for each t0 ∈ Z+ , x0 ∈ Rn . For X ∈ SnN , t ∈ Z+ , i ∈ D, let us consider the linear operators Π1i (t)X =

r X

AT k (t, i)Ei (t, X)Ak (t, i),

Π2i (t)X =

k=0

Π3i (t)X =

r X

AT k (t, i)Ei (t, X)Bk (t, i),

k=0 r X

BkT (t, i)Ei (t, X)Bk (t, i),

Ei (t, X) =

N X

pt (i, j)X(j).

j=1

k=0

With the above notation we introduce the discrete-time system (6)

X(t, i) = Π1i (t)X(t + 1) + C T (t, i)C(t, i)− −[Π2i (t)X(t + 1)][DT (t, i)D(t, i) + Π3i (t)X(t + 1)]−1 [Π2i (t)X(t + 1)]T

of generalized Riccati equations (DTSGRE). Theorem 1. Assume that system (1) is stochastic stabilizable. Then the optimal control of OP1 is given by u e(t) = Fe(t, ηt )e x(t), where (7)

Fe(t, i) = −[DT (t, i)D(t, i) + Π3i (t)Xmin (t + 1)]−1 [Π2i (t)Xmin (t + 1)]T

with Xmin (t) the minimal bounded solution of (6) and x e(t) = xFe (t), x e(t0 ) = x0 . The optimal value of the cost functional is J1 (t0 , x0 , u e) =

N X

πt0 (i)xT 0 Xmin (t0 , i)x0 ,

πt0 (i) = P{ηt0 = i}.

i=1

Proof. Since system (1) is stochastic stabilizable, by Theorem 6.1 in [14] the DTSGRE (6) has a positive semidefinite bounded solution Xmin (t) which is minimal in the class of positive semidefinite bounded solutions of (6). Also, it is known that Xmin (t, i) = lim Xτ (t, i), where Xτ (t, i), 0 ≤ t ≤ τ , i ∈ D, is τ →∞

the positive semidefinite solution of (6) with final value Xτ (τ, i) = 0.

5

(8)

Linear quadratic optimization problems

311

By Lemma 3.2 in [11], for v(t, x, i) = xT Xmin (t, i)x we have τ X E[(|C(t, ηt )e x(t)|2 + |D(t, ηt )e u(t)|2 )|ηt0 = i] = =

t=t0 T x0 Xmin (t0 , i)x0

− E[e xT (τ + 1)Xmin (τ + 1, ητ +1 )e x(τ + 1)|ηt0 = i]

for all τ ≥ t0 , i ∈ Dt0 where Ds = {i ∈ D | πs (i) > 0} for each s ∈ Z+ . Since Xmin (τ, i) ≥ 0 and Xmin (t) is bounded, from (8) we have u e ∈ U1 (t0 , x0 ) and J1 (t0 , x0 , u e) ≤

(9)

N X

πt0 (i)xT 0 Xmin (t0 , i)x0 .

i=1

Further, by Lemma 3.2 in [11], for v(t, x, i) = xT Xτ (t, i)x we have τ −1 X

E[(|C(t, ηt )xu (t)|2 + |D(t, ηt )u(t)|2 )|ηt0 = i] =

t=t0

= xT 0 Xτ (t0 , i)x0 +

τ −1 X

 E (u(t) − Fτ (t, ηt )xu (t))T (DT (t, ηt )D(t, ηt )+

t=t0

 + Π3ηt (t)Xτ (t + 1))(u(t) −Fτ (t, ηt )xu (t))|ηt0 = i , where Fτ (t, i) is the feedback gain associated with Xτ (t, i) constructed as in (7) with Xτ (t, i) instead of Xmin (t, i). Since Xτ (t, i) ≥ 0, we can write (10)

τ −1 X

E[(|C(t, ηt )xu (t)|2 + |D(t, ηt )u(t)|2 )|ηt0 = i] ≥ xT 0 X(t, i)x0 .

t=t0

Letting τ → ∞ in (10), we obtain X (11) J1 (t0 , x0 , u) ≥ πt0 (i)xT 0 Xmin (t0 , i)x0 . i∈Dt0

Writing (11) for u(t) = u e(t) and taking into account (9) we get J1 (t0 , x0 , u) ≥

N X

πt0 (i)xT e). 0 Xmin (t0 , i)x0 = J1 (t0 , x0 , u

i=1

This shows that u e(t) is the optimal control which completes the proof.



4. THE SOLUTION OF OP2

Since in (3) no assumption concerning the sign of the weighting matrices M (t, i), L(t, i) and R(t, i) was made, it is possible that J2 (t0 , x0 , ·) be unbounded from bellow.

312

Vasile Dragan and Toader Morozan

Let V(t0 , x0 ) =

inf u∈U2 (t0 ,x0 )

6

J2 (t0 , x0 , u), (t0 , x0 ) ∈ Z+ × Rn be the value

function associated to OP2. Definition 2. We say that OP2 is well possed if −∞ < V(t0 , x0 ) < ∞ for all t0 ∈ Z+ and x0 ∈ Rn . To make clearer the statement of the next results we adopt the notation: N Q(t) = (Q(t, 1), . . . , Q(t, N )), Π(t)X = ((Π(t)X)(1), . . . , (Π(t)X)(N )) ∈ Sn+m ,     M (t, i) L(t, i) Π1i (t)X Π2i (t)X Q(t, i) = , (Π(t)X)(i) = . LT (t, i) R(t, i) (Π2i (t)X)T Π3i (t)X With the pair Σ = (Π, Q) we associate the so called dissipation operator N e Σ of `∞ (Z+ , SnN ) DΣ : `∞ (Z+ , SnN ) → `∞ (Z+ , Sn+m ) and the subsets ΓΣ and Γ by Σ (DΣ X)(t) = ((D1Σ X)(t), (D2Σ X)(t), . . . , (DN X)(t)),   Π1i (t)X(t + 1) + M (t, i) − X(t, i) L(t, i) + Π2i (t)X(t + 1) Σ (Di X)(t) = (L(t, i) + Π2i (t)X(t + 1))T R(t, i) + Π3i (t)X(t + 1)

(12)

for arbitrary X = {X(t)}t≥0 ∈ `∞ (Z+ , SnN ), (13)

ΓΣ = {X = {X(t)}t≥0 ∈ `∞ (Z+ , SnN ) | (DΣ X)(t) ≥ 0, R(t) + Π3 (t)X(t + 1)  0, t ≥ 0},

(14)

e Σ = {X = {X(t)}t≥0 ∈ `∞ (Z+ , SnN ) | (DΣ X)(t)  0, t ≥ 0}. Γ

Let us consider the system (15) X(t, i) = Π1i (t)X(t + 1) + M (t, i) − [L(t, i) + Π2i (t)X(t + 1)][R(t, i)+ Π3i (t)X(t + 1)]−1 [L(t, i) + Π2i (t)X(t + 1)]T ,

t ∈ Z+ , i ∈ D

of discrete-time Riccati equations. Definition 3. We say that Xs (t) = (Xs (t, 1), . . . , Xs (t, N )), t ∈ Z+ , is a stabilizing solution of (15) if (16)

F Xs (t, i) = −[R(t, i) + Π3i (t)Xs (t + 1)]−1 [L(t, i) + Π2i (t)Xs (t + 1)]T

is a stabilizing feedback gain for system (1). Theorem 2. Assume that a) the system (1) is stochastic stabilizable; b) the set ΓΣ is not empty. Then OP2 problem is well possed. Moreover, we have (17)

V(t0 , x0 ) =

N X i=1

πt0 (i)xT 0 Xmax (t0 , i)x0

7

Linear quadratic optimization problems

313

for all t0 ∈ Z+ and x0 ∈ Rn , where {Xmax (t)}t≥0 is the maximal bounded solution of DTSGRE (15) which verifies R(t, i) + Π3i (t)Xmax (t + 1)  0,

(18)

t ∈ Z+ , i ∈ D.

Proof. First, we remark that under assumptions a) and b), by Theorem 4.2 in [14] the DTSGRE (15) has a maximal and bounded solution Xmax (t) which satisfies condition (18). Also, it follows from Remark 1 that U2 (t0 , x0 ) is not empty, ∀t0 ∈ Z+ , x0 ∈ Rn . By Lemma 3.2 in [11], for v(t, x, i) = xT Xmax (t, i)x, whatever u ∈ U2 (t0 , x0 ) we have " # T   τ −1 X xu (t) xu (t) (19) E Q(t, ηt ) + E[xT u (τ )Xmax (τ, ητ )xu (τ ) u(t) u(t) t=t0

=

X

πt0 (i)xT 0 Xmax (t0 , i)x0 +

τ −1 X

E[(u(t) − Fe(t, ηt )xu (t))T (R(t, ηt )+

t=t0

i∈Dt0

Π3ηt (t)Xmax (t + 1))(u(t) − Fe(t, ηt )xu (t))]. Since the left hand side of (19) converges for τ → ∞, the right hand side is also convergent. Letting τ → ∞ in (19) and taking into account (4) we obtain X J2 (t0 , x0 , u) = πt0 (i)xT (20) 0 Xmax (t0 , i)x0 + i∈Dt0

+

∞ X

h


T E u(t) − Fe(t, ηt )xu (t) R(t, ηt )+

t=t0



i + Π3ηt (t)Xmax (t + 1) u(t) − Fe(t, ηt )xu (t) for all u ∈ U2 (t0 , x0 ) , (t0 , x0 ) ∈ Z+ × Rn . Further, (20) together with (18) imply X (21) J2 (t0 , x0 , u) ≥ πt0 (i)xT 0 Xmax (t0 , i) x0 i∈Dt0

for all u ∈ U2 (t0 , x0 ) , (t0 , x0 ) ∈ Z+ × Rn . Hence X (22) V (t0 , x0 ) ≥ πt0 (i)xT 0 Xmax (t0 , i) x0 . i∈Dt0

Thus, we deduce that the linear quadratic optimization problem under consideration is well-posed. It remains to show that in (22) we have equality. To this end, we choose a decreasing sequence of positive numbers {εj }j≥0 such

314

Vasile Dragan and Toader Morozan

8

that lim εj = 0. We associate the cost functionals j→∞

(23)

J εj (t0 , x0 , u) = J2 (t0 , x0 , u) + εj

∞ X

  E |xu (t)|2 ,

t=t0

u ∈ Ue2 (t0 , x0 ), where Ue2 (t0 , x0 ) = {u = {u(t)}t≥0 ∈ U2 (t0 , x0 ) | xu (t, t0 , x0 ) is such that the series (23) is convergent}. Let Vj (t0 , x0 ) = inf J εj (t0 , x0 , u). Since Ue2 (t0 , x0 ) ⊂ U2 (t0 , x0 ) u∈Ue2 (t0 ,x0 )

J εj

and J2 (t0 , x0 , u) ≤ (t0 , x0 , u) , u ∈ Ue2 (t0 , x0 ), we deduce that Vj (t0 , x0 ) ≥ V (t0 , x0 ) for all j ≥ 0. Consider the DTSGRE (24) X(t, i) = Π1i (t)X(t + 1) + M (t, i) + εj In − [L(t, i) + Π2i (t)X(t + 1)] × × [R(t, i) + Π3i (t)X(t + 1)]−1 [L(t, i) + Π2i (t)X(t + 1)]T .
It is defined by the pair Σj
= Π(t), Qj (t) , where Π(t) is as before and Qj (t) = Qj (t, 1), . . . , Qj (t, N ) with   M (t, i) + εj In L(t, i) Qj (t, i) = . LT (t, i) R(t, i) e Σj is not empty since Γ e Σj ⊃ ΓΣ . By Theorem 5.4 in [14] For each j ≥ 0, Γ we deduce that for each j ≥ 0, DTSGRE (24) has a bounded and stabilizing
solution Xsj (t) = Xsj (t, 1), . . . , Xsj (t, N ) , t ≥ 0. It follows from Proposition 5.1 in [14] that Xsj (t) coincides with the maximal solution of (24). Further, from Theorem 4.3 in [14] we deduce that Xsj (t, i) ≥ Xsj+1 (t, i) ≥ Xmax (t, i) for all j ≥ 0 and lim Xsj (t, i) = Xmax (t, i) for all t ≥ 0, i ∈ D. As in the first j→∞

part of the proof we deduce that ∞ h X X
T εj T j πt0 (i)x0 Xs (t0 , i)x0 + (25) J2 (t0 , x0 , u) = E u(t)−Fsj (t, ηt )xu (t) t=t0

i∈Dt0



i × R(t, ηt ) + Π3ηt Xsj (t + 1) (u(t) − Fsj (t, ηt )xu (t) j for all u ∈ Ue2 (t0 , x0 ), where Fsj (t, i) = F Xs (t, i) is a stabilizing feedback associated with Xsj (t). Take the control ujs (t) = Fsj (t, ηt )xjs (t), {xjs (t)}t≥t0 , the solution of system (1) with u(t) replaced by ujs (t). Since Xsj (t) is the stabilizing solution of (24), we have ujs = {ujs (t)}t≥t0 ∈ Ue2 (t0 , x0 ). Taking u = ujs in (25) we obtain X
ε j J2 j t0 , x0 , ujs = πt0 (i)xT 0 Xs (t0 , i)x0 .

i∈Dt0

9

Linear quadratic optimization problems

This leads to V (t0 , x0 ) ≤ Vj (t0 , x0 ) ≤

P i∈Dt0

315

j πt0 (i)xT 0 Xs (t0 , i)x0 for all j ≥ 0.

Letting j → ∞ we obtain X (26) V (t0 , x0 ) ≤ πt0 (i)xT 0 Xmax (t0 , i)x0 ,

∀ (t0 , x0 ) ∈ Z+ × Rn .

i∈Dt0

From (26) and (22) we get (17) and the proof is complete.



Definition 4. We say that a control uopt = {uopt (t)}t≥t0 ∈ U2 (t0 , x0 ) is called an optimal control for the linear quadratic optimization problem under consideration if V (t0 , x0 ) = J2 (t0 , x0 , uopt ) ≤ J2 (t0 , x0 , u) for all u ∈ U2 (t0 , x0 ). The following result provides a sufficient condition for the existence of an optimal control for OP 2. Proposition 3. If DTSGRE (15) has a bounded and stabilizing solution {Xs (t)}t≥0 which satisfies (27)

R(t, i) + Π3i (t)Xs (t + 1)  0,

then the linear quadratic optimization problem under consideration has an optimal control given by uopt (t) = Fs (t, ηt )xs (t), where Fs (t, i) is defined in (16) and {xs (t)}t≥t0 is the solution of system (1) for u(t) = Fs (t, ηt )xs (t) and the initial condition xs (t0 ) = x0 . Proof. Since {Xs (t)}t≥0 is the bounded and stabilizing solution of (15), the control uopt = Fs (t, ηt ) xs (t) is admissible. The conclusion follows immediately from (20) for u = uopt and taking into account (27).  Now, we prove a result which provides a necessary and sufficient condition for the existence of an optimal control. Theorem 4. Assume that a) the assumptions of Theorem 2 are fulfilled; P b) N i=1 pt (i, j) > 0 for all t ≥ 0 and j ∈ D; c) π0 (i) = P{η0 = i} > 0 for 1 ≤ i ≤ N . Then the following assertions are equivalent: (i) for any (t0 , x0 ) ∈ Z+ × Rn the optimization problem OP 2 admits an optimal control u bt0 ,x0 (t), t ≥ t0 , that is V (t0 , x0 ) = J2 (t0 , x0 , u bt0 ,x0 ) ; (ii) we have

(28) lim T e (t, t0 ) = 0, ∀t0 ∈ Z+ , t→∞

F

ξ

where TFe (t, t0 ) is the linear evolution operator on SnN defined by the sequence of Lyapunov operators {LFe (t)}t≥0 , LFe being defined by (5) with Fe(t, i) instead of F (t, i) and Fe(t, i) = F Xmax (t, i) and k · kξ is the Minkovski norm (see [13]).

316

Vasile Dragan and Toader Morozan

10

If (i) or (ii) are fulfilled, then the optimal control of the problem under consideration is given by uopt (t) = Fe (t, ηt ) x b(t), where x b(t) is the solution of system (31) below. Proof. Let us assume that (i) is fulfilled. Let (t0 , x0 ) ∈ Z+ × Rn and b). From (20) u b = {b u(t)}t≥t0 ∈ U2 (t0 , x0 ) be such that V (t0 , x0 ) = J2 (t0 , x0 , u we get (29)

V(t0 , x0 ) =

X

πt0 (i)xT 0 Xmax (t0 , i)x0 +

∞ X

E[(b u(t) − Fe(t, ηt )b x(t))T

t=t0

i∈D

×(R(t, ηt ) + Π3ηt (t)Xmax (t + 1))(b u(t) − Fe(t, ηt )b x(t))], where x b = xub (t) is the optimal trajectory. Combining (17) and (29) yields ∞ X

(30)

h
T E u b(t) − Fe (t, ηt ) x b(t) ×

t=t0


i ×(R(t, ηt ) + Π3ηt (t)Xmax (t + 1)) u b(t) − Fe(t, ηt )b x(t) = 0. On account of (27), the last equation leads to u b(t) = Fe (t, ηt ) x b(t) a.s. t ≥ t0 . Substituting this equality in (1), we deduce that x b(t) is the solution of the problem h (31) x b(t + 1) = A0 (t, ηt ) + B0 (t, ηt ) Fe0 (t, ηt ) +

r X


i b(t), wk (t) Ak (t, ηt ) + Bk (t, ηt )Fe(t, ηt ) x

x b(t0 ) = x0 ,

k=1

with given initial value. It follows from assumptions b) and c) that Dt0 = D. Since u b ∈ U2 (t0 , x0 ), from (4) we have   (32) lim E |b x(t)|2 |ηt0 = i = 0, i ∈ D. t→∞

If ΦFe (t, t0 ) is the fundamental matrix solution of (31), then (32) may be rewritten as h i T lim E xT ∀i ∈ D, (t0 , x0 ) ∈ Z+ × Rn . 0 ΦFe (t, t0 )ΦFe (t, t0 )x0 |ηt0 = i = 0, t→∞

By the representation theorem (see [10]), the last equation is equivalent to
∗ (t, t )J (i)x = 0 for all (t , x ) ∈ Z × Rn , i ∈ D, where J = lim xT T 0 0 0 0 + 0 e t→∞

F

(In , . . . , In ) ∈ SnN . Recalling that

∗

 ∗ 

T (t, t0 ) = T ∗ (t, t0 )J = max sup xT 0 TFe (t, t0 )J (i)x0 , ξ ξ Fe Fe i∈D |x0 |≤1

11

Linear quadratic optimization problems

317

we deduce that (33)

lim TF∗e (t, t0 ) ξ = 0.

t→∞

Finally, using the properties of the Minkovski norm (see [13]), we deduce that (33) is equivalent to (28). So, the implication (i) ⇒ (ii) does hold. To prove the converse implication, we remark that by the representation theorem in [10] if (28) holds then (32) holds, too. This means that the control u b(t) = Fe (t, ηt ) x b(t) is admissible. Further, from (20) and (17) we deduce that u b is an optimal control and thus the proof is complete.  Remark 2. From the definition of the stabilizing solution of a system of discrete-time Riccati equations of stochastic control (see [14]) we deduce that the maximal solution Xmax (t) of DTSGRE (15) is a stabilizing solution if and only if there exist β ≥ 1 and q ∈ (0, 1) such that (34)



T e (t, t0 ) ≤ βq t−t0 F ξ

for all t ≥ t0 ≥ 0. From Theorem 4 we deduce that the condition verified by the maximal solution of (15), which is equivalent to the existence of an optimal control of the problem under consideration, is weaker than (34). This can explain why the result proved in Proposition 3 only provides a sufficient condition for the existence of an optimal control. Theorem 5. Assume that a) the coefficients of system (1) and the weights of the cost functional (3) are periodic sequences with period θ ≥ 1; b) the assumptions of Theorem 2 are fulfilled. Under these assumptions the following assertions are equivalent: (i) for any (t0 , x0 ) ∈ Z+ × Rn the optimization problem described by the controlled system (1), the cost functional (3) and the class of admissible controls U2 (t0 , x0 ) has the optimal control u bt0 x0 = {ut0 x0 (t)}t≥t0 , i.e., V (t0 , x0 ) = J2 (t0 , x0 , u bt0 x0 ); (ii) the DTSGRE (15) has a bounded stabilizing solution {Xs (t)}t≥0 which satisfies (27). Proof. The implication (ii) ⇒ (i) follows from Proposition 3. If (i) is fulfilled, reasoning as in the proof of Theorem 4, we deduce, by Theorem 4.1 in [10], that Fe is the stabilizing feedback gain for system (1). This allows us to conclude that the maximal solution {Xmax (t)} coincides with the stabilizing solution of (15). Thus, the proof is complete. 

318

Vasile Dragan and Toader Morozan

12

REFERENCES [1] H. Abou-Kandil, G. Freiling and G. Jank, On the solution of discrete-time Markovian jump linear quadratic control problems. Automatica J. IFAC 32 (1995), 5, 765–768. [2] W.P. Blair and D.D. Sworder, Feedback control of a class of linear discrete systems with jump parameters and quadratic cost criteria. Internat. J. Control 21 (1975), 5, 833–841. [3] E.K. Boukas and K. Benjelloun, Robust control for linear systems with Markovian jumping parameters. Preprints of 13th IFAC World Congress, San Francisco, USA, 1996, 451–456. [4] H.J. Chizeck, A.S. Willsky and D.Castanon, Discrete time Markovian jump linear quadratic optimal control. Internat. J. Control 43 (1986), 1, 213–231. [5] O.L.V. Costa, Linear minimum mean square error estimation for discrete time markovian jump linear systems. IEEE Trans. Automat. Control 39 (1994), 8, 1685–1689. [6] O.L.V. Costa and M.D. Fragoso, Discrete time LQ-optimal control problems for infinite Markov jump parameter systems. IEEE Trans. Automat. Control 40 (1995), 12, 2076– 2088. [7] O.L.V. Costa, Mean-square stabilizing solutions for discrete time coupled algebraic Riccati equations. IEEE Trans. Automat. Control 41 (1996), 4, 593–598. [8] O.L.V. Costa, M.D. Fragoso and R.P. Marques, Discrete-time Markov Jump Linear Systems. Springer-Verlag, London, 2005. [9] J.L. Doob, Stochastic Processes. Wiley, New York, 1967. [10] V. Dragan and T. Morozan, Mean square exponential stability for some stochastic linear discrete time systems. Eur. J. Control 12 (2006), 4, 373–399. [11] V. Dragan and T. Morozan, The linear quadratic optimization problem for a class of discrete-time stochastic linear systems. Internat. J. Innovative Computing, Information and Control 4 (2008), 9, 2127–2137. [12] V. Dragan and T. Morozan, Discrete-time Riccati type equations and the tracking problem. ICIC Express Lett. 2 (2008), 2, 109–116. [13] V. Dragan T. Morozan, Discrete time linear equations defined by positive operators on ordered Hilbert spaces. Rev. Roumaine Math. Pures Appl. 53 (2008), 2-3, 131–166. [14] V. Dragan and T. Morozan, A class of discrete time generalized Riccati equations. Preprint no. 1/2008, “Simion Stoilow” Institute of Mathematics of the Romanian Academy. To appear in J. Difference Equations Appl. [15] M.D. Fragoso, Discrete-time jump LQG problem. Internat. J. Systems Sci. 20 (1989), 12, 2539–2545. [16] K. Furuta and M. Wongsaisuwan, Discrete time LQG dynamic controller design using plant Markov parameters. Automatica J. IFAC 31 (1995), 1317–1324. [17] B.E. Griffiths and K.A. Loparo, Optimal control of jump linear gaussian systems. Internat. J. Control 42 (1985), 4, 791–819. [18] Y. Ji, H.J. Chizeck, X. Feng and K. Loparo, Stability and control of discrete-time jump linear systems. Control Theory Adv. Tech. 7 (1991), 2, 247–270. [19] M. Mariton, Jump Linear Systems in Automatic Control. Marcel Dekker, New York, 1990. [20] T. Morozan, Stability and control of some linear discrete-time systems with jump Markov disturbances. Rev. Roumaine Math. Pures Appl. 26 (1981), 1, 101–119. [21] T. Morozan, Optimal stationary control for dynamic systems with Markovian perturbations. Stoch. Anal. Appl. 1 (1983), 3, 299–325.

13

Linear quadratic optimization problems

319

[22] T. Morozan, Discrete-time Riccati equations connected with quadratic control for linear systems with independent random perturbations. Rev. Roumaine Math. Pures Appl. 37 (1992), 3, 233–246. [23] T. Morozan, Stability and control for linear discrete-time systems with Markov perturbations. Rev. Roumaine Math. Pures Appl. 40 (1995), 5-6, 471–494. [24] T. Morozan, Stabilization of some stochastic discrete-time control systems. Stochastic Anal. Appl. 1 (1983), 1, 89–116. [25] T. Morozan, Dual linear quadratic problem for linear control systems with jump Markov perturbations. Rev. Roumaine Math. Pures Appl. 41 (1996), 5-6, 363–377. [26] J. Zabczyk, Stochastic control of discrete-time systems. In: Control Theory and Topics in Functional Analysis. Vol. III, pp. 187–224. IAEA, Vienna, 1976. Received 28 April 2009

Romanian Academy “Simion Stoilow” Institute of Mathematics P.O. Box 1-764, 014700 Bucharest, Romania [email protected] [email protected]