A RICCATI EQUATION FOR STOCHASTIC H1 D. Hinrichsen
Institut fur Dynamische Systeme Universitat Bremen D-28334 Bremen F.R.G. Fax: +49 421 2184235
[email protected]
Keywords: Riccati equation, LMI, Stochastic systems, Robust stability
Abstract In this note we report on a new kind of algebraic Riccati equation which we encountered when studying an H 1 type problem of disturbance attenuation for stochastic linear systems. The same equation occurs in the analysis of stability radii of linear systems with both deterministic and stochastic uncertainties. The associated linear matrix inequality is also considered.
1 Introduction The objective of this paper is to introduce and discuss a new type of Riccati equation/inequality which we encountered when investigating an H 1 type problem of disturbance attenuation for a rather general class of stochastic linear systems. Consider systems described by Ito stochastic di erential equations of the form
dx(t) = Ax(t)dt + A0 x(t)dw1 (t) + B0 v(t)dw2 (t) +B1v(t)dt + B2 u(t)dt (1) z (t) = C1 x(t) + D11 v(t) + D12 u(t) y(t) = C2 x(t) + D21 v(t); where wi ; i = 1; 2 are independent normalized scalar Wiener processes. Setting all the D's to zero and v = z in (1) where is an unknown matrix we obtain an Ito stochastic di erential equation where both the drift term Ax(t)dt and the di usion term A0 x(t)dw1 (t) are perturbed. Thus (1) is a suciently general class of stochastic systems to model the simulaneous presence of deterministic and stochastic parameter uncertainties. It is this feature, i.e. the simultaneous presence of the terms B1 v(t)dt and B0 v(t)dw2 (t) which will lead us later to a new kind of Riccati equation/inequality.
A. J. Pritchard
Mathematics Institute University of Warwick Coventry CV4 7AL UK Fax: +44 1203 524182
[email protected]
We now give an interpretation of the system equations (1) in the spirit of H 1 control. In this context v is viewed as a disturbance which adversely a ects the to be controlled output z . This e ect is to be ameliorated via control action u based on dynamic feedback from the measured output y. A feedback controller K : y 7! u has to be chosen in such a way that the closed loop system cl is stabilized. The e ect of the disturbances on the to be controlled output z of cl is then described by the perturbation operator Lcl : v 7! z of cl which (for zero initial state) maps nite energy disturbance signals v to the corresponding nite energy output signals z of the closed loop system. The size of this linear operator is measured by the induced norm and the problem is to determine, for any > 0 whether or not there exists a stabilizing controller K achieving kLcl k < . The LMI approach to H 1 control [3], [6] has shown that the mathematical centrepiece of the state space theory of deterministic H 1 control is the Bounded Real Lemma. This lemma characterizes the norm of the perturbation operator of a given system in terms of a parametrized algebraic Riccati Inequality or the corresponding parametrized Linear Matrix Inequality. In this note we will not study the stochastic counterpart of the H 1 control problem but only present a \stochastic" version of the Bounded Real Lemma which provides the basis for a solution of the stochastic disturbance attenuation problem for systems of the form (1), see [5]. The perturbation operator of a given stochastic system will be characterized, equivalently, by the solvability of
a new linear matrix inequality (11), a Rational Riccati Equation (13) plus a simple LMI, a Rational Riccati Inequality (12) plus a simple LMI. These equations and inequalities will be introduced in the next section. Their solvability will be characterized in Section 3. Because of space limitations most proofs will be omitted. Full details can be found in [5].
2 A Rational Riccati Equation
Let Hn (R) denote the set of symmetric matrices in Rnn . It is known (see [1]) that (2) is stable in the above sense Consider the linear stochastic system described by the fol- if and only if, for every (some) positive de nite matrix lowing Ito stochastic di erential equation Q0 2 Hn (R), there exists P 2 Hn (R); P 0 such that dx(t) = Ax(t)dt + A0 x(t)dw1 (t) + PA + A P + A0 PA0 = Q0 : (6) B0 v(t)dw2 (t) + Bv(t)dt (2) z (t) = Cx(t) + Dv(t); It can be shown that every internally stable system (2) is also externally stable in the sense that for zero initial where state it responds to v() 2 L2w (R+ ; R` ) with an output z () 2 L2w (R+ ; Rq ) and (A; A0 ; B0 ; B ) 2 Rnn Rnn Rn` Rn` (C; D) 2 Rqn Rq` : (3) kz ()kL2w (R+;Rq) kv()kL2w (R+;R`) ; v 2 L2w (R+ ; R` ) (7) w1 ; w2 are independent normalized real scalar Wiener processes on a probability space ( ; F ; ) with respect to for some constant 0. The operator an increasing family (Ft )t2R+ of -algebras Ft F . In L : L2w (R+ ; R` ) ! L2w (R+ ; Rq ) particular, for i; j = 1; 2; t; s 2 R+ ; t > s,
E ((wi (t) ? wi (s))(wj (t) ? wj (s))) = ij (t ? s) de ned by where ij is the Kronecker symbol. ` In (2) the input process v(t) is viewed as a stochastic (Lv)(t)= Cx(t; v; 0) + Dv(t); t 0; v 2 Lw (R ; R ) (8) disturbance and the output process z (t) as a vector of describes the e ect of the disturbances v() 2 L (R ; R` ) w the to be controlled variables. The system equation con2
+
2
tains multiplicative state and input dependent noise terms which may be interpreted as white noise parameter perturbations of the matrices A and B : dx(t) = (A + A0 w_ 1 (t))x(t)dt + (B + B0 w_ 2 (t))v(t)dt In this paper we provide all spaces Rk ; k 1 with the usual inner product h; i and the corresponding 2-norm k k. For any 0 < T < 1 we denote by L2w ([0; T ]; Rk ) the space of nonanticipative stochastic processes y() = (y(t))t2[0; T ] with respect to (Ft )t2[0; T ] (see e.g. [2]) satisfying
ky()kL2w 2
Rk) = E
([0;T ];
Z T 0
!
ky(t)k dt < 1 : (4) 2
An anologous notation is used for the in nite time interval R+ = [0; 1). For arbitrary 0 < T < 1, v 2 L2w ([0; T ]; R` ) and x0 2 Rn , there exists a unique solution x() = x(; v; x0 ) 2 L2w ([0; T ]; Rn ) of (2) with x(0) = x0 , i.e. x() is a continuous nonanticipative stochastic process satisfying the Ito integral equation
x(t) = x + 0
Z t
Z t 0
(Ax(s) + Bv(s))ds
(5)
w1 (s) ; t 2 [ 0; T ]; [A0 x(s) B0 v(s)]d w 2 (s) 0 see [2]. Moreover x() has bounded L2 second moments on [ 0; T ]. De nition 2.1 The system (2) is said to be internally (exponentially mean square) stable if there exist constants M 1; ! > 0, such that Ekx(t; 0; x0 )k2 Me?!tkx0 k2; x0 2 Rn ; t 0: +
+
on the output of the system and is called the perturbation operator of (2). Its norm kLk is de ned as the minimal
0 such that (7) is satis ed. kLk may be considered as a measure of the worst e ect of the disturbance on the to be controlled output, and consequently it is of interest to reduce this norm by feedback control. This is the stochastic disturbance attenuation problem and its deterministic counterpart is the object of H 1 -theory. As pointed out in the introduction it is fundamental for the LMI and the Riccati equation approach to H 1 -control that the norm kLk can be characterized by parametrized algebraic Riccati equations. In order to develop a stochastic counterpart of the Bounded Real Lemma we consider { as in the deterministic case { an associated optimal control problem over arbitrary nite time intervals [0; T ]; T > 0. Given > 0 and T > 0 we de ne the cost caused by v 2 L2w ([0; T ]; R` ) at initial state x0 2 Rn by 2
JT (x0 ; v) =
Z T 0
E kv(t)k ? kz (t)k dt 2
2
2
where
z () = z (; v; x0 ) = C (; v; x0 ) + Dv() is the output signal of (2) generated by the initial state
x0 2 Rn and the input v. Note that the cost J1 2 (0; v) over the in nite time interval [0; 1) is nonnegative for all v 2 L2w (R+ ; R` ) 2if and only if kLk . In order to
(0; v ) 0 for all v 2 L2 (R ; R` ) we examine whether J1 w + will analyze (in the next section) the nite time optimal control problem 2
Minimize JT (x0 ; v) for v 2 L2w ([0; T ]; R` )
(9)
where it is required that I + B PB 0. Comparing (13) with (14) and (15) we see that, loosely speaking, the Rational Riccati Equation (13) is a combination of a continuous time and a discrete time algebraic Riccati equation.
where x0 2 Rn ; T > 0 are arbitrary but xed. We introduce the following notation:
L(P ) = PA + A P + A0 PA0 ? C C H 2 (P ) = 2 I` + B0 PB0 ? D D 2 H` (R) K (P ) = PB ? C D 2 Rn` and
M (P ) = KL((PP)) HK (2P(P) ) :
(10)
The following lemma can be shown using Ito's formula [2]. Lemma 2.2 Suppose P () : [ 0; T ] 7! Hn (R) is continuously di erentiable, T > 0 given. Then, for any x0 2 Rn ; v() 2 L2w ([0; T ]; R` ), 2
JT (x0 ; v) = hx0 ; P (0)x0 i ? Ehx(T ); P (T )x(T )i + Z T
x(t) ; M (P (t)) x(t) dt E v(t) v(t) 0 where x() = x(; v; x0 ) and M (P ) is de ned by (10).
hx(t); P_ (t)x(t)i +
Our aim is to show that (2) is internally stable and kLk < if and only if
The following proposition shows that the solvability of the Rational Riccati Inequality together with H 2 (P ) 0 imply that (2) is internally stable and kLk < . Proposition 2.4 Suppose that (11) holds for some pair ( ; P ) 2 (0; 1) Hn (R) with P 0. Then (2) is internally stable and kLk < . Proof: Stability follows since M (P ) 0 implies L(P ) 0 so that the stability criterion (6) is satis ed. To prove kLk < choose " > 0 suciently small and P = P 0 such that M (P ) "2 I . Then, setting P (t) = P and x0 = 0 in Lemma 2.2, we obtain for all v() 2 L2w (R+ ; R` ) and all T > 0,
2 T
Z T
J (0; v) E
"
2
0
Z T 0
x(t) ; M (P (t)) x(t) v(t) v(t)
dt
E kv(t)k dt 2
since P 0. It follows that z () = z (; v; 0) 2 L2w (R+ ; Rq ) M (P ) = KL((PP)) HK (2P(P) ) 0: (11) and Z 1 E kz (t)k2dt kLvk2L2w (R+;Rq) = 0 holds for some P 2 Hn (R) with P 0. By a well-known Z 1 2 2 criterion for the positive de niteness of symmetric 2 2 ( ? " ) E kv(t)k2 dt 0 block matrices this condition is equivalent to H 2 (P ) 0 plus the rational matrix inequality: 2 ` for all v() 2 Lw (R+ ; R ). This concludes the proof. 2 L(P ) ? K (P )H (P )?1 K (P ) 0: (12) In the next section we will obtain the converse of the previous proposition. The associated rational matrix equation is
L(P ) ? K (P )H 2 (P )?1 K (P ) = 0: (13) Note that in the deterministic case (A0 = 0; B0 = 0)
3 Solvability of the Rational Riccati Equation
this equation reduces to the Riccati equation, well known In order to prove that internal stability of (2) and kLk < from H 1 control and also in the theory of stability radii imply solvability of the Rational Riccati Equation for (if D = 0, see [4]): some P 2 Hn (R) with P 0, we consider the nite time control problem (9) for arbitrary but xed x0 2 PA + A P ? C C (14) optimal n ; T > 0. ?(PB ? C D)( 2 I` ? D D)?1 (PB ? C D) = 0 R We rst determine the cost of an arbitrary control which is composed of a feedback control and an open loop conWhereas this equation is polynomial (quadratic) in P , the trol. LHS of (13) or (12) is a rational matrix function of P . We therefore call (12) a Rational Riccati Inequality, (13) Lemma 3.1 Suppose x0 2 Rn , v 2 L2w (R+ ; R` ), F () 2 a Rational Riccati Equation and (11) the associated linear C ([ 0; T ]; R`n ) and matrix inequality. ? xF () = x ; F ()xF () + v() ; x0 Remark 2.3 Recall that the discrete time algebraic Riccati equation corresponding to (14) with D = 0 has the is the solution of form dxF (t) = (A + BF (t))xF (t)dt + A0 xF (t)dw1 (t) + ? 1 P ?A PA?C C +A PB (I + B PB ) B PA = 0 (15) B0 F (t)xF (t)dw2 (t) + B0 v(t)dw2 (t) + Bv(t)dt (16)
Proposition 3.4 Suppose (2) is internally stable and k Lk < . Then (21) has a unique solution PT () on [ 0; T ] 2 2 JT (x ; FxF + v) = hx ; PF (0)x i + (17) for every T > 0. Moreover, the cost functional JT 2 (x ; v) Z T h D is minimized by the feedback control vT (t) = FT (t)xFT (t) 2 Ei E hv; NxF i + hNxF ; vi + v; H 2 (PF )v dt; where
2 ? F (23) T (t) = ?H (PT (t)) K (PT (t)) ;
2 where PF (t) solves and xFT () satis es X_ (t) + [I F (t)]M (X (t)) F I(t) = 0; X (T ) = 0: (18) dxFT (t) = (A + BFT (t))xFT (t)dt + A xFT (t)dw (t) + B FT (t)xFT (t)dw (t); xFT (0) = x : 2 2 2 The optimal cost is and N (t) = K (PF (t)) + H (PF (t))F (t). In particular, if v = 0 then
2 min J (24) ( x ; v ) = x ; P T (0)x : T 2 v2Lw
2
2 JT (x ; FxF ) = hx ; PF (0)x i: (19) It remains to examine what happens as T ! 1. The Proof: Applying Lemma 2.2 with P () = PF 2 () and following lemma is proved in an analogous way as in the F ()xF () + v() instead of v(), a short calculation yields deterministic case, extending the optimal control on some (17). Setting v = 0 in (17) we obtain (19). interval [0; T ] to a larger interval [0; T 0] by zero. The following lemma establishes a lower bound for the Lemma 3.5 Suppose (2) is internally stable and kLk
0 there exists a (unique) with xF (0) = x0 . Then 0
0
0
0
0
1
1
0
2
0
0
0
0
0
0
0
0
0
0
0 2
2
1
1
0
0
0
0
0
0
solution of (21) backwards in time on a maximal interval (t? (T ); T ]. For t # t? (T ) the solution of (21) may cease to exist either by \explosion" or by converging towards the boundary of Df . By Lemma 3.2 it can be shown that the solution can never explode in [0; T ], for all T > 0. To prove that it cannot converge towards the boundary @Df in [0; T ] requires a deeper analysis based on the following Lemma, see [5]. Lemma 3.3 Suppose (2) is internally2 stable, kLk < , F (2) 2 C ([ 0; T ]; R`n ); T > 0 and PF () solves (18) with PF (T ) = 0. Then 2 I ? D D 0 and 2
H 2 (PF (t)) ( 2 ? kLk2 )I` ; t 2 [ 0; T ]:
(22)
On the other hand, by Lemma 3.2, 2
hx ; PT ( )x i = JT ? (x ; vT ? ) ?ckx k ; for all x 2 Rn ; 2 [ 0; T ]. Therefore ?cIn PT (t) 0; t 2 [0; T ] (25) and it follows from Lemma 3.5 that PT (t) converges as
2 T ! 1 for every t 0. Moreover, PT (t) = PFT (t) where FT is de ned by (23). Hence Lemma 3.3 implies that, for all t 2 [ 0; T ], 0
0
0
0 2
0
2
H 2 (PT (t)) = H 2 (PF T (t)) ( 2 ? kLk2 )I` 0: (26) Since PT (t) = PT ?t (0), the limit limT !1 PT (t) = limT !1 PT (0) = P is constant. By (25) and (26) P 0 and H 2 (P ) 0:
Applying this Lemma the following proposition can be proved which shows that (21) has a solution on the whole interval [0; T ], for all T > 0, and that the solution of the optimal control problem (9) can be expressed in terms of Taking limits in the integral equation equivalent to (21) this solution. we see that P is a solution of (13).
Our main result is the following Stochastic Bounded Real Remark 3.8 (i) The previous theorem also provides a Lemma. basis for studying stability radii [4] of stochastic systems although here some dicult problems remain Theorem 3.7 For any set of data (3) and > 0 the open, see [5]. following conditions are equivalent: (ii) The results of this paper can be extended to systems (i) the system (2) is internally stable and kLk < ; (2) where the Wiener processes w1 (t), w2 (t) are not (ii) there exist > 0 and P 2 Hn (R); P 0 satisfying independent, as shown in [5]. This should lay the foundation for deriving analogous L(P ) ? 2 I ? K (P )H 2 (P )?1 K (P ) = 0; results for a more general class of stochastic systems H 2 (P ) 0: where the two terms A0 x(t)dw 1 (t) and B0 v (t)dw2 (t) PN i i in (2) are replaced by i=1 A0 x(t)dw (t) and PN i i i (iii) there exists P 2 Hn (R); P 0 such that the2 Rational i=1 B0 v (t)dw (t), respectively, with (not necesRiccati inequality (12) is satis ed with H (P ) 0; sarily independent) scalar Wiener processes wi ; i = 1; : : : ; N . (iv) there exists P 2 Hn (R); P 0 such that M (P ) 0; (v) there exists P 2 Hn (R); P 0 such that 2 3 PA + A P + A0 PA0 PB C BP
2I + B0 PB0 D 5 R(P )= 4 [1] G. Da Prato and J. Zabczyk. Stochastic Equations in C D I In nite Dimensions. Encyclopedia of Mathematics and is positive de nite. its Applications, Cambridge University Press, 1992. Proof: (i) ) (ii): Suppose (i). Replacing C by C = [2] A. Friedman. Stochastic Di erential Equations and C D Applications. Probability and Mathematical Statistics I and D by D = 0 in (8) we obtain a 28, Academic Press, 1975. perturbation operator L for the modi ed data. Then kL k < for suciently small > 0 and so applying The- [3] P. Gahinet and P. Apkarian. A Linear Matrix Inequality approach to H1 control. Int. J. Robust and orem 3.6 to the modi ed data we nd that there exists Nonlinear Control 4: 421-448, 1994. P 2 Hn (R); P 0 satisfying the conditions in (ii). By stability P 0. [4] D. Hinrichsen and A. J. Pritchard. Real and com(ii) ) (iii) is trivial. plex stability radii: a survey. In D. Hinrichsen and The equivalence (iii) , (iv) follows from the de niteness B. Martensson, editors, Control of Uncertain Systems, criterion for symmetric 2 2 block matrices. volume 6 of Progress in System and Control Theory, The equivalence (iv) , (v) follows from pages 119{162, Basel, 1990. Birkhauser. 2 3 2 3 I 0 ?C I 0 0 M (P ) 0 [5] D. Hinrichsen and A. J. Pritchard. Stochastic H 1 . 4 0 I ?D 5 R(P ) 4 0 I 05 = Technical Report Nr. 366, Inst. f. Dynamische Sys0 I 0 0 I ?C ?D I teme, Universitat Bremen 1996, submitted. Finally (iv) ) (i) is a consequence of Proposition 2.4. [6] T. Iwasaki and R. E. Skelton. All controllers for the general H1 control problem: LMI existence conditions and state space formulas. Automatica 30: 1307-1317, On the basis of the previous theorem an H 1 type dis1994. turbance attenuation problem for the stochastic system (1) can be resolved, see [5]. The stabilizing controllers [7] B. P. Molinari The stabilizing solution of the discrete K achieving kLcl k < are characterized by LMI's which algebraic Riccati equation. Trans. Autom. Control 20: { in the special case that A0 = 0; B0 = 0 { reduce to 396{399, 1975. the LMI's characterizing the ?suboptimal controllers in the general deterministic case, see [3], [6]. If B0 = 0 and [8] T. Morozan. Parametrized Riccati equations assoD = 0 the rational Riccati equation (13) reduces to the ciated to input-output operators for time-varying algebraic Riccati equation: stochastic di erential equations with state-dependent noise. Institutul de Matematica al Academiei Romane, ? 1 PA + A P + A0 PA0 ? C C ? PBB P = 0: Preprint No. 37, Bucarest 1995. In [8] Morozan has used the corresponding di erential Riccati equation to characterize the norm of the perturbation operator of a time-varying stochastic system with B0 = 0 and D = 0.
References
