A CONVERGENT ALGORITHM FOR THE OUTPUT COVARIANCE ...

c 1997 Society for Industrial and Applied Mathematics

SIAM J. CONTROL OPTIM. Vol. 35, No. 1, pp. 341–361, January 1997

019

A CONVERGENT ALGORITHM FOR THE OUTPUT COVARIANCE CONSTRAINT CONTROL PROBLEM∗ G. ZHU† , M. A. ROTEA‡ , AND R. SKELTON§ Abstract. This paper considers the optimal control problem of minimizing control effort subject to multiple performance constraints on output covariance matrices Yi of the form Yi ≤ Y i , where Y i is given. The contributions of this paper are a set of conditions that characterize global optimality, and an iterative algorithm for finding a solution to the optimality conditions. This iterative algorithm is completely described up to a user-specified parameter. We show that, under suitable assumptions on problem data, the iterative algorithm converges to a solution of the optimality conditions, provided that this parameter is properly chosen. Both discrete- and continuous-time problems are considered. Key words. covariance control, convergent algorithm AMS subject classifications. 49K15, 93C05, 93C55, 93C60 PII. S0363012994263974

1. Introduction. Consider the following linear time-invariant system: x˙ p (t) = Ap xp (t) + Bp u(t) + Dp wp (t), (1.1)

yp (t) = Cp xp (t), z(t) = Mp xp (t) + v(t),

where xp is the state, u the control, wp represents process noise, and v is the measurement noise. The vector yp contains all variables whose dynamic responses are of interest. The vector z is a vector of noisy measurements. Suppose that we apply to the plant (1.1) a full state feedback stabilizing control law of the form (1.2)

u(t) = Gxp (t)

or a strictly proper output feedback stabilizing control law given by (1.3)

x˙ c (t) = Ac xc (t) + F z(t), u(t) = Gxc (t).

Then the resulting closed-loop system is

(1.4)

x(t) ˙ = Ax(t) + Dw(t),     yp (t) Cy y(t) = = x(t) = Cx(t), Cu u(t)

∗ Received

by the editors March 2, 1994; accepted for publication (in revised form) November 27, 1995. This research was supported in part by National Science Foundation grants ECS-91-08493 and ECS-93-58288 and NASA grant NAG8-220. http://www.siam.org/journals/sicon/35-1/26397.html † Space Systems Control Laboratory, Purdue University, West Lafayette, IN 47907. Present address: Cummins Engine Company, Inc., 1900 McKinley Ave, MC 50197, Columbus, IN 47201 ([email protected]). ‡ School of Aeronautics and Astronautics, 1282 Grissom Hall, Purdue University, West Lafayette, IN 47907 ([email protected]). § Space Systems Control Laboratory, 1293 Potter Engineering Center, Purdue University, West Lafayette, IN 47907 ([email protected]). 341

342

G. ZHU, M. ROTEA, AND R. SKELTON

where for the state feedback case we have x = xp and w = wp , while for the output feedback case we have x = [xTp xTc ]T and w = [wpT v T ]T . Moreover, formulas for A, C, and D are easy to obtain from (1.1) and (1.2) or (1.3). Consider the closed-loop system (1.4). Let Wp and V denote positive definite symmetric matrices with dimensions equal to the process noise wp and measurement vector z, respectively. Define W = Wp , if the state feedback controller (1.2) is used in (1.4) or W = block diag [Wp , V ] if (1.3) is used in (1.4). Let X denote the closedloop controllability Gramian from the (weighted) disturbance input W −1/2 w. Since A is stable, X satisfies (1.5)

0 = AX + XAT + DW DT .

T T Partition the performance output yp in (1.4) into yp := [y1T , y2T , . . . , ym ] , where mi yi = Ci x ∈ R for i = 1, 2, . . . , m. In this paper we are interested in finding controllers of the form (1.2) or (1.3) that minimize the (weighted) control energy trace RCu XCuT with R > 0, and satisfy the constraints

(1.6)

Yi = Ci XCiT ≤ Y i ,

i = 1, 2, . . . , m,

where Y i > 0 (i = 1, 2, . . . , m) are given and X solves (1.5). This problem, which we call the the output covariance constraint (OCC) problem, is defined as follows. The OCC Problem. Find a static state feedback or full-order dynamic output feedback controller for system (1.1) to minimize the OCC cost (1.7)

JOCC = trace RCu XCuT ,

R > 0,

subject to (1.5) and (1.6). The OCC problem may be given several interesting interpretations. For instance, assume first that wp and v are uncorrelated zero-mean white noises with intensity matrices Wp > 0 and V > 0. That is, let E be an expectation operator, and (1.8)

E[wp (t)] = 0,

E[wp (t)wpT (t − τ )] = Wp δ(τ ),

E[v(t)] = 0,

E[v(t)v T (t − τ )] = V δ(τ ).

Letting E∞ [·] := limt→∞ E[·] and W = Wp for the case of state feedback or W = block diag [Wp , V ] for the output feedback case, it is easy to see that the OCC is the problem of minimizing E∞ uT Ru subject to the OCCs Yi := E∞ yi (t)yiT (t) ≤ Y i . As is well known, these constraints may be interpreted as constraints on the variance of the performance variables or lower bounds on the residence time (in a given ball around the origin of the output space) of the performance variables [10]. The OCC problem may also be interpreted from a deterministic point of view. To see this, define the L∞ and L2 norms 2

(1.9)

k yi k∞ := supt≥0 yiT (t)yi (t), R∞ 2 k wk2 := 0 wT (t)w(t)dt,

and define the (weighted) L2 disturbance set (1.10)

W := {w : R → Rnw and k W −1/2 wk2 ≤ 1}, 2

where W > 0 is a real symmetric matrix. Then, for any w ∈ W, we have [17, 18] (1.11)

2

k yi k∞ ≤ σ[Yi ],

i = 1, 2, . . . , m,

OUTPUT COVARIANCE CONTROL

343

and (1.12)

kui k2∞ ≤ [Cu XCuT ]ii ,

i = 1, 2, . . . , nu ,

where nu is the dimension of u. (Here, σ[·] denotes the maximum singular value and [ ]ii is the ith diagonal entry.) Moreover, [17, 18] show that the bounds in (1.11) and (1.12) are the least upper bounds that hold for an arbitrary signal w ∈ W. Thus, if we define Y i := Imi 2i in (1.6) and R := diag [r1 , r2 , . . . , rnu ] in (1.7), the OCC problem is the problem of minimizing the (weighted) sum of worst-case peak values on the control signals given by   nu X 2 ri sup kui k∞ (1.13) JOCC = i=1

w∈W

subject to constraints on the worst-case peak values of the performance variables of the form (1.14)

sup kyi k2∞ ≤ 2i ,

i = 1, 2, . . . , m.

w∈W

This interpretation is important in applications where hard constraints on responses or actuator signals cannot be ignored, such as space telescope pointing and machine tool control. Control problems related to the OCC problem defined here have been considered before by several authors. See, for example, [6, 9, 5, 1, 3, 15, 16] for work in multiobjective optimal control with quadratic cost functionals, [13, 14, 4, 19] for the so-called variance constraint control problems, and [12] for the so-called generalized H2 control problem. In the above references, one may find two different approaches for solving this class of optimal control problems: the approach based on solving the optimality conditions corresponding to the optimization problem at hand [4, 16, 19] and the approach based on reducing the given problem to a finite dimensional convex optimization problem [1, 3, 12]. In this paper, we follow the approach initiated in [4, 19]. Here, we consider a more general and realistic problem, i.e., the OCC problem, than the one studied in [4, 16, 19], and provide an iterative algorithm for solving the optimality conditions corresponding to this problem. Our main contribution is in the algorithm itself. This iterative algorithm is completely described up to a user-specified parameter. We show that the algorithm converges to a solution of the optimality conditions (assuming that one exists), provided that the user-specified parameter is properly chosen. Both discrete- and continuous-time problems are considered. The paper is organized as follows. Section 2 provides optimality conditions for the continuous-time OCC problem in the case of state feedback. These conditions comprise one algebraic Riccati equation and one Lyapunov equation. The Riccati equation has a forcing term depending on a matrix Q (which represents the Kuhn– Tucker multipliers) that must be determined. An algorithm for finding this matrix Q is given, and its convergence analyzed. Section 2 concludes with the extension of the state feedback results to the output feedback case. Section 3 is the discrete-time version of section 2. An example is presented in section 4 to illustrate the performance of the algorithm. Section 5 gives the conclusions of this work. The notation used in this paper is fairly standard. Given the continuous-time algebraic Riccati equation 0 = ATp K + KAp − KBp R−1 BpT K + CpT QCp ,

344

G. ZHU, M. ROTEA, AND R. SKELTON

we say that K is the stabilizing solution if K = K T satisfies the Riccati equation and Ap − Bp R−1 BpT K has all eigenvalues in the open left half plane. Similarly, given the discrete-time algebraic Riccati equation K = ATp KAp − ATp KBp (R + B T KBpT )−1 BpT KAp + CpT QCp , we say that K is the stabilizing solution if K = K T satisfies the Riccati equation and Ap − Bp (R + Bp KBpT )−1 BpT KAp has all eigenvalues in the open unit disk. Note that when the continuous (or discrete) stabilizing solution exists it is unique. Moreover, if Q = QT ≥ 0, the stabilizing solution is positive semidefinite. 2. The OCC algorithm for continuous systems. 2.1. The OCC algorithm for state feedback. In this section we consider the case of state feedback. With the state feedback controller (1.2) the closed-loop system matrices in (1.4) are given by (2.1)

A = Ap + Bp G,

D = Dp ,

C y = Cp ,

Cu = G.

The following theorem provides conditions for optimality in the state feedback case. THEOREM 2.1. Suppose there exists a matrix Q∗ = block diag [Q∗1 , Q∗2 , . . . , Q∗m ] ≥ 0, (2.2)

Q∗i = Q∗i T ∈ Rmi ×mi ,

i = 1, 2, . . . , m,

such that the algebraic Riccati equation 0 = ATp K + KAp − KBp R−1 BpT K + CpT Q∗ Cp

(2.3)

has the (unique) stabilizing solution K ∗ . Define G∗ = −R−1 BpT K ∗ ,

(2.4)

and let X ∗ denote the unique solution of the Lyapunov equation 0 = (Ap + Bp G∗ )X + X(Ap + Bp G∗ )T + Dp Wp DpT ,

(2.5)

and define Yi = Ci X ∗ CiT (i = 1, 2, . . . , m). Then if 0 = (Yi − Y i )Q∗i and Yi ≤ Y i

(2.6)

for all i = 1, 2, . . . , m, we have that G∗ given by (2.4) is an optimal solution to the OCC problem defined in (1.7). Proof. Let Q∗ be given by (2.1) and define the following LQ problem: (2.7)

min J(G, X) = trace RGXGT + (G,X)

m X

trace (Ci XCiT − Y i )Q∗i

i=1

subject to Ap + Bp G stable and (2.8)

0 = (Ap + Bp G)X + X(Ap + Bp G)T + Dp Wp DpT .

Using a simple completion of square argument, it is easy to see from (2.3), (2.4), and (2.5) that (G∗ , X ∗ ) solves (2.7).

OUTPUT COVARIANCE CONTROL

345

Now let G denote a feasible controller (arbitrary but fixed) for the OCC problem. That is, (Ap + Bp G) is stable and Ci XCiT ≤ Y i (for all i = 1, 2, . . . , m), where X is the closed-loop Gramian corresponding to G. From the previous paragraph, we get that ∗

∗

∗

∗

∗ T

JOCC (G , X ) = trace RG X (G ) +

m X

trace (Ci X ∗ CiT − Y i )Q∗i

i=1

(2.9)

≤ trace RGXGT +

m X

trace (Ci XCiT − Y i )Q∗i

i=1

≤ trace RGXGT . Using the fact that 0 = (Ci X ∗ CiT − Y i )Q∗i , from (2.9) we obtain (2.10)

trace RG∗ X ∗ (G∗ )T ≤ trace RGXGT .

This last inequality, together with the fact that G∗ is also feasible for the OCC problem because Ci X ∗ CiT ≤ Y i (for all i = 1, 2, . . . , m), implies that G∗ is a solution to the OCC problem. From (2.3) and (2.4), it follows that the solution of the OCC problem with static state feedback is an LQ controller with a special choice of output-weighting matrix Q. Therefore, our algorithm for solving the conditions in Theorem 2.1 needs only to iterate on Q. Before giving the algorithm we would like to mention that the existence of Q∗ satisfying the conditions of Theorem 2.1 is necessary in certain cases. For example, from Theorem 5.8 of [5], it follows that, when the constraints in (1.6) are scalar and (for example) the pairs (C1 , Ap ), . . . , (Cm , Ap ) do not have imaginary axis unobservable modes, then a diagonal Q∗ exists if a solution to the OCC problem exists. See also [3]. The case of block diagonal matrices Q does not seem to appear in the published literature. It should be noted that the emphasis of the present paper is an algorithm for computing Q∗ (and thus a controller that solves the OCC problem) under the assumption that a matrix Q∗ satisfying the conditions of Theorem 2.1 exists. This algorithm is given next. To give this algorithm we need to introduce the following operator. Let M denote a real symmetric matrix, and suppose that (2.11)

M = [U1 U2 ] block diag [Ep , En ] [U1 U2 ]T

is the (real) Schur decomposition of M , where Ep and En are diagonal matrices containing the strictly positive and nonpositive eigenvalues of M in decreasing order, respectively, and [U1 U2 ] is an orthogonal matrix. Define  (2.12)

P[M ] =

0 U1 Ep U1T

if M ≤ 0, otherwise.

Note that if M is a symmetric matrix with block diagonal structure, the operator P[·] preserves the block structure; i.e., P[M ] has the block structure of M . The following algorithm for solving the conditions in Theorem 2.1 is the main contribution of this paper.

346

G. ZHU, M. ROTEA, AND R. SKELTON

THE OCC ALGORITHM. b (1) Given Ap , Bp , Dp , Cp , Wp , R, Y = block diag [Y 1 , Y 2 , . . . , Y m ], an initial point Q(0) = block diag [Q1 (0), Q2 (0), . . . , Qm (0)] > 0, and constants α > 0, 0 < β < 1, let j = 0 and go to 2). (2) Compute K(j) ≥ 0 and G(j) by solving (2.13)

0 = ATp K(j) + K(j)Ap − K(j)Bp R−1 BpT K(j) + CpT Q(j)Cp , G(j) = −R−1 BpT K(j).

(3) Compute X(j) by solving (2.14)

0 = [Ap + Bp G(j)]X(j) + X(j)[Ap + Bp G(j)]T + Dp Wp DpT .

(4) Set Yi (j) = Ci X(j)CiT for i = 1, 2, . . . , m. (5) Let Y b (j) = block diag [Y1 (Q(j)), Y2 (Q(j)), . . . , Ym (Q(j))], and (2.15)

b

Q(j + 1) = βQ(j) + (1 − β)P[Q(j) + α{Y b (j) − Y }] ,

j = j + 1, go to 2). Several stopping criteria may be used to guarantee that the OCC algorithm terminates in a finite number of steps. In this paper, we propose stopping the algorithm whenever the first equation in condition (2.6) is satisfied to a given numerical accuracy. This can be achieved by checking if the inequality (2.16)

m X

||(Yi (j) − Y i )Qi (j)|| < 

i=1

holds, where  > 0 is the specified tolerance. Inequality (2.16) must be tested after step 4). If (2.16) holds, we stop the algorithm and declare G(j), Q1 (j), Q2 (j), and Qm (j) to be a numerical solution to the OCC problem; if (2.16) does not hold, the algorithm continues. The rest of this section is devoted to showing that, under the assumption (A1)

(Ap , Bp ) is stabilizable and Ap has no eigenvalues on the imaginary axis,

if there exists Q∗ satisfying the conditions in Theorem 2.1, then the OCC algorithm will find it, provided that α is properly chosen. More specifically, under the assumptions mentioned, we will show that the sequence of matrices {Q(j)}∞ j=0 generated by ˆ which satisfies all the conditions of the OCC algorithm (see (2.15)) has a limit Q Theorem 2.1. Thus, the OCC algorithm converges to a globally optimal solution to ˆ implies that, given any the OCC problem. Note that the existence of the limit Q  > 0, there exists an integer j such that inequality (2.16) holds. Note that the OCC algorithm is well posed in the sense that the unique positive semidefinite solution K(j) to the Riccati equation (2.13) and the solution X(j) to the Lyapunov equation (2.14) exist at each iteration. This follows from assumption (A1) and the fact that, at each iteration, Q(j) ≥ 0. As is well known [7], since (Ap , Bp ) is stabilizable and the pair (Q(j)1/2 Cp , Ap ) has no imaginary axis unobservable modes, K(j) ≥ 0 exists, is unique, and renders Ap − Bp R−1 BpT K(j) asymptotically stable. To establish our results we need to introduce the following operators: (2.17)

Tβ [Q] := βQ + (1 − β)T [Q], b

T [Q] := P[Q + α{Y b (Q) − Y }],

347

OUTPUT COVARIANCE CONTROL

where α > 0 and β ∈ (0, 1) are parameters of the OCC algorithm. Note that, with this notation, the sequence of matrices Q(j) generated by the OCC algorithm is {Tβj [Q(0)]}∞ j=0 , where Q(0) ≥ 0 is block diagonal. THEOREM 2.2. Consider the OCC algorithm, and suppose that α > 0, 0 < β < 1, and assumption (A1) holds. Suppose that the algorithm converges; that is, the ∗ ∗ ∗ ∗ ∗ sequence {Tβj [Q(0)]}∞ j=0 converges to Q . Then Q := block diag [Q1 , Q2 , . . . , Qm ] satisfies the sufficient conditions in Theorem 2.1 for optimality. In other words, if the OCC algorithm converges, the resulting controller u=−R−1 BpT K ∗ x, where K ∗ solves (2.3), is a global optimal solution to the given OCC problem, where Q∗ is the limit of the convergent sequence {Tβj [Q(0)]}∞ j=0 . The proof of Theorem 2.2 requires the following lemma. LEMMA 2.3. For any symmetric matrices M = M T and N = N T of the same dimensions, the following statements hold: 1. P[M + N ] = M if and only if M ≥ 0, N ≤ 0, and M N = 0. 2. kP[M ] − P[N ]k ≤ kM − N k, where k · k denotes the Frobenius norm. Proof. First, we shall show the necessity part of 1. The property M ≥ 0 is a direct consequence of the definition of P[·] in (2.12). Next, we show that N ≤ 0 and M N = 0. Let M + N have the Schur decomposition (2.18)

M + N = [U1 U2 ] block diag [Ep , En ][U1 U2 ]T = U1 Ep U1T + U2 En U2T ,

where Ep > 0 and En ≤ 0. Thus, from (2.12) and P[M + N ] = M we obtain (2.19)

P[M + N ] = U1 Ep U1T = M.

Subtracting this last equation from (2.18) we obtain (2.20)

N = U2 En U2T ≤ 0.

Since U1T U2 = 0, from (2.19) and (2.20) it follows that M N = U1 Ep U1T U2 En U2T = 0. Second, we shall show the sufficiency part of 1. Let M ≥ 0 and N ≤ 0 be given and suppose that M N = 0. Note that if either M or N is zero, the sufficiency of property 1) is trivial. Now suppose that M 6= 0 and N 6= 0. The real Schur decompositions of M and N are M = U1 Ep U1T ,

N = U2 En U2T ,

where Ep > 0, En < 0, U1T U1 = I, U2T U2 = I, and U1T U2 = 0. Then (2.21)

P[M + N ] = P[U1 Ep U1T + U2 En U2T ] = U1 Ep U1T = M.

Finally, we show property 2. Let M and N have the following Schur decompositions: (2.22)

M = [U1 U2 ] block diag [Ep , En ][U1 U2 ]T , ˆ1 U ˆp , E ˆn ][U ˆ1 U ˆ2 ] block diag [E ˆ 2 ]T , N = [U

Ep > 0, ˆp > 0, E

En ≤ 0, ˆn ≤ 0. E

348

G. ZHU, M. ROTEA, AND R. SKELTON

Let M + := P[M ] = U1 Ep U1T > 0, M − := U2 En U2T ≤ 0, ˆ1 E ˆp U ˆ1T > 0, N + := P[N ] = U ˆ2 E ˆn U ˆ2T ≤ 0. N − := U Note that M = M + + M − , N = N + + N − , M + M − = 0, and N + N − = 0. Then

(2.23)

kM − N k2 = k(M + − N + ) + (M − − N − )k2 = kM + − N + k2 + kM − − N − k2 −2traceM − N + − 2traceM + N − .

Since −traceM − N + ≥ 0 and −traceM + N − ≥ 0, we obtain (2.24)

kM − N k2 ≥ kM + − N + k2 = kP[M ] − P[N ]k2 ,

which completes the proof. The following lemma, essentially due to [2], is also required for the proof of Theorem 2.2. LEMMA 2.4. Consider the plant defined in (1.1) and suppose that assumption (A1) holds. Let K denote the unique stabilizing solution to the Riccati equation (2.3) with Q = QT ≥ 0. Then K(Q) is a real analytic function of Q = QT ≥ 0. Proof of Theorem 2.2. By Lemma 2.4, the state feedback control gain G(Q) in (2.4) is a continuous function of Q = QT ≥ 0. Hence, the block output covariance matrix Y b (Q) is continuous with respect to Q = QT ≥ 0. Since the operator P[·] is continuous, we obtain that T [·] and Tβ [·] are well defined and continuous for any ∗ Q = QT ≥ 0. Suppose that {Tβj [Q(0)]}∞ j=0 converges to Q , i.e., (2.25)

lim Tβj [Q(0)] = Q∗ .

j→∞

Since β ∈ (0, 1) and P[·] preserves the block structure, we may conclude that, for each j, Tβj [Q(0)] has the correct block diagonal structure and is positive semidefinite. Thus, Q∗ = block diag [Q∗1 , Q∗2 , . . . , Q∗m ] and Q∗ ≥ 0. From the continuity properties of Tβ , we obtain   Tβ [Q∗ ] = Tβ lim Tβj [Q(0)] = lim Tβj+1 [Q(0)] = Q∗ . (2.26) j→∞

j→∞

That is, Q∗ is a fixed point of Tβ [·]. Since β 6= 1, from (2.17), we get (2.27)

b

Q∗ = T [Q∗ ] = P[Q∗ + α{Y b (Q∗ ) − Y }] . b

Let M = Q∗ and N = α[Y b (Q∗ ) − Y ]. From Lemma 2.3, we conclude that b

b

α[Y b (Q∗ ) − Y ] ≤ 0 and αQ∗ [Y b (Q∗ ) − Y ] = 0 . Since α > 0, the above inequalities imply that Q∗ satisfies (2.6). Hence, Q∗ satisfies the conditions in Theorem 2.1. This completes the proof.

349

OUTPUT COVARIANCE CONTROL

The following result shows that there is always a choice for the parameter α in the OCC algorithm that will guarantee its convergence, provided that the conditions in Theorem 2.1 admit one solution. THEOREM 2.5. Suppose that assumption (A1) holds. Assume also that there exists Q∗ satisfying the conditions in Theorem 2.1. Then, given any Q(0) ≥ 0 ∈ Rny ×ny with the appropriate block diagonal structure, there exists an α∗ > 0 such that if ˆ 0 < α ≤ α∗ , the sequence {Tβn [Q(0)]}∞ n=0 will converge to some Q ≥ 0 satisfying the conditions in Theorem 2.1. That is, the OCC algorithm will converge to a global optimal solution of the given OCC problem. In order to prove Theorem 2.5, we need a few intermediate results and definitions. Let Q = QT ≥ 0 be given and K denote the (unique) stabilizing solution to 0 = ATp K + KAp − KBp R−1 BpT K + CpT QCp .

(2.28)

Then, with the state feedback gain G = −R−1 BpT K, the lth output covariance of the closed-loop system Yl (l = 1, 2, · · · , m) is given by Yl = Cl XClT , where X is the unique solution to 0 = (Ap + Bp G)X + X(Ap + Bp G)T + Dp Wp DpT .

(2.29) Now, let (2.30)

Q = block diag[Q1 , Q2 , . . . , Qm ],

l Ql := [qij ] ∈ Rml ×ml ,

and Y b = block diag[Y1 , Y2 , . . . , Ym ] .

(2.31)

Below, we compute the derivative of Y b with respect to the weighting matrix Q given in (2.30). We do this using vector notation. Let Q be given by (2.30) and define the operator svec by  1  q  q2    (2.32) svec[Q] =  .  ∈ Rn ,  ..  qm where (2.33)

i

q :=

√

 2

i i qm q22 qi i i i i √11 , q12 √ √i mi , . . . , q1m , , . . . , q , . . . , , q 23 2m i i 2 2 2

T .

Note also that the operator svec defined in (2.32) preserves the Frobenius norm; i.e., if Q is given by (2.30), we have (2.34)

||Q|| = ||svec[Q]|| .

Moreover, svec[·] is a linear operator.

350

G. ZHU, M. ROTEA, AND R. SKELTON

Let y = svec[Y b ] ,

(2.35)

where Y b is given by (2.31). Define also the symmetric matrix Ei = svec−1 [ei ] ,

(2.36)

where ei ∈ Rn has a one in the ith row and zeros elsewhere, and svec−1 is the inverse of the operator svec. LEMMA 2.6. Consider the system defined in (1.1), and suppose that assumption (A1) holds. Let Q = QT ≥ 0 be given by (2.30), and define q = svec[Q]. Let y be given by (2.35). Then the partial derivative of y ∈ Rn with respect to q ∈ Rn is (2.37)

∂y = −[Hij ] = −[2trace(Pi Bp R−1 BpT Pj X)], ∂q

i, j = 1, 2, . . . , n,

where Pi is the unique solution to 0 = Pi (Ap + Bp G) + (Ap + Bp G)T Pi + CpT Ei Cp

(2.38)

with Ei given by (2.36). Moreover, if Q = QT ≥ 0, the matrix-valued function H(Q) = [Hij ] is continuous and it satisfies H(Q) ≥ 0. Proof. Let yi denote the ith component of y. From the definition of the operator svec (see, for example, (2.32)) it follows that yi = trace(Ei Cp XCpT ).

(2.39)

Using the Lyapunov equations (2.29) and (2.38) it follows from (2.39) that yi = trace(Pi Dp Wp DpT ),

(2.40)

where Pi is the solution to (2.38). Hence, from (2.40), we get ∂yi = trace(Pij Dp Wp DpT ), ∂qj

(2.41)

i where qj is the jth component of q and Pij = ∂P ∂qj . Now to generate Pij , differentiate equation (2.38) with respect to qj to obtain

0 = Pij (Ap + Bp G) + (Ap + Bp G)T Pij − Pi Bp R−1 BpT

(2.42)

∂K ∂K − Bp R−1 BpT Pi . ∂qj ∂qj

From the Riccati equation (2.28) and the Lyapunov equation (2.38) we get Pj =

∂K , ∂qj

where Pj solves (2.38) with the “E-matrix” equal to Ej . Hence, from (2.42), we obtain Z ∞ exp[(Ap + Bp G)T t][Pi Bp R−1 BpT Pj Pij = − 0

(2.43)

+ Pj Bp R−1 BpT Pi ] exp[(Ap + Bp G)t]dt.

OUTPUT COVARIANCE CONTROL

351

Finally, from (2.29), (2.41), and (2.43) we obtain (2.44)

∂y = −[Hij ] = −[2trace(Pi Bp R−1 BpT Pj X)] , ∂q

which gives (2.37). The continuity of H(Q) follows from the fact that, on the set of positive semidefinite matrices Q, the matrix-valued functions Pi , Pj , and X are all continuous. Note also that Hij = trace (Pi Bp R−1 BpT Pj X) = hX 1/2 Pi Bp R−1/2 , X 1/2 Pj Bp R−1/2 i, where hM, N i = traceM N T is the standard inner product on the space of matrices Rnx ×nu , where nx and nu are dimensions of the plant states and controls. Thus, H is of the form (2.45)

H = [hX 1/2 Pi Bp R−1/2 , X 1/2 Pj Bp R−1/2 i],

which shows that H ≥ 0. The following results may be found in [11]; see Propositions 3.2.3 and 12.3.7. LEMMA 2.7. Assume that F : Rn → Rn is Fr´echet differentiable on a convex set 0 D 0 ⊂ Rn . Then for any x and y ∈ D0 0 , (2.46)

kF(y) − F (x)k ≤ sup σ{F 0 [x + t(y − x)]}kx − yk , 0≤t≤1

where F 0 (·) denotes the Fr´echet derivative of F(·) and σ[·] denotes the maximum singular value of [·]. LEMMA 2.8. Suppose that T : Rny ×ny → Rny ×ny is nonexpansive on the closed, convex set D0 . That is, for any x, y ∈ D0 , we have (2.47)

kT (y) − T (x)k ≤ ky − xk .

Assume, further, that T D0 ⊂ D0 and that D0 contains a fixed point of T . Then for any β ∈ (0, 1) and x0 ∈ D0 the iteration (2.48)

xk+1 = βxk + (1 − β)T (xk ),

k = 0, 1, . . . ,

converges to a fixed point of T in D0 . Proof of Theorem 2.5. The proof of Theorem 2.5 consists of two steps. First, we show the nonexpansive property of operator T defined in (2.17). By assumption, there exists Q∗ satisfying the conditions in Theorem 2.1. Define a subset of Rny ×ny as follows:

(2.49)

D0 := {Q ≥ 0 ∈ Rny ×ny : Q = block diag[Q1 , Q2 , . . . , Qm ] and kQ − Q∗ k ≤ kQ(0) − Q∗ k},

where ny is the dimension of yp , and Q(0) is the initial output-weighting matrix for the OCC algorithm. It is obvious that the set D0 is compact (i.e., closed and bounded) and convex. Let D0 0 be a set defined by (2.50)

D0 0 := {q = svec[Q] ∈ Rn : Q ∈ D0 } .

352

G. ZHU, M. ROTEA, AND R. SKELTON

It is clear that D0 0 is convex, because svec[·] is a linear operator and D0 is convex. Let q ∈ D00 and define y(q) = svec[Y b (Q)], and F [q] = q + αy(q).

(2.51)

Note that F[·] is well defined and Fr´echet differentiable, with respect to q, in D0 0 . In fact, from Lemma 2.6, it follows that the Frechet derivative of F [·] is F 0 [q] = I − αH(q),

(2.52)

where H(q) is defined in (2.37). (Here, we think of H as a function of q = svec[Q] instead of a function Q.) Now, let Qν and Qµ in D0 be given. Define q ν = svec[Qν ] and q µ = svec[Qµ ]. Then, since svec[·] preserves the Frobenius norm, we have b

b

kT [Qν ] − T [Qµ ]k = kP[Qν − α{Y b (Qν ) − Y }] − P[Qµ − α{Y b (Qµ ) − Y }]k ≤ kQν − Qµ − α[Y b (Qν ) − Y b (Qµ )]k

(2.53)

= kq ν − q µ − α[y ν − y µ ]k, where y ν := svec[Y b (Qν )] and y µ := svec[Y b (Qµ )]. Since q ν and q µ belong to D0 0 , using Lemma 2.7, we have kq ν − q µ − α[y ν − y µ ]k = kF[q ν ] − F [q µ ]k ≤ sup σ[F 0 [tq ν + (1 − t)q µ ]]kq ν − q µ k 0≤t≤1

= sup σ[I − αH{[tq ν + (1 − t)q µ ]}]kq ν − q µ k.

(2.54)

0≤t≤1

Since H is a continuous function over the compact set D0 0 , there exists an α∗ > 0 such that for any q ν ∈ D0 0 , q µ ∈ D0 0 , and 0 ≤ t ≤ 1, we have σ{H[tq ν + (1 − t)q µ ]} ≤ 2/α∗ .

(2.55)

Thus, since for any q ν and q µ ∈ D0 0 and any t ∈ [0, 1], H[tq ν + (1 − t)q µ ] ≥ 0, we have sup σ{I − αH[tq ν + (1 − t)q µ ]} ≤ 1

(2.56)

0≤t≤1

for any α ≤ α∗ . Therefore, using (2.53) and (2.54), for any α ≤ α∗ we obtain (2.57)

kT [Qν ] − T [Qµ ]k ≤ kq ν − q µ k = kQν − Qµ k.

Hence, for any α ≤ α∗ , the operator T is nonexpansive on D0 . Replacing Qµ by Q∗ proves that for any Q ∈ D0 (2.58)

kT [Q] − T [Q∗ ]k ≤ kQ − Q∗ k ≤ kQ(0) − Q∗ k.

Now, using Lemma 2.3 and the fact that Q∗ satisfies the conditions of Theorem 2.1, we conclude that T [Q∗ ] = Q∗ . This equation and (2.58) imply (2.59) therefore, T [Q] ∈ D0 .

kT [Q] − Q∗ k ≤ kQ(0) − Q∗ k;

OUTPUT COVARIANCE CONTROL

353

Second, we shall show the convergence of the OCC algorithm, that is, the convergence of the sequence {Tβj [Q(0)]}∞ j=0 . Since T D0 ⊂ D0 , D0 is convex and contains a fixed point of T , from Lemma 2.8 we obtain that the sequence {Tβj [Q(0)]}∞ j=0 generated by the iteration (2.60)

Q(j + 1) = Tβ [Q(j)] = βQ(j) + (1 − β)T [Q(j)]

ˆ The fact that Q ˆ satisfies the sufficient converges to a fixed point of T in D0 —say, Q. conditions in Theorem 2.1 is the direct consequence of Theorem 2.2. 2.2. The OCC algorithm for full-order dynamic feedback. The extension of the state feedback case to the full-order dynamic feedback case is straightforward. In fact, the state feedback OCC algorithm can be applied to solve the full-order dynamic feedback OCC problem. Here, for system (1.1), we assume that assumption (A1) holds and that (A2)

(Mp , Ap ) is detectable.

˜ that As is well known [7], under assumption (A2), there exists a unique matrix X satisfies the Riccati equation (2.61)

˜ T V −1 Mp X ˜ + XA ˜ Tp − XM ˜ + Dp Wp DpT 0 = Ap X p

˜ T V −1 is asymptotically stable. Moreover, X ˜ ≥ 0. With this matrix X, ˜ and Ap − XM p we define (2.62)

˜ T V −1 . F = XM p

˜ and F denote the THEOREM 2.9. Consider the plant defined in (1.1). Let X matrices in (2.61) and (2.62). Suppose that there exists a matrix Q∗ = block diag[Q∗1 , Q∗2 , . . . , Q∗m ] ≥ 0, (2.63)

Q∗i = Q∗i T ∈ Rmi ×mi ,

such that the algebraic Riccati equation (2.64)

0 = ATp K + KAp − KBp R−1 BpT K + CpT Q∗ Cp

has the (unique) stabilizing solution K ∗ . Define (2.65)

G = −R−1 BpT K ∗ ,

and let X ∗ denote the unique solution to the Lyapunov equation (2.66)

0 = (Ap + Bp G)X + X(Ap + Bp G)T + F V F T ,

˜ + X ∗ )C T (i = 1, 2, . . . , m). Then if and define Yi = Ci (X i (2.67)

0 = (Yi − Y i )Q∗i and Yi ≤ Y i

for all i = 1, 2, . . . , m, the dynamic controller (2.68)

x˙ c (t) = (Ap + Bp G − F Mp )xc (t) + F z(t), u(t) = Gxc (t)

is an optimal solution to the OCC problem defined in (1.7).

i = 1, 2, . . . , m,

354

G. ZHU, M. ROTEA, AND R. SKELTON

A proof of this theorem may be obtained by combining Theorem 2.1 in this paper and Lemma 4.2 and Theorem 4.1 in [12]. The result in [12] shows how to reduce the OCC problem (and other H2 -like problems) with output feedback to an equivalent problem with state feedback. ˜ and F in (2.61) and (2.62) do not depend on the Note that the matrices X ∗ weighting matrix Q . To find a matrix Q∗ satisfying the conditions in Theorem 2.9, we can use the OCC algorithm given in section 2. This requires that we replace Dp , ˜ T , respectively. Wp , and Y i in the OCC algorithm F , V , and Y i − Ci XC i 3. Discrete-time version. The discrete-time version of the OCC problem is very much like the continuous-time case. Here, we give the definition of the OCC problem and the main results. Consider the following discrete system: (3.1)

xp (k + 1) = Ap xp (k) + Bp u(k) + Dp wp (k), yp (k) = Cp xp (k), z(k) = Mp xp (k) + v(k).

Suppose that we apply to the plant (3.1) a full state feedback stabilizing control, i.e., (3.2)

u(k) = Gx(k),

or a strictly proper stabilizing control xc (k + 1) = Ac xc (k) + F z(k),

(3.3)

u(k) = Gxc (k).

Then the closed-loop system has the following form: (3.4)

x(k + 1) = Ax(k) + Dw(k),     Cy yp (k) = x(k) = Cx(k), y(k) = Cu u(k)

where the definitions of matrices A, B, and C and vectors x, w, and y are as in the continuous-time case. As in section 1, let Wp > 0 and V > 0 denote symmetric matrices with dimensions equal to the dimensions of wp and z, respectively. Define W = Wp if (3.2) is used in (3.4) or W = block diag[Wp , V ] if (3.3) is used. Let X denote the closed-loop controllability Gramian from the input W −1/2 w. Since A is stable, X is given by X = AXAT + DW DT .

(3.5)

As in the continuous-time case, we seek a solution to the following optimal control problem. The Discrete-Time OCC Problem. Find a state feedback stabilizing controller (3.2) or a strictly proper output feedback stabilizing controller (3.3) for the system (3.1) to minimize the OCC cost (3.6)

JOCC = traceRCu XCuT ,

R > 0,

subject to (3.7)

Yi = Ci XCiT ≤ Y i ,

where X is given by (3.5).

i = 1, 2, . . . , m,

355

OUTPUT COVARIANCE CONTROL

The discrete-time OCC problem has interpretations similar to the ones of the continuous-time case. For example, the discrete-time OCC problem may be interpreted as the problem of minimizing a weighted sum of the worst-case peak values of the control signals ui subject to constraints on the worst-case peak values of the response yi , when the disturbance w is unknown but has bounded energy. This is because, as in the continuous-time case, discrete-time gains from `2 to `∞ may also be computed using controllability Gramians [18]. 3.1. State feedback case. In this section we consider the case of state feedback. The following theorem provides conditions for global optimality. Its proof is similar to that of Theorem 2.1 and is omitted. THEOREM 3.1. Suppose there exists a matrix Q∗ = block diag[Q∗1 , Q∗2 , . . . , Q∗m ] ≥ 0, (3.8)

Q∗i = Q∗i T ∈ Rmi ×mi ,

i = 1, 2, . . . , m,

such that the algebraic Riccati equation (3.9)

K = ATp KAp − ATp KBp (R + BpT KBp )−1 BpT KAp + CpT Q∗ Cp

has the (unique) stabilizing solution K ∗ . Define (3.10)

G∗ = −(R + BpT K ∗ Bp )−1 BpT K ∗ Ap ,

let X ∗ denote the unique solution of the Lyapunov equation (3.11)

X = (Ap + Bp G∗ )X(Ap + Bp G∗ )T + Dp Wp DpT ,

and define Yi = Ci X ∗ CiT (i = 1, 2, . . . , m). Then, if (3.12)

0 = (Yi − Y i )Q∗i and Yi ≤ Y i

for all i = 1, 2, . . . , m, we have that G∗ given by (3.10) is an optimal solution to the OCC problem defined in (3.6). The following algorithm may be used to find a matrix Q∗ and consequently a matrix G∗ for the OCC problem satisfying the conditions in Theorem 3.1. THE DISCRETE-TIME OCC ALGORITHM. (1) Given Ap , Bp , Dp , Cp , Wp , R, Y i = block diag[Y 1 , Y 2 , . . . , Y m ], an initial point Q(0) = block diag[Q1 (0), Q2 (0), . . . , Qm (0)] > 0, and constants α > 0, 0 < β < 1, let j = 0 and go to 2). (2) Compute K(j) ≥ 0 and G(j) by solving K(j) = ATp K(j)Ap + CpT Q(j)Cp (3.13)

−ATp K(j)Bp [R + BpT K(j)Bp ]−1 BpT K(j)Ap , G(j) = −[R + BpT K(j)Bp ]−1 BpT K(j)Ap .

(3) Compute X(j) by solving (3.14)

X(j) = [Ap + Bp G(j)]X(j)[Ap + Bp G(j)]T + Dp Wp DpT .

(4) Set Yi (j) = Ci X(j)CiT for i = 1, 2, . . . , m.

356

G. ZHU, M. ROTEA, AND R. SKELTON

(5) Let Y b (j) = block diag[Y1 (Q(j)), Y2 (Q(j)), . . . , Ym (Q(j))], and (3.15)

b

Q(j + 1) = βQ(j) + (1 − β)P[Q(j) + α{Y b (j) − Y }],

j = j + 1, go to 2). In (3.15), the operator P[·] is as defined in (2.12). The same stop criterion as the continuous-time case is proposed for the discretetime OCC algorithm. That is, (2.16) needs to be tested after step 4) for a given tolerance  > 0. If (2.16) holds, G(j), Q1 (j), Q2 (j), and Qm (j) are numerical solutions for the given OCC problem; otherwise, the algorithm continues. In the rest of this section, we will assume the following: (A3)

(Ap , Bp ) is stabilizable and Ap has no eigenvalues on the unit circle.

Note that the discrete-time OCC algorithm is well posed in the sense that the unique positive semidefinite solution K(j) to the Riccati equation (3.13) and the solution X(j) to the Lyapunov equation (3.14) exist at each iteration. This follows from assumption (A3) and the fact that, at each iteration, Q(j) ≥ 0. As is well known [8], since (Ap , Bp ) is stabilizable and the pair (Q(j)1/2 Cp , Ap ) has no unobservable modes on the unit circle, K(j) ≥ 0 exists, is unique, and renders Ap − Bp (R + BpT K(j)Bp )−1 BpT K(j)Ap asymptotically stable. A close examination of the proofs of the continuous-time results given in Theorems 2.2 and 2.5 reveals that the convergence property of the continuous-time algorithm follows from (i) the properties of the operator P[·] given in Lemma 2.3, (ii) the properties of the stabilizing solution to the continuous-time Riccati equation given in Lemma 2.4, (iii) the formula for the derivatives of the output covariance matrices Y1 (Q), Y2 (Q), . . ., Ym (Q), with respect to Q, given in Lemma 2.6. Certainly, property (i) above holds in the discrete-time case because the operator P[·] is the same. Also, it is relatively easy to show that, under assumption (A3), property (ii) extends to the discrete-time setting. Finally, property (iii) above also holds in the discrete-time case, provided that the Lyapunov equation (2.38) is replaced by its discrete-time counterpart and the matrices R and X in (2.37) are replaced by R+BpT KBp and X−Dp Wp DpT , respectively. Thus, we may now conclude the following result. THEOREM 3.2. Suppose that the assumption (A3) holds. Assume also that there exists Q∗ satisfying the conditions in Theorem 3.1. Then, given any Q(0) ≥ 0 ∈ Rny ×ny with the appropriate block diagonal structure, there exists an α∗ > 0 such that ˆ if 0 < α ≤ α∗ , the sequence {Tβj [Q(0)]}∞ j=0 will converge to some Q ≥ 0 satisfying the conditions in Theorem 3.1. That is, the discrete-time OCC algorithm will converge to a global optimal solution of the given OCC problem. 3.2. Full-order dynamic feedback. As in the continuous-time case, the discretetime state feedback results can be readily extended to solve the discrete-time OCC problem with output feedback. Consider the system (3.1) and suppose that (A4)

(Mp , Ap ) is detectable.

˜ that satisfies the Riccati equation Then there exists a unique matrix X (3.16)

˜ Tp − Ap XM ˜ pT (V + Mp XM ˜ pT )−1 Mp XA ˜ Tp + Dp Wp DpT ˜ = Ap XA X

357

OUTPUT COVARIANCE CONTROL

˜ T (V + Mp XM ˜ T )−1 Mp is asymptotically stable; see, for example, [8]. and Ap − Ap XM p p ˜ ≥ 0. With this matrix X, ˜ we define Moreover, X (3.17)

˜ pT (V + Mp XM ˜ pT )−1 . F = Ap XM

The next result gives a solution to the OCC problem with strictly proper output feedback controllers. The proof follows the continuous-time case and is omitted. ˜ and F denote the THEOREM 3.3. Consider the plant defined in (3.1). Let X matrices in (3.16) and (3.17). Suppose that there exists a matrix Q∗ = block diag[Q∗1 , Q∗2 , . . . , Q∗m ] ≥ 0, (3.18)

Q∗i = Q∗i T ∈ Rmi ×mi ,

i = 1, 2, . . . , m,

such that the algebraic Riccati equation (3.19)

K = ATp KAp − ATp KBp (R + BpT KBp )−1 BpT KAp + CpT Q∗ Cp

has the (unique) stabilizing solution K ∗ . Define (3.20)

G = −(R + BpT K ∗ Bp )−1 BpT K ∗ Ap ,

let X ∗ denote the unique solution to the Lyapunov equation (3.21)

˜ pT )F T , X = (Ap + Bp G)X(Ap + Bp G)T + F (V + Mp XM

˜ + X ∗ )C T (i = 1, 2, . . . , m). Then, if and define Yi = Ci (X i (3.22)

0 = (Yi − Y i )Q∗i and Yi ≤ Y i

for all i = 1, 2, . . . , m, the dynamic controller (3.23)

xc (k + 1) = (Ap + Bp G − F Mp )xc (k) + F z(k), u(k) = Gxc (k)

is an optimal solution to the OCC problem defined in (3.6). ˜ and F are Note that, as in the continuous-time case, the computation of X independent of the selection of the output-weighting matrix Q. Hence, we can apply the discrete-time OCC algorithm with state feedback to solve the discrete-time fullorder output feedback OCC problem under the assumption that the optimal solutions are strictly proper. This requires that in the algorithm given in section 3.1, we replace ˜ T , and Y i − Ci XC ˜ T , respectively. Dp , Wp , and Y i with F , V + Mp XM p i 4. An example. We consider the continuous-time OCC problem defined in (1.7) for the plant (1.1) with the following system matrices:       0 1 0 0 0 1  , Bp =  0  , Dp =  0  , (4.1a) Ap =  −1 −0.1 0 0 −10 1 1   (4.1b) Mp = 1 1 0 ,   1 0.5 0 (4.1c) Cp =  0 0 0.5  . 1 1 0

358

G. ZHU, M. ROTEA, AND R. SKELTON

60

Iteration Number

50

40

30

20

10 1

2

3

4 alpha

5

6

7

FIG. 4.1. Iteration number versus α.

Both the process noise wp and the measurement noise v are scalar variables, while the performance variable yp has three components. The weighting matrices required to define the OCC problem (1.7) are taken to be (4.2)

Wp = 1,

V = 0.01,

and R = 1.

Below, we consider two different OCC problems. These two problems differ in the grouping of the performance variables yi used to define the constraints (1.6). For each problem, we consider both state feedback and dynamic output feedback. 4.1. Problem 1. Here, the OCC problem has the performance constraints (4.3)

Y1 ≤ 0.035,

Y2 ≤ 0.050,

Y3 ≤ 0.050,

where Y1 , Y2 , and Y3 denote the output covariance (1×1) matrices introduced in (1.6), corresponding to the first, second, and third performance variables, respectively. Note that this OCC problem can be also solved by the ellipsoid algorithms given in [1, 3, 12] or the quadratically convergent algorithms given in [16]. First, we consider the case of state feedback. We use the algorithm described in section 2.1 with the following parameters: (4.4)

Q(0) = I3 ,

β = 0.1,

 = 10−6 .

To assess the effect of the user-specified parameter α, we ran the the algorithm with 1.0 ≤ α ≤ 7.25. Figure 4.1 shows the number of iterations required to meet the stopping criteria of the algorithm versus α. Clearly, as α approaches 1 or 7.25, the iteration number increases. From Figure 4.1, it follows that there exists an optimal α which uses the least number of iterations. Finding such an optimal α in terms of the system and specification matrices remains an open problem.

359

OUTPUT COVARIANCE CONTROL TABLE 4.1 Solution to Problem 1 with α = 4.5.

Iteration number 9

Iteration number 22

State feedback design Constraints Optimal cost Spec. (Y i ) Actual (Yi ) JOCC 0.0350 0.0314 0.0500 0.0123 0.0234 0.0500 0.0500 Output feedback design Constraints Optimal cost Spec. (Y i ) Actual (Yi ) JOCC 0.0350 0.0314 0.0500 0.0126 0.0340 0.0500 0.0500

Qi 0.0000 0.0000 1.4268

GT 0.0237 −0.9522 −0.0948

Qi 0.0000 0.0000 2.3765

GT 0.0193 −1.3839 −0.1374

Table 4.1 shows the results of running the algorithm with α = 4.5. Both state and output feedback cases are computed. In the state feedback case, G denotes the state feedback gain. In the output feedback case, G denotes the controller output matrix; see (2.65) and (2.68). The controller input matrix F is precomputed according to (2.62). In this case we have (4.5)

F = [ 0.4412 0.7633 0.4796 ]T .

From Table 4.1, we can see that both controllers are feasible, since Yi satisfies the bound Yi ≤ Y i . The only active constraint is the third one, i.e., Y3 = Y 3 ; hence, the corresponding output weight Q3 is nonzero. As expected, the optimal cost JOCC with output feedback is bigger than that with state feedback. 4.2. Problem 2. Now, the OCC problem has the performance constraints (4.6)

Y1 ≤ 0.035,

Y2 ≤ 0.050 × I2 ,

where Y1 denotes the (1 × 1) output covariance matrix corresponding to the first performance output and Y2 denotes the (2×2) output covariance matrix of the second and third performance outputs grouped together. Table 4.2 shows the results of running the algorithm with α = 30 for both state and output feedback cases. The other parameters required by the algorithm are those in (4.4). For the output feedback case the input gain matrix F of the controller given in (4.5). From Table 4.2, we can see that both controllers are feasible. As expected, the optimal cost JOCC with output feedback is bigger than that with state feedback. Also, note that the constraint on the second output group Y2 ≤ 0.05 × I2 is sufficient for the output covariance constraints of Problem 1 in (4.3), that is, Y2 ≤ 0.05 and Y3 ≤ 0.05. As expected, the costs of Problem 2 for both state and output feedback cases are bigger than those of Problem 1. 5. Conclusion. In this paper we have considered the so-called output covariance constraint (OCC) control problem. This is the problem of minimizing control effort subject to matrix inequality constraints on several closed-loop covariance matrices. Optimality conditions for characterizing a global solution are provided. In the state feedback case, these conditions comprise one algebraic Riccati equation, one Lyapunov equation, and a coupling condition. The unknown in this system of equations is a

360

G. ZHU, M. ROTEA, AND R. SKELTON TABLE 4.2 Solution to Problem 2 with α = 30.

Iteration number 31

Iteration number 65

State feedback design Constraints Optimal cost Spec. (Y i ) Actual (Yi ) JOCC 0.0300 0.0313   0.0123 0.0014 0.050×I2 0.0235 0.0014 0.0499

Qi 0.0000   0.0019 0.0527 0.0527 1.4277

GT 0.0212 −0.9542 −0.0950

Output feedback design Constraints Optimal cost Spec. (Y i ) Actual (Yi ) JOCC 0.0350 0.0314   0.0126 0.0014 0.050×I2 0.0341 0.0014 0.0499

Qi 0.0000   0.0035 0.0919 0.0919 2.3809

GT 0.0149 −1.3878 −0.1379

matrix Q which may be interpreted as a matrix of Kuhn–Tucker multipliers. We have given an iterative algorithm to find such a matrix Q. Under the assumption that the optimality conditions admit a solution Q, we have shown that the iterative algorithm converges to one such solution, provided that the step size parameter α is properly chosen. Using the separation property of a closed-loop covariance matrix given in [12], we have shown how to modify the state feedback algorithm to solve the OCC problem with output feedback. Both discrete- and continuous-time problems have been solved. Finally, an example is presented to demonstrate the applicability of our results. REFERENCES [1] S. P. BOYD AND C. H. BARRATT, Linear Controller Design: Limits of Performance, PrenticeHall, Englewood Cliffs, NJ, 1991. [2] D. F. DELCHAMPS, A note on the analyticity of the Riccati metric, in Algebraic and Geometric Methods in Linear Systems Theory, Lectures in Applied Mathematics, Vol. 18, AMS, Providence, RI, 1980. [3] A. M. EUDARIC, Ellipsoid Method for Multiobjective Control with Quadratic Performance Measures, Master thesis, Purdue University, 1992. [4] C. HSIEH, R. E. SKELTON, AND F. M. DAMRA, Minimum energy controllers with inequality constraints on output variances, Optimal Control Appl. Methods, 10 (1989), pp. 347–366. [5] P. P. KHARGONEKAR AND M. A. ROTEA, Multiple objective optimal control of linear systems: The quadratic norm case, IEEE Trans. Automat. Control, 36 (1991) pp. 14–24. [6] N. T. KOUSSOULAS AND C. T. LEONDES, The multiple linear quadratic Gaussian problem, Internat. J. Control, 43 (1986), pp. 337–349. [7] V. KUCERA, A contribution to matrix quadratic equations, IEEE Trans. Automat. Control, 17 (1972), pp. 344–347. [8] V. KUCERA, The discrete Riccati equation of optimal control, Kybernetika, 8 (1972), pp. 430–447. [9] P. M. MAKILA, On multiple criteria stationary linear quadratic control, IEEE Trans. Automat. Control, 34 (1989), pp. 1311–1313. [10] S. MEERKOV AND T. RUNOLFSSON, Output residence time control, IEEE Trans. Automat. Control, 34 (1989), pp. 1171–1176. [11] J. M. ORTEGA AND W. C. RHEINBOLDT, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970. [12] M. A. ROTEA, The generalized H2 control problem, Automatica J. IFAC, 29 (1993), pp. 373– 385. [13] R. E. SKELTON AND M. DELORENZO, Space structure control design by variance assignment, J. Guidance Control Dynamics, 8 (1985), pp. 454–462.

OUTPUT COVARIANCE CONTROL

361

[14] H. T. TOIVONEN, Variance constrained self-tuning control, Automatica J. IFAC, 19 (1983), pp. 415–418. [15] H. T. TOIVONEN, A multiobjective linear quadratic Gaussian control problem, IEEE Trans. Automat. Control, 29 (1984), pp. 279–280. [16] H. T. TOIVONEN AND P. M. MAKILA, Computer-aided design procedure for multiobjective LQG control problems, Internat. J. Control, 49 (1989), pp. 655–666. [17] D. A. WILSON, Convolution and Hankel operator norms for linear systems, IEEE Trans. Automat. Control, 34 (1989), pp. 94–97. [18] G. ZHU, M. CORLESS, AND R. SKELTON, Robustness properties of covariance controllers, in Proce. of Allerton Conf., Monticello, IL, September 1989. [19] G. ZHU AND R. E. SKELTON, Mixed L2 and L∞ problems by weight selection in quadratic optimal control, Internat. J. Control, 53 (1991), pp. 1161–1176.