Convergence Properties of the Riccati Difference Equation in Optimal

110

IEEE TRAXSACTIONS ON AUTOMATICCONTROL,

VOL. AC-29, NO. 2. FEBRUARY 1984

Convergence Properties of the Riccati Difference Equationin Optimal Filtering of Nonstabilizable Systems SIEW WAH CHAN, GRAHAM CLIFFORD GOODWIN. MEMBER. IEEE, AND KWAI SANG SIN

Abstract-This paper studies the convergence and properties of the solutions of the Riccati difference equation. Special emphasis is given to systemswhicharenotnecessarily stabilizable (in the filtering sense), particularlythosehavinguncontrollableroots 011the unit circle. Besides generalizing and unifying previous tiork. the results have application to a number of important problems including filtering and control of -stems with purely deterministic disturbances such as sinusoids and drift compu-

nents.

I. INTRODUCTION

T

HE Riccatidifferenceequation arises in linear quadratic optimalcontrol and filtering problems. The filtering and control problems are duals of each other. We shall therefore give emphasis tooptimal filtering and briefly summarizethedual interpretations in optimal control. The convergence and properties of the solutions of the Riccati equation were first studied by Kalman and Bucy [1]-[3] subject to both controllability and observability assumptions. These assumptionswere later relaxed to detectability and stabilizability by Wonham [4] (for continuoustimesystems) and Caines and Mayne [9] (for discrete time systems). Corresponding results have also been established[lo] for time varying detectableand stabilizable systems. However, these results do not apply to systems which are not stabilizable (in the filtering context); a simple example being the system: x(t+l)=ax(t);

y(r)=x(t)+u(t)

)a121

(1.11

where { r : ( t ) } denotes measurement noise. It is clearly desirable to clarify the question of asymptotic stability and or time invariance of the K h a n filter when applied to such a trivial problem. An important class of systems to whichthe results do not applyarethosewhichhavepurelydeterministicdisturbances including sine waves and drift terms. These disturbances can be modeledbyobservable but nonstabilizable state spacemodels hailing uncontrollable roots on the unit circle. In fact, the question of stochastic adaptive control of systems haling sinusoidal disturbanceswastheauthors’ original motivation[13] to study the topic addressed in the current paper. The nonstabilizable case for continuous time systems has been

treated by a number of authors including Kucera [6], Martensson [5], Willems [7], and Cdlier and Willems 181. In [5] it was shown that if thesystemisdetectable but not stabilizable, thenthe algebraicRiccatiequation has morethanonesymmetric nonnegative definite solution. It was also shown that, with respect to the usual ordering of symmetric matrices, the largest solution is the only one that lields an asymptotically stable filter. A more detailed study of all symmetric nonnegative definite solutions that arise whenthesystem is not stabilizable was carried out by Kucera in [6]. In his paper all nonnegative definite solutions to the algebraic Riccati equation were characterized by a distributive lattice. Parallel results for discrete systemswere described in [ll].Willems in [7] evaluated an expression for all thesymmetricsolutions(positive and negativesemidefinite) to the algebraic Riccati equation. In [5], Martensson also discussed the convergence propertiesof thesolutions of theRiccati differential equationtowarda stationary solution for some special cases although a general proof of convergence was not D ven. In arecent paper [SI, theconvergence of thematrixRiccati differential equation for systems that are not necessarily stabilizable was discussed. A criterion for convergence was derivedand a technique was developed to determine which solution, if any, the solutions of the Riccati differential equation converge to. However, the discussion excludes uncontrollable roots on the imaginary axis (which corresponds to uncontrollable roots on the unit circle in the discrete case). To the authors’ knowledge, there has been no systematic study of the convergence of the matrix Riccati difference equation in the discrete case for nonstabilizable systems. These problems are addressed in the current paper including the case where there are uncontrollable roots on the unit circle. The layout of the paper is as follows. In SectionI1 we wil briefly describe the filtering problem and introduce the Riccati difference equation and the limiting algebraic Riccati equation. InSection I11we definethe strung solution to thealgebraic Riccati equation as the one giving rise to a filter having roots on or inside theunitcircle.Wediscuss existence, uniqueness, and other properties of the strong solution. In Section IV we present themainnew results of thepaper on theconvergence of the solutions of the Riccati difference equationto the strong solution. Finally. in Section V we discuss some implications of the results.

11. O P ~ M AFILTERING L Manuscript received March 15. 1982: revised J q u a r y 10. 1983. Paper recommended by W. E. Schmitendorf.PastChamnan of the Optmal Systems Committee.This work was supported by the Australian Research Grants Committee and the Radio Research Board. S. W. Chan and G. C. GoodxO. Case 2, 1u1> I : Herethesystemisunstabilizable and thus from Theorem 3.1 vi) there are at least two nonnegative solutions to the ARE (in fact there are exactly two in this case, Z = 0 and Z = a’ - 1). The strong (in fact stabilizing) solution here 2 , = a’ - 1 since it gives an asymptotic stable filter with A = l / a where Il/al< 1. With 2 = 0 the filter is unstable since A = a N l t h la I > 1. To ensure that lim, x E( r ) = 2,. Theorem 4.2 requires 2, > 0. othernlse . Z ( r ) = 0 for all t giving rise to an unstable filter. Theorem 4.2 showsthat stabilizability of (.4,D ) is not necessary for filter stability or timeinvariancecontrary to the impression given in [15. p. 82 and problem 4.3. p. 841 Case 3, la1 = I : With 1u1=1, the system has an uncontrollable root on the unit circle, and henceis not stabilizable. Furthermore, Z = a’ - 1 = 0 making Z = 0 the only nonnegative solution to the ARE. Hence, Z, = c i s the strong (but not stabilizing) solution giving a filter with A = a where In1 = 1 [compare to Theorem 3.1 iv), v)]. Theorem 4.3 ensures that lim, -,E(t) = Z, for ( 2 ,- 2,) > 0 or Z, = Z,. Since Z, = 0, lim, -,Z(t) = Z, for Z o > 0 in this case. If Z, = 1 at time t o =1. then the associated RDE is

(5.2)

When la1 2 1 , then this system is not stabilizable (in the filtering

Z ( t + l ) = - - Z- ( t ) . Z(t)+l ’

Z( t o ) = Z(1)

= Z,

=l. (5.7)

The optimal control is known to satisfy:

The analytic solution of (5.7) is 1 Z(t)=-. t

where

as l / t (not exponentially fast -see Theorem 4.3) and the resulting steady-state Kalman filter is Thus, Z ( t ) converges to Z,

(5.14)

u(t) =-L(t)x(t)

=0

t(r+l)=%(t).

e - .

(5.15)

(5.9) and

Of course, the estimates generated by the time-varyingKalman filter (5.3) converge (in mean square) to x, as 1 tends to c o , and thus asymptotically the correct initial conditions are "generated" for the steady-state filter (5.9). 2) Consider the MA equation r(t)=~(t)+C1w(t-l)+

~ ( t ) =( B T S ( t + l ) B + r ) - ' B T S ( t + l ) A

+C,p(t-n).

(5.10)

~(t)=~~s(r+l)~+s2-~(t)~[~~~(t

(5.16)

S ( N ) =o,v.

(5.17)

The resulting closed loop system is x(t+l)=[A-BL(t)]x(t);

An appropriate state space model is

x(t0)=X,.

Wenowdefine B and E (analogously toand filtering context) as

(5.18)

D in the

-

B P (r-1/2)TB (5.19)

L11

E'E

a.

(5.20)

Then the dual results to Theorems 4.1 to 4.3 are as follows. Applyingthe transformation given in (2.4) to (2.7) we see that Theorem 5.1, Subject to the resulting systemis uncontrollable. In fact, it is not stabilizi) ( E , A ) is detectable, able unless the polynomial C ( z - ' ) = I + Clz-' + . . . Cnz-" is ( A , B ) is stab&&,le, stable, but it is observable provided C, # 0. Thus: i i i )a,&,> 0, then we can i)when c(z-')h a roots on the unit then ur- ,S( t ) = ,'j, (exponentially fast) where s, corresponds Theorem 4.3 to conclude that x ( t ) will converge to 2 s giving all to the stabilizing solution of the associated ARE and where S( t ) roots of the filter on or inside the unit circle provided 2 0 - 2 , > 0, is the solution of the RDE (5.16). the ktthe resulting closed i i )when C ( z - ' ) has no root on the unit circle then we can loop system (5.18) is asymptotic~ystable. l i converge to 2 , neorem 5.2: Subject to: apply Theorem 4.2 to conclude that Z ( t ) w all roots of the filter inside the i) there are no unobservable modes of ( E , A ) on the u ~ t (exPonenti& fast) circle for any 2 , > 0. circle, Thelattersituation corresponds to theusual assumption made ( A , B ) is &&ilizable, in MA models that the zeros can be assumed to be inside the unit iii) w,,o, circle. This clarifies [15, example 4.2, p. 1131. , fast) where S ( t ) isthe then l i n ~ + - ~ S ( t ) = S(exponentially 3) In filtering Or stochastic control Problem With determinis- solution of the RDE (5.16). In the k t the resulting closed loop stable. tic disturbances Such as periodic disturbances and t m d s , then an system (5.18)is observable model can always be constructed [13]. However, this Theorem 5.3: subject to model will have uncontrollable modes on the unit circle to i) ( A , B ) is controllable, describe the deterministic disturbances. Theorem 3.1 iv) tells us ii) (o,v - S,) > or Q , =~ S,, that the strong Of the ARE will have roots On the unit then Em, - ,S( t ) = s, where S ( t ) is the solution of the RDE from Theorem 4.39 x ( t ) converge to Z s provided (5.16). In the k t the resultingclosed loop system (5.18) has 2 , satisfies the conditions of the theorem. This implies that in roots inside or on the unit circle. steady state, it is reasonable to assume that thesystem is deA specific ex,ple for optimal control is scribed by an ARMA model of the form: x ( t + l ) = a x ( t ) + u x( t( )l ); = X , , a > l (5.21) A ( q - ' ) y ( t ) = B(4-').ct>+C(4-')w(t> N-1 J=0,x(N)2+ U(t)2. (5.22) where C( q-') has roots on and inside the unit circle; the roots on r =Io the unit circle also appear in A ( q - ' ) and B ( q - ' ) . This justifies the key modeling assumption made in [13] where adaptive control This is the dual of (5.1). Thesystem has an unstable unobof systems such is discussed. servable mode (in the control context).system The is stabilizable 4) Finally, we present the dual results for linear quadratic and thus the stabilizing solution to the ARE exists and is unique. optimal control. Moreover,Theorem 5.2 ensures that Eml+ - , S ( t ) = S, (the Consider the following system: stabilizing solution) giving a stable closed loop system with pole at l/a for any > 0. x(t+l)=Ax(r)+Bu(t); x(t,)=i0 (5.12)

+

~

-

VI. CONCLUSION

with cost function N-1 J=x(N)To~Vx(N)+ I =fg

x(t)'ax(t)+u(r)'ru(t). (5.13)

This paper has discussed the solutions of the algebraic Riccati equation and the Riccati difference equation when the system is observable or detectable but not necessarily stabilizable (in the filtering sense). Particular emphasis has beengiven to systems

Authorized licensed use limited to: Anatol Tunik. Downloaded on May 28, 2009 at 01:32 from IEEE Xplore. Restrictions apply.

116

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL.

having uncontrollable roots on the unit circle. Possible applications of the results to problems in filtering and control have been briefly discussed. The significance of the new results presented here is that they clanfy some important misconceptions in the literature concerning the conditions for asymptotic time invariance of the Kalman filter. These misconceptions may have unnecessarily limited the class of problems to which the Kalman filter has been applied in the past.

AC-29,NO. 2, FEBRUARY 1984

Hence, y,*x, = 0 which gives

+ x:Qxl

y,*i'?Cyl

= 0.

Again we have D 'xl

=cyl = 0

which implies A'x,

= Ax,

+x,

DTxl = 0 .

Part ix): We display the proof for a) only. The proof for b) is analogous. For the only if part of the proof, let

- I A l = l , i = 0 , 1 , . - - , 2 p - l , X - ~ = Y - ~ = O . (A.l)

Similarly the proof can be carried out for i = 2 , 3 , - ..,p._l, thus showing that there is an uncontrollable mode of multiphclty p of ( A , D ) on the unit circle. To show that the multiplicity is not greater than p consider the case i = p, then

However,

Then

[-yp.

A'xi

+ C'CA-'QX, - C'CA-ly,

=AX,

+~ , - 1

(A.2)

xp*][x;']#o

by Lemma 3.1 ii), (B.5), and Lemma C.l, i.e.,

- A - ' Q x , + A - ' ~ , = A ~ , + ~ , - ~(A.3) .

Multiplying (A.3) by

YP*Xp-' # 0.

CTcand adding it to (A.2) gives us

A'x, - ACTCyl - CTCy;-l = A X ; + X ; - ' .

(A.4)

Thus, D'xp and Cyp cannot be zero at the same time. Hence, it is not true that

Multiplying (A.3) by A gives

Arxp =Ax,

-QX;+Y;=AA~,+A~,-~.

Premultiplying (A.4) by A*y: results gives A*Ay:C'Cy,

and (A.5)by x: and adding the

+ A*Ay,*x; + A*y,*x;-l xTQx; + Y;X; y;"-lA'~;

+ A*y:C'G?;-'

(A.5)

-

-

+ xP-'

D Txp= 0.

This completes the on& if part of the proof. The if part of the result can be established as follows. From Lemma 3.1 vii) we see that if there is an uncontrollable mode A of multiplicity p of ( A , D ) on the unit circle such that

(A.6)

or since A*A = l A I 2 =1, we have y:C'cyi

+ xjrQx, + A*y,*C'cy,-l + A*y:xj-l + yi*_,A'xj

= 0.

(A.7)

then M will have an eigenvalue A on theunit eigenvectors

circle with p

We now consider different values of i. When i = 0, xi-' = = 0, we have yz?,*c'Cyo + x$Qxo = 0.

- -

Hence, y:C 'Cy,

=0

and xgQxo = x,*DDTxo= 0, giving D'x,

= Cyo = 0.

Cyo = 0 implies using (A.4) A'x, =0

= x,, which together with D'x, qualifies A as an uncontrollable eigenvalue of ( A , D ) with

Ihl = l . When i = 1 we have [since yo = 0 from Lemma 3.1 vii)]

yTcTcyl

+ x:Qxl

+ A*yTx, = 0.

From Lemma 3.1 ii),j3 = (A*)-' is also an eigenvalue of M with p eigenvectors associated with it. In this case where I A j = 1, fi = A and /3 .has to be in the same Jordan block as X. To show this we argue by contradiction: if B is in a different Jordan block, then it can be shown (using similar argument as in the only if part of the proof) that both j3 and A are uncontrollable modes of ( A , D on ) the unit circle but in different Jordan blocks. This is clearly not possible since otherwise the number of uncontrollable modes of ( A , D ) on the unit circle would change. Thus, M will have an eigenvalue h of multiplicity 2 p on the unit circle. From the above we see that eigenvalues of M on the unit circle cannot have odd multiplicities. Part x): Let the generalized eigenvectors of a, be

From Lemma 3.1 i i )and (B.5) of Appendix B and Lemma C.l of Appendix C, we have

[~~],["1+'];..,[~;"'-'] bI+l br+p,-l by Lemma 3.1 ii)

Authorized licensed use limited to: Anatol Tunik. Downloaded on May 28, 2009 at 01:32 from IEEE Xplore. Restrictions apply.

CHAN et

117

01. : CONVERGENCEPROPERTIES OF RICCATI DIFFERENCEEQUATION

[-3':

If

where I?-' is a vectorpolynomial in ai of the form

are used in the construction of Z, then

is a chain of generalized left eigenvectors associated with(a:)-'. Let the chain of generalized eigenvectors associated with 9 + (a:)-' be

[~],[~::]....'[yi+pJ~l]. Then by (B.5) of Appendix B =0

for

u;",suj+r

]I:;[

a,*,,]

z = (X*)-'(X*Y)x-' for all i , j , s, r.

=0

r,s=0,1;.-,pi-1. (A.ll)

Finally, if X is nonsingular, then Z = YX-' is a solution to the ARE [Lemma 3.1 i) b)] and, since

i , j , s, r

which together with Lemma C.l implies

[ - b;+,

= u ?J + suJ.+ r

Equations (A@, (AlO), and(A.ll) all imply that ( X * Y ) is Hermitian if the conditions of the lemma are satisfied. Conversely, if the conditions of the lemma are violated, then at least one inequality will occur in the equations corresponding to (A.lO) and ( A X ) since the indexes run over different sets. Hence, the matrix X*Y will not be Hermitian.

Xj+pJ- 1

%+,[X:]

which together with Lemma C.l implies

Letthe chain of generalizedeigenvectorsassociatedwith (a:)-' be

(A.8)

it is clear that Z is Hermitian if and only if X*Y is Hermitian. 0

Pi=

APPENDIX B F'ROPER~ESOF GENERALIZED LEFTAND RIGHT E~GENVALUES

fi

If Aj an eigenvalue of M of multiplicity p . , then the generalized eigenvectors zj+,, r = 0,l; . . p . -1 w i$ rank 1,2; . ..passociatedwiththeeigenvalue A j iadsfy the set of nontrivid solutions

Then by (B.5) of Appendix B l++s

[

c;+j

=

di+r

{ # O0

r+s=p,-1 otherwise.

=

each pair of aiand B;

If exactly pi eigenvectors are used in the construction of

(M-Ajl)Zj=O ( M - A j 1 ) Z j + , = Zj+,-'

(A.9)

(B.1) (B.2)

A row vector wi # 0 is called a left eigenvector of M associated with A, if

s+r=p,-2. Thus, from (A.9)

r=1,2;..,pj-1.

U;(M-A,I)=O

03.3)

and a,+,# 0 is called a generalized left eigenvector of rank (s 1) associated with an eigenvalue A; of M if =f oo r aWs +~ r+=~p( ,M- 2- A , I ) =~W = 1~ ,+2~, - ~ - -, , p ; - l . (B.4)

+

~.+,[~,~~] which together with Lemma

It is well known [16], [17] that, without loss of generality, the eigenvectors can be chosen such that

C.1 implies

b:+sc,c,+, = a,?+sdi+r

for all s + r = p i -2.(A.10)

Letthe chain of generalizedeigenvectorsassociatedwith where lyjl = 1 be

w,+~z~+,#O, y,

=0

i = j and s + r = p , - l = p , - l

otherwise.

APPENDIX

c

LEMMA C.1 (Seeproof of Lemma 3.1 ix) for the fact that y, has even multiplicity.) Let 2j+s, s = 0;. . , 2 p , -1 be the chain of generalized left eigenvectors associated with ( l / y J ) * . By (B.5)

'J+S

]I[::

=

{ #= OO

r+s=2pj-1 otherwise. sequence

[;:I:]

Let z , ~ = , and ',, r = 0; . .,p - 1 be, respectively, thegeneralizedright andleft eigenvectors of the symplectic matrix M (3.1) associated, respectively, with the eigenvalues A; and ( l / h , ) * of multiplicity p,. Let vJ = of vectors. If

Authorized licensed use limited to: Anatol Tunik. Downloaded on May 28, 2009 at 01:32 from IEEE Xplore. Restrictions apply.

[2.1

j = 0.;

.,q be a

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL.

118

q + p j = 0, EE8109.1981. then

r = 0 ; . .,PI-1 3 j = o ...,q

(c.1)

Rep.

”.

(c.2)

proof: n e proof will be &ed a) For r = 0 we have

out by induction, using (3.2).

and the result follows immediately. b) Assume (C.2) is true for r = 0; . .,k - 1; k < p , . From (C.l) and (3.2) we obtain: C-

which implies

and taking into account the definition of (pi-,)*

G\,en in (3.3)

and the induction hypothesis, we conclude:

AC-29,NO. 2, FEBRUARY 1984

[13] G. C., Goodwinand S. W. Chan, ‘‘Deterpinistic and stochastic adaptwe control with purely determinisbcbturbances,” Dep. Elec. Comput. Ekg., University of Newcastle, New South Wales, Australia, Tech. [14] G. C. Goodwin and R. Payne, L. Dynamic System Identificatiorz- Experiment Design and Data Analysis. New York: Academic, 1977. [15] B. D. 0. Anderson and J. B. Moore, Oprinzal Filtering. Englewood Cliffs, NJ: Prentice-Hall, 1979. [16] M. C.Pease 111. Merhods of Matrix Algebra. NewYork:Academic, 1965. [17] D. K.Faddeevand V. N.Faddeeva, Computational MethodF of Linear Algebra. San Francisco. CA: Freeman, 1963.

Siew Wah Chan wasborninSingapore on February 27,1951. He received the B.E. degree inelectricalengineering(computer)fromthe University of Newcastle,NewSouthWales, Australia, in 1977. Since September 1980 he has been a Professional Officer in theDepartment of Electrical and Comuuter Endeerine. Universiw of Newcastle, New S o d WalesTAustralia,-wherehe worked on adaptive control using microprocessor systems, high frequency electronics for adaptive array processing, digital systems for aP.W.M. inverter for variable speed drive machines, and general microprocessor hardware and software develonment. Currentlv he is enzaeed in research work for the PhD. deueein -r------~ theDepartment of E l e c t r i z a n d Computer E n e e e h g , U i v e G i t y of Newcastle. His researchinterests lie inthegeneralarea of filtering, prediction, and control of discrete-time systems. ~-_I

Aistraha. in 1964, 1966. and 1970. respectively.’ From 1970 to 1974he m-as a Lecturer in the REFERENCES

R.E. Kalman, “Contributions to the theory of optimal control,” Bol. SOC.Mat. Mex., vol. 5. pp. 102-119, 1961. R. E. Kalman, “When is a h e a r control system optimal?,” Trans. A S M E , J . Baric Eng., p 5160, Mar., 1964. R. S. Bucy, “Global %eory,f the ficcati equation,” J . Cornput. Syst. Sci.. V O ~1.. pq; 349-361. 1967. W. M. Wonham. On amatrixRiccatiequation of stochastic control,” SIAM J . Conrr.. vol. 6, no. 4. pp. 681-698, 1968. K. Martensson, “On the matrix Riccati equation,”Inform. Sci.. vol. 3, pp. 17-49, Jan. 1971. V. Kucera, “On nonnegative definite solutions to matrix quadratic equations,” Automarica, vol. 8, pp. 413-423, 1972. J. C. WiUems, “Least-squares stationary optimal control and algebraic Riccati equation,” IEEE Trans. Automat. Contr., vol. AC-16, pp. 621-634,1971. F. M. Callier and J. L. Willems, “Criterion for the convergence of thesolution of the Riccatidifferentialequation.” IEEE Trans. Automat. Conrr.. vol. AC-26. ny; 6, pp. 1232-1242, 1981. P. E. Caines and D. Q. Mayne, On the discrete-time matrix Riccati equation of optimal control.” Int. J . Conrr.. vol.12. pp. 785-794. Nov. 1970. B. D. 0. Anderson and J. B. Moore. “Detectabilitv and stabilizability of time varying discrete time h e a r systems,” SIriM J . Conrr. Oprimiz., vol. 19, Jan. 1981. V. Kucera, “The discrete R~ccatiequation of optimal control.’‘ Kybernetika (Prague), vol. 8, pp. 430-447, 1972. -, “A contribution to matrix quadratic equations,’‘IEEE Trans. Automat. Contr.. vol. AC-17. pp. 344-357, 1972.

-

.

< . 3

been-nith the Univ&ity of Newcastle, New South Wales. Australia. wherehe is currently Professor and Head of the Department of Electrical and Computer Engineering. He has written a number of papers in the area of system identification and adaptive control and is the coauthor ofConrrol Theon, (London:Oliverand Boyd.1970): DwamicSystem Idenrlficatlon: E.ypenmenr Design and Data Analysis(New York: Academic, 1977). and Adaptive Filtering. Prediction and Control (Englewood Cliffs. NJ: Prentice-Hall, 1984). *%