Controllability of linear stochastic systems

724

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 46, NO. 5, MAY 2001

Controllability of Linear Stochastic Systems Nazim Idrisoglu Mahmudov

Abstract—We discuss several concepts of controllability for partially observable stochastic systems: complete controllability, approximate controllability, and stochastic controllability We show that complete and approximate controllability notions are equivalent, and in turn they are equivalent to the stochastic controllability for linear stochastic systems controlled with gaussian processes. We derive necessary and sufficient conditions for these concepts of controllability. These criteria reduce to the well-known rank condition. Index Terms—Linear regulator, linear stochastic systems, stochastic controllability.

I. INTRODUCTION

I

N RECENT years, controllability problems for different kind of dynamical systems have been considered in many publications. The extensive list of publications can be found in [1]–[5]. Most literature in this direction so far has been concerned, however, with the deterministic controllability problems. For deterministic systems, the basic controllability concepts have been well investigated. For stochastic systems the situation is less satisfactory. Only a few papers deal with the stochastic controllability problems. In [6], [7] via Lyapunov techniques “null” controllability is discussed. References [8] and [9] contain results concerning generalization of the recurrence notions of weak controllability, controllability, and strong controllability. In [13]–[18], relationships between (approximate) controllability of the stochastic system and corresponding deterministic system have been studied. In the present paper, we systematically study several types of controllability of partially observable linear time-invariant stochastic systems generated by Wiener processes, which are natural generalizations of complete and approximate controllability concepts well-known in the theory of controllability of infinite dimensional deterministic systems (see [10]–[12]). Controllability of partially observable linear autonomous infinite-dimensional stochastic systems is studied in [14]–[17], and controllability of finite dimensional stochastic systems is studied -stochastic conin [13] and [18]. In [13], the concept of trollability for linear stochastic systems with pertubations of a semimartingale type in finite dimensional space is studied. Developing a variational approach to controllability based on the duality theory for extremal problems, the authors established relationships between the corresponding controllability concepts for stochastic and deterministic systems. The basic result proved Manuscript received July 30, 1999; revised May 20, 2000 and October 6, 2000. Recommended by Associate Editor Q. Zhang. The author is with the Department of Mathematics, Eastern Mediterranean University, Gazi Magusa, Mersin 10, Turkey (e-mail: [email protected]). Publisher Item Identifier S 0018-9286(01)03605-4.

in this reference is that a stochastic system is -stochastiif and only cally controllable on each interval -controlif the corresponding deterministic system is (see [19] and the references lable on each such interval therein). Using this result, they obtained rank criteria for -stochastic controllability. The paper is organized as follows. In Section II, we define complete controllability, approximate controllability, and stochastic controllability concepts for linear stochastic systems. We introduce the stochastic controllability operator and point out a link between and the . Under the natural ascontrollability matrices for all , we give an sumption of invertibility explicit formula for a stochastic control which steers any given initial state to any other state. In Section III, we develop a variational approach to present necessary and sufficient conditions for positiveness and cowhich are formulated in ercivity of a linear operator . terms of convergence of sequence operators In particular, we show that uniform convergence to zero of is equivalent to coercivity of the operator (complete controllability), and strong convergence of to zero is equivalent to positiveness of (approximate controllability). These facts are used to study the relationship among the controllability concepts defined in Section II. In Section IV, we study the link of controllability concepts for partially observable linear stochastic systems defined in Section II and controllability of the corresponding deterministic systems. We establish the equivalence between stochastic controllability concepts and the corresponding notion of controllability for deterministic systems. Our analysis shows the following. 1) The complete controllability of the linear stochastic is equivalent to controllability of system on the corresponding deterministic system on every (see Theorem 10). 2) The approximate controllability of the linear stochastic is equivalent to controllability of the corsystem on responding deterministic system on every (see Theorem 11). 3) The complete (approximate) controllability of the linear stochastic system is equivalent to stochastic controllability of the same system with the gaussian control set (see Theorem 13). 4) For the analytically varying stochastic linear systems complete controllability, approximate controllability, stochastic controllability with gaussian controls and controllability of the corresponding deterministic systems coincide (see Corollary 14).

0018–9286/01$10.00 ©2001 IEEE

MAHMUDOV: CONTROLLABILITY OF LINEAR STOCHASTIC SYSTEMS

725

5) Explicit rank conditions for these controllabilities can be derived (see Corollary 14).

It is known that and

In other words, there exist such that

II. DEFINITIONS A. Partially Observable Systems be a probability space with filtration on which are given -measurable random variable with gaussian probability law, with mean and covariance Wiener processes with values in and , are respectively. The variable and the processes is the space of linear transmutually independent. to . is the space of all formations from square integrable processes, adapted to the family . is a domain, and is an image of the linear operator A self-adjoint operator on a Hilbert space is nonnegaif for all ; is positive tive if for all nonzero is coercive if there exists a such that ( for all . . We consider the following class of finite-dimensional par: tially observable system in the interval Let

where

which is a Wiener process, and also a Wiener process (see [25]). The following result is proven in [23]. , then the Kalman filter Lemma 1: If

is the solution of

(2) where

,

is the solution of

(1) ,

where ,

, , , , each of whose entries is a bounded

function. Using standard techniques, we can transform the partially observable problem (1) into a full observation problem. To summarize this, we need the following basic results from Kalman filtering theory. We consider the family of -algebras generated by and generated by where the processes correspond to the state and observation processes, respectively, when the control is . A natural definition of a control set is the Hilbert space of all , adapted to This definition square integrable processes expresses the fact that we want the control to be determined by observing the observation process. However, there is an immediate difficulty that the observation process depends on control. That is why we restrict the set of admissible controls. More precisely, we consider

Remark 2: The process plays a key role in filtering problems. The point is that, first, this process turns out to be a Wiener process, and second, it contains the same information as the does. More precisely, it means that for all process the -algebras and coincide. B. Definitions of Controllability The solution of the linear stochastic differential equation (2) can be written as follows:

Denote this solution by . Here, mental solution of linear time-varying system

and With this definition of admissibility, it is immediate that if then This property makes it easy to obtain the Kalman filter. Introduce an important subclass of admissible controls, dethose defined by a linear feedback on the obsernoted by vation

is the funda-

We say that (2) is smoothly varying on iff and are smooth as functions of for and (2) is analytically varying on iff and are analytic on . Now, let us introduce the following operators and sets: 1) the operator defined by

726


and the set of all states attainable from

in time

3) stochastically controllable on

Clearly, the adjoint is defined by

2) The controllability operator

if

i.e., given an arbitrary it is possible to steer from to within a distance from all points in the point at time with a probability arbithe state space trarily close to one.

associated with (1)

C. Minimum Energy Principle The tween

below

gives

relationship for

which belongs to and the controllability matrix

3) The set of in time

lemma

respectively. Lemma 5: For every process 1)

beand ,

there exists a such that

-attainable nonrandom points from (3) such that

2) (4)

The concept of -attainability has been introduced by in Sunahara and Boyarski in [20] (see also [15]). For fixed etc. in [6] and Klamka and Socha in [7] gave conditions for via Lyapunov approach for stochastic systems of Ito type and Ito-Gihman type, respectively. In [20], it is shown is closed in for each for weak solutions that to a large class of stochastic control systems. The following theorem is in a sense similar to that of for a strong solutions of linear stochastic systems and shows that we need not introduce complete stochastic controllability and approximate stochastic controllability concepts separately [see Definition 4(c)]. Theorem 3: The following statements hold:

Proof: The proof can be found in [18]. For the linear stochastic system (1), we define the following concepts. Definition 4: The system (1) is said to be if all the points in 1) completely controllable on can be reached from the point at that is, if time

2) approximately controllable on

if

i.e., given an arbitrary it is possible to steer from the point to within a distance from all points in the at time ; state space

3) for all

(5) Proof: 1) The proof of (3) can be found in [21, Th. 5.6, p. 165]. From 1), it follows that there 2) Let such that exists

Now, definition of the operator and Fubini’s stochastic theorem lead to desired representation (4)


727

3) Let us show representation (5). Since is the following operator: invertible for all

So, is strictly convex. On the other hand, is weakly lower semicontinuous on . Then, the problem of minimizing . Using variathe functional (6) has a unique solution tional approach (see [22]), we can show that an optimal solution has the following form: (10)

is well defined, and it is obvious that The next lemma gives a formula for a control transto an arbitrary (for deterministic ferring the state analogue of this formula see [4] and [3]) and shows that minimum energy principle holds for the system (1). the matrix Lemma 6: Assume that for arbitrary is invertible. Then the control 1) for arbitrary

Using this in (7), we have

Hence

which implies

and, consequently, we obtain transfers

to

at time , where and can be found from the following representation

2) among all controls the control time

transferring to at minimizes the integral

Proof: The proof is similar to that of [3, Prop. 1.1, p. 14] or [2, Th. 5, p. 108] and, hence, will be omitted. III. A VARIATIONAL APPROACH TO CONTROLLABILITY CONCEPTS Let us consider the following linear regulator problem. To minimize (6) where

defined by

(11) Thus, (9) holds. Substituting (11) in (10), we obtain (8). The lemma is proved. Theorem 8: The following conditions are equivalent: and ; 1) ; 2) converges as in uniform operator 3) topology; converges to zero operator as 4) in uniform operator topology. 2) This equivalence is well known. Proof: 1) 3) Suppose . Then, for all 2) and for all

(7) and

and

are Hilbert spaces,

is a parameter. Lemma 7: There exists a unique optimal control which the functional (6) takes on its minimum value and

at

(8) (9) Proof: Suppose can show that

and

Therefore, for all

We conclude that is bounded with respect to . Furthermore, by resolvent identity

. One

So, topology.

converges as

in uniform operator

728

3) 4)


3) 4): Assume that for all . Since and self adjoint. we deduce that

4): This implication is obvious. 2): Suppose

Then, small

as

as is nonnegative

. For sufficiently

, we can write

Theorems 8 and 9 are also of independent interest. So, for all

, we have IV. THE RANK CONDITION In this section, we study the relationship between the controllability concepts of the stochastic system (1) and controllability of the associated deterministic system in the interval (12)

which implies

and, consequently

Theorem 9: The following conditions are equivalent: is dense in ; 1) ; 2) converges to zero operator as 3) in strong operator topology; converges to zero operator as 4) in week operator topology. 2) This equivalence is well known. Proof: 1) 1): Let be strongly convergent to zero 3) . Consider an arbitrary and the operator as functional (6) with this . By (9), selecting sufficiently small, to be close to . So, is dense in . we can make . Then, for arbitrary , there exists 1) 3): Let in such that as . a sequence We have

where is the control at which the functional (6) takes on its is given, then we can make minimum value. If for some sufficiently large and then, we can to be sufficiently small so that for all select

Thus, for all , i.e., converges to as . By (9) and by the arbitrariness of , this implies the to zero operator as . strong convergence of

where Theorem 10: The following conditions are equivalent. 1) the stochastic system (1) is completely controllable on ; ; 2) converges to zero as in uniform 3) topology; 4) the deterministic system (12) is controllable on every ; 5) the stochastic system (1) is completely controllable on . every Proof: 1) 2) 3) These equivalences follow from Theorem 8. 1) 4) Assume that the stochastic system (1) is controllable Then, by 2) for some on (13) To prove controllability of the defor all and terministic system (12), we use relationship between . Using the representations (3) and (4), we write in terms of and use the inequality : for all there (13) to show coercivity of exists s process


If for and

729

follows, by the Lebesgue-dominated convergence theorem and the equality

otherwise, that is, if can be rewritten as follows:

Taking the limit as

, then the above inequality that for every

we obtain

for all and for all Thus, (12) is controllable on every 4) 5): Suppose that the system (12) is controllable on every . Then, the matrix is invertible for all and the following operator is well defined

It is not hard to see that 5) 1): This implication is obvious. The theorem is proved. Theorem 11: The following conditions are equivalent: 1) the stochastic system (1) is approximately controllable ; on ; 2) converges to zero as in strong 3) topology; 4) the deterministic system (12) is controllable on every ; 5) the stochastic system (1) is approximately controllable . on every 2) 3) This implications follow from TheProof: 1) orem 9. 4) Let the stochastic system (1) be an approximately 1) or, equivalently, for all controllable on

as that is the stochastic system (1) is approximately . controllable on every 1): This implication is obvious. 5) Corollary 12: The stochastic system (1) is completely conif and only if it is approximately controllable trollable on . on We now wish to discuss the situation when the control has to be a gaussian process. The following theorem gives the relationship between stochastic controllability and controllability of the deterministic system. Theorem 13: 1) Complete (approximate) controllability of the system (1) implies stochastic controllability of the same . system with the control set 2) Stochastic controllability of the system (1) with the implies controllability of the correcontrol set sponding deterministic system (12). Proof: 1) If the system (1) is completely controllable, then by Theorem 10, it is completely controllable on every , so is invertible for all Then, Lemma 6 shows that the control

will transfer the point to which means that an arbitrary point can be reached from an by means of gaussian control . Thus, arbitrary such that there exists the gaussian control

From this and (5), we have

From here, we see that

for all a subsequence

and, consequently, there is such that for all

The latter means that the deterministic system (12) is approxi. mately controllable on every 5) Suppose that strongly as 4) for every Since for all it

Now, stochastic controllability with the control set follows from Chebyshev’s inequality. 2) Assume that the system (1) is stochastically controlfor all lable, in other words, and . To prove complete (approximate) controllability of the same system (1), look at the seand such that , quences and as and take fixed . From controllability of the system (1), we obtain the such that existence of the sequence

which implies the convergence of to converges to zero in probability. Hence, in probability. Since is a Gaussian random

730


variable for all , then the characteristic functions of these random variables are

for some integer , then the system (2) is completely (approximately) controllable. V. CONCLUSION

where

and The convergence of to in probability implies the convergence of the characteristic functions, that is for all where valid when

. The last convergence is (14)

From the first convergence in (14), the system (12) is approximately controllable, or equivalently, is controllable. We now look for a simpler condition analogous to the rank condition for time-varying systems. is smooth (respectively, anThe matrix function and when (2) alytic,) as a function of . Introis smoothly (respectively, analytically) varying on matrix functions duce the

and let

Corollary 14: Let and be analytic functions on and is an arbitrary fixed element of . Then, the following conditions are equivalent: for some in1) teger ; for some integer 2) ; 3) the deterministic system (12) is controllable on every non; trivial subinterval of 4) the stochastic system (2) is completely controllable on ; every nontrivial subinterval of 5) the stochastic system (2) is approximately controllable on ; every nontrivial subinterval of 6) the system (2) is stochastically controllable with the conon every nontrivial subinterval of . trol set , so the Remark 15: In the time-invariant case, above condition says that (2) is controllable on any nontrivial span an interval if and only if the columns of dimensional space, which is equivalent to the rank condition . and be smooth functions on Corollary 16: Let . If there exists any such that

Corollary 14 says that the analytically varying system (1) is controllable if and only if the rank condition holds. The rank . The condition does not depend on the diffusion matrix -stochastic controllability concept defined by Dubov and Mordukhovich applied to the systems (1) is an approximate null controllability. The approximate controllability and the approximate null controllability concepts are equivalent for the system is invertible. Analogue of Corollary 14 for the (1) because time-invariant system is provided by Mahmudov and Denker in [18]. They proved that the time-invariant system (1) is controllable if and only if controllability subspace coincide with all . Zabczyk in [8] and Ehrhard and Kliemann in [9] studied the time-invariant case when controllability subspace is a proper . They proved that the linear stosubspace of the state space chastic time-invariant system with the above property and with diffusion matrix is stochastically time independent controllable if and only if

and is a matrix of type 1. Some of the proofs given in Section II and Section III do not depend on the dimension of the state space. Although almost similar results can be obtained for infinite-dimensional partially observable linear systems, the proofs have to be modified due to the fact that, for infinite-dimensional systems, the notions complete and approximate controllability do not coincide. Complete and approximate controllabilities, and their relationship in infinite-dimensional spaces, will be discussed in a subsequent paper (see [26]). ACKNOWLEDGMENT The author expresses his gratitude to the referees for a number of helpful comments and suggestions. The author is also thankful to J. Shibliev for a thorough revision of the English of this paper. REFERENCES [1] R. E. Kalman, “A new approach to linear filtering and prediction problems,” J.Basic Eng., ser. Transactions ASME Ser D, vol. 82, pp. 35–45, 1960. [2] E. D. Sontag, Mathematical Control Theory: Deterministic Finite Dimensional Systems. New York: Springer-Verlag, 1990. [3] J. Zabczyk, Mathematical Control Theory. Boston, MA: Birkhauser, 1992. [4] A. Bensoussan, G. Da Prato, M. C. Delfour, and S. K. Mitter, Representation and Control of Infinite Dimensional Systems. Boston, MA: Birkhauser, 1993, vol. 2. [5] R. Curtain and H. J. Zwart, An Introduction to Infinite Dimensional Linear Systems Theory. New York: Springer-Verlag, 1995. [6] Y. Sunahara, T. Kabeuchi, S. Asada, S. Aihara, and K. Kishino, “On Stochastic Controllability for Nonlinear Systems,” IEEE. Trans. Automat. Contr., vol. AC-19, pp. 49–54, 1974. [7] J. Klamka and L. Socha, “Some remarks about stochastic controllability,” IEEE Trans. Automat. Contr., vol. AC-22, pp. 880–881, 1977. [8] J. Zabczyk, “Controllability of Stochastic Linear Systems,” Syst. Control Lett., vol. 1, pp. 25–31, 1981. [9] M. Ehrhard and W. Kliemann, “Controllability of Stochastic Linear Systems,” Syst. Control Lett., vol. 2, pp. 45–153, 1982.


[10] H. O. Fattorini, “Some remarks on complete controllability,” SIAM J. Control, vol. 4, pp. 686–694, 1966. [11] D. L. Russel, “Nonharmonic Fouier series in the control theory of distributed parameter systems,” J. Math. Anal. Appl, vol. 18, pp. 542–560, 1967. [12] R. Triggiani, “Controllability and observability in Banach space with bounded operators,” SIAM J. Control, vol. 13, pp. 462–491, 1975. [13] M. A. Dubov and B. S. Mordukhovich, “Theory of controllability of linear stochastic systems,” Differential Equations, vol. 14, pp. 1609–1612, 1978. , “On controllability of infinite-dimensional linear stochastic sys[14] tems,” in Proc. 2nd IFAC Symp. Stochastic Control, Vilnius, May 1986, pp. 307–310. [15] A. E. Bashirov and K. R. Kerimov, “On controllability conception for stochastic systems,” SIAM J. Control Optim., vol. 35, no. 2, pp. 384–398, 1997. [16] A. E. Bashirov, “On weakening of the controllability concepts,” in Proc. 35th IEEE Conf. Decision Control, vol. 1, Kobe, Japan, December 11–13, 1996, pp. 640–645. [17] A. E. Bashirov and N. I. Mahmudov, “On concepts of controllability for deterministic and stochastic systems,” SIAM J. Control Optim., vol. 37, no. 6, pp. 1808–1822, 1999. [18] N. I. Mahmudov and A. Denker, “On Controllability of linear stochastic systems,” Int. J. Control, vol. 73, pp. 144–151, 2000. [19] B. S. Mordukhovich, Approximation Methods in Problems of Optimization and Control. Moscow, Russia: Nauka, 1988. [20] A. Boyarski, “Finite-dimensional attainable sets for stochastic control systems,” J. Optim. Theory Appl., vol. 22, no. 3, pp. 429–445, 1977. [21] R. S. Liptser and A. N. Shiryaev, Statistics of Random Processes. New York: Springer-Verlag, 1977.

731

[22] N. I. Mahmudov and A. E. Bashirov, “First order and second order necessary conditions of optimality for stochastic systems,” in Proc. Steklov Mathematical Inst. Seminar, Statistics Control Stochastic Processes, 1997, pp. 283–296. [23] A. Bensoussan, Stochastic Control of Partially Observable Systems. London, U.K.: Cambridge Univ. Press, 1992. [24] W. Rudin, Functional Analysis. New York: McGraw-HilCompany, 1973. [25] A. Lindquist, “On feedback control on linear stochastic systems,” SIAM J. Control, vol. 11, no. 2, pp. 323–343, 1973. [26] N. I. Mahmudov, “Controllability of linear stochastic systems in Hilbert spaces,” J. Math. Anal. Appl., to be published.

Nazim Idrisoglu Mahmudov was born in 1958 in Cebrayil Province, Azerbaijan. He received the B.Sc. and M.Sc. degrees in mathematics from the Baku State University, Baku, Azerbaijan, and the Ph.D. degree in mathematics in 1985 from Institute of Cybernetics of Azerbaijan Academy of Sciences, Baku. He is currently an Associate Professor at the Eastern Mediterranean University, Turkey. He has done research in stochastic optimal controls, controllability of linear and nonlinear systems. His research interests include the areas of control theory, stochastic control, differential equations, and linear and nonlinear systems.