Michael S. Branickytt. Vivek S. BorkarO .... a< v
Extended abstract presented at 11th International Conference on Analysis and Optimization of Systems, INRIA, France, June 15-17, 1994. Summary accepted to 1994 IEEE Conference on Decision and Control.
Revised
A Unified Framework for Hybrid Control: Background, Model, and Theory* Michael S. Branickytt
Vivek S. BorkarO
Sanjoy K. Mittert¶
April 1994 Revised June 1994
Abstract We propose a very general framework for hybrid control problems which encompasses several types of hybrid phenomena considered in the literature. A specific control problem is studied in this framework, leading to an existence result for optimal controls. The "value function" associated with this problem is expected to satisfy a set of "generalized quasi-variational inequalities" which are formally derived. Key Words: jumps.
1
Hybrid control, optimal control, quasi-variational inequalities, autonomous jumps, impulsive
Introduction
Hybrid control systems are control systems that involve both continuous dynamics and controls, as well as discrete phenomena. Some examples include computer disk drives [13], transmissions and stepper motors [9], constrained robotic systems [2], and intelligent vehicle/highway systems [22]. More generally, such systems arise whenever one mixes logical decision-making with the generation of continuous control laws. such as in modern flight control systems. In this paper, our focus is on the case where the continuous dynamics is modeled byr a differential equation x(t) = J(t),
t>_O
(1.1)
Here, x(t) is the continuous component of the state taking values in some subset of a Euclidean space. ~(t) is a controlled vector field which generally depends on x(t), the continuous component u(t) of the control policy, and the aforementioned discrete phenomena. We shall make this more precise later on. The discrete phenomena generally considered are of four types: (1) autonomous switching, (2) autonomous jumps, (3) controlled switching, and (4) controlled jumps. In this paper, we study a model that subsumes all these phenomena and study an associated control problem. The paper is organized as follows. The next section details the discrete phenomena arising in hybrid systems listed above, including some simple examples. Section 3 reviews models of hybrid *Work supported by the Army Research Office and the Center for Intelligent Control Systems under grants DAAL0392-G-0164 and DAAL03-92-G-0115. tLaboratorv for Information and Decision Systems and Center for Intelligent Control Systems, Dept. of Electrical Engineering and Computer Science, Massachusetts Institute of Technology. tGraduate Student. LIDS, MIT, 77 Mass. Ave., 35-415, Cambridge, MA 02139-4307. E-mail: branickyOlids.mit.edu. "Professor. Dept. of Electrical Engineering, Indian Institute of Science, Bangalore 560012, India. ¶Professor. LIDS, MIT, 77 Mass. Ave., 35-304, Cambridge, MA 02139-4307.
1
systems from the control and dynamical systems literature. Section 4 abstracts the phenomena found in hybrid systems to a unified framework, which is related to the other models reviewed. Section 5 defines an optimal control problem in this framework. All assumptions used in obtaining the remaining results are expressly stated there. The necessity of such assumptions is discussed throughout the sequel. The existence of an optimal control for this problem is established in Section 6. Section 7 gives a formal derivation of the associated generalized quasi-variationalinequalities. Finally, Section 8 concludes with a list of some open issues. Closely related issues occur in the study of piecewise deterministic processes, an excellent account of which can be found in [12]. Finally, we collect some notation used throughout. First, we make use of the abbreviations ODEs (ordinary differential equations), FA (finite automata/on), and DEDS (discrete event dynamical systems; see [15]). R, It + , Z, N denote the reals, nonnegative reals, integers, and nonnegative integers, respectively. For x C R, [xJ denotes the greatest integer less than or equal to x, and, in an abuse of common notation, Fxl denotes the least integer greater than x. N denotes the set {1, 2,..., N}. Below we deal with continuous time systems, such as ODEs, that are affected by events at discrete instants, such as jumps in state. We will use [t] to denote the time less than or equal to t at which the last "jump" or "event" occurred. If the event is unclear, we will subscript the variable, so that [t]p denotes the time at which the variable p last jumped. Throughout, v ip] and x[t] will be shorthand for
v(LpD) and x([t]), respectively. Other notation is common. For example, X\U represents the complement of U in X; U represents the closure of U, U ° its interior, OU its boundary; f(t+), f(t-) denote the right-hand and left-hand limits of the function f at t, respectively; a function is right-continuous if f(t +) = f(t) for all t; C(X, Y) denotes the space of continuous functions with domain X and range Y; v T denotes the transpose of vector v.
2
Hybrid Phenomena
In this section, we briefly examine the discrete phenomenon that arise in the study of hybrid systems: (1) autonomous switching, (2) autonomous jumps, (3) controlled switching, and (4) controlled jumps. We also discuss how finite automata may be viewed as evolving in continuous time, which sets the stage for their interacting with ODEs below. AUTONOMOUS SWITCHING. Here the vector field J(.) changes discontinuously when the state x(-) hits certain "boundaries" [19, 21]. The simplest example of this is when it changes depending on a "clock" which may be modeled as a supplementary state variable [9]. An example of autonomous switching is the following: Example 2.1 Consider the following model of a system with hysteresis [21]: X1
=
x2 -(X
1)
Jx2
= H(0(xl,X2)) -
(X2)
where the multi-valued function H is shown in Figure 1. The functions X, 0 depend on the exact system under consideration. Note that this system is not just a differential equation whose right-hand side is piecewise continuous. There is "memory" in the system, which affects the value of the vector field. Indeed, such a system naturally has a finite automaton associated with the function H, as pictured in Figure 2.
2
H
Ai /
a
.
.
.c
b
d
\|/
Figure 1: Hysteresis function, H. a< v 1/e, one can show that u(.) - -e is optimal.
8
Conclusions
We examined the phenomena that arise in hybrid systems and reviewed several models of hybrid systems from the literature. We then proposed a very general framework for hybrid control problems which encompasses these hybrid phenomena. A specific control problem was then studied in this framework, leading to an existence result for optimal controls. The "value function" associated with this problem is expected to satisfy a set of "generalized quasi-variational inequalities" which were formally derived. The foregoing presents some initial steps towards developing a unified "state space" paradigm for hybrid control. Several open issues suggest themselves. We conclude with a brief list of some of the more striking ones. 1. A daunting problem is to characterize the value function as the unique viscosity solution of the generalized quasi-variational inequalities (7.4)-(7.7). 2. Many of our assumptions can possibly be relaxed at.the expense of additional technicalities or traded off for alternative sets of assumptions that have the same effect. For example, the condition d(Ci, Ai) > 0 could be dropped by having c 2 penalize highly the impulsive jumps that take place too close to Ai. (In this case, Assumption 4 has to be appropriately reformulated.) 23
3. Example 6.6 show that Assumption 3 cannot be dropped. In the autonomous case, however, the set of initial conditions that hit a C °O manifold are of measure zero [21]. Thus, one might hope that an optimal control would exist for almost all initial conditions in the absence of Assumption 3. The system of Example 6.7 showed this to be false. Likewise, in the systems of Example 6.6 we have, respectively, no optimal control for the sets {(Xl,X 2 ) I X2 < 0.X 1 < 1,x
2
+ 1 > X1 }
and {(xI,x2) I x 2
x1} U
([O, 1] x [-a/2,0] - B([1, -1]T,
1))
where a = 2 - v2 and B(x, r) denotes the ball of radius r about the point x. It remains open how to relax the conditions Assumptions 3 and 4. This might be accomplished through additional continuity assumptions on G, !l, and cl. 4. An important issue here is to develop good computational schemes to compute near-optimal controls, which is currently a topic of further research. See [10] for some related work. This is a daunting problem in general as the results of [7] show that the hybrid systems models discussed in Section 3 can simulate arbitrary Turing machines (TMs), with state dimension as small as three. It is not hard to conceive of (low-dimensional) control problems where the cost is less than 1 if the corresponding TM does not halt, but is greater than 3 if it does. Allowing the possibility of an impulsive jump at the initial condition that would result in a cost of 2, one sees that finding the optimal control is equivalent to solving the halting problem. 5. Another possible extension is in the direction of replacing Sio by smooth manifolds with boundary embedded in a Euclidean space. See [8] for some related work. 6. In light of Definition 7.1, all the proofs seem to hold if Assumption 1 is relaxed to only consider distances "from the right," that is if infi d+(Ai, Di) > 0, with d+(Ai,Di)
inf
t>O,u(.),xEDi
E (x,u(.)) E Ai
° where Et(x, u(.)) denotes the solutions under fi with initial condition x and control u(.) in U[ot). Here, time can be used as a "distance" in light of the uniform bound on the fi; we consider t > 0 by adding that caveat that if we jump directly onto Ai, we do not make another jump until we hit it again. Presumably one must also make some transversality or continuity assumptions for well-posedness. This would allow the results to extend to many more phenomena, including those examples in [8].
References [1] PANOS J. ANTSAKLIS, JAMES A. STIVER, AND MICHAEL D. LEMMON. Hybrid system modeling
and autonomous control systems. In [14], pp. 366-392. [2] ALLEN BACK, JOHN GUCKENHEIMER, AND MARK MYERS. A dynamical simulation facility for
hybrid systems. Technical Report 92-6, Mathematical Sciences Institute. Cornell University, April 1992. Revised. Also in: [14], pp. 255-267. [3] D. D. BAINOV AND P. S. SIMEONOV. Chichester, England, 1989.
Systems with Impulse Effect. Ellis Horwood Limited,
24
[4] ALAIN BENSOUSSAN AND JACQUES-LOUIS
LIONS.
Impulse Control and Quasi-Variational In-
equalities. Gauthier-Villars, Paris, 1984. [5] PATRICK BILLINGSLEY. Convergence of ProbabilityMeasures. Wiley, New York, NY, 1968.
[6] MICHAEL S. BRANICKY. Topology of hybrid systems. In Proc. 32nd IEEE Conference on Decision and Control, pp. 2309-2314, San Antonio, TX, December 13-17, 1993. [7] MICHAEL S. BRANICKY. Universal computation and other capabilities of hybrid and continuous dynamical systems. Technical Report LIDS-P-2218, Laboratory for Information and Decision Systems. Massachusetts Institute of Technology, December 1993. Revised, June 1994. Also in: Theoretical Computer Science, to appear. [8] ROGER W. BROCKETT. Smooth multimode control systems. In Clyde Martin, ed., Berkeley-Ames Conference on Nonlinear Problems in Control and Fluid Dynamics, pp. 103-110. Math-Sci Press, Brookline, MA, 1983. [9] ROGER W. BROCKETT. Hybrid models for motion control systems. Technical Report CICS-P364, Center for Intelligent Control Systems, Massachusetts Institute of Technology, March 1993. Also in: H. L. Trentelman and J. C. Willems, eds., Essays in Control: Perspectives in the Theory and its Applications (Birkhauser, Boston, 1993) pp. 29-53. [10] 0. L. V. COSTA AND M. H. A. DAVIS. Impulse control of piecewise deterministic processes. Math. Control Signals Syst., 2:187-206, 1989. [11] M. G. CRANDALL, H. ISHII, AND P.-L. LIONS. User's guide to viscosity solutions of second order
partial differential equations. Bulletin of the American Mathematical Society, 27(1):1-67, 1992. [12] M. H. A. DAVIS. Markov Models and Optimization. Chapman and Hall, London, 1993. [13] ALEKS GOLLU AND PRAVIN P. VARAIYA.
Hybrid dynamical systems.
In Proc. 28th IEEE
Conference on Decision and Control, pp. 2708-2712, Tampa, FL, December 1989. [14] ROBERT L. GROSSMAN, ANIL NERODE, ANDERS P. RAVN, AND HANS RISCHEL, EDS. Hybrid
Systems, vol. 736 of Lecture Notes in Computer Science. Springer-Verlag, New York, 1993. [15] YU-CHI Ho, ED. Discrete Event Dynamic Systems: Analyzing Complexity and Performance in
the Modern World. IEEE Press, Piscataway, NJ, 1992. [16] JOHN G. HOCKING AND GAIL S. YOUNG. Topology. Dover, New York, 1988.
[17] FRANK MORGAN. Geometric Measure Theory. Academic Press, Inc., Boston, MA, 1988. [18] JAMES R. MUNKRES. Topology. Prentice-Hall, Englewood Cliffs, NJ, 1975. [19] ANIL NERODE AND WOLF KOHN. Models for hybrid systems: Automata, topologies, stability.
Technical Report 93-11, Mathematical Sciences Institute, Cornell University, March 1993. Revised. Also in: [14], pp. 317-356. [20] MARC H. RAIBERT Legged Robots that Balance. MIT Press, Cambridge, MA, 1986. [21] LucIo TAVERNINI. Differential automata and their discrete simulators. Theory, Methods, and Applications, 11(6):665-683, 1987.
Nonlinear Analysis,
[22] PRAVIN P. VARAIYA. Notes on Optimization. Van Nostrand Reinhold Company, New York, 1972. [23] L. C. YOUNG. Lectures on the Calculus of Variations and Optimal Control Theory. Chelsea, New York, 2nd edition, 1980. [24] JERZY ZABCZYK. Optimal control by means of switching. Studia Mathematica, 65:161-171, 1973.
25
