Hierarchical Control of Piecewise Linear Hybrid Dynamical Systems Based on Discrete Abstractions Panos J. Antsaklis Department of Electrical Engineering University of Notre Dame Notre Dame, IN 46556, USA Tel. +1-219-631-5792 Fax +1-219-631-4393
[email protected]
Xenofon D. Koutsoukos Xerox Palo Alto Research Center 3333 Coyote Hill Road Palo Alto, CA 94304, USA Tel. +1-650-812-4385 Fax +1-650-812-4334
[email protected]
Abstract In this paper, a novel methodology for analysis of piecewise linear hybrid systems based on discrete abstractions of the continuous dynamics is presented. An important characteristic of the approach is that the available control inputs are taken into consideration in order to simplify the continuous dynamics. Control specifications such as safety and reachability specifications are formulated in terms of partitions of the state space of the system. The approach provides a convenient general framework not only for analysis, but also for controller synthesis of hybrid systems. The research contributions of this paper impact the areas of analysis, verification, and synthesis of piecewise linear hybrid systems.
1 Introduction A great amount of research work has already been done in the hybrid systems area during the past decade; see for example [3] and the references there in. A survey of different models and methodologies can be found in [4]. The approach presented in this paper is directly related to supervisory control framework for hybrid systems [38, 39, 20]. Similar approaches based on approximations of the continuous dynamics by a discrete event system have also been proposed in [31, 33, 12, 24]. The hybrid system model typically used in the supervisory control framework consists of a plant described by nonlinear differential or difference equations, a discrete event controller described by a deterministic finite automaton, and an interface which provides the means for the communication between the plant and the controller. In the model proposed in the present work, we restrict ourselves to linear dynamics evolving in discrete-time. However, we consider a plant that may also contain discrete dynamics. More importantly, we consider a larger class of control inputs which may contain both discrete and continuous inputs. In addition, we take into consideration both discrete and continuous disturbances. In this paper, new methodologies to solve important control problems in hybrid systems are presented. Our work is motivated by the need to address challenging problems in the control and coordination of modern complex engineering The partial financial support of the National Science Foundation (ECS99-12458) and the Army Research Office (DAAG55-98-1-0199) is grate-
fully acknowledged.
1
applications such as autonomous vehicles, chemical and manufacturing plants, and multiple robotic systems. Hybrid systems are modeled as discrete-time dynamical systems. A mathematical model that can capture both discrete and continuous phenomena is formulated. The continuous dynamics are described by linear difference equations and the discrete dynamics by finite automata. The interaction between the continuous and discrete parts is defined by piecewise linear maps characterized by sets of linear equalities and inequalities. We refer to this class of systems as piecewise linear hybrid dynamical systems in order to emphasize the hybrid nature of the systems and problems of interest. Piecewise linear hybrid dynamical systems is an important class of systems with many practical applications. The introduced model is general enough to describe important engineering applications, but simple enough to facilitate the development of analysis, and more importantly, synthesis tools. Piecewise linear hybrid dynamical systems have an efficient representation for modeling and simulation. Furthermore, current modeling tools such as MATLAB ,S IMULINK, and S TATEFLOW offer the necessary flexibility for modeling and simulation of this class of systems. A systematic methodology for analysis of piecewise linear hybrid systems based on discrete abstractions of the continuous dynamics is presented. Analysis and synthesis methodologies based on discrete abstractions have been studied extensively in the hybrid system literature; see for example [2, 20] and the references there in. In order to analyze hybrid systems and design control algorithms, it is desirable to induce dynamical systems in finite quotient spaces that preserve the properties of interest and then study the simplified models. In this paper, we propose a new methodology for the construction of discrete abstractions of the continuous dynamics. An important characteristic of the approach is that the available control inputs are taken into consideration in order to simplify the system. The main mathematical tool to be used is the predecessor operator applied recursively to subsets of the hybrid state space. The application of the predecessor operator corresponds to partition refinement into finer partitions that allow the formulation of conditions that guarantee the existence of appropriate controls for the objectives of interest. Typical control specifications investigated in this paper are formulated in terms of partitions of the state space of the system. Examples include safety problems, where the controller guarantees that the plant will not enter an unsafe region for example guaranteeing that two interacting robots will not collide. Also reachability problems where the controller drives the plant from an initial operating region or state to a desired one; this is the case for example in the startup procedure of a chemical plant. In order to study safety specifications for piecewise hybrid dynamical systems, we introduce the notion of quasideterminism. Quasideterminism represents the case when the future behavior only for the next time interval of the given system can be uniquely determined by the current state of the induced system. We show that this property can be used to formulate conditions for safety specifications for piecewise linear hybrid dynamical systems. The safety conditions can be tested using efficient linear programming techniques. We also present an algorithm for the computation of the maximal safe set based on the approach in [41, 25]. Reachability conditions are also formulated. Our approach is based on conditions that guarantee that the state can be forced to reach a desirable region of the state space by selecting appropriate controls. It should be emphasized that we are interested only in the case when reachability between two regions is defined so that the state is driven to the target region without entering a third region.This is a problem of great practical importance in hybrid systems since it is often desirable to drive the state to a target region of the state space while satisfying constraints on the state and input during the operation of the system. Piecewise linear systems arise very often as mathematical models for practical applications. For example, piecewise linear systems can be used to model systems with discontinuous dynamics that arise because of saturation constraints, hysteresis, friction in mechanical systems and so on. For another example, in order to avoid dealing directly with a set of nonlinear differential equations one may choose to work with linear equations and switch among these simpler models. Furthermore, piecewise linear systems arise in the switching control paradigm [27, 28] where the behavior of the plant is
2
controlled by switching between different controllers for each region of the state space. It should be noted that the class of piecewise linear systems has been studied extensively in the circuit theory community; see for example [22] and the references therein. Here, we are interested in approaches that have been developed for modeling, analysis, and synthesis of hybrid control systems. The first investigations of piecewise linear hybrid systems can be found in [35, 36, 37]. The main problems studied in this framework are stability, controllability, and input-output regulation. Piecewise linear dynamical systems have been considered also in [10, 5, 6]. A methodology for approximating the reachable states is developed and a supervisory control framework is used for controller design. A class of hybrid systems which is similar to piecewise linear hybrid systems is considered in [7, 8, 9]. These systems are described by linear dynamic equations subject to linear inequalities involving real and integer variables. Finally, piecewise linear systems were also studied in [15] to develop computational algorithms for the analysis of nonlinear and uncertain dynamical systems. The hybrid system model used in this paper can be viewed as a input-output hybrid automaton evolving in discretetime. Hybrid automata provide a general modeling formalism for the formal specification and algorithmic analysis of hybrid systems [1]. Formalisms for input/output hybrid automata have been also proposed in [26, 41, 23]. A related approach to the work presented in this paper is based on the modeling formalism of hybrid automata and uses bisimulations to study the decidability of verification algorithms [14, 21, 2]. Bisimulations are quotient systems that preserve the reachability properties of the original hybrid system and therefore, problems related to the reachability of the original system can be solved by studying the quotient system. The idea of using finite bisimulations for the analysis and synthesis of hybrid systems is similar to the approximation of the continuous dynamics with discrete event systems. The main contributions of the paper are the following. An algebraic system theoretical framework is developed for the analysis, verification, and synthesis of piecewise linear hybrid dynamical systems. This framework enables us to develop a novel methodology for analysis of piecewise linear hybrid systems based on discrete abstractions of the continuous dynamics. Our approach is based on systematic methodology for refinement of the state space partition. The main characteristic of the approach is that the available control inputs are taken into consideration in order to simplify the continuous dynamics. Algorithms for reachability analysis of discrete-time piecewise linear hybrid systems are presented in detail. It should be noted that these algorithms can be applied in the general case when the discrete dynamics contain controllable and uncontrollable events and the continuous dynamics contain control inputs and disturbances. The research contributions of this work impact the areas of reachability analysis, verification, and synthesis of piecewise linear hybrid systems. Note that the main results of this paper have appeared in [16]; early results have been reported in [18, 17, 19]. This paper is organized as follows. In Section 2, we present the modeling framework for discrete-time hybrid dynamical systems. Section 3 contains the necessary mathematical preliminaries that are used to formally define piecewise linear hybrid dynamical systems. Our mathematical model is presented in Section 4 and is illustrated using a temperature control system. In Section 5, we use an algebraic system theory framework to describe our motivation for using discrete abstractions for the analysis of hybrid systems. We also describe the proposed hierarchical control architecture using an algebraic system theory framework and we formalize the conditions under which such a hierarchical scheme can be used for control design of hybrid systems. In Section 6, we present a methodology for backward reachability analysis of piecewise linear hybrid systems. First, we formally define the notion of partition refinement by characterizing the set of polyhedral partitions as a lattice. Then, we define the predecessor operator for PLHDS, and we present computer algorithms for backward reachability analysis based on the predecessor operator. In Section 7, we study the safety problem for piecewise linear hybrid systems. We define the notion of quasideterminism and how it can be used to formulate safety conditions. We also describe an algorithm based on linear programming techniques for testing the safety conditions, and we illustrate the approach using the temperature control system. In addition, we
3
present an algorithm for the computation of the maximal safe set of piecewise linear hybrid dynamical systems. In Section 8, we study the reachability problem and we formulate conditions that guarantee reachability between piecewise linear regions. We also present an approximation technique for the computation of the coreachable set of a piecewise linear region based on the quantization of the state space. Finally, concluding remarks are presented in Section 9.
2 Discrete-time Hybrid Dynamical Systems Hybrid systems are modeled by the discrete-time dynamical system
( + 1) = f (q(t); x(t); u(t)) q (t + 1) = Æ (q (t); x(t); (t)) y (t) = g (q (t); x(t))
x t
(1) (2) (3)
where
2 f0; 1; 2; : : :g < is the time index, x 2 X < = 7 6 7 prec;q0 (P ) = x 2 : > > > : 0:8259 0:1354 0:0387 ; 1
2
3
(79)
= 0. Using Proposition 4.3, we consider the following set of
s.t. for i
(78)
4
1
(82)
2
Next, we compute the set
( ) = fxjA x + B u + E d 2 P g:
prec;q1 P
1
1
1
(83)
We consider the following set of linear inequalities: GA1 x
+ GB u 1
u u
27
w
GE1 d
1 0:5
(84) (85) (86)
We apply the Fourier-Motzkin elimination method in order to eliminate the control variable u. We also solve the linear programming problems
min
d
s.t. for i
giT E1 d
(87)
2D
1
= 1; 2; 3; 4 and we obtain [d; d ; d; d] = [1; 1; 0; 0]. Using Proposition 3 we have that 8 2 3 2 39 > 0 :6634 0 :1997 19 :4580 > > 66 0:1997 0:2641 77 " x # 66 4:3953 77> < = 7 6 7 prec;q1 (P ) = x 2 < j 6 64 0:1997 0:2641 75 x 64 0:6847 75> : > > > : 0:6634 0:1997 4:0507 ; 1
2
3
4
1
2
(88)
2
( )
( )
The sets prec;q0 P and prec;q1 P are shown in Figure 12. 30
20
10
q
P 0
q
0
1
−10
−20
−30 −10
−5
0
5
10
15
20
25
30
35
40
Figure 12: Final partition for the temperature control system.
( )
(
The set pre R is computed using the algorithm presented in Subsection 6.3. Since both discrete transitions q0 ; q1
(
)
)
and q1 ; q0 are feasible we get
( ) = f(q ; q ); prec;q0 (P ) [ prec;q1 (P )g:
pre R
0
(89)
1
In the following, we illustrate how we can test the safety condition R sented by the logical formula
pre(R). The set pre(R)jX can be repre-
( ^ ^ ^ ) _ ( ^ ^ ^ ) 01
02
03
04
11
12
13
(90)
14
( )
( )
where the atomic formulas ij correspond to the linear inequalities that define the sets prec;q0 P and prec;q1 P in pre RjX c . Using DeMorgan’s laws, the set Q can be Equations (82) and (88) respectively. We define the set Q
=[ (
)]
represented by
:[( ^ ^ ^ ) _ ( ^ ^ ^ )] , :( ^ ^ ^ ) ^ :( ^ ^ ^ ) , (: _ : _ : _ : ) ^ (: _ : _ : _ : ) [ , (: i ^ : j ) 01
01 01
02
02
03
04
03
02
i=1;2;3;4; j =1;2;3;4
11
04
11
03
04
0
1
28
12
12
11
13
13
12
14
14
13
14
= Si;j Qij . Each set Qij is described by the logical formula (: i ^ : j ) and therefore, it is polyhedral. The condition RjX pre(R)jX can be checked by testing the feasibility of the linear Therefore, the set Q can be written as Q
0
programming problems:
min
x
1
2 P \ Qij For the temperature control system, we have that RjQ = pre(R)jQ = fq ; q g and P \ Qij = ; for i = 1; 2; 3; 4 and j = 1; 2; 3; 4. Therefore, the region R is safe as it can be seen in Figure 12. 2
(91)
x
s.t.
0
1
7.3 Maximal Safe Set In Subsection 7.2, we formulated conditions that guarantee that a given region R is safe. In the case when the safety
conditions are not satisfied and R is not safe, it is possible that there exists a region R0
R which is safe.
Such a
region can be computed as the maximal safe set contained in R. The problem of computing the maximal safe set for hybrid systems has been studied in [41, 25, 44]. Here, we present how the algorithm presented in [41] can be applied to PLHDS. Algorithm for the computation of maximal safe set
= ;
INPUT: R0 X R1 while Rk 6 Rk 1 Rk pre Rk 1 k = k+1;
= = (
end OUTPUT: R
= R; k = 1; ) \ Rk ; 1
= Rk
The maximal safe set is computed as a fixed point of the iterative procedure described above. At the k th iteration of the algorithm, we compute the set Rk pre Rk 1 \ Rk 1 which contains all the states in Rk for which there exist controls so that the state will remain in Rk . If there exists a fixed point iteration, then clearly the corresponding set R is safe. Furthermore, we have that Rk Rk 1 and therefore the set R is the maximal safe set contained in R.
= (
)
The algorithm involves the computation of the predecessor operator at every iteration. In [41, 25], this computation is accomplished by solving a Hamilton-Jacobi-Bellman equation derived from a game theoretical formulation of the problem. In the case of PLHDS, the predecessor operator can be computed using the algorithms for elimination of quantifiers presented in Section 5. The proposed procedure is semi-decidable. If the algorithm terminates, it provides the maximal safe piecewise linear set contained in R, however its termination is not guaranteed. The advantage of computing the maximal safe set of a PLHDS, is that based on this set, a controller can be designed which is maximally permissive. Such a controller is optimal in a sense, since it does not restrict the behavior of the plant in a conservative way. However, the algorithm for computing the maximal safe set is not computationally efficient. For PLHDS, the number of linear constraints increases exponentially at each iteration of the algorithm. On the other hand, the safety conditions presented in Subsection 7.2 do not guarantee that the corresponding controller will be maximally restrictive. However, they provide constructive conditions that guarantee that a given region is safe and they can be used to determine what are the appropriate control inputs that guarantee safety. A class of discrete-time systems for which this procedure is decidable has been presented in [42].
29
8 Reachability In this section, we study the reachability problem for piecewise linear hybrid dynamical systems. We present a reachability algorithm based on the successive computation of the predecessor operator. In general, the proposed procedure is semi-decidable and its termination is not guaranteed. In order to formulate a constructive algorithm for reachability, we consider two approaches. First, we consider an upper bound on the time horizon and we examine the reachability only for the predetermined finite horizon. Second, we formulate a termination condition for the reachability algorithm based on a grid-based approximation of the piecewise linear regions of the state space. It should be emphasized that we are interested only in the case when reachability between two regions R1 and R2 is defined so that the state is driven to R2 directly from the region R1 without entering a third region. This is a problem of practical importance in hybrid systems since it is often desirable to drive the state to a target region of the state space while satisfying constraints on the state and input during the operation of the system. Consider, for example, an unmanned underwater vehicle with control policies that allow various combinations of screw speeds (on and off), stern plane positions (up, level, down), and rudder positions (left, right, straight). A control goal for such a system can be described by a target region of the state space which represents a desirable set of displacements and velocities for the vehicle. However, while the system is driven to the target regions the dispacements and velocities must be approprietly constrained to guarantee safe operation.
( )2
Definition 5 Given two regions R1 ; R2 Q X , we say that R2 is directly reachable from R1 , if every state q; x R1 can be driven in R2 in finite time without entering a third region.
The problem of deciding if a region R2 is directly reachable from R1 can be solved by recursively computing all
=
the states that can be driven to R2 from R1 using the predecessor operator. We only consider regions of the form R1 Q1 ; P1 and R2 Q2 ; P2 for which P1 and P2 are adjacent polyhedral regions of the primary partition. In this case,
(
)
=(
)
(
1)
( )=
-dimensional hyperplane h x the regions P1 and P2 have a common boundary which is represented by a n g T x w. The reachability problem between any two regions can be solved by finding a path consisting of adjacent reachable regions. Note that if the regions R1
= (q ; P ) and R = (q ; P ) have identical continuous parts, then the 1
2
2
reachability problem can be solved by considering the set of feasible transitions for the polyhedral region P .
8.1 Finite Time Horizon
( )
Consider the regions R1 and R2 and the initial state q; x 2 R1 and assume that we can disable the state from crossing all the boundaries of R1 but h x . It is still possible that the hybrid system will be blocked in the sense that the state will
()
()
never exit the region R1 through the hyperplane h x . Note that this can happen since we want to drive the state from R1 to R2 without entering a third region. In this case there is a trade-off between driving the state into the target region and satisfy the constraints for the state trajectory. The risk of violating the operational conditions of a system while stirring the state to a desired operating point must be addressed. Thus, this formulation of the reachability problem that takes into consideration constraints in the state trajectory is more important than considering only the state into the target region, both in theory and in practice.
()
Our approach is based on conditions that guarantee that state can be forced to cross the hyperplane h x in finite time by selecting appropriate controls. For this purpose, we consider a finite time horizon defined by N T where T is
N
the sampling period and N 2 . Consider a PLHDS described by the equations (13)-(15) and assume that the initial condition is q t0 ; x t0 2 R1 .
( ( ) ( ))
30
( ( ) ( )) 2 R there exist for t t < t + kt and (q (t +
Definition 6 The region R2 is directly N -reachable from R1 if for every initial state q t0 ; x t0 control inputs for the PLHDS and k kt ; x t0 kt 2 R2 ..
) ( + ))
2 N; 0 < k N so that (q(t); x(t)) 2 R
1
( )
0
0
1
0
0
N R of all states that can be driven from the region R to R in the finite time We define the coreachable set CRR 2 1 2 1 Q X Q X t N T without entering a third region. The predecessor operator pre ! can be used to compute the N R using the following algorithm. set CRR 2 1 N R Algorithm for the computation of CRR 2 1
( )
:2
2
( )
=R ; N ( R ) = ;; CRR 1 k = 0; while :(Rk Rk ) AND k N Rk = pre(Rk ) \ R ; N (R ) = CRN (R ) [ Rk ; CRR R1 1 k = k + 1; R0
2
2
+1
+1
1
2
2
+1
end Given the regions R1 and R2 , we compute all the states that can driven from R1 to R2 . Note that at every iteration k of the algorithm we consider the intersection of the set pre Rk with the set R since we are interested only in states
( )
1
that can be driven to R2 directly from the region R1 without entering a third region. At every iteration of the algorithm we have to apply the predecessor operator to a piecewise linear region of the state space. The resulting region is still piecewise linear, it can be represented using only linear equalities and inequalities, and it can be computed using the algorithms for elimination of quantifiers presented in Section 5. The above algorithm can be used to determine if the region R2 is N -reachable from R1 using the following theorem.
=(
)
=(
)
Q1 ; P1 and R2 Q2 ; P2 . Then, Theorem 3 Consider a PLHDS described by (13)-(14) and the regions R1 N R is piecewise linear and the region R is directly N -reachable from R if and only if R the set set CRR 2 2 1 1 1 N R . CRR 2 1
( )
( )
( )
N R is piecewise linear since it is computed using finite unions and intersections of piecewise Proof The set set CRR 2 1 linear sets. At the k iteration of the algorithm, the set Rk contains all the states in R1 that can be driven in R2 in t kT . N R then there exists controls so that every state q; x 2 R can be driven to R in t N T without If R1 CRR 2 1 2 1 2 entering a third region.
( )
( )
( )
N R is piecewise linear, the reachability problem between R and R can be Furthermore, since the set CRR 2 1 2 1 solved using linear programming techniques similarly to the safety conditions (see Subsection 7.2).
For regions that are not adjacent, a feasible path connecting these regions which consists of adjacent must be established. Note that this can be done at the higher level of abstraction, since the necessary information is the existence of a control policy and not the actual policy. Example - Temperature Control System We illustrate the reachability algorithm using the temperature control system presented in Subsection 4.
31
Consider the regions R1
= (fq ; q g; P ) and R = (fq ; q g; P ) where P = fx 2 < j(0 x 20) ^ ( 20 x 0)g 0
1
1
2
2
1
and P2
0
1
2
1
(92)
2
= fx 2 < j(0 x 20) ^ (0 x 5)g: 2
1
(93)
2
It is desirable that every state from R1 can be driven to R2 without entering a third region. Such a specification may arise, for example, at the startup procedure of the system, where it is required for the state of the system to reach the safe region x2 without entering the unsafe region x1 > .
0
5
20
In order to compute the set of states in the region R1 that can be driven to R2 using appropriate control inputs, we apply the reachability algorithm presented in Subsection 8.1. The coreachable set of states for three iterations of the algorithm is shown in Figure 13. 30
30
k=1
k=0 20
20
10
10
0
0
−10
−10
−20
−20
−30 −30
−20
−10
0
10
20
−30 −30
30
30
−20
−10
0
10
30
k=2
k=3
20
20
10
10
0
0
−10
−10
−20
−20
−30 −30
30 30
20
−20
−10
0
10
20
−30 −30
30
−20
−10
0
10
20
30
Figure 13: Coreachable set for the temperature control system.
=3
T . Therefore, there exists control policy which selects the The region R2 is directly reachable from R1 in t control input u 2 U and possibly forces appropriate discrete transitions so that every state q; x 2 R1 can be driven
to the region R2 .
( )
2
8.2 Grid-Based Approximation In this section, we formulate an approximation-based methodology in order to guarantee that the algorithm for the successive computation of the predecessor operator will terminate. The reachability algorithm based on the successive computation of the predecessor operator is semi-decidable and therefore, its termination is not guaranteed. In the following, we present such an example.
( + 1) = Ax(t) with # 1 :1036 0 :0315 A= 0:1051 0:9984 :
Example Consider the discrete-time linear system x t
"
32
(94)
Suppose we are given the partition shown in Figure 14 described by the following hyperplanes:
() h (x) h (x) h (x) h1 x 2 3 4
= = = =
x1 x1 x2 x2
+3 +2 1 +1
(95) (96) (97) (98)
1.5
1
0.5
P1
P2 0
−0.5
−1
−1.5 −3
−2
−1
0
1
2
3
Figure 14: Primary partition for the system. We consider the regions
= fx 2
