Discrete Event Control of an Unmanned Aircraft Mehdi Fatemi, James Millan, Jonathan Stevenson, Tina Yu, and Siu O’Young
Abstract— This paper describes the application of a limitedlookahead discrete event supervisory controller that handles the control aspects of the Unmanned Aerial Vehicle (UAV) “Sense and Avoid” (SAA) problem. The controlled UAV and the approaching uncontrolled intruding aircraft that must be avoided are treated as a hybrid system. The UAV control decision making process is discrete, while the embedded flight model dynamics of the aircraft are continuous. Simulation results for the controlled system are presented for a variety of flight scenarios.
I. INTRODUCTION Unmanned Aerial Vehicles (UAVs) are quickly becoming a cost effective tool for real-time airborne surveillance, providing services for both military and civilian applications. One of the main inhibitors to the general acceptance of UAVs for commercial applications is the danger of collisions that these vehicles pose to other aircraft and to terrestrial obstacles. UAVs primarily operate over wireless data links and could therefore risk loss of vehicular control during a communications outage between the UAV and the ground based operator. In order to prevent such accidents, the US Federal Aviation Administration (FAA) and UK Civil Aviation Authority (CAA) are likely to require that all UAVs employ onboard intelligence and control systems that provide the equivalent of “see-and-avoid” capabilities as compared to the presence of a pilot in a manned aircraft [1] [2]. In an unmanned aircraft this capability is called “sense and avoid” (SAA), since the intruding aircraft may be sensed by a variety of instruments including electro-optical/infrared (EO/IR) or radar systems for example. This paper does not deal with the sensor-related aspects of SAA, but describes the development of a supervisory controller that implements a collision avoidance strategy using hybrid modeling and discrete event supervisory techniques. A. Continuous Dynamics with Decision-Making The proposed SAA control system is an excellent example of a hybrid system: the aircraft dynamics are best modeled with continuous models, while discrete decisions must be taken in order to enforce the safety of both aircraft. This gives rise to discrete mode changes in the aircraft’s dynamics that classify it as a hybrid system. M. Fatemi and T. Yu are with the Faculty of Computer Science Memorial University of Newfoundland, St.John’s, Newfoundland, Canada
[email protected],
[email protected] J. Stevenson and S. O’Young are with the Faculty of Engineering and Applied Science Memorial University of Newfoundland, St.John’s, Newfoundland, Canada
[email protected],
[email protected] J. Millan is with the Institute for Ocean Technology, National Research Council, St.John’s, Newfoundland, Canada
[email protected]
The control task can be cast as a discrete event supervisory control problem by abstracting the continuous dynamics of the aircraft. It is difficult to synthesize an offline controller for such a system since the state space of the aircraft approaching the UAV (i.e. the intruder) is a set of continuous variables. The technique used in this paper is to compute the controller in an online fashion on a limited time horizon. The discrete-event model for the online computation is based on a discrete state abstraction of a continuous model of the flight dynamics of both aircraft. The discrete event model is assembled by computing many continuous simulations in order to predict the future state of the aircraft. Events are generated by the continuous systems in response to guards that have been placed on the continuous trajectories. This approach is described in [3], in which the movements of a pair of marine craft were controlled in this manner.
B. Related Work Hybrid models are well-suited to the concept of aircraft control and coordination. The concept of using predicted trajectories of both aircraft to form anti-collision decisions is the basis of the proposed Auto ACAS (Automated Aircraft Collision Avoidance System) [4] which is being designed for manned aircraft. Hybrid modeling was used in an experimental project in which cooperative control experiments were carried out with fixed-wing UAVs [5]. This paper studies the application where the other agents are noncooperative. In [6], hybrid models based on impulse differential inclusions were used to compute state reachability for a medium altitude long endurance (MALE) UAV approached by a manned intruder aircraft. In the case of this paper, the hybrid modeling is embodied in the discrete switching of continuous dynamics. In [7], nonlinear programming and hybrid simulation embedded in a graph search algorithm was utilized to find an optimal hybrid trajectory. The technique presented here differs in that the optimality of the control is not specified in the continuous trajectories, but in the discrete domain: optimality is expressed in the traditional DES sense of the supremal controllable sublanguage [8] of a plant with respect to some safety specification. The limited lookahead DES control concept utilized in this paper is in part inspired by [9]. Limited lookahead theory does not deal with blocking, since there is always a possibility that the controller will block. In [10], a technique was introduced to handle this situation by defining a special reserved emergency shutdown (ESD) action that always guarantees safe, nonblocking behaviour.
C. Paper Outline In section II the modeling details of this paper are discussed starting with the aircraft dynamical models. The framework that is used to capture the hybrid model and to synthesize the controller is also discussed, and the model parameters and other important quantities are defined. In section III the details of the controller design are developed, including the switched continuous model (SCM) and the discrete event specifications. Finally, in section IV a scenario involving a UAV and intruder aircraft collision is presented, the controller is applied and the results of this simulation are discussed. II. MODELING Once an intruding aircraft is detected by the sensors on the UAV, a model is formulated in which the continuous dynamics of both aircraft must be captured. The model used for both aircraft is described below. A. Aircraft Dynamics The flight dynamics of both the UAV and the potential threat aircraft are modeled using a relatively simple pointmass model in a similar manner to [6]. The overall formulation of this model is as follows: V cos (u3 ) cos (ψ ) x˙ y˙ V cos (u3 ) sin (ψ ) h˙ = V sin (u ) (1) 3 CD Sρ 2 1 V˙ − V − g sin(u ) + u 3 2m m 1 CL Sρ ψ˙ V sin (u ) 2 2m T in which the state vector is x = x y h V ψ . The state variables x and y are the horizontal displacement, and h the altitude of the aircraft in a geographic (local vertical) reference plane, while V is the true air speed (TAS) and ψ is the heading angle. There are two constants g, the acceleration due to gravity, and m, the mass of the aircraft (a reasonable assumption since an encounter between two aircraft will be of relatively short duration). The lift L and drag D terms appearing in (1) are functions of the altitude (due to the variation of air density): CD Sρ (h) CL Sρ (h) and D = 2 2 In these equations, CL and CD are the drag and lift coefficients and are assumed to be constant. Parameter S is the aircraft main wing area. The air density ρ (h) is calculated according to the ISA (International Standard Atmosphere) standard atmosphere T model. Finally, the inputs to the model are u1 u2 u3 , which are, respectively, the thrust, bank angle and flight path angle (e.g. angle of descent or climb). This model is reasonable, assuming normal (i.e. non-aerobatic) maneuvers by both aircraft, providing that the angle of attack and side-slip angles remain small throughout the encounter. Note that this model has been adopted from [6] with the simplification that the effect of wind factors in 3D space have been omitted. It is true that there will be uncertainty in L=
the aircraft positions due to the wind in an absolute sense (i.e. versus a fixed reference on the ground). However both vehicles are flying within the same air mass and hence will be subject to the same wind conditions in the small airspace volume under consideration here. Hence, the relative position of the UAV versus that of the intruder aircraft will be the same no matter what the wind conditions are. B. Switched Continuous Model The switched continuous model was proposed in [11]. It is easily instantiated as a computational model, acting as a container or wrapper for the numerous continuous simulations that will be generated in order to compute the hybrid state space of the controller. The SCM presents a discrete-event layer to the outside world; thus enabling it to encapsulate the continuous dynamics of a model while still retaining the full nonlinear continuous dynamics of the modeled system. The SCM is a container for many different continuous dynamical models. Consider a continuous system model (CSM) s = ( f , Ψ, x0 ) where the function f is a continuous dynamical model which may be of the form x˙ = f (x, u,t), as in the aircraft dynamics of (1). Ψ is a set of partitioning functionals Ψ = {Fi (x) : Rn → R, i ∈ N, 6= ∞} where each Fi is a continuously differentiable functional. The initial condition x0 is the state of the system at some initial simulation time t0 . Consider the SCM be defined as an automaton-like triple: G = (F , Γ, s0 ) where F = {si : i = 1, 2, 3, ...} is an infinite set of CSMs and Γ is the enabled system function, embodying a selection mechanism. Let A = {α ⊂ F : 1 ≤ |α | < ∞} be the set of non-empty finite subsets of F , and Γ : F → A . s0 is the initial continuous system model, which in the example of this paper would correspond to the mode the UAV is operating in when the intruder is first sensed. Other modes of operation, for example turning, diving, climbing etc. are embodied by additional continuous system models which differ either in parameters, or state for example. Thus, a SCM represents a set of dynamical models that can be switched amongst discretely. C. Synchronization With DES The SCM must be able to be synchronized with discrete event models. The abstraction layer of the SCM has both input and output discrete events. Continuous trajectories generate events whenever they cross some partition of the system. Thus, an event σout may be associated with each functional, and a set of events Σout exists for a collection of partitions. The events are termed output events because they are seen by the discrete event world as being generated by the continuous models spontaneously. These output events are the technique that is used to provide synchronization with external discrete event models. Additionally, a unique discrete event σin is associated with each of the continuous
TABLE I
TABLE II
M ODEL S IMULATION PARAMETERS
UAV C ONTROL A CTIONS
Mass, m Wing Surface Area, S Drag Coefficient CD Lift Coefficient CL Nominal Thrust, u1
UAV 11 0.55 0.0512 0.530 12
Intruder 5000 28.2 0.034 0.427 4000
Units kg m2 N
dynamics s ∈ F , and can be treated as the ‘guard’ or initiating mechanism to cause a particular continuous system model to be executed. The set of input events Σin are the mechanism for the discrete event world to communicate (or generate input) to the continuous simulations embedded within the SCM. D. Dealing With Uncertainty For the purposes of this paper, it is assumed that there exists a sensor package on the UAV that is capable of identifying only the position of the intruding aircraft in T three-space, x y h , thus, its heading and speed are also unknown (at least initially). It will also be assumed that the aircraft type is unknown, thus all of the model parameters are also unknown. We propose to deal with the uncertainty by assuming reasonable worst-case conditions for the intruder: i.e. a fast, highly-maneuverable aircraft, which in this case we have chosen as the Beechcraft King Air. In addition to accounting for parametric and state unknowns, it is also necessary to account for uncontrollable actions that influence the aircraft’s flight path: that is, the evasive actions taken by the pilot. For maximum safety, if it is assumed that at every moment the intruder is attempting to collide with the UAV, then there can be no worse scenario. Thus, in the simulations that are conducted to construct the online controller, the intruder is modeled with a closed-loop control system that tracks and attempts to collide with the UAV. This antagonistic stance ensures that if the controller is safe for this scenario (the UAV is able to evade the intruder), then it will be safe for any other. Additional safety can also be built into the problem by increasing the allowable separation between aircraft. In this paper, for control synthesis and simulation, an Aerosonde aircraft is used for the UAV. The specific model parameters used for the simulations of both aircraft are given in Table I. III. CONTROL DESIGN This section gives the details of the modeling of the SCM and of the discrete event specification from which the online supervisor is created.
σin rHi rLo str lLo lHi sdsd+
u1 12.0 12.0 12.0 12.0 12.0 12.0 12.0
u2 -30.0 -15.0 0 15 30 45 -45
u3 0 0 0 0 0 -5 -5
Description turn right hard turn right medium fly straight turn left medium turn left hard hard turn left and dive hard turn right and dive
actions. Without loss of generality, we concentrate mostly on u2 , the bank angle, due to the fact that it plays a more important role in flight control, although we use path angle as well for emergency shutdown. Adding additional controller inputs increases the computational complexity of the controller synthesis, and of course should be considered for a real control design. Table II lists the control actions (input events) available to the system. The set of input events is thus Σin = {rHi,rLo,str,lLo,lHi,sd+,sd-} For example, rHi indicates maximum angle of right turn, and rLo is used for the mid-level right turn, which is also a reasonable control action. There are two events sd+ and sd-, which are reserved control events designed to be used by the controller in the event of emergency (i.e. ESD). In this case, the UAV will be banked very steeply and caused to dive. These are designed to be used by the controller only in the event of a potential block being reached. In fact, this move is considered to be risky to the UAV’s survival. Ultimately though, the concern is clearly with the survival of the intruder: that is, the UAV should always assume that the intruder is manned, and as such, should sacrifice itself to ensure that there is no collision. B. State Space Partitioning The partitioning of the continuous state space should be designed with the discrete event specifications in mind. Ultimately, the output events (associated with the boundaries of the continuous regions) are the symbols that make up the SCM’s language Lscm ⊆ Σ∗out . Table III gives the partitioning functionals. For this controller design, the continuous dynamics of both aircraft are lumped into a single simulation in which the state vector includes the individual state vectors of both aircraft, thus the partitioning functionals may be expressed in terms of the state vectors of both aircraft. In this table x ∈ R10 is defined as the combined state vector of both aircraft. The relative displacement between the aircraft r and their relative angle θ in the horizontal plane are defined as
A. Control Actions As indicated in section II, there are three control inputs available in the UAV aircraft model: thrust (u1 ), bank angle (u2 ), and path angle (u3 ). In fact, each of these variables can be considered as an abstraction for actual aircraft control
r=
p (xuav − xint )2 + (yuav − yint )2 + (huav − hint )2 (yuav −yint ) θ = tan−1 (x uav −xint )
TABLE III
σout alarm ay+ ayesd tick
Functional F1 (x,t) = r − 500 F2 (x,t) = r − 900 F3 (x,t) = r − 1000 F4 (x,t) = h − 1000 F5 (x,t) = sin(2π t/∆t)
Direction ↓ ↓ ↑ ↓ l
Condition violated min. separation entering warning zone exiting warning zone reaches relative altitude controller update
h (m)
C ONTINUOUS S TATE PARTITIONS
2600 2400 Init. 2200 1500 1200 900
1200
600
800
300 [q1] ay+
esd
0 −300
[q2]
y (m)
Init.
400 0
−400
[q2]
x (m)
tick
ay-
[q3]
ay[q1]
Fig. 2.
Simulated aircraft trajectories.
tick
ay[q4] tick ay-
90 [q5]
ay-
Fig. 1.
tick
2000 m
120
[q6]
60 Init.
1500 m
Finite state machine specifications 1000 m
150
The time base of the controller is set to ∆t = 5 seconds, corresponding to the tick event. The direction column refers to the direction of the trajectory crossing a partition.
30
500 m
180
0
C. Discrete Event Specification The specification for this control system is very simple: avoid collision (or close approach) with the intruding aircraft. Thus, the specification is simply given by the lower finite state machine (FSM) in Fig. 1. The initial state is indicated by the hexagonal node, and state marking is indicated by the shading. The event set of this FSM is Σout = {tick,ay+,ay-,alarm}. By placing the alarm event in the event set, this inhibits the event from being generated in the product. The second specification (top) ensures that shutdown is permitted at any time, taking the system to a marked state. The controller is formed by taking the synchronous product of the FSM specifications along with the SCM. At any time step, the initial state for the continuous simulation model, derived from sensors, can be loaded into the SCM and the legal discrete event behaviour for the limited horizon is computed by exhaustively evaluating the safe reachable states. The result of the reachability at any given time step is a control decision that must be made from the set of available legal ones. IV. SIMULATION A. Computation The controller synthesis and simulation of the aircraft are both modeled in MATLAB. The system modeling and control synthesis functions have been implemented using an experimental software package called H Y S YNTH. H Y S YNTH permits the modeling of continuous and discrete objects and implements methods that enable the user to produce limited lookahead discrete event supervisors. Typically, the
210
330
240
300 270
Fig. 3.
Relative distance r and heading theta.
specifications are modeled in finite state machines, while the plant continuous dynamics are modeled in a switched continuous model object. B. Scenario Based on the generic dynamic of section II, the flight trajectories for both aircraft can be simulated using MATLAB ODE solvers. Initially, u1 , u2 , and u3 are set to be fixed for both aircraft during the simulation so that the actual flight paths can be observed without any control systems. The 3dimensional flight trajectories for the UAV (blue stars), and the intruder aircraft (red stars) are shown in Fig. 2. Dashed lines show relative distance r between the two planes. In this scenario, the UAV is loitering, perhaps circling a target, and the intruder enters the nearby airspace climbing and moving towards the UAV. While they do not collide, the approach is too close, violating the 500 m circle. Basically, the planes cannot be closer to each other than this limitation distance due to the potential for collision. In Fig. 3, the relative
TABLE IV
V. CONCLUSIONS
C ONTROL S EQUENCE Time t0 t1 t2 t3 t4 t5
σin Graph Size† rHi 115 rHi 44 lHi 184 lHi 217 str 217 str 217 † in transitions
Priority First Second First First First First
distance between the aircraft is plotted versus the relative angle θ in the horizontal plane (the compass heading) and also the two action distances are plotted as thick red circles. C. Results Fig. 4 illustrates the near-miss scenario of the previous section, but this time with the online controller working. This figure represents the time history of the actual aircraft flight paths for 25 seconds, along with the hypothetical reachable trajectories of both aircraft as computed by the controller. The thick black trace in the upper part of the figure is the trajectory of the intruder aircraft during the entire simulation (traveling from left to right). The aircraft does not see the UAV and is simply continuing along its way, climbing slightly while also turning slightly to the left. The UAV’s trajectory is also indicated with a thick black line in the lower part of the figure. The dark blue traces are the possible UAV trajectories at each time step, projected 3 steps into the future. The red traces are the hypothetical antagonistic trajectories that could be taken by the intruder, also projected 3 steps into the future. The green circles represent points at which events are detected and where the system control is exercised. Recall that the controller is presumed to have only the position of the intruder and no other information. For clarity, the emergency shutdown traces have all been eliminated from the figure, but they must always be present to ensure nonblocking safety of the controller. Table IV gives the sequence of control actions (σin ) taken by the UAV controller based on the computation of the safe reachable states at each of the time steps. The graph size indicated in the table is a measure of how many traces have been eliminated during the reachability calculation. Initially at time step t0 , many of the future intruder trajectories will lead to a collision. In the next step the outlook is more bleak as the graph of safe reachable states is even smaller. However, once the intruder aircraft has passed, the graph increases in size and any of the control actions becomes safe. As a final note, the ‘Priority’ column in the table refers to the choice of control action to be taken from the available set of safe actions: those actions that lead to nonblocking states and are ESD state reachable are preferable to ones that lead only to shutdown. Thus, the former are first priority while the latter are second priority.
AND
FUTURE WORK
This example has demonstrated the application of discrete event control to the development of a sense and avoid system for autonomous aircraft. The limited lookahead technique is particularly well-suited to this problem since relative approach speeds of two aircraft limit the control problem to a relatively short time horizon. It is computationally feasible to compute the entire useful state space for this small time window, so this is the ideal application of limited lookahead discrete event control. Other areas of autonomous aircraft control might also be amenable to this technique, particularly planning and coordination tasks. This paper has described a preliminary study that was conducted as a stand alone simulation only. One future goal will be to conduct further real-time or near-real time studies using a synthetic environment. This will provide a more realistic flight simulation for both aircraft including disturbances, enabling the project team to evaluate the robustness of this control technique. Eventually, the control technique may be tested on real UAV aircraft in an unmanned trial for safety reasons. The project team has has three Mk 4.2 Aerosonde UAVs to conduct such work. In this example, the control choice made by the UAV was made by a human in the loop. Typically, this would not be possible since the aircraft might be flying completely autonomously. More work is needed in this area to automate the control choice. VI. ACKNOWLEDGMENTS This work was carried out for the RAVEN (Remote Aerial Vehicles for Environmental monitoring) research program of Memorial University of Newfoundland (MUN). The project is jointly supported by the Atlantic Canada Opportunities Agency (ACOA), the National Research Council of Canada (NRC), Institute for Ocean Technology (IOT) and the Institute for Aerospace Research (IAR), Provincial Airlines Limited (PAL), the National Science and Engineering Research Council (NSERC) and Defence Research and Development Canada (DRDC). R EFERENCES [1] Order 7610.4K special military operations, US Department of Transport, Federal Aviation Administration, February 2004. [2] Unmanned Aerial Vehicle operations in UK Airspace - Guidance, Civilian Airspace Authority, UK Directorate of Airspace Policy, London, UK, November 2004. [3] J. P. Millan and S. D. O’Young, “Discrete event approach to hybrid system control using embedded simulation,” IFAC Journal of Control Engineering Practice, 2008, article in press. [4] B. Sundqvist, “Auto-ACAS - robust nuisance-free collision avoidance,” in Proceedings of the 44th IEEE Conference on Decision and Control. IEEE, December 2005, pp. 3961–3963. [5] S. Bayraktar, G. Fainekos, and G. Pappas, “Hybrid modeling and experimental cooperative control of unmanned aerial vehicles,” Department of Computer and Information Science, University of Pennsylvania, Technical Report MS-CIS-04-32, December 2004. [6] E. Cruck and J. Lygeros, “Sense and avoid system for a MALE UAV,” in Proceedings of Guidance, Navigation and Control Conference and Exhibit. American Institute of Aeronautics and Astronautics, August 2007.
t5
Altitude (m)
2600 2500
t0
2400
t2
t1
t4
t3
2300 2200 1400 1200 UAV Start
1000 800 600
200 400
0 200
−200 0
−400 −200
y(m)
−600 −400
−800 x(m)
Fig. 4.
UAV and intruder trajectories (black) along with the hypothetical trajectories generated by the controller (intruder in red, UAV in blue).
[7] O. Stursberg, “A graph search algorithm for optimal control of hybrid systems,” in Proceedings of the 43rd IEEE Conference on Decision and Control, December 2004, pp. 1412–1417. [8] C. G. Cassandras and S. Lafortune, Introduction to Discrete Event Systems. Kluwer Academic Publishers, 1999. [9] S. Chung, S. Lafortune, and F. Lin, “Limited lookahead policies in supervisory control of discrete event systems,” IEEE Transactions on
Automatic Control, vol. 37, no. 12, pp. 1921–1935, December 1992. [10] J. P. Millan and S. D. O’Young, “On-line supervisory control of hybrid systems using embedded simulations,” in Proceedings of the IEEE 8th International Workshop on Discrete Event Systems WODES06, July 2006. [11] J. P. Millan, “Online discrete event control of hybrid systems,” Ph.D. dissertation, Memorial University of Newfoundland, October 2006.
