Distributed Simulation of Forward Reachable Set-Based Control for Multiple Pursuer UAVs Chern Ferng CHUNG1, Ali Haydar GÖKTOĞAN2, Kaijiun CHEANG3 and Tomonari FURUKAWA4 ARC Centre of Excellence for Autonomous Systems School of Mechanical and Manufacturing Engineering, The University of New South Wales1,3,4 {c.chung1,k.cheang3}@student.unsw.edu.au,
[email protected] Australian Centre for Field Robotics, University of Sydney, NSW Australia2
[email protected] Abstract. A geometrical measure of a vehicle’s dynamics, called the Forward Reachable Set (FRS), serves as a useful tool in a Pursuit-Evasion (PE) game. By incorporating the FRS into the pursuer’s control strategy, it can determine if it can capture an evader of higher manoeuvrability as well as serve as a guidance mechanism for the pursuer’s trajectory. However, as most current applications of FRS-based pursuer control mainly deals with a single evader and centralized simulation, a distributed architecture for an existing FRS-based pursuit strategy for multiple pursuer Unmanned Aerial Vehicles (UAVs), is proposed, with an evader allocation extension to capture multiple evader UAVs. The advantages of such distribution are the relative feasibility and robustness. As the load of solving the central pursuer objective is divided amongst the pursuers, each of their individual control actions can be calculated faster. Also, performance degradation in one UAV will not have as major an adverse effect on the entire system, as compared to a centralized one. The distributed architecture allows for easier integration and implementation since most real UAVS systems and advanced simulation platforms are distributed. This paper presents the distributed FRS-based pursuer control strategy and architecture along its integration and simulation on the Real-time Multi UAV Simulator (RMUS). 1. INTRODUCTION In a Pursuit-Evasion (PE) scenario, it was shown that the relative dynamic capabilities between the pursuer and evader affect the outcome of the game [1],[2]. If the evader is more agile laterally than the pursuer, whilst only slightly slower in terms of maximum forward acceleration, there is a high chance that the evader can escape from the pursuer. A study by Lawrence [3] reveals information related to the dynamics of a vehicle in the game, called the Forward Reachable Set (FRS), played an important role in the trajectory control of the pursuers to maximise their chances of capturing such an evader. It states a key result in a single-pursuer-singleevader (1P1E) game that should the evader’s FRS be contained with the pursuer’s FRS at all times, under the control actions of the pursuer, capture is guaranteed. One of the earliest applications of the FRS was seen in a 1P1E game [4] to the evasive strategy of an evader. Results demonstrated how an evader may escape using FRS analysis. Another work [5] incorporated the FRS by demonstrating in simulation, the control of multiple pursuers with homogenous properties to capture a single evader of a relatively larger FRS using centralized control. However, these algorithms fail to address the computation loads and performance of the pursuers in a realistic large scale game involving multiple evaders. Such a game tends to have a dynamic environment and demands the pursuers control to have the feasible and robust implementation. A popular approach to achieve both the feasibility and robustness in homogeneous multi-UAV applications is to share/distribute the control [6],[7] that is, share the
load of solving the central pursuer control problem amongst the UAVs in the network i.e. to “divide-andconquer”. This is achieved by converting central objective into distributed objectives and introducing communication between pursuers to exchange the intended action plans. In the distributed control architecture, compared to the centralized one, the individual control actions can be calculated faster. Also, in such a system, no one pursuer is central to the success of the pursuer team. Failure of a single pursuer slightly reduces the performance but does not catastrophically affect the system. The distributed pursuer system architecture can be simulated on a distributed simulation environment such as Real-time Multi UAV Simulator (RMUS) [8]. Simulation also allows the human operators/designers to assess important information about the pursuer systems; such as the FRSs, UAV trajectories, which cannot be visualized during the demonstrations with the real UAVs. For realistic capture demonstrations, some physical proximity between pursuers and evaders may be required at the capture state. This is high risk operation to perform with the real UAVs and may incur loss of UAVs. Our pursuer system consists of integration of three components; control strategy, architecture and implementation. This paper presents the distributed architecture for multiple pursuers using FRS based control strategy and its implementation to engage multiple evaders of higher manoeuvrability. Various capture game scenarios are performed and visualized in RMUS. SimTecT 2006 Refereed
This paper is organized as follows: Section 2 defines the FRS and describes the FRS based pursuer control strategy to capture multiple evaders. Section 3 presents the distributed pursuer architecture that employs the FRS based strategy. Section 4 presents in the RMUS architecture used for the PE game simulation, and Section 5 concludes the paper. 2.
FRS-BASED CONTROL STRATEGY
2.1 The Forward Reachable Set The Forward Reachable Set (FRS) of a UAV is dependent on its motion model. Consider the general dynamic model of the UAV a:
x& a = f a (x a , u a , t ) where x is the state vector of the ath UAV describing its position, velocity and attitude with respect to a fixed reference frame, u denotes the set of control inputs available to the UAV, f is a set of dynamic equations (also known as dynamic model) that governs the motion of the UAV and t denotes the time. The dynamic model often has model parameters, such as stall speed, max. turn rate, rate of climb and descent, etc. that impose constraints on the motion of the UAV. Another form of constraint is on its control inputs, such as thrust saturation, and can be stated generally as
u min ≤ u ≤ u max The control and dynamic constraints determine the FRS A(t , t + Δt ) | x(t ) , where A denotes the set of all future states x(t + Δt ) reachable by the UAV at a future time t + Δt , given its current state at time t [9], as illustrated in Figure 1.
Δt under state changes and normal flight conditions and hence can be assumed to have constant shape. 2.2 FRS Coverage and Control Strategy In the case of a 1P1E game, analysis using FRS provides insights for the pursuer to determine if it can capture the evader. Figure 2 a) shows that if the pursuer is positioned such that its FRS fully envelope the evader’s FRS, there exist trajectories from t to t + Δt for the pursuer to capture the evader. In the case whereby the evader is more manoeuvrable than the pursuer, its FRS well tend to have a smaller minimum radius of turn boundary (see Figure 1), then more than one pursuer is needed to coordinate and fully envelope the evader’s FRS [5], as shown in Figure 2 b). This enveloping of the evader’s FRS is termed as FRS coverage. a)
b)
Figure 2: a) FRS coverage showing one pursuer is sufficient to capture the evader b) FRS coverage showing two pursuers are need to capture the evader as evader’s FRS geometry is relatively wider Figure 3 a) illustrates the control strategy based on FRS coverage for an N-pursuer-M-evader scenario, where N > M > 1. The planning of the FRS coverage for each evader a=ei (i=1,2…,M) is executed first.
y z-axis paper
x
a) Figure 1: Top-down view of a FRS of an UAV at a future time t + Δt As the focus of this paper is on the architectural design for algorithm implementation and simulation, a few assumptions are made. Environmental factors such as terrain, practical communication are not considered here. The motions of the UAVs are simplified to a 2dimensional plane and only the top-down views of the UAVs’ FRSs are used to explain the FRS control strategy. As the focus is on the algorithm rather than the actual geometry of the FRS, it is assumed that UAVs of the same side have homogeneous properties. Also, there is negligible change in the FRS geometry for a given
b) Figure 3: a) Overview the FRS Control Strategy b) Execution of the FRS Feedback Control during Capturing Phase The results of the FRS coverage planning are Capture States (CSs), states which pursuer a=pj (j=1,2…,N) must reach simultaneously in such a way that total FRS coverage occurs. The degree of FRS coverage is
defined mathematically as Aei / U Apj . It is assumed that pursuer knows the motion model of the evader a priori and that there are sufficient pursuers to ensure total FRS coverage of every evader simultaneously. Also, the initial separations of the pursuers are such that capture is possible within a given operation time. In general, an evader vehicle has a detection region (shown by the circle) and they react to any pursuer if a pursuer is in the region. Pursuers are assumed to have global and accurate GPS knowledge of the evaders’ states at all times. If total FRS coverage can be achieved while the pursuers are outside of the evaders’ detection regions, then the pursuers gain an advantage over the evaders before the evaders can react to them. In a multiple evader case, the CSs from the FRS coverage is used in the evader allocation which consequently divides all the pursuers into smaller subteams and one evader is allocated to each sub-team. The remaining of the FRS strategy is as follows: 1. The pursuers predict the future states of the evaders, given the evaders’ current course of action at some time t+Tc , where Tc is some time horizon, that they are able to reach the CSs simultaneously.
pursuer reaches a certain capture proximity with the evader. It is assumed that a pursuer will not change its objective once an evader has been allocated to it. Using FRS-based control strategy, selecting an evader is equivalent to selecting a capture state corresponding to that evader. Hence FRS coverage needs to be performed prior to the EA task. The FRS coverage planning for each evader may be distributed amongst the pursuers, with the preference of one pursuer planning the FRS coverage and CSs for one evader. The pursuers chosen to do the planning are arbitrary using any method, for example, UAVs with the lowest and even number IDs will do the computation. The resultant CSs from the FRS coverage may then be communicated through the pursuer network for EA. The method to perform EA has to be distributed and computationally feasible, and will be discussed in Section 3.2. 3.
DISTRIBUTED PURSUER ARCHITECTURE
3.1 Overview The distributed pursuer architecture that employs the proposed strategy is shown in Figure 4.
2. This is followed by the trajectory control leading to the capture of an allocated evader occurs in two phase: i. A Positioning phase where the pursuers execute in feedforward trajectory control while they have not reach the CSs, as the evader is nonresponsive during this time. ii. A Capturing phase where after reaching the CSs, the pursuers execute FRS feedback control as they proceed to capture the evader after reaching the CSs, as shown in Figure 3 b). The pursuers coordinate at the same sampling rate δtpj. As the pursuers converge about the evader, they constantly aim to maximize their chances of capturing the evader by maximizing the FRS coverage [5]. 2.3 Evader Allocation (EA) When there are multiple evaders, the each pursuer needs to know which evader to designate/allocate as its target. During/after the EA, the status of each pursuer/evader can be in one of the modes shown in Table 1. Table 1: Pursuer and Evader Modes
Figure 4: General architecture of the distributed pursuer system An important aspect of distributed architecture is the inter-UAV communication. As each pursuer controls itself individually, there is a need for communication to exchange data with other pursuers to make its decisions. The data is then processed by each UAV as it continuously goes on a sense-plan-execute operation during the game. 3.2 EA in the Distributed Architecture The evader allocation process performed in the distributed architecture is illustrated in Figure 5. It uses an auction-based method [10] with the distance d lei to the lth capture state of the evader i as a metric. The objective is to select the CS of the smallest d.
The pursuer is said to be ‘Coordinated’ if there are other pursuers coordinating with it to capture its selected evader. Likewise, the mode of the corresponding evader UAV is set to ‘Selected’. The evader is considered ‘Captured’ when any of the
The EA process begins by projecting the CSs obtained from FRS coverage planning forward at t+Tc . This is done by predicting each evader’s state xei at t+Tc given their current course of action and their motion model. Note that as the CSs are relative to the evader states, the
planned CSs are then transformed accordingly given the xei(Tc) such that they maintain the same FRS coverage configuration. These transformed CSs are then communicated through the pursuer network until all pursuers have the CSs for EA.
problem can then be solved by minimizing the individual objectives min jpj such that the constraints γp1 = γp2 =…= γpN are satisfied. Figure 6 shows that for the positioning phase, the objective jpj and parameter γ are
jpj = Tc , γ = Tc..
Figure 5: Evader allocation process Each pursuer will calculate its set of d values for all CSs and communicate the results through the pursuer network. As each pursuer receives a set of d values from another pursuer, it will compare the received set with its own computed set. Any d values from the received set that are smaller than the receiving pursuer’s d values, the corresponding non-optimal CSs are eliminated from the receiving pursuer’s choices. After receiving the results from all pursuers in the network, if there are still CSs left in the pursuer’s choices, then it will select the optimal CS with the smallest d and go into ‘Allocated’ mode. In the case where by the number of pursuers are more than the number of total CSs, the pursuers with eventually no CSs left in its choices will proceed in ‘Unallocated’ mode. Pursuers that have selected the same evader will the form sub-teams to coordinate their actions and go into ‘Coordinated’ mode. EA stops once all the pursuers or evaders are in ‘Allocated’ or ‘Selected’ mode.
Figure 6: Example of distributing the centralized objective J to distributed individual objectives during the positioning phase. This is such that the pursuers have to arrive simultaneously at the CSs in the shortest time. Tc may be chosen to be the longest time resulting amongst the individual trajectories planned [12] in the same subteam. After reaching the CSs, the pursuers may execute FRS feedback control with a synchronized feedback sampling rate δtpj, thereby ensuring that the pursuers coordinate their motions simultaneously. Hence for the capturing phase,
jpj = Aei/Apj, γ = δtpj. This distribution of the centralized problem allows each distributed problems to be solved on a different platform, hence an easier integration into a distributed simulator. Also, as the parameters are reduced in the individual utility, the optimization becomes faster to be solved.
3.3 Distributed Trajectory Planning and Control After EA, the pursuer UAVs need plan their trajectories and control their motions along the planned path to capture the evaders. The distributed constraints satisfaction technique [11] is used to distribute the centralized problem, hence the FRS-based pursuit strategy, in the architecture. This means that each pursuer plans and controls its trajectory individually and coordinates its motion through individual objectives to satisfy a common constraint. In the FRS-based strategy, the objective of the pursuers during the positioning phase is to minimize the time Tc to the CSs, while the objective during the capturing phase is to maximize FRS coverage till capture. To distribute the trajectory planning and control problem amongst the pursuer UAVs, the centralized objective J for each phase is converted into distributed control problems to minimize the individual objectives jpj constrained by a common parameter γpj. The global
3.4 Communication As mentioned in the previous sections, timing plays a very important role in the planning and coordination of motion in FRS-based strategy. Hence, a Time-Triggered Synchronized (TTS) [13] communication scheme is used. A TTS communication scheme is defined [13] as: “a synchronization that occurs at a time where τpj ∈ {Δ,2 Δ …}, where Δ is the synchronization period. ” The message that is transmitted to pursuer pj from pursuer pq where q ≠ j, is denoted as:
Ypj (τ ) ∈ { y pq (τ )}
q≠ j
where the content of the message depends on the mode and control of the pursuer. Figure 7 shows the communication schemes with the messages amongst the pursuers. The Message Decoder
(as shown previously in Figure 4) in each pursuer UAV interprets the message that it currently receives.
In its simplest form, a script code consists of information about the simulation objects such as UAVs, sensors, and 3D terrain. It is also possible to script both stationary and moving virtual cameras. The simulation scenario then can be visualised from those cameras. The Time Server generates periodic or aperiodic time synchronisation messages. Upon reception of a time synchronisation message, the event handlers in each module activate the sequence of operation for the next simulation time step. Pausing of the time server makes it possible to analyse in detail, the overall system state for that particular time step.
Figure 7: The TTS communication scheme amongst the pursuers 4.
SIMULATION IN RMUS
The Real-time Multi UAV Simulator (RMUS) [8] is a distributed simulation environment that allows us to test the performance of the algorithms, control system, both inter and intra platform communications used in multiUAV systems. These algorithms, once validated, can be ported with no or minimum modifications to the real UAV hardware ready for real flight tests. Figure 8 shows a simplified architecture of the RMUS modules used in the PE game simulations. Depending on complexity of the simulation and the processing power requirements, the RMUS modules can be configured to run either on a single computer or they can be distributed over a number of computers on a network.
The Mission Planner (MP) module can be used, either interactively or through the scripting, to set up PE game simulation scenarios. Functionality of the MP can be enhanced by multiple plug-in modules. Typical plug-in modules for MP are the Flight Dynamics Models (FVDMs) and Path Planner Modules (PPMs). The numbers of UAVs in the PE game scenarios, their types, FVDMs, initial paths, sensors carried by each vehicle, etc. are defined in MP.
Figure 9: A screen shot from the FS module showing three pursuers and one evader in a PE game simulation. Similar to the MP module, functionality of the Flight Simulator (FS) module can be extended by plug-in modules. The Visual Plug-in Modules (VPMs) provide 3D models for the simulated visual entities. The captured screen image from the FS module in Figure 9 shows a group of UAVs in which their 3D appearances defined in VPMs. The FS module optionally supports Human-In-the-Loop (HIL) interface for the simulated UAVs. Through the HIL, a human pilot may control a selected evader’s escape actions.
Figure 8: The RMUS system architecture. In the RMUS environment, the simulation modules communicate with each other through the virtual channels of the RMUS’s novel communication framework CommLibX [8]. It provides hardware and operation system independent abstraction for the communication in and between RMUS modules and clusters. The overall RMUS configuration, simulation scene, vehicle models, FRS geometry, etc. can be scripted. The SimCompiler reads the script code and configures the individual RMUS modules.
The sensor carried by the UAVs can be grouped as flight sensors and mission sensors. The flight sensors are those used for navigation and control such as Global Positioning System (GPS), Inertial Measurement Unit (IMU), and the barometric pressure sensors. The mission sensors are specifically selected for a particular mission type. Infra-red (IR), colour and monochrome cameras, millimetre wave radar are just few examples to the mission sensors. Simulation of these sensors is not trivial [8],[14] and generally requires considerable amount of processing power. The Sensor Server module provides sensor readings to requesting objects. In a typical configuration, UAVs carry a variety of flight sensors and at least one mission sensor. As the simulated UAV objects fly in the
simulated scene, they send their Position, Velocity and Attitude (PVA) data to the sensor server. Given the PVA data for a UAV, predefined sensor properties, and relative orientation of the sensor with respect to the UAV, the sensor server calculates and publishes the sensor readings for that UAV. The PE Game Server is the main module for the reachable set based control. Different algorithms and strategies are encapsulated into separate plug-in modules to be used by the server. The desired algorithms can be performed by loading the corresponding plug-in modules. The Data Logger module can be programmed to log all messages on all virtual channels or it can log messages on selected virtual channels. The data logging process can also be activated automatically when a predefined system state is reached. This is particularly useful to concentrate on the desired phase (Figure 3) of the PE game scenario. The logged files can later be replayed for further analysis or demonstrations. 5.
Chung, C. F.; Furukawa, T. & Göktoğan, A.H. (2006) “Coordinated Control for capturing a Highly Maneuverable Evader using Forward Reachable Sets”, IEEE Int. Conf., Robotics and Automation, Orlando, in print.
6.
Feddema, J.T.; Lewis, C. & Schoenwald, D.A. (2002) "Decentralized Control of Cooperative Robotic Vehicles: Theory and Application." IEEE Trans Robotics and Automation, vol. 18, no. 5, Oct, pp. 852864.
7.
Beard, R.W.; McLain, T.W.; Goodrich, M.A. & Anderson, E.P. (2002) "Coordinated Target Assignment and Intercept for Unmanned Air Vehicles", IEEE Trans Robotics and Automation, vol. 18, no. 6, Dec, pp. 911-922.
8.
Göktoğan, A.H.; Ridley, M.; Nettleton, E. & Sukkarieh, S. (2003) “Real Time Multi-UAV Simulator”, IEEE International Conference on Robotics and Automation (ICRA'03), pp: 2720-2726
9.
Schättler, H. (1998) “Time-Optimal Feedback Control for Nonlinear Systems: A Geometric Approach.” Geometry of Feedback and Optimal Control, Marcel Dekker, pp. 381-421.
10.
Gerkey, B.P. & Mataric, M (2002) “Sold!: Auction Methods for Multirobot Coordination”, IEEE Trans Robotics and Automation, vol. 18, no. 5, Oct, pp. 758768.
11.
Yokoo, M; Etzioni, O; Ishida, T & Jennings N. (2001) “Distributed Constraint Satisfaction: Foundations of Cooperation in Multi-Agent Systems”, Springer Series on Agent Technology, Springer Verlag, Edition 1.
12.
Furukawa, T.; Bourgault F.; Durrant-Whyte, H.F. & Dissanayake, G. (2004) “Dynamic Allocation and Control of Coordinated UAVs to Engage Multiple Targets in a Time-Optimal Manner”, International Conference on Robotics and Automation (ICRA ’04), New Orleans, LA, pp. 2353-2358.
13.
Speranzon, A. & Johansson, K. H. (2003) "On some communication schemes for distributed pursuitevasion games", Proceedings of 42nd IEEE Conference on Decision and Control, pp. 1023-1028.
14.
Göktoğan, A.H.; Sukkarieh; S.; Cole, D. & Thompson, P. (2005) “Airborne Vision Sensor Detection Performance Simulation”, Simulation Conference and Exhibition, SimTecT'05, Sydney, Australia, 2005 and Interservice/Industry Training, Simulation and Education Conference (I/ITSEC'05), Orlando, FL, USA.
CONCLUSION AND FURTHER WORKS
A distributed architecture for the FRS-based pursuer control strategy had been proposed and implemented in the RMUS. Although the initial simulations are promising, the implementation still needs to be further enhanced. The currently implemented FRS is simplistic as its geometry does not change with the states of the UAVs. The implementation may also be extended to Hardware-In-The-Loop (HWIL) simulation. Practical asynchronous communication issues that are present in real pursuer systems should be addressed in future works. 6.
5.
ACKNOWLEDGMENT
This work is supported by the ARC Centre of Excellence programme, funded by the Australian Research Council (ARC) and the New South Wales State Government. REFERENCES 1.
Isaacs, R. (1965) “Differential Games: A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization”, Dover Publications.
2.
Salmon, D. M., & Heine, W. (1973) "Reachable Set Analysis - An Efficient Technique for Performing Missile/Sensor Tradeoff Studies", AIAA Journal, vol. 11, no.7, July, pp. 927-931.
3.
Lawrence, R. V. (1998) "Interceptor Line-of-Sight Rate Steering: Necessary Conditions for a Direct Hit", Journal of Guidance, Control and Dynamics, vol. 21, no. 3, May-June, pp. 471-477.
4.
Zanardi, C.; Herve, J.Y. & Cohen, P. (1995) “Escape Strategy for a Mobile Robot Under Pursuit”, Proceedings. of IEEE Int. Conference on Systems, Man and Cybernetics, Vol. 4, pp. 3304-3309.
