AUTONOMOUS AND COOPERATIVE MULTI-UAV ... - CiteSeerX

AUTONOMOUS AND COOPERATIVE MULTI-UAV GUIDANCE IN ADVERSARIAL ENVIRONMENT

by UGUR ZENGIN

Presented to the Faculty of the Graduate School of The University of Texas at Arlington in Partial Fulfillment of the Requirements for the Degree of

DOCTOR OF PHILOSOPHY

THE UNIVERSITY OF TEXAS AT ARLINGTON May 2007

c by UGUR ZENGIN 2007 Copyright All Rights Reserved

To my family for their support and making me who I am.

ACKNOWLEDGEMENTS I would like to thank several people who made this research possible. First, I would like to acknowledge the guidance of my supervising professor Dr. Atilla Dogan during my PhD studies. Our frequent meetings and fruitful discussions, thanks to his immense availability, have been very beneficial. It was only due to his continuous encouragement that I could publish several journal and conference papers during my studies. I would also like to thank him for the moral and financial support he had offered, without which, it would have been very challenging for me to maintain my focus consistently on education and research. I would also like to thank my dissertation committee members Dr. Don Wilson, Dr. Frank Lewis, Dr. Kamesh Subbarao and Dr. Wen Chan, for their constructive comments and for taking time to serve in my dissertation defense committee. I especially would like to thank to Dr. Subbarao for teaching me several courses and being available for any kind of discussion during my years at UT Arlington. I wish to thank the Department of Mechanical and Aerospace Engineering at UT Arlington for the financial support provided to me for my conference paper presentations in conferences. I am also grateful to Ms. Barbara Howser, the subject librarian of Aerospace Engineering for her immense help in conducting literature survey and obtaining research information from various sources. I am grateful to all the teachers who have taught me from elementary school to college, in Turkey. I would also like to thank former and current labmates of the Computer Aided Control System Design Lab, especially Sriram Venkataramanan (dear bro) for his continuous encouragement, even from overseas, and Eunyoung Kim for sharing her food iv

with me in my endless nights in the lab. My many other friends have been a wonderful community during my UTA years, providing support and entertainment, especially the Turkish community and UTA Soccer Club team members. Lastly, I am extremely grateful to my family for their continuous support, encouragement and patience. This dream would have never came true without them. April 20, 2007

v

ABSTRACT

AUTONOMOUS AND COOPERATIVE MULTI-UAV GUIDANCE IN ADVERSARIAL ENVIRONMENT Publication No. UGUR ZENGIN, Ph.D. The University of Texas at Arlington, 2007

Supervising Professor: Dogan, Atilla The research presented in this dissertation is aimed at developing rule-based autonomous and cooperative guidance strategies for UAVs to perform missions such as path planning, target tracking and rendezvous while reducing their risk/threat exposure level, and avoiding threats and/or obstacles by utilizing measurement information provided by sensors. First, a mathematical formulation is developed to represent the area of operation that contains various types of threats, obstacles, and restricted areas, in a single framework. Once constructed, there will be no need to distinguish between threats, obstacles and restricted areas as the framework already contains the information on what needs to be avoided and the level of penalty for a given position in the area. This framework provides the mathematical foundation for the guidance strategies to make intelligent decisions during the execution of the mission and also provides scalar metrics to assess the performance of a guidance strategy in a given mission.

vi

The autonomous guidance strategies are developed by using a rule-based expert system approach with the requirements of completing assigned mission or task, avoiding obstacle/restricted-areas, minimizing threat exposure level, considering the dynamic and communication constraints of the UAVs and avoiding collision. All these requirements and objectives are quantified and prioritized to facilitate the development of guidance algorithms that can be executed in real–time. The strategies consist of a set of “decision states”, which contain rules to determine how the host UAV should move by generating heading and speed signals. Cooperation of multiple UAVs is modeled by minimizing a cost function, which is constructed based on the level of threat exposure for each UAV and distance of each UAV relative to the target. This improves the performance of the system in the terms of increasing the total area of coverage of the sensors onboard the UAVs, increasing the flexibility of the UAVs to search for better trajectories in terms of obstacle/restricted–area avoidance and threat exposure minimization, and improving the estimation by providing additional sources of measurement. c Finally, the performances of the algorithms are evaluated in a MATLAB/Simulink

based simulation environment, which includes the dynamics of each vehicle involved, the models of sensor measurement and data communication with different sampling rates, and the discrete execution of the algorithms. The simulation results demonstrate that the proposed algorithms successfully generates the trajectories that satisfy the given mission objectives and requirements.

vii

TABLE OF CONTENTS ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

iv

ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

vi

LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xii

LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xvi

CHAPTER 1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

1.1

Introduction and Motivation . . . . . . . . . . . . . . . . . . . . . . . . .

1

1.2

Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

1.2.1

Research Objectives . . . . . . . . . . . . . . . . . . . . . . . . .

4

1.2.2

Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6

1.3

Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

1.4

Dissertation Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

2. LITERATURE SEARCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

2.1

Autonomous Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

10

2.2

Adversarial Environments and Map Building . . . . . . . . . . . . . . . .

11

2.3

Path Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

12

2.3.1

Single Vehicle Path Planning

. . . . . . . . . . . . . . . . . . . .

13

2.3.2

Multi Vehicle Path Planning . . . . . . . . . . . . . . . . . . . . .

16

Target Pursuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

18

2.4.1

Pursuit-Evasion Games . . . . . . . . . . . . . . . . . . . . . . . .

19

2.4.2

Ground target tracking and detection . . . . . . . . . . . . . . . .

20

2.4.3

Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21

2.4

viii

2.5

Cooperative Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

23

3. FORMULATION OF ADVERSARIAL ENVIRONMENT . . . . . . . . . . .

25

3.1

Formulation of Area of Operation . . . . . . . . . . . . . . . . . . . . . .

25

3.2

Probabilistic Threat Exposure Map (PTEM) . . . . . . . . . . . . . . . .

28

3.3

Time-Variant Probabilistic Threat Exposure Map . . . . . . . . . . . . .

30

3.4

Regions by the Level of Threat Exposure . . . . . . . . . . . . . . . . . .

32

3.5

Gradient Search on PTEM . . . . . . . . . . . . . . . . . . . . . . . . . .

33

3.6

Minima of PTEM on a Closed Disk . . . . . . . . . . . . . . . . . . . . .

34

3.7

Computational Efficiency of Evaluating PTEM . . . . . . . . . . . . . . .

37

4. TRAJECTORY PLANNING ALGORITHM . . . . . . . . . . . . . . . . . . .

39

4.1

. . . . . . . . . . . . . . . . . . . .

39

4.1.1

Dynamic Constraints of the UAV . . . . . . . . . . . . . . . . . .

40

4.1.2

Strategy Design . . . . . . . . . . . . . . . . . . . . . . . . . . . .

41

Comparison with the other strategies . . . . . . . . . . . . . . . . . . . .

48

5. ESTIMATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

56

4.2

Trajectory Planning Strategy (TPS)

5.1

Estimation via heading and speed . . . . . . . . . . . . . . . . . . . . . .

57

5.2

Estimation via position . . . . . . . . . . . . . . . . . . . . . . . . . . . .

60

5.3

Comparison of the estimation algorithms . . . . . . . . . . . . . . . . . .

62

5.4

Data Fusion and Estimation . . . . . . . . . . . . . . . . . . . . . . . . .

63

6. POINT SEARCH GUIDANCE ALGORITHM . . . . . . . . . . . . . . . . . .

68

6.1

6.2

Target Following Strategy . . . . . . . . . . . . . . . . . . . . . . . . . .

68

6.1.1

Restricted Area Search . . . . . . . . . . . . . . . . . . . . . . . .

71

6.1.2

Strategy Constraints . . . . . . . . . . . . . . . . . . . . . . . . .

72

6.1.3

Strategy cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

77

Simulation Environment and Results . . . . . . . . . . . . . . . . . . . .

81

6.2.1

81

Modular Structure of the Simulation Environment . . . . . . . . . ix

6.2.2

Implementation of the Strategy and Simulation Results . . . . . .

82

7. GRADIENT SEARCH GUIDANCE ALGORITHM . . . . . . . . . . . . . . .

98

7.1

Target Following Strategy . . . . . . . . . . . . . . . . . . . . . . . . . .

98

7.1.1

Preliminary Definitions . . . . . . . . . . . . . . . . . . . . . . . .

99

7.1.2

Decision Factors and Strategy States . . . . . . . . . . . . . . . . 104

7.1.3

Computation of Desired Heading and Admissible Range

7.1.4

Computation of Desired Speed and Admissible Range . . . . . . . 114

7.1.5

Scheduling Scheme for Heading Difference Constraint . . . . . . . 117

7.1.6

Detection of Local Minima . . . . . . . . . . . . . . . . . . . . . . 118

. . . . . 108

7.2

Simulation Results and Comparison with the Point Search Guidance . . . 119

7.3

Feasibility and Implementation of the Algorithm . . . . . . . . . . . . . . 129

8. COOPERATIVE GUIDANCE ALGORITHM . . . . . . . . . . . . . . . . . . 131 8.1

8.2

Cooperative Target Following Strategy . . . . . . . . . . . . . . . . . . . 131 8.1.1

Formulation of Cooperation . . . . . . . . . . . . . . . . . . . . . 132

8.1.2

Implementation of Cooperative Strategy . . . . . . . . . . . . . . 135

8.1.3

Improving Computational Efficiency . . . . . . . . . . . . . . . . 136

8.1.4

Collision Avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . 138

8.1.5

Guidance Strategy for Each Individual UAV . . . . . . . . . . . . 140

Simulation Environment and Results . . . . . . . . . . . . . . . . . . . . 142 8.2.1

Modular Structure of the Simulation Environment . . . . . . . . . 142

8.2.2

Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . 145

9. VIRTUAL TARGET TRACKING . . . . . . . . . . . . . . . . . . . . . . . . 158 9.1

Trajectory planning via virtual target tracking . . . . . . . . . . . . . . . 159

9.2

Rendezvous via virtual target tracking . . . . . . . . . . . . . . . . . . . 167

10. CONCLUSIONS AND FUTURE WORK

. . . . . . . . . . . . . . . . . . . . 170

10.1 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . 170 x

10.2 Directions for Future Study . . . . . . . . . . . . . . . . . . . . . . . . . 175 Appendix A. TRANSFORMATION BETWEEN INERTIAL AND LOCAL FRAMES . . . 178 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 BIOGRAPHICAL STATEMENT . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

xi

LIST OF FIGURES Figure

Page

1.1

Target tracking while avoiding obstacles and threats . . . . . . . . . . . .

3

3.1

Probabilistic threat exposure map of an operation area . . . . . . . . . .

31

3.2

Local minima of a PTEM . . . . . . . . . . . . . . . . . . . . . . . . . . .

36

3.3

Target coordinate system . . . . . . . . . . . . . . . . . . . . . . . . . . .

37

4.1

Heading search cone and target direction . . . . . . . . . . . . . . . . . .

40

4.2

Incremental distance in one simulation step . . . . . . . . . . . . . . . . .

41

4.3

Numbering of the points and reference heading . . . . . . . . . . . . . . .

45

4.4

Heading and speed search . . . . . . . . . . . . . . . . . . . . . . . . . . .

46

4.5

Speed change of the UAV guided by the strategies TPS, SCH and SIM . .

50

4.6

Threat exposure level along the trajectory of each strategy . . . . . . . .

51

4.7

Comparison of the trajectories at 1401 sec . . . . . . . . . . . . . . . . .

52

4.8


53

4.9


53

4.10 Comparison of the trajectories at 2001 sec . . . . . . . . . . . . . . . . .

54


54


55

5.1

Comparison of the estimation algorithms . . . . . . . . . . . . . . . . . .

63

5.2

Comparison of fitting error of x position estimation for UAV-1 . . . . . .

66

6.1

Proximity and sensor circles . . . . . . . . . . . . . . . . . . . . . . . . .

69

6.2

Searching to avoid restricted high threat region . . . . . . . . . . . . . . .

71

6.3

Heading constraint cone and heading difference cone . . . . . . . . . . . .

72

xii

6.4

No overlap between UAV heading and heading difference constraint . . .

74

6.5

Overlap between UAV heading and heading difference constraint . . . . .

74

6.6

Searchable points in the target proximity range . . . . . . . . . . . . . . .

75

6.7

Steering UAV into the proximity circle . . . . . . . . . . . . . . . . . . .

76

6.8

Decision tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

77

6.9

Modular structure of a singleUAV simulation environment . . . . . . . . .

82

6.10 UAV trajectory generated by the strategy to follow the target

. . . . . .

84

6.11 Actual, measured and estimated target heading and speed and commanded UAV heading and speed . . . . . . . . . . . . . . . . . . . . . . . . . . . .

85

6.12 Insufficient UAV trajectory generated by the strategy to follow the target

87


88

6.14 UAV trajectory generated by the strategy to follow a slowing target . . .

89


90

6.16 UAV trajectory generated by the strategy to follow a slowing and stopping target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

91


92

6.18 UAV trajectory generated by the strategy to follow a slowing and stopping target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93


94

7.1

Proximity & sensor circles, HC & HDC cones, and LOS vector . . . . . . 100

7.2

General proximity disk cone . . . . . . . . . . . . . . . . . . . . . . . . . 102

7.3

PR cone in various cases of position of Cr relative to Cp , (a) Cr (k) 6⊂ Dp (k) and Cr (k) ∩ Cp (k) = ∅, (b),(c) Cr (k) 6⊂ Dp (k) and Cr (k) ∩ Cp (k) 6= ∅, (d) Cr (k) ⊂ Dp (k) and Cr (k) ∩ Cp (k) = ∅ . . . . . . . . . . . . . . . . . . . . . 103

7.4

SHR cone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

xiii

7.5

Tangent heading bounding the UAV turn . . . . . . . . . . . . . . . . . . 111

7.6

Inertial and UAV local coordinate systems . . . . . . . . . . . . . . . . . 115

7.7

Heading difference constraint angle ψHDC . . . . . . . . . . . . . . . . . . 118

7.8

Detection of local minima . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

7.9

Case 1:UAV trajectories following an accelerating target . . . . . . . . . . 121

7.10 Case 1: Commanded signals for the UAV . . . . . . . . . . . . . . . . . . 121 7.11 Case 2: UAV trajectories following a slowing target . . . . . . . . . . . . 122 7.12 Case 2: Commanded signals for the UAV . . . . . . . . . . . . . . . . . . 123 7.13 Case 2: Computation time of the algorithms at each update instant . . . 123 7.14 Case 3: UAV trajectories following a move-stop-move-stop target . . . . . 125 7.15 Case 3: When the target stops in a non-restricted region . . . . . . . . . 126 7.16 Case 3 (New Algorithm): When target stops in a restricted area . . . . . 126 7.17 Case 3: Commanded signals for the UAV . . . . . . . . . . . . . . . . . . 127 8.1

Construction of cost functions . . . . . . . . . . . . . . . . . . . . . . . . 133

8.2

Formation of proximity disks around target . . . . . . . . . . . . . . . . . 134

8.3

Local threats effecting the UAV at a position . . . . . . . . . . . . . . . . 137

8.4

Collision avoided trajectories . . . . . . . . . . . . . . . . . . . . . . . . . 139

8.5

Top level modular structure of the simulation environment . . . . . . . . 142

8.6

Modular structure of a UAV subsystem . . . . . . . . . . . . . . . . . . . 143

8.7

Points that minimize PTEM on proximity disk (rmin ) and the corresponding UAV trajectories in cooperative cases . . . . . . . . . . . . . . . . . . . . . 147

8.8

Points that minimize PTEM on proximity disk (rmin ) and the corresponding UAV trajectories in noncooperative cases . . . . . . . . . . . . . . . . . . 148

8.9

Changes of the radii of the proximity disks by the cooperation strategy . 149

8.10 UAV-1 commanded signals for Kdc = 1 and Kdc = 0 . . . . . . . . . . . . 150 8.11 UAV-2 commanded signals for Kdc = 1 and Kdc = 0 . . . . . . . . . . . . 150 xiv

8.12 Comparison of the threat exposure level of UAV-1 . . . . . . . . . . . . . 152 8.13 Comparison of the threat exposure level of UAV-2 . . . . . . . . . . . . . 153 8.14 Comparison of the total threat exposure level of UAVs . . . . . . . . . . . 153 8.15 Comparison of the trajectory length and threat exposure level of UAV-1 . 155 8.16 Comparison of the trajectory length and threat exposure level of UAV-2 . 155 8.17 Comparison of the total trajectory length and total threat exposure level of UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 8.18 UAV-2 estimates for Ktc = 0 and noncooperative case . . . . . . . . . . . 157 9.1

UAV trajectory with rp = 5 m . . . . . . . . . . . . . . . . . . . . . . . . 159

9.2

UAV and virtual target heading and speed with rp = 5 m . . . . . . . . . 160

9.3

UAV trajectory with rp = 500 m . . . . . . . . . . . . . . . . . . . . . . . 161

9.4

UAV and virtual target heading and speed with rp = 500 m . . . . . . . . 162

9.5

UAV trajectory with rp = 2000 m . . . . . . . . . . . . . . . . . . . . . . 163

9.6

UAV and virtual target heading and speed with rp = 2000 m . . . . . . . 163

9.7

UAV trajectory with rp = 5000 m . . . . . . . . . . . . . . . . . . . . . . 164

9.8

UAV and virtual target heading and speed with rp = 5000 m . . . . . . . 165

9.9

UAV trajectory with fmax = 0.025 . . . . . . . . . . . . . . . . . . . . . . 165

9.10 Comparison of the threat exposure level and trajectory length for rp and fmax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 9.11 Trajectories of the UAVs and the virtual targets . . . . . . . . . . . . . . 167 9.12 Speed of the UAVs and the virtual targets . . . . . . . . . . . . . . . . . 168

xv

LIST OF TABLES Table 6.1 6.2 6.3

Page Effects of proximity range on the total UAV trajectory length (Ltraj ) and Threat Exposure Level (TEL) . . . . . . . . . . . . . . . . . . . . . . . .

95

∗ Effects of heading difference constraint(ψHDC ) on the total UAV trajectory length (Ltraj ) and Threat Exposure Level (TEL) . . . . . . . . . . . . .

95

Effects of fr on the total UAV trajectory length (Ltraj )and Threat Exposure Level (TEL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

96

7.1

Decision factors and strategy states (N/A: Not Applicable) . . . . . . . . 107

7.2

Parameters: Initial Position (IP) [km], Speed (IS) [km/h], Heading (IH) [deg], Sensor Range (SR) [km], Proximity Range (PR) [km] . . . . . . . . 120

7.3

Comparison of the algorithms . . . . . . . . . . . . . . . . . . . . . . . . 127

xvi

CHAPTER 1 INTRODUCTION This chapter provides the introduction and motivation for the research presented in this dissertation. The problem statement and the specific objectives of this research are also presented.

1.1

Introduction and Motivation UAVs (Unmanned Aerial Vehicles) are very promising for the technological lead-

ership of the nation and essential for improving the security of society, owing to their potential to perform dangerous, repetitive tasks in remote and hazardous environments. While recent technological advances have enabled the development of unmanned vehicular systems and recent implementations have proven their benefits in both military and civilian applications, the full benefit of unmanned systems will be utilized when they can operate autonomously. The primary requirements of autonomy are the capabilities of detecting internal and external changes, and of reacting to them without human intervention in a safe and efficient manner. Furthermore, the area where autonomous UAVs are tasked to operate may very often have various types of obstacles, threats and/or restricted regions. Thus, another challenge for the operation of autonomous UAVs is to be able to efficiently carry out the assigned tasks without compromising the success of the mission by flying into obstacles and/or restricted areas or becoming exposed to unacceptable level of threat risk. This challenge can be addressed by the development of intelligent guidance strategies and their implementation in autonomous guidance and control systems (AGCS) to “pilot” unmanned vehicles. Among the missions that can 1

2 be performed autonomously by UAVs in an adversarial environment are target tracking, path planning and rendezvous, just to name a few. Since all these different missions are to be autonomously operated in an adversarial environment, the first and the most important requirement for the development of autonomous guidance strategies for such missions is the construction of a map of the area of operation to formulate possibly various types of obstacles/restricted-areas and sources of threat/risk. Then, based on the formulation of the area of operation as a map, autonomous guidance strategies can be developed for each type of mission. Moreover, if autonomous guidance strategies can be formulated in a generic framework such that they can be easily modified for different types of missions, the development cost of AGCS for UAV operation in adversarial environment can be significantly reduced. In addition to capability of autonomy of a single UAV, coordination and cooperation [1–3] of multiple UAVs is a key enabler for full utilization of UAV systems. Although the addition of individual UAVs operating autonomously would be beneficial to a mission, groups of UAVs operating under an effective coordination strategy will perform much more efficiently. For example, tracking highly mobile targets is a type of mission that can significantly benefit from the use of UAVs with capabilities of autonomy and cooperation, especially when the pursuit is to take place in an adversarial environment. Another example is the rendezvous problem for multiple vehicles in an adversarial environment. Cooperative guidance of multiple UAVs in such an environment will certainly improve the likelihood of the simultaneous arrival of the vehicles at the target location. Figure 1.1 depicts an adversarial area of operation and scenarios where multiple UAVs are guided to perform missions in this area with various obstacles, no–fly zones, restricted areas and “threats”. For the successful execution of these missions, there should be a cooperative AGCS that leads to feasible decisions for the motion of the UAVs. For example, during a target pursuit scenario, the AGCS should generate guidance

Target

No-Fly Zone

Pursuer

No-Fly Zone

3

bs Threat

Restricted Area

O

Pursuer

Obstacle

ta

cl e

Threat

Figure 1.1. Target tracking while avoiding obstacles and threats. commands for the pursuers to ensure the tracking of the target while avoiding obstacles, threats and restricted areas. The computation of the commanded signals should take into account the dynamic constraints of the vehicles and the communication issues between them. Otherwise, the vehicles may not be able to implement the commanded maneuvers. Moreover, the decisions made by AGCS of each vehicle should be based on cooperative criteria so that an optimal trade–off is achieved between the increases in benefit and risk of employing multiple vehicles. Other missions that can be performed in this adversarial environment is path planning to reach from an initial to a final target position while avoiding adversaries or the rendezvous of multiple UAVs at final target position.

1.2

Problem Statement In this research, we investigate the problem of autonomous and cooperative guid-

ance of UAVs in adversarial environments. The emphasis is on (i) the mathematical formulation of the area of operation, (ii) development of single-UAV guidance and multipleUAV cooperation strategies (iii) development and implementation of the corresponding algorithms in a simulation environment.

4 1.2.1

Research Objectives Considering very diverse areas of application and very promising potential of uti-

lizing autonomous and collaborating UAVs, this research effort is aimed at developing rule-based autonomous and cooperative guidance strategies for UAVs to perform missions such as path planning, target tracking and rendezvous while avoiding threats and/or obstacles by utilizing measurement information provided by sensors. The guidance strategies should be implemented in algorithms that can be executed by onboard computers in real-time. Furthermore, the strategies should be used by multiple UAVs to increase the mission performance and decrease the risk of UAVs flying into obstacles or no-fly zones. In missions involving target tracking, the information regarding the motion of the target should be estimated [4–6] utilizing sensor measurements obtained by UAVs. An integrated simulation environment should be developed that includes obstacles, communication and dynamics of the target and UAVs and used in the development and evaluation of the guidance and estimation algorithms.

1.2.1.1

Formulation of the area of operation:

The first task of the proposed research is to develop a mathematical formulation to represent the area of operation, which contains, various types of threats, obstacles, restricted areas, in a single framework. The framework should enable the modeling of uncertainty in the information and facilitate real–time update of the information when needed. Once constructed, there should be no need to distinguish between threats, obstacles and restricted areas while the framework still contains the information on what needs to be avoided and the level of penalty for a given position in the area. The framework should provide the mathematical foundation for the guidance strategy to make intelligent decisions during the execution of the mission. It should also provide scalar metrics to assess the performance of a guidance strategy in a given mission. While

5 the framework represents the entire area of operation, it can be used to extract local information to infer decisions so that the computational burden is manageable.

1.2.1.2

Formulation of the objectives and the strategy:

The strategy has various requirements: (i) completion of assigned mission or task, (ii) obstacle/restricted-area avoidance, (iii) minimization of threat exposure level, (iv) consideration of the dynamic and communication constraints of the UAVs, (v) collision avoidance (vi) measurement of the target and estimation/prediction of its states and (vii) real–time execution. All these requirements and objectives should be quantified and prioritized to facilitate the development of a guidance algorithm that can be executed in real–time. The developed rule-based guidance strategies should generate commanded heading and speed signals for the UAVs. The strategies shall consist of a set of “decision states”, which contain rules to determine how the host UAV should move. The rules within each state should depend on the dynamic states of the host UAV, communicated states of the other UAVs in the team, estimated/predicted states of the target and the current status of the area of operation in terms of threats, obstacles and restricted areas.

1.2.1.3

Formulation of cooperation:

Cooperation of multiple UAVs should improve the performance of the system in three different aspects: (i) increase the total area of coverage by the sensors of the UAVs, (ii) increase the flexibility of the UAVs to search for better trajectories in terms of obstacle/restricted–area avoidance and threat exposure minimization and (iii) improve the estimation by providing additional sources of measurement. However, multiple UAVs, if not operating in a cooperative manner, would increase the risks of (i) collision with obstacles and with each other, (ii) incursion into restricted areas, (iii) unnecessarily high level of total threat exposure, and (iv) losing the target permanently. Thus, the guidance

6 strategy for each UAV should have a feedback from a cooperation algorithm to utilize the full benefits of multiple-UAV operation while keeping the risks at minimal levels. The cooperation algorithm shall minimize a cost function which is constructed based on the level of threat exposure for each UAV and distance of each UAV relative to the target.

1.2.1.4

System Integration and Verification:

A simulation platform should be employed in both the development and validation phases of the algorithms. The simulation environment should include dynamics of each vehicle involved, and models of sensor measurement and data communication with different sampling rates. It should also model the discrete execution of the guidance and estimation algorithms.

1.2.2

Assumptions The design of the cooperative and autonomous guidance strategies are developed

under the following assumptions: • Map of the area of operation represents a 2-D model of the environment. • Each vehicle is equipped with an onboard controller to execute the commanded signals to move the vehicle along the trajectory generated by the guidance strategy. • Closed-loop dynamics of each vehicle can be modeled by first-order transfer functions. • Dynamic constraints of each vehicle are independent of its states of the vehicle. • Each vehicle has a wide-area position sensor with 360-degree sensor footprint. • Each vehicle has a communication device to share measurement and position information with other vehicles. • Each vehicle has its own estimator to fuse data provided by the on-board sensors and information transmitted from other vehicles.

7 1.3

Contributions The following list summarizes the contributions of this research, which are already

published in Refs. [7–14] and/or reported in this dissertation. • Development of a mathematical formulation of the operation environment to represent threats, obstacles and restricted regions in a single framework, which models the uncertainity in the information and facilitates real-time update of the information when needed. • Development of rule-based guidance strategies for autonomous path planning and target tracking in adversarial environments that take into account the dynamics of the UAVs and that can be executed in real-time. • Development of measures of effectiveness (MoE) to evaluate the trajectories generated by the strategy and assessment of the performance of the guidance strategy in a given mission. • Development of a cost function for placing multiple UAVs around a target in PTEM (Probabilistic Threat Exposure Map) that models an adversarial environment and the development of a cooperation strategy based on the defined cost function for a team of UAVs to pursue a target in the adversarial environment while considering collision avoidance. • Extension of the target pursuit strategy to path planning problem by introducing and employing the concept of virtual target tracking. Employment of virtual target tracking for path planning overcomes some disadvantages of potential-field-like approaches, such as getting trapped in a local minimum. Further, the definition of restricted regions can provide hard avoidance quarantees by considering the obstacle’s shape and dimensions.

8 1.4

Dissertation Layout Chapter 2 will provide an extensive literature survey of previous work related to

this research. In Chapter 3, formulation of the adversarial operation environment will be given. Chapter 4 provides an overview of the point search guidance algorithm developed in this research for path planning in a dynamic adversarial environment. Chapter 5 provides a summary and comparison of the estimation algorithms used to estimate target motion based on position measurements in target tracking application. Chapter 6 presents the point search guidance algorithm developed for target tracking in adversarial environment. A gradient search guidance algorithm is introduced and a comparison of all target tracking guidance algorithms developed in this dissertation is given in Chapter 7. In Chapter 8, a cooperation algorithm is developed to provide the UAVs with the capability of operating in coordination during a mission. Chapter 9 presents the other application areas of the developed target tracking guidance algorithms such as trajectory planning and rendezvous of UAVs by introducing a virtual target approach. Finally, in Chapter 10, the dissertation ends with concluding remarks and discussion of related future research.

CHAPTER 2 LITERATURE SEARCH There has been an increasing effort of research in the broad area of Unmanned Aerial Vehicles (UAV) for military as well as civilian applications. In military applications, UAVs are not intended to replace pilots and manned aircraft, but to provide added capabilities to perform highest risk missions in remote and harsh environments, to operate continuously and to complete missions that require longer duration than the ones that manned systems are capable of. Since UAVs need not be concerned with the support of a human pilot, theoretically they can be smaller, less expensive and more agile. The increased agility results from being able to make sharper turns with higher gravitational force that otherwise cause a pilot to become unconscious. There are many different types of missions that single or multiple UAVs could conceivably be assigned to accomplish in any environment. One particular environment and type of missions of interest is the battlefield and military missions which can include battle damage assessment, delivery of weapons, target following, target destroying and reconnaissance, just to name a few. Although UAV development has primarily been initiated due to the military needs, there are also several civilian applications of interest today. Among these are search and rescue operations, urban surveillance, traffic monitoring, weather observation, surveillance of natural disaster sites such as areas hit by an earthquake or flood and investigation of areas that are inaccessible to humans such as volcanos in action or sites with radioactive radiation [15]. Despite the very large variety of missions and applications, one common requirement for more effective and succesful implementation of UAVs is the capability of autonomous, to some extend, operation in adversarial environments. 9

10 2.1

Autonomous Systems While recent technological advances have enabled the development of unmanned

vehicular systems and recent implementations have proven their benefits in both military and civilian applications, the full benefit of unmanned systems will be utilized when they can operate autonomously. The primary requirements of autonomy are the capabilities of detecting internal and external changes, and of reacting to them without human intervention in a safe and efficient manner. This can be achieved by developing and implementing autonomous guidance and control systems (AGCS) to “pilot” unmanned vehicles [2,16–24]. The recent advances in autonomous “self-piloting” UAVs can be found in [25]. To achieve the capability of autonomy, a system should be equipped with sensing, communication, navigation and onboard computing subsystems. In addition to these subsystems, autonomus decision making capability is needed to guide the vehicle in assigned tasks. Such a capability can be developed by using expert system approaches such as rule-based expert systems, fuzzy expert systems, frame-based expert systems [8, 26–28]. Brief introductions to various types of expert systems and their comparisons are provided in [29]. Rule-based sytems are the most popular expert systems where the knowledge is added in the form of rules and a solution is provided to the problem based on the relevant data. A unique feature of a rule-based expert system is its explanation capability which enables the system to review its reasoning and explain its decisions [30]. Another advantage of rule-based expert systems is that they organize decision making very efficiently. They have been built and applied in many areas such as business, engineering, medicine and geology, power systems and mining. Further, their applicability has been shown for the design of intelligent guidance systems for ground vehicles and intelligent flight control systems [31–33]. The rule-based expert systems consist of a rule-base providing the required knowledge, a backward chaining inference engine to perform reasoning and

11 a knowledge-base compiler to optimize the reasoning process. Other advantages of rulebased expert systems include ease of programming, testing, debugging and breakdown of the problem in a consistent and natural manner [31].

2.2

Adversarial Environments and Map Building For the development of the capability of autonomy using expert system approach,

the area in which the UAVs will be operating should be quantified with a suitable formulation. This is essential for the definition and organization of the knowledge about the area and for the development of decision states in the interference engine. Formulation of the area of operation is further challenging when UAVs operate in environments with adversarial components such as threats, obstacles, targets and no-fly/restricted zones. These adversaries may not be even fully known at the start of a mission. Thus, there is a growing interest in finding different approaches to deal with the uncertainties in the information of the area of operation. This leads naturally to considering probabilistic models to mathematically formulate the area of operation and incorporate the uncertainty. The most common method for building a probability map of an uncertain area of operation is grid-based occupancy maps, which assign probalities of adversary/target existence to the individual cells of the grid [34–40]. Such a map usually contains a point estimate of the probability for each cell [34–37, 40]. Since the full knowledge of the environment is not always available to the planner, it is more realistic to use sets that describe the range of possible values. This can be incorporated by a mathematical formulation that uses Beta distributions [38, 39]. The grid-based occupancy map approach, however, is mostly applied to environments with static adversaries. Bayesian estimation is used as a probabilistic approach to dealing with the dynamic uncertain environments [18, 41]. Further, the Chapman-Kolmogorov equation is used to predict the future movements of dynamic obstacles. Bayesian rules are also used for

12 finding the posterior probabilistic map, which is defined in terms of joint conditional distribution and computed from the data acquired by noisy sensors [42]. When the measurements are collected in a centralized manner, this architecture may cause a single point of failure in the central processing unit. Thus, decentralized approaches might be more preferable [36]. One such approach is to assign a map builder for each sub-area of the operation environment and to have the map builder exchanging information. Another approach is to use a stochastic model that represents different subregions with different stochastic properties such as mean and covariance [43]. Another problem of interest is to construct a map of the area of operation by using probability distribution functions for all the adversaries in the environment [44–47]. These types of maps are also called density distribution functions of threat sources or hospitability maps. By utilizing the sum of Gaussian distributions of the threats [44, 45, 47], various types of “threat” sources and no-fly zones or restricted areas can be characterized in the same probabilistic framework. This also provides a likelihood or a “weight” for each point on the terrain’s surface for the vehicle to move and maneuver.

2.3

Path Planning Using the formulation of the area of operation, an autonomus system should deter-

mine the safest path for a vehicle to move through the area of operation to accomplish a given mission. Recently, path planning has been an important research topic in the field of autonomous aerial vehicles. The early methods developed were based on the construction of a Voronoi diagram with the known locations of threats, obstacles and targets. By using search algorithms such as Djikstra and Bellman-Ford, a preliminary path can be generated from the Voronoi graph. In the worst case, which defines the upper bound on the running time of an algorithm for any input, the Djikstra algorithm is computationally more attractive than the Bellman-Ford algorithm [48, 49]. Another search algorithm

13 that can be used for path generation is the k-shortest path algorithm [50]. However, considering the dynamic constraints of the UAV, all the initial polygonal paths generated by these algorithms have to be refined to a flyable path. Several different methods are used for this purpose such as discretizing and smoothing [48], using filters that model the kinematics of vehicle [50], utilization of differentially flat systems [51], and using the dynamics of a set of virtual masses in a virtual force field emanating from each threat [52], which is similar to the potential field approaches [23, 53–55] in robotics. One drawback of these methods is that the effects of uncertainities in the locations of the threats are not considered. This means, the location of the threats and their effects are assumed to be known deterministically even in the case of pop-up threats. Complexity of path planning is increased when temporal requirements are added to the problem. One example of such problems is to compute the path so that the given mission is completed within a prespecified time for increasing the safety of the UAVs. In the case of UAV teams, another objective for the UAVs is to arrive at their assigned locations simultaneously to increase the element of surprise and decrease the risk of detection. This sort of optimization problem is called rendezvous problem [12, 48, 56, 57]. UAV applications requiring path planning can be classified as follows.

2.3.1

Single Vehicle Path Planning In single vehicle planning, the main requirement from an autonomous vehicle is to

move from the initial position to some other positions to execute the necessary tasks in the area of operation. One specific type of application is for a single UAV to reach a single target through an area of operation with static obstacles and threats [44,45] . Single UAV-multiple targets with static obstacles or threats [45] is another type of application that requires the determination of the sequence for the UAV to reach multiple targets through an area of operation with static obstacles and threats. Another application type

14 using single UAV is to complete a mission through an area of operation with moving obstacles and threats [16, 41, 58], which requires a more sophisticated path planning design procedure. Single vehicle path planning approaches can be classified as follows.

2.3.1.1

Graph-based Approaches

In graph-based approaches the knowledge of the environment is represented as a graph. Thus, the complexity of the graph depends on the environment surroundings such as obstacles, threats or no-fly zones rather than the physical size of the environment. The graph consists of nodes that correspond to positions in the environment. The nodes are linked to each other with defined costs. The objective is to find the path with the minimum total cost that is computed through all the links that connect the inital node to the final node. There are several methods to represent the environment as a graph such as generalized Voronoi diagrams [59], Voronoi graphs [60,61], Delaunay triangulation [22,62], visibility graphs [61, 63, 64] and topographical mapping [65, 66]. These graphs can also be integrated in a single architecture to generate a more efficient path [61]. After the representation of the environment as graph, a graph search algorithm such as Djikstra, Bellman-Ford or A* algorithms [49, 67] are used to find the shortest path.

2.3.1.2

Probabilistic Approaches

The most common probabilistic approach in path planning is the probabilistic roadmap method. This method is used particularly for the motion planning of robots with many degrees of freedom. It consists of two phases, a learning phase in which a graph called roadmap is constructed and a query phase in which a feasible path is found. Probabilistic roadmap is constructed by generating random free configurations of the robot and connecting them by using a local planner. The nodes of this graph are the configurations and the edges are the paths computed by a local planner. The planner

15 searches the graph to find a sequence of edges connecting the start and goal nodes in the roadmap, and finally a feasible path is generated. However, most research studies in this area [68–71] deal with path planning problems in known environments and they are efficient only if the queries do not require passing through narrow corridors. This is also known as “narrow passage” problem [72]. Nevertheless, this problem can be solved by a probabilistic roadmap algorithm that uses a powerful local planner [73]. Probabilistic roadmap algorithms can also be used at unknown environments [74, 75]. Main idea to achieve this is to initially generate a few random nodes in the environment map and then to create the path gradually by choosing the nodes that are in the detection region of the sensors. Furthermore, it decreases the cost of the paths by shrinking the local free node regions and erasing the nodes which are redundant. Nevertheless, it can generate more nodes randomly if necessary to guide the robots to pass through narrow passages. There are also other probabilistic approaches in the literature for robot planning that use probabilistic cell decomposition [76], collision probability between robot and the obstacles [77] and a workspace, which is divided into a finite number of cubes with the probability of each cube becoming a dead end [78]. As mentioned earlier, most of the recent probabilistic approaches developed for UAV path planning are based on the probabilistic map of the area of operation [1, 44, 45, 79, 80] and they should be able to deal with the uncertainties in the information of threats, obstacles and targets.

2.3.1.3

Evolution-based Approaches

Evolution-based approaches are a class of optimization techniques, which use a population of solutions and random modifications to these solutions to form a structured stochastic search. By the stochastic search, a random population of paths is generated. Encoding of these paths can be a sequence of simple splines or B-splines. The solutions obtained from this search are evaluated based on a fitness function and the one with

16 the best fitness value is selected as a candidate solution. This solution is then used to generate new candidate solutions by random modifications. This evolution process is continued until a certain stop criterion is reached. The evolutionary algorithms are very efficient at solving combinatorial optimization problems and more robust to uncertainities compared to directed search methods. Further, they can easily be implemented in problems with a high number of constraints and adapt to the special characteristics of the problem under consideration [20]. A brief summary of several evolutionary path planners can be found in [81]. Evolutionary approaches have been successfully applied to offline/online path planning problems for UAVs [16,19,20,82–84] and autonomous underwater vehicles [85]. By using an evolutinary approach, a near optimal solution to problems involving a dynamically changing environment with varying terrain, dynamic obstacles and moving targets can be found [16,83]. The reason for obtaining sub-optimal solutions is premature convergence that prevents the evolutionary algorithm from reaching global optimal solution. To overcome this problem, a framework of parallel evolutionary algorithms, in which several solutions grow simultaneously and compete with each other, can be used [82]. This provides more exploration capability to the planning algorithm and decreases the likelihood of being trapped in local optimal solutions.

2.3.2

Multi Vehicle Path Planning When multiple vehicles are used in a common mission, path planning requirements

should be satisfied for all vehicles as a team as opposed to for each vehicle individually. In addition to path planning, target management is also necessary to assign vehicles to the targets such as to minimize the group path length to the targets, to minimize group threat exposure, to maximize the number of vehicles reaching each target [34, 35, 50, 86]. There are two different approaches to path planning for multiple vehicles in terms of system architecture and communication requirements.

17 2.3.2.1

Centralized Approaches

Centralized approaches are characterized by their system architecture in which the entire system is managed by one of the agents in the team or by a single command and control center. This approach can be applicable for a team of UAVs consisting of reasonable number of vehicles. One hierarchical design methodology for centralized path planning of multi-vehicle systems is grouping sequence of available team sources and planning the assignment of feasible routes at the upper level [87]. Further, central controller considers the timing sequence of team activities. The controller at lower layer generates feasible trajectories in real time for each vehicle to follow. Another hierarchical design methodology for centralized approach is to generate the inputs for the team leader where the position and input of the leader is sent to followers to maintain the formation [51]. Further, the central controller accounts for the probability of UAV loss during the mission, which effects the operation of other vehicles [88]. This is done by a stochastic formulation that improves the survival probabilities and maximizes expected mission score.

2.3.2.2

Decentralized Approaches

When the team consists of large number of vehicles, problems might occur with the application of centralized approach, especially in complex missions. Among these problems are the slow response to dynamic conditions, intractable solutions for large teams, communication problems, high band-width requirements and the leader becoming a central point of failure [89]. These problems might cause the lack of responsiveness of the system to the changes in the environment. Thus, using a decentralized approach provides significant simplification in the team-level planning process and a substantial decrease in the volume of information flow among the agents. Decentralization process

18 can be seen as the division of a complex problem into subproblems, which can be solved by the components of the system. Decentralized approach is extensively used for the path planning problem of multiple UAVs, where each aircraft plans its trajectory using a receding horizon strategy based on mixed integer linear programming [90]. This provides a solution to the conflicts between vehicles in a sequential, decentralized fashion by allowing each aircraft to take into account the latest trajectory and loiter pattern of the other aircraft. Decentralized approaches are also applicable for the missions that a team of UAVs operates under timing constraints [91, 92]. In such applications, rather than using UAV state trajectories for all other UAVs on the team centrally, only the critical timing information is determined, which is represented by a coordination function and UAV trajectories are determined in a decentralized fashion on the individual UAVs.

2.4

Target Pursuit Unlike path planning, when the target involved in a mission is mobile, the problem

is usually referred to as “tracking” or “pursuit” [93–98]. Dynamic nature of the mobile target brings additional constraints to the path planning problem such as considering the states of the target and maintaining the visibility of the target. Tracking highly mobile targets is a type of mission that can significantly benefit from the use of UAVs with capabilities of autonomy, especially when the pursuit is to take place in an environment where various sources of threats, obstacles and restricted areas may exist. The objective in autonomous tracking of a target is to enable the UAV to track the target given the present and predicted states of the target and map of the area of operation while respecting the dynamic constraints and avoiding obstacles and collisions. In a military scenario, multiple UAVs can be used to track enemy or escort a friendly convoy [93, 94, 99, 100] while avoiding no-fly zones and possible sites of SAMs (Surface-to-Air-missiles). In a border patrol application, UAVs can be employed to track intruders while staying within

19 the border and avoiding high elevation. In a law enforcement scenario, criminals might be pursued or a specific vehicle might be tracked for protection while avoiding high buildings or residential areas. As a wild life protection effort, animals can be tracked while avoiding high elevation. In such applications, there are several common features such as (i) mobile objects to be pursued (ii) measurement and estimation of the position of the moving object (iii) adversaries to be avoided.

2.4.1

Pursuit-Evasion Games In pursuit-evasion games, a pursuer or a group of pursuers search for an evader

(target) while a group of evaders can either move around randomly in the area of operation or try to avoid the pursuers. Target pursuit ends when all evaders are captured. In the earliest formulation of the pursuit-evasion game theory, it was first assumed that both players know the current values of the state variables. Thus, the game is played with full information. This was the greatest limitation on the present form of the theory especially for military based games [101–103]. In practical applications, this resulted in overly conservative pursuit policies applied to inaccurate maps built from noisy measurements. The first rigorous formulation of the pursuit-evasion problem was given in a game, which is assumed to be played in an environment with a discrete graph where the approach was to find a trajectory to clear the entire graph until the pursuer touches the evader [102]. Secondly, a visibility-based version was formulated where the domain was changed from discrete graphs to continuous polygonal free spaces [103], with an approach that the pursuer need not to touch the evader, but can instead see the evader from a distance with an unlimited range (but cannot see through walls). Nevertheless, the most vital research area was the extension of these cases of complete information to the cases of partial information, which has appeared recently in the literature [104–107]. This is achieved by using probabilistic approaches similar to map building and path planning

20 problems. The probabilistic approach avoids the conservativeness inherent to worst-case assumptions on the motion of the evader [104–106] and also takes into account the fact that the measurement is not precise and that only inaccurate a priori map of the terrain may be known. Further, a pursuit-evasion game is described as a Markov game where the system evolution is governed by a translation probability function that models the uncertainity affecting the players’ motion [107]. The lack of information about the area of information and sensors inaacuracy are also added in the same framework by considering a partial information markov game.

2.4.2

Ground target tracking and detection One of the important target tracking problems is the ground target tracking by

autonomous UAVs. Several strategies have been developed to track a ground vehicle, which is able to change its heading and vary its speed (maneuvering) [94, 95]. In such strategies, UAVs are assumed to track the ground vehicle with an offset vector as well as pre-defined loitering modes such as a sinusoidal motion, which might limit the capabilities of the strategies. Further, the UAVs are assumed to be operating in a friendly environment where threat exposure and restricted regions are not a concern. Another approach is the enhancement of a multiple hypothesis (MHT) algorithm to make it usable for tracking moving ground targets [98]. Included in these enhancements are bias estimation, road modeling, stopped target motion and stationary target tracking. A survey about target tracking over the past 40 years and its applicability to tracking ground targets can be found in [108]. Further information about detection of a ground vehicle can be found in [3, 109–112], where new developments and trends in Ground Moving Target Indicator (GMTI) radars and Synthetic Aperture Radars (SAR) are explained. The improvement of the ground moving target indication performance and the optimal placement of GMTI UAVs carrying out surveillance of several ground targets are stud-

21 ied as well [3, 111]. GMTIs can also be used in assignment-based algorithms for data association in ground target tracking for targets employing evasive maneuvers such as move-stop-move maneuver [113].

2.4.3

Estimation Ground target tracking is much more difficult than the aerial target tracking due

to the variable terrain structures. This can change the target’s motion patterns significantly and increases the obscurity in the measurement data [114]. Further, ground target tracking presents unique challanges such as high target density and maneuverability, high clutter and low visibility [108]. Thus, simple state estimation algorithms have evolved into more complicated algorithms to be used for tracking highly maneuvering ground targets. Estimation algorithms are used to process the raw data detected from the sensors, since raw data sometimes might be unusable for the control and guidance systems. A series of papers [115–119] provides a comprehensive survey of estimation in maneuvering target tracking. Estimation problem becomes even more interesting when the sensor data is obtained from multiple sensors, since efficiently fusing all the sensor data increases the accuracy of the estimation.

2.4.3.1

Multi Sensor

When the measurements are obtained from multiple sensors [120], there are two cases to be distinguished. In the first case, sensors operate synchronously, which means that the sampling times of all the sensors are the same. In the second, which is a more realistic case, sensors operate asynchronously [121] and provide data at different rates with different communication delays. When the measurements obtained asynchronously in a centralized tracking algorithm, the possibility that the measurements are received in the wrong order due to time delays must also be considered. This is called the out-of-

22 sequence measurements situation, which is being dealt by using different approaches in the literature [122–124]. This requires special consideration in carrying out the update. If the measurements originate from different sensors, then the sampling interval might be negative in general. This is called negative-time measurement update.

2.4.3.2

Asynchronous Measurements

There are several approaches in the literature [125–132] dealing with the asynchronous sensors in multiple sensor applications. In references [125, 126], a Bayesian approach is used to deal with asynchronous measurements. A new method is presented in [125] incorporating out-of-sequence measurements based on the “particle” filter. In reference [126], a tracking algorithm is developed for multi-sensor single target tracking in the presence of asynchronous measurements and high clutter levels. This algorithm also uses “particle” filtering methods to perform prediction and update. Additionally, it does not require batch processing, which is an important requirement to obtain a good initialization for some tracking algorithms such as Kalman filter-based algorithms. By “particle” filtering, the posterior distribution of interest with a set of weighted particles is represented. If the weights are chosen accurately, the expectations of the posterior distribution can be made close to the equivalent expectations of the set of weighted particles by using more particles. Basically, particle filtering is a method of updating these particles as the time progresses and weighting them correctly [131]. References [127,130] provide contributions in decentralized estimation for asynchronous sensors by designing a multiple lag out-of-sequence measurement algorithm for a linear measurement model based on the minimum variance estimation algorithm. Further, using the interacting multiple model (IMM), the algorithm for optimal single-lag out-of-sequence filtering is extended to multiple-model setting for out-of-sequence case.

23 Another approach to deal with asynchronous sensor data without performing a sequential processing is data fusion. The data fusion problem for the observations obtained from asynchronous sensors can be briefly stated as finding the conditional probability density function of the system state, conditioned on the observation histories of all the sensors [128]. The algorithm developed in [128] can be used for both similar and dissimilar sensors with arbitrary sampling rates and communication delays. In reference [129], a multi-sensor fusion algorithm is developed using multidimensional data association for multi-target tracking. Observations are obtained from multiple asynchronous sensors with different sampling intervals and a sliding window technique is used to update the estimates after each frame of measurements.

2.5

Cooperative Planning Cooperation among multiple UAVs is a key capability for utilizing the full potential

of the UAVs. The goal achieved by enabling a cooperative behavior in a system is to increase the success rate of the mission. Cooperative behavior in this context can be defined as the coordinated action of multiple vehicles to optimally perform a given task. The initial research in cooperative planning with applications to robotics was conducted [133]. In recent years, cooperative planning for UAVs has also been extensively studied for a variety of applications including formation flying of UAVs [134–138], cooperative path planning [19, 87, 88, 90, 139, 140], cooperative rendezvous [48, 57, 91, 92], coordinated target assignment and intercept [17, 21, 40] and cooperative target tracking [93, 94, 97, 99, 100]. A common approach in cooperative planning research is to optimize a suitably chosen cost function. In most cooperative path planning and rendezvous problems, a single cost function is selected to represent the total threat cost of the team [48, 57, 140]. This reduces the problem to choosing a path description and velocity along the path for

24 each UAV that will minimize the total cost function for the team. However, this approach requires a precalculation of a family of paths and the velocities along the paths. Thus, this optimization problem involves searching over an infinite number of solutions. For practical purposes, the search space of the optimization is usually reduced dramatically by considering only finite number of paths. Further, this optimization must be carried out prior to the mission and when changes occur in the threat scenario during the mission. In addition to the threat cost, in some applications, the fuel cost is also added to the cost function [91, 92]. Further in coordinated target assignment problems, the team power (number of UAVs prosecuting the same target) and the spread of the intercepted targets (number of targets to be intercepted) can be added to the cost function as well [21]. An application of cooperative target tracking recently studied is convoy protection [94, 99, 100]. In such applications, both target and environment are friendly where UAVs are commanded to fly directly over the target to provide surveillance and cooperation is provided through formation control. However, when the area of operation has adversarial elements in it, UAVs cannot always fly directly over the target trajectory. This may increase the risk that the UAVs are subject to and the likelihood of losing the target. In a recent study [93], the problem of tracking a hostile target is adressed by defining a stand-off distance for the UAVs while the sensor coverage is maximized by adjusting the relative phase angle between the UAVs. Nevertheless, since the threat cost of the UAVs and the team is is not incorporated into the coordination, the UAVs will always track the target with the constant stand-off distance and will not be able to adjust their distances to the target based on their threat exposure levels.

CHAPTER 3 FORMULATION OF ADVERSARIAL ENVIRONMENT Adversarial environment is an environment where threat exposure should be minimized, and obstacles and restricted areas should be avoided. In this dissertation, “threat” is used as a broad term to describe the risk or cost for a UAV to occupy a given location at a given time as well as obstacles and restricted regions in the area of operation. When a UAV is flying in an area with multiple threats, safety of the flight is characterized by the probability of the UAV becoming disabled at a certain location, specified by its x– and y– coordinates relative to a frame of reference, (x, y) at a certain time t. To be able to construct the problem in a probabilistic framework, several events are defined and their probabilities are determined.

3.1

Formulation of Area of Operation Let Ei (x, y, t) be the event that the UAV becomes disabled by ith source of threat

at the position of (x, y) at time t in the area of operation. E(x, y, t) is the event that the UAV becomes disabled by at least one of the threat sources at the position (x, y) at time t. Then, let fp,i (x, y) and ft,i (t) be probability density functions (pdf ) such that the probability of the UAV becoming disabled by ith threat source at the neighborhood of (x, y) at time t is pi (x, y) = fp,i (x, y)ft,i (t)∆x∆y∆t,

(3.1)

where ∆x and ∆y are to define the area of a neighborhood of (x, y) and ∆t is to define a neighborhood of t. Note that, fp,i (x, y) models the dependency of becoming disabled on position and ft,i (t) models the dependency of becoming disabled on time. As an example, 25

26 fp,i (x, y) can be characterized by a Gaussian pdf, which specifies the concentration point (location) of the threat by the mean value and the level of penalty of flying close to it by the variance. Regarding the time dependency of becoming disabled, various possible pdf s can be used for ft,i (t). For example, a uniform pdf for ft,i (t) means that the threat exposure level of the UAV at a given position does not depend on time itself but the amount of elapsed time in the neighborhood of that position. If the level of exposure of the UAV to threats increases as it stays longer in the area of operation, then an increasing probability density function of time should be defined for ft,i (t). If there are, in the area of operation, threats that become less effective in disabling a UAV or the UAV becomes less vulnerable to, then ft,i (t) should be defined as a decreasing probability density function of time. Note that, in this case, the probability of a UAV becoming disabled still increases as it stays in the area of operation, however the rate of increase becomes smaller. Now, let S(x, y, t) be a certain event that the UAV follows trajectory S to reach (x, y) at time t. Then, the conditional probability of the event that the UAV becomes disabled by ith source of threat at the position of (x, y) at time t under the condition that the UAV follows trajectory S is defined as pS,i (x, y, t) = P [Ei (x, y, t)|S(x, y, t)] =

Z

fp,i (x, y) ft,i (t) l1 (x, y, t) l2 (x, y, t) dt,

(3.2)

t

where l1 and l2 are used to define the neighborhood at a point on trajectory S (e.g. radar signature area of the vehicle). Also note in Eq. (3.2) (x, y) are functions of time and thus fp,i is also a function of time. To better explain the dependency of the conditional probability on both position and time, Eq. (3.2) will be presented in a special case where the UAV enters the area

27 of operation at time t0 , moves until time t1 , afterwards stops and hovers at the same position. In this case, Eq. (3.2) can be written as Z t1 pS,i (x, y, t) = fp,i (x, y) ft,i (t) l1 (x, y, t) l2 (x, y, t) dt t0 Z t + fp,i (x(t1 ), y(t1 )) ft,i (t) l1 (x(t1 ), y(t1 ), t) l2 (x(t1 ), y(t1 ), t) dt, (3.3) t1

where fp,i is constant after time t1 because the UAV does not change its position. If l1 and l2 do not explicitly depend on time, then Z t1 pS,i (x, y, t) = fp,i (x, y) ft,i (t) l1 (x, y, t) l2 (x, y, t) dt t0

+ fp,i (x(t1 ), y(t1 )) l1 (x(t1 ), y(t1 )) l2 (x(t1 ), y(t1 ))

Z

t

ft,i (t) dt,

(3.4)

t1

Note that in the above equation, the second term shows how the probability increases even when the position of the UAV does not change. Let us assume that ft,i (t) is a uniform pdf, i.e. it is constant in the interval when it is not zero. Let us further assume that during the time the UAV stays in the area of operation, ft,i (t) is nonzero. Then, Z t1 pS,i (x, y, t) = ft,i (t0 ) fp,i (x, y) l1 (x, y, t) l2 (x, y, t) dt t0

+ fp,i (x(t1 ), y(t1 )) l1 (x(t1 ), y(t1 ), t) l2 (x(t1 ), y(t1 ), t) ft,i (t0 ) (t − t1 ),(3.5) where note in the first term that fp,i (x, y) is still implicitly a function of time since the UAV moves until t1 . If there are N number of sources of threat in the area of operation, then the conditional probability of the UAV becoming disabled by at least any one of the sources of threat at the position of (x, y) at time t under the condition that it follows trajectory S is pS (x, y, t) = P [E(x, y, t)|S(x, y, t)] .

(3.6)

Since E(x, y, t) =

N [

i=1

Ei (x, y, t)

(3.7)

28 and Ei (x, y, t) are not necessarily disjoint events, pS (x, y, t) ≤

N X i=1

pS,i (x, y, t) =

N Z X

fp,i (x, y) ft,i (t)l1 (x, y, t)l2 (x, y, t)dt

(3.8)

S

i=1

by Union Bound [141]. Thus we can easily compute an upper bound on the probability of a UAV becoming disabled if it follows a certain trajectory in an area with multiple threat sources. If l1 and l2 are assumed to be constant for any position and time on the trajectory and ft,i (t) is the same for all threat sources, then " N # Z X pS (x, y, t) ≤ l1 l2 ft (t) fp,i (x, y) dt, t

(3.9)

i=1

since fp,i (x, y) and ft (t) are probability density functions and therefore integrable.

3.2

Probabilistic Threat Exposure Map (PTEM) In the formulation of the probability of becoming disabled, introduced in the pre-

vious section, the dependency on position (the part of Eq. (3.9) in square brackets) is defined to be the Probabilistic Threat Exposure Map (PTEM), which quantifies the risk of exposure to sources of threat as a function of position. This concept is particularly useful in defining, in a single framework, various types of threats such as objects or locations that need to be avoided as far as possible, obstacles or restricted areas that should not be entered. This probabilistic map is not meant to provide the actual map of the area of operation, but to provide a way to plan the trajectory of a UAV to avoid obstacles and accomplish the given mission such as path planning and target tracking. All the threat sources (e.g. exposure to enemy radar, obstacles, and no-fly zones or restricted areas) are characterized in the same probabilistic framework using the sum of probability distributions of threats, obstacles and restricted areas. As mentioned earlier, if a threat is characterized by a Gaussian pdf, there are two parameters needed to fully specify the threat; the mean value specifies the concentration point (location) of the threat source

29 and the variance specifies how the threat can have an area (or volume for 3-D case) of effectiveness. The mean values and variances of each Gaussian function are specified such that the obstacles, restricted areas, no-fly zones and threats in the area of operation are all represented. Gaussian distribution can be used to model enemy radars or missiles as well as obstacles and restricted areas. Note that there is not necessarily one-to-one correspondence between the actual obstacles/restricted-areas/threats and the Gaussian functions used in the construction. An actual obstacle, for example, may require multiple Gaussian functions while a Gaussian function may be enough to represent a threat and restricted area, together. Further, a uniform distribution can also be used to model obstacles or restricted areas. However, note that, a map with a uniform distribution can only be used with a guidance algorithm that does not need differentiation of PTEM such as the ones described in Chapters 4 and 6. For the algorithms described in Chapters 7 and 8 only differentiable probability distribution functions can be used such as a Gaussian distribution. Once PTEM is constructed, there is no need to distinguish between the types of threats, obstacles, or no-fly zones and the use of the probabilistic map is sufficient for decision making. This is because the map already contains the information on the penalty of flying close to a source of threat or a restricted area. The PTEM quantifies the threat exposure level at a given position in the area of operation. A position in the area of operation is defined by its vector, r, relative to the origin of a reference coordinate system. Let r be the representation of vector r, i.e. r = [x y]T , where x and y are the components of vector r along the x- and y- axes of the reference frame. The same notation will be used for all the other vectors and their representation throughout this dissertation, i.e. v is a

30 vector with its representation v in the reference frame. By using this notation, PTEM equation of an area of operation, modelled by Gaussian distributions, can be written as: N X 1 1 T −1 p exp − (r − µi ) Ki (r − µi ) (3.10) f (r) = 2 2π det(K ) i i=1 where µi and Ki are the mean vector and the covariance matrix of the ith threat, respectively and defined as 



 µx,i  µi =  , µy,i





2  σx,i 0  Ki =   2 0 σy,i

Note that, threats modelled by Gaussian distributions are characterized by the wellknown Multidimensional Gaussian Law. If there are additional threat sources, obstacles or restricted-areas that should be modelled by any other distribution, for example uniform pdf, then the summation in Eq. (3.10) that formulates PTEM will have terms with uniform pdf as

u(r) =

    

1 (b−a)(d−c)

0

a ≤ x ≤ b AND c ≤ y ≤ d ,

(3.11)

(x < a OR x > b) AND (y < c OR y > d)

The evaluation of PTEM at a given position in the area will quantify the cumulative penalty of being at the position due to all the adversaries. Fig. 3.1 shows a sample PTEM constructed by a set of Gaussian and Uniform pdf s. Note that if the parameters of pdf s are constants, then PTEM is time-invariant.

3.3

Time-Variant Probabilistic Threat Exposure Map Recall that the probability of becoming disabled is time-dependent through tem-

poral pdf as in Eq. (3.9). While PTEM constitutes the position dependency of the probability of becoming disabled, PTEM can also be time-varying to formulate the timevariation of threat exposure level at a given position. For example, the area of operation

31

0.08 0.06 0.04 0.02 0 50

−60 0

−40 −20 0 20 40 −50

60

Figure 3.1. Probabilistic threat exposure map of an operation area. might consists of threats with time varying properties, e.g. the position or effectiveness of the threats might be changing with time. Another example is when obstacles are moving or restricted areas are updated over time. If all the threat sources are characterized by Gaussian pdf, time-variant PTEM formulation will be fR(t) (r) =

N X i=1

1 T −1 p exp − (r − µi ) Ki (r − µi ) 2 2π det(Ki ) 1

(3.12)

where 







2 0   µx,i (t)   σx,i (t) µi (t) =   , Ki (t) =   2 µy,i (t) 0 σy,i (t)

The time variation of the mean value µi (t) of the ith threat source represents the case where the concentration point (location) of the threat moves over time. As the variance Ki (t) varies with time, the area of effectiveness of the threat increaes or decreases.

32 3.4

Regions by the Level of Threat Exposure The area of operation can be divided into different subregions based on the value

of PTEM. The classification of a given location based on the subregion that it is in will facilitate the development of path planning and target tracking strategies, particularly the rule-based guidance algorithms. In this dissertation, the area of operation is divided in three subregions. Subregions may be constituted by disconnected zones depending on the structure of PTEM. As will be explained in the subsequent chapters, some strategies consider all three subregions while others use only one subregion in the formulation of decision factors.. (a) Region 1(A1 ): In this region, the value of PTEM is always less then or equal to fmax . A1 (t) = {r : f (r) ≤ fmax }

(3.13)

(b) Region 2(A2 ): This is the region where the value of PTEM is between fmax and fr . A2 (t) = {r : fmax < f (r) < fr }

(3.14)

(c) Region 3(A3 ): This is the region where the value of PTEM is greater than or equal to fr . A3 (t) = {r : f (r) ≥ fr }

(3.15)

Such regions quantified by the lower limit are also called restricted areas (Ar ), where the UAV should never enter. Note that the three regions are not fixed over time in the area of operation if the probabilistic map itself is time-variant with respect to position and effectiveness area. Nevertheless, if the position and effectiveness area of the threats are fixed but the effects of the threats are still time-variant, e.g. the level of threat exposure of the UAV increases as it stays longer in the area of operation, these three regions will be fixed over time.

33 3.5

Gradient Search on PTEM If the PTEM is differentiable, i.e. the area of operation is all modelled by Gaussians

or any other differentiable distribution functions, the gradient search approach, which is extensively used in robotics and optimization literature [142–149] can be employed. This determines the direction of minimum increase or steepest descent of the PTEM. In other words, it can be easily determined in which direction the UAV should move to minimize the threat exposure level or maximize the likelihood of avoiding a restricted region or a collision. Since the PTEM is constructed as the sum of differentiable functions, the determination of gradient or the sharpest-descent direction can be carried out by utilizing the sum of “directional” derivatives of f (r) given in Eq. (3.10) along the axes of the reference coordinate system of choice. Let u be a direction, at position r, whose angle from the positive x-axis is ψ. Thus, the vector defining direction u is u = cosψ Ib + sinψ Jb

(3.16)

b Jb are the unit vectors of x- and y-axes of the reference frame, respectively. where I, Then, the directional derivative of f (r) along direction u at position r is Du f (r) = fx (r) cos ψ + fy (r) sin ψ.

(3.17)

where fx (r) and fy (r) are the partial derivatives of f (r) with respect to x and y, respectively: ( ) 2 2 (x − µx,i )2 σy,i + (y − µy,i )2 σx,i x − µx,i fx (r) = − exp − 3 2 2 2πσx,i σy,i 2σx,i σy,i i=1 ( ) N 2 2 X (x − µx,i )2 σy,i + (y − µy,i )2 σx,i y − µy,i exp − fy (r) = − 3 2 2 2πσ σ 2σx,i σy,i x,i y,i i=1 N X

(3.18)

(3.19)

At a given position, to find the direction along which the threat exposure is reduced the most, i.e. the steepest descent, the directional derivative in Eq. (3.17) should be

34 minimized over angle ψ. Namely, “the minimizing direction”, umin , at position r in terms of its angle from the positive x-axis is ψmin (r) = argminψ Du f (r)

(3.20)

which yields “the minimizing angle” at position r as −1

ψmin (r) = tan

−fy (r) −fx (r)

(3.21)

Then, the vector representing the minimizing direction is umin = cos ψmin Ib + sin ψmin Jb

(3.22)

Note, further, that the gradient of f (r) is [150] ∇f (r) = fx (r)Ib + fy (r)Jb

(3.23)

In terms of the gradient, the minimum value of the directional derivative is given by min Du f (r) = −k∇f (r)k

(3.24)

Note that this is, in fact, the value of the directional derivative along direction umin , i.e. min Du f (r) = Dumin f (r) = fx (r) cos ψmin + fy (r) sin ψmin .

3.6

(3.25)

Minima of PTEM on a Closed Disk In this section, the formulation of minima of PTEM on a closed disk is derived.

This formulation will be utilized in the optimization of a cost function for a cooperation strategy that will be presented in Chapter 8. A closed disk centered at rc and with radius rd is defined as D(rc , rd ) = r : |r − rc | ≤ rd

(3.26)

35 Note that PTEM, formulated as f (r) in Eq. (3.10), is a continuous function of r, i.e. x and y. Thus, by the Extreme Value Theorem [150], there exists at least a point, rmin ∈ D such that rmin = argmin f (r)

(3.27)

which is either within D or on the boundary of D, bdD. First, local minima of PTEM are determined and the ones within D are selected. Further, PTEM has to be minimized on bdD. Among the local minima of PTEM and the minima on bdD, the minimum value is determined and the corresponding position is designated as rmin . Note that at a local minimum of PTEM the partial derivatives of f (r) with respect to x and y are 0 at the same time. Thus, the Eqs. (3.18) and (3.19) can be solved for local minima. Namely, fx (r) = 0

and

fy (r) = 0

(3.28)

However, the solution to Eq. (3.28) gives all local minima as well as maxima and saddle points. To identify the local minima among the solutions to Eq. (3.28), the secondpartials-test is applied. From Eqs. (3.18) and (3.19), the second partial derivatives of f (r) are obtained as ( ) 2 2 (x−µx,i )2 σy,i + (y−µy,i )2 σx,i (x−µx,i )2 1− exp − (3.29) fxx (r) = − 3 2 2 2 2πσ σ µ 2σ σ y,i x,i x,i x,i y,i i=1 ( ) N 2 2 2 2 2 X (x−µ ) σ + (y−µ ) σ (y−µy,i ) 1 x,i y,i y,i x,i 1− exp − (3.30) fyy (r) = − 3 2 2 2 2πσx,i σy,i µy,i 2σx,i σy,i i=1 ( ) N 2 2 X (x−µx,i )2 σy,i + (y−µy,i )2 σx,i (x − µx,i )(y − µy,i ) exp − (3.31) fxy (r) = − 3 3 2 2 2πσx,i σy,i 2σx,i σy,i i=1 N X

1

Figure 3.2 shows all the positions at which a PTEM has local minimum. Then, the set of positions within the closed disk at which a PTEM has local minimum is defined as SLM = r ∈ D : fx (r) = 0, fy (r) = 0, fxx (r)fyy (r) − [fxy (r)]2 > 0, fxx (r) > 0 (3.32)

36 10 local minimum

5

02 5

2 0.0

0.

8

6

0.025

4 0 .02 5 0.025

y(km)

5

0.02

2

0 5

02

0.02 5

−2

−4

5

0.02

−6

0.02 5 0.025

−8

−10 −10

0.0 25

0.

−8

−6

−4

−2

0 x(km)

2

4

6

8

10

Figure 3.2. Local minima of a PTEM. Now, the next step is to determine the minima of PTEM on bdD. Note that, for a given D, (x, y) coordinates of any point on its boundary can be parameterized by angle θ (See Fig. 3.3) as x = xc + rd cos θ

(3.33)

y = yc + rd sin θ

(3.34)

Substituting x and y from the above equations into Eq. (3.10) yields the formulation of PTEM with a single parameter θ for a given D. ( ) N 2 2 X (xc + rd cos θ − µx,i )2 σy,i + (yc + rd sin θ − µy,i )2 σx,i 1 exp − f (θ) = − 2 2 2πσx,i σy,i 2σx,i σy,i i=1 (3.35) Derivative of f (θ) gives the derivative of PTEM on bdD.

37 y r d

D (xc ,yc)

r

c

θ

r

x

Figure 3.3. Target coordinate system.

fθ (θ) =

N X i=1

( ) 2 2 (xc + rd cos θ − µx,i )2 σy,i + (yc + rd sin θ − µy,i )2 σx,i Υi exp − (3.36) 2 2 2σx,i σy,i

where Υi = −

1 3 3 4πσx,i σy,i

2 2 −2rd sin θσy,i (xc + rd cos θ − µx,i ) + 2rd cos θσx,i (yc + rd sin θ − µy,i )

Then, the set of positions on bdD at which PTEM has local minimum is defined as SBM = r ∈ bdD : fθ (θ) = 0, fθθ (θ) > 0

(3.37)

Finally, among the points in SLM and SBM , the ones with the smallest value of PTEM are Smin = {r ∈ SLM ∪ SBM : f (r) ≤ f (z), ∀z ∈ SLM ∪ SBM }

(3.38)

where note that the cardinality of Smin is possibly greater than 1.

3.7

Computational Efficiency of Evaluating PTEM Although a PTEM represents an entire area of operation including all the threats,

obstacles and restricted areas, it is not computationally demanding. This is, first, because its storage in each UAV will be carried out by simply pre-loading a set of numbers representing the parameters of the distribution functions used and/or updating them

38 during the mission when needed. Second, the guidance strategies will need to compute the values of PTEM only at few points and its directional derivatives only at the current UAV position at each execution instant. Of course, this will be done based on the preprogrammed probability distribution functions and their analytically determined partial derivatives in the on-board computer. Furthermore, when new information about the area is obtained during a mission, the corresponding PTEM can be easily updated by either varying the parameters of the distribution functions or adding new ones. Note here that adding a new function is simply increasing the upper limit of the summation, which is carried out using a single function call with different parameters, for both evaluation of the map and of the partial derivatives.

CHAPTER 4 TRAJECTORY PLANNING ALGORITHM To be able to guide the autonomous UAVs from initial location to target locations, an algorithm that can plan a route between the two locations is required to let the UAV navigate itself along the planned route. This route is also called “path” when it is not time-variant, while in most practical scenarios a “trajectory” is needed, which defines a position at a given time. Further, if the trajectory to be planned is in an adversarial environment, then there should be trade-offs between conflicting objectives.

4.1

Trajectory Planning Strategy (TPS) In the area with multiple time varying threats, the trajectory planning for a UAV

to go to a prespecified target location has three main objectives: (i) avoid restricted areas, (ii) minimize the threat exposure level, (iii) reach the proximity of the target, where the first objective has a higher priority. While avoiding the restricted areas and minimizing the threat exposure, proximity of the target should be reached. Proximity of the target to be reached is a strategy parameter, which is specified before the UAV starts its mission. By strategy parameter, it is meant that the parameter is not imposed by the UAV dynamics and can be changed according to the mission requirements. In other words, by the proximity of the target we define how much the UAV should get closer to the target. To achieve these objectives, the strategy generates commanded heading, ψcmd , and speed, Vcmd , for the UAV. 39

40 4.1.1

Dynamic Constraints of the UAV While generating the commanded signals, the strategy should take into account

the dynamic constraints of the UAV, which are: (a) Heading change constraint: The UAV can not change its heading instantaneously. The rate of heading change is limited due to the dynamics. This is modeled as a maximum heading constraint, ψmax , which is the maximum heading angle change the UAV can make in each simulation step. ψmax is assumed to be constant regardless of the speed of the UAV, but note that in practice it might be function of speed as well as other states of the UAV. Based on the ψmax a heading constraint cone, which moves with the UAV, is defined to be the range of admissible heading angles (see Fig. 4.1). It is bounded by ψR and ψL that are defined to be the maximum turning angle to the right and left, respectively. Note that, counter-clockwise is the positive direction for all the angles. In this cone, PTEM is evaluated at a number of points equally distributed over an arc of a circle, which are called possible flying points. ψ

UAV Target position

ψ

L

ψ

ψ

R

max

ψ

t

UAV

Figure 4.1. Heading search cone and target direction.

41 (b) Minimum and maximum speed constraints: The UAV has a flyable minimum, Vmin , and maximum, Vmax , speeds. Vmin is chosen to be 30 percent greater than the stall speed of the UAV. Note that, the speed constraints are also assumed to be constant, but in practice they are also functions of the UAV states. (c) Minimum and maximum acceleration constraints: Acceleration capability of the UAV is limited. It is quantified by minimum and maximum acceleration constraints amax and amin . Note that negative value of acceleration implies deceleration.

4.1.2

Strategy Design To achieve the main objectives in the order of priority, the strategy should command

the UAV’s heading and speed while considering the dynamic constraints. Since the UAV is flying in an area with multiple threats with time varying properties, PTEM is a function of time as well as position. Thus, minimization of the threat exposure requires search of the minimizing point in both position and time where the UAV can possibly fly. If the UAV maintains its current speed (Vc ), the incremental distance that the UAV travels from its current position Pi to the next position Pi+1 until the next computation time, is computed by the current speed and the simulation time step 4t (see Fig. 4.2). Furthermore, the incremental distance and the heading of the UAV along with the current

Vc

P

t

i+1

ψ

.

Pi

Figure 4.2. Incremental distance in one simulation step.

42 position will determine the next position of the UAV. In other words, the next position of the UAV depends on the choice of the heading, which will also effect the threat exposure of the UAV. This is because, the next position of the UAV and the time at which the UAV will reach that position will determine the level of threat exposure since the threat exposure is a function of position and time. When the minimum level of threat exposure is sought, then the probability of becoming disabled should be evaluated at various positions and time instants. As mentioned above, the next position of the UAV can be varied by changing the heading of the UAV and the time at which to reach the next position can be varied by changing the speed of the UAV.

4.1.2.1

Admissible Range of Constraints

Note that, the variation of the heading is limited by the heading constraint, ψmax , i.e. the heading can vary only within the heading constraint (HC) cone. Thus, the HC cone is the range/set of admissible heading angles. In addition to limitation on the heading, the variation of the speed is also limited by both speed constraints and acceleration constraints. The speed variation is limited by acceleration constraints because the UAV needs to reach a position (next position) in a specified amount of time (simulation step time). The acceleration range that will not violate the speed constraints of the UAV is calculated as follows. Assuming that the UAV maintains a constant acceleration in the next simulation interval aVmax =

Vmax − Vc , 4t

(4.1)

which is the acceleration that will increase the UAV speed to its max value. aVmin =

Vmin − Vc , 4t

(4.2)

43 which is the acceleration that will reduce the UAV speed to its minimum. Using aVmax and aVmin , the acceleration range due to the speed constraints is defined Ra1 = [aVmin , aVmax ].

(4.3)

The UAV acceleration should be in this range because, otherwise the UAV speed will be higher than Vmax or lower than Vmin , which is a direct violation of the speed constraints. The second acceleration is due to the acceleration constraints

Ra2 = [amin , amax ].

(4.4)

The admissible acceleration range is the intersection of these two ranges

Ra = [alower , aupper ] = Ra1 ∩ Ra2 .

(4.5)

which implies that

alower =

   aV , aV ≥ amin min min

(4.6)

   aV , aV max max ≤ amax

(4.7)

  amin , aVmin < amin .

aupper =

  amax , aVmax > amax .

where alower is the lower limit and aupper is the upper limit on the admissible acceleration range. Considering the admissible acceleration range, the admissible speed range is obtained as follows;

RV = [Vlower , Vupper ]

(4.8)

44 where

4.1.2.2

Vlower = Vc + alower 4t

(4.9)

Vupper = Vc + aupper 4t.

(4.10)

Discretization of Admissible Ranges

As stated earlier, when there is a risk of higher threat exposure level, then a compromise should be made between the shortest flight trajectory and the trajectory with the lowest threat exposure. When the trade-off is required between directly going towards the target with the current speed or minimizing the threat exposure level by changing the heading and speed without violating the dynamic constraints, then the heading and speed from their corresponding admissible ranges should be determined to minimize the threat exposure level. To find the minimizing pair of heading and speed, the heading and speed ranges are discretized, which results in search points in heading and speed ranges. Points in the heading range are called “heading search points” and points in the speed range are called “speed search points”. These search points are numbered to specify the order in which the probability of becoming disabled is evaluated. In other words, the search is conducted in two directions, one with the heading variation, other with the speed variation. Thus, there are two sets of numbering: one for heading (heading numbering) and the other for speed (speed numbering). Since the final objective is to go to the target position, the preferred heading for the UAV is ψt , which is the heading directly towards the target. Moreover, the preferred speed of the UAV is its current speed to prevent any unnecessary acceleration or deceleration. Using ψt and HC cone, a reference heading (ψref ) is determined. If the target direction (ψt ) is in the HC cone, the reference heading is the target direction. If the target direction is not in the HC cone, then the permissible largest left/right turn

45 which will get the heading of the UAV closer to the target direction will be the reference heading (see Fig. 4.3). ψt

Since strategy wants the UAV to get to the target as soon

ψref 1

2

3

ψt , ψref 5

ψc

4 5

11 3 1

ψc

2 4

6

5 4

8

9 10 11

ψc

8 6

7

8

9

7

6

7

10

3

9

2 10

1

ψref ψt

Figure 4.3. Numbering of the points and reference heading.

as possible, the numbering for the heading starts from the reference heading. When the reference heading is aligned with right/left edge of the heading search cone, it continues on to the next point to the right/left respectively, and so on. If the reference heading is within the heading search cone, it moves to the next point to the right, then the next point to the left, then again to the second point on the right, to the second point on the left (if there is any), and so on (see Fig 4.3). For the speed, the numbering starts from the point corresponding to zero acceleration or current speed as shown in Fig. 4.4 and it continues on to the next point to the up/down respectively, and so on. As stated earlier, distance between the heading search points (points on Arc1 in Fig. 4.4) and the current UAV position is Vc 4t. In addition to the heading search points, another set of points is defined with the same heading but with distance of 2 Vc 4t. These are the points on Arc2 in Fig. 4.4. Additionally, at the points on Arc2, PTEM is evaluated only for the current speed or zero acceleration to be able to monitor any approaching high threat region. This evaluation at the points on Arc2 is conducted

46 V y Vupper

10 8

ψ

6

7

5

2 1

3

1

2

3

4

5

UAV

,Vc

ref

4

6

7

Arc1

9 11

Arc2

Vlower x

Figure 4.4. Heading and speed search. in the order of heading numbering. If the probability of becoming disabled is higher than fr at a searching point on Arc 2, then its corresponding point on Arc1 is eliminated. Additionally, the evaluation of PTEM is conducted through the speed search points in the order of the speed numbering at each heading search point that is not eliminated in the previous evaluation. During the search in the direction of speed search points, the search stops when one of the following occur: (a) local minimum in speed direction is found (b) PTEM becomes less than fmax This search procedure is repeated for all the heading search points in the order of the heading numbering. The evaluation of PTEM stops when one of the following occurs: (a) minimum PTEM for the heading search point is greater than the one from the previous search (b) PTEM becomes less than fmax Note that if a value of PTEM less than fmax is found in any speed search, the strategy ends the search and chooses the corresponding speed and heading.

47 4.1.2.3

Strategy Cases

By considering the three threat level (the subregions) of PTEM described in Chapter 3-Section 3.4 and the dynamic constraints of the UAV, the strategy explained above results in 5 cases while evaluating the search points to command the heading and the speed of the UAV. (a) CASE 1: In this case, all the search points on Arc1 are in A1 . Thus, there is no concern about the UAV becoming disabled. Thus, the strategy should steer the UAV directly to the target position. This is done by commanding the reference heading and the current speed of the UAV. (b) CASE 2: In this case, some of the search points on Arc1 are in A1 and some of them are in A2 . Since some of the search points are in A2 , by conducting the search procedure explained above, the strategy tries to minimize the threat exposure if the search point corresponding to the reference heading is in A2 . If the PTEM value corresponding to the reference heading and current speed is less than fmax , i.e. in A1 , obviously the reference heading and current speed will be commanded. (c) CASE 3: In this case, all the search points on Arc1 are in A2 and none of the search points in Arc2 are in A3 . Thus, the position corresponding to the reference heading with current speed is definitely in A2 . Thus, the strategy will conduct the search explained to find the minimizing pair of heading and speed to command. (d) CASE 4: In this case, some of the search points on Arc2 are in A2 and some of them are in A3 . This is the case where the UAV is in a low risk of getting into Ar . To prevent the risk of getting closer the Ar , the search points of Arc2, which are in A3 are marked and the corresponding points on Arc1 are eliminated. The search is conducted at the remaining points to find the minimizing pair of heading and speed to command.

48 (e) CASE 5: In this case, all the search points on Arc2 are in A3 . This is the case where the UAV is in a high risk of getting into Ar . To prevent the UAV from getting into the Ar , the sharpest possible turn should be taken. To achieve this, the search is conducted only at the max right/left heading to find the minimizing pair of heading and speed to command.

4.2

Comparison with the other strategies To assess the benefit of adjusting the UAV speed as well as heading based on the

information about the time variation of PTEM, the trajectory planning strategy (TPS), explained in the previous section, is compared with two other strategies, explained below. (i) Strategy for Commanded Heading (SCH): With this strategy, at each computation, the current PTEM is used to find the heading that would minimize the threat exposure level. The strategy uses the same algorithm as TPS does to conduct only the heading search. Even if this strategy is provided with the current map, it does not know or use the time variation of the map. Thus, it does not adjust the UAV speed and the UAV maintains its initial speed throughout the mission. (ii) Strategy with Initial Map (SIM): This strategy is the same as SCH except it is provided with only PTEM of the area at the initial time. Thus, even if PTEM is changing, it makes decisions for the commanded heading based on the initial status of the threats. Furthermore, since it does not carry out speed search it also commands the initial speed throughout the mission. In the simulation case for the comparison, two Gaussian pdf s are considered to be time-variant. One of the pdf s is changing the concentration point (location), i.e, the mean (µx ,µy ) of the pdf of the threat is located at (−3, −29.5) km initially and moving along the positive y-axis as shown in Figures 4.7-4.12. The other Gaussian pdf located at (−20, 0) km is changing the area of effectiveness between 1800 and 2200 seconds,

49 i.e, the variance (σx ,σy ) of the pdf is (1.9, 1.8) km initially. As the time increase from 1800 seconds to 2000 seconds the variance decreases linearly to (0.9, 0.8) km and goes back to its initial value linearly between 2000 and 2200 seconds. The TPS trajectory is shown with solid line, SCH trajectory is shown with dashed line and SIM trajectory is shown with dashed-dotted line in Figures 4.7-4.12. For this simulation, fmax and fr are selected to be 0.0001 and 0.005, respectively, which are also shown in Figures 4.7-4.12. The minimum UAV speed for the mission is selected to be 1.3Vstall where Vstall is 30 m/sec and the maximum UAV speed is 100 m/sec. In all the strategies, the UAV starts the mission with a speed of 50 m/sec. The minimum and maximum acceleration constraints are −10 m/sec2 and −10 m/sec2 , respectively. Maximum heading change (ψmax ) in a simulation step (3 seconds in the cases presented) is 25 degrees. As seen from Figures 4.7 all the three strategies commands the same trajectory up to 1400 seconds. This can also be seen from the Figures 4.5 and 4.6 that they have the same speed and threat exposure level up to 1400 seconds. At 1400 seconds, the UAV with the TPS starts increasing its speed to pass through the region that the threat is approaching to. In other words, it tries to pass through the region while the threat exposure level is less. Since the threat is approaching to the UAV , the UAV also starts maneuvering to the right at 1600 seconds to avoid the high threat exposure level and reaches a speed 20% higher than its initial speed (see Fig. 4.5). In other words, since the threat exposure level is increasing in the direction the UAV is heading, the UAV speeds up and maneuvers to avoid it. Since SCH uses only current PTEM, it recognizes the approaching threat later than the TPS does and starts to maneuver later. Moreover, it cannot adjust its speed. Thus, it takes a wider turn to avoid the threat but it is exposed to a higher level of threat than the TPS. However, SIM is exposed to the highest threat level, since it does not know anything about the time-variation of PTEM and passes through the threat region, which is shown in Fig. 4.8. It is also shown in Fig. 4.6 that the

50 threat exposure level increases significantly in SIM as the threat approaches where the SCH has less and the TPS has the least threat exposure level. Besides, even if the UAV with TPS is exposed to an approaching threat region, it gets out of that region quicker than the others because of the speed adjustment. 230 TPS SCH SIM 220

210

200

V(km/h)

190

180

170

160

150

140

0

500

1000

1500

2000

2500

3000

3500

t(sec)

Figure 4.5. Speed change of the UAV guided by the strategies TPS, SCH and SIM.

Figs. 4.9 and 4.10 show that as the UAV approaches the second target position, the TPS recognizes that threat exposure level is decreasing in the area the UAV is approaching. This is because a threat located at (−20, 0) km (the second time-variant one) is decreasing its area of effectiveness. Thus, it decreases its speed to the minimum to wait for the threat level to drop before it enters the area. After flying with a minimum speed for about 150 seconds, the UAV realizes that the effectiveness area of the threat starts increasing again and the TPS thus commands a speed 80% higher than the minimum speed to get to the second target position and leave the region as soon possible. And

51 this speed is maintained since there is no any other significant change in PTEM until the UAV gets to the final target. On the other hands, the UAV guided by the SCH finds itself in a high threat region since the threat starts increasing the effectiveness area after 2000 seconds. Since the UAV is directed and close to the second target position, it keeps flying under this exposure for about 200 seconds (see Fig. 4.6). The UAV with SIM tries to minimize the threat exposure according to PTEM of the area at the initial time. After reaching the second target position, again all the strategies command pretty much the same trajectory since there is no time variation for the threats affecting this region. Since there is no time variation, the UAV guided by TPS maintains its speed and reaches the last target position earlier. The total time for TPS, SCH and SIM are 2880 seconds, 3201 seconds and 3141 seconds, respectively. 0.06 TPS SCH SIM 0.05

0.04

0.03

0.02

0.01

0

0

500

1000

1500

2000

2500

3000

3500

Figure 4.6. Threat exposure level along the trajectory of each strategy.

52 Finally, all the UAVs guided by three different strategies reach the last target position. To evaluate the overall performance of the strategies, the tthreat exposure level can be calculated by integrating the curves in Fig. 4.6. By doing so, it is shown that TPS trajectory has the least threat exposure level of 0.1221. The SCH trajectory has the total threat exposure level of 0.1552 and the SIM trajectory is the longest and has the worst total threat exposure level of 0.2746. The trajectory lengths for TPS, SCH and SIM are 158.03 km, 160.20 km and 157.20 km, respectively. time is 1401sec 40

0.0001

0.0 0.000501

0.0

5 0.00

05

0. 00 0

0.005 05 0.0 01 0 0.0

05

.0

0 05 0.00 15 0.000

0.0

01

−20

1 00

1

0.005

0.0 0

0.0

00

−10

0.0

5 00

0.

1 00 0.0 1 0 0 .0 0

0.00 01 1

0.0

5

0

0 0.0

0.00.00 00 5 1

0. 00 01

y(km)

01 00

0.

0.00

01

0

00 1

10

05

0 0. 1 .000

05

0.0001

20

0.0001 0.0

05

05

0.0

0.0

30

−30

−40 −40

−30

−20

−10

0 x(km)

10

20

30

40

Figure 4.7. Comparison of the trajectories at 1401 sec.

53 time is 1650sec 40

30

0.0

01 0. 00

1 00 0.0

5

0.0 05

01 0.

05 0.0

5

0. −20

0.00

0.005 0.0001

1

0 .00 0.00001

−10

0.

00

01

5

0

1 00 0.0

0.005

00 0.005 01

0.005

0.0001 0.005

00

y(km)

0.0001

1 00 1 0. 5 0.0 0.000 000.00 01

0.005

00

0.

10

00

0.

0 0.0 001 .00 5

0.005

05 0.0001

20

0.0001

01.00 0 5 .00

0

−30

−40 −40

−30

−20

−10

0 x(km)

10

20

30

40


time is 1851sec 40

0.

30

00

5

0.0001

1

00

0 0.

0 0.

05

05

1 00

0.0001

1

0 01

00 0. 0.0001

−10

0.005 1 0.0001 00 5 0.00.00

05

00 0.0 1

05 0.0

0.0

−20

0.0 05

0.005

0.000

y(km)

5 0.000001 0.

0.0

0.0

001

0.0

10

0.0001 0. 000.005 01 0.0 5 0 .0 00 0 1 0.0 05

0. 00 5 01 0.00

0.0001 0.0 05

20

05 0.0

0.0001

01 0.00

−30

−40 −40

−30

−20

−10

0 x(km)

10

20

30

40


54 time is 2001sec 40

1 0.0005 0 0.0

30

0.00 01 0.005

0.00

0.

5

0.0001

0

0.0 0

y(km)

5 0.0001

005 0.00 1 0.0

01

1

1

00

00

0.0

0.00

10

0. 00 5

1 0.000

0.0

0.0001

00

5

0.0001 5

1 00 0.0

0.

00 5

0.00

05 0.0

0.005

1

−10

0.000

−20

0.0 0.005 00 1

0.005

0.005 0.0001

20

1

0 00

0. 0.0 05 0.0001 5 .00

0.0001

0 −30

−40 −40

−30

−20

−10

0 x(km)

10

20

30

40


time is 2151sec 40

0.0001

0

0.0001

01

00 0. 0.0001

01

1 00 0.00.005

0.005

01

00

0 1 .0001 0.000 0.005

05

0.0

1 00 0.0

0.0 0.0 00 05 1

0.0001

1 05 00 0.0 0.0

0. 00 5

−20

0.00

0.0 00

05

00 5

−10

05

0.0

0.

0.

0.005

y(km)

0.0 00 1

0.0

0.0 00 1

05 0.0

10

01

05

1

0.0

0.0001

0.00051 0.00

0.

20

5 00 0. 0.005

00

30

−30

−40 −40

−30

−20

−10

0 x(km)

10

20

30

40


55

time is 3201sec

00 0. 0.0001 5 0.00

0.005

05 0.0 001 0.0

0.0001

0.005

01

01

1 00 0.0

−20

0.005

0.00.00 00 5 1

0.

05 0.0

5

1

0 0.0 0.0001

−10

0.0 0.005 001

05

00

00 0

01

00

0

0.0

0.

0.

y(km)

5

0.0001 5 0.00

00

0.

5

0.00

00

0.

10

0.00 01 0 0.005 .00 01

001

0.005

20

0.0

30

0.0001

01

0.00

5

40

0.0

001

05 001 0.0

0.0

−30

−40 −40

−30

−20

−10

0 x(km)

10

20

30

40


CHAPTER 5 ESTIMATION In this dissertation, it is assumed that the UAVs are equipped with sensors that can measure only target position. However, the guidance algorithms cannot use the noisy measurements. Furthermore, they need the states of the target motion in addition to the ones that are measured. Thus, an estimation algorithm is needed to filter the noise from the measurements and, further, to estimate the speed and heading states of the target in addition to the position. The estimation algorithm is also used to predict the motion of the target when no measurement is available. When the target is lost, for example, while conducting a maneuver to avoid a restricted area, the guidance algorithms can use the predicted states of the target to steer the UAV until the target enters back into the sensor range. In most cases, a model for the motion of the target is not available or determination of what model the target is moving is not possible. Thus, in this research, dynamic model based estimation techniques are not used. Instead, a second-order leastsquare estimation technique is used. Let Te be the “estimation update period”, i.e., the estimation is executed only at every Te seconds. The position of the target is assumed to be measured in terms of x and y coordinates by sensors onboard the UAV. x˜t (k) = xt (k) + νx (k)

(5.1)

y˜t (k) = yt (k) + νy (k)

(5.2)

where x˜t (k) and y˜t (k) are measured target position, xt (k) and yt (k) are actual target position and νx (k) and νy (k) are the samples of Gaussian white noise on x and y measurements at the k th (current) instant, with standard deviations σx and σy , respectively. 56

57 5.1

Estimation via heading and speed In this approach, the estimation algorithm is developed assuming that the heading

and speed measurements of the target are available. However, as stated earlier, target position is measured, not the speed and heading. Even though the heading and the speed of the target are not measured, the estimation models can be developed for the estimation of heading and the speed. Then by using the estimated heading and speed, the predicted position of the target is computed. Since the estimation model requires “measured” heading and speed, they are computed from the measured positions as follows; y˜t (k) − y˜t (k − 1) −1 ˜ ψt (k) = tan x˜t (k) − x˜t (k − 1) p (˜ yt (k) − y˜t (k − 1))2 + (˜ xt (k) − x˜t (k − 1))2 V˜t (k) = ∆t

(5.3) (5.4)

where ψ˜t (k) and V˜t (k) are considered to be “measured” target heading and speed values at the k th instant. Initially, the estimation strategy assumes a batch mode of processing where all previous samples are used to produce estimates. The more samples usually mean better estimate. However, the problem with the batch processing is that as more data arrives, calculations will have to be repeated with a larger amount of data, which will result in a computational and memory burden. To resolve this issue, the batch processing is used only until a certain amount of measurement data is obtained. Once a pre-specified number (np ) of measurements are obtained, then a sequential mode of estimation is initiated. For the sequential estimation, only the previous estimation and the current measurement are needed.

58 A second-order estimator is used to estimate the heading and the velocity of the target with the measurement model ˜ ψ(k) = H(k) Xψ (k) + νψ (k)

(5.5)

˜ V(k) = H(k) YV (k) + νV (k)

(5.6)

where νψ (k) and νV (k) are the arrays of noise samples, and measurement arrays/windows, estimation parameters and basis function are given as follows. ˜ ψ(k) =

˜ V(k) =

T

ψ˜t (1) ψ˜t (2)· · ·ψ˜t (np − 1)

Xψ (k) =

V˜t (1) V˜t (2)· · ·V˜t (np − 1) T

YV (k) =

dV (k)

eV (k)



1

Te

aψ (k)

    H(k) =    

bψ (k)

T

cψ (k)

fV (k)

T Te2 Te2

1 .. .

2 Te .. .

4

1

(np − 1) Te

(np − 1) Te

.. .



       2 

Note that, since heading and speed measurements are calculated using nonlinear equations, νV (k) and νψ (k) are not zero mean white noise, although they are treated as zero mean white noises in the formulation of the estimation. To start linear least squares estimation, at least 4 position measurements are needed. The equations for the estimation [4] are: (a) Batch estimation −1 ˜ H(k)T H(k) H(k)T ψ(k) b V (k) = H(k)T H(k) −1 H(k)T V(k) ˜ Y b ψ (k) = X

(5.7) (5.8)

59 b ψ (k) and Y b V (k) contain the estimated coefficients for the second-order where X estimator.

(b) Sequential estimation h i b b b ˜ Xψ (k + 1) = Xψ (k) + K(k + 1) ψ(k + 1) − H(k + 1)Xψ (k) (5.9) −1 K(k + 1) = P(k)HT (k + 1) H(k + 1)P(k)HT (k + 1) + W −1 (k + 1) (5.10) P(k + 1) = [I − K(k + 1)H(k + 1)] P(k)

(5.11)

where K is the Kalman gain matrix, which is used to modify the previous best correction bψ (k) by an additional observation to account for the information in the (k + 1)th X

measurement subset. P is the covariance update matrix and W is the noise covariance. b1 and covariance We start the sequential estimation process by an a priori estimate, X estimate P1 , which are obtained at the end of the batch processing.

Using the batch and sequential processes, heading and velocity estimates of the target are computed by ψbt (k) = b cψ (k) (k Te )2 + bbψ (k) k Te + b aψ (k)

Vbt (k) = fbV (k) (k Te )2 + ebV (k) k Te + dbV (k)

(5.12) (5.13)

From the estimated heading and velocity, the predicted position of the target at the next update instant is calculated by x bt (k + 1) = xt (k) + Vbt (k)cosψbt (k)∆t

ybt (k + 1) = xt (k) + Vbt (k)sinψbt (k)∆t

(5.14) (5.15)

where the previous position (xt (k),yt (k)) are from the previous measured position when measurement was available. If there is no measured position from the previous computation time, then the previous estimated position (b xt (k),b yt (k)) is used.

60 5.2

Estimation via position It is known that using nonlinear equations for the calculation of “measured” heading

and speed from the measured positions changes the noise characteristics of the noisy measurements. Thus, instead of estimating heading and speed of the target directly, in this section, the position of the target is estimated by using position measurements. Then, the kinematic equations are used to calculate the heading and the speed of the target. For this estimation again the least-squares estimation technique with batch processing mode, based on a sliding window of measurements, is employed. Namely, only a specified number of measurements is stored and the measurement array is updated with new measurement by removing the oldest measurement and thus retaining the size of the array. In fact, this is essential when there is a transition between different mode of motions. However, note that, there is no attempt made in this work to determine the optimal size of the measurement window. A second-order estimator is used to estimate both x and y positions of the target with the measurement model x ˜(k) = H(k) Xx (k) + νx (k)

(5.16)

y ˜(k) = H(k) Yy (k) + νy (k)

(5.17)

where νx (k) and νy (k) are the arrays of noise samples, and measurement arrays/windows, estimation parameters and basis function are given as follows. x ˜(k) =

y ˜(k) =

x˜t (km − n + 1) x˜t (km − n + 2)· · ·˜ xt (km )

T

y˜t (km − n + 1) y˜t (km − n + 2)· · ·˜ yt (km )

T

61 Xx (k) =

Yy (k) =

dy (k)



1

    H(k) =    

ax (k)

bx (k) ey (k)

cx (k)

T

fy (k)

T

1 .. .

(km − n + 2) Te .. .

2 (km − n + 1) Te 2 (km − n + 2) Te .. .

1

(km Te )

(km Te )2

(km − n + 1) Te

        

where the superscript T denotes the matrix transpose, km corresponds to the time instants when the measurement is taken and n = min{nsw , km }, i.e. the size of measurement window will increase by 1 at every time instant when the measurement is taken until the maximum size (nsw ) is reached. To start linear least squares estimation with a second-order estimator, at least 3 position measurements are needed. Note that, the previous estimator needs at least 4 position mesurements. This is because at least 4 position measurements are needed to calculate 3 heading and speed “measurements”. The equations for the estimation are: −1 H(k)T H(k) H(k)T x ˜(k) b y (k) = H(k)T H(k) −1 H(k)T y Y ˜(k)

b x (k) = X

(5.18) (5.19)

b b where X(k) and Y(k) contain the estimated coefficients for the second-order estimator. Thus, the estimated positions at the current update instant will be x bt (k) = b cx (k) (k Te )2 + bbx (k) k Te + b ax (k)

ybt (k) = fby (k) (k Te )2 + eby (k) k Te + dby (k)

(5.20) (5.21)

62 Since we have the position estimates as functions of time, we can also use them to estimate the speed and heading of the target. Note that x b˙ t (k) = 2 b cx (k) k Te + bbx (k)

yb˙ t (k) = 2 fby (k) k Te + eby (k)

(5.22) (5.23)

Using the kinematic equations, estimated target speed and the heading at the current update instant can be written as q

2 2 x b˙ t (k) + yb˙ t (k) h i −1 ˙ ˙ b ψt (k) = tan ybt (k)/x bt (k)

Vbt (k) =

(5.24) (5.25)

Also note that we can use the position estimation functions to predict the position of the target at the (k + 1)th (next) update instant by simply replacing k with k + 1 in Eqs. (5.20) and (5.21). Further note that x bt (k + 1) and ybt (k + 1) are the components of

b rt (k + 1) in Eq. (7.4). 5.3

Comparison of the estimation algorithms While both estimation algorithms use a least-squares estimation technique, they are

different in the following aspects. (i) The first estimation initially starts in batch mode and then switch to sequential mode. The second estimation is always in batch mode with a sliding window. (ii) The first estimation uses the kinematic relations to compute, from the position measurements, the “measured” speed and heading, based on which the speed and heading are estimated by the least-squares. Then, the estimated position is computed by the kinematic relations. The

second approach directly estimates the

position from the position measurements. Fig. (5.1) presents the comparison of the two estimation approaches in a simulation case. In this case, the target continuously turns right while its speed increases until 900 sec and the measurement is lost at 700 sec. As

63 Estimated heading of the target ψ(deg)

400 300 200 100 0

100

200

300

400

500 600 700 t(sec) Estimated speed of the target

800

900

1000

100

200

300

400 500 600 700 t(sec) Estimated position of the target

800

900

1000

V(km/h)

200 150 100 50 0 4

x 10

OLD NEW TARGET

y(m)

1 0 −1 −15000

−10000

−5000 x(m)

0

5000

Figure 5.1. Comparison of the estimation algorithms. can be seen from Fig. (5.1), the new estimation is better because the heading estimation converges faster than that of the old estimation, the speed never converges with the old estimation, and the position estimation in the new approach does not deviate as much after the measurement is lost.

5.4

Data Fusion and Estimation In Section 5.2, an estimation algorithm is developed using the least-squares estima-

tion technique with batch processing mode, based on a sliding window of measurements obtained by sensors onboard a single UAV. In this section, a data-fusion algorithm is developed and will be used when multiple UAVs, pursuing the same target, can share their measurements of the target. It is assumed that the UAVs can transmit their onboard measurements of the target to other UAVs in the team when they are in communication range. Measurements received from other UAVs are used onboard a UAV to fuse with or substitute for its own measurements.

64 The position of the target is assumed to be measured in terms of x and y coordinates by sensors onboard the ith UAV when the target is in sensor range x˜t,i (k) = xt (k) + νi (k)

(5.26)

y˜t,i (k) = yt (k) + νi (k)

(5.27)

where x˜t,i (k) and y˜t,i (k) are measured target position by the ith UAV, xt (k) and yt (k) are the actual target position, and νi (k) is the sample of Gaussian white noise on x and y measurements with standard deviation σi onboard the ith UAV at the k th discreet time instant. A second-order least-squares estimator was designed in Section 5.2 to estimate both x and y positions of the target based on the measurement arrays/windows x ˜(k) =

y ˜(k) =

x˜t,i (km − n + 1) y˜t,i (km − n + 1)

x˜t,i (km − n + 2) y˜t,i (km − n + 2)

··· ···

x˜t,i (km )

T

y˜t,i (km )

T

Note that in Section 5.2, the measurement arrays/windows for the ith UAV were constructed only by measurements taken onboard. In the case of a team of cooperative UAVs communicating their measurements of the target, the measurement arrays/windows for a UAV will include measurements from the other UAVs. Since the same technique is used for y-measurement array, the following discussion is limited to x-measurement array. The x-measurement array for the ith UAV in the case of communicating UAVs is defined as x ˜(k) =

x˜0t,i (km

− n + 1)

x˜0t,i (km

− n + 2)

···

x˜0t,i (km )

T

(5.28)

65 where

x˜0t,i (l) =

   x˜t,i (l)       2 σj

x˜ (l) + σj2 +σi2 t,i         x˜t,j (l)

, case-1 σi2 σj2 +σi2

x˜t,j (l)

, case-2

(5.29)

, case-3

where l ∈ {km − n + 1, · · · , km}, case-1 is when the ith UAV has only its own measurement, case-2 is the optimal fusion [4] of measurements when the ith UAV has its own measurement and the measurement from the j th UAV, and case-3 is when the ith UAV does not have its own measurement but receives measurement from the j th UAV. Note that in case-2, it is assumed that the ith UAV knows the covariance of measurements onboard the j th UAV as well as its own. Note also that x˜t,i (l) and x˜t,j (l) are measurements of the target taken at the same real time even though x˜t,j (l) is received by the ith UAV after a delay. In this research, it is assumed that the communication from one UAV to another takes one update period. It is also assumed that the communication brings the time stamp of a measurement along with the measurement itself. Thus, measurements from other UAVs, even though received with delay, are fused with onboard measurements that were taken at the same time and stored in the measurement array. Similarly, if no measurement was taken onboard at the time specified by the time stamp of the received measurement, the received measurement is inserted at the entry of the measurement array as indicated by the accompanying time stamp. To demonstrate the benefit of fusing measurements from other UAVs, two simulation cases were run with two UAVs. In both cases, the target starts at initial po-

66 Fitting error for x position estimate of UAV1 60

50

fitting error

40

30

20

10 UAV1 measurements only Both UAV measurements 0 0

100

200 t(sec)

300

400

Figure 5.2. Comparison of fitting error of x position estimation for UAV-1. sition (−5000, −20000) meters and moves on a trajectory described by the speed in meter/second and the heading in radian as functions of time Vt (t) = 1000 [− sin(0.00015 t) + 0.099] 3 t t ψt (t) = 1 − + + 3 sin(0.002 t) 220 615

(5.30) (5.31)

The simulation time was 450 seconds and the update period was 3 seconds. UAV-1 has a sensor with noise covariance σ12 and UAV-2 has 4 σ12 , which means that measurements by UAV-2 are more noisy. In both cases, the size of the measurement arrays is 50. In the first case, the UAV-1 uses only its own measurements and, in the second case, it fuses its measurements with the measurements sent from UAV-2. The performance of the two cases are compared based on the fitting error of the least-squares estimation.

67 Using the estimate in the quadratic error that is minimized by the linear leastsquares estimation technique, the fitting error, a scalar and dimensionless quantity, is defined as the sum of the normalized residuals [4] 1 J (l) = 2 σ1 ∗

l X

km =max{1,l−49}

h

i2 b m) x˜t,1 (km ) − H(km )X(k

(5.32)

2 b m ) = [a(km ) b(km ) c(km )]T is where H(km ) = [1 km km ] is the basis function and X(k

the estimates of the coefficients of the second-order model. Note that the lower limit and as well as the upper limit of the summation move according to the moving measurement array of size 50. Figure 5.2 shows the fitting errors during the simulation of the two cases. Since the size of the measurement array increase during the first 150 seconds (In the simulation the sampling period is 3 seconds and the maximum size of the measurement array is 50), the values of the fitting errors, as the sums of the residuals, also increase. During the entire simulation, the fitting error of the estimation based on data fusion is consistently smaller than that with only UAV-1 measurements, even though the fusion is performed with measurements that have higher noise covariance. Therefore, it is beneficial to perform data fusion whenever the measurements from other UAVs are available.

CHAPTER 6 POINT SEARCH GUIDANCE ALGORITHM When the target involved in a mission is mobile, this requires the planning of a trajectory to follow the target given the estimated and predicted states of the target and a map of the environment. Further, the dynamic constraints of the vehicle(s) should be taken into account and obstacles/threats and restricted regions should be avoided.

6.1

Target Following Strategy Following a target in an area with multiple threats requires a trade-off between

two possibly conflicting objectives:(i) to maintain the proximity of the target and (ii) to minimize the level of threat exposure. The first objective is quantified by the proximity circle, a circle centered at the target position and moving with the target as it moves in the area of operation (see Fig. 6.1). The intelligent following strategy is to keep the UAV within this circle as long as it is possible. Namely, |rU AV − rt | ≤ rp ,

(6.1)

where rp is the radius of the proximity circle, and rU AV and rt are the position vectors of the UAV and target, respectively. The radius of the proximity circle, rp , is a design parameter of the strategy and quantifies how close the UAV should follow the target. It is assumed that the position of the target is provided for the strategy by an onboard sensor that can only measure the position of the target relative to the UAV. It

68

69 is also assumed that the target position can always be measured as long as the target is within the sensor range, i.e |rU AV − rt | ≤ rs ,

(6.2)

where rs is the sensor range or the radius of the sensor circle as shown in Fig. 6.1. In other words, the target position is measured during the time when the target is within a circle that is centered at the UAV and moves with it, and whose radius rs is equal to the range of the sensor. However, note that, depending on the type of the sensor, this assumption may not be valid in practice where the sensor has a limited view angle and/or the measurement is obstructed by obstacles such as buildings or terrain.

y

ψUAV rs

ψ

UAV

t

Target r UAV r

rp

t

x

Figure 6.1. Proximity and sensor circles.

The radius of the proximity circle, rp , determines the level of trade-off between following the target closely and minimizing the threat exposure level. The bigger the proximity circle, the larger the area where the strategy can search for a trajectory with the lowest threat exposure level. However, as the proximity circle gets bigger, the UAV will follow the target from farther distances. Furthermore, the risk of the target getting outside the sensor range will be greater. Regardless of the size of the proximity circle,

70 the strategy may steer the UAV so far away from the target that the target is no longer in the sensor range. This may happen when the strategy needs to avoid a high threat region or a restricted area (Ar ). Another situation that puts the target outside the sensor range occurs when the target suddenly changes its speed or heading. When the target is no longer within the sensor range, the position as well as the heading and speed of the target are estimated by the estimation model from the measurement obtained while the target is moving within the sensor circle. The prediction is utilized to steer the UAV to the target close enough to resume the position measurement. Because the strategy steers the UAV based on the position at which the target will be at the next computation time not on the current position of the target, even when the target is in the sensor range, the estimation model is utilized to predict the position of the target at the next computation time. The strategy has three main objectives, which are, in the order of priority, (i) keep/take the UAV out of Ar (ii) keep/put the UAV within the proximity circle (iii) minimize the threat exposure level To achieve these objectives, the strategy will generate commanded heading, ψcmd , and speed, Vcmd , for the UAV. When it does so, it takes into account the heading change constraint and minimum/maximum speed constraints of the UAV as described in Chapter 4. The strategy should not generate any heading or speed command that would violate these constraints. To achieve the objectives without violating the constraints, the strategy uses several rules for the heading and speed difference between the target and the UAV. The strategy evaluates a number of points equally distributed over an arc of circle bounded by ψR and ψL to generate heading and speed command in each simulation step (the smaller arc inFig. 6.2). These points are to evaluate the PTEM to find its minimum value within the constraints of the UAV. The radius of the arc is determined

71 by the current UAV speed and the simulation time step. As the UAV speeds up, the search distance ahead is increased. As the UAV speed decreases the search is done in a closer distance. The points in this arc are called possible flying points. Moreover, this arc stretches over the permissible heading range.

Figure 6.2. Searching to avoid restricted high threat region.

6.1.1

Restricted Area Search To find if the UAV is in the risk of getting into Ar , instead of searching only next

possible flying points, the strategy uses a second search arc in which the candidate search points are also distributed with equal intervals as shown in Fig. 6.2, which shows how the immediate risk of getting into a restricted area is quantified. a) If some of the search points on the second arc are outside the restricted area, it is possible to avoid it using one of the search points on the first arc. b) If all the points on the second arc are in the restricted area, there is high risk that the UAV would enter the restricted area unless

72 a course change is taken. c) There is no immediate risk, if any of the search points on the first arc are chosen. Search points from the candidates are chosen depending on the dynamic constraints of the UAV and the strategy parameters by means of the intersection areas between cones and circles. In each computation time, the strategy evaluates the PTEM at all the search points, if there are any, to determine the heading with the lowest threat exposure level. If only some of the points are in the restricted region, the UAV is in the low risk of getting into Ar . If all of the points in the second arc are in the restricted region, the UAV is defined to be in the high risk of getting into Ar . If none of the points are in the restricted region, then there is no immediate risk for the UAV to get into Ar . ψ

ψ

ψ

UAV

target

L

ψ

UAV

ψ max

R

ψ

HDC

Target

Figure 6.3. Heading constraint cone and heading difference cone.

6.1.2

Strategy Constraints In addition to dynamic constraints, two other mechanisms are also used in order

to keep the UAV in the proximity of the target: (i) keep the heading difference small (ii) keep the speed difference small. The first mechanism is quantified by ψHDC , heading

73 difference constraint angle (see Fig. 6.3) and the second mechanism is quantified by speed difference constraint.

6.1.2.1

Heading difference constraint(HDC)

To keep the heading difference between the UAV and the target small, it should be bounded. This is necessary because the turning radius of a vehicle increases as long as the speed of the vehicle increases. If the heading difference is not bounded, the strategy may change the UAV heading drastically to minimize the threat exposure when the UAV is within the proximity circle. This, in turn, may increase the risk of the target getting outside the proximity circle and eventually outside the sensor range. To restrict the motion of the UAV in and around the direction where the proximity circle is heading, ψHDC is introduced. Since the correspondence between the vehicle heading and the target heading may be very restrictive in a slow pursuit case especially if the target stops, a linear relationship is developed between (ψHDC ) and the estimated target speed (Vest ). ∗ ) as long as the Vest is There should be a constant heading difference constraint (ψHDC

greater than a pre-specified threshold speed (Vthres ) and starting from the Vthres , ψHDC should go to π as Vest goes to zero. Basically, this means that there is no more heading difference constraint between the UAV and the target if the Vest is zero.    ψ∗ , Vest ≥ Vthres HDC ψHDC = ∗   ψHDC −π Vest + π, Vest < Vthres .

(6.3)

Vthres

Similar to the heading constraint cone, a heading difference (HD) cone is defined around the target heading moving with the target (see Fig. 6.3). |ψU AV − ψtarget | < ψHDC

(6.4)

The strategy should keep or put the heading of the UAV in the HD cone by commanding the heading angle of the UAV so that the heading difference between the UAV

74

Figure 6.4. No overlap between UAV heading and heading difference constraint. and the target will be bounded as specified. However, as stated earlier, the UAV cannot change its heading more than ψmax in a given computation interval. In other words, the heading constraint cone defines the admissible range of heading angles for the UAV and the HD cone specifies the desired range of heading angles for the UAV.

ψUAV ψmax

ψmax

ψ HDC

ψ t ψ H D

C

Searchable Heading Range

Figure 6.5. Overlap between UAV heading and heading difference constraint.

75 If there is no overlap between these two cones, as shown in Fig. 6.4, the closest heading to the target heading, which is obviously the farthest right or left heading of the UAV will be chosen. As a special case if there is 180 degrees heading difference, both of the farthest right and left headings will obtain the heading of the UAV at the same closeness to the target heading. So, the heading, which will provide the lowest threat exposure level will be chosen.

Figure 6.6. Searchable points in the target proximity range.

The searchable heading range (SHR), as shown in Fig. 6.5, is defined to be the overlap cone between the heading constraint and heading difference cones. The intersection area between the SHR cone and the proximity circle determines the search points for minimizing the threat exposure (see Fig. 6.6). If this intersection area is empty, it means the UAV will be outside the proximity circle if the commanded heading is chosen from the SHR cone. To avoid this, the proximity cone (as shown in Fig. 6.7) is defined to be the set of all the headings that would take the UAV into the proximity circle from its current position. Since putting the UAV into the proximity circle has higher priority

76 than minimizing the threat exposure, when the UAV is outside the proximity circle, only the intersection between the proximity cone and HC (heading constraint) cone is investigated. Since the proximity circle is moving with the target, this will also protect the UAV to get into a limit cycle, which is very objectionable.

Figure 6.7. Steering UAV into the proximity circle.

6.1.2.2

Speed difference constraint

To be able to follow the target, the speed difference between the UAV speed (VU AV ) and the speed of the target (Vt ) should also be bounded, which is defined by |VU AV − Vt | < VDC

(6.5)

In this chapter, speed difference constraint VDC is defined to be zero, which means that the strategy wants the UAV’s speed to be equal to the speed of the target as long as the speed of the target is within the UAV’s speed constraints. Otherwise, the UAV speed is selected to be Vmin or Vmax . Note that, this is the case where UAV is inside the proximity

77 circle. If the UAV falls outside the proximity circle, it will be commanded to catch up with the target as will be explained in the following section.

6.1.3

Strategy cases To achieve the objectives under the given constraints, the strategy uses some rules

on the heading and speed difference between the target and the UAV. These rules will vary depending on whether (a) the UAV is in the risk of getting into Ar . (b) there is an overlap between HC and HD cones. (c) the UAV is in the proximity circle. (d) there is an overlap between HC and proximity cones. Depending on the answers (YES/NO) to the above questions there are 7 different cases as shown in Fig. 6.8:

YES

YES

1

HIGH RISK

NO

YES

YES IN PROXIMITY

NO

YES IN PROXIMITY

CIRCLE

2

NO

RISK REGION

OVERLAP BETWEEN HC and HD CONES

NO

7

NO

CIRCLE

3

4

YES

5

Figure 6.8. Decision tree.

OVERLAP WITH PROXIMITY CONE

NO

6

78 CASE 1 : In this case, the UAV is in the high risk of getting into Ar . To steer the UAV out of the high risk region as soon as possible, the possible sharpest turn should be taken. To achieve this, the max right/left heading is selected without considering ψHDC . The commanded speed is selected to be Vmin to increase the turning rate of the UAV and to prevent any further incursion.    ψR , ψcmd =   ψL ,

fXY (XL , YL ) ≥ fXY (XR , YR ),

(6.6)

fXY (XL , YL ) < fXY (XR , YR ).

Vcmd = Vmin

(6.7)

where (XL , YL ) and (XR , YR ) are the positions corresponding to the left and right headings, respectively. CASE 2 : In this case, the UAV is in the low risk of getting into Ar . To steer the UAV away from the restricted region, the points that are in the restricted region in the second arc of the circle are marked. Corresponding points are eliminated in the first arc of the circle and then at the remaining points the PTEM is evaluated. If some of these points are in the proximity circle, then the heading with the corresponding search point that will give the lowest threat exposure level (ψmin−pr ) is chosen and the commanded speed is chosen to be equal to the Vest provided as long as it is within the UAV speed constraints. ψcmd = ψmin−pr    Vmin ,    Vcmd = Vest ,      Vmax ,

(6.8) Vest ≤ Vmin , Vmin < Vest < Vmax ,

(6.9)

Vest ≥ Vmax .

where Vest is the estimated speed of the target. CASE 3 : In this case, none of the points are in the proximity circle. So, to get the UAV closer to the predicted target position, the corresponding heading (ψcl ) which will

79 steer the UAV closest to the predicted target position is chosen. In other words, ψcl is the corresponding heading of the search point on the first circle of arc with the smallest distance to the predicted position of the target. Moreover, the commanded speed is chosen to be equal to the Vest provided it is within the UAV speed constraints. ψcmd = ψcl    Vmin ,    Vcmd = Vest ,      Vmax ,


(6.11)

Vmax ≤ Vest .

CASE 4 : In this case, there is no risk of getting into Ar and there is an overlap between HC cone and HD cone, so the points in the searchable heading range are evaluated. Among the search points in the proximity circle, the corresponding heading which will give the lowest probability of becoming disabled (ψmin−pr ) is chosen. Vest is the commanded speed also in this case provided it is within the UAV speed constraints. ψcmd = ψmin−pr    Vmin ,    Vcmd = Vest ,      Vmax ,


(6.13)

Vmax ≤ Vest .

CASE 5: In this case, none of the points in SHR are in the proximity circle. However, since there is an overlap between SHR and the proximity cone, the UAV heads towards the proximity circle. Thus, to get closer to the target, the heading (ψcl ) within the HC cone that will move the UAV closest to the predicted target position is chosen. In this case, ψcl is within the proximity cone. Along ψcl direction, the strategy evaluates the PTEM at five more points at every V × ∆t intervals. If the PTEM is less than fr at all these points, then the strategy will command the speed to increase to put the UAV in the proximity circle as soon as possible. The required velocity (Vreq ) that will take

80 the UAV into the proximity circle in the next computation is computed. Then, Vcmd is computed considering Vref and the speed constraints of the UAV ψcmd = ψcl    Vmin ,    Vcmd = Vreq ,      Vmax ,

(6.14) Vreq ≤ Vmin , Vmin < Vreq < Vmax ,

(6.15)

Vmax ≤ Vreq .

If, at any one of the points, the PTEM is greater than fr , then there is a risk for the UAV to get into Ar . To prevent the UAV from approaching Ar , the strategy commands the sharpest turn while the UAV prevents its current speed . CASE 6: In this case, there is no overlap between SHR and the proximity cone. Thus, the UAV heads away from the proximity circle. To turn the UAV towards the proximity circle, the UAV is commanded to turn to ψcl (which happens to be the sharpest turn in this case) and match the target speed. ψcmd = ψcl    Vmin ,    Vcmd = Vest ,      Vmax ,


(6.17)

Vmax ≤ Vest .

CASE 7: In this case, there is no risk of getting into Ar . Additionally, there is not an overlap between heading constraint cone and HD cone. Thus, to make the heading difference between the UAV and the target smaller, the corresponding max right/left

81 heading is selected and the commanded speed is selected to be equal to the Vest provided it is within the UAV speed constraints.    ψR , ψcmd =   ψL ,    Vmin ,    Vcmd = Vest ,      Vmax , 6.2 6.2.1

fXY (XL , YL ) ≥ fXY (XR , YR ),

(6.18)

fXY (XL , YL ) < fXY (XR , YR ). Vest ≤ Vmin , Vmin < Vest < Vmax ,

(6.19)

Vmax ≤ Vest .

Simulation Environment and Results Modular Structure of the Simulation Environment The guidance and estimation algorithms are implemented in a modular simulation

environment. As shown in Fig. 6.9, the simulation model is organized in 5 major systems as Target Dynamics, UAV Dynamics & Control, Sensor Module, Estimation Algorithm and Guidance Algorithm for simulating cases where a singleUAV pursue a target in an adversarial environment. Target Dynamics module generates the trajectory of the target in the area. UAV Dynamics & Control module consists of the model of the UAV Dynamics and the low level controller. In this research, the closed-loop UAV dynamics is modeled as two firstorder transfer functions, from the commanded speed and heading to the actual speed and heading.

Sensor Module receives x and y components of the actual target position and

superimposes them with Gaussian white noise to model the measurement noise. This module also receives UAV position from UAV Dynamics & Control subsystem. UAV position is compared with the actual target position to model the fact that sensors do not provide measurement when the target is beyond the sensor range. Thus, when the

82

Guidance Algorithm

Speed & Heading Commands

UAV Dynamics & Control

UAV States

Sensor Module

Target Measurement

Estimation Algorithm

Estimated Target States

Target Dynamics

Figure 6.9. Modular structure of a singleUAV simulation environment. radial distance to the target is larger than a specified “sensor range”, Sensor Module does not provide any output. Estimation Algorithm takes target measurements from Sensor Module. The measurements are used to compute the estimates of the target position, speed and heading as explained in Chapter 5. Estimation Algorithm module is not run continuously unlike the other modules discussed above. It is executed only at discrete time instants as specified by the estimation update period. Guidance Algorithm module is also executed only at discrete time instants, but not necessarily at the same rate, i.e., its update period might be different from that of Estimation Algorithm. Guidance Algorithm module executes the rule-based intelligent guidance algorithm as explained in Section 6.1.

6.2.2

Implementation of the Strategy and Simulation Results For the implementation of the strategy, 5 cases are selected where 17 sources of

threat and 3 no-fly zones are scattered over a 120 km X 120 km area of operation. These cases are selected to show the full capabilities of the algorithm. They cover most of the possible scenarios such as when (i) UAV starts the pursuit while behind the target (ii) UAV starts the pursuit while ahead of the target (iii) target continuously accelerates (iv) target continuously decelerates and then maintains constant speed that is less than

83 the minimum speed of the UAV (v) target stops in a restricted region, and (vi) target stops outside the restricted regions. The PTEM of the area is defined and constructed as explained in Section 3.2. In all the simulations, the UAV is considered to have a minimum speed of 180 (km/h) and a maximum speed of 650 (km/h) and a maximum heading angle of (30) deg. Simulation time step is 3 seconds for all simulations. Strategy ∗ design parameters fr and ψHDC are selected to be 0.0028 and 45 deg, respectively. The

estimation algorithm explained in Section 5.1 is utilized to estimate the states of the target as well as to predict the future position. In all the figures showing the trajectories, the UAV trajectory is shown with a solid line, the actual target trajectory is shown with a dash-dot line and the estimated target trajectory is shown with a dashed line. Figs. 6.10 and 6.11 show the first case where the target is moving on a third order trajectory with an increasing speed according to the following functions of time for speed in m/s and heading in radians Vt (t) = 1000 sin(0.00002 t) + 0.109 3 t t ψt (t) = 1.7 − + + 3 sin(0.002 t) 200 615

(6.20) (6.21)

The target’s initial position, heading and velocity are (17,-35) km, (97.4) deg, (392.4) km/h, respectively. UAV’s initial position, heading and velocity are (11,-38) km, (30) deg, (288) km/h, respectively.

The UAV is assumed to have a 9 km sensor range and

proximity range is chosen to be 7.5 km. As seen from Fig. 6.10, strategy has generated a feasible trajectory for the UAV which follows the target while minimizing the threat exposure and avoiding no-fly zones. Note from Fig. 6.10 that initially the UAV is ahead of the target. Since the target is behind, the UAV turns around to head toward the target. During the pursuit, the UAV loses the target at 180 seconds since the target has entered a restricted region. While the UAV flies around the restricted area, the distance to the target gets larger than the sensor range, which prevents the UAV from

84

28

Contour plot of the probalistic threat exposure map

0.

00

30

0.0028

0.0

02

8

20

0.

00

0 28

00

0.

0.0028

y(km)

0.00

28

10

28

0.0028

0.002 0.002 8

8

−10

−20

0.00

28

0.

00

28

−30

−40

−30

−20

−10 x(km)

0

10

Figure 6.10. UAV trajectory generated by the strategy to follow the target. taking the measurement of the position of the target. Since there is no measurement, the UAV uses the predicted trajectory generated by the estimator to continue on the pursuit. Eventually, when the target leaves the restricted region, the UAV is able to fly closer and resume the measurement at 240 seconds. In this case simulated, since the time interval with no measurement is short and the UAV has obtained enough measurement earlier, the deviation of the predicted trajectory from the actual one is small. Thanks to the satisfactory performance of the estimator, the UAV regains the measurement of the target. Fig. 6.11 shows that during the time when the target is lost and no measurement

85

ψ(deg)

ψtargetactual 350

ψtargetmeasured

300

ψtargetestimated

250

ψUAVcommanded

200 150 100 50 0 0

100

200

300

400

500

t(sec)

Vtargetmeasured

800

V(km/h)

600 Vtargetactual

700

Vtargetestimated

600

VUAVcommanded

500 400 300 200 100 0

100

200

300

400

500

600

t(sec)

Figure 6.11. Actual, measured and estimated target heading and speed and commanded UAV heading and speed. is taken from 180 to 240 seconds, the measured signals do not have any noise in them and are exactly the same as the estimated ones. This is due to the fact that there is in fact no measurement and the last estimated signals are used instead. Note also from Fig. 6.11 that the commanded UAV speed is always within the constraints and the UAV matches its speed with the estimated speed of the target as long as the UAV is inside the proximity circle and there is no restricted areas. Nevertheless, since Case 5 occurred several times, the UAV speed is commanded to its maximum speed regardless of the estimated target speed. In the simulation all the angles are restricted to the range of 0 degree to 360 degrees. Thus, the commanded heading

86 angle seems to have a jump from 10 to 340 degrees at about 15 seconds. However, this is not a jump but the continuation of the commanded heading change in the clock-wise direction while the UAV is turning around towards the target. The sensor noise for the position measurements is Gaussian white noise with a zero mean and standard deviation of 100 meters. Since the “measured” heading and speed are in fact computed from the measured positions by using nonlinear equations, their means and standard deviations are different as can be seen in Fig. 6.11. Yet, they are assumed to be zero mean white noise in the estimation process. The second case is shown in Figs. 6.12 and 6.13 where the target’s initial position, heading and velocity are (28,-35) km, (97.4) deg, (392.4) km/h, respectively and target moves with the same functions of time for speed and heading as in the first case. UAV’s initial position, heading and velocity are (22,-41) km, (30) deg, (288) km/h, respectively. The UAV is assumed to have a 9 km sensor range and proximity range is chosen to be 7.5 km.

As seen from the Fig. 6.12, the UAV cannot continue the pursuit of the

target. This is because the target passes through a bigger threat region. During the time that the UAV avoids the threat region, the predicted target trajectory significantly diverges from the actual trajectory. Since the UAV follows the predicted trajectory as seen from Fig. 6.13, it cannot detect the target again. This case shows the limitation of the estimation algorithm. If the target moves unpredictably, letting the target go and staying out of the sensor range for a long time may cause the UAV to lose the target permanently. Nevertheless,

as shown in the first case, the incorporation of the

estimation algorithm into the guidance strategy certainly gives the UAV an opportunity to regain the target. This increases the success of the pursuit while avoiding restricted areas and minimizing the threat exposure level.

87 Contour plot of the probalistic threat exposure map

0.

00

28

10

0 28

0.0028

00

0.

0.0028

0.002 8

y(km)

−10

−20

0.00 28

−30

−40

0

5

10

15 x(km)

20

25

28

−5

00 0.

−10

30

35

Figure 6.12. Insufficient UAV trajectory generated by the strategy to follow the target. Figs. 6.14 and 6.15 show the third case where the target’s initial position, heading and velocity are (-15,-35) km, (34.4) deg, (356.4) km/h, respectively and moves in the area according to the following functions of time for speed in m/s and heading in radians Vt (t) = 1000 0.099 − sin(0.00002 t) 3 t t + + 5 sin(0.0017 t) ψt (t) = 0.6 − 250 950

(6.22) (6.23)

UAV’s initial position, heading and velocity are (-11,-30) km, (30) deg, (288) km/h, respectively. The UAV is assumed to have a 7.5 km sensor range and proximity range is chosen to be 5 km. Note from Fig. 6.14 that since the UAV is initially ahead of the

88 ψtargetactual ψtargetmeasured

300

ψ(deg)

ψtargetestimated ψUAVcommanded

200

100

0 0

50

100

150

200

250

300

350

400

450

t(sec) 800

V(km/h)

600 400

Vtargetactual Vtargetmeasured

200

Vtargetestimated 0 0

50

100

150

200

250

300

350 VUAVcommanded 400

450

t(sec)

Figure 6.13. Actual, measured and estimated target heading and speed and commanded UAV heading and speed. target, the UAV turns around to head toward the target. Furthermore,

in this case

the target reduces its speed to 108 km/h in 345 seconds and maintains that speed in the rest of the pursuit as seen from Fig. 6.15. Note that the speed of the target is much less than the minimum speed of the UAV during the most of the pursuit. Thus, the UAV reduces its speed to the minimum speed and loiters within the proximity circle when there is no restricted area while minimizing the threat exposure level as shown in Fig. 6.14. Nevertheless, the UAV speed is commanded to its maximum value several times to catch the proximity circle. In all the time during the pursuit, the commanded speed is within the constraints. Furthermore, there seems to be a jump in the commanded heading angle due to the specified range of angles when the UAV flies with a heading angle close to 0 or 360 degrees. Also there seems to be an amplification in the noise when the target is moving with a heading close to zero. Again, this is due to the conversion of the angles


0.0

028

40

28

00

28

0.

0.

00

30

0.0

02

8

20

y(km)

0.00

28

10

0

28

00

0.

0.0028

0.002 0.002

8

8

−10

−20

0.00

28

0.

00

28

−30

−40

−35

−30

−25

−20 −15 x(km)

−10

−5

0

Figure 6.14. UAV trajectory generated by the strategy to follow a slowing target. outside the range into their corresponding values in the range (e.g. 355 deg is the same as -5 deg). This case is presented to mainly show that when the target moves slower than the minimum speed of the target, the strategy generates commanded heading to keep the UAV within the proximity circle while minimizing the threat exposure. The fourth case is shown in Figs. 6.16 and 6.17 where the target’s initial position, heading and velocity are (-1,-32) km, (57.29) deg, (356.4) km/h, respectively and moves

90 ψtargetactual ψtargetmeasured

400

ψtargetestimated

ψ(deg)

300

ψUAVcommanded

200 100 0 0

500

1000

1500 t(sec)

2000

2500 Vtargetactual

3000

Vtargetmeasured Vtargetestimated

V(km/h)

600

VUAVcommanded 400 200 0 0

500

1000

1500 t(sec)

2000

2500

3000

Figure 6.15. Actual, measured and estimated target heading and speed and commanded UAV heading and speed. in the area according to the following functions of time for speed in m/s and heading in radians Vt (t) = 1000 0.099 − sin(0.00015 t) 3 t t + + 5 sin(0.002 t) ψt (t) = 1 − 220 615

(6.24) (6.25)

UAV’s initial position, heading and velocity are (5,-30) km, (30) deg, (288) km/h, respectively. The UAV is assumed to have a 7.5 km sensor range and proximity range is chosen to be 5 km. In this case target continuously reduces its speed to 0 km/h and stops at 660 seconds in the middle of a restricted region during the rest of the pursuit as seen from Fig. 6.16 and Fig. 6.17. In this case, the UAV flies along the boundary of the restricted area when the distance to the target is greater than the radius of the proximity circle. Whenever possibly (the distance from the boundary of the region is smaller than the radius of the proximity circle), as Fig. 6.17 shows, the UAV tries to minimize the


5

0 28

0.0028

00

0.

0.0028

−5

y(km)

−10

0.002

8

−15

−20

−25

0.00 28

0.

00

28

−30

−35 −15

−10

−5

0

5

10

15

20

x(km)

Figure 6.16. UAV trajectory generated by the strategy to follow a slowing and stopping target. threat exposure while circling around the restricted region. As seen from Fig. 6.17, the UAV reduces its speed to the minimum speed and maintains it during the rest of the pursuit. However, note that the UAV speed is commanded to its maximum value several times to catch the proximity circle before the target completely stops. Note that, as the target speed get closer to zero, there is an increase in the noise for heading measurements due to the fact that target is heading around 90 degrees. This basically means that the position change in the x direction is very small, which results in the amplification of the noise. Thus, according to Eq. (5.3), the noise is amplified. Nevertheless, note that when

92 the target stops or moves with a very low speed, strategy does not take target heading into consideration in computing the UAV heading command. ψtargetactual 400

ψ(deg)

300

ψtargetmeasured ψtargetestimated ψUAVcommanded

200 100 0 0

200

400

600

800

1000

1200

1400

V(km/h)

t(sec)

600

Vtargetactual

400

Vtargetestimated

Vtargetmeasured VUAVcommanded

200 0 0

200

400

600

800

1000

1200

1400

t(sec)

Figure 6.17. Actual, measured and estimated target heading and speed and commanded UAV heading and speed.

Figs. 6.18 and 6.19 show the fifth case where the target’s initial position, heading and velocity are (-5,-20) km, (57.29) deg, (356.4) km/h, respectively and target moves with the same functions of time for speed and heading as in the fourth case. UAV’s initial position, heading and velocity are (-3,-17) km, (30) deg, (288) km/h, respectively. The UAV is assumed to have a 7.5 km sensor range and proximity range is chosen to be 5 km. In this case target constantly reduces its speed and at 660 seconds it stops in a region where there is no restricted areas during the rest of the pursuit as seen from Fig. 6.18 and Fig. 6.19.

In this case, the UAV loiters inside the proximity circle while

minimizing the threat exposure. The UAV speed is commanded to its minimum value

93 Contour plot of the probalistic threat exposure map 20

0.

00

15

28

10

5

y(km)

0 28 0.0028

00

0.

0.0028

−5

−10

0.002

8

−15

−20

−25 −20

−15

−10

−5

0.00 0 28

5

10

15

20

x(km)

Figure 6.18. UAV trajectory generated by the strategy to follow a slowing and stopping target. and maintains it during the pursuit as seen from Fig. 6.19. Due to the same reasons discussed above, there is an increase in the noise for heading measurements. In all cases, it is shown that threats and no-fly zones are avoided perfectly, yet the UAV may violate these regions for a very short time while trying to avoid them. This does not mean that the UAV has failed because fr is a parameter of the strategy and it is specified with a safety margin which will protect the UAV from violating the real restricted areas. The reason behind flying very close to restricted areas is that the UAV tries to be in the proximity circle all the time, which means that it tries to compromise

94 400

ψtargetactual ψtargetmeasured

ψ(deg)

300

ψtargetestimated ψUAVcommanded

200 100 0 0

100

200

300

400

500 t(sec)

600

700

800

600

900

1000

Vtargetactual

V(km/h)

Vtargetmeasured 400

Vtargetestimated VUAVcommanded

200

0 0

100

200

300

400

500 t(sec)

600

700

800

900

1000

Figure 6.19. Actual, measured and estimated target heading and speed and commanded UAV heading and speed. between safer and closer trajectories to the target. Thus, as the proximity circle gets bigger, total threat exposure level will decrease, which means that the UAV will have a chance to follow a much safer trajectory. However, if the proximity radius is greater than the sensor range, then the UAV will very likely lose the target. The effects of the parameters of the strategy on the total length and total probability of the UAV are given in tabular forms in the remainder of this section, which are obtained from the first case where the target is continuously accelerating. In this case, total length of the target trajectory is 75.216 km and total threat exposure level is 0.1273. Table 6.1 shows that as the proximity range increases, total threat exposure level decreases and the total length of the UAV trajectory decreases accordingly. Increasing the proximity range will give the strategy more flexibility to minimize PTEM while it is inside the proximity circle. Furthermore, as long as the UAV stays in the proximity circle it will make fewer maneuvers to get closer to the target, which means that a safer

95 and a shorter trajectory will be followed. As the ratio of the sensor to proximity ranges approaches 1, following the target may become harder. Once rp becomes greater than the rs , the target will be more likely to be outside the sensor range.

In this case if the

Table 6.1. Effects of proximity range on the total UAV trajectory length (Ltraj ) and Threat Exposure Level (TEL)

rp (km) 0.5 3.0 5.0 7.5 8.0

UAV Ltraj (km) 91.177 86.964 85.702 84.987 84.721

TEL 0.0755 0.0459 0.0340 0.0307 0.0291

estimator is not good enough to follow the target trajectory, the UAV cannot catch the target again, and thus the amount of measurements about the target movement might not be enough to develop a good estimation model. Note that in all the cases, the threat exposure level is much less than that if it would follow exactly the trajectory of the target.

∗ Table 6.2. Effects of heading difference constraint(ψHDC ) on the total UAV trajectory length (Ltraj ) and Threat Exposure Level (TEL)

∗ ψHDC (rad.) π/4 π/6 π/12 π/18 π/36

UAV Ltraj (km) 84.987 81.906 81.906 81.906 81.906

TEL 0.0307 0.0326 0.0688 0.0689 0.0694

96 ∗ As seen from Table 6.2 , if the heading difference constraint constant (ψHDC ) is

decreased, the threat exposure level will increase because there will be fewer points in the searchable area and the UAV will not be allowed to turn its heading to the points which will minimize its threat exposure more. However, this parameter does not effect ∗ the total length significantly. Note that if the ψHDC is specified bigger than π/4, the UAV

would lose the target permanently. Thus, even if the threat exposure level is reduced ∗ as ψHDC is increased, the heading difference should be restricted especiaaly during high

speed pursuits. Table 6.3 shows the effect of fr on the threat exposure level and the total length of the trajectory. As fr is increased, the probability of the UAV becoming disabled will increase due to the fact that it will have permission to enter higher threat regions. However, note that when fr is greater than 0.01, any further increase in fr does not affect

Table 6.3. Effects of fr on the total UAV trajectory length (Ltraj )and Threat Exposure Level (TEL)

fr 0.0028 0.0050 0.0075 0.0100 0.0200 0.1000 0.5000 1.0000

UAV Ltraj (km) 84.987 83.834 83.538 83.217 83.217 83.217 83.217 83.217

TEL 0.0303 0.0454 0.0529 0.0535 0.0535 0.0535 0.0535 0.0535

the UAV motion and thus the probability and the length stays the same. This is because even if there is no restricted area, the strategy still minimizes PTEM as long as it stays in the proximity range of the target. Note that, even this case, threat exposure level is

97 much less than the threat exposure level if the target’s trajectory was followed exactly. Note also that this maximum threat exposure level can be further decreased by varying the other parameters of the strategy such as increasing the proximity range.

CHAPTER 7 GRADIENT SEARCH GUIDANCE ALGORITHM As in the algorithm described in Chapter 6, the main goal of this rule-based intelligent strategy is also to generate feasible speed and heading commands for UAVs to safely pursue a moving target in an area with multiple sources of threats and/or restricted zones. This goal requires a trade-off between three possibly conflicting objectives given in the order of their priorities:(i) to avoid restricted regions and obstacles (ii) to maintain the proximity of the target, (iii) to minimize the level of threat exposure. Thus, the strategy should guide the UAV autonomously to keep it within a pre-specified proximity of the target as well as try to minimize the threat exposure at any time while avoiding restricted areas.

7.1

Target Following Strategy The strategy guides a UAV by generating commanded heading, ψcmd , and speed,

Vcmd . While generating these commands, the strategy takes the dynamic constraints of the UAV into account. Another requirement for the strategy, addressed in the new algorithm, is that the strategy algorithm be executed on-line during the pursuit on an onboard computer/micro-processor. Thus, the strategy should be computationally feasible regarding the flight characteristics of the UAV and the configuration of the on-board processor/computer. While the problem statements are the same, this chapter presents a new and significantly improved approach that has resulted in a more systematic and analytic formulation of the guidance strategies, a computationally more efficient algorithm that 98

99 generates more feasible commanded signals for the pursuing UAV. Specifically, the algorithm described in this chapter uses a gradient search approach in minimizing threat exposure and avoiding restricted areas and systematically utilizes mathematical tools such as level curves, inertial and moving-rotating frames and rotation matrices, vector representations of directions and geometric relations between cones, circles and straight lines. Furthermore, strategy states of this algorithm are better organized, which makes the algorithm much easier for reading and programming. The decision factors in this algorithm are analytically quantified while the other algorithm has to evaluate PTEM at numerous points for determining the decision factors, which increases the computational burden of the algorithm. Thus, this algorithm develops a computationally more feasible guidance algorithm, without compromising the performance of tracking, avoiding obstacles/restricted-areas and minimizing threat exposure.

7.1.1

Preliminary Definitions In this section, mathematical and geometric preliminaries are introduced to famil-

iarize the reader with concepts, parameters and constraints used for the description and formulation of the strategy. Note that throughout the dissertation C, D and V refer to circle, disk and cone, respectively. Let Ts be the “guidance update period”, i.e., the commanded heading and speed are updated only when time t = kTs , k ∈ {0, 1, 2, · · · }. The positions of the UAV and the target are defined relative to the fixed reference frame where the PTEM is presented as a function of position. As seen from Fig. 7.1, the position vectors of the UAV and the target are defined to be rU AV and rt with (xu , yu ) and (xt , yt ) coordinates, respectively.

100 y

ψUAV

ψR

ψL

v

t

x

x

ψ ma

ψ ma

ψ

HC cone ψ HDC

UAV rs

ψ

L

r

HDC

Target

UAV

rp r

HDC cone

t

x

Figure 7.1. Proximity & sensor circles, HC & HDC cones, and LOS vector. The headings of the UAV and the target are angles measured from positive x-axis as ψU AV and ψt , respectively. Let L, “line of sight” (LOS) vector, be the position vector of target with respect to UAV and be written as L = (xt − xu )Ib + (yt − yu )Jb Thus, the LOS angle, ψLOS , can be computed as yt − yu −1 ψLOS (k) = tan xt − xu

(7.1)

(7.2)

Then, the time rate of change of ψLOS is calculated as (y˙ t − y˙ u )(xt − xu ) − (x˙ t − x˙ u )(yt − yu ) ψ˙ LOS (k) = (xt − xu )2 + (yt − yu )2

(7.3)

Proximity circle, Cp (k), denotes a circle, at the k th (current) update instant, centered at the predicted target position at the (k + 1)th (next) update instant and with a specified radius, rp (see Fig. 7.1). Namely, Cp (k) = z ∈ R2 : |z − b rt (k + 1)| = rp ,

(7.4)

101 Similarly, Proximity disk, Dp (k), is defined to be the region bounded by Cp (k), i.e. Dp (k) = z ∈ R2 : |z − b rt (k + 1)| ≤ rp ,

(7.5)

Dp (k) is introduced to define the proximity of the target and thus it is a design parameter that quantifies how close to the target the strategy should keep the UAV during the pursuit. Note that the proximity of the target is the objective of the strategy with the second highest priority. Once this objective is secured, the strategy should try to achieve the last objective, minimizing the threat exposure level. In this regard, Dp (k) quantifies the trade-off between these two objectives. Similar to Cp (k), Reachability circle, Cr (k), is defined to be a circle, centered at the current UAV position (see Fig. 7.2) and has a radius determined by the speed of the UAV and the guidance update period, Ts : Cr (k) = z ∈ R2 : |z − rU AV (k)| = Ts × VU AV (k) ,

(7.6)

where rU AV (k) and VU AV (k) are the samples of the UAV position and speed at the k th update instant, respectively. Note that Cp (k) is centered at the predicted target position at the next update instant while Cr (k) is centered at the UAV position at the current update instant. Thus, these two circles are used to determine (i) whether it is possible stay within Dp (k) until the next update instant and (ii) if not, the range of directions that would steer the UAV towards Dp (k). Based on ψHDC (k), HDC (Heading Difference Constraint) Cone (VHDC ) is defined to be the range of headings that are considered to be close to the estimated heading of the target, ψbt . Thus, as shown in Fig. 7.1, VHDC moves with the target, is centered at its estimated heading and expands in both clockwise and counter-clockwise directions by

ψHDC (k). Similarly, HC (Heading Constraint) Cone, VHC , defines the range of headings that are admissible when only the dynamics of the UAV is considered. VHC , as shown in

102 Fig. 7.1, moves with the UAV, is centered at its heading and expands in both clockwise and counter-clockwise directions by ψmax . Cp FPR cone

Target ψ

UAV

ψ

R

ψ

L

Cr

UAV

PR cone

Figure 7.2. General proximity disk cone.

When the UAV is outside the proximity disk, Dp (k), it is desirable to know (i) whether it is possible to move into Dp (k) at the current speed and (ii) if it is, the range of feasible headings that would steer the UAV towards Dp (k). To be able to answer the first question, the PR (Proximity Range) Cone, VP R , is constructed, as shown in Fig. 7.2, by the intersection points of Cr (k) and Cp (k). Namely, Obviously, if Cr (k) 6⊂ Dp (k) and Cr (k) ∩ Cp (k) = ∅ (Note that ∅ denotes the empty set), then VP R = ∅ and thus the answer to question (i) is negative (see (a) of Fig. 7.3). When Cr (k) 6⊂ Dp (k) and Cr (k)∩Cp (k) 6= ∅ (see (b) and (c) of Fig. 7.3), however, the feasibility of VP R should be investigated by determining its intersection with VHC . If VHC ∩ VP R = ∅, then the answer to question (i) is still negative. When VHC ∩ VP R 6= ∅, the answer to question (i) is affirmative, and the answer to question (ii) is the intersection cone, which is defined to be FPR (Feasible Proximity Range) Cone, i.e. VF P R = VHC ∩ VP R as shown in Fig. 7.2. When

103 Cr (k) ⊂ Dp (k), as seen in (d) of Fig. 7.3, Cr (k) ∩ Cp (k) = ∅. However, in this special case, VP R = R2 since any direction will lead to Dp (k), and thus, VF P R = VHC .

Cp

Cr

PR=∅ PR (a)

(b)

PR= ΙR2

PR (c)

(d)

Figure 7.3. PR cone in various cases of position of Cr relative to Cp , (a) Cr (k) ⊂ 6 Dp (k) and Cr (k) ∩ Cp (k) = ∅, (b),(c) Cr (k) 6⊂ Dp (k) and Cr (k) ∩ Cp (k) 6= ∅, (d) Cr (k) ⊂ Dp (k) and Cr (k) ∩ Cp (k) = ∅. The objective of the strategy with the highest priority is to always avoid the restricted areas. To be able to do so, the range of the headings that would steer the UAV towards a restricted area, especially when the restricted area is close-by, should be determined. This can be accomplished by utilizing the directions that are tangent to the level curve of the PTEM that passes through a given UAV position. Note that the tangent directions are always normal to the gradient, ψmin (k). The angles of the two tangent directions are referred to as ψtgR (k) and ψtgL (k) as shown in Fig. 7.4. Then, SHR (Safe Heading Range) Cone, VSHR , is defined to be the range of directions within the VHC through which the directional derivative of the PTEM is zero or negative. In

104 other words, if the heading of the UAV is within VSHR , the UAV is guaranteed to fly away from the close-by restricted area.

restricted region

level curves

ψ tg

ψL

R

ψ UAV

UAV

ψ min

SHR cone

ψ tg

L

ψR

Figure 7.4. SHR cone.

7.1.2

Decision Factors and Strategy States The intelligent strategy first computes the “desired” heading and speed as well

as the “admissible” ranges of heading and speed. The desired signals are computed in accordance with the three objectives of the strategy without considering the dynamic and strategy-imposed constraints. At the same time, the admissible ranges are determined based on the states of the UAV, the local PTEM and the constraints. Then, the strategy generates the commanded signals (heading and speed) by considering the desired signals and their respective admissible ranges. When a desired signal is within the respective admissible range, then obviously the desired signal is assigned as the commanded signal. Otherwise, the boundary of the admissible range that is closest to the desired signal is selected to be the commanded signal. This section of the chapter introduces the decision factors and decision states that are defined based on the strategy objectives and their

105 assigned priorities and used by the strategy to infer the desired heading, desired speed and their admissible ranges. There are four decision factors that are used to determine the decision states: Factor1 : This factor determines whether the UAV is in a risk of getting into a restricted region. It is quantified by using VSHR and evaluating the PTEM at three positions that would be the positions of the UAV in n update periods ahead if it flies with its current speed in the current, maximum right and maximum left heading directions (ψi , i = {1, 2, 3}). Thus, these three prospective positions are calculated as xi (k + n) = xc + Vc (k) n Ts cos ψi (k)

(7.7)

yi (k + n) = yc + Vc (k) n Ts sin ψi (k)

(7.8)

where n is the number of update periods that would take for the UAV to make a 90 degreeturn by utilizing the maximum available turn rate. It is computed by n = ceil [π/ (2ψmax )] where ceil is a function that rounds a real number to the nearest integer towards positive infinity. Then, the PTEM is evaluated at these three positions. If all f (xi , yi ) are greater than or equal to fr , then the UAV is in HIGH risk since all the headings steer the UAV to a restricted area. If only some of f (xi , yi ) is greater than fr and VSHR = ∅ then the UAV is still in HIGH risk since VSHR = ∅ means that the entire VHC is directed completely towards the restricted area If VSHR 6= ∅ and there are some f (xi , yi ) less than fr , then the UAV is considered to be in LOW risk. If all f (xi , yi ) are less than fr , then the risk is NONE since none of the directions steers the UAV to the restricted area. Factor 2 : This factor determines whether the UAV will be within the proximity of the target at the next update instant. This is quantified by employing VP R , VHC and

106 VSHR depending on the answer to Factor 1. When Factor 1 is NONE (i.e. no risk of getting into Ar ), the intersection of VP R and VHC is taken. If VHC ∩ VP R 6= ∅

(7.9)

then the answer is YES, i.e. it is feasible for the UAV to be within Dp (k + 1). Otherwise, Factor 2 is NO. When Factor 1 is LOW, VHC is replaced by VSHR in Eq. (7.9). Note that in the case when VP R = R2 , i.e. Cr (k) is completely in Dp (k), Factor 2 is always YES when Factor 1 is NONE or LOW. When Factor 1 is HIGH, the answer to Factor 2 is not defined. Factor 3 : This factor determines whether the headings of the UAV and the target are close. This is quantified by the intersection of VHC and VHDC when Factor 1 is NONE. If VHC ∩ VHDC 6= ∅

(7.10)

then the answer is YES, i.e. the UAV and the target headings are close, otherwise the answer is NO. When Factor 1 is LOW, VHC is replaced by VSHR in Eq. (7.10). Factor 4 : This factor determines whether the UAV is heading towards Dp (k) only when Factor 1 is NONE or LOW, Factor 2 is NO and Factor 3 is YES. This is quantified by comparing ψLOS (k) with (VHC ∩ VHDC ) or (VSHR ∩ VHDC ). When Factor 1 is NONE and ψLOS (k) ⊂ (VHC ∩ VHDC )

(7.11)

then the answer is YES, namely the UAV is heading towards Dp (k), otherwise the answer is NO. When Factor 1 is LOW, VHC is replaced by VSHR in Eq. (7.10) as done in Factors 2 and 3. Depending on the answers to these decision factors, the strategy states are defined to distinguish between the cases with different priorities since different methods are to be used to compute the desired signals and their respective admissible ranges. Note that

107 Table 7.1. Decision factors and strategy states (N/A: Not Applicable)

STATES 1 2 3 4 5 6 7 8 9

DECISION FACTORS 1 2 3 4 NONE NO YES YES NONE NO YES NO NONE NO NO N/A NONE YES N/A N/A LOW NO YES YES LOW NO YES NO LOW NO NO N/A LOW YES N/A N/A HIGH N/A N/A N/A

this corresponds to the knowledge base part of a rule-based system, which is represented as a set of rules. For example, to    If        AND    Rule1 = AND       AND       THEN

decide strategy state 1 a rule can be defined as F actor 1 is NONE

< antecedent 1 >

F actor 2 is NO

< antecedent 2 >

F actor 3 is YES

< antecedent 3 >

F actor 4 is YES

< antecedent 4 >

(7.12)

Strategy state is 1 < consequent 1 >

Note that database part of a rule-based system includes a set of facts (antecedents) used to match against the IF parts of the rules in the knowledge base. Table 7.1 summarizes all the rules to determine the states of the strategy based on the decision factors. In States-1 to -4, the UAV is not in any risk of flying into a restricted area. In State-1, the UAV is not in Dp , but the headings of the UAV and the target are close and the UAV is heading towards Dp . However, In State-2, the UAV is not heading towards Dp . Note that Factor 4 is considered only when there is NONE or LOW risk of flying into a restricted region and the headings of the UAV and the target are close. In State-3, the UAV is not in the proximity disk and moreover the headings of the

108 UAV and the target are not close. As the simulation results will reveal, this state occurs rarely due to the imposed heading difference constraint by the strategy. In State-4, the UAV is within Dp . Note also that this is the state that strategy tries to keep the UAV in as much as possible during the mission. In States-5 to -8, the UAV is in low risk region. In both States-5 and -6 , the UAV is outside the proximity disk and the headings of the UAV and the target are close. However, in State-6, the UAV is not heading towards Dp . In State-7, the UAV is not in the proximity disk and moreover the headings of the UAV and the target are not close. In State-8, the UAV is within Dp . Note that Factors 3 and 4 are not considered in States-4 and -8 because the UAV is already in the proximity of the target. In State-9, the UAV is in high risk region. Thus, other decision factors are not considered.

7.1.3

Computation of Desired Heading and Admissible Range As stated earlier, the commanded heading is computed based on the desired heading

ψdes and the AHR (Admissible Heading Range) cone, VAHR , which is defined to be the cone that consists of the feasible heading directions. Depending on the strategy state at a given time, different criteria are used to determine ψdes and VAHR . Note that the rules in this section and the next section can be defined to be the rules that represents a strategy to select the commanded signals. Recall that the strategy has three objectives with different levels of priority. There are three headings, ψbt (k), ψmin (k) and ψLOS (k), one of which is chosen as the desired

heading, ψdes (k), at the k th update instant, based on the objective of the strategy in a given strategy state. In States-1 and -2, the objective with the highest priority (avoidance of the restricted regions) is considered to be “achieved” since Factor 1 is NONE. However, the second objective needs to be targeted since Factor 2 is NO, i.e., Dp should be intercepted.

109 Since the Factor 3 is YES, the headings of the UAV and the target are already close. To achieve the second objective, a pursuit guidance law is employed for the UAV to intercept Dp as soon as possible: ψdes (k) = ψLOS (k) + KD ψ˙ LOS (k)

(7.13)

where the first term represents the well-known velocity guidance [151] and the derivative term, with gain KD , is added to improve the pursuit performance. The admissible heading range is selected such that the commanded heading does not violate the dynamic constraint of the UAV and the heading difference constraint imposed by the strategy itself, i.e. VAHR = VHC ∩ VHDC .

(7.14)

Note that the desired heading selection strategy is the same for States-1 and -2. However, the gain, KD , in Eq. (7.13) will be selected differently. Another difference originates from the selection of the desired speed as explained in the next section. In State-3, similar to the first two states, the objective with the highest priority is achieved but the second objective is not. Furthermore, since the Factor 3 is NO, the headings of the UAV and the target are not close to each other. In a high speed pursuit, flying in a direction different from that of the target will lead the UAV to lose the proximity of the target very soon. In other words, it will lead to the failure of the second objective. To turn the UAV in the same direction with the target, ψdes (k) is selected to be ψbt (k) and VAHR is selected to be VHC to allow the UAV to make the sharpest

turn possible. Simulation experiments have shown that, during high speed pursuits, the strategy employed in States-1 and 2 would not be as efficient particularly due to occurrence of restricted regions between Dp and the UAV. This is partly the reason why a different strategy is employed in State-3. Further, note that as pursuit speed decreases,

110 it will be less likely for this state to occur because, as to be explained in Section 7.1.5, the heading difference constraint will be relaxed. In State-4, since the first two objectives of the strategy are achieved, the third one can be targeted. Thus, the strategy, in this state, should try to minimize the threat exposure level while ensuring that the other objectives are not compromised. Note that strategy should steer the UAV in a direction that would minimize threat exposure, which, in turn, implies that the UAV is guided away from any possible restricted area. In other words, objective with the highest priority has no conflict with the third objective. However, moving in a direction to minimize the threat exposure may be in conflict with the second objective, i.e. staying in Dp . Thus, in State-4, an efficient compromise between the second and the third objective should be formulated while considering that the second objective has a higher priority. As stated earlier, the third objective is quantified by the steepest descent direction, ψmin (k). Namely, if the third objective was the only concern, ψmin (k) would be the desired heading. On the other hand, if the UAV was commanded to fly tangent to Cp , the UAV would never leave Dp . Thus, a direction tangent to Cp is defined to quantify the second objective. Recall that the computation of ψmin (k) is already introduced in Section II. Now, calculation of the tangent direction will be presented. Then, the method will be explained that computes ψdes (k) based on ψmin (k) and the tangent direction to quantify the trade-off between the two conflicting objectives. First, the admissible heading range is determined to ensure that the dynamic constraints of the UAV are not violated. Furthermore, any direction that would certainly move the UAV outside Dp (k) should be eliminated. Thus, VAHR = VHC ∩ VP R .

(7.15)

VHDC is not considered (i.e Factor 3 is not considered) in this state because Factor 2 is already YES.

Note that the most likely heading of the UAV is ψmin (k) provided it is

111 ψ

y

y

tg

1

L

proximity circle

y r Target

L2

tg

ψ

temp

UAV ∧

r

t

r

y y

ψ L1

x

L

tg

2

L2

x

Figure 7.5. Tangent heading bounding the UAV turn. within VAHR . Hence, a temporary heading, ψtemp (k) is defined to be ψmin (k) if ψmin (k) ∈ VAHR and, otherwise, the boundary of VAHR that is closest to ψmin (k). ψtemp (k) is used to define a local coordinate system, as shown in Fig. 7.5, whose origin is at the current UAV position and y-axis, yL , has angles ψtemp (k) from the positive x-axis of the inertial reference frame. The two points where yL intersects Cp are defined to be yL1 and yL2 . Appendix A explains how yL1 and yL2 are calculated. Note, from Fig. 7.5, that yL1 (k) is always negative and yL2 (k) is always positive in State-4 since the UAV is in Dp . Since the UAV is likely to head towards yL2 , the two tangent directions of Cp at this point need to be computed. Recall that Cp is centered at the predicted target position, b rt (k + 1), at the next update instant. If the radial

direction, rtg (k), from the center of Cp to yL2 is known, then the tangent directions are easily computed as they are normal to the radial directions, as shown in Fig. 7.5. Note that rtg (k) = ryL (k) − b rt (k + 1) 2

(7.16)

112 where ryL (k) is the position vector of point yL2 relative to the inertial reference frame. 2

By employing the inverse of the transformation used in Eq. (A.2), the representation of ryL (k) in the inertial frame is computed as 2   ryL2 (k) = RTLI (k) 

0

yL2 (k)



  + ru (k)

(7.17)

After carrying out matrix multiplication and addition, Eq. (7.17) yields ryL (k), written 2

as a vector, ryL (k) = xu (k) + yL2 (k) cos[ψtemp (k)] Ib 2 + yu (k) + yL2 (k) sin[ψtemp (k)] Jb

(7.18)

Substituting Eq. (7.18) in Eq. (7.16) yields rtg (k) = xu (k) − x bt (k + 1) + yL2 (k) cos[ψtemp (k)] Ib + yu (k) − ybt (k + 1) + yL2 (k) sin[ψtemp (k)] Jb Then the tangent angles, shown in Fig. 7.5, can be calculated as: yt (k+1)+yL2 (k) sin[ψtemp (k)] π −1 yu (k)−b ψtg1,2 (k)=tan ∓ xu (k)−b xt (k+1)+yL2 (k) cos[ψtemp (k)] 2

(7.19)

(7.20)

Among the two tangent directions, ψtg1 and ψtg2 , the one, referred to as ψtg (k), that is closest to the current UAV heading, ψU AV (k), is selected to quantify the second objective. For example, in Fig. 7.5, ψtg (k) = ψtg1 (k) since it is closer to the UAV heading. Once the two objectives are quantified by ψtemp (k) to minimize threat exposure and ψtg (k) to stay with the proximity of the target, the next step is to consolidate these two possibly conflicting objectives in an efficient way. Let utemp (k) and utg (k) be the unit vectors along ψtemp (k) and ψtg (k) directions, respectively. Namely, utemp (k) = cos ψtemp (k)Ib + sin ψtemp (k)Jb utg (k) = cos ψtg (k)Ib + sin ψtg (k)Jb

(7.21) (7.22)

113 Then, the vector with the desired heading direction is constructed as the weighted vectorial sum of rtemp (k) and rtg (k) as rdes (k) = 1 − α(k) utemp (k) + α(k) utg (k)

(7.23)

where α(k), a real number between 0 and 1, is used to quantify the level of trade-off between the two objectives. Note that when the UAV is close to the center of Cp , there is no immediate danger of leaving Dp and thus minimizing the threat exposure should be the primary objective. This implies that α(k) should be 0 so that rdes (k) = utemp (k). On the other hand, when the UAV is close to Cp , there is an immediate risk of leaving the proximity of target and thus rdes (k) = utg (k) (i.e. α(k) = 1) so that the UAV does not head towards outside of Dp . When the UAV is between these two extreme cases, α(k) should take a value between 0 and 1 so that the two objectives are consolidated. This logic is formulated by a scheduling scheme for α(k) as    1, αsch (k) ≤ αtg    αsch (k)(αtemp −1) α(k) = , αtg < αsch (k) ≤ αtemp αtemp −αtg      0, otherwise

(7.24)

In this scheduling, αtg and αtemp are strategy design parameters and αsch (k) quantifies how close the UAV is to leaving Dp as αsch (k) =

yL2 (k) 2 rp

(7.25)

Note that when αsch (k) is close to 0, the UAV is actually close to Cp (k) but heading towards inside of Dp (k) and thus this case is not considered as a danger of leaving Dp (k). By using the angle of the resultant vector in Eq. (7.23), the desired heading angle in State-4 is computed as ( ) 1−α(k) sin ψ (k)+α(k) sin ψ (k) temp tg ψdes (k) = tan−1 1−α(k) cos ψtemp (k)+α(k) cos ψtg (k)

(7.26)

114 In States-5 to -8, exactly the same strategies as in States-1 to -4, respectively, for computing ψdes (k) have been implemented. The only difference in these states is that there is a LOW threat region instead of having NONE threat region. Thus, VHC is replaced by VSHR during the calculation of VAHR in all these states. In State-9, the objective with the highest priority is not considered to be “achieved” since Factor 1 is HIGH. Thus, the UAV should be commanded to make a turn to stop approaching the restricted area. Since ψmin (k) is the direction of the sharpest descent, it is selected to be ψdes (k). Furthermore, the UAV should make the turn as fast as possible, thus, VAHR is selected to be VHC to allow the UAV to make the sharpest turn possible.

7.1.4

Computation of Desired Speed and Admissible Range In the previous section, the computation of the commanded heading at each update

instant is presented. Once the commanded heading, ψcmd (k), is computed at the k th update instant, the strategy computes the desired speed, Vdes (k), based on ψcmd (k) and the current decision state. Furthermore, an admissible speed range is determined based on the speed and acceleration constraints of the UAV. This section presents the computation of Vdes (k) and the admissible speed range. Note that during a pursuit mission, the target speed might be varying drastically. Further, the path of the UAV, determined by the commanded heading, might be significantly different from that of the target because the strategy steers the UAV to avoid restricted areas and minimize threat exposure level. Thus, the commanded speed is determined to address objective-2, i.e. to help maintain or obtain the proximity of the target in almost all decision states. The exception is State-9 where the commanded speed is determined to help avoid restricted areas. In States-1 to -8, the desired speed is calculated by a proportional control algorithm based on two different error signals. The first one, speed error, is the difference between

115 the speed of the UAV and the estimated target speed. This is used to ensure that the UAV speed will be adjusted as the target speed varies. The second one, the position error, is defined to quantify the distance between the UAV and Dp .

y

proximity circle

y v

ψ

v

v

(x t, y t)

y

t

L

L2

Target ∧

r

t

y

L1

ψ UAV r

e

cmd

(x u ,y u) UAV

x

L x

Figure 7.6. Inertial and UAV local coordinate systems.

To compute the position error, a local reference frame is defined as shown in Fig. 7.6, such that its origin is at the UAV position and its y-axis, yL , has an angle of ψcmd (k) from the positive x-axis of the inertial frame. Note that this local frame is similar to the one used in the previous section except the angle used to define the orientation relative to the inertial frame. Then, the position error, e(k), is defined to be the arithmetic mean of the two intersection points of yL -axis with Cp . Namely, e(k) =

yL1 (k) + yL2 (k) 2

(7.27)

Note that yL1 and yL2 can be calculated by using the same approach detailed in Appendix A with ψcmd (k) that replaces ψtemp (k). In addition to the error calculation, the

116 signs of the intersection points determine the position of the UAV relative to Dp . If both yL1 (k) and yL2 (k) are positive, the UAV is outside but commanded towards Dp . If one of them is positive and the other is negative, then the UAV is in Dp . If both are negative, Dp is behind the UAV and the UAV is commanded away from Dp . Furthermore, when yL -axis does not intersect Cp , i.e. Eq. (A.6) does not have real solutions, the UAV is outside of Dp and commanded neither towards nor directly away from Dp . The proportional control algorithm is formulated as Vdes (k) = VU AV + Ks Vbt (k) − VU AV (k) + Ke e(k)/Ts

(7.28)

where Vbt (k) is the estimated target speed, Ks and Ke are the proportional gains for the

speed and position errors, respectively. Note that different values can be assigned to the gains in different states. For example, for the simulations presented in Section 7.2, in State-1 and State-5, the gains are twice as big as the values used in other states. This is because, in these states, the UAV is outside Dp but heading towards it (Recall that Factor-4 is YES). Thus, a greater speed increase should be commanded so that it will take the UAV a shorter time to attain Dp . As stated earlier, in State-9, the objective with the highest priority needs to be addressed. Namely, there is a HIGH risk of incursion into a restricted area and the sharpest descent direction is commanded to turn the UAV away from the restricted area. The speed command is also utilized to improve the performance of the strategy. Note that the lower the speed of a UAV, the sharper turns it can make. Thus, the minimum speed possible, given the deceleration constraint of the UAV, is commanded, i.e. Vdes (k) = VU AV (k) + amin Ts .

(7.29)

Once the desired speed is determined, the admissible range for the speed should be determined to compute the commanded speed so that the speed and acceleration con-

117 straints of the UAV are not violated. The upper and the lower bounds of the admissible speed range are calculated as Vmaxconst (k) = min Vmax , VU AV (k) + amax Ts Vminconst (k) = max Vmin , VU AV (k) + amin Ts 7.1.5

(7.30) (7.31)

Scheduling Scheme for Heading Difference Constraint Recall that VHDC is introduced to serve as a strategy imposed constraint on the

heading of the UAV. This is necessary because the turning radius of a vehicle increases as its speed increases. If the heading difference is not bounded, the strategy may change the UAV heading drastically to minimize the threat exposure when the UAV is within Dp (k). This, in turn, may increase the risk of the target getting outside Dp (k) and eventually outside the sensor range. To restrict the motion of the UAV in and around the direction where Dp (k) is heading, ψHDC (k) is introduced. However, as stated earlier, this constraint, if it was fixed, would become a liability in the case of low speed and even more so when the target stops. Thus, a scheduling scheme is developed to impose a heading difference constraint in high speed pursuit and to relax it when the target moves slow or stops. The scheduling should be done in such a way that no discontinuity is introduced in the computation of the commanded heading. According to the scheduling ∗ scheme used (see Fig. 7.7), ψHDC (k) is set to a constant, ψHDC , when the estimated target

speed, Vbt , is high (i.e. greater than Vthres2 ); when the target slows down, the constraint is

gradually relaxed by linearly increasing ψHDC (k). This increase is performed with such a slop that ψHDC (k) becomes 180◦ when Vbt is equal to another threshold, Vthres1 . When

the target moves very slow or stops (i.e. Vbt is less than Vthres1 ), ψHDC (k) is set to 180◦ , which effectively removes the constraint.

118 ψ

HDC

π ψ* HDC

V

thres

1

V

thres

2

V est

Figure 7.7. Heading difference constraint angle ψHDC . 7.1.6

Detection of Local Minima Note that umin (k) is the sharpest descent direction at the current update instant,

k. Thus, the directional derivative of PTEM in this direction Dumin (k) f (r(k)) ≤ 0

(7.32)

Let r(k + 1) be the position vector of the UAV at the next update instant, k + 1 if it flies with the current speed in the sharpest descent direction. The directional derivative at r(k + 1) in the direction of umin (k) is referred to as Dumin (k) f (r(k + 1)). Thus, the local minima is considered to be present ahead if (see Fig. 7.8) Dumin (k) f (r(k)) · Dumin (k) f (r(k + 1)) < 0

(7.33)

When this condition occurs there is a local minimum of the PTEM ahead if the UAV would fly in the sharpest descent direction. If the strategy keeps commanding ψmin through such a local minima, the simulation experiments have shown that the commanded heading may show an unnecessary oscillation. To prevent this, during the occurrence of local minima, a vectorial sum of the two sharpest descent directions is computed as umin,avg (k) = umin (k) + umin (k + 1). Thus, the angle of this new direction is −1 sin ψmin (k) + sin ψmin (k + 1) ψmin,avg (k) = tan cos ψmin (k) + cos ψmin (k + 1)

(7.34)

(7.35)

119 which replaces ψmin (k) in the current execution of the strategy.

5

4

0.025

3

u (k) min

2

u (k+1) min

1

0

−1

local minima 4

5

6

7

8

Figure 7.8. Detection of local minima.

7.2

Simulation Results and Comparison with the Point Search Guidance The performance evaluation of the guidance algorithm presented in this chapter

(referred to as NEW algorithm) and its comparison with the algorithm explained in Chapter 6 (referred to as OLD algorithm) are carried out in an integrated simulation environment described in Chapter 6-Section 6.2.1. For a fair comparison, both of the guidance algorithms are implemented with the same estimation technique given in Section 5.2 To test the full capabilities of the strategy, 3 different tracking cases, which are representative of most likely scenarios, are simulated. In the 30 km X 30 km area

120 Table 7.2. Parameters: Initial Position (IP) [km], Speed (IS) [km/h], Heading (IH) [deg], Sensor Range (SR) [km], Proximity Range (PR) [km]

Case 1 2 3

Target UAV IP IS IH IP IS IH (7,-13) 80 97.40 (8,-14) 144 60 (2.85,-13) 144 23.00 (4,-12) 144 60 (4,-12) 80 57.30 (5,-11) 144 60

SR PR 2.0 1.5 2.0 1.5 2.0 1.0

of operation, there are 17 threat sources along with the restricted regions. In all the simulations, the UAV is considered to have a minimum speed of 72 km/h, a maximum speed of 180 km/h, a maximum acceleration of 2 m/s2 , a maximum deceleration of -2 m/s2 , and a turning rate of 10 deg/s with a guidance and estimation update periods of 3 seconds. The first order transfer functions have time constants 0.5 and 0.01 for heading and speed responses of the UAV, respectively. Strategy design parameters fr ∗ and ψHDC are selected to be 0.025 and 45 deg, respectively. Scheduling parameters for

heading difference constraint Vthres1 and Vthres2 are Vmin and 0.2 Vmin , respectively. The α-scheduling parameters αtg and αtemp are 0.05 and 0.2, respectively. The proportional controller gains for the speed and position errors are 0.5 and 0.0004, respectively. The derivative gain, KD , in Eq. 7.13 is 10 for States -1 and -5, and 60 in States -2 and -6. The standard deviations of the noise added to the x and y positions of the target to obtain the measurements are selected to be 0.05 km for Cases 1 and 2 and 0.1 km for Case 3. The initial conditions, sensor and proximity ranges in each case are given in Table 7.2. In each case, the target is tracked first by the new algorithm and then by the old algorithm. The performance of each algorithm is illustrated by presenting the UAV trajectory (see Figs. 7.9, 7.11 and 7.14) and commanded signals (see Figs. 7.10, 7.12 and 7.17).

121 UAV trajectory generated by OLD algorithm

0.

02

5

UAV trajectory generated by NEW algorithm

0.025

0.025

0.

02

02

5

0.0 2

y(km)

5

02

5 02 0. 0.025

0.

5

0

0.025

0.025 0.02

02

0.

25

0.025

−10

5

0.025

5

0.025

0.0

y(km)

0.0−5 25

−5

5

0 0.025

0.025

5

0.025

5

0.

0.025

5 02

10

0.025

25

10

0.

UAV path Est. target path Act. target path Proximity disk Sensor range

0. 0

UAV path Est. target path Act. target path Proximity disk Sensor range −15

−10

0.0

−10

25

5 0.02

−15 0.0 −10 25

−5 x(km)

0

−5 x(km)

0

5

5

Figure 7.9. Case 1:UAV trajectories following an accelerating target. Commanded headings

ψ(deg)

200 150 100 50 0

OLD algorithm NEW algorithm 100

200

300

400

500 t(sec)

600

700

800

900

1000

V(km/h)

Commanded speeds 180 140 100 60 0

OLD algorithm NEW algorithm 100

200

300

400

500 t(sec)

600

700

800

900

1000

Figure 7.10. Case 1: Commanded signals for the UAV. Figs. 7.9 and 7.10 show the first case where an accelerating target is moving on a third order path according to the following functions of time for speed in m/s and heading in radians Vt (t) = 1000 sin(0.00002 t) + 0.022 3 t t ψt (t) = 1.7 − + + 3 sin(0.002 t) 200 650

(7.36) (7.37)

122 Target accelerates until it reaches its maximum speed and continues with this speed. As seen from Fig. 7.9, target passes through three restricted regions. UAV, when guided by the new algorithm, avoids these regions while continuing the pursuit of the target even if target gets outside the sensor range during the second and the third restricted regions. When guided by the old algorithm, UAV cannot reattain the target once lost during the third restricted region. Fig. 7.10 shows that the new algorithm commands more turns while the commanded speed seems to involve more accelerations with the old algorithm. However note that both algorithms generates commanded signals within the specified

UAV trajectory0.generated by OLD algorithm 02 5 0. 02 5

10

0.025 0. 02

y(km)

0.025

25

0.025

−5 0.0 2

0.025

5

5 02

02

0.0

0.025

0.025

0.

0.

5

5

0.025

−10

0.0 2

y(km)

5

0 0.025

5 02

0.

−15

−15

−10

−15 0.025

−5 x(km)

0

0.0

5 0.02 −5 x(km)

−10

25

−15 −10

UAV path Est. target path Act. target path 0.025 Proximity disk Sensor range

5

0.02

0.025

−5

UAV path Est. target path Act. target path Proximity disk Sensor range

0.025

UAV trajectory generated by NEW0.algorithm 02 5 0. 02 5 0

5

0.025

5

10

0.025

dynamic constraints.

0

5

5

Figure 7.11. Case 2: UAV trajectories following a slowing target.

123 Commanded headings OLD algorithm NEW algorithm

ψ(deg)

400 0 −400 0

500

V(km/h)

180

1500 2000 t(sec) Commanded speeds

2500

3000

3500

1500

2500

3000

3500

OLD algorithm NEW algorithm

120 60

1000

0

500

1000

2000 t(sec)

Computation time (sec)

Figure 7.12. Case 2: Commanded signals for the UAV. 0.6 0.4 OLD algorithm NEW algorithm

0.2 0 0

500

1000

1500

t(sec)

2000

2500

3000

3500

Figure 7.13. Case 2: Computation time of the algorithms at each update instant. Figs. 7.11, 7.12 and 7.13 show the second case where the target, while moving on the path shown in Fig 7.11, reduces its speed to 36 km/h in 600 seconds and then maintains it in the rest of the pursuit. Target moves according to the following functions of time for speed in m/s and heading in radians Vt (t) = 1000 0.040 − sin(0.00005 t) 3 t t + + 4.8 sin(0.002 t) ψt (t) = 0.4 − 225 1050

(7.38) (7.39)

Note that the speed of the target is much less than the minimum speed of the UAV during the most of the pursuit. Thus, guided by either algorithm, the UAV reduces its speed to the minimum speed, as shown in Fig. 7.12, and tries to loiter within the proximity disk when there is no restricted area while minimizing the threat exposure level as shown in Fig. 7.11. Loitering of the UAV can also be seen from Fig. 7.12, which shows the commanded headings. In both cases, the commanded heading seems constantly changing

124 since the UAV is constantly commanded to make turns to minimize the threat exposure and stay within the proximity disk. However, note that the commanded heading with the old algorithm has low-magnitude but high-frequency oscillations, particularly during 750-1300 sec and 1500-1800 sec when the minimizing direction is towards the outside of the proximity circle. The new algorithm does not generate these oscillations because the utilization of the tangent direction of the proximity circle. Note that there is no pre-defined loitering mode in either algorithm. In fact, there is no need for the strategies to have a seperate loitering mode because the strategies put the UAV in loitering autonomously based on the speed of the proximity circle and the local PTEM within the proximity disk. Fig. 7.13 shows the computation time (CT) required by the algorithms at each update instant. This information is provided only in this case because the CT does not depend on the scenario simulated. First note the variation in the CT of the old algorithm shows that its each strategy-state requires different CTs while CT of the new algorithm is almost uniform. More importantly, Fig. 7.13 demonstrates the significant improvement in the computational efficiency of the new algorithm over the old one. Note that the instantaneous spikes were due to the CPU usage by the operating system during the simulations. The third case is shown in Figs. 7.14, 7.15, 7.16 and 7.17. In this case target continuously reduces its speed and stops at 660 seconds in a region where there is no restricted area by using the functions of time for speed in m/s and heading in radians Vt (t) = 1000 0.035 − sin(0.000054 t) 3 t t + + 3 sin(0.002 t) ψt (t) = 1 − 220 615

(7.40) (7.41)

125 UAV trajectory generated by NEW algorithm

0.

10

02

5

y(km)

y(km)

0.02 UAV path 5 Est. target path .025 Act. target path −5 0 Proximity disk Sensor range

0

0.025 UAV path Est. target path .025Act. target path −5 0 Proximity disk 5 0.02 Sensor range

0.025

0

0.025

0.025

0

6

8

5

0.02

0.025

−10 2 4 x(km)

0.025

0.025 5

0.025

0.025 5

−2

0.

02

5

−10

0.025

0.025

10

UAV trajectory generated by OLD algorithm

−2

0

2 4 x(km)

6

8

Figure 7.14. Case 3: UAV trajectories following a move-stop-move-stop target. After staying in this region for 140 seconds, the target, at 800 seconds, starts accelerating and reaches its maximum speed at 1000 seconds by using the functions of time for speed in m/s and heading in radians Vt (t) = 1000 − 0.1593 + sin(0.0002 t) 3 t t ψt (t) = 1.6355 − + + 3 sin(0.002 t) 168 650

(7.42) (7.43)

At 1100 seconds, it starts decelerating again and stops at 1360 seconds in a restricted region during the rest of the pursuit as seen from Fig. 7.14. Target completes its trajectory according to the following functions of time for speed in m/s and heading in radians Vt (t) = 1000 0.040 − sin(0.00015 (t − 1100)) 3 t − 1100 t − 1100 ψt (t) = 2.3489 − + + 3 sin(0.002 (t − 1100)) 190 650

(7.44) (7.45)

126 UAV trajectory generated by OLD algorithm

UAV trajectory generated by NEW algorithm 0

−1

−1 y(km)

y(km)

0

−2

−3

−2

6

8

−3

6

x(km)

8 x(km)

y(km)

Figure 7.15. Case 3: When the target stops in a non-restricted region. 9 0. 02 5 8.5 8 7.5 −2.5

−2

−1.5 −1 x(km)

−0.5

0

Figure 7.16. Case 3 (New Algorithm): When target stops in a restricted area. This case shows the full capability of the strategies for autonomously putting the UAV in loitering mode both with and without restricted areas close-by. As seen from Fig. 7.15, when the target stops in a region where there is no close-by restricted region, the new algorithm puts the UAV in loitering around the local minimum of the PTEM inside the proximity circle. The old algorithm, however, cannot keep the UAV within the proximity circle while trying to put the UAV in loitering around the local minimum. This once again shows the benefit of utilizing the proximity-circle tangent-direction. Also note from Fig. 7.14 that when the target stops the second time within a restricted area, a small portion of the proximity circle is still outside the restricted area. The new algorithm loiters the UAV around this portion of the proximity circle (see Fig. 7.16) while the old algorithm flies the UAV around the restricted area. This shows the benefit

127 of utilizing the pursuit guidance, with the right choice of the gain, based on the LOS angle and its derivative in the new algorithm. Further, the UAV guided by the old algorithm permanently loses the target while flying around the restricted area because Commanded headings

ψ(deg)

1000


500 0 −500 0

200

V(km/h)

180

400

600

800 1000 Commanded speeds

1200

1400

1600

1800

1200

1400

1600

1800


120 60 0

200

400

600

800

1000 t(sec)

Figure 7.17. Case 3: Commanded signals for the UAV.

the estimation deviates too much due to the lack of measurement. Fig. 7.17 shows that the commanded signals by both algorithms are similar except during the times when the heading signal of the old algorithm has high-frequency low-magnitude oscillation. The increase in the commanded signal by the old algorithm during the last portion of the simulation is due to the fact that the estimation falsely shows acceleration of the target.

Table 7.3. Comparison of the algorithms Case 1 2 3

TELR LR TIPR ACT (sec) O N O N O N O N 0.41 0.34 1.06 1.11 0.46 0.40 0.360 0.037 0.14 0.15 1.70 1.72 0.91 0.82 0.375 0.036 0.25 0.21 1.90 1.78 0.65 0.68 0.355 0.036

Table 7.3 is constructed to provide a quantitative comparison of the two algorithms. TELR (Threat Exposure Level Ratio) is defined to be ratio of the cumulative threat

128 exposure level of the UAV to the target. The cumulative threat exposure levels for the UAV and the target are calculated by integrating PTEM over the respective trajectories and can be used to quantify the risk of flying through that trajectory. LR (Length Ratio) is defined to be the ratio of the UAV trajectory length to the target trajectory length. TIPR (Time In Proximity Ratio) is the ratio of the total time when the UAV stays within the proximity of the target to the total simulation time. In the calculation of TIPR for each algorithm, the strategy-states that corresponds to the UAV being within the proximity circle are used, e.g., for the new strategy, the total time when the strategy is in states-4 or 8. Average computation time (ACT) is calculated by averaging the calculation time per update instant over the whole simulation time. For example, for case-2, ACT for each algorithm is the average of the plots given in Fig. 7.13. Table 7.3 shows that the new algorithm outperforms the old algorithm in cases-1 and -3 and performs very close in case-2 in terms of threat exposure level. In terms of trajectory length and time spent within the target proximity, the old algorithm performs better in cases-1 and -2 but in case-3, the new algorithm generates better results. In terms of the computational efficiency, the new algorithm is about 10 times better than the old algorithm. Therefore, it can be concluded that the new algorithm has improved the computational efficiency without compromising the performance and even slightly improved the performance in some cases. Improvement in the computational efficiency is mainly due to the fact that the gradient search approach is utilized, which eliminates the need to evaluate the PTEM at numerous search points. Further, the introduction of the pursuit-guidance techniques based on LOS angle, proportional control for speed command and the weighted vectorial summation of minimizing direction and proximity-circle tangent helps maintain the level of performance.

129 7.3

Feasibility and Implementation of the Algorithm To quantify the computational feasibility of the algorithms, an approximation of

MIPS (Million Instructions Per Second) that each algorithm requires is computed. In computation of MIPS, each of four basic operators is counted as one instruction, a logical operator is counted as 2 instructions, each trigonometric function is counted as 25 instructions, square-root operation is counted as 15 instructions and so on [152]. According to this approximation, the new algorithm requires (20000+ 1800 × N ) instructions, where N is the number of Gaussian distribution functions that constitutes the PTEM. For example, the new algorithm, when N = 17 as in the simulations and with the update rate of 1 Hz requires 0.05 MIPS. As a comparison, the old algorithm, under the same conditions, would require 0.4 MIPS by the formula (79000 + 600 × N (Ns + 2)), where Ns is the number of search points (Ns = 30 in [8]). Note that the MIPS of the old algorithm is about ten times larger than that of the new one, which is consistent with the ratio of computation times observed in the simulation. This significant improvement of the computational efficiency in the new algorithm, as stated earlier, is mainly due to the smaller number of PTEM evaluations required. The old algorithm uses an approach of evaluating PTEM on a discrete set of search points on two circular arcs to approximately find a feasible direction to minimize the threat exposure level and avoid restricted areas. However, with the gradient approach used in the new algorithm, only two directional derivatives of PTEM, computed using analytical expressions provided, are sufficient to determine the exact direction that leads to minimum threat exposure at a given position. The algorithms are programmed as M-file S-functions to be executed within the Matlab/Simulink-based simulation environment. To generate a code that is executable in a microprocessor, Real-time Workshop [153] of Matlab can be utilized. This toolbox generates optimized, portable and customizable ANSI C or C++ code from Simulink Models to create stand-alone implementations of models that operate in real-time. Since

130 our S-functions are written as M-files, a TLC (Target Language Compiler) file for Ccode generation should be provided for each block. Alternatively, the S-functions can be rewritten as C or C++ file. The optional Real-Time Workshop Embedded Coder works with Real-Time Workshop to generate efficient embeddable source code. These tools support a growing set of targets such as Motorola MPC555 and TI C6000. The embedded target or Motorola MPC555 enables the deployment of the production code generated from Real-Time Workshop Embedded Coder directly on to MPC5xx microcontrollers. The MPC56x series processors can perform up to 100 MIPS at a clock rate of 66 MHz [154]. MPC555 is a floating point processor that performs up to 63 MIPS at a clock rate of 40 MHz. This processor is being used in many UAV applications including Piccolo family of autopilots. TI C6000 supports C6711 which is a 32-bit floating point processor that performs up to 1200 MIPS at a clock rate of 150 MHz. Another approach is to utilize SDK (Software Developer’s Kit) feature of a commercial-off-the-shelf (COTS) autopilot to implement the algorithm. For example, XT endermp of MicroPilot SDK enables the implementation of user codes, called mp plug-ins, that are run simultaneously with MicroPilot autopilot code. These mp plug-ins can be used to collect and process data from external sensors, implement specialized guidance and control algorithms and override existing ones, and access unused processor hardware. The current microprocessor used in MicroPilot is a Freescale MC68332 with 7.5 MIPS at a clock rate of 25 MHz. An upgrade to the 68332, which is called 523x series, provides a better performance with 144 MIPS at a clock rate of 150 MHz.

CHAPTER 8 COOPERATIVE GUIDANCE ALGORITHM 8.1

Cooperative Target Following Strategy The cooperation strategy, presented in chapter, generates way points in such a

way that total threat exposure level of the team of the UAVs as well as the distance from which the UAVs follow the target are small. First, a cost function of cooperation is defined to quantify these possibly conflicting requirements and the tradeoff between them. The cost function is the weighted sum, over all UAVs in the team, of the minimum value of PTEM on a given proximity disk and the distance of the minimizing point to the target. Each UAV is assigned a proximity disk and the proximity disks are distributed around the target in a formation such that the target is always in the area of intersection. Based on the cost function, the cooperation strategy, developed in this chapter, determines how to vary the radii of the proximity disks at discrete instants such that the cost function is minimized. The guidance algorithm, developed in Chapter 7, is employed to track the way points while avoiding the restricted areas and obstacles. Another form of cooperation among the UAVs is implemented in the estimation of target trajectory based on noisy measurements. If the UAVs are in communication range, they receive, with a delay, each other’s measurements of the target. If target measurement is not available for the host UAV, measurements from the other UAV will substitute the measurements required for the estimation. If measurements for both the host UAV and the other UAVs are available, an optimal data fusion developed in Chapter 5-Section 5.4 is employed to improve estimation performance. When no measurement is available, estimation algorithm is still utilized to predict the target motion. 131

132 8.1.1

Formulation of Cooperation When a dynamic target is to be pursued in an adversarial environment, there are

two main requirements as explained in previous chapters. The pursuers should stay close to the target and their threat exposure should be minimized at a given time. The first requirement is quantified by defining a proximity disk around the target and the second requirement is quantified by PTEM. If the pursuit needs to be carried out by multiple vehicles, a cooperative scheme should be defined to carry out the common task in a more effective manner. To relate the possibly conflicting objectives in a cooperative manner, a cost function is defined that take into consideration the requirements for the entire team, rather than for individual pursuers. Jt = Ktc

N X j=1

fmin,j + α Kdc

N X

dmin,j

(8.1)

j=1

where N is the number of pursuer UAVs, fmin,j is the minimum value of PTEM on the proximity disk associated with j th UAV, dmin,j is the distance to the target from the point where the value of PTEM is fmin,j , and Ktc and Kdc are the respective weighting coefficients. Note that α is used as a normalization factor to ensure that the distance term and threat exposure term assume values in the same order. Note also that the calculation of fmin is carried out as explained in Section 3.6. When the cardinality of Smin is greater than 1, i.e. there are multiple points with the same minimum value, the one with the smallest distance to the target location is selected as this obviously minimizes the cost function. The rationale for this cost function is that the UAVs should be placed where PTEM has minimum value within their corresponding proximity disk. That is, the UAVs will satisfy the first requirement by remaining within their proximity disk and the second requirement by staying where threat exposure is minimum. For a given target position in PTEM, the values of fmin and dmin depend on the formation of the proximity disks

133 2.5 2 0.025

1.5

d

TARGET

min,1

fmin,1

y(km)

1 dmin,2

0.5 0 −0.5 fmin,2

−1 1

2

3

4

5

x(km)

Figure 8.1. Construction of cost functions. and their radii. While increasing radius may yield a smaller fmin , it may result in longer dmin (see Fig. 8.1). This helps quantify the trade-off between the two requirements when they conflict. Further, through the summation over all pursuer UAVs in a team, the cost function quantifies the requirements and their trade-off for all UAVs as a team in a cooperative manner. The weighting coefficients are set such that Ktc + Kdc = 1

(8.2)

and they can be adjusted according to mission requirements. When threat minimization of the UAVs is the only requirement, Ktc = 1, which means that the distance of the UAVs to the target is not a concern for the mission. On the other hand, when the UAVs should follow the target as close as possible without considering threat exposure, Kdc = 1. In

134 the cases where both requirements should be considered, the weighting coefficients are selected between 0 and 1 to quantify a desired level of tradeoff.

a1

rp1

θ

rp2

a2

Figure 8.2. Formation of proximity disks around target.

Figure 8.2 demonstrates the parameterization of the proximity disk formation around a target in a two-UAV case. The formation of the proximity disks is determined by distances a1 , a2 and angle θ. The centers of the proximity disks are always located on the formation line, which makes angle θ with the normal direction to the target heading. The intersection points of the proximity circles with the formation line are parameterized by a1 and a2 , which are chosen such that the target is always located in the intersection of the proximity disks. Note that, with constant a1 and a2 , as radii rp1 and rp2 change, the centers of the proximity disks move on the formation line. Further, as θ increases, the proximity disks rotate around the target in the counter-clockwise direction while keeping the target in their intersection area. For example, when θ = 0, one proximity disk is located on the right and the other on the left relative to the heading of the target. When θ = 90 deg, one proximity disk is ahead of and the other behind the target.

135 8.1.2

Implementation of Cooperative Strategy A cooperative strategy, based on the cost function defined in Eq. (8.1) should

determine how to change the sizes of the proximity disks and their formation so that the cost function is minimized as the target moves in the area. An underlining assumption here is that a guidance strategy is employed by each UAV to stay at the point with minimum PTEM on its proximity disk as the target moves, and the radius and the formation of the proximity circle are adjusted by the cooperative strategy. Note that, for a fixed formation of the proximity disks, the cost function in Eq. (8.1) depends only on rp1 and rp2 at a given location in PTEM. The cooperative strategy developed in this research focuses on adjusting rp1 and rp2 to minimize the cost function as the target is pursued. Target pursuit is initiated with specified initial values of rp1 and rp2 . At every update instant of the cooperation strategy, a gradient search approach is utilized to determine how to adjust rp1 and rp2 simultaneously so that the value of the cost function is reduced the most. Note that this approach can as easily be applied for N number of UAVs. In the two-UAV case, the partial derivatives of the cost function with respect to rp1 and rp2 need to be calculated. It is, however, difficult to determine analytical expressions for the partial derivatives. Instead, the partial derivatives at the k th update instant are approximated as J r (k) + ∆r , r (k) − J r (k), r (k) t p p p t p p 1 1 2 1 2 Jtrp1 (k) ∼ = ∆rp1 J r (k), r (k) + ∆r − J r (k), r (k) t p p p t p p 1 2 2 1 2 Jtrp2 (k) ∼ = ∆rp2

(8.3) (8.4)

where ∆rp1 and ∆rp2 are the increments for the radii of the proximity disks and need to be selected as strategy parameters. Once the partial derivatives are computed, then, as

136 explained in Chapter 3-Section 3.2, “the minimizing direction” or “the steepest descent direction” can be calculated as −1

ψJtmin (k) = tan

−Jtrp2 (k) −Jtrp1 (k)

!

(8.5)

Then, the cooperative strategy updates the radii of the proximity disks as rp1 (k + 1) = rp1 (k) + ∆rp cos ψJtmin (k)

(8.6)

rp2 (k + 1) = rp2 (k) + ∆rp sin ψJtmin (k)

(8.7)

When the benefit of updating the radii is zero or very small, the cooperative strategy should skip the updates and keep the radii unchanged. This is determined by the magnitude of the gradient of the cost function. The cooperative strategy updates the radii only if the magnitude of the gradient is greater than a threshold value, which is also a strategy parameter.

8.1.3

Improving Computational Efficiency Since this cooperative algorithm operates on PTEM, its computational performance

is directly proportional to the number of summation in Eq. (3.10), i.e., the number of pdf s used to define PTEM. At every update instant, the cooperative algorithm needs to calculate the value of cost function Jt in Eq. (8.1) three times to compute the partial derivatives in Eqs. (8.3) and (8.4). Note from Eq. (8.1) that fmin and dmin for each UAV should be computed. Especially the evaluation of fmin , as explained in Section 3.6, is computationally intensive. To improve the computational efficiency, instead of all pdf s that define PTEM, a small subset of the pdf s can be used in the evaluation of PTEM at a given point in the area. This subset should consist of pdf s that have considerable contribution to the value of PTEM at the point of interest. Fig. 8.3 shows, as an example, a portion of a

137 PTEM that consists of Gaussian pdf s. Note that 99% of the effect of a Gaussian pdf is concentrated within a circle centered at the mean value and with radius of 3 σ, i.e., three times the standard deviation. That is, if a point of interest is beyond the 3 σ circle of a pdf, the effect of that pdf at that point is negligible. Thus, only pdf s that have the point of interest within their 3 σ circles are considered in the evaluation of PTEM. For example, in Fig. 8.3, a UAV at (0.2, −4.2) is affected only by three pdf s. In this study, for the sake of convenience, effective area of a Gaussian pdf is modeled as a circle even for a Gaussian pdf with different σx and σy . In such a case, the bigger standard deviation is used to define the circle of effectiveness. 0.025

0

−2

0.025

−4

y(km)

0.025

−6 5

0.02

−8

25

0.025

0.0

−10

−12 −6

−4

−2

0

2

4

6

8

x(km)

Figure 8.3. Local threats effecting the UAV at a position.

138 Note that when the position of the target is not within the circle of effectiveness of any pdf, then the cost function defined in Eq. (8.1) is zero. This is because the value of PTEM is approximately zero everywhere on any proximity disk, which means that the target position is chosen to be rmin , which, in turn, implies that fmin = 0 and dmin = 0.

8.1.4

Collision Avoidance The cooperative strategy, operating on the defined cost function, guides the UAV

to act as a team to pursue a target while optimizing the tradeoff between following the target closer and reducing threat exposure level. There might be regions of PTEM where the two requirements are not in conflict, i.e. the closer to the target, the smaller the level of threat exposure. In such regions, the cooperative strategy, according to the cost function, would steer all UAVs closer to the target. To avoid potential risk of collision, the cooperative strategy, as executed onboard each UAV, modifies PTEM by adding a pdf for every other UAV in the team. Recall that in PTEM there are regions that are defined as restricted areas where the value of PTEM is greater than a threshold, fr . Restricted areas are used to quantify areas where there are no-fly zones or obstacles and thus the UAVs should never enter. Using the same concept, the proximity of every other UAV operating in the area should be defined as restricted area for each UAV. The only difference is that each of these restricted areas should be moving with the corresponding UAV. This is achieved simply by adding a pdf to PTEM for every other UAV in the area with a mean value that tracks the position of the corresponding UAV as it moves. In the case of Gaussian pdf s, PTEM has an additional pdf for every other UAV whose mean value is always equal to the position of the other UAV. Further, the variances of such pdf s are determined to secure an area around each UAV as restricted areas. These restricted areas are defined as “collision avoidance” circles. Once the radius of such a

139 circle is specified as a strategy parameter, rca , the variance of the corresponding Gaussian pdf is determined from 2 1 r fr = √ exp − ca2 σ 2π σ

(8.8)

Rearranging this equation yields √ 2 2σ 2 ln fr 2π σ + rca =0

(8.9)

which is solved for σ to secure a restricted area around each UAV with the specified radius. Figure 8.4 shows the collision avoided trajectories generated for 4 UAVs flying into a region simultaneously. The radius of the collision avoidance circle around each UAV is selected to be 0.25 km.

12

0.

02

10

5 0. 02

5

y(km)

8

6

4

0.025

0.

02

5

2

0.0

25

0 −10

−8

−6 x(km)

−4

−2

Figure 8.4. Collision avoided trajectories.

140 8.1.5

Guidance Strategy for Each Individual UAV While the cooperative guidance algorithms adjust the radii of the proximity disks

for the UAVs to minimize the cost function for the entire team, rmin , the point on each proximity disk with minimum threat exposure, is computed at every update instant. For the effective implementation of this cooperative strategy, each UAV in the team should be guided to stay at the corresponding rmin as the target moves and the cooperative strategy adjusts the proximity disks. For each UAV to effectively track the corresponding rmin while avoiding restricted areas in PTEM, the gradient search guidance strategy explained in Chapter 7, is employed to generate feasible heading and speed commands. This rule-based intelligent guidance strategy was developed for a single UAV to autonomously pursue a moving target in an adversarial environment. The rules were defined for three objectives in the order of priority as (i) to avoid restricted areas, (ii) to maintain the proximity of the target, and (iii) to minimize threat exposure. The same concepts of PTEM and proximity disk were used to quantify the objectives. The strategy were organized into nine decision states, only one of which is active based on four factors. The factors determined whether the UAV (i) is close to a restricted area, (ii) is in the proximity disk, (iii) has a heading direction close to the heading of the target, and (iv) is heading towards the proximity disk. In summary, the intelligent guidance strategy was used to avoid restricted areas, to move the UAV into the proximity disk and keep it there if possible, and minimize the threat exposure while staying within the proximity disk. Thus, note that the strategy had decision states to guide the UAV towards the proximity disk while avoiding any restricted area. This is the feature that the implementation of the cooperative guidance strategy requires, i.e. to intercept rmin while avoiding restricted areas. In other words, the cooperative guidance strategy generates waypoints for each UAV to follow and, in turn, the intelligent guidance strategy generates commanded heading and speed signals for the UAV to intercept the waypoints.

141 To be able to employ the intelligent guidance strategy without any modification in the structure of the algorithm, the following interface changes are carried out. For the intelligent guidance algorithm, a very small proximity disk is defined that is located at rmin of the corresponding “cooperative” proximity disk. With this change, the intelligent guidance strategy steers the UAV to intercept rmin while avoiding restricted areas. Note that the intelligent guidance algorithm never executes its own threat minimization routines since the UAV never enters the small proximity disk that moves with rmin . This is exactly what is intended because there is no need for the intelligent guidance algorithm to minimize threat as the minimization is already carried out by the cooperative guidance algorithm. Further, since the UAV never enters the small proximity disk at rmin , the intelligent guidance algorithm does not execute its routines that reduce the heading difference between the UAV and the target. This is also intended because the intelligent guidance is now used to track rmin , not the target itself. In fact, with this minor interface change, the intelligent guidance algorithm uses, among the nine original states, only those that operates on the line-of-sight angle to intercept the target (rmin in this case) and that utilize PTEM to avoid restricted areas. Another interface change is in the position error for the calculation of speed command. In the original intelligent guidance strategy, the distance from the UAV position to the intersection of the commanded heading direction with the proximity disk was used in the proportional speed control law. In the current implementation, the line-of-sight distance to rmin is used instead. By employing the intelligent target tracking algorithm, it is guaranteed that each UAV will follow its corresponding rmin while avoiding restricted areas. Further, the heading and speed commands are feasible for the UAVs as the intelligent guidance algorithm takes into account the dynamic constraints of the UAV. Another advantage of employing the intelligent guidance algorithm in tandem with the cooperative guidance strategy is seen in threat exposure minimization. In the current implementation, the UAVs are

142 guided to follow rmin , analytically computed position with minimum threat exposure on a given proximity disk while the intelligent guidance algorithm, as originally developed, extracts local information of PTEM to steer the UAV in the threat exposure minimizing direction.

8.2 8.2.1

Simulation Environment and Results Modular Structure of the Simulation Environment The guidance and estimation algorithms are implemented in a modular simulation

environment. As shown in Fig. 8.5, at the top level, the simulation model is organized in three major systems as Target Dynamics, UAV-1 and UAV-2 for simulating cases where two UAVs pursue a target in an adversarial environment. Target Dynamics Module (TDM) generates the trajectory of the target in the area. UAV-1 and UAV-2 modules receive the position of the target from TDM as it is assumed that sensors onboard UAVs measure target position. At this level, UAV modules send each other their position information as well as their measurement of the target. Target Position

Target Dynamics Module

UAV-1 Module

UAV-1 Position Target Measurement

UAV-2 Module

UAV-2 Position Target Measurement

Figure 8.5. Top level modular structure of the simulation environment.

UAV modules are also organized in a modular structure as shown in Fig. 8.6. As the top level view of the system indicates, UAV subsystem receives three sets of inputs:

143 (i) position of the target, (ii) position of the other UAV, and (iii) measurement of the target sent from the other UAV. UAV Dynamics & Control module consists of the model of the UAV Dynamics and the low level controller. In this research, the closed-loop UAV dynamics is modeled as two first-order transfer functions, from the commanded speed and heading to the actual speed and heading. Sensor Module receives x and y components of the actual target position and superimposes them with Gaussian white noise to model the measurement noise. This module also receives UAV position from UAV Dynamics & Control subsystem. UAV position is compared with the actual target position to model the fact that sensors do not provide measurement when the target is beyond the sensor range. Thus, when the radial distance to the target is larger than a specified “sensor range”, Sensor Module does not provide any output.

Target Measurement & UAV position Sent to Other UAV

Position of Other UAV

Target Measurement by Other UAV

Communication Module

Speed & Cooperative Heading & Commands Individual Guidance Algorithms

UAV Dynamics & Control

UAV States

Sensor Module

Target Measurement

Estimation Algorithm

Target Position

Figure 8.6. Modular structure of a UAV subsystem.

Estimated Target States

144 Estimation Algorithm takes target measurements from Sensor Module and that sent from the other UAV when the other UAV is in communication range. The measurements are used to compute the estimates of the target position, speed and heading as explained in Chapter 5. Estimation Algorithm module is not run continuously unlike the other modules discussed above. It is executed only at discrete time instants as specified by the estimation update period. Cooperative & Individual Guidance Algorithm (CIGA) module is also executed only at discrete time instants, but not necessarily at the same rate, i.e., its update period might be different from that of Estimation Algorithm. CIGA module executes the cooperative guidance algorithm and the rule-based intelligent guidance algorithm as explained in Section 8.1. Note that CIGA module receives the position of the other UAV, which is used in two separate routines. Collision avoidance routine as explained in Section 8.1.4 uses the position of the other UAV. Its more subtle use is in cooperative guidance algorithm. Recall from Section 8.1.1 that cooperative strategy minimizes the cost function in Eq. (8.1). As implemented, the cooperative guidance algorithm is run onboard every UAV in the team in a decentralized manner. To compute the cost function onboard a UAV, the position of the other UAV and the corresponding proximity disk as well as its own position and proximity disk should be known. Under the assumptions that the pre-programmed initial radius of the proximity disk for the other UAV is known and that the other UAV runs exactly the same cooperative algorithm, it is sufficient to receive only the position information from the other UAV. Receiving the UAV position and the target measurement, Communication Module sends them to the other UAV if the other UAV is in communication range. Position of the other UAV is an input to this module and used in computing the radial distance between the two UAVs to determine whether they are in communication range.

145 8.2.2

Simulation Results In the simulation environment described above, a case of target pursuit by two

cooperating UAVs is simulated to analyze the performance of the cooperation strategy. Additional cases are simulated with UAVs that pursue the target without any cooperation to quantify the performance of cooperation compared to noncooperative pursuit. In all cases simulated, a target moves through an environment in an area of 15 km by 15 km that is adversarial for the pursuing UAVs. The adversarial area is formulated by a PTEM that consists of 19 Gaussian pdf s with a specified fr . Time dependency of becoming disabled, as formulated in Eq. (3.9), ft (t), is modeled by using the same uniform distribution for all the threat sources. The interval of the uniform distribution is [0, tsim ] where tsim is the simulation time, which is 1100 secs for the case simulated. Note that, l1 and l2 in Eq. (3.9) are selected to be unit length. The target, starting at the initial position of (4,-13) km, moves in the area according to the following functions of time for speed in m/s and heading in radians Vt (t) = 1000 sin(0.00002 t) + 0.022 3 t t ψt (t) = 1.7 − + + 3 sin(0.002 t) 200 650

(8.10) (8.11)

Noisy position measurements onboard UAV-1 and UAV-2 are modeled by adding a zero mean Gaussian noise with 50 and 100 m standard deviations, respectively, to the actual position of the target. The total path length of the target trajectory during the simulation is 35.9 km. Threat Exposure Level (TEL) along the target trajectory is calculated by Eq. (3.9) to be 0.0233. In the simulation, both UAV-1 and UAV-2 are considered to have a minimum speed of 72 (km/h) and a maximum speed of 180 (km/h). Maximum turn rate of both UAVs is 10 deg/sec. Their initial heading and speed are 60 deg and 144 km/h, respectively. UAV-1’s initial position is (8,-14) km and UAV-2’s initial position is (6,-12) km. The

146 UAV-1 and UAV-2 are assumed to have a 2 km and 1.5 km sensor ranges, respectively. Their communication range is 4 km and any data sent are received by the other UAV with a delay of one update period. Figures 8.7 and 8.8 show rmin calculated on the corresponding proximity disk at every update instant and the UAV trajectories that are generated by following rmin as way points, in cooperative and noncooperative cases, respectively. In all cases, the proximity disks at the initial time have the same radius of 1.1 km. Among the cases simulated, cooperative case means that the radius of the proximity disk for each UAV is adjusted by the cooperation strategy. The cooperation strategy, in these simulations, is allowed to increase the radii of the proximity disks, for UAV-1 up to 1.7 km and for UAV-2 up to 1.4 km. For both UAVs, the radii of the proximity disks can be reduced to 0.005 km. This lower limit selected for proximity disk radius is to show how the cooperation strategy guides the UAVs very close to the target. When both UAVs are brought close to the target, the collision avoidance feature of the guidance strategy is also demonstrated. Note that the collision avoidance circle around each UAV is selected to have a radius of 0.25 km. Recall that the cost function in Eq. (8.1) has weighting coefficients to quantify the trade-off between threat exposure, Ktc and distance to the target, Kdc . The cooperation case is simulated with various values of the weighting coefficients and the corresponding trajectories on PTEM are shown in Fig. 8.7. Comparison of the subplots in Fig. 8.7 shows that as a smaller value of Kdc is selected, the UAVs are allowed to pursue the target from a farther distance. When Kdc = 1, the cooperation strategy reduces the radii of the proximity disks to their minimum value so that the UAVs fly on the trajectory of the target. Note that when the target passes through a restricted area, the cooperation strategy may compute points of rmin also within the restricted area. In such a case, the rule-based intelligent guidance algorithm does not follow the way points and instead

147

UAV trajectories for Ktc=0 and Kdc=1

UAV trajectories for Ktc=0.4 and Kdc=0.6

15

15

5

5 02

02 0.

0.

10

10 5

25

y(km)

−10

−10

−5 x(km)

5

0.02 5

25

0.025

0

0.02

UAV1 trajectory UAV2 trajectory 0.025 Target trajectory UAV1 minimums UAV2 minimums

0. 0 5

−15

−5

0.02

−15

10

−15

UAV trajectories for Ktc=0.6 and Kdc=0.4

−10

−5 x(km)

5

5

0.02

0.02

5 0.0 25

5 0.0 25

0.025

0.02

−15

5

5


−10

0.025

02

0.02

−5

0 0.

0.02

0.025

25 5

5 02 0.

0.0

0.02

0.025

5

0

0.025

5

0.0

0.02

y(km)

02 5

0.025

5

25

0.0

0.

0.

02 5

2 0.0

0. 5

0

0.025

5

15

5

5 02

02 0.

0.

10

10 5

02 5

25

y(km)

0

5

25

10

25

5 0.02


−10 0.025

0.0

25

5 0.02

0.0

−5

0.02 5

0.02

−15

−15

−10

−5 x(km)

0

5

5

0.02

−5 x(km)

0.025

0. 0 5

−10

5

5

0.02

0.02

−15

0.025

02


−10

0 0.

0.02

−5

25 5

5

0.025

0.0

0.02

0.025

0.02

5 02 0.

0.025

5

0.0

0

25

0.0

0.

0.

02 5

2 0.0

0.025

5 y(km)

10

UAV trajectories for Ktc=1 and Kdc=0

15

−15

02 5

0. 5

0.025

5

02 5

10

Figure 8.7. Points that minimize PTEM on proximity disk (rmin ) and the corresponding UAV trajectories in cooperative cases.

148 UAV trajectories for noncooperative centered case

UAV trajectories for noncooperative formation case

15

15

5

5 02

02 0.

0.

10

10 5

25

−5 x(km)

y(km)

25

0.025

5

10

5

25 0.0


−10

0

0.02

5

25 0.0

−5

0.02 5

0.02

−15

−15

−10

−5 x(km)

0

5

−10

5

0. 0 5

−15

0.025

0.02

0.02

0.02

−15

5

5


−10

0 0.

0.02

−5

0.025

02

0.02

0.025

25 5

5 02 0.

0.0

0.02

0.025

5

0

0.025

5

0.0

0.02

y(km)

02 5

0.025

5

25

0.0

0.

0.

02 5

2 0.0

0. 5

0.025

5

02 5

10

Figure 8.8. Points that minimize PTEM on proximity disk (rmin ) and the corresponding UAV trajectories in noncooperative cases. steers the UAVs around the restricted area. Note also that as Kdc is selected smaller than 0.5, way points of rmin generated by the cooperation algorithm do not change much. This is because the cooperation strategy does not look any further once a local minima of the cooperation cost function is reached. Fig. 8.9 shows how the radii of the proximity disks are adjusted by the cooperation strategy during the pursuit. As can be seen from the comparison of the subplots in this figure, the variation of the radii depends on the weighting coefficients selected. In the case where Ktc = 0 and Kdc = 1, the cooperation strategy lowers the radii to the minimum value in order to reduce the distance cost because the threat exposure has zero weighting in the cost function. On the other extreme, when Ktc = 1 and Kdc = 0, the proximity disks are expanded to their corresponding maximum size because distance is

149 Ktc=0 and Kdc=1

Ktc=0.4 and Kdc=0.6 1800 UAV1 UAV2

1000 800 600 400 200 0 0

200

400

600 t(sec)

800

1000

proximity circle radius (rp)(meter)


1200

1400 1200 1000 800 600 400 200 0 0

1200

UAV1 UAV2

1600

200

400

Ktc=0.6 and Kdc=0.4

1000

1200

1700 UAV1 UAV2

1600 1400 1200 1000 800 600 400 200 200

400

600 t(sec)

800

1000

1200



800

Ktc=1 and Kdc=0

1800

0 0

600 t(sec)

UAV1 UAV2

1600 1500 1400 1300 1200 1100 1000 0

200

400

600 t(sec)

800

1000

1200

Figure 8.9. Changes of the radii of the proximity disks by the cooperation strategy. not penalized in the cost function. However, note that the strategy reduces the radii rarely, when smaller proximity disks results in lower threat exposure. Figures 8.10 and 8.11 show the comparison of the commanded signals generated by the rule-based intelligent guidance strategy when the cooperation strategy has Kdc = 1 and Kdc = 0 for UAV-1 and UAV-2. Note that the commanded signals in the case with Kdc = 1 is more oscillatory as compared to the case with Kdc = 0. When Kdc = 1, the cooperation strategy minimizes the cost function by guiding the UAVs close to the target since threat minimization is not required with Ktc = 0. When the UAVs fly very close to each other, the collision avoidance routine is active and constantly maneuvers the UAVs to prevent collision. On the other hand, when Kdc = 0, the UAVs follow the target from a distance and collision is not an issue. Thus, the commanded signals are smoother.

150 UAV1 600 ψ(deg)

400

ψcomm for Kdc=1

200

ψcomm for Kdc=0

0 −200

V(km/h)

−400 0 180 160 140 120 100 80 60 0

200

400

600 t(sec)

800

1000

Vcomm for Kdc=1 Vcomm for Kdc=0 200

400

600 t(sec)

800

1000

Figure 8.10. UAV-1 commanded signals for Kdc = 1 and Kdc = 0 . UAV2 600 ψ(deg)

400

ψcomm for Kdc=1

200

ψcomm for Kdc=0

0 −200

V(km/h)

−400 0 180 160 140 120 100 80 60 0

200

400

600 t(sec)

800

1000

Vcomm for Kdc=1 Vcomm for Kdc=0 200

400

600 t(sec)

800

1000

Figure 8.11. UAV-2 commanded signals for Kdc = 1 and Kdc = 0. When the proximity disks are very large, the routine that improves the computational efficiency as explained in Section 8.1.3, may cause inaccuracy in the calculation of rmin . For example, note an isolated point of rmin at around (8,-6) within restricted region in the case with Ktc = 1 and Kdc = 0 in Fig. 8.7. This instantaneous inaccuracy is a side effect of the concept of “circle of effectiveness” introduced in Section 8.1.3. Recall

151 that a UAV considers, in its calculation of rmin , only those Gaussian pdf s that have the UAV position within their 3 σ radii. If a proximity disk is very large and the corresponding UAV is somewhere close to the boundary circle, then there might be Gaussian pdf s that have a significant contribution to PTEM on the disk but the UAV position is not within their circles of effectiveness. Instead of distance relative to the UAV position, the “effective” pdf s could have been determined by considering the intersection of the proximity disks and the circle of effectiveness. However, this would have added another level of computation complexity. Since the circles of effectiveness are much larger than even the maximum size proximity disks, this problem occurs instantaneously and very rarely. Thus, the benefit of the routine that improves the computational efficiency outweighs the effect of this drawback. Noncooperative case means that the radii of the proximity disks remain constant throughout the simulation. Still, rmin on each proximity disk of fixed radius is calculated as way points. These simulation results present two different noncooperative cases based on the formation of the proximity disks: (i) The proximity disks are centered at the target position, as done in the case of target pursuit by a single UAV in Chapters 6 and 7 (see the left part of Fig. 8.8), and (ii) One proximity disk is on each side of the target, as done in the cooperative cases (see the right part of Fig. 8.8). In noncooperative cases, the radii of both proximity disks remain at their initial value of 1.1 km. Larger proximity disks may potentially reduce TEL. However, simulation experiments have shown that with proximity disks with radius greater than 1.1 km, the UAVs, if not cooperating, are not able to follow the target. This is because the target goes beyond the sensor ranges of the UAVs for a long period of time and thus the UAVs lose the target permanently in early phases of the simulations. In the cooperative cases, on the other hand, radii of the proximity disks can become larger when so required by the cooperation strategy. This is because the cooperative strategy enlarges proximity disks only temporarily, during the

152 time when the level of threat exposure increases. Even in such a case, communication between the UAVs helps predict the target trajectory better. To quantify the trajectories shown in Figs. 8.7 and 8.8 and to comparatively analyze the cooperative and noncooperative target pursuit, two performance metrics are computed in each case simulated. The threat exposure level of each UAV is computed as formulated in Eq. (3.9) along the trajectory generated in the simulation. Another threat exposure level of the same UAV is computed along the target trajectory. The ratio of these two threat exposure levels is defined to be the Threat Exposure Level Ratio (TELR). Length Ratio (LR) is defined to be the ratio of a UAV trajectory length to the target trajectory length. UAV1

Threat exposure level ratio

0.35 Noncooperative centered Noncooperative formation Cooperative

0.3

0.25

0.2

0.15

0.1 0

0.2

0.4 0.6 Threat cost weight (Ktc)

0.8

1

Figure 8.12. Comparison of the threat exposure level of UAV-1.

Figures 8.12, 8.13 and 8.14 show the TELR of the team of two UAVs seperately and together in the cooperative case as Ktc is increased from 0 to 1 with 0.1 increments and in two noncooperative cases. First, note that in all cases, total TELR is less than 1, which means that the total threat exposure of the two UAVs along their trajectories is less than that of even a single UAV moving along the trajectory of the target. As can be seen in

153 UAV2


0.35

Noncooperative centered Noncooperative formation Cooperative

0.3

0.25

0.2

0

0.2

0.4 0.6 Threat cost weight(Ktc)

0.8

1

Figure 8.13. Comparison of the threat exposure level of UAV-2. UAV1 and UAV2

Total threat exposure level ratio

0.7 Noncooperative centered Noncooperative formation Cooperative

0.65 0.6 0.55 0.5 0.45 0.4 0.35 0

0.2

0.4 0.6 Threat cost weight (Ktc)

0.8

1

Figure 8.14. Comparison of the total threat exposure level of UAVs. Figs. 8.7 and 8.8, the target moves through three restricted areas, which have a very high level of threat exposure. On the other hand, the rule-based intelligent guidance algorithm, employed in both cooperative and noncooperative cases, always guides the UAVs to avoid any restricted area. Second, note that as Ktc increases, total TELR in the cooperative case consistently decreases. This is expected because Ktc is the weighting coefficient for the threat exposure component of the cooperation cost function in Eq. (8.1). With a larger Ktc , the cooperation strategy adjusts the radii of the proximity disks

154 as to reduce the threat exposure level of the team. Note also that, in the noncooperative cases, the total TELR, when the proximity disks are centered at the target location, is greater than that when the proximity disks are located on each side of the target. When the proximity disks of same size are centered, the UAVs follow the target closer, which, in turn, results in higher threat exposure. When Ktc in the cooperative case is greater than 0.3, TELR is lower than that in both cooperative cases. This clearly illustrates the benefit of cooperation based on the defined cost function between the UAVs when they pursue a common target. In addition to the benefit of lower threat exposure, the cooperation also reduces the risk of losing the target permanently in a pursuit. See in Fig. 8.7 that, in all cooperative cases, even with smaller Kdc (less emphasis on distance), both UAVs follow the target throughout the whole simulation. However, in both noncooperative cases, one of the UAVs loses the target permanently by the end of the simulation as seen in Fig. 8.8. There are two mechanisms that contribute to these results. In noncooperative cases, the radii of the proximity disks remain constant even when the cooperation strategy would make them smaller. Larger proximity disks may unnecessarily allow the UAVs to fly away from the target. When the cooperation reduces the size of a proximity disk, the corresponding UAV is forced to fly closer to the target. Even when one UAV is close to the target and has the target within the sensor range, through communication, the other UAV, while flying in a distance to the target to reduce the overall threat exposure, is provided with target measurements. Figures 8.15, 8.16 and 8.17 show the performance comparison using both TELR and LR. Note that both Figs. 8.14 and 8.17 have TELR in y-axis, but Fig. 8.17 has LR in its x-axis. TELR and LR in Figs. 8.14 and 8.17 are the sum of the corresponding quantities of the two UAVs in the team shown in Figs. 8.12, 8.13, 8.15 and 8.16. Note that, the benefit of cooperation is seen more clearly in Figs. 8.14 and 8.17, while some

155 UAV1


0.35

0.3


0.1 0.2 0

0.25

0.2

0.15

0.1 1.24

0.4

1.26

0.5

0.3

0.8 0.6 0.7 0.9 1 1.28

1.3 1.32 1.34 Trajectory length ratio

1.36

1.38

Figure 8.15. Comparison of the trajectory length and threat exposure level of UAV-1. UAV2


0.35

0.3

0.25

0

Noncooperative centered Noncooperative formation Cooperative 0.1

0.2

0.3 0.2 0.5 1.2

1.25

0.9 1 0.8

0.6

0.4

0.7

1.3 Trajectory length ratio

1.35

1.4

Figure 8.16. Comparison of the trajectory length and threat exposure level of UAV-2. exceptions can be seen, in Figs. 8.12, 8.13, 8.15 and 8.16, in the trend of decreasing threat exposure level and trajectory lengths by adjusting the weighting coefficients. Further, the UAVs invidually perform worse in some of the weighting coefficients compared to noncooperative formation case while they perform better as a team. This is due to the fact that the cooperation is quantified by a cost function based on the UAVs as a team but not individually. If both UAVs fly through the target trajectory, LR would be 2 as it is the sum of the trajectory lengths of the two UAVs divided by the trajectory length

156 UAV1 and UAV2

Total threat exposure level ratio

0.7 0.65

0

0.1


0.6 0.55

0.2

0.5 0.45 0.4 0.35

0.8 0.9

1

0.3 0.5 0.4 0.6 0.7

2.6

2.65 2.7 Total trajectory length ratio

2.75

Figure 8.17. Comparison of the total trajectory length and total threat exposure level of UAVs. of the target. Each marker on Figs. 8.15, 8.16 and 8.17 corresponds to a simulation case. Diamonds () refer to the cooperative cases and the number next to each diamond shows the value of the corresponding Ktc . Bullet (•) refers to the noncooperative case when the proximity disks are in formation and triangle (N) is the noncooperative case when the proximity disks are centered. Fig. 8.17 clearly demonstrate the fact that the cooperative cases outperform the noncooperative cases in terms of both total threat exposure and total trajectory length. When Ktc is greater than 0.3, the cooperation results in lower threat exposure as well as shorter trajectory length. It is interesting to see that the cooperation gives shorter trajectory length than that even when the proximity disks are centered at the target position (thus, the UAVs are supposed to follow the target closer) and this does not result in higher threat exposure. Given the fact that these two requirements in general conflict each other, it is very significant to demonstrate that the cooperation improves the performance in both directions. Figure 8.18 shows another benefit of cooperation by comparing the estimated signals in cooperative and noncooperative cases. The cooperative case shown is with

157

ψ(deg)

UAV2 250

ψactual

200

ψest noncooperative

150

ψest for Ktc=0.5

100 50 0

V(km/h)

200 150

200

400

600 t(sec)

800

1000

400

600 t(sec)

800

1000

Vactual Vest noncoperative Vest for Ktc=0.5

100 50 0

200

Figure 8.18. UAV-2 estimates for Ktc = 0 and noncooperative case. Ktc = 0.5 and the noncooperative case is the one with the proximity disks in formation. Estimation in the case of cooperation performs better as demonstrated in Chapter 5Section 5.4. This is because optimal data fusion is utilized when measurements from both UAVs are available. Further, when its own measurement is not available, a UAV uses the measurements from the other UAV when they are in communication.

CHAPTER 9 VIRTUAL TARGET TRACKING In this chapter, the target tracking algorithm developed in Chapter 7 is applied to the problems of trajectory planning and rendezvous through the introduction of the concept of virtual target. The target tracking algorithm presented in Chapter 7 is to guide a UAV to pursue a target in an adversarial environment. The trajectory planning algorithm developed in Chapter 4 is to determine a safe and feasible trajectory for a UAV to go to a target position in an adversarial environment. In this chapter, utilizing the concept of virtual target, the target tracking algorithm is used to function as a trajectory planning algorithm. However, note that in trajectory planning problem, there is no mobile target involved. Since the target tracking algorithm assumes the presence of a mobile target, the concept of virtual target tracking is introduced. Note also that the target tracking algorithm relies on an estimation algorithm to obtain the motion information of the mobile target. In the application to trajectory planning, there is no need for an estimation routine since the target tracking algorithm has direct access to the virtual target motion that is planned for the UAV to go to the assigned target location. Further, the concept of virtual target tracking is easily applied to rendezvous problem of multiple UAVs in an adversarial environment. In this case, the target tracking algorithm of each UAV involved pursues the virtual target assigned to the host UAV and the motion of the virtual targets are scheduled such that all the UAVs arrive at the target location simultaneously.

158

159 15

5

02

0.025

0. 10 25

0.

02

5

0.0

0.025 0.025

5

0.02

y(km)

5

0

5

02

0.02

5 0.0 25

0.

0.02

−5

5

0.02

5

0.025

−10 UAV trajectory 0.02 trajectory Virtual target 5

−5

0

0. 02

5

0.025

5

10

x(km)

Figure 9.1. UAV trajectory with rp = 5 m. 9.1

Trajectory planning via virtual target tracking As stated previously, the objective in trajectory planning is for the host UAV to

go the assigned target position. Thus, a virtual target trajectory has to be designed in order to use target tracking algorithm. Based on the initial position of the UAV and the assigned target position, first, the heading of the virtual target is calculated such that it moves on a non-maneuvering trajectory (constant heading and constant speed) towards the target position. The speed of the virtual target is set to the cruise speed of the UAV.

Namely, the virtual target, starting from the initial position of the UAV,

moves with constant speed on a straight line to the target position. Using the states of the virtual target, the target tracking algorithm guides the UAV towards the target position as if pursuing a target. Recall that rp defines how close the UAV should follow

160

V(km/h)

150 100 50

Vvirtual Vcomm

0 0

100

200

300

400 t(sec)

500

600

700

800

200

300

400 t(sec)

500

600

700

800

ψ(deg)

150 100 50 0

ψvirtual ψcomm 100

Figure 9.2. UAV and virtual target heading and speed with rp = 5 m. the target in the target tracking strategy. Thus, choosing a very small rp will generate the shortest trajectory to follow the virtual target while bigger rp will give a longer but safer trajectory. Since the target tracking algorithm follows the proximity disk instead of the actual target position, the radius of the proximity disk is reduced as the virtual target approaches to the final target position. Radius rp starts decreasing when the virtual target reaches the last 10% of its trajectory and keeps decreasing until it reaches the minimum value. This is intended to ensure that UAV goes to the final target position as the virtual target reaches the final position. To demonstrate the applicability of this approach, a simulation case is run where a trajectory is planned to go from initial position of (8.5,-13) km to a final position of (-4,14) km. The speed of the virtual target is selected to be 40 m/sec. The UAV that the trajectory is planned for has a maximum speed of 50 m/sec and a minimum speed of 50 m/sec as well as 10 deg/sec turning rate. The target tracking algorithm is run for several rp values. Additionally, the trajectory planning algorithm of Chapter 4 is run for the same start and end positions on the same PTEM with various values of fmax . Note

161 15

5

02

0.025

0. 10 25

0.

02

5

0.0

0.025 0.025

5

0.02

y(km)

5

0

5

02

0.02

5 0.0 25

0.

0.02

−5

5

0.02

5

0.025

−10 UAV trajectory Virtual target 0.02 trajectory 5 0.025 −5

0

5

0. 02

5

10

x(km)

Figure 9.3. UAV trajectory with rp = 500 m. that fmax in trajectory planning algorithm corresponds to rp in tracking algorithm and both can be used for specifying the trade-off level between shorter path length and lower threat exposure. As seen from Fig. 9.1, choosing rp = 5 m generates a straight trajectory for the UAV except avoiding the restricted regions. It can be seen from Fig. 9.2 that the heading of the UAV always converges to the constant heading of the virtual target while not avoiding the restricted regions. The speed of the UAV converges to the speed of the virtual target as well. Further, the speed of the UAV decreases to the minimum since the virtual target reaches to the final position earlier than the UAV and its speed becomes zero. The decrease in the speed of the UAV is due to the proportional speed control defined in Chapter 7, which is based on the position error and speed difference error.

162

V(km/h)

200 150 100

Vvirtual

50

Vcomm

0 0

100

200

300

400 t(sec)

500

600

700

800

600

700

800

200 ψ(deg)

150 100

ψvirtual

50 0 0

ψcomm 100

200

300

400 t(sec)

500

Figure 9.4. UAV and virtual target heading and speed with rp = 500 m. If rp is increased to 500 m, we see from Fig. 9.3 that the generated trajectory has small deviations from the virtual target trajectory. It can be seen from Fig. 9.4 that the commanded heading of the UAV is more oscillatory until it reaches the first restricted region around 200 seconds. This is due to the fact that a bigger proximity circle allows target tracking algorithm to minimize the threat exposure more. If the proximity disk radius is increased further, the deviations from the virtual target trajectory will increase and the commanded heading of the UAV will be more oscillatory as seen from Figs. 9.5 and 9.6. Note also that the speed of the UAV is higher than the virtual target speed since the UAV minimizes the threat exposure more and falls behind the virtual target (see Fig. 9.6). This increases the position error and contribute to the commanded speed.

An unnecessarily large proximity disk radius gives more flexibility to the target tracking algorithm to minimize the threat exposure. Nevertheless, this causes an unnecessarily long trajectory as seen from Fig. 9.7. Further, it causes UAV to loiter around a local minimum since the target tracking algorithm steers the UAV around a local

163 15

5

02

0.025

0. 10 25

0.

02

5

0.0

0.025

5

5

y(km)

0.02

0.025

0

5

02

0.02

5 0.0 25

0.

0.02

−5

5

0.02

5

0.025

−10 UAV trajectory Virtual target 0.02 trajectory 5 0.025 −5

0

5

0. 02

5

10

x(km)

Figure 9.5. UAV trajectory with rp = 2000 m.

V(km/h)

150 100

Vvirtual

50

Vcomm

0 0

100

200

300

400 t(sec)

500

600

700

800

500

600

700

800

ψ(deg)

200 100

ψvirtual ψcomm

0 0

100

200

300

400 t(sec)

Figure 9.6. UAV and virtual target heading and speed with rp = 2000 m.

164 15

5

02

0.025

0. 10 25

0.

02

5

0.0

0.025 0.025

5

0.02

y(km)

5

0

5

02

0.02

5 0.0 25

0.

0.02

−5

5

0.02

5

0.025

−10

UAV trajectory Virtual target trajectory 0.02 5 0.025 −5

0

5

0. 02

5

10

x(km)

Figure 9.7. UAV trajectory with rp = 5000 m. minimum as long as the local minimum is within the proximity disk. As the virtual target moves towards the final position, the proximity disk, after some time, leaves the local minimum out. When this happens, the target tracking algorithm commands higher speed, as seen in Fig. 9.8, for the UAV to move towards the middle of the proximity disk through the position error in the proportional speed control law. For comparison of the virtual target approach with the trajectory planning algorithm presented in Chapter 4, the same simulation case is run with the trajectory planning algorithm for several different fmax values. When fmax is selected to be equal to restricted region threshold fr , the shortest trajectory that avoids the restricted regions is generated (see Fig. 9.9). Greater fmax value generates safer trajectories by allowing larger heading changes and farther distances from the restricted regions.

Fig. 9.10 presents

165

V(km/h)

150 100 Vvirtual 50 0 0

Vcomm 100

200

300

400

500

600

700

800

900

1000

1100

t(sec)

ψvirtual

0

ψcomm

−200 −400 0

100

200

300

400

500

600

700

800

900

1000

t(sec)

Figure 9.8. UAV and virtual target heading and speed with rp = 5000 m.

15

5

02

0.025

0. 10 25

0.

02

5

0.0

0.025

5

0.025

0.02 5

02 0.

5

25

0

0.0

0.02

5

y(km)

ψ(deg)

200

5 0.02

−5

0.02

5

0.025

−10

UAV trajectory 0.02

5

−5

0

0.025

5

0. 02 5 10

x(km)

Figure 9.9. UAV trajectory with fmax = 0.025.

1100

166


0.5 0.05 0.25 0.45 0.005 0.5 0.025 1 0.4 0.015

5 3

1.5 2

0.35

rp fmax

0.01 0.3 0.005

0.25

0.001 0.0005

0.2 1

1.05

1.1

1.15 1.2 1.25 Trajectory length ratio

1.3

1.35

1.4

Figure 9.10. Comparison of the threat exposure level and trajectory length for rp and fmax . a comparison of the trajectories generated by the virtual target approach and the trajectory planning algorithm in terms of the trajectory Length Ratio (LR) and Threat Exposure Level Ratio (TELR).

Note that as fmax increases, the trajectory tracking

algorithm generates shorter paths with higher threat exposure. The trend in the virtual target approach is not that clear. For rp > 2, as rp increases, both trajectory length and threat exposure increase. This is because UAV spends more time loitering around the local minima and thus overall flight time increases, which leads to longer trajectory and higher threat exposure. When rp < 2, smaller rp yields shorter trajectory and higher threat exposure. In summary, the trajectory planning algorithm performs better. On the other hand, the virtual target tracking approach performs comparable to the trajectory planning algorithm even though it was not developed for trajectory planning. Further, with virtual target tracking approach, the time arrival of the final position can be controlled by adjusting the speed of the virtual target. Another advantage of the virtual

167 UAV1 trajectory UAV2 trajectory 0.025 UAV3 trajectory

25

0

0. 10

Virtual target 1 Virtual target 2 Virtual target 3

25

0.

02

5

0.0

0.025

5

0.0

0.025

5

0.02

y(km)

25

0

5

02

0.

0.02 5 0.0 25

25

0.0

0.02

0.025

5

5 0.02

−5

−10

0.02 5

−10

−5

0

0.025 5

0. 02 5 10

x(km)

Figure 9.11. Trajectories of the UAVs and the virtual targets. target approach is that the generated trajectories are safer as the commanded speed is adjusted when flying close to restricted areas.

9.2

Rendezvous via virtual target tracking In rendezvous problem, the objective of the UAVs, starting from different positions,

is to arrive at the same final target position simultaneously. For this purpose, a different virtual target trajectory has to be designed for each UAV. Based on the starting positions of the UAVs, each virtual target will have a different initial heading and speed on a straight line trajectory.

Since the UAVs operate in an adversarial environment, they

deviate from the virtual target trajectories to avoid restricted regions. This causes delays in their arrival time to the rendezvous point. To synchronize their arrival, the virtual

V(km/h)

168 180 160 140 120 100 80 60 0

Vvt1 VUAV1 100

200

300

400

500 t(sec)

600

700

800

900

1000

200

300

400

500 t(sec)

600

700

800

900

1000

600

700

800

900

1000

V(km/h)

160 140

Vvt2

120

V

UAV2

100 80 60 0

100

V(km/h)

160 140

Vvt3

120

VUAV3

100 80 60 0

100

200

300

400

500 t(sec)

Figure 9.12. Speed of the UAVs and the virtual targets. target trajectories are updated during the mission based on the current positions of the UAVs and the remaining time for the rendezvous. This results in adjustments of the heading and the speed of the virtual targets. In the simulation shown in Fig. 9.11, three UAVs starting from the positions (8.5,13) km, (-9,-5) km and (13,-5) km are guided to rendezvous at the final target position (-4,14) km at 1000 secs. Based on the starting positions of the UAVs, rendezvous time and position, the non-maneuvering target trajectories are generated. Note that, UAV-1 has the longest distance while UAV-2 has the shortest distance to the rendezvous position. Further, UAV-1 has to avoid more restricted regions than the other two UAVs. Thus, it follows the virtual target that has the fastest speed as seen from Fig. 9.12.

The UAV-2

maintains almost a constant minimum speed since it is closest to the target and does not avoid any restricted regions. The virtual target trajectories are updated in every 75

169 secs and thus new heading and speed of the virtual targets are calculated based on a straight line trajectory. As seen from Fig. 9.12, the virtual target 1 has the highest speed during the mission and the virtual target 3 increases its speed at the end of the mission. This is due to the fact that the UAV-1 flies the longest trajectory and UAV-3 avoids a restricted region just before it reaches to the rendezvous position. All three UAVs reach simultaneously at the proximity of the rendezvous position, which is defined to be 100 m in this simulation. The collision avoidance circle around each UAV has a radius of 10 m. Note that, if they are selected to be greater than the proximity of the rendezvous position, then the UAVs will never be able to get to the proximity of the rendezvous position, since they will repel each other as they get closer to the target position.

CHAPTER 10 CONCLUSIONS AND FUTURE WORK This chapter summarizes the research work, presents the conclusions and suggests directions for future research.

10.1

Summary and Conclusions First, the concept of PTEM (the Probabilistic Threat Exposure Map) is introduced

as a mathematical formulation of the area of operation in terms of threats, obstacles and restricted areas. PTEM defines various types of threats, obstacles and restricted areas in a single framework that quantifies the threat exposure level as a function of position. Probabilistic threat exposure map is composed of threat sources with multidimensional Gaussian or uniform probability density functions. Threat sources modelled by Gaussians are considered to be stationary or moving based on the time variance of mean and variance. PTEM is also used to determine the conditional probability of the UAV becoming disabled by a source of threat under the condition that it follows a certain trajectory at a certain time to go from one location to another. Based on PTEM, a new probabilistic approach is introduced for trajectory planning of UAVs. Utilizing the time-variant probabilistic threat exposure map, a trajectory planning strategy is introduced for a UAV flying in a dynamic environment with the objective of reaching multiple targets while avoiding restricted areas and minimizing the threat exposure level. The trajectory planning strategy, using the information about the time-variation of the map, generates the commanded heading and speed for the UAV to minimize its threat exposure level. Simulation results have shown the efficiency of the 170

171 strategy with the knowledge of time-variance of the map over two other strategies. One of them uses the current PTEM, but not the time-variation, and generates only commanded heading without commanding any change in the speed. The other one is provided with PTEM of the area only at the initial time and, based on that, generates the commanded heading. Strategies are applied to a case where one of the threats is changing its position and another one is changing its area of effectiveness. The commanded heading angle in all three strategies is generated within the turn rate constraints of the UAV. It is shown that if the time-variance of PTEM is known, the speed of the UAV should be adjusted as well as heading to benefit from the knowledge of time-variance. The strategy with the knowledge of time-variance is generating the trajectory with the least threat exposure level when compared to other two strategies. Moreover, it commands the UAV not only what path to follow but also how to adjust its speed while satisfying the given velocity and acceleration constraints since the map is time-variant. For the problem of pursuing a moving target in an adversarial environment modeled by PTEM, a rule-based intelligent guidance strategy is developed. This strategy is based on minimization of PTEM by a point search algorithm. The point search guidance algorithm evaluates a number of points equally distributed over an arc of circle bounded by the maximum right and left heading angles of the UAV. The strategy has three main objectives, which are, in the order of priority, (i) keep/take the UAV out of Ar , (ii) keep/put the UAV within the proximity circle and (iii) minimize the threat exposure level. These objectives are achieved by the strategy generating commanded heading and speed signals for the UAV. Furthermore, dynamic constraints are taken into consideration. During the pursuit, only the position of the target is measured with an unknown sensor noise if and when the target is within the sensor range. The speed and heading of the target are estimated satisfactorily based on the noisy measurement data by a second order linear least square estimator with a batch and sequential mode of processing. Several

172 cases were run in a Matlab based simulation environment to show the effectiveness of the strategy for almost all possible pursuit scenarios. A case is also presented to show the limitation of the estimation algorithm. When the target moves unpredictably while outside the sensor range, the UAV may lose the target permanently. However, as shown in some other cases, the incorporation of the estimation algorithm into the pursuit strategy certainly gives the UAV an opportunity to regain the target. This increases the success of the pursuit and survavibility by giving the UAV the ability of moving away from the target to avoid restricted areas and to minimize threat exposure level while following the target. The performance of the pursuit strategy can be adjusted via variation of the strategy parameters such as maximum allowable threat exposure level, specified proximity distance to the target and the allowable heading angle difference between the target and the UAV. The variation of the parameters can be utilized to quantify the trade–off between following the target closer and following the target with lower threat exposure level. Another rule-based intelligent guidance algorithm is developed to improve the computational efficiency without compromising the performance over the point search guidance algorithm. This new and significantly improved approach has resulted in a more systematic and analytical formulation of the guidance strategy. Specifically, the new approach uses gradient search in minimizing threat exposure and avoiding restricted areas, and systematically utilizes mathematical tools such as level curves, inertial and movingrotating frames and rotation matrices, vector representations of directions and geometric relations between cones, circles and straight lines. The approximation of the Million Instructions per Second (MIPS) of each algorithm and the computation time that each algorithm requires as observed in simulations have demonstrated that the new algorithm is about ten times better than the old algorithm in terms of computational efficiency. The simulations have further revealed that the new algorithm is at least as effective as

173 the old one in terms of the performance. In all simulation cases, guided by the new algorithm, the UAV safely avoided restricted-areas/obstacles while continuing the pursuit of the target. But, the old algorithm has caused the UAV to lose the target permanently, in two cases, even if the new improved estimation algorithm is used. The simulations have also shown that the new algorithm performs better in terms of minimizing the threat exposure while the old algorithm is slightly better in trajectory length and time to stay within the proximity circle when the target moves fast and accelerates. However, when the target moves slowly or stops, the new algorithm tends to perform better in all performance metrics. The computational improvement is due to the fact that the gradient search approach has eliminated the need to evaluate the PTEM at numerous search points. Further, utilization of the pursuit-guidance techniques based on Line of Sight (LOS) angle, proportional control for speed command and the weighted vectorial summation of minimizing direction and proximity-circle tangent has enabled the new algorithm to perform as well as the old algorithm. Since the dynamic constraints of the UAV are considered by strategy in decision making, the simulation results shown, as expected, that the heading and speed commands are always feasible and thus the firstorder response of the UAV follows the commanded signals very closely. The simulation experiments and the comparison of the approximations of MIPS that is required by the algorithm and that is available in the current real-time microprocessors have shown that the new algorithm is easily implementable for real-time execution. Pursuit of a mobile target in an adversarial environment by multiple UAVs is addressed next. A cost function is defined for placing multiple UAVs around a target in PTEM and a cooperation strategy based on the defined cost function for a team of UAVs to pursue a target in the adversarial environment is developed. The cooperation strategy is implemented along with the gradient search rule-based intelligent guidance algorithm in an integrated simulation environment with various modules such as target dynamics,

174 UAV dynamics, estimation, sensor and communication subsystems. To evaluate the performance of the cooperation strategy, a number of simulation cases are run where a target moves in an adversarial environment and is followed by two UAVs. First of all, simulations show that the rule-based intelligent guidance algorithm complements the cooperation strategy very well. The UAVs successfully follow the way points generated by the cooperation strategy while avoiding obstacles and restricted areas. Cooperative cases simulated are compared with noncooperative cases where two UAVs pursue the same target without any cooperation between them. In both cooperative and noncooperative cases, points that minimizes threat exposure on the proximity disks of the UAVs are computed as way points at every update instant. Noncooperative case means that the proximity disks are of fixed size throughout the pursuit. In cooperative cases, on the other hand, the radii of the proximity disks are adjusted, at every update instant, by the cooperation strategy to minimize the cost function. Simulations show that in noncooperative cases at least one of the two UAVs loses the target in early stages of the pursuit permanently while avoiding restricted areas if larger proximity disks are used. In relatively small size proximity disks are used, the UAVs, not cooperating, do not lose the target, not until late stages of the pursuit. In cooperative cases, on the other hand, the UAVs can follow the target even when the cooperation strategy increases the sizes of the proximity disks larger than those in the noncooperative case. Larger proximity disks indicates larger search area for minimizing threat exposure. This is, in fact, observed by the fact that the cooperation yields smaller threat exposure for the two UAV team. In the noncooperative case, larger fixed-size proximity disks may yield smaller threat exposure, if the target is not lost, at the expense of longer trajectory (i.e. UAVs follow the target from a farther distance in average). A very significant finding of the simulation results is that the cooperation yields shorter trajectory length as well as smaller threat exposure level. This is achieved by the cooperation strategy

175 that constantly adjusts the sizes of the proximity disks to minimize the cost function that consists of two terms, one for each requirement. This finding clearly demonstrates the benefit of cooperation among the UAVs when they follow a target in an adversarial environment. Another contribution to the success of the cooperation strategy is by the cooperative estimation scheme. By sharing measurements and fusing them optimally, the quality of estimation is improved, which, in turn, allows the cooperation strategy to significantly increase the sizes of the proximity disks when needed. Finally, a virtual target tracking approach is implemented for trajectory planning of the UAVs by using the gradient search algorithm developed for target tracking. Simulation results have shown that by adjusting the guidance strategy parameters such as proximity disk radius, shortest and the safest trajectories can be obtained. Further, it is shown that by adjusting the trajectories of several virtual targets the developed tracking algorithm (guidance strategy) is also applicable for other UAV applications such as rendezvous of UAVs.

10.2

Directions for Future Study Building on the conclusions above, suggestions for the future research directions

are outlined below. PTEMs used in this dissertation are generated manually and are not based on an adversarial environment in a real application. The generation of the PTEM should be automated to be able to used in real-time applications, which might require the online update of the PTEM. By automating the generation of PTEM, design specifications such as mean, variance and the number of Gaussians needed to model the environment should be calculated given the number of threats, obstacles and their dimensions and shapes. For example, in a real Homeland Security, Defense and Emergency Response (HSDER) application, a disaster environment such as an environment exposed to a nuclear detonation

176 can be represented by a PTEM. Particularly, the hot zones in the disaster environment can be modeled as restricted areas by a PTEM. Further, the PTEM can be updated if any other follow-up attacks occur or the new detection information is received during the mission. The performance of the guidance strategies developed in this dissertaion are evaluated through a simulation environment. It is known that simulations cannot always substitute for field tests but it is also realized that flight test operations of UAVs are challenging. Nevertheless, this work can be taken to practical implementation by using simple and small UAV platforms such as the ones developed at the University of Texas at Arlington-Autonomous Vehicles Lab (UTA-AVL). Since real ground target tracking applications require highly sophisticated sensor systems such as GMTI and SAR, the virtual target tracking approach can be used by these small UAVs in trajectory planning applications. Note that this approach dooes not require real target measurements and thus fairly simple to implement compared to real target tracking. Moreover, since virtual target tracking approach is developed based on the real target tracking guidance algorithm, the implementation of this approach by using simple UAV platforms will give an insight into how the target tracking guidance algorithm will perform in real applications assuming that the measurement and estimation processes give fairly accurate results. Further, the more realistic UAV models can be included to increase the fidelity of the simulations and to have a better insight on how the algorithms would work in real UAV platforms. In the current simulations, closed-loop dynamics of each vehicle is modeled by first-order transfer functions. However, these models can easily be changed by high fidelity dynamics models and controllers. For example, the dynamics of the UAVs operated by UTA-AVL can be identified by using real flight data and can be implemented in the simulations. This will replace the first-order models by the closed loop models with PID controllers.

177 The simulations in this dissertation have shown the benefit of integrating the estimation algorithm with the guidance algorithm. However, simulation results also demonstrate the limitation of the estimation algorithm when the time interval without any measurement is very long, especially when the target transition between different modes of motion such as when it decelerates and then stops. This is one of the areas that is the subject of future research. Another future area of research might be the improvement of the guidance strategies. These improvements might include the consideration of non-convex restricted regions, elevation of the area of operation and intelligent targets. Note that, in the current simulations all the restricted regions were selected to be convex. In the case of nonconvex restricted areas, the guidance strategies should be improved to ensure that UAVs are not trapped in a non-convex regions. Further, the guidance strategies can be improved to handle 3-D regions such as by changing proximity disk to a proximity sphere. Another improvement on the guidance algorithms should be the consideration of the evasive maneuvers of a target. Note that, in the current simulations, target does not pick up on follower cues and does not re-plan its track. In fact, the guidance algorithm developed for the UAV to track the target can be modified and used to guide the target to avoid detection. Another future work on the guidance algorithms should be the modification of the guidance algorithms to provide the capability of pursuing multiple targets. Finally, the global information of the area of operation can be utilized to generate the globally optimal trajectory and can be compared to the trajectories generated by the guidance algorithms developed in this dissertation.

APPENDIX A TRANSFORMATION BETWEEN INERTIAL AND LOCAL FRAMES

178

179 Transformation between the inertial reference frame and the local frame is utilized to facilitate the computation of yL1 and yL2 . Note from Fig. 7.5 that the rotation matrix from the inertial frame to the local frame is    sin ψtemp (k) − cos ψtemp (k)  RLI (k) =   cos ψtemp (k) sin ψtemp (k)

(A.1)

The predicted target position, then, can be written in the local frame by employing the rotation matrix as b rt,L (k + 1) = RLI (k) [b rt (k + 1) − ru (k)]

(A.2)

where b rt is the representation of b rt in the inertial frame and b rt,L is the representation in the local frame. Equation (A.2) gives the components of b rt in the local frame as x bt,L (k + 1) = [b xt (k + 1) − xu (k)] sin ψtemp (k) −[b yt (k + 1) − yu (k)] cos ψtemp (k)

ybt,L (k + 1) = [b xt (k + 1) − xu (k)] cos ψtemp (k) +[b yt (k + 1) − yu (k)] sin ψtemp (k)

(A.3)

(A.4)

Now, Cp can be easily formulated in (xL , yL ) local frame as [xL − x bt,L (k + 1)]2 + [yL − ybt,L (k + 1)]2 = rp2

(A.5)

As Fig. 7.5 implies, yL1 (k) and yL2 (k) are the solution to Eq. (A.5) when xL = 0 since yL1 and yL2 are defined to lie on yL -axis. yL1,2 (k) = ybt,L (k + 1) ∓

q

rp2 − x bt,L (k + 1).

(A.6)

REFERENCES [1] S. Butenko, R. Murphey, and P. M. Pardalos, Cooperative Control: Models, Applications and Algorithms. Orlando, FL: Kluwer Academic Publishers, April 2003. [2] J. Finke, K. M. Passino, and A. Sparks, “Cooperative control via task load balancing for networked uninhabited autonomous vehicles,” in Proceedings of the 42nd IEEE Conference on Decision and Control, vol. 1, Maui, Hawaii, Dec. 2003, pp. 31–36. [3] A. Sinha, T. Kirubarajan, and Y. Bar-Shalom, “Optimal cooperative placement of gmti UAVs for ground target tracking,” in Proceedings of IEEE Aerospace Conference, 2004, pp. 1859–1868. [4] Y. Bar-shalom, X. Li, and T. Kirubarajan, Estimation with Applications to Tracking and Navigation. John Wiley & Sons, Inc., 2001. [5] M. E. Polites and et al, “Optimal state estimation with failed sensor discrimination and identification,” Journal of Guidance, Control and Dynamics, vol. 27:3, MayJune 2004. [6] Y. Bar-shalom and X. Li, Estimation and Tracking: Principles, Techniques and Software. Artech House, Inc., 1993. [7] U. Zengin and A. Dogan, “Real-time target tracking for autonomous UAVs in adversarial environments: A gradient search algorithm,” IEEE Transactions on Robotics, vol. 23, no. 2, April 2007. [8] A. Dogan and U. Zengin, “Unmanned aerial vehicle dynamic-target pursuit by using a probabilistic threat exposure map,” AIAA Journal of Guidance, Dynamics and Control, vol. 29, no. 4, pp. 944–954, 2006. 180

181 [9] U. Zengin and A. Dogan, “Real-time target tracking for autonomous UAVs in adversarial environments: A gradient search algorithm,” Proceedings of the 45th IEEE Conference on Decision and Control, pp. 697–702, Dec. 2006. [10] ——, “Cooperative target tracking for autonomous UAVs in an adversarial environment,” in Proceedings of the AIAA Guidance, Navigation and Control Conference and Exhibit, Keystone, CO, Aug. 2006, pp. 3315–3329. [11] ——, “Target tracking by UAVs under communication constraints in an adversarial environment,” in Proceedings of the AIAA Guidance, Navigation and Control Conference and Exhibit, San Francisco, CA, Aug. 2005. [12] ——, “Multi-UAV rendezvous in hostile environment,” in Proceedings of Ankara International Aerospace Conference, Ankara, Turkey, Aug. 2005, pp. AIAC–2005– 043. [13] ——, “Probabilistic trajectory planning for UAVs in dynamic environments,” in Proceedings of AIAA 3rd ”Unmanned Unlimited” Technical Conference, Workshop and Exhibit, Chicago, IL, Sep. 2004, pp. 605–616. [14] ——, “Dynamic target pursuit by UAVs in probabilistic threat exposure map,” in Proceedings of AIAA 3rd ”Unmanned Unlimited” Technical Conference, Workshop and Exhibit, Chicago, IL, Sep. 2004, pp. 954–969. [15] T. Schouwenaars, “Safe trajectory planning of autonomous vehicles,” Ph.D. dissertation, MIT, 2006. [16] D. Rathbun, S. Kragelund, A. Pongpunwattana, and B. Capozzi, “An evolution based path planning algorithm for autonomous motion of a UAV through uncertain environment,” in Proceedings of the 21st Digital Avionics Systems Conference, vol. 2, Indianapolis, Indiana, Oct. 2002, pp. 8D2–1–8D2–12.

182 [17] M. Flint, M. Polycarpou, and E. F. Gaucherand, “Cooperative control for multiple autonomous UAV’s searching for targets,” in Proceedings of the 41st IEEE Conference on Decision and Control, Las Vegas, Nevada, Dec. 2002, pp. 2823–2828. [18] M. Jun, A. I. Chaudry, and R. D’Andrea, “The navigation of autonomous vehicles in uncertain dynamic environments: A case study,” in Proceedings of the 41st IEEE Conference on Decision and Control, Las Vegas, Nevada, Dec. 2002, pp. 3770–3775. [19] A. Pongpunwattana and R. Rysdyk, “Real-time planning for multiple autonomous vehicles in dynamic uncertain environments,” Journal of Aerospace Computing, Information, and Communication, vol. 1, pp. 580–604, Dec 2004. [20] I. K. Nikolas, K. P. Valavanis, N. C. Tsourveloudis, and A. N. Kostaras, “Evolutionary algorithm based offline/online path planner for UAV navigation,” IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, vol. 33, no. 6, pp. 898–912, 2003. [21] R. Zhu, D. Sun, and Z. Zhou, “Cooperation strategy of unmanned air vehicles for multitarget interception,” Journal of Guidance, Control and Dynamics, vol. 28, no. 5, pp. 1068–1072, 2005. [22] J. Thomas, A. Blair, and N. Barnes, “Towards an efficient optimal trajectory planner for multiple robots,” in Proceedings of the 2003 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, Las Vegas, Nevada, Oct. 2003, pp. 2291–2296. [23] S. Waydo and R. M. Murray, “Vehicle motion planning using stream functions,” in Proceedings of the 2003 IEEE International Conference on Robotics&Automation, Taipei, Taiwan, Sep. 2003, pp. 2484–2491. [24] I. Kaminer, A. Pascoal, E. Hallberg, and C. Silvestre, “Trajectory tracking for autonomous vehicles: An integrated approach to guidance and control,” Journal of Guidance, Control and Dynamics, vol. 1, no. 21, pp. 29–38, 1998.

183 [25] D. N. Borys and R. Colgren, “Advances in intelligent autopilot systems for unmanned aerial vehicles,” in Proceedings of the AIAA Guidance, Navigation and Control Conference and Exhibit, San Francisco, CA, Aug. 2005. [26] J. Sumita, “An application of fuzzy expert concept for unmanned air vehicles,” in Proceedings of the 2nd AIAA ”Unmanned Unlimited” Systems, Technologies, and Operations Aerospace, Land, and Sea Conference and Workshop Exhibit, San Diego, CA, Sep. 2003. [27] ——, “Autonomous nas flight control for UAV by fuzzy concept,” in Proceedings of AIAA Guidance, Navigation and Control Conference and Exhibit, Rhode Island, NY, Aug. 2004. [28] M. A. Kovacina, D. Palmer, G. Yang, and R. Vaidyanathan, “Multi-agent control algorithms for chemical cloud detection and mapping using unmanned air vehicles,” in Proceedings of the 2002 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, Lausanne,Switzerland, Oct. 2002, pp. 2782–2788. [29] J. Efstathiou, “Expert systems, fuzzy logic and rule-based control explained at last,” Transactions of the Institute of Measurement and Control, vol. 10, no. 4, pp. 198–206, 1988. [30] M. Negnevitsky, Artificial Intelligence: A Guide to Intelligent Systems. England: Addison-Wesley, 2002. [31] A. Niehaus and R. F. Stengel, “An expert system system for automated highway driving,” IEEE Control Systems Magazine, vol. 11, no. 3, pp. 53–61, 1991. [32] R. F. Stengel, “Toward intelligent flight control,” IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, vol. 23, no. 6, pp. 1699–1717, 1993. [33] J. P. Wangermann and R. F. Stengel, “Principled negotiation between intelligent agents: A model for air traffic management,” Artificial Intelligence in Engineering, vol. 12, no. 3, pp. 177–187, 1998.

184 [34] Y. Jin, A. A. Minai, and M. M. Polycarpou, “Cooperative real-time search and task allocation in UAV teams,” in Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, Hawaii, Dec. 2003, pp. 3585–3590. [35] Y. Yang, A. A. Minai, and M. M. Polycarpou, “Decentralized cooperative search by networked UAVs in an uncertain environment,” in Proceedings of the American Control Conference, Boston, MA, July 2004, pp. 5558–5563. [36] A. Mahajan, J. Ko, and R. Sengupta, “Distributed probabilistic map service,” in Proceedings of the 41st IEEE Conference on Decision and Control, Las Vegas, Nevada, Dec. 2002, pp. 130–135. [37] M. Tomono, “Planning a path for finding targets under spatial uncertainties using a weighted voronoi graph and visibility measure,” in Proceedings of the 2003 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, Las Vegas, Nevada, Oct. 2003, pp. 124–129. [38] L. F. Bertuccelli and J. P. How, “Search for dynamic targets with uncertain probability maps,” in Proceedings of the American Control Conference, Minneapolis,MN, June 2006, pp. 737–742. [39] ——, “Robust UAV search for environments with imprecise probability maps,” in Proceedings of the 42nd IEEE Conference on Decision and Control, Seville, Spain, Dec. 2005, pp. 5680–5685. [40] Y. Y. M. M. Polycarpou and K. Passino, “A cooperative search framework for distributed agents,” in Proceedings of IEEE International Symposium on Intelligent Control, Mexico City, Mexico, Sep. 2001. [41] M. Jun and R. D’Andrea, “Probability map building of uncertain dynamic environments with indistinguishable obstacles,” in Proceedings of the American Control Conference, Denver, CO, June 2003, pp. 3417–3421.

185 [42] J. P. Hespanha, H. Kizilocak, and Y. S. Ateskan, “Probabilistic map building for aircraft-tracking radars,” in Proceedings of the American Control Conference, vol. 6, Arlington, VA, Jun. 2001, pp. 4381–4386. [43] Y. D. Kwon and J. S. Lee, “A stochastic map building method for mobile robot using 2-d laser range finder,” Journal of Autonomous Robots, vol. 7, pp. 187–200, 1999. [44] A. Dogan, “Probabilistic path planning for UAVs,” in Proceedings of the 2nd AIAA ”Unmanned Unlimited” Systems, Technologies, and Operations Aerospace, Land, and Sea Conference and Workshop Exhibit, San Diego, CA, Sep. 2003. [45] ——, “Probabilistic approach in path planning for UAVs,” in Proceedings of the IEEE International Symposium on Intelligent Control, Houston, TX, Oct. 2003, pp. 608–613. [46] S. Kanchanavally, R. Ordonez, and J. Layne, “Mobile target tracking by networked uninhabited autonomous vehicles via hospitability maps,” in Proceedings of the American Control Conference, Boston, MA, July 2004, pp. 5570–5575. [47] A. S. Miralles and M. A. S. Bobi, “Global path planning in gaussian probabilistic maps,” Journal of Intelligent and Robotic Systems, Jan. 2004. [48] T. McLain, P. Chandler, S. Rasmussen, and M. Pachter, “Cooperative control of UAV rendezvous,” in Proceedings of the American Control Conference, Arlington, VA, June 2001, pp. 2309–2314. [49] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Expert Systems Design and Development. New York: McGraw-Hill, 2001,2nd Ed. [50] R. Beard, T. McLain, and M. Goodrich, “Coordinated target assignment and intercept for unmanned air vehicles,” in Proceedings of the 2002 IEEE International Conference on Robotics Automation, Washington, DC, May 2002, pp. 2581–2586.

186 [51] S. Li, J. Boskovic, and et al, “Autonomous hierachical control of multiple unmanned combat air vehicles (ucavs),” in Proceedings of the American Control Conference, Anchorage, AK, May 2002, pp. 274–79. [52] S. Bortoff, “Path planning for UAV,” in Proceedings of the American Control Conference, Chicago, IL, June 2000, pp. 364–68. [53] O. Khatib, “Real-time obstacle avoidance for manipulators and mobile robots,” The International Journal of Robotics Research, vol. 5, no. 1, pp. 90–99, 1986. [54] Y. K. Hwang and N. Ahuja, “A potential field approach to path planning,” IEEE Transactions on Robotics and Automation, vol. 8, no. 1, pp. 23–32, Feb. 1992. [55] P. Iniguez and J. Rosell, “Probabilistic harmonic-function-based method for robot motion planning,” in Proceedings of the 2003 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, Las Vegas, Nevada, Oct. 2003, pp. 382–387. [56] T. McLain and R. Beard, “Trajectory planning for coordinated rendezvous of unmanned air vehicles,” in Proceedings of the AIAA Guidance, Navigation and Control Conference and Exhibit, Denver, CO, Aug. 2000. [57] P. R. Chandler, M. Pachter, and S. Rasmussen, “UAV cooperative control,” in Proceedings of the American Control Conference, Arlington, VA, June 2001, pp. 50–55. [58] S. Twigg, A. Calise, and E. Johnson, “On-line trajectory optimization including moving threats and targets,” in Proceedings of AIAA Guidance, Navigation and Control Conference and Exhibit, Rhode Island, NY, Aug. 2004. [59] O. Takahashi and R. J. Schilling, “Motion planning in a plane using generalized voronoi diagrams,” IEEE Transactions on Robotics and Automation, vol. 5, no. 2, pp. 143–150, Apr. 1989.

187 [60] D. Gu, W. Kamal, and I. Postlethwaite, “A UAV waypoint generator,” in Proceedings of the AIAA 1st Intelligent Systems Technical Conference, Chicago,IL, Sep. 2004. [61] E. Masehian and M. R. Amin-Naseri, “A voronoi diagram-visibility graph-potencial field compound algorithm for robot path planning,” Journal of Robotic Systems, vol. 21, no. 6, pp. 275–300, June 2004. [62] J. Cortes, S. Martinez, and F. Bullo, “Robust rendezvous for mobile autonomous agents via proximity graphs in arbitrary dimensions,” IEEE Transactions on Automatic Control, vol. 51, no. 8, pp. 1289–1298, 2006. [63] J.-C. Latombe, Robot Motion Planning. London: Kluwer Academic, 1991. [64] D. Turra, L. Pollini, and M. Innocenti, “Real-time unmanned vehicles task allocation with moving targets,” in Proceedings of AIAA Guidance, Navigation and Control Conference and Exhibit, Rhode Island, NY, Aug. 2004. [65] Y. Saab and M. VanPutte, “Shortest path planning on topographical maps,” IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 29, no. 1, pp. 139–150, 1999. [66] D. Eladio and M. Luis, “Probability of success and uncertainty analysis in path planning,” in Proceedings of the 2003 IEEE International Conference on Robotics&Automation, Taipei, Taiwan, Sep. 2003, pp. 4203–4208. [67] H. I. Yang and Y. J. Zhao, “Trajectory planning for autonomous aerospace vehicles amid known obstacles and conflicts,” Journal of Guidance, Control and Dynamics, vol. 6, no. 27, pp. 997–1008, 2004. [68] L. E. Kavraki, M. N. Kolountzakis, and J. Latombe, “Analysis of probabilistic roadmaps for path planning,” IEEE Transactions on Robotics and Automation, vol. 14, no. 1, pp. 166–171, Feb. 1998.

188 [69] A. M. Ladd and L. E. Kavraki, “Measure theoretic analysis of probabilistic path planning,” IEEE Transactions on Robotics and Automation, vol. 20, no. 2, pp. 229–242, Apr. 2004. [70] L. E. Kavraki, P. Svestka, J.-C. Latombe, and M. H. Overmars, “Probabilistic roadmaps for path planning in high-dimensional configuration spaces,” IEEE Transactions on Robotics and Automation, vol. 12, no. 4, pp. 566–580, Apr. 2004. [71] N. M. Amato, O. B. Bayazit, L. K. Dale, C. Jones, and D. Vallejo, “Choosing good distance metrics and local planners for probabilistic roadmap methods,” IEEE Transactions on Robotics and Automation, vol. 16, no. 4, pp. 442–447, Aug. 2000. [72] J.-C. Latombe, “Motion planning: A journey of robots, molecules, digital actors, and other artifacts,” International Journal of Robotics Research, vol. 18, no. 11, pp. 1119–1128, Nov. 1999. [73] P. Isto, “Constructing probailistic roadmaps with powerful local planning and path optimization,” in Proceedings of the 2002 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, Lausanne,Switzerland, Oct. 2002, pp. 2323–2328. [74] Z. Lee and X. Chen, “Path planning approach based on probabilistic roadmap for sensor based car-like robot in unknown environments,” in Proceedings IEEE International Conference on Systems, Man and Cybernetics, The Hague, Netherlands, Oct. 2004, pp. 2907–2912. [75] C. Lanzoni, A. sanchez, and R. Zapata, “Sensor-based motion planning for carlike mobile robots in unknown environments,” in Proceedings of the 2003 IEEE International Conference on Robotics Automation, Taipei, Taiwan, Sep. 2003, pp. 4258–4263. [76] F. Lingelbach, “Path planning using probabilistic cell decomposition,” in Proceedings of the 2002 IEEE International Conference on Robotics Automation, New Orleans, LA, Apr. 2004, pp. 467–472.

189 [77] E. Horiuchi and K. Tani, “A probabilistic approach to path planning with object boundary uncertainties,” in Proceedings of Fifth International Conference-’91 ICAR, Pisa, Italy, 1991, pp. 1698–1701. [78] S. Jun and G. Shin, “A probabilistic approach to collision-free robot path planning,” in Proceedings of the 1988 IEEE International Conference on Robotics Automation, Philadelphia, PA, 1988, pp. 220–225. [79] H. Hu and M. Brady, “Dynamic global path planning with uncertainity for mobile robots in manufacturing,” IEEE Transactions on Robotics and Automation, vol. 13, no. 5, pp. 760–767, Oct. 1997. [80] R. Sharma, “Locally efficient path planning in an uncertain, dynamic environment using a probabilistic model,” IEEE Transactions on Robotics and Automation, vol. 8, no. 1, pp. 105–110, Feb. 1992. [81] C. Hocaoglu and A. C. Sanderson, “Planning multiple paths with evolutionary speciation,” IEEE Transactions on Evolutionary Computation, vol. 5, no. 3, pp. 169–191, June 2001. [82] D. Jia and J. Vagners, “Parallel evolutionary algorithms for UAV path planning,” in Proceedings of AIAA 1st Intelligent Systems Technical Conference, Chicago, Illinois, Sep. 2004. [83] B. J. Capozzi and J. Vagners, “Navigating annoying environments through evolution,” in Proceedings of the 41st IEEE Conference on Decision and Control, Orlando, Florida, Dec. 2001, pp. 646–651. [84] R. Vaidyanathan, C. Hocaoglu, T. S. .Prince, and R. D. Quinn, “Evolutionary path planning for autonomous air vehicles using multiresolution path representation,” in Proceedings of the 2003 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, Maui, Hawaii, Oct. 2001, pp. 69–76.

190 [85] A. Alvarez, A. Caiti, and R. Onken, “Evolutionary path planning for autonomous underwater vehicles in a variable ocean,” IEEE Journal of Oceanic Engineering, vol. 29, no. 2, pp. 418–429, April 2004. [86] L. Xu and U. Ozguner, “Battle management for unmanned aerial vehicles,” in Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, Hawaii, Dec. 2003. [87] F. Lian and R. Murray, “Real-time trajectory generation for the cooperative path planning of multi-vehicle systems,” in Proceedings of the 41st IEEE Conference on Decision and Control, Las Vegas, Nevada, Dec. 2002, pp. 2823–2828. [88] J. S. Bellingham, M. Tillerson, M. Alighanbari, and J. P. How, “Cooperative path planning for multiple UAVs in dynamic and uncertain environments,” in Proceedings of the 41st IEEE Conference on Decision and Control, Las Vegas, Nevada, Dec. 2002, pp. 2816–2822. [89] M. B. Dias and A. Stentz, “A comparative study between centralized, marketbased, and behavioral multirobot coordination approaches,” in Proceedings of the International Conference on Intelligent Robots and Systems, Las Vegas, Nevada, Oct. 2003. [90] T. Schouwenaars, J. How, and E. Feron, “Decentralized cooperative trajectory planning of multiple aircraft with hard safety guarantees,” in Proceedings of AIAA Guidance, Navigation and Control Conference and Exhibit, Rhode Island, NY, Aug. 2004. [91] R. Beard and T. McLain, “Cooperative path planning for timing-critical missions,” in Proceedings of the American Control Conference, Denver, Colorado, June 2003, pp. 296–301.

191 [92] T. McLain, P. Chandler, and M. Pachter, “A decomposition strategy for optimal coordination of unmanned air vehicles,” in Proceedings of the American Control Conference, Chicago, IL, June 2000, pp. 369–373. [93] E. W.Frew and D. A. Lawrance, “Cooperative stand-off tracking of moving targets by a team of autonomous aircraft,” in Proceedings of the AIAA Guidance, Navigation and Control Conference and Exhibit, San Francisco, CA, Aug. 2005. [94] R. Sengupta, J. K. Hedrick, and et al, “Strategies of path planning for a UAV to track a ground vehicle,” in Proceedings of the Second Annual Symposium on Autonomous Intelligent Networks and Systems, Menlo Park, CA, 2003. [95] S. C. Spry, A. R. Girard, and J. K. Hedrick, “Convoy protection using multiple unmanned aerial vehicles:organization and coordination,” in Proceedings of American Control Conference, Portland, OR, June. 2005, pp. 3524–3529. [96] J. S. Jang and C. J. Tomlin, “Control strategies in multi-player pursuit and evasion game,” in Proceedings of the AIAA Guidance, Navigation and Control Conference and Exhibit, San Francisco, CA, Aug. 2005, pp. 3702–3711. [97] C. Schumacher, “Ground moving target engagement by cooperative UAVs,” in Proceedings of American Control Conference, Portland, OR, June. 2005, pp. 4502– 4505. [98] P. J. Shea and et al, “Precision tracking of ground targets,” in Proceedings of IEEE Aerospace Conference, vol. 3, 2000, pp. 473–482. [99] S. Spry, A. Vaughn, and X. Xiao, “A vehicle following methodoly for UAV formations,” in Proceedings of the Fourth International Conference on Cooperative Control and Optimization, Boston, MA, 2003, pp. 244–251. [100] E. Frew, S. Spry, and T. McGee, “Flight demonstrations of self directed collaborative navigation of small unmanned aircraft,” in Proceedings of AIAA 3rd ”Un-

192 manned Unlimited” Technical Conference, Workshop and Exhibit, Chicago, IL, Sep. 2004. [101] R. Isaacs, Differential Games. New York: Wiley, 1965. [102] T. D. Parsons, Pursuit-evasion in a graph, In Y. Alani and D.R. Lick, editors, Theory and Application of Graphs,. Springer-Verlag, 1976. [103] I. Suzuki and M. Yamashita, “Searching for a mobile intruder in a polygonal region,” SIAM Journal on Computing, vol. 21, no. 5, pp. 863–888, 1992. [104] A. Antoniades, H. J. Kim, and S. Sastry, “Pursuit-evasion strategies for teams of multiple agents with incomplete information,” in Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, Hawaii, Dec. 2003, pp. 756–61. [105] R. Vidal, O. Shakernia, H. J. Kim, D. H. Shim, and S. Sastry, “Probabilistic pursuit-evasion games: Theory, implementation, and experimental evaluation,” IEEE Transactions on Robotics and Automation, vol. 18, no. 5, pp. 662–69, Oct. 2002. [106] J. P. Hespanha, H. J. Kim, and S. Sastry, “Multiple-agent probabilistic pursuitevasion games.” Phoenix,AR: Proceedings of the 38th IEEE Conference on Decision and Control, Dec. 1999, pp. 2272–2277. [107] J. P. Hespanha, M. Prandini, and S. Sastry, “Probabilistic pursuit-evasion games:a one-step nash approach,” in Proceedings of the 39th IEEE Conference on Decision and Control, Sydney,Australia, Dec. 2000, pp. 2272–2277. [108] C. Chong, D. Garren, and T. P. Grayson, “Ground target tracking - a historical perspective,” in Proceedings of IEEE Aerospace Conference, 2000. [109] J. N. Entzminger, C. A. Fowler, and W. J. Kenneally, “Jointstars and gmti: Past, present and future,” IEEE Transactions on Aerospace and Electronic Systems, vol. 35, no. 2, pp. 748–761, April. 1999.

193 [110] J. K. Jao, “Theory of synthetic aperture radar imaging of a moving target,” IEEE Transactions on Geoscience and Remote Sensing, vol. 39, no. 9, pp. 1984–1992, Sep. 2001. [111] N. Pulsone, C. Rader, and B. Mathew, “Improving ground moving target indication performance.” [112] D. J. V. Pierce and J. J. SantaPietro, “Visualization of ground target designation from an unmanned aerial vehicle,” in Proceedings of SPIE,The International Society for Optical Engineering, vol. 3395, 1998, pp. 76–86. [113] L. Lin, Y. Bar-Shalom, and T. Kirubarajan, “New assignment-based data association for tracking move-stop-move targets,” IEEE Transactions on Aerospace and Electronic Systems, vol. 40, no. 2, pp. 714–725, Apr. 2004. [114] C. Ke, J. G. Herrero, and J. Llinas, “Comparative analysis of alternative ground target tracking techniques,” in Proceedings of the Third International Conference on Information Fusion, vol. 2, July 2000, pp. WEB5/3 – WEB510. [115] X. R. Li and V. P. Jilkov, “Survey of maneuvering target tracking. part i: Dynamic models,” IEEE Transactions on Aerospace and Electronic Systems, vol. 39, no. 4, pp. 1333–1364, Oct. 2003. [116] ——, “Survey of maneuvering target tracking. part ii: Ballistic target models,” in Proceedings of SPIE,The International Society for Optical Engineering, vol. 4473, Aug. 2001, pp. 559–581. [117] ——, “Survey of maneuvering target tracking. part iii: Measurement models,” in Proceedings of SPIE, The International Society for Optical Engineering, vol. 4473, Aug. 2001, pp. 423–446. [118] ——, “Survey of maneuvering target tracking. part iv: Decision-based methods,” in Proceedings of SPIE, The International Society for Optical Engineering, vol. 4728, April 2002, pp. 511–534.

194 [119] ——, “Survey of maneuvering target tracking. part v: Multiple-model methods,” in Proceedings of SPIE, Conference on Signal and Data Processing of Small Targets, vol. 5204, San Diego, CA, Aug. 2003. [120] M. Lodaya and R. Bottone, “Least squares approach to asynchronous data fusion,” in Proceedings of SPIE, The International Society for Optical Engineering, vol. 4048, 2000, pp. 333–344. [121] Y. Zhou, “A kalman filter based registration approach for asynchronous sensors in multiple sensor fusion applications,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Quebec,Canada, May 2004, pp. 293–296. [122] Y. Bar-Shalom and X. Li, Multitarget-Multisensor Tracking: Principles and Techniques. CT: YBS Publishing, 1995. [123] P. J. Lanzkron and Y. Bar-Shalom, “A two-step-method for out-of-sequence measurements,” in Proceedings of IEEE Aerospace Conference, 2004, pp. 2036–2041. [124] M. Mallick, K. Zhang, and X. R. Li, “Comparative analysis of multiple-lag outof-sequence measurement filtering algorithms,” in Proceedings of SPIE, Signal and Data Processing of Small Targets, vol. 5204, Bellingham,WA, Aug. 2003, pp. 175– 187. [125] M. Orton and A. Marrs, “A bayesian approach to multi-target tracking and data fusion with out-of-sequence measurements,” in IEE, Savoy Place, London, UK, 2001. [126] A. Marrs, “Asynchronous multi-sensor tracking in clutter with uncertain sensor locations using bayesian sequential monte carlo methods,” in Proceedings of IEEE Aerospace Conference, vol. 5, 2001, pp. 52 171–52 178.

195 [127] M. Mallick and S. C. C. Carthel, “Advances in asynchronous and decentralized estimation,” in Proceedings of IEEE Aerospace Conference, vol. 4, 2001, pp. 41 873– 41 888. [128] A. Alouani and T. R. Rice, “On asynchronous data fusion,” in Proceedings of the Annual Southeastern Symposium on System Theory, 1994, pp. 143–146. [129] T. Kirubarajan, H. Wang, Y. Bar-Shalom, and K. R. Pattipati, “Efficient multisensor fusion using multidimensional data association,” IEEE Transactions on Aerospace and Electronic Systems, vol. 37-2, pp. 386–398, 2001. [130] E. M. Nebot, M. Bozorg, and H. F. Durrant-Whyte, “Decentralized architecture for asynchronous sensors,” Journal of Autonomous Robots, vol. 6, pp. 147–164, 1999, netherlands. [131] M. Orton and W. A. Fitzgerald, “A bayesian approach to tracking multiple targets using sensor arrays and particle filters,” IEEE Transactions on Signal Processing, vol. 50, no. 2, pp. 216–223, 2002. [132] W. D. Blair, T. R. Rice, B. S. McDole, and E. M. Sproul, “Least squares approach to asynchronous data fusion,” in Proceedings of SPIE, The International Society for Optical Engineering, vol. 1697, 1992, pp. 130–141. [133] Y. U. Cao, A. S. Fukunaga, A. B. Kahng, and F. Meng, “Cooperative mobile robotics: Antecedents and directions,” vol. 4, 1997, pp. 1–23. [134] E. Franco, T. Parisini, and M. M. Polycarpou, “Cooperative control of discretetime agents with delayed information exchange: a receding-horizon approach,” in Proceedings of the 43rd IEEE Conference on Decision and Control, Atlantis, Paradise Island, Bahamas, Dec. 2004, pp. 4274–4279. [135] R. O. Saber, W. B. Dunbar, and R. M. Murray, “Cooperative control of multivehicle systems using cost graphs and optimization,” in Proceedings of the American Control Conference, Denver, Colorado, June 2003, pp. 2217–2222.

196 [136] M. R. Anderson and A. C. Robbins, “Formation flight as a cooperative game,” in Proceedings of AIAA Guidance, Navigation and Control Conference and Exhibit, Boston, MA, Aug. 1998, pp. 244–251. [137] L. P. F. Giulietti and M. Innocenti, “Autonomous formation flight,” IEEE Control Systems Magazine, vol. 20, pp. 34–44, Dec 2000. [138] M. Pachter, J. J. D’Azzo, and A. W. Proud, “Tight formation flight control,” AIAA Journal of Guidance, Dynamics and Control, vol. 24, no. 2, pp. 246–254, 2001. [139] C. Zheng, M. Ding, and C. Zhou, “Cooperative path planning for multiple air vehicles using a coevolutionary algorithm,” in Proceedings of the 1st International Conference on Machine Learning and Cybernetics, Beijing, China, 2002, pp. 219– 224. [140] R. Beard and T. McLain, “Multiple UAV cooperative search under collision avoidance and limited range communication constraints,” in Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, Hawaii, Dec. 2003. [141] H. Stark and J. W. Woods, Probability, Random processes, and Estimation Theory for Engineers. Upper Saddle River, NJ: Prentice-Hall, Inc., 1994. [142] K. Konolige, “A gradient method for realtime robot control,” in Proceedings of the International Conference on Intelligent Robots and Systems, Takamatsu, Japan, Jan. 1996, pp. 639–646. [143] C. Choi and J. Lee, “Dynamical path-planning algorithm of a mobile robot: Local minima problem and nonstationary environments,” Mechatronics, vol. 6, no. 1, pp. 81–100, June 1996. [144] C. Zhao, M. Farooq, and M. Bayoumi, “Collision-free path planning for a robot with two arms cooperating in 3-d work space,” in Proceedings of the International Conference on Robotics and Automation, Minneapolis,Minnesota, Apr. 1996, pp. 2835–2840.

197 [145] N. F. Foster, G. S. Dulikravich, and J. Bowles, “Three-dimensional aerodynamic shape optimization using genetic evolution and gradient search algorithms,” in Proceedings of AIAA 34th Aerospace Sciences Meeting and Exhibit, Reno, NV, Jan. 1996. [146] E. Burian, D. Yoerger, A. Bradley, and H. Singh, “Gradient search with autonomous underwater vehicles using scalar measurements,” in Proceedings of the IEEE Symposium on Autonomous Underwater Vehicle Technology, Monterey, CA, June 1996, pp. 86–98. [147] I. M. Mitchell and S. Sastry, “Continuous path planning with multiple constraints,” in Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, Hawaii, Dec. 2003, pp. 5502–5507. [148] P. Ogren, E. Fiorelli, and N. E. Leonard, “Cooperative control of mobile sensor networks: Adaptive gradient climbing in a distributed environment,” IEEE Transactions on Automatic Control, vol. 49, no. 8, pp. 1292–1302, 2004. [149] R. Bachmayer and N. E. Leonard, “Vehicle networks for gradient descent in a sampled environment,” in Proceedings of the 41st IEEE Conference on Decision and Control, Las Vegas, Nevada, Dec. 2002, pp. 112–117. [150] R. Larson, R. P. Hostetler, B. H. Edwards, and D. E. Heyd, Calculus with Analytic Geometry. Boston, NY: Houghton Mifflin Company, 2002. [151] H. L. Pastrick, S. M. Seltzer, and M. E. Warren, “Guidance laws for short-range tactical missiles,” Journal of Guidance, Control and Dynamics, vol. 4, no. 2, pp. 98–108, 1981. [152] T. Minka,

“The lightspeed matlab toolbox,”

2005. [Online]. Available:

http://research.microsoft.com/ minka/software/lightspeed/ [153] “The

real-time

workshop

toolbox

user

guide.”

[Online].

Available:

http://www.mathworks.com/access/helpdesk/help/pdf doc/rtw/rtw ug.pdf

198 [154] F.

Semiconductors,

“Mpc5xx

microprocessors.”

[Online].

Available:

http://www.freescale.com/webapp/sps/site/prod summary.jsp?code=MPC563

BIOGRAPHICAL STATEMENT Ugur Zengin was born in Tercan, Erzincan, Turkey, in 1980. He received his B.S. and M.S. degrees from Istanbul Technical University, Turkey, in 2001 and 2003, respectively, in Aeronautical Engineering and Ph.D. degree from The University of Texas at Arlington in 2007, in Aerospace Engineering. From 2001 to 2003, he was with the department of Aeronautical Engineering, Istanbul Technical University as a graduate assistant. From 2003 to 2007, he was with the department of Aerospace Engineering, University of Texas at Arlington, as a graduate teaching and research assistant in Computer Aided Control System Design (CACSD) Lab. He was the recepient of Who’s Who Among Students in American Universities and Colleges Award in 2005 and John De Young Outstanding Research Award in 2006. His current research interests are path planning and target tracking for UAVs, guidance and navigation of autonomous vehicles, system identification, estimation and control theory. He is an AIAA and IEEE member.

199