Optimal UAV Path Planning in a 3D Threat Environment by Using Parallel Evolutionary Algorithms Nuri Özalp The Scientific and Technological Research Council of Turkey- TUBITAK
[email protected] Abstract- In recent years, unmanned aerial vehicles-UAVs represent one of the most demanding technologies in aeronautics, and they have tremendous appeal because of their operability with considerable autonomy (by using minimal human intervention). UAVs have to operate in complex environments with different constraints such as obstacles, threatening zones, UAV kinematics, etc. In this technology, path planning plays a crucial role for high autonomy operations, although absolute autonomy is still an open question. In this paper, we tried to discuss, how a feasible path planning for a UAV can be done in the 3dimensional environment by avoiding threats such as a radar network which contains several radars with different detection ranges. The proposed methodology is implemented with using genetic algorithms, and a parallel approach is used for reducing path planning calculations. The environment is represented as 3 dimensional structure by using World Wind, which is an open-source and accurate 3D environment browser. The developed methodology can provide fast and safe routes for autonomous single UAVs or operatorassisted flight. Index Terms— Path planning, Genetic Algorithm, Optimization, UAV, 3D environments.
Ozgur Koray Sahingoz Turkish Air Force Academy Computer Engineering Department 34149, Yesilyurt-Istanbul, Turkey
[email protected] I. INTRODUCTION A UAV is a powered self-piloted aircraft, which flies without human operators. It can carry different size of payloads, sensors, and communication equipment. Although, UAVs have limited communication and sensing capabilities, because of their advantages on size, weights, mobility and the price; they can be efficiently used both in civil aviation and military operations. Some examples of these applications include surveillance, providing air support, tracking ground vehicles, crop dusting, etc. In the aerial related projects, which need long exploration and observation time, there may be problems about weakness of human being. To overcome this problem, researchers have to spend more times and money for securing the life of the pilots and controllers in the plane. On the other hand, by using pilotless vehicles, such as unmanned aerial vehicles-UAVs, more effective and less expensive vehicles can be produced. UAVs are typically used in three type environments of D’s, which stand for dangerous, dirty, and dull [1]. Dangerous environment term is used for operationally risky areas for both aircrafts and pilots. Dirty defines especially the environment which may be contaminated by chemical, biological, or nuclear particles. Finally, dull is used for tasks, which require long lasting operations, which are not desired to use manned planes.
At the same time, UAVs can be used to solve some problems quickly and less costly. They spend less power, and they are more eco-friendly regarding to noise and wastes. Compared with manned planes that maintain the same objective; UAVs are more advantageous in terms of maintenance, expenditure and energy saving. While the number of UAVs increases, the need for autonomous path planning for them also increases. UAVs generally work in dynamic environments, which contain lots of variable from the shape of the ground, vehicles, or changes in the atmosphere. While they are flying in high velocity, they should be capable of reacting to this type of variables changes quickly. Thus, they became one of the main focuses of interests and many researches works on this topic. Path planning of UAVs is an extremely complicated problem. Route planning plays a vital role as flight durations of UAVs are restricted. Predetermined targets should be patrolled in optimum period. Path planning operation should protect UAVs from the threats that can be faced. Each of these notes is part of expenditure of path planning. Thus, optimum path planning will both minimize the risks and enable us saving time and fuel. In most of the route planning algorithms for ground vehicles, two dimensional scenarios are used for solving the problem. On the other side for UAVs, there is a need of taking into account 3dimensional environments for generating real scenarios. There are several algorithms in literature about UAV path planning such as potential fields [2], Particle Swarm Optimization [3], Ant Colony Optimization [4], A* (Shortest path search based on A-star algorithm)[5], Genetic Algorithms [6] and Artificial Bee Colony [7] algorithms. However, most of them used 2 dimensional solutions for UAVs. A genetic algorithm (GA) is a population based problem solving technique which treats possible solutions as biological entities and it produce new solutions from the bests of the previous solution with some operator. GA supports multi-objective optimization, and it can be developed to work in parallel. It uses different operators and selection methods for generating new generation solutions.
In this paper, we used GA for path planning of UAVs by getting into account the 3 dimensional environment definitions not only for terrain information but also for threat information. Threats are generally defined as radars (or radar containing missiles, aircrafts or anti-aircraft), which can be defined as a radar equation. Each radar type has intrinsic parameters. In this paper, we assumed all radars have a circular threat zone. However, this can be extended by adding different parameter. By using this information, a UAV path is planned before the mission, by using the available terrain and threat information to find an optimal solution. The remainder of this paper is organized as follows: Firstly, related works about UAV path planning algorithms are explained in section 2. Then, as an open-source 3D environment browser, World Wind is explained and the proposed usage of GA is shown in Section 3. Threats with their avoidance approach and Experimental results are described in Section 4 and 5 respectively. Finally, conclusions and future works are outlined. II. RELATED WORK There are several researches on usage of GA in the path planning structures of UAVs. Topcuoglu and et al. developed a 3D synthetic terrain for UAV path planning and implemented an offline planner in this environment [8]. Route planning is designed with GA and meta-heuristic techniques. Route planning is done in accordance with the technical constraints is represented by B-spline curve structures. According to the UAV route planning maneuverability of planes are drawn and turning angles are constructed to be greater than allowable minimum curvature radius maneuverability. UAVs can avoid the restriction on a land area from the source to the destination. In [9], an offline task planning for UAVs to go target points optimally by using genetic algorithm in accordance with TuAF (Turkish Air Force) needs. 2D target points are demonstrated on Google maps, and there was an interface, which is capable of modeling, taking package and storing in Matlab Simulink. The system also calculates the routes for each UAV with different number. Experiments are done with different parameters, and their results are shown as a feasible solution.
Some path planning methods have deterministic and probabilistic threads that can be compared with the related works in [10]. Online and offline algorithms were run, and their performance is calculated and analyzed. In the result of this research, advantages and disadvantages of different path planning methods were shown and compared, authors showed that the deterministic path planning approach needs more information and much time for minimum numbers of image, on the other hand, probabilistic approaches are more flexible and adaptive. Some path planning strategies [11] are studied for an autonomous UAVs to track a vehicle on the ground has capabilities of changing its head and speed. UAVs use different tracking algorithms, and if the tracked vehicle does not move or its speed is under chosen threshold value, UAV starts to move in a circular path or a curved path. If the speed of the vehicle is over chosen thresholds, UAV starts to track as a sinusoidal path to balance speed changes and calibrate width dynamically. Multi Objective evolutionary algorithms are also used in some researches [12]. Authors aimed to find solutions related to conflicting goals of minimizing length of the path and maximizing margin of safety. They used NSGA-II algorithm for solving this type of problems their algorithms generated a curved path which is represented using B-Spline curves. To get an overview of the state during recovery scenarios is indispensable. UAVs quickly and efficiently obtain the necessary information. In [13], authors offered a solution to 3D environments to explore including the disaster areas presents one approach for path planning in a partially or completely destroyed building. Using multiple UAV coordination and task allocation results a decrease in the problem’s solution time. The use of multiple UAVs in a distributed manner without any central coordination instance and field reconnaissance work includes planning as well as a target-oriented way for multi-UAV communication with the use of UAVs is prevented from using unnecessary discovery potential areas and land-based. Proposed algorithm enables to calculate the fastest routes for exploration uses the simplicity of the gradient method.
III. IMPLEMENTATION OF THE SYSTEM In this section, two main components of the proposed system are detailed: Genetic Algorithm and 3D mapping with Environment Browser. A. 3D Environment Browser A UAV flying environment consists of different types of entities, such as mountains, ground objects, radar, vehicles, etc. Let’s put an image. In a real simulation of UAVs, it is required to use 3D mapping of the environment. Most of the related works on the topic either uses 2D mapping, or they developed their 3D environment simulation areas and test their system according to virtually created environment maps. It is important to use real 3D mappings for reaching efficient and effective solutions. World Wind is a client interactive 3D geographic imaging and information system, which maps the surface of the Earth, as well as a directly overhead perspective, is explored [14]. 3D visualization is most powerful of a surface perspective through the eyes of an astronaut when zooming in/out and watching the land surface perspective. Viewing from directly overhead and relatively World Wind from the surface, close to the traditional 2D map viewer similar to 3D visualization. Refresh rates for interactive flyovers and to enable the creation of large-scale field data, World Wind caches and makes extensive use of advanced 3D terrain rendering algorithms. World Wind provides rich display properties, representation of geographic data and many different geometric shapes. World Wind project is an open-source code with in 3D representation. It has easy integration and can work all possible operating systems. Also, it can work on the web, and geographic data are taken from NASA servers. It includes the high resolution data and allows reaching instant geographic coordinates. On the project, each point is accessible by means of latitude, longitude, and elevation. These values are important for defining the place of an entity in 3D space. There are many interactive feature of the system for displaying the systems theatre. It is executed in a layered structure; therefore representation can be easily prepared in many overlapping layers. Anybody can easily set up a map server in his local
computer and work offline. It is Java and OpenGL based software, which has some prominent features such as transparency, dragging, selecting, zooming and navigation. It can read data in many formats such as [15]: Imagery: JPG, PNG, GeoTIFF, JPEG2000, etc. Government: NITF, MrSID, RPF (CADRG, CIB, etc.), DTE, etc. GIS: Shapefile, KML, VPF, GML, GeoJSON, GeoRSS, GPX, NME, etc. Coordinate Systems: Lat/Lon, UTM, MGRS. Datums: WGS84, NAD27, etc. B. Usage of Genetic Algorithm Genetic algorithms are a kind of optimization problem whose main purpose is to solve complex problems. It is an intuitive research method based on the foundation of the process of natural development of living things in the universe. The goal is to find global optimization of mathematical functions. First of all, GA is used by Dutch to solve iteration of the genetic algorithm problems of laws of evolution [9]. The aim of GA is to achieve always getting better solution, which is defined in a chromosome structure. To reach this goal, the sequences of genes are changed for generating better offsprings. As a result of its evolutionary structure, GA keeps the better offsprings and eliminates worse ones in an iterative structure. It generates a set of solutions, which is called as a population, rather than a single solution. This also increases the percentage of success in reaching the best solution (or at least acceptable solution). By using GAs, a Start
Generate initial population
Evaluate all individuals
significant improvement can be achieved in terms of performance, especially for NP hard problems. In the proposed system, algorithm is run according to the flow diagram in Figure 1. 1. Firstly, a possible solution is encoded in a chromosome structure. 2. All possible solutions (chromosomes) construct a population. 3. Initial population is constructed with random ordered chromosomes. 4. Each chromosome’s fitness function is calculated and stored. This value shows the quality of the chromosome. 5. Some chromosomes are selected to be parents, which will inherit their genes to the new individuals. This selection can be done according to different selection algorithms. 6. In the evolution of these individuals, some crossover and mutation operators are used. These operators change the place of genes and try to reach a better cost individual. 7. A local optimization technique is used for finding a better order of genes by doing swapping. 8. After that, costs of all individuals are calculated again and same processes are done iteratively until reaching a stopping criterion. When the iterations reached the stopping criteria, such as iterating a specified number of times, or a period of times, etc.; best cost individual is selected. Chromosome Structure is in the form of an order of control points, which defines a point with x, y, and z geographical coordinates. A Yes Stop ?
Best Individuals
No Selective reproduction
Crossover
Mutation
Heuristics2opt
Fig. 1. Evolutionary Algorithm Flow Diagram
Result
chromosome should contain all CP in it, which means all control points should be visited. Each CP should take place in this structure only once, which means UAV should reach each CP only once. Although there are different implementations of this chromosome structure, we accept each UAV should return to its base; therefore, after visiting all control points it should return to the initial base. Generation of Initial Population is one of the important decisions to make. It affects how fast best solution can be found. Initial populations of GA are traditionally generated randomly. A. Fitness Function Searching the best UAV path typically means searching the shortest path. This is the main case of the TSP type problems. In this paper, mainly total distance of the UAVs is used for calculating main cost function (fitness function). To generate new generation, there is a need to select two parents’ chromosome from the population. There are two important factors for implementing a Parent Selection (PS) method: parent selection should allocate reproductive opportunities for each individual, and to generate new offsprings, best cost chromosomes should have more opportunity than low cost ones. There are mainly two types of PS methods: Tournament Selection and roulette wheel selection. Because of the speed and simplicity of the implementation, tournament selection is used for PS in the proposed algorithm. Two types of Crossover Operators are used in the proposed algorithm: One Point crossover and Two Point crossover. In the former one, firstly, one crossing point is determined and the subchromosomes are exchanged according to this point. From the first parent the sub chromosome is selected from the first gene to the specified position, and then from the second parent, subchromosome is selected from the specified point to the end to combine a new chromosome as shown in Figure 2. In this approach some genes can be identical in the newly generated individual, therefor it can correcting by changing the last one with an unused gene.
0
6
7
1
5
3
8
2
4
9
0
5
4
1
8
3
9
2
6
7
0
6
7
1
8
3
9
2
6
4
Fig. 2. One Point Crossover Operator Two-point crossover is similar to the one point crossover except that there are two different selected points, and all genes between these points are swapped. According to this, two different offsprings can be produced as shown in Figure 3. If some genes are duplicated they can be reorganized by changing the second gene with an unused one. The two-point crossover is better for large populations, while the one point crossover can be preferred for small populations. 1
7
6
2
5
4
7
8
3
Parent 1
7
8
9
4
5
8
1
2
3
Parent 2
1
7
6
4
5
8
3
9
2
7
8
9
2
5
4
1
3
6
Offspring1 Offspring2
Fig.3. Two Point Crossover Mutation is used to maintain the genetic diversity of the population. It changes one gene from the chromosome. While changing it with the other, generally gene array between these genes are also reversed to maintain the minimum number of changes as shown in Figure 4. 0
6
7
1
5
3
8
2
4
9
0
6
2
8
3
5
1
7
6
0
Fig. 4. Mutation
IV. THREATS AND AVOIDANCE In a real 3D environment, especially in war theatre, there are lots of threats in the environment of the mission. These threats can be classified in 5 main domains: Radars Missiles Anti-aircrafts Aircrafts UAVs As a detection mechanism, either a UAV can be detected by radar, or it can be seen or heard by a human operator. In the above list, first three ones are the static threats, and they can have different size of radar coverage. The last two ones are mobile threats, and they are not in the scope of this paper. It is left to the future works. A. Adding Threats Radar has the shape of a cylindrical threedimensional coverage, and mainly they are located on the ground. A threat of radar can be defined as a radar equation. Each radar type has intrinsic parameters. In this paper, we assumed all radars have a circular threat zone. However, this can be extended by adding different parameter. B. Virtual Points around Radar Avoiding from a threat is a challenging issue in the UAV path planning problem. If 3D environments are taken into account, then this problem is getting harder. To solve this avoidance problem we added eight virtual points around the threats. These points are distributed around the threat with 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° to the diameter of the circle. It is aimed to circulate the UAV around the coverage of the threat. C. Detecting Threats While a UAV follows up a route between any two points, it is important not to enter the radar coverage of the threat. Therefore, there is a need to implement a new formula to avoid the threats. The main problem is can be defined as follows, “if we want to go from control point B to control point V, how can we decide whether the threat, centered at point M, cover our path or not”.
V
T
M
B
Fig. 5. Detecting Threats. The proposed algorithm is one of the applications of the cosines theorem. As shown in Figure 5, first the starting point (B) with radar circle tangent point (T) and the radar center point (M) using the distance between the angle cos(MBT) and cos(MBV) are calculated according to Equation 1 and 2 respectively. (
(
)
)
After that, these two values are compared. If cos(MBT) ≤ cos(MBV) then this means the path of the UAV is crosscut the threat zone of the radar. Therefore, the UAV should maneuver for avoiding the threat. D. Avoiding Threat It is aimed to avoid from threats by using virtual points around the threats. This approach does not make any changes in the structure of the genetic algorithm, and it not only finds the most convenient way to update but also the value of the fitness function and the path continues on this path. Firstly, the nearest point to the pre-calculated path is determined by finding nearest virtual point. Then nearest point of the target point, which is on the other side of radar and nearest virtual point of a moving point are determined. By using this iterative approach, the destination is reached by using these virtual points as depicted in Figure 6. New cost of |BC| is calculated according to Equation 3.
The proposed algorithm is tested with different numbers of control points, different number of threads and according to the GA parameters listed in Table 2. TABLE 2. GENETIC ALGORITHM PARAMETERS Parameters Values
Fig. 6. Avoiding threats by using Virtual Points E. Conversion of dimensional coordinates In 3D environments, working with Ellipsoid areas is a challenging issue. Therefore in most of the 3D studies it is neglected only direct path lengths are taken into consideration. World Wind system gives us realistic 3d information, and as a result, it can be calculated by using standard calculations. Ellipsoidal distance between two points whose azimuth values are known is calculated according to Vincenty’s formulas described in [16]. How distance between two points in geographic coordinate system can be calculated is showed in Figure 7. V. EXPERIMENTAL RESULTS We have explained the necessary concepts for calculating a 3D Path planning for UAVs. The proposed algorithm produces a feasible trajectory. However, the path production time is still too long for single core machines, especially for trying to use in real time applications. Therefore, we developed a parallel version of the proposed algorithm, which is suitable for multicore/multi CPU machines. We implement this algorithm in Java by using World Wind, which is an open source and 3D environment browser. Test platform is described in Table 1.
Number of Control Points Number of Chromosomes in a Population Crossover Rate (%) Mutation Rate (%) Elitism Rate (%) # of Threads # of Cores
202,411,662,963 100 100 20 20 1,2,4,8,16 8
To show the efficiency and correctness of the proposed algorithm, we simulated 4 different 3d Path planning which has a different number of control point as 202, 411, 662 and 963 respectively. Figure 7 represents the results of the execution time and the speedup of the proposed parallel GA for different number of threads, and Figure 8 shows the calculated cost for 411 CPs mission.
Fig.7. Execution Time and Speedup
TABLE 1. TEST PLATFORM PROPERTIES Properties CPU RAM Cache Op. Sys. IDE Java
Values Intel(R) Quad i7 CPU
[email protected] GHz Dual Channel DDR3-1333 SDRAM 8 MB-256 KB-32 KB (L3-L2-L1) Windows 7 Ultimate, 64 bits Net Beans 7.2, 64 bits JDK 7.10
Fig. 8. Cost Calculation
a) Altitude between 840m-1126m, area of 195 km2
b) Altitude between 820m-1021m, area of 275 km2
c) Altitude between 760m-980m, area of 150 km2
d) Altitude between 600m-910m, area of 135 km2
e) Altitude between 904m-1778m, Path Length 220,5 km
f) Altitude between 450m-815m area of 140 km2
Fig. 9. The proposed algorithm’s path planning results on different regions of Turkey
Different solutions of the proposed algorithm are also depicted in Figure 9 (a-f). The graphics are drawn on the World Wind 3D Environment Browser, and the solution path length, minimum and maximum altitudes and problem theaters are also shown in the different regions of Turkey. VII. CONCLUSION AND FUTURE WORKS In this paper, we tried to solve the 3D path planning problem of UAVs by using evolutionary algorithms. The proposed algorithm uses population based approach, by using Genetic Algorithms (GAs), with same initial and final depot points in the environment. The theatre of the UAVs also contains different types and sizes of threats. In this part of the study, we assumed all threats are static treats, and they have uniform radar coverage. The parallel computational model for GA is constructed, and it is combined with the local search heuristics. System is simulated in a 3D map (by using World Wind) built from real terrain data, and the simulation results showed that the proposed method is effective and feasible. As a future work, it is planned to run the algorithm on a real-time system. Thus, we will add different type of additional constraints (UAV kinematic constraints, environmental constraints, mission specific constraints, etc.) to the proposed system. ACKNOWLEDGEMENT This work has been conducted within the Multi-Cooperation Autonomous UAVs-MUCAVproject (project number 3920-S513000), which is part of the Avionics and Air Defense Systems research program funded by The National Research Institute of Electronics and Cryptology (TUBITAK BILGEM). We thank all project partners for their work and contributions to the project. REFERENCES [1] R. K. Barnhart, S. B. Hottman, D. M. Marshall and E. Shappee, "Introduction to Unmanned Aircraft Systems", CRC Press, Taylor & Francis Group, 2011. [2] K. S. Al-Sultan and M. D. S. Aliyu, “A New Potential Field-Based Algorithm for Path
Planning”, Journal of Intelligent and Robotic Systems, vol.17, no.3, pp. 265-282, 1996. [3] V. Roberge, M. Tarbouchi and G. Labonte, "Comparison of Parallel Genetic Algorithm and Particle Swarm Optimization for RealTime UAV Path Planning," IEEE Transactions on Industrial Informatics, vol.9, no.1, pp.132-141, Feb. 2013 [4] C. Zhang, Z. Zhen, D. Wang and M. Li, "UAV Path Planning Method Based on Ant Colony Optimization", Control and Decision Conference (CCDC), 2010 Chinese, pp.37903792, 2010. [5] Y. Qu, Q. Pan and J. Yan, “Flight Path Planning of UAV Based on Heuristically Search and Genetic Algorithms”, Industrial Electronics Society, IECON 2005, 31st Annual Conference of IEEE, 2005. [6] D. Ambrosinoa, A. Sciomachena and M. Scutellàb, “A Heuristic Based On Multi Exchange Technics For A Regional Fleet Assignment Location- Routing Problem”, Computers & Operation Research 36, pp. 442-460, Italy, 2007. [7] L. Lei and Q. Shiru, "Path Planning For Unmanned Air Vehicles Using an Improved Artificial Bee Colony Algorithm," Control Conference (CCC), 2012, 31st Chinese, pp. 2486-2491, 2012. [8] I. Oz, H. R. Topcuoglu and M. Ermis,"A Meta-Heuristic Based Three-Dimensional Path Planning Environment for Unmanned Aerial Vehicles", Sage Journals, 2012. [9] Pakkan and M. Ermis, “Assignment of a Fleet of Unmanned Aerial Vehicles to Multi Targets Using Genetic Algorithms”, Journal of Aeronautics and Space Technologies, vol.4, no.3, pp. 77-84, 2010. [10] E. Yanmaz, R. Kuschnig, M. Quaritsch, C. Bettstetter and B. Rinner, “On Path Planning Strategies for Networked Unmanned Aerial Vehicles”, IEEE, INFOCOM, Workshop on M2MCN’2011, 2011. [11] J. Lee, R. Huang, A. Vaughn, X. Xiao, J. Karl Hedrick, M. Zennaro and R. Sengupta, “Strategies of Path-Planning for a UAV to Track a Ground Vehicle”, AINS’2003, 2003.
[12] S. Mittal and K. Deb, “Three-Dimensional Offline Path Planning for UAVs Using Multi objective Evolutionary Algorithms”, Evolutionary Computation IEEE Conference, pp.1-6, 2007. [13] C. Rasche, C. Stern, L. Kleinjohann and B. Kleinjohann, “A Distributed Multi-UAV Path Planning Approach for 3D Environments”, Automation, Robotics and Applications ICARA), pp.7-12, 2011. [14] D.G. Bell, F. Kuehnel, C. Maxwell, R. Kim, K. Kasraie, T. Gaskins, P. Hogan and J. Coughlan, "NASA World Wind: Open source GIS for Mission Operations", IEEE Aerospace Conference, pp.1-9, 2007. [15] Complete Guide to World Wind, http://goworldwind.org/features/ (accessed on May 2013). [16] T. Vincenty, "Direct and Inverse Solutions of Geodesics on the Ellipsoid with Application of Nested Equations”, Survey Review, pp. 88–93, 1976.
