variety of military and civilian applications, such as border patrol, weather data collection, search and rescue, and in- telligence gathering. It is possible to deploy ...
Toward Developing an Optimal Cooperative Search Algorithm for Multiple Unmanned Aerial Vehicles Pedro DeLima and Daniel Pack Department of Electrical and Computer Engineering United States Air Force Academy Colorado Springs, CO 80840 {pedro.lima, daniel.pack}@usafa.edu ABSTRACT
different trajectory planning algorithms was compared by Enright and Frazzoli [5] for a single UAV approaching multiple targets in a convex, unobstructed space. Also, Jizhou et al. [6] used multiple images from a single UAV to reconstruct 3D images of buildings.
In search and surveillance operations, a team of small Unmanned Aerial Vehicles (UAVs) can provide a robust solution that surpasses in efficiency what can be achieved by a single aircraft with comparatively superior mobility and sensors. The key to unlock such potential is in cooperative decentralized control strategies that allow each UAV to independently determine its actions while aiming at optimizing the team’s objectives through collaboration. In this paper we present the results of a statistical analysis that demonstrates the efficacy of the distributed search technique proposed by the authors in [1]. Three metrics are used to measure the search performance: dynamic coverage, heterogeneity of the coverage, and energy consumption.
Although many single UAV applications can be performed through manual operation, only autonomous systems are suitable for a growing number of applications. In recent years several independent results, particularly in the development of joint search strategies [7] and path coordination [8], have made clear the synergistic benefit of using multiple systems working cooperatively. In turn, the emergence of multiple-UAV solutions have led to an even greater need for flight and decision automation, due to the long recognized limitations of manual operation of multiple, multi-tasking UAVs [3].
KEYWORDS: cooperative, unmanned aerial vehicles, search algorithm, autonomous, decentralized.
In this paper we consider multiple UAVs persistently searching a specified area and focus on a decentralized search algorithm capable of achieving superior search performance through collaboration between team members. The proposed search algorithm, originally introduced in [1], allows each UAV to determine its own heading based on a weighted combination of four independent factors: (1) distance from neighboring UAVs, (2) distance from approaching search boundaries, (3) elapsed time since a subsection of the mission area was last visited, and (4) current heading. Although the algorithm itself has already been demonstrated in real-world tests involving groups of two UAVs such as the one shown in Figure 1 [9], the independent contribution of each of the four aforementioned factors in the heading calculation has not yet been the subject of accurate analysis in the context of the joint search efficiency.
1. INTRODUCTION Autonomous multiple UAVs under decentralized control possess a number of benefits that make them ideal for a wide variety of military and civilian applications, such as border patrol, weather data collection, search and rescue, and intelligence gathering. It is possible to deploy UAVs to areas where an elevated risk makes it unadvisable for manned vehicles to operate. Moreover, decentralized systems are intrinsically robust, and the potential capability of automated guidance and decision making allows a team of UAVs to jointly perform a mission for extended periods of time [2] with little human supervision [3]. Traditionally, researchers have strived to develop a single sophisticated system to perform complex tasks. In Saripalli et al. [4], control strategies were developed for a single UAV with the ultimate goal of providing sensory data as well as communication with ground units. The efficiency of
978-1-4244-2249-4/08/$25.00 ©2008 IEEE
This paper’s contribution lies on an in-depth statistical analysis of the impact of each factor that affects the computation of future UAV heading directions in the proposed search al-
506
Figure 1. One of the UAV Platforms Used to Fly Search Missions Guided by the Proposed Search Algorithm. gorithm and how those factors affect the overall search effectiveness. To measure the efficiency of a combined search effort by multiple UAVs, three metrics are employed. The first metric is the dynamic coverage, which encapsulates a group of UAVs’ ability to revisit each section of the search area with a high frequency. The second metric is heterogeneity, which measures the negative effect of having some of the sections of the search area visited frequently while others remain comparatively neglected. Finally, the third metric focuses on energy efficiency by measuring the total amount in degrees of turns performed by all UAVs while autonomously searching an area.
Figure 2. The Square and Circle Areas Indicate the Overlaying of the Footprints of the UAV’s Two Sensors. visible sporadically, sensing with only one type of sensor provides less likelihood of target detection. The proposed decentralized search algorithm expresses individual and collective goals in four weighted directional vectors which, when summed, provide the desired future heading h of the UAV as shown in Equation 1. h = wg vg + wU AV vU AV + wc vc + wm vm ,
In Section 2 the proposed cooperative decentralized search strategy is summarized. In Section 3 the search efficiency of four heading equations representing incremental stages of completion of the proposed approach are analyzed and compared through a rigorous statistical analysis, followed by a summary of the main findings and final conclusion.
(1)
where vg is the goal vector, vU AV is the UAV spread vector, vc is the vector that repels the UAV from the search area boundaries, vm is the momentum vector, and the ws are their corresponding weights. New desired headings are calculated at fixed discrete moments triggered by the arrival of new sensor data. Figure 3 shows an example of how the four vectors manifest for a particular UAV with the search boundary on its left and another cooperating UAV to its right. Shaded areas in the figure represent areas that have already been searched by sensors. In actual execution, the shaded areas are composed of a series of sensor footprints, such as the ones shown in Figure 2, sequentially plotted on the map of the search area every time a new sensor reading is taken.
2. COOPERATIVE SEARCH The primary goal of the decentralized search algorithm is to allow each UAV to guide itself in response to changes in the environment in a manner that generates an efficient collective search for the targets of interest. A UAV perceives the environment through its sensors, communication with other members, and pre-existing knowledge of the search area. We assume that each UAV is equipped with an omnidirectional radio frequency (RF) sensor with a limited sensing range, and a non-gimballed camera capable of detecting visible targets through image processing. Furthermore, due to filtering and processing limitations, we assume that new sensing data is available at fixed intervals. Figure 2 shows an example of the area on the ground covered by both sensors, a shape we refer to as the sensors’ footprint. Note that the area covered by the RF sensor is a circle, while the camera senses a rectangle aligned along the UAV’s current heading. Since the ground targets are assumed to emit or be
In detail, the vector vg points in the direction of the area around the UAV that has not been visited by any other member of the group for the longest time. It is in the interest of the group to investigate such areas because there is a greater likelihood of detecting targets in them. For implementation, each UAV stores in its memory a map of the search environment. At every iteration, each UAV receives the current position and heading of all other UAVs in the team, and updates its search map. However, due to the fact that
507
Figure 4. The Unitary Vector vg Points in the Direction of the Trapezoid-Shaped Section with Less Previously Covered (Shaded) Area, in this Case p5 .
Figure 3. The Four vVectors Involved in the Determination of a UAV’s Heading h. the targets are mobile and that their detectability is intermittent, the desire to revisit an area for additional sensing should grow with time. For this reason, at every iteration the history values associated with all locations in the map are decreased by a small factor. By comparing the average values of different sections of the map around the UAV, it is possible to determine which one is the most attractive area.
The second vector, vU AV , is the vector responsible for the UAV to UAV avoidance. Its presence increases the spread of the UAVs during the search effort, therefore increasing the group’s coverage. Given the total number of UAVs available nU AV and their respective locations xi (i[1, nU AV ]), vU AV can be calculated as shown in Equation 4.
For the sake of reducing the computational cost of the method, the average value of the map is calculated in n trapezoid-shaped sections spaced in front and around the UAV at pj (j[1, n]) points as shown in Figure 4. The distance from the center of the aircraft to the most distant point of the trapezoid-shape area used to evaluate the map is the search range rs , and it is one of the adjustable parameters of the algorithm. Equations 2 and 3 show how the normalized vg is obtained. vg =
pm − x , |pm − x|
m = arg min (M (pj ) + β ∗ anglej ) , j[1,n]
vU AV =
n U AV i=1
x − xi , |x − xi |2
(4)
where xi is the position of UAV i. Note that, different from vg , this vector is not normalized. As a matter of fact, vU AV is inversely proportional to the distance between UAVs. The vector vc is used to maintain the UAV within a predetermined search area by operating as an obstacle avoidance vector that concerns itself only with search boundaries within a radial comfort range rc around the UAV. In this manner, the UAV is only “pushed” away from the boundaries within a certain distance from it, preventing this vector from pushing all UAVs to the center of the search area at all times while at the same time discouraging the UAVs from moving outside of the search area. The ideal comfort range rc is influenced by the dynamics of a UAV, the shape of a sensor footprint, and the number of UAVs in the effort. Adjusting rc for each scenario can be challenging and therefore we define it as the second parameter to be set through optimization. Equation 5 describes how vc is obtained.
(2)
(3)
where x is the current position of the UAV, M is a function that calculates the sum of the intensity of the cells of the map that fall within the trapezoid-shaped area indicated by the point pj positioned around the UAV at a distance equal to its largest sensor range in the configuration shown in Figure 4, β is a small positive number, anglej is the angle of visage of the point pj with respect to the UAV’s current heading, and m is the index of the point with minimal history value that is closest to the UAV’s current heading.
vc =
nc x − bk , |x − bk |2
k=1
508
(5)
desired heading hn representing the integration of n substrategies of the heading decision equation. The first, most basic component of the search cost is its intention to simply remain within the search area, without which measuring the success of a search inside a delimited area becomes inconsequential. The vector vc encapsulates such goal, however, since it does not provide a heading outside the comfort zone, vm is used in such circumstances to generate a functional h1 as shown in Equation (6). In effect, under this search strategy a UAV is made to simply fly in a straight trajectory while outside the comfort range that extends for rc from the search borders. Since this strategy is the minimum necessary for a team to achieve measurable values for the three goals, the results of the simulations based on h1 can be treated as the baseline for comparison with the other strategies. wc vc , if within rc from borders, (6) h1 = , otherwise. vm
Table 1. Optimized Decentralized Search Algorithm Parameters. wg
wU AV
wc
rc (km)
rs (km)
0.63
0.26
0.27
0.25
0.99
where nc is the number of boundaries closer than rc to the UAVs, and bk is the point of the boundary k closest to the UAV. Finally, vm is the momentum vector that incorporates in h the desire to maintain the previous heading. It has the additional benefit of resulting in search strategies that require less fuel by minimizing the number and intensity of maneuvers. Therefore, vm is a normalized vector with the direction of the current heading of the UAV.
3. STATISTICAL ANALYSIS OF SIMULA-
Second, we introduce to the search cost the intention of a UAV to navigate to an area that has not been visited for the longest time with the addition of vg . Since vg is determined inside the entire search area, a functional desired heading equation no longer requires the addition of vm . In this manner, as shown in Equation (7), h2 is composed only of vc and vg . As the next step, we add to the desired heading the concept of cooperative coverage by incorporating the repelling forces of vU AV to form h3 as shown in Equation (8).
TION RESULTS As previously stated, the proposed search strategy based on the determination of h has as its primary goals (1) to increase the rate at which each area is revisited (dynamic coverage), (2) to reduce the variability of the visitation frequency (a.k.a. heterogeneity) across the entire search area, and (3) to minimize energy consumption by reducing the number and intensity of maneuvers performed by the UAVs. In this section we show statistically supported simulation results that justify the composition of Equation (1) by demonstrating the direct impact each component has on the three stated goals. Due to the particular impact of the vector vm and its direct influence on energy consumption and hence the flight duration, the impact of the other three vectors will be discussed in the first subsection, followed by a more in-depth analysis of the impact of the momentum vector.
h2 = wg vg + wc vc ,
(7)
h3 = wg vg + wU AV vU AV + wc vc ,
(8)
In order to measure the impact of each heading equation over the three stated goals the following metrics were used. To measure the average frequency in which any given point in the search area is visited, the dynamic coverage of the search area is measured by obtaining the average visitation history value of the cells of the search area’s map matrix. To measure the variability, or heterogeneity, of the coverage effort, the standard deviation of the cells of the map matrix at a given time is used. Note that the more uniform the search is, the smaller the standard deviation of the coverage in the history matrix, and therefore the goal is to design a search cost that leads to minimum heterogeneity. Finally, the amount of energy required by each UAV in a team to search an area is measured by the total amount of turns, in degrees, involved in all maneuvers conducted by the UAV in order to follow the desired heading stipulated by the applied heading equation.
3.1. Impact of vc , vg , and vU AV Since the specific values of the parameters of the proposed search strategy have a direct impact on its efficiency, for the purpose of the comparisons done in this section, wg , wU AV , wc , rc , and rs have been fixed to the values stated in Table 1. These particular values were obtained in [1] using an evolutionary algorithm-based optimization process performed with the goal of maximizing the coverage capability of a team of UAVs using the proposed h. Adopting an incremental approach, we break down the proposed search strategy into four sub strategies, each with a
509
Figure 5. Average Dynamic Coverage of Teams of Four UAVs Subject to Three Different Strategies to Compute the Desired Heading h.
Figure 6. Heterogeneity Levels Achieved by Teams of Four UAVs Under Three Different Strategies to Compute the Desired Heading h. Lower Levels are Desired as They Indicate Greater Homogeneity of the Search Effort.
Table 2. Dynamic Coverage, Heterogeneity, and Total Amount of Turns at the End of One Hour for Search Conducted Using h1 , h2 , and h3 . Results in Mean (Standard Deviation) Notation. Goals Dyn. Coverage Heterogen. Turns (o ) 1 h 0.736 (0.032) 0.057 (0.014) 2631 (59) h2 0.758 (0.015) 0.039 (0.004) 5210 (100) h3 0.779 (0.009) 0.033 (0.003) 5861 (77)
of simulation. As expected, the addition of vg and vU AV respectively in h2 and h3 each had a beneficial impact on the dynamic coverage, both at the initial and steady state stages of the search. Analyzing the final dynamic coverage after one hour of simulation, all three methods generated significantly different means (with probability of less than 0.001 of being equal i.e. p < 0.001) when compared to one another, with the results obtained by the application of h3 showing an average gain of more than 5% over the baseline.
The simulations involved a team of four UAVs, randomly initialized inside a 4 km x 4 km search area, conducting a search mission for one hour while the three metric were measured with UAVs employing in different runs the h1 , h2 , and h3 heading sub strategies. All UAVs were set to a cruise speed of 65 km/h with a maximum allowed turn rate of 2o /s, modelling typical responses of small fixed wing aircrafts. Each UAV model is outfitted with sensors with a radial maximum range of 400 m and capable of sensing such area for targets every 2 seconds. To account for the variability of the results, 50 simulation runs were performed for each scenario, with the final results listed in Table 2. All results were evaluated by pairs against the null hypothesis of equal means (test limit α = 0.05) using the standard t-test when the equal variance hypothesis could not be rejected (test limit α = 0.1), or using the Satterthwaite’s approximation to the t-test otherwise.
Figure 6 displays the results pertaining to the second goal, minimization of the heterogeneity of the search. Although these are independent measures, the results follow the same trend of the ones obtained for dynamic coverage, with heterogeneity of the search decreasing with the addition of vg and subsequently also with vU AV . When statistically analyzed, the application of h1 , h2 , and h3 also results in significantly different (p < 0.001) final search heterogeneity after one hour of simulation. Finally, the results for the total degrees of turns (Table 2) show three significantly different (p < 0.001 for all pairs) outcomes. In the total degrees of turns the tradeoff of the previous improvements credited to the addition of the vectors vg and vU AV becomes clear. With the addition of vg , the average degrees of turns per UAV was 1.98 times greater than the baseline established by h1 , this ratio increasing to 2.23 with the addition of vU AV in h3 . Although the exact relationship between the characteristics of the maneuvers and the ultimate flight autonomy of an UAV is aircraftdependent, increases of this magnitude are bound to have an
The average evolution of the dynamic coverage, the first goal of the proposed search algorithm, when applying the proposed decentralized search algorithm using h1 , h2 , and h3 can be seen in Figure 5 with respect to the entire one hour
510
Table 3. Dynamic Coverage, Heterogeneity, and Total Amount of Turns at the End of One Hour for Search Conducted Using h4 with Different Values for the Weight wm . Results in Mean (Standard Deviation) Notation. Goals Dyn. Coverage Heterogen. Turns (o ) 3 h 0.779 (0.009) 0.033 (0.003) 5861 (77) h4 (wm =1) 0.776 (0.010) 0.031 (0.002) 4315 (106) h4 (wm =2) 0.771 (0.011) 0.034 (0.003) 3733 (118) h4 (wm =3) 0.750 (0.014) 0.039 (0.004) 3220 (105) h1 0.736 (0.032) 0.057 (0.014) 2631 (59)
Figure 7. Dynamic Coverage Achieved by Teams of Four UAVs Guided by h4 with Different Values for wm . For the Purpose of Comparison, the Outcomes of h3 and the Baseline h1 are Shown in Grey.
impact on the total time the group can persistently search an area.
3.2. Impact of vm Having shown the tradeoffs of the improvements on the first two metrics brought by the addition of the vg and vU AV , we now show how vm can be used to manage the increase of flight maneuvers. To follow in the same structure, we now introduce h4 , described in Equation (9), which incorporates all four vectors. h4 = wg vg + wU AV vU AV + wc vc + wm vm ,
Figure 8. Heterogeneity Levels Achieved by Teams of Four UAVs Guided by h4 with Different Values for wm . Lower Levels are Desired as they Indicate Greater Homogeneity of the Search Effort. For the Purpose of Comparison, the Outcomes of h3 and the Baseline h1 are Shown in Grey.
(9)
Note that the desired heading h4 has the same structure of h initially presented in Section II, but with weight wm left as a variable. Table 3 shows the impact of applying h4 with gradual increases on the value of wm over the three search metrics. For easier comparison, the results of the baseline h1 and from h3 are also displayed in the same table.
the weight wm = 1. In such a case, the dynamic coverage was not found to be significantly different (p = 0.26), even though a significant reduction (p < 0.001) in the total amount of maneuvers was observed, leading to an average reduction of 26.4% over the maneuvers performed when h3 was applied. Further comparison with the results of the application of h3 also reveals that the heterogeneity of the search conducted using h4 with wm = 1 was significantly smaller (p < 0.001) than that achieved when h3 was used. This result in particular indicates that the introduction of vm with small enough wm can have beneficial impacts on the energy consumption of the UAVs without incurring on negative effects on the other two metrics.
As it can be seen in Table 3 and Figures 7 and 8, the greater the value assigned to weight wm , the greater the loss of search efficiency concerning the first two metrics. On the other hand, greater energy efficiency is also achieved with the increase of wm . Conducting the same statistical analysis as performed in the previous subsection, results in significantly different (p < 0.01) outcomes for all metrics across all levels of wm and their comparisons with the results using h1 and h3 . The only exception to this is the comparison of the dynamic coverages achieved using h3 and h4 with
511
4. CONCLUSION
[9] Yoon, Y., S. Gruber, L. Krakow, and D. Pack, “Autonomous target detection and localization using cooperative unmanned aerial vehicles,” to appear in Proc. Int. Conf. on Cooperative Control and Optimization, Jan. 2008.
The incremental addition of the four basic components that constitute the overall search method has shown that allowing UAVs to seek areas that have not been visited for the longest time while maintaining distance from one another through the use of the vectors vg and vU AV has a great positive impact on dynamic coverage and heterogeneity of the search. However, these improvements are counterbalanced by a severe increase in the amount of turning requested of the UAVs, leading to greater energy consumption. The complete proposed heading equation h was shown to address this issue with the addition of vm , which has a direct impact on energy consumption and can also be adjusted to regulate such tradeoff to match the endurance requirements of each mission.
REFERENCES [1] DeLima, P., D. Pack, and J. C. Sciortino, Jr, “Optimizing a search strategy for multiple mobile agents,” Proc. SPIE Defense and Security, Orlando, Fl., Mar. 2007. [2] Everaerts, J., N. Lewyckyj, and D. Fransaer, “PEGASUS: design of a stratospheric long endurance UAV system for remote sensing,” Proc. International Society for Photogrammetry and Remote Sensing Congress, Istanbul, Turkey, pp. 2933, 2004. [3] Van Breda, L., “Operator performance in multi Maritime Unmanned Air Vehicle control,” Report TNO-TM 1995 A-76, TNO Human Factors Research Institute, Soesterberg, The Netherlands, 1995. [4] Saripalli, S., D.J. Naffin, and G.S. Sukhatme, “Autonomous flying vehicle research at the University of Southern California,” Proc. International Workshop on Multi-Robot Systems, pp. 73-82, 2002. [5] Enright, J.J., and E. Frazzoli, “UAV routing in a stochastic, time-varying environment,” Proc. IFAC World Congress, Prague, Czech Republic, 2005. [6] Jizhou, W., L. Zongjian, and L. Chengming, “Reconstruction of buildings from a single UAV image,” Proc. International Society for Photogrammetry and Remote Sensing Congress, Istanbul, Turkey, pp. 100-103, 2004. [7] Bourgault, F., T. Furukawa, and H.T. Durrant-Whyte, “Coordinated decentralized search for a lost target in a bayesian world,” Proc. IEEE/RJS Conference on Intelligent Robotic Systems, Las Vegas, NV, pp. 48-53, Oct. 2003. [8] Simeon, T., S. Leroy, and J. Laumond, “Path coordination for multiple mobile robots: a resolution-complete algorithm,” IEEE Transactions on Robotics and Automation, vol. 18, no. 1, pp. 42-49, Feb. 2002.
512
