A Cooperative Search and Coverage Algorithm with ... - MDPI

sensors Article

A Cooperative Search and Coverage Algorithm with Controllable Revisit and Connectivity Maintenance for Multiple Unmanned Aerial Vehicles Zhong Liu *, Xiaoguang Gao and Xiaowei Fu

ID

School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, China; [email protected] (X.G.); [email protected] (X.F.) * Correspondence: [email protected]; Tel.: +86-158-2973-2829 Received: 24 March 2018; Accepted: 5 May 2018; Published: 8 May 2018







Abstract: In this paper, we mainly study a cooperative search and coverage algorithm for a given bounded rectangle region, which contains several unknown stationary targets, by a team of unmanned aerial vehicles (UAVs) with non-ideal sensors and limited communication ranges. Our goal is to minimize the search time, while gathering more information about the environment and finding more targets. For this purpose, a novel cooperative search and coverage algorithm with controllable revisit mechanism is presented. Firstly, as the representation of the environment, the cognitive maps that included the target probability map (TPM), the uncertain map (UM), and the digital pheromone map (DPM) are constituted. We also design a distributed update and fusion scheme for the cognitive map. This update and fusion scheme can guarantee that each one of the cognitive maps converges to the same one, which reflects the targets’ true existence or absence in each cell of the search region. Secondly, we develop a controllable revisit mechanism based on the DPM. This mechanism can concentrate the UAVs to revisit sub-areas that have a large target probability or high uncertainty. Thirdly, in the frame of distributed receding horizon optimizing, a path planning algorithm for the multi-UAVs cooperative search and coverage is designed. In the path planning algorithm, the movement of the UAVs is restricted by the potential fields to meet the requirements of avoiding collision and maintaining connectivity constraints. Moreover, using the minimum spanning tree (MST) topology optimization strategy, we can obtain a tradeoff between the search coverage enhancement and the connectivity maintenance. The feasibility of the proposed algorithm is demonstrated by comparison simulations by way of analyzing the effects of the controllable revisit mechanism and the connectivity maintenance scheme. The Monte Carlo method is employed to validate the influence of the number of UAVs, the sensing radius, the detection and false alarm probabilities, and the communication range on the proposed algorithm. Keywords: multi-UAVs; search and coverage; digital pheromone; distributed receding horizon optimizing; collision avoidance; connectivity maintenance; minimum spanning tree; potential field

1. Introduction Recently, multiple UAVs have received more and more attention for their accomplishments in both military and civil applications. Cooperative search and coverage is one major application of multiple UAVs equipped with sensors [1], such as camera, radar, and sonar. The goal of cooperative search and coverage is to control multiple UAVs to find several unknown ground targets scattered in a given surveillance region, while maximally reducing the uncertainty of the environment and minimizing the search time [2]. In the cooperative search and coverage problem, there are two main technical issues that should be considered [3]. (1) The environment representation and update. This focuses on how to represent Sensors 2018, 18, 1472; doi:10.3390/s18051472

www.mdpi.com/journal/sensors

Sensors 2018, 18, 1472

2 of 35

targets existence and uncertainty in the environment, and how to treat sensor observation results as evidence to update the knowledge of the environment so that the UAVs’ belief can reflect the true existence or absence of the targets within areas of the surveillance region; (2) search path planning. This focuses on how develop cooperative control methods that enable UAVs to move in such a way as to maximize the possibility of finding targets or minimize the uncertainty in the environment. Extensive studies have been carried out on these two key issues. The environment representation is the primary problem of cooperative search and coverage. In general, a common method is to make the whole surveillance area become smaller cells, and each cell is associated with values, such as probability or uncertainty, thereby constituting a map for the search region. Each UAV maintains a map of the surveillance region that serves as the UAV’s knowledge about the state of the region. There are several types of the map, such as the occupancy map [4–6], the target probability map [7–9], the uncertainty map [10–12], and so on. In fact, these maps can collectively be called the cognitive map, which is used to represent the environment and is incrementally updated based on the new sensor observations. The continuous updating of the cognitive map reflects the collecting and processing of new sensory information about the environment for the UAVs. However, each UAV can only obtain local sensory information about the whole surveillance region due to its limited sensing range. In addition, considering the non-ideal sensor performance, there are a great variety of errors and uncertainties on the sensory information from the UAVs. Therefore, the sensory information from multiple UAVs needs to be combined through some update and fusion schemes so that the best knowledge of the environment can be obtained. Most available update and fusion schemes are broadly based on Bayesian theory [2,13,14] and Dempster-Shafer theory [15]. However, few of the existing update and fusion methods can guarantee that all individual cognitive maps can be converged to the same one that reflects the true target existence status of each cell in the whole surveillance region. Search path planning is an important problem for efficient search and coverage by a team of UAVs. It is concerned with cooperative controlling of the movements of multi-UAVs in order to guarantee optimal search paths that maximize the possibility of finding targets and minimize the uncertainty in the environment. Different authors have developed several search path planning methods, such as reinforcement learning [15], potential field [16], group dispersion pattern [17], intelligence algorithm [18], dynamic programming [19], gradient optimization [20], mixed integer linear programming [21], Voronoi partitioning [22,23], and receding horizon optimization (RHO) [24–26]. In [22,23], the convex region is partitioned into Voronoi cells so that there is only one agent in each Voronoi cell according to the position of the agents. Therefore, the distributed multi-agents coverage control problem is converted into the coverage of Voronoi cell for single agent. Then, a gradient descent control law is designed to continually drive the agents toward the centroids of their Voronoi cells. In this way, the whole convex region can be covered by multi-agents without violating the collision constraint. For the reason of the fact that the RHO could effectively handle the dynamic changes of the environment and constrain movements of UAVs, it is used to solve the cooperative search and coverage problems. Reference [24] presents a receding horizon cooperative search algorithm that jointly optimizes paths and sensor orientations for a team of UAVs searching for a mobile target. In reference [25], a receding horizon, motion-planning algorithm is used to obtain the optimal search path in the given horizon. In reference [26], the distributed model predictive control method is presented to solve the cooperative search moving targets problem. In addition, the attractant and repulsion pheromones are introduced to improve the effective cooperation between UAVs. Although these above methods have been verified to be effective for the multi-UAVs cooperative search and coverage problems, they lack a controllable revisit mechanism to guide the UAVs to revisit sub-areas with large target probability or high uncertainty, so that the capability to capture the target and the efficiency of coverage are low.

Sensors 2018, 18, 1472

3 of 35

Furthermore, the sufficient information exchange and sharing in the whole team of UAVs is quite essential for cooperative search and coverage, and thus it is required that the UAVs could maintain a connected communication network. Connectivity maintenance has been applied in the multiple agents system. For example, in [27], a decentralized power iteration algorithm is designed to estimate the connectivity of the multi-agents network. Then, a gradient control law for each agent is proposed to maintain global connectivity. Reference [28] uses the potential fields to drive the agents to a full connected configuration while avoiding collisions with each other. However, these methods cannot be applied directly in the cooperative search and coverage problem in this paper. These methods often generate a full connected topology with dense communication links, which may largely restrict the motion of the UAVs and jeopardize the efficiency of cooperative search and coverage. In fact, adding communication links improves the network connectivity, but the movement of the UAVs will be extremely limited. Disconnecting links may destroy the network connectivity, but on the other hand, that will reduce UAV movement restrictions and provide more freedom to explore wider areas and increase the efficiency of searching and covering [25]. Thus, the tradeoff between the search coverage enhancement and the connectivity preservation should be considered, which aims to maintain the connected network while relaxing the motion constraints on the UAVs as possible. In this paper, we mainly study cooperative search and coverage for a given bounded rectangle region, which contains several unknown stationary targets, by a team of UAVs with non-ideal sensors and a limited communication range. The goal of mission is to minimize the search time, while gathering more information about the environment and finding more targets. During the mission process, the collision avoidance and network connectivity constraints must be guaranteed. The main contributions to this paper contain three aspects. (1) A distributed update and fusion scheme for the cognitive map is proposed. We prove that this update and fusion scheme can guarantee that all individual cognitive maps converge to the same one that reflects the true target existence status of each cell in the search region; (2) considering the revisit requirement for the sub-areas in the search region, we develop a controllable revisit mechanism based on a digital pheromone. This mechanism can control the UAVs to revisit sub-areas with large target probability or high uncertainty; (3) aiming to achieve the tradeoff between the search coverage enhancement and the connectivity maintenance, a connectivity maintenance control strategy based on the minimum spanning tree (MST) topology optimization is presented. The network topology is optimized using the MST, and only the communication links in the MST topology are maintained. Thus, the UAVs remove the redundant links without violating the global connectivity condition, and hence obtain more freedom to be dispersed for the search and coverage enhancement. The structure of this paper is organized as follows. In Section 2, the mission scenario, together with the problem formulation, is provided. In Section 3, as the representation of the environment, cognitive maps that include the target probability map (TPM), the uncertain map (UM), and the digital pheromone map (DPM) are constituted. We design an update and fusion scheme for the individual cognitive maps such that they are all converge to the same one that reflects the true environment. We also develop a controllable revisit mechanism based on DPM. This mechanism can command the UAVs to revisit some important areas where they have a high target probability or that they have not explored for a long time. In Section 4, we propose our path planning algorithm for multi-UAVs cooperative search and coverage operation. The path planning is performed in a distributed fashion. Each UAV solves local rolling time domain optimization problem, and obtains its own optimal path to search and cover the surveillance region. In this path-planning algorithm, the movement of UAVs is restricted by the potential fields to satisfy the collision avoidance and connectivity maintenance constraints. In addition, the tradeoff between the coverage enhancement and the connectivity maintenance is achieved using the MST topology optimization strategy. We present simulation and experimental results in Section 5, followed by summary and conclusions in Section 6.

Sensors 2018, 18, 1472

4 of 35

2. Problem Formulations As Sensors shown in18,Figure 1, there is a team of UAVs (Ai , i = 1, 2, . . . , N) performing cooperative 2018, x FOR PEER REVIEW 4 of 36 search and coverage mission in an unknown surveillance region that contained several stationary ground 2. Problem Formulations targets (T j , j = 1, 2, . . . , M). In unknown environment, for the UAVs, the targets number and their locations areAs unknown priori. The UAVs need to use(Aon-board sensors (e.g., cameras) to observe shown in aFigure 1, there is a team of UAVs i, i = 1, 2, …, N) performing cooperative search some coverage mission an the unknown region that contained several of stationary ground areas of and the environment, sointhat UAVssurveillance can incrementally obtain knowledge the environment and targets (T j, j = 1, 2, …, M). In unknown environment, for the UAVs, the targets number and their find targets. The decision on where to explore next is driven by the objective to increase the chance of locations are unknown a priori. The UAVs need to use on-board sensors (e.g., cameras) to observe some finding targets or reducing the uncertainty in the environment. For this purpose (1) we need to design areas of the environment, so that the UAVs can incrementally obtain knowledge of the environment a cognitive map as the environment representation, represent uncertainty and find targets. The decision on where to explore next to is driven by thetargets objectiveexistence to increase and the chance in the environment. Inorother words, each UAV maintains a cognitive map of whole surveillance of finding targets reducing the uncertainty in the environment. For this purpose (1)the we need to design a cognitive map asUAV’s the environment representation, to represent(2) targets existence andfusion uncertainty in need region that serves as the knowledge of the environment; an update and scheme the environment. In other words, each UAV maintains a cognitive map of the whole surveillance region to be designed to guarantee all cognitive maps converge to the same one that reflects the targets’ true that serves as the UAV’s knowledge of the environment; (2) an update and fusion scheme need to be existence or absence in each cell of the search region. As the UAVs move around and observe sub-areas designed to guarantee all cognitive maps converge to the same one that reflects the targets’ true of the region, the corresponding cells of the cognitive maps are updated to incorporate the information existence or absence in each cell of the search region. As the UAVs move around and observe sub-areas gained through thethe on-board sensors, as the communication network. The cognitive of the region, corresponding cellsas ofwell the cognitive maps are updated to incorporate theupdated information maps should sametheand couldsensors, reflectasthe true environment; (3)network. we need designcognitive a distributed gained be through on-board well as the communication Theto updated mapscoverage should be control same andalgorithm could reflectthat the true environment; (3) we need to a distributed search search and plans the optimal paths fordesign the UAVs to follow to search and coverage control algorithm that plans the optimal paths for the UAVs to follow to search and and coverage the surveillance region, while guarantying the constraints of the collision avoidance and coverage the surveillance region, while guarantying the constraints of the collision avoidance and connectivity preservation. connectivity preservation.

UAV Communication network

Target

Field of view Surveillance region

Figure 1. Target search by multiple UAVs.

Figure 1. Target search by multiple UAVs. 2.1. The Description of Search Environment

2.1. The Description ofinSearch As shown FigureEnvironment 2, an L × W rectangular region Ω is uniformly divided into Lx × Ly cells of the size. The cell that is located in the m-th row and n-th column is identified by its identity number As same shown in Figure 2, an L × W rectangular region Ω is uniformly divided into Lx × Ly cells c = m + (n − 1) × Lx, c ∊ {1, 2, …, Lx × Ly}. Δx and Δy denote the length and width of the cells, respectively. of the same size. The cell that is located in the m-th row and n-th column is identified by its identity Δx and Δy may be chosen as the flight distance of the UAV in a time step with the constant cruising numberspeed. c = mEach + (ncell − is1)located × Lx , with c ∈ {1, 2, . . . μ,c L L }. ∆x and ∆y denote the length and width of the its center = x[x× c, yc]Ty, in which xc and yc are coordinated with its center. cells, respectively. ∆x and ∆y may be chosen as flight of the UAVis in a time step with ζc ∊ {0, 1} indicates whether a target exists in cell cthe or not, i.e., distance ζc = 1 indicates a target present in cell T c, and cruising ζc = 0 indicates no target present in that with cell. To describe assumes the constant speed. Each iscell is located itsbetter center µc = the [xc ,problem, yc ] , in itwhich xc that and yc are (i) there is, atits most, one target each and (ii) there are noathreats obstacles in the surveillance coordinated with center. ζ c ∈in{0, 1}cell indicates whether targetand exists in cell c or not, i.e., ζ c = 1 region. indicates a target is present in cell c, and ζ c = 0 indicates no target is present in that cell. To better describe the problem, it assumes that (i) there is, at most, one target in each cell and (ii) there are no threats and obstacles in the surveillance region.

Sensors 2018, 18, 1472

5 of 35

Sensors 2018, 18, x FOR PEER REVIEW

y

5 of 36

North

△x

c

△y

W

m

n L

East x

Figure 2. A rectangular surveillance region is uniformly divided into Lx × Ly cells of the same size.

Figure 2. A rectangular surveillance region is uniformly divided into Lx × Ly cells of the same size. 2.2. Simplified Dynamic Model of UAV

2.2. Simplified Dynamic Model of UAV

We assume that the UAVs move on a fixed plane above the surveillance region. Let xi(k) = [μi(k),

T represents the position of Ai, in which xi(k) i(k)]T denote state of Aimove at timeon k. μai(k) = [xiplane (k), yi(k)] Weψassume thatthe the UAVs fixed above the surveillance region. Let xi (k) = [µi (k), and y i (k) are the planar coordinates of its projection onto surveillance region. ψi(k) is the heading T represents ψi (k)]T denote the state of Ai at time k. µi (k) = [xi (k), yi (k)]the the position of Ai , in which xi (k) angle. The two-dimension motion of UAV in a horizontal plane is analyzed, and the kinematics of and yi (k) are the planar coordinates of its projection onto the surveillance region. ψi (k) is the heading the UAV are angle. The two-dimension motion of UAV in a horizontal plane is analyzed, and the kinematics of the v ⋅ Δt ⋅ cosψ i (k )  UAV are ] i   xi (k + 1) = xi (k ) + In[ c h x ·cos ψi (k)   xi (k + 1) = xi (k) + In vcΔ·∆t (1) ψ∆x i i (k ) (1)  yi (k + 1) = yi (k ) + In[ vc ⋅hΔtv⋅ sin ](k) · ∆t · sin ψ c i  yi (k + 1) = yi (k ) + In Δy ∆y

In Equation (1), vc represents the constant cruising speed, Δt represents time step. In Equation (1), vc represents the constant cruising speed, andand ∆t represents thethe time step. Operator Operator In [·] indicates a rounding operation that maps the flight distance of Ai over a time step Δt In [·] indicates a rounding operation that maps the flight distance of Ai over a time step ∆t to the cell to the cell index increments (Δm, Δn) in surveillance region. index increments (∆m, ∆n) in surveillance region. For simplicity, it is assumed that the UAV is equipped with an autopilot that holds constant Foraltitude simplicity, it is assumed thatneed the toUAV is guidance equipped with an low-level autopilot that holds constant and ground speed. We only design inputs to this autopilot system altitudefor and ground speed. We paper, only need to design guidance inputs low-level autopilot system target searching. In this the design guidance inputs are ui(k) to = ψthis i(k), which are constrained bysearching. the dynamicInlimits of UAV,the i.e.,design the turning rate Δψ i(k) ∊ [−Δψ , Δψ According to Equation for target this paper, guidance inputs are max ui (k) =max ψ].i (k), which are constrained by (1), the dynamics of the UAVs can be described as follows. At each Δt, A i can only move from the the dynamic limits of UAV, i.e., the turning rate ∆ψi (k) ∈ [−∆ψmax , ∆ψmax ]. According to Equation (1), current cell to the neighboring cells, due to the constraints of maneuverability. More specifically, the dynamics of the UAVs can be described as follows. At each ∆t, Ai can only move from the current there are eight possible flight orientations defined as ψi(k) ∊ {1(east), 2(northeast), 3(north), cell to the neighboring cells,6(southwest), due to the constraints of Due maneuverability. More specifically, there are eight 4(northwest), 5(west), and 7(south)}. to its physical curvature radius constraints, possiblethe flight defined as ψi (k) {1(east), 3(north), 4(northwest), UAVorientations can only change its orientation at ∈ most once in2(northeast), a Δt. In this case, Ai has only three possible5(west), orientation (turnDue left, go or turn right, denoted by Δψ i(k) ∊ {l(left),the s(straight), r(right)}) 6(southwest), andchoices 7(south)}. tostraight, its physical curvature radius constraints, UAV can only change for the next time step. Thus, the maximum turning angle is Δψ max = 45°. its orientation at most once in a ∆t. In this case, Ai has only three possible orientation choices (turn left, go straight, or turn right, denoted by ∆ψi (k) ∈ {l(left), s(straight), r(right)}) for the next time step. 2.3. Communication Model Thus, the maximum turning angle is ∆ψmax = 45◦ . In this paper, we only consider the limited communication range and ignore the bandwidth

limitation, theModel communication delay, and the interruption that will be the future research direction 2.3. Communication

extending the current work. Thus, two UAVs can directly exchange information if the distance

In between this paper, only the limited communication range and ignore bandwidth themwe is no moreconsider than the communication range Rc. The network topology of the l the UAVs at time k can be modeled as an undirected graph G(k) = (V, E(k)). V = {A 1 , …, A N } is the vertices set and limitation, the communication delay, and the interruption that will be the future research direction E(k)the = {(A i, Aj)| (Ai, Aj) ∊ V; ||μi(k) − μj(k)|| ≤ Rc} is the edge set, in which ||·|| denotes the 2-norm for extending current work. Thus, two UAVs can directly exchange information if the distance between vectors. Let Ni(k) = {Aj|||μi(k) − μj(k)|| ≤ Rc; j = 1, …, N, and j ≠ i} denote the set of neighbors of Ai at them is no more than the communication range Rc . The network topology of the l UAVs at time k time k. The adjacency matrix can be expressed as can be modeled as an undirected graph G(k) = (V, E(k)). V = {A1 , . . . , AN } is the vertices set and E(k) = {(Ai , Aj )| (Ai , Aj ) ∈ V; ||µi (k) − µj (k)|| ≤ Rc } is the edge set, in which ||·|| denotes the 2-norm for vectors. Let N i (k) = {Aj |||µi (k) − µj (k)|| ≤ Rc ; j = 1, . . . , N, and j 6= i} denote the set of neighbors of Ai at time k. The adjacency matrix can be expressed as ( A(k) = [ aij ] =

ωij , ( Ai , A j ) ∈ E(k) 0, otherwise

(2)

Sensors 2018, 18, 1472

6 of 35

in which ω ij > 0 is the weight of the wireless link (Ai , Aj ). In this paper, ω ij is defined as ω ij = (dij /1000)3

(3)

in which dij = ||µi (k) − µj (k)|| is the distance between Ai and Aj . From Equation (3), we can see that a greater between Ai and Aj results in the larger weight of the link (Ai , Aj ). Then, the Laplacian matrix can be expressed as  N   ∑ a ,i = j ij L(k) = [lij ] = (4) j=1,j6=i   − a , i 6= j ij G(k) is full connected if direct communication exists between every two vertices of the graph. G(k) is connected if a sequence of edges (a route) exists for any two vertices; otherwise, G(k) is unconnected. The information sharing is quite indispensable in the cooperative search and coverage mission, and thus the network connectivity must be preserved. Let 0 = λ1 (k) ≤ λ2 (k) ≤ . . . ≤ λN (k) be the ordered eigenvalues of the Laplacian matrix L(k). According to the algebraic graph theory, if and only if λ2 (k) > 0, the graph G(k) is connected. The second smallest eigenvalue of L(k), λ2 (k), is also called as the algebraic connectivity of the graph. 3. Cognitive Map One of the most important aspects of search and coverage mission is to create a representation of the environment that contains information about targets existence and uncertainty within each cell. In this section, the notion of cognitive map is used as the environmental representation. We firstly construct the target probability map (TPM), which is used to describe the probabilities of the cells being occupied by a target. Next, the other two maps are introduced based on TPM. One is uncertainty map (UM), and it can describe the uncertainty degree of the environment. The other is digital pheromone map (DPM), which is mainly used to establish the controllable revisit mechanism. Integrating TPM, UM, and DPM, the cognitive map can be designed. 3.1. The Target Probability Map If prior intelligent information obtained is not accurate, UAV can not absolutely know the distribution of the targets. The TPM is used to describe target existing probability of each cell. The target existing probability pc (k) ∈ [0, 1] is modeled as a Bernoulli distribution, i.e., pc (k) = P(target present in cell c at time k), (1 − pc (k)) = P(no target present in cell c at time k). The higher pc (k) is, the more likely the UAV believes that a target is in cell c. pc (k) = 0.5 indicates that the UAV has no knowledge about target existence in cell c, because the probability that a target is present is equal to the probability that no target is present in cell c. Based on the above notations, the TPM is defined as M i,TPM (k) = {pi,c (k)|c ∈ Ω}

(5)

In the mission processing, each UAV maintains its own TPM. The knowledge of each UAV on target existence state in cell c is based on its sensor observations and the shared information from the neighboring UAVs by communication. So, the update of individual TPM has two stages: update TPM based on sensor observations and update TPM based on shared information. 3.1.1. Update TPM Based on Sensor Observations It is assumed that Ai takes observations via a downward-facing camera. The field of view (FoV) is a circle with sensing radius Rs that covers some cells in the surveillance region Ω. Therefore, at time k, the set of the covered cells inside the FoV of Ai denotes Φi,k Φi,k = {c ∈ Ω: ||µc − µi (k)|| ≤ Rs }

(6)

Sensors 2018, 18, 1472

7 of 35

The sensor observations about cell c at time k are defined as Zi,c,k ∈ {0, 1}, in which Zi,c,k = 1 indicates “target detection” and Zi,c,k = 0 indicates “non-target detection”. However, the sensor is non-idear. The performance of the sensor can be described by detection probability pd and false alarm probability pf , i.e., P(Zi,c,k = 1|ζ c = 1) = pd and P(Zi,c,k = 1|ζ c = 0) = pf . We assume that, for all cells and UAVs, pd and pf are constant and known prior. According to the sensor observations Zi,c,k , the pi,c,k , pi,c (k) is update via following rule, which is based on Bayesian theory

pi,c,k =

p( Zi,c,k |ζ c =1) pi,c,k−1 p( Zi,c,k |ζ c =1) pi,c,k−1 + p( Zi,c,k |ζ c =0)(1− pi,c,k−1 )

=

      

pd pi,c,k−1 , pd pi,c,k−1 + pf (1− pi,c,k−1 ) (1− pd ) pi,c,k−1 , (1− pd ) pi,c,k−1 +(1− pf )(1− pi,c,k−1 )

c ∈ Φi,k and Zi,c,k = 0

pi,c,k−1 ,

c∈ / Φi,k

c ∈ Φi,k and Zi,c,k = 1

(7)

By introducing the nonlinear transformation that described in Equation (8), the update rule is rewritten as shown in Equation (9). 1 Qi,c,k , ln( − 1) (8) pi,c,k  pf   ln pd , c ∈ Φi,k and Zi,c,k = 1 1− p (9) Qi,c,k = Qi,c,k−1 + υi,c,k ;υi,c,k , ln 1− p f , c ∈ Φi,k and Zi,c,k = 0 d   0, c ∈ / Φi,k From Equation (9), we can see that the evolution of Qi,c,k depends on the number of detections that is taken over cell c up to time k, which is denoted as mi,c,k . In fact, mi,c,k → +∞, pi,c,k → 0 if no target is present in cell c, and pi,c,k → 1 if a target is present in cell c. The symbol “→” indicates “approaches”. It means that the converged TPM, M i,TPM (k), can reflect the true existence and absence of the targets. To prove this view, the convergence property of the updating method (Equation (9)) is analyzed in Theorem 1. Theorem 1. Given the initial prior TPM 0 < pi,c,0 < 1 for Ai ., and 0 < pf < 0.5 < pd < 1, the following conclusions hold by implementing the updating rule in Equation (9).

•

If ζ c = 1, which indicates a target is present in cell c, as mi,c,k → +∞, then Qi,c,k → −∞ (i.e., pi,c,k → 1) and (Qi,c,k /mi,c,k ) → pd ln(pf /pd ) + (1 − pd )ln(1 − pf /1 − pd );

•

If ζ c = 0, which indicates no target is present in cell c, as mi,c,k → +∞, then Qi,c,k → +∞ (i.e., pi,c,k → 0), and (Qi,c,k /mi,c,k ) → pf ln(pf /pd ) + (1 − pf )ln(1 − pf /1 − pd ).

The proof of Theorem 1 is seen in Appendix A. From Equation (9), we can find several interesting properties. If 0 < pf < 0.5 < pd < 1, which means that the sensor could provide useful information, then pi,c (k) > pi,c (k + 1) if a target is present in cell c and pi,c (k) < pi,c (k + 1) if no target is present in cell c. Therefore, the upper bound pmax and the lower bound pmin of the target existing probability are introduced. If pi,c,k ≥ pmax , the UAV has obtained enough evidence to support its belief in a target existing in cell c. If pi,c,k ≤ pmin , the UAV confirms that no target exists in cell c. In order to confirm whether there is a target in cell c or not, the UAVs needs to detect cell c several times to update the target existence probability to approach the upper bound pmax or the lower bound pmin . Theorem 2 shows how to estimate the minimum number of observations required in a given cell c. Theorem 2. Given the initial prior TPM 0 < pi,c,0 < 1 for Ai , and 0 < pf < 0.5 < pd < 1.

•

+ required in cell c to satisfy the If a target is present in cell c, the minimum number of observations mavg condition pi,c,k ≥ pmax is estimated as

Sensors 2018, 18, 1472

8 of 35

+ mavg ≥

•

ln pd ln

h

pf pd

pi,c,0 (1− pmax ) pmax (1− pi,c,0 )

i

− pf + (1 − pd ) ln 11− pd

(10)

− required in cell c to satisfy If no target is present in the cell c, the minimum number of observations mavg the condition pi,c,k ≤ pmin is estimated as

− mavg ≥

ln pf ln

pf pd

h

pi,c,0 (1− pmin ) pmin (1− pi,c,0 )

i

− pf + (1 − pf ) ln 11− pd

(11)

The proof of Theorem 2 is seen in [3]. We can see that the sensor performance is better, e.g., when the detection probability pd and the lower the false alarm probability pf are larger, + or m− is smaller. This means that the better the the minimum number of observations required mavg avg performance of the sensor, the faster the TPM converges. 3.1.2. Update TPM Based on Shared Information First, according to the sensor observations Zi,c,k , each UAV Ai at time k updates its own TPM using Bayesian rule in Equation (9). Hi,c,k = Qi,c,k + vi,c,k (12) Then, each UAV Ai exchanges the updated TMP Hi,c,k to neighboring UAVs, and updates its own TPM again (map fusion) by following consensus protocol N

Qi,c,k =

∑ ωi,j,k Hj,c,k

(13)

j =1

in which ω i,c,k = 1 − (|N i (k)|/N), ω i,j,k = (1/N) for j ∈ N i (k), and ω i,c,k = 0 for j ∈ / N i (k). N i (k) is the set of neighbors of Ai at time k. Similar to the Theorem 1, the rationality and the convergence property of the distributed update and fusion method in Equation (13) is analyzed in Theorem 3. Theorem 3. Given the initial prior TPM 0 < pi,c,0 < 1 for Ai ., and 0 < pf < 0.5 < pd < 1, if the network topology G(k) of the UAVs is connected at all times, the following conclusions hold by implementing the distributed update and fusion scheme in Equation (13).

•

If ζ c = 1, which indicates a target is present in the cell c, as mc,k → +∞, then Qi,c,k → −∞ (i.e., pi,c,k → 1) and (Qi,c,k /mc,k ) → (pd /N)ln(pf /pd ) + ((1 − pd ) /N)ln(1 − pf /1 − pd );

•

If ζ c = 0, which indicates no target is present in the cell c, as mc,k → +∞, then Qi,c,k → +∞ (i.e., pi,c,k → 0), and (Qi,c,k /mc,k ) → (pf /N)ln(pf /pd ) + ((1 − pf ) /N)ln(1 − pf /1 − pd ). N

in which mc,k = ∑ mi,c,k represents the total number of detections that taken over the cell c up to time k by the i =1

whole team of UAVs. The proof of Theorem 3 is seen in Appendix B. Theorem 3 gives two important conclusions, as follows 1.

As mc,k → +∞, the converged TPM, M i,TPM (k), i = 1, 2, . . . , N, which is updated according to the distributed update and fusion scheme in Equation (13), can reflect the true existence and nonexistence of the targets. Thus, the rationality of the distributed update and fusion scheme in Equation (13) can be proven.

Sensors 2018, 18, 1472

2.

9 of 35

The relationship between the average detected rate of the cells and the convergence speed of the TPM is explicated in Theorem 3. For example, if a target is present in the cell c, as mc,k → +∞ Sensors 2018, 18, x FOR PEER REVIEW

2.

9 of 36

Qi,c,k Qtargets. p of the distributed update 1 − pfand fusion scheme in nonexistence of the Thus, the rationality i,c,k = mkc,k → pd ln f + (1 − pd ) ln ,a Equation (13) can be proven. mc,k pd 1 − pd Nk

(14)

The relationship between the average detected rate of the cells and the convergence speed of the TPM is explicated in Theorem 3. For example, if a target is present in the cell c, as mc,k → +∞

in which (mc,k /(Nk)) represents the global average detected rate of the cell c by the UAVs. Q Q of Qki,c,k approaching Equation (14) shows that the speed −∞ or +∞ is proportional to the global 1− p p = → p ln + (1− p )ln a m 1− p m p (14) average detected rate of the cell c. This conclusion is important for improving the effectiveness of Nk cooperative search and coverage. in which (m c,k/(Nk)) represents the global average detected rate of the cell c by the UAVs. Equation i , c, k

i ,c , k

f

f

d

c, k

c, k

d

d

d

(14) shows that the speed of Qi,c,k approaching −∞ or +∞ is proportional to the global average detected

Whenrate weofimplement Equation there isfor a data overflow problem which is caused by extremely the cell c. This conclusion(13), is important improving the effectiveness of cooperative search and coverage. large or small value of Qi,c,k . Thus, we set a bound Qmax > 0, such that Qi,c,k ∈ [−Qmax , Qmax ]. 3.2. The

When we implement Equation (13), there is a data overflow problem which is caused by extremely large or small value of Qi,c,k. Thus, we set a bound Qmax > 0, such that Qi,c,k ∊ [−Qmax, Qmax]. Uncertainty Map

3.2. The Uncertainty As mentioning above,Map if pmin < pi,c,k < pmax , Ai cannot confirm whether there is a target in cell c or As mentioning if pmin < pfor i,c,k < A pmax Ai cannot whether there is a target inof cellthe c orcells in the not; in other words, cell c above, is uncertain order confirm to quantify the uncertainty i . ,In not; in other words, cell c is uncertain for A i. In order to quantify the uncertainty of the cells in the surveillance region, we define the following uncertainty associated with cell c, which is a function of surveillance region, we define the following uncertainty associated with cell c, which is a function of its target existing probability pi,c,kpi,c,k its target existing probability ηi,c,k = e−Kη |Qi,c,k | (15) −K Q (15) ηi ,c , k =e η in which Kη > 0 is a gain parameter. |·| means absolute operator. According to Equations (8) and (15), in whichbetween Kη > 0 is a gain absolute operator. the relationship η i,c,kparameter. and pi,c,k|·| ismeans shown in Figure 3. According to Equations (8) and (15), i ,c , k

the relationship between ηi,c,k and pi,c,k is shown in Figure 3. 1 0.9 0.8

Uncertainty ηi,c,k

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Target existing probability pi,c, k

Figure 3. Uncertainty verse target probability in cell.

Figure 3. Uncertainty verse target probability in cell. It is seen that when the cell c has pi,c,k = 0.5, it has maximal uncertainty ηi,c,k = 1, which indicates that cell c is unknown for the UAVs completely. When cell c has pi,c,k = 1 or 0, it has minimal It is seen that when the cell c has pi,c,k =are 0.5, it has maximal uncertainty η i,c,kor=not. 1, which indicates uncertainty ηi,c,k = 0. In this case, the UAV completely sure about the target present In the that cell c is unknown for the UAVs completely. When cell c has p = 1 or 0, it has minimal uncertainty process of cooperative search and coverage, more attention shouldi,c,k be paid to the cells with higher uncertainties. on theare above notations, we can define UM as η i,c,k = 0. In this case,Based the UAV completely sure aboutthe the target present or not. In the process of

cooperative search and coverage, more attention uncertainties. Mi, UM(k) =should {ηi,c,k|c ∊be Ω} paid to the cells with higher(16) Based on the above notations, we can define the UM as 3.3. The Digital Pheromone Map

3.3. The

Theorem 3 shows the convergence TPM M i,UMspeed (k) =of{ηthe ∈ isΩ}proportional to the global average i,c,k |c detected rate of the cells in the surveillance region. It means that, in order to reduce the uncertainty of the cells in the surveillance region as soon as possible, it is essential to improve the global average Digital Pheromone Map

(16)

Theorem 3 shows the convergence speed of the TPM is proportional to the global average detected rate of the cells in the surveillance region. It means that, in order to reduce the uncertainty of the cells in the surveillance region as soon as possible, it is essential to improve the global average detected rate of the cells for the UAVs, which equates to improving the controllable revisit ability of the UAVs. For this reason, we develop a controllable revisit mechanism that is based on the characteristic of

Sensors 2018, 18, 1472

10 of 35

pheromone, such as “secrete, propagate and evaporation”. This mechanism can be used to control the UAVs to revisit sub-areas with large target probability or high uncertainty, and hence improve the performance of search and coverage. The digital pheromone map is defined as M i,DPM (k) = { si,c,k |c ∈ Ω}

(17)

in which si,c,k is the pheromone strength in the cell c at time k. Digital pheromone supports three primary operations: (i) release: Pheromone can be released quantitatively into the cell; (ii) propagate: Pheromone propagates from a cell to its neighboring cells; (iii) evaporate: Pheromone gradually evaporates to zero over time. In order to simulate these three primary operations of pheromone, the propagation factor Gs ∈ (0, 1) and the evaporation factor Es ∈ (0, 1) are defined. Equation (18) describes evolution of the pheromone strength in cell c at time k si (c, k ) = (1 − Es ){(1 − Gs )[si (c, k − 1) + k i (c, k) · ds ] + g(c, k)}

(18)

in which ds is the release amount of pheromone at time k. si (c, k − 1) is the pheromone strength in cell c at time (k − 1). The binary variable ki (c, k) ∈ {0, 1} is the pheromone releasing switch in cell c at time k. g(c, k) denotes the amount of pheromone propagated to cell c from its neighboring cells during the time period (k − 1, k], which can be described by the following Equation (19). g(c, k) =

∑

c0 ∈ N (c)

Gs [ s ( c 0 , k − 1) + k i ( c 0 , k ) · d s ] |N(c0 )| i

(19)

in which N(c) is the neighbor set of the cell c, the neighboring cells are denoted as c0 ∈ N(c), and |N(c0 )| is the number of the neighboring cells of cell c0 . si (c0 , k − 1) is the pheromone strength in neighboring cell c0 at time (k − 1). ki (c0 , k) ∈ {0, 1} is switch coefficient of the pheromone releasing in the neighboring cell c0 at time k. The key of controllable revisit mechanism is determining the switch coefficient of the pheromone releasing in the cells of the DPM at time k. The switching coefficient ki (c, k) indicates whether the cell c autonomously releases pheromone or not. If ki (c, k) = 1, cell c will release the pheromone, and the released pheromone will propagate to the neighboring cells. In this way, the pheromone fields will be formed and attract the UAVs to revisit cell c. In order to drive the UAVs to revisit the cells that have large target probability or high uncertainty, the pheromone releasing switch of cell c should be turned on in the following two cases. Case 1: If target existing probability in cell c satisfies the condition pi,c,k > 0.5, it indicates that the UAVs are more likely to believe that a target exists in cell c. However, the UAVs do not have enough evidence to support their beliefs, since pi,c,k < pmax . In this case, the UAVs are required to detect cell c again (revisit) so as to confirm target present state of cell c. Thus, the pheromone releasing switch of cell c should be turn on (i.e., ki (c, k) = 1), in order to attract the UAVs to revisit cell c. Once pi,c,k ≥ pmax or pi,c,k ≤ pmin , the switch should be turned off (i.e., ki (c, k) = 0) immediately, and, finally, the pheromone in cell c evaporates to zero over time. Case 2: Assume that τ c is the last revisited time of cell c, T0 is a pre-defined time threshold, and t is current time. If (t − τ c ) > T0 , then ki (c, k) = 1, which means cell c has not been explored for a long time and should be revisited; otherwise, ki (c, k) = 0. Once cell c has revisited by a UAV, its pheromone releasing switch is turned off. After a period of time, if the condition (t − τ c ) > T0 is satisfied again, the switch in cell c will be turn on again. 4. Distributed Path Planning Algorithm for Cooperative Search and Coverage Optimal search and coverage paths can be designed based on the cognitive maps of the UAVs. Since the cognitive map of each UAV is incrementally updated based on its sensor observations and

Sensors 2018, 18, 1472

11 of 35

the shared information from other UAVs by communication, each UAV continually re-plans its path to guarantee the team of UAVs identifies maximum number of targets or gathers maximum information about environment. Therefore, multi-UAVs cooperative search and coverage problem is an on-line Sensors 2018, 18, x FOR problem. PEER REVIEWIn this section, we plan optimal paths of the UAVs for 11cooperative of 36 dynamic optimization search and coverage in the frame of distributed receding horizon optimizing. information about environment. Therefore, multi-UAVs cooperative search and coverage problem is an on-line dynamic optimization problem. In this section, we plan optimal paths of the UAVs for 4.1. Distributed Receding Horizon Optimizing Model for Cooperative Search and Coverage cooperative search and coverage in the frame of distributed receding horizon optimizing.

In the frame of distributed receding horizon optimizing as shown in Figure 4, at time k, 4.1. Distributed Receding Horizon Optimizing Model for Cooperative Search and Coverage Ai optimizes its control inputs Ui (k ) and computes its own path Xi (k) based on the received the frame of distributed receding horizon optimizing as shown in Figure 4, at time k, current stateInXthe −i ( k ) of its neighboring UAVs. Ai computes its own path by solving the following local A i optimizes its control inputs U i (k ) and computes its own path X i (k ) based on the received the optimization problem ∗ neighboring UAVs. Ai computes its own path by solving the following current state X −i (k ) of its U (20) i ( k ) = argmax Ji (Xi ( k ), Ui ( k ), X−i ( k )) Ui ( k )

local optimization problem

 *  Xi (kU+ 1|kmax ) =Jfi i((XXi (ik(),k U+i (qk |),kX),−U k )+ = arg ))k + q | k )) i ( ki ( i (q   U (k )  xi ( k + q | k ) ∈ Ξ s.t. ) = f i ( X i ( k + q | k ), U i ( k + q | k ))  X i ( k + q + 1 | ku  i (k + q|k) ∈ Θ    x (k + q | k ) ∈ Ξ   q = 1, 2, .i . . , T; i = 1, 2, . . . , N s.t.  i

(20)

(21)

(21) ui ( k + q | k ) ∈ Θ   Define Ji as the “gain” at decision time theNUAV dynamic model. Ξ and Θ denote q =step 1, 2,...,k. T ; if i= is 1, 2,...,

the feasible state set and admissible control input set of UAV, respectively. Ui (k) = {ui (k + 1|k), . . . , Define Ji as the “gain” at decision time step k. fi is the UAV dynamic model. Ξ and Θ denote ui (k + T|k)} denote the control inputs sequence of Ai over the time horizon [k + 1:k + T], which is the feasible state set and admissible control input set of UAV, respectively. U i (k ) = {ui(k + 1|k), …, determined at time step k. According to the simplified dynamic model of UAV, the control inputs u ui(k + T|k)} denote the control inputs sequence of Ai over the time horizon [k + 1:k + T], which is is the choose orientation ψi of the aircraft. Xi (simplified k) = {xi (kdynamic + 1|k), model . . . , xiof (kUAV, + T|k)} the prediction state determined at time step k. According to the the is control inputs u sequence, which is defined asψthe planned path of A . X−i+(1|k), k) = …, {xj (k)|j N i (k)} denote the received the i of the aircraft. X i (k ) =i {xi(k xi(k + ∈ T|k)} is the prediction state is the choose orientation current sequence, state of the neighboring UAVs of A . Generally, in order to satisfy the collision avoidance and i which is defined as the planned path of Ai. X −i (k ) = {xj(k)|j ∊ Ni(k)} denote the received connectivity maintenance constraints, Ai needs to obtain the future state sequence of its neighboring the current state of the neighboring UAVs of Ai. Generally, in order to satisfy the collision avoidance UAVs, denoted X−i (k +maintenance q k) = {xj (kconstraints, + 1|k), . . .A,i xneeds . . . , xthe + T|k)|j ∈ sequence N i (k), q =of1, its 2, . . . , T}, j (k + q|k), j (k future and connectivity to obtain state ∗ * (k + 1|k), . . . , x * (k + T|k)}. However, due to the bandwidth when Aneighboring plans its own path X ( k ) = {x X ( k + q | k ) = {x j (k + 1|k), …, x j (k + q|k), …, x j (k + T|k)|j ∊ N i (k), q = 1, 2, UAVs, denoted −i i i i i * limitation realistic wireless it is…, hard the future sequence its owncommunication path X i (k ) = {xi*links, (k + 1|k), xi* (kto+receive T|k)}. However, duestate to the …, of T},the when Ai plans of the all neighboring UAVs within one sampling time. In this paper, we adopt a feasible approach to bandwidth limitation of the realistic wireless communication links, it is hard to receive the future reduce the communication between UAVs. Each UAV A only receives the current state X ( k ) of its state sequence of the all neighboring UAVs within one isampling time. In this paper, we adopt −i a feasible approach to current reduce the communication between Each UAV A receives the neighboring UAVs. Using state X−i (k) and the UAVUAVs. dynamic model, Ai i only can estimate the future ) and state Xon the UAV dynamic current state −i (k ) of its neighboring −i (kthe state sequence of theX neighboring UAVs atUAVs. next TUsing time current steps. Based estimate state sequence of model, Ai can estimate the future state sequence neighboring UAVs at next T time steps. Basedproblem the neighboring UAVs, Ai computes its own pathof bythe solving the following local optimization on the estimate state sequence of the neighboring UAVs, A i computes its own path by solving the in Equations (20) and (21). In order to guarantee the estimation accuracy and reduce the computation following local optimization problem in Equations (20) and (21). In order to guarantee the estimation time, the planning horizon must be shorter. accuracy and reduce the computation time, the planning horizon must be shorter. Communication Network

X 1 (k )

X −1 ( k )

Optimization in the planning horizon (RHO)

x1 (k ) u1* [k + 1: k + Tp ]

A1

X i (k )

...

X −i (k )

Optimization in the planning horizon (RHO) * xi (k ) ui [ k + 1: k + Tp ]

Ai

X N (k )

...

X − N (k )

Optimization in the planning horizon (RHO) * x N (k ) uN [k + 1: k + Tp ]

AN

Figure 4. Diagram of distributed receding horizon optimizing frame for multi-UAVs search and coverage.

Figure 4. Diagram of distributed receding horizon optimizing frame for multi-UAVs search and coverage.

Sensors 2018, 18, 1472

12 of 35

Sensors 2018, 18, x FOR PEER REVIEW

12 of 36

4.2. Search Path Decision Process

4.2. Search Path Decision Process

The2018, whole process of the search path decision strategy is illustrated in Figure 5.three There Sensors 18, x FOR PEER REVIEW 12 ofare 36 The whole process of the search path decision strategy is illustrated in Figure 5. There

are

three sub-stages wholedecision decision process: (i) prediction; (ii) decision; and (iii) acting. sub-stagesin in the the whole process: (i) prediction; (ii) decision; and (iii) acting. 4.2. Search Path Decision Process

Strat atprocess k=0 The whole of the search path decision strategy is illustrated in Figure 5. There are three obtaining the state k = k +1 update Cognitive Map sub-stages in the wholexdecision process: (i) prediction; (ii) decision; and (iii) acting. i(k) of Ai at the (TPM, UM, DPM)

current time k Strat at k = 0

Target existing probability update Cognitive Map Uncertainty (TPM, UM, DPM) Digital pheromone strength

obtaining the state k = k +1 xi(k) of Ai at the xi(k) current time k

sensor Target existing Optimal path probability Pi* (k ) Vehicle observations dynamics Uncertainty selection (Acting) Zi,c,k *

Expanding planning tree

xi(k)

(Prediction)

(Prediction)

(Decision) U (k) Digital pheromone strength

P i (k )

Expanding planning tree P i (k )

sensor observations Zi,c,k

Optimal UAVs X − ipath (k ) from neighboring Pi* (k ) Vehicle dynamics selection (Acting) U*(k)

(Decision)

Figure 5. Flow diagram forsingle single UAV UAV search decision strategy. Figure 5. Flow diagram for searchpath path decision strategy.

X − i (k ) from neighboring UAVs

4.2.1. Prediction Stage

4.2.1. Prediction Stage

Figureusing 5. Flow diagram for single UAV search decisionmodel, strategy. In this stage, current state xi(k) and the UAVpath dynamic each UAV Ai generates its

In this stage,state using currentXstate x (k) and the UAV dynamic model, each UAV Ai generates its step, predicted sequence, i (k ) = i{xi(k + 1|k), …, xi(k + q|k), …, xi(k + T|k)}, at next time 4.2.1. Prediction Stage predicted state sequence, X ( k ) = {x (k + 1|k), . . . , x (k + q|k), . . . , x (k + T|k)}, at next time step, i i (k + q). i in which xi(k + q|k) represents the ipredicted state at time In this stage, state xi(k) anddynamic thestate UAVmodel, model, UAV Ai one generates in which xi (kAccording + q|k)using represents the predicted atdynamic timeA(k + q). to current the simplified UAV i can onlyeach move from cell to its another X ) = UAV {x i(k +possible 1|k), …,orientation xmodel, i(k + q|k), xi(k +the T|k)}, next time step, predicted state to sequence, i (konly neighboring cell, has three choices for nextattime step, i.e., turn to left,another According the and simplified dynamic A…, only move from one cell i can go straight, and turn only right, based on the and orientation. predicted in which xi(kcell, + q|k) represents thethree predicted statecurrent atorientation timeposition (k + q). choices neighboring and has possible for the Therefore, next timethe step, i.e., turn left, According to theright, dynamic Areflects i can only move from Therefore, one cells cell to another Xsimplified = {xi(kUAV + 1|k), …, xi(kmodel, + T|k)} set of reachable from future state stateand sequence i (k ) based go straight, turn on the current position andthe orientation. thethe predicted neighboring cell, and hasthe only three possible orientation choices for thestate nextxtime step, i.e., turn left, time step (k + 1) to future time step (k + q), based on the current i(k) at time k. These reachable sequence Xi (k) = {xi (k + 1|k), . . . , xi (k + T|k)} reflects the set of reachable cells from the future time go straight,form and turn right, based on the current position and orientation. Therefore, the predicted expanding shown in Figure 6. The expanding is denoted step (k +cells 1) to theanfuture timeplanning step (k tree, + q),asbased on the current state xi (k)planning at timetree k. These reachable X ( k ) = {x i (k + 1|k), …, x i (k + T|k)} reflects the set of reachable cells from the future state sequence i     P ( k ) = { Pi (k+1| k) , …, Pi (k +q| k) , …, Pi (k +T | k) }, in which Pi (k +q| k) is the set of the predicted reachable as i cells form an expanding planning tree, as shown in Figure The expanding planning tree is denoted 6. 3T candidate time step the future on the i(k) at time k. These reachable cells(kat+ 1) thetofuture step step (k + (k q).+It q),isbased clear that thecurrent expanding tree contains timetime state xplanning e e e e as cells Pi (kform ) = {anPexpanding , . . . , Ptree, Pi (kl6. +The T kexpanding ) }, l in which Pi (ktree + lqis kdenoted ) is the set of the k ) , . .in. ,Figure planning i ( k +asqshown i ( k + 1 k )planning paths; the l-th path can be denoted as Pi (k) = {Pi (k + 1|k), …, Pi (k + q|k), …,

(k+1| kin (k +(cell) ∈ l(k predicted cellsPi (kat future (k P+ q).| ikIt isqthe clear that expanding planning tree =+ {T|k)}, inistep which the2,the predicted reachable as P i (Pkil)(kreachable P ) , …, +qthe | kwaypoint ) , …, P Ttime | k) }, P which the + q|k) = 1, …, T. ()k is + | k set ) , qof i i i (k +qP l l T l T cells at 3thecandidate future timepaths; step (k the + q).l-th It is path clear that expandingas planning 3 candidate contains can the be denoted Pi (k) =tree {Picontains (k + 1|k), . . . , Pi (k + q|k), . . . , l(k) l(k paths; the inl-th paththe can be denoted as l (kPi+ = ∈ {PiP ++ 1|k), …, Pil2, (k . .+. ,q|k), e Pi l (k + T|k)}, which waypoint (cell) P q|k) ( k q ) , q = 1, T. …, k i i

Pil(k + T|k)}, in which the waypoint (cell) Pil(k + q|k) ∈ Pi ( k + q | k ) , q = 1, 2, …, T.

The l-th path Pi l ( k + 3 | k ) P l (k ) = i

l

Pi (k + 2| k)

{Pi l (k + 1| k ),

l The l-thPpath i (k + 2 | k ), Pi ( k + 3 | k ) P l (k ) = l i

Pi l (k + 1| k ) l

Pi (k + 3 | k )} {Pi l (k + 1| k ),

Pil (k + 2| k)

Pi l (k + 2 | k ),

Pi l (k + 1| k )

Pi l (k + 3 | k )} :Pi (k + 1| k );

: Pi (k + 2 | k );

:Pi (k + 3 | k );

Figure 6. Illustration of recursive 3-step planning tree (T = 3). :Pi (k + 1| k );

: Pi (k + 2 | k );

:Pi (k + 3 | k );

Figure 6. Illustration of recursive 3-step planning tree (T = 3).

Figure 6. Illustration of recursive 3-step planning tree (T = 3).

4.2.2. Decision Stage In this state, in the basis of the current knowledge about the environment (such as, the target existing probability pi,c,k , the uncertainty η i,c,k , and the digital pheromone strength si,c,k ) available via the cognitive map, and the positions and orientations of the team of UAVs, each UAV uses a

Sensors 2018, 18, 1472

13 of 35

multi-objectives optimization function Ji to select and update its search path. In other words, at each time step k, Ai evaluates the value of Ji associated with each path and selects the optimal path and determines the corresponding optimal control input (heading angle) sequence U*(k) = {ψi * (k + 1|k), . . . , ψi * (k + T − 1|k)}. 4.2.3. Acting Stage The first item ψi * (k + 1|k) in the optimal decision sequence U*(k) is implemented to guide Ai to visit the cells that waiting to be visited, and Ai updates its own cognitive map into the next round of cycle. 4.3. Multi-Objectives of the Cooperative Search and Coverage Mission In this section, we investigate different objectives of the search and coverage mission, which include (i) environment exploration; (ii) target discovery and environment coverage; (iii) collision avoidance and (iv) connectivity maintenance. So, we define the reward of environment exploration JA , the reward of target discovery and environment coverage JB , the cost of collision avoidance JC , and the cost of connectivity maintenance JD , respectively. 4.3.1. Environment Exploration The main objective of the mission is exploring the whole environment to decrease the uncertainty about the existence and nonexistence of the targets in the cells of the environment. In other words, the UAV should follow the path where there is maximum uncertainty in the cognitive map. Thus, if Ai selects the l-th candidate path, Pi l (k) = {Pi l (k + 1|k), . . . , Pi l (k + q|k), . . . , Pi l (k + T|k)}, to follow, the reward of environment exploration JA can be defined as T

JA (i, l, k ) =

∑

∑

ηi,c,k

(22)

q=1 c∈Φi ( Pl (k +q|k )) i

in which Φi (c) represents the set of cells that would be searched by Ai along the path Pi l (k). 4.3.2. Target Discovery and Environment Coverage Although the main objective of the search mission is to explore the environment to gather more information about it, one could also be interested in exploiting that information to concentrate the UAVs around the targets to capture them as soon as possible. The objective of target discovery and environment coverage is to distribute the UAVs across the environment while aggregating in more important sub-areas. These more important sub-areas refer to the cells that have high target probability (Case 1 in Section 3.3) or have not been explored for a long time (Case 2 in Section 3.3). Based on this consideration, we design the controllable revisit mechanism based on DPM in Section 3.3. Thus, if Ai selects the l-th candidate path, Pi l (k) = {Pi l (k + 1|k), . . . , Pi l (k + q|k), . . . , Pi l (k + T|k)}, to follow, the reward of target discovery and environment coverage JB can be defined as T

JB (i, l, k) =

∑

∑

si,c,k

(23)

q=1 c∈Φi ( Pl (k +q|k )) i

4.3.3. Collision Avoidance We use the concept of virtual rivaling force, which is borrowed from [29], to solve the collision avoidance problem. The main idea of the rivaling force mechanism is to treat the paths of other UAVs as “soft obstacles” to be avoided in path selection. The virtual rivaling force F j →i exerted by Aj to Ai at time k is non-zero if the relative position and heading conditions are both hold. The relative position condition imposes the requirement that Aj needs to be sufficiently close to Ai . The relative heading

Sensors 2018, 18, 1472

14 of 35

Sensors 2018, 18, x FOR PEER REVIEW

14 of 36

4.3.3. Collision Avoidance condition means that, if Aj exerts a rivaling force on Ai , Aj must be moving in the same or opposite direction as We Ai , use approximately. overall rivaling force is exerted by from all the other UAVs Ai at time the concept ofThe virtual rivaling force, which borrowed [29], to solve theupon collision The main idea of the rivaling force mechanism is to treat the paths of other UAVs k is F i =avoidance ∑j 6=i F j →problem. i. as “soft obstacles” to beofavoided in path selection. virtual rivaling forceusing Fj→i exerted by rivaling Aj to Ai at force is A schematic diagram multi-UAVs collisionThe avoidance strategy virtual time k is non-zero1 if the relative 2position and heading3 conditions are both hold. The relative position shown in Figure 7. Pi (k + 1|k), Pi (k + 1|k), and Pi (k + 1|k) are three candidate waypoints; θ 1 , θ 2 , condition imposes the requirement that Aj needs to be sufficiently close to Ai. The relative heading and θ 3 are the angles between the direction of the virtual rivaling force F i and the directions in the condition means that, if Aj exerts a rivaling force on Ai, Aj must be moving in the same or opposite candidate waypoints {Pi 1 (k + 1|k),The Pi 2overall (k + 1|k), andforce Pi 3 (k + 1|k)}. θ 1 , θupon 2 , and direction as Ai, approximately. rivaling exerted by allThe the angles other UAVs Ai θat3 satisfy 1 0 ≤ θ 1 0 and c toc 0.5Rc , then Vi,j,k c i,j,k ) < 0.5Rc , Vi,j,k > 0 C

70

60

50

40

30

20

10

0

0

100

200

300

400

500

600

700

0

100 200 300 400

500 600 /m

rc 2

Rc 2 Rc = 1000m rc = 800m

800

900

1000

Ai

Aj

Figure 10. The potential field VCi,j,k is generated to maintain the connectivity between Ai and Aj.

C is generated to maintain the connectivity between A and A . Figure 10. The potential field Vi,j,k i j

Therefore, the potential field that aims to restrict the movement of Ai inside the allowable position constrained set ξi, k is defined as

Therefore, the potential field that aims to restrict the movement of Ai inside the allowable position Vi ,Ck (c ) =  Vi ,Cj , k (c) constrained set ξ i, k is defined as (29) j∈ N ( k ) C C Vi,k (c) = ∑ Vi,j,k (cl) (29) l l i

Thus, if the UAV Ai selects the l-th candidate path, Pi (k) = {Pi (k + 1|k), …, Pi (k + q|k), …, j∈ Ni (k ) Pil(k + T|k)}, to follow, the cost of connectivity maintenance JD can be defined as

Thus, if the UAV Ai selects the l-th candidate path, Pi l (k) = {Pi l (k + 1|k), . . . , Pi l (k + q|k), . . . , T −1 C l J ( i , l , k ) = V ( P ( k + q | k) (30)  D i ,k i Pi l (k + T|k)}, to follow, the cost of connectivity maintenance JD can be defined as q =0 The total performance index function J(i,Tl,−k) 1 is the weighted sum of JA, JB, JC, and JD

∑

C

l

(i, l, k) = V k )(−Pλi (kJ + q k) J (i, l , k ) = λJD A J A (i , l , k ) + λB J B (i, l ,i,k C C (i, l , k ) − λD J D (i, l , k )

q =0

(31)

(30)

The total performance index function J(i, l, k) is the weighted sum of JA , JB , JC , and JD J (i, l, k) = λA JA (i, l, k) + λB JB (i, l, k) − λC JC (i, l, k) − λD JD (i, l, k)

(31)

Sensors 2018, 18, 1472

17 of 35

5. The Simulation Validation and Results Analysis The simulations are carried out in Microsoft Visual C++6.0 on a 2.4GHz, 2GB RAM laptop (Lenovo ThinkPad T420si, Beijing, China). 5.1. Effect of the Controllable Revisit Mechanism Based on Digital Pheromone In Scenario 1, there are four UAVs performing search and coverage mission over a 2 km × 2 km rectangular region. The surveillance region is uniformly divided into 50 × 50 cells of the same size. The size of each cell is 40 m × 40 m. Three targets are scattered in the surveillance region. Tables 1 and 2 list the initial settings of the UAVs and targets respectively. The communication range is Rc = 4000 m. Thus, the communication range is large enough to maintain the direct communication between every two vertices so that the movement of the UAVs can be not restricted by the network connectivity constraint. The sensing radius is Rs = 60 m. The detection and false alarm probabilities are pd = 0.9 and pf = 0.3, respectively. The time step in simulations is Ts = 0.1 s. Table 1. The initial settings of 4 UAVs in Scenario 1. UAVs Ai

Position (xi , yi )/m

Occupant Cell (m, n)

Heading ψi /(◦ )

A1 A2 A3 A4

(−620, −980) (−220, −980) (−180, −980) (580, −980)

(10, 1) (20, 1) (30, 1) (40, 1)

45 135 45 135

Table 2. The initial settings of 3 targets in Scenario 1. Targets Tj

Position (xj , yj )/m

Occupant Cell (m, n)

T1 T2 T3

(−620, −620) (−620, 580) (580, −620)

(10, 10) (10, 40) (40, 10)

In Scenario 1, we compared our proposed algorithm with the method of reference [16]. The essential difference of the two methods is that the controllable revisit mechanism based on digital pheromone is not taken account by the method of reference [16]. Thus, in order to verify the controllable revisit mechanism based on digital pheromone and enhance the capacities of target capture and region coverage for the UAVs, the following two groups of experiments are carried out:

• •

Group A: the controllable revisit mechanism is considered (the proposed method); Group B: the controllable revisit mechanism is not considered (the method of reference [16]).

5.1.1. Group A: With the Controllable Revisit Mechanism First, when A1 arrives at the cell (10, 10) and its sensor observations is “target detection”, so the pheromone releasing switch of the cell (10, 10) in A1 ’s cognitive map is turned on and the cell (10, 10) releases the pheromone to attract the A1 to revisit it. At t = 1.4 s, as shown in Figure 11, T1 presenting in the cell (10, 10) is confirmed by A1 . Then, A4 arrives at the cell (10, 40), and its sensor observation is “target detection”, so the cell (10, 40) releases the pheromone to attract the A4 to revisit it. At t = 7.7 s, as shown in Figure 12, T2 presenting in the cell (10, 40) is confirmed by A4 . At last, at t = 18.2 s, T3 in the cell (40, 10) is confirmed by A2 , as shown in Figure 13. Figure 14 shows the minimum distance between the UAVs. The collision distance is the length of the square cell, which is 40 m. The minimum distance is never lower than the collision distance; it means that any two UAVs are always not in the same cell; thus, the collision avoidance is guaranteed.

Sensors 2018, 18, 1472

18 of 35

y/m y/m y/m

Sensors 2018, 18, x FOR PEER REVIEW Sensors 2018, 18, x FOR PEER REVIEW 1000 Sensors 2018, 18, x FOR PEER REVIEW 1000 800 1000 800 600 800 600 400 600 400 200 400 200 0 200 0 -200 0 -200 -400 -200 -400 -600 -400 -600 -800 -600 -800 -1000 -800 -1000

18 of 36 18 of 36 18 of 36 T2 T2 T2

T1 T1 T1

T3 T3 T3

-1000

0 500 1000 A1 -500 A2 A3 A4 x/m -500 0 500 1000 A1 A2 A3 A4 -1000 x/m -1000 0 500 A1 -500 A4 A1000 Figure 11. t = 1.4 s, A1 found T1 inA2cellx/m (10,A10) (Group in Scenario 1). 3 -1000

y/m y/m y/m

Figure 11. t = 1.4 s, A1 found T1 in cell (10, 10) (Group A in Scenario 1). Figure 11. t = 1.4 s, A1 found T1 in cell (10, 10) (Group A in Scenario 1). Figure 11. 1000 t = 1.4 s, A1 found T1 in cell (10, 10) (Group A in Scenario 1). 1000 800 1000 800 600 800 600 400 600 400 200 400 200 0 200 0 -200 0 -200 -400 -200 -400 -600 -400 -600 -800 -600 -800 -1000 -800 -1000

T2 T2 T2

T1 T1 T1

T3 T3 T3

-1000

0 500 1000 A1 -500 A2 A3 A4 x/m -500 0 500 1000 A A A A -1000 1 2 x/m 3 4 -1000 500 A1 -500T2 inA2cell 0(10,A40) A4 A 1000 Figure 12. t = 7.7 s, A4 found (Group in Scenario 1). 3 x/m -1000

y/m y/m y/m

Figure 12. t = 7.7 s, A4 found T2 in cell (10, 40) (Group A in Scenario 1). Figure 12. 1000 t = 7.7 s, A found T in cell (10, 40) (Group A in Scenario 1). Figure 12. t = 7.7 s, A44 found T22in cell (10, 40) (Group A in Scenario 1). 1000 800 1000 800 600 800 600 400 600 400 200 400 200 0 200 0 -200 0 -200 -400 -200 -400 -600 -400 -600 -800 -600 -800 -1000 -800 -1000

T2 T2 T2

T1 T1 T1

T3 T3 T3

-1000

0 500 1000 A1 -500 A2 A3 A4 x/m -500 0 500 1000 A1 A2 A3 A4 -1000 x/m -1000 500 1000 A1 -500T3 in A2cell0(40, A310) (Group A4 Figure 13. t = 18.2 s, A2 found A in Scenario 1). -1000

x/m

Figure 13. t = 18.2 s, A2 found T3 in cell (40, 10) (Group A in Scenario 1). Figure 13. t = 18.2 s, A2 found T3 in cell (40, 10) (Group A in Scenario 1). Figure 13. t = 18.2 s, A2 found T3 in cell (40, 10) (Group A in Scenario 1).

Sensors 2018, 18, 1472

19 of 35

Sensors 2018, 18, x FOR PEER REVIEW

19 of 36

Sensors 2018, 18, x FOR PEER REVIEW

19 of 36

900

Minimum Collision Minimum Collision

Minimum distance between the UAVs /m Minimum distance between the UAVs /m

900 800 800 700 700 600 600 500 500 400 400 300 300 200 200 100 40 100 0 40 0 0 0

2

4

6

8

2

4

6

8

10

12

14

16

18

10

12

14

16

18

Time /s Time /s

Figure 14. Minimum distance between the UAVs (Group A in Scenario 1). Figure 14. Minimum distance between the UAVs (Group A in Scenario 1). Figure 14. Minimum distance between the UAVs (Group A in Scenario 1).

5.1.2. Group B: Without the Controllable Revisit Mechanism 5.1.2. Group B: B: Without 5.1.2. Group Without the the Controllable Controllable Revisit Revisit Mechanism Mechanism The snapshots of finding the targets T2, T3, T1 are shown in Figures 15–17, respectively. From The snapshots ofoffinding the targets T , T2,3 ,TT3,1 T areare shown in Figures 15–17, respectively. From From these snapshotswe finding the targets shown Figures 15–17, respectively. theseThe snapshots, can conclude that,2 Tdue to 1lacking the in controllable revisit mechanism, the snapshots, we can conclude that, due to lacking the controllable revisit mechanism, the capabilities of these snapshots, canforconclude that, in due to lacking the controllable revisit mechanism, the capabilities of the we UAVs target capture Group B are lower than Group A. Specifically, as shown the UAVs for target capture in Group B areinlower than Group A. Specifically, as shown in Figure 15, capabilities ofalthough the UAVs target capture B are Group A. as shown in Figure 15, Afor 1 travels through theGroup cell (10, 10)lower at thethan beginning of Specifically, the search process, A1 although A travels through the cell (10, 10) at the beginning of the search process, A does not revisit 1 1 in Figure 15, although A 1 travels through the cell (10, 10) at the beginning of the search process, A does not revisit the cell (10, 10) to confirm whether there is a target in this cell or not, due to lacking1 the cell 10) to confirm whether there iswhether a target in thisiscell or not, lacking controllable does not(10, revisit the cell (10, 10) to confirm there target indue thisto cell or not,the due to lacking the controllable revisit mechanism. Therefore, the target Ta1 is not captured (found and confirmed) revisit mechanism. Therefore, the target T is not captured (found and confirmed) early enough and is 1 the controllable revisit mechanism. Therefore, 1 is notincaptured (found and early enough and is confirmed by A 1 until t =the 33.2target s, as Tshown Figure 17. Figure 18confirmed) shows the confirmed by A until t = 33.2 s, as shown in Figure 17. Figure 18 shows the minimum distance between 1 early enough and isbetween confirmed by A1 in until t = 33.2 s, asbeshown in Figure 17. Figure 18 in shows the minimum distance the UAVs Group B. It can seen that two UAVs are never the same the UAVs in Group B. It can be seen that two UAVs are never in the same cell; thus, the collision minimum distance between the UAVs in Group B. It can be seen that two UAVs are never in the same cell; thus, the collision avoidance is guaranteed. avoidance is guaranteed. cell; thus, the collision avoidance is guaranteed. 1000

1000 800 800 600

T2

600 400

T2

y/m y/m

400 200 200 0 0 -200 -200 -400

T1

-400 -600 -600 -800 -800 -1000 -1000

T3

T1

-1000

T3

A1 -500

-1000

A1

A2

-500

A2

0 x/m 0 x/m

A3 A3

500 500

A4 A4

1000 1000

Figure 15. t = 6.9 s, A2 found T2 in cell (10, 40) (Group B in Scenario 1). Figure 15. tt == 6.9 6.9 s, s, A A2 found Figure 15. found T T2 in in cell cell (10, (10, 40) 40) (Group (Group BB in in Scenario Scenario 1). 1). 2

2

Sensors 2018, 18, 1472

20 of 35

Sensors 2018, 18, x FOR PEER REVIEW Sensors 2018, 18, x FOR PEER REVIEW

20 of 36 20 of 36

1000 1000 800 800 600

T2 T2

600 400 400 200 y/my/m

Sensors 2018, 18, x FOR PEER200 REVIEW

20 of 36

0

0 1000 -200 -200 -400

800

T2

600

-400 -600 400 -600 -800 200 y/m

-800 -1000

T3 T3

T1 T1

0

-1000 -1000 -200 -1000 -400

A1 -500 A1 -500

A2 A2

0 x/m 0 x/m

A3 A3

500

1000

500

1000

A4 A4

Figure 16. t = 30.2 s, A2 found T3 in cell (40, 10) (Group B in Scenario 1). T3 -600 s, A found T in cell (40, 10) (Group Figure 16. t = 30.2 B in Scenario 1). Figure 16. t = 30.2 s, A22 found T33 in cell (40, 10) (Group B in Scenario 1). T 1000

1

-800

1000 -1000 800

-1000

A1 -500

800 600

A2

0 x/m

A3

500

1000

A4

T T2

2 T3 in cell (40, 10) (Group B in Scenario 1). Figure 16. t = 30.2 s, A2 found 600 400 400 200

1000 800

y/my/m

200 0 600 0 -200 400

T2

-200 200 -400 y/m

-400 -600

T1 T1

0

-600 -200 -800 -400

-800 -1000 -600 -1000 -1000 -800-1000

T3 T3

T1

A1 -500 A1 -500

A2 A2

0 x/m 0 x/m

A3 A3

T 500 3

1000

500

1000

A4 A4

Figure 17. t =-1000 33.2 s,-1000 A1 found T1 in cell0 (10, 10)500(Group1000 B in Scenario 1). A1 -500 A2 A A4 x/m(10,3 10) (Group Figure 17. t = 33.2 s, A1 found T1 in cell B in Scenario 1). Figure 1200 17. t = 33.2 s, A1 found T1 in cell (10, 10) (Group B in Scenario 1). Figure 17. t = 33.2 s, A1 found T1 in cell (10, 10) (Group B in Scenario 1).

1200

Minimum Collision Minimum Collision Minimum

1200

Collision 1000 Minimum distance between the UAVs /m

Minimum distance between the UAVs /m /m Minimum distance between the UAVs

1000 1000 800 800

800

600 600

400 400

200

600

400

200

200 40

0 40 0 0 40 0 0 0 18. Figure

5

5

10

10

15

15

20 Time /s

20

25

30

25

35

30

35

Time /s

5 10 15 20 25 B in Scenario 30 Minimum distance between the UAVs (Group 1).35

Time the /s Figure 18. Minimum distance between UAVs (Group B in Scenario 1). Figure 18. Minimum distance between the UAVs (Group B in Scenario 1). From Figure the time the targets are between confirmed the UAVs in B Groups A and1).B, it can be 18.when Minimum distance thebyUAVs (Group in Scenario

concluded that the controllable revisit mechanism can concentrate the UAVs around the targets to

From time when the targets are confirmed byUAVs the UAVs in Groups A itand can be From thethe time when theastargets are confirmed the in Groups Athe and B, canB,beitconcluded capture them as soon possible, and enhance theby capacity of target capture for UAVs. From the time when the targets are confirmed by the UAVs in Groups A and B, it can be concluded that the controllable revisit mechanism can concentrate the UAVs around the targets to that the controllable revisit mechanism can concentrate the UAVs around the targets to capture them concluded thatasthe controllable revisit mechanism can concentrate the UAVsfor around the targets to capture them soon as possible, and enhance the capacity of target capture the UAVs. as soon as possible, and enhance the capacity of target capture for the UAVs. capture them as soon as possible, and enhance the capacity of target capture for the UAVs.

Sensors 2018, 18, 1472

21 of 35

Sensors 2018, 18, x FOR PEER REVIEW

21 of 36

Furthermore, revisitmechanism mechanismcan can enhance region Furthermore,ininorder orderto toverify verify that that the the controllable controllable revisit enhance thethe region coverage capacity of the UAVs and then improve the performance of mission operation, we analyze coverage capacity of the UAVs and then improve the performance of mission operation, we analyze and compare the global in Groups Groups A and compare the globalaverage averageregion regionrevisited revisitedrate rateand andthe theglobal global average average uncertainty uncertainty in and B, as shown in Figures 19 and 20, respectively. A and B, as shown in Figures 19 and 20, respectively. First, wewe defined the the following global average region revisited rate (“global” meansmeans it is averaged First, defined following global average region revisited rate (“global” it is over the N UAVs and the region capacitycapacity of the UAVs. averaged over the N (L UAVs (Lx × to Ly)evaluate cells) to evaluate thecoverage region coverage of the UAVs. x × Land y ) cells) ! 1 NN ϑ  !  N1 N  σ k 1=   σ i , k  ×100% =  1 i , kϑi,k×100% σk = × × 100% ε i , kε N σi =i,k 1  100% = NNi =1∑ N i∑ =1 i =1 i,k

(32) (32)

in which σ is the global average region revisited rate at time k. σi,k indicates the region revisited in which σk isk the global average region revisited rate at time k. σi,k indicates the region revisited rate of Ai at time k. ϑi,k indicates the number of the cells that are being revisited by Ai at time k. εi,k rate of Ai at time k. ϑi,k indicates the number of the cells that are being revisited by Ai at time k. εi,k indicates the number of the cells that need to be revisited at time k in the cognitive map of Ai. indicates the number of the cells that need to be revisited at time k in the cognitive map of Ai . Generally, Generally, ϑi,k ≤ εi,k, and if εi,k = 0, we set σi,k = 0. ϑi,k ≤ εi,k , and if εi,k = 0, we set σi,k = 0. Then, we defined the following global average uncertainty to evaluate the performance of Then,operation. we defined the following global average uncertainty to evaluate the performance of mission mission operation. N N 11 ηηk = = (33) (33) ,c ,k ∑ ηηii,c,k k N N((L L xx × ×LLyy)) i∑ i =1 c∈ Ω =1 c ∈ Ω



It It can bebe seen cells in in Group Group A can seenfrom fromFigure Figure1919that thatthe theglobal globalaverage averageregion region revisited revisited rate of the cells A is higher in Group B, due to considering controllable revisit mechanism GroupAAand is higher thanthan that that in Group B, due to considering thethe controllable revisit mechanism inin Group and lacking the controllable mechanism in Group B. In Group after 35 about 35target s, the existence target lacking the controllable revisit revisit mechanism in Group B. In Group A, afterA,about s, the existence status of most of the cells has been so confirmed, so that the averagerate revisited rate remains status of most of the cells has been confirmed, that the average revisited remains substantially substantially zero about 35 s.atHowever, at some moments = 32.2 the average revisited zero after about 35after s. However, some moments (e.g., t = (e.g., 32.2 ts), the s), average revisited rate is rate is approximately 100%. This is because the number of cells currently being revisited is approximately 100%. This is because the number of cells currently being revisited is approximately approximately equal to the number of cells that need to be revisited at these moments. Thus, we can equal to the number of cells that need to be revisited at these moments. Thus, we can conclude that the conclude that themechanism controllablecan revisit mechanism cancoverage enhancecapacity the region coverage controllable revisit enhance the region of the UAVs. capacity of the UAVs. It can be seen from Figure 20 that compared with Group B, the global average uncertainty in It can be seen from Figure 20 that compared with Group B, the global average uncertainty in Group A decreases to 0 more quickly. In the controllable revisit mechanism, we design the digital Group A decreases to 0 more quickly. In the controllable revisit mechanism, we design the digital pheromone as the “guidance information”, which is used to concentrate the UAVs to revisit sub pheromone as the “guidance information”, which is used to concentrate the UAVs to revisit sub mission area with high target probability or maximum uncertainty. In this way, the global average mission area with high target probability or maximum uncertainty. In this way, the global average detected rate of the cell increases accordingly. Therefore, according to Theorem 3, in Group A the detected rate of the cell increases accordingly. Therefore, according to Theorem 3, in Group A the convergence speed ofof the cognitive is ishigher convergence speed the cognitivemaps mapsininthe theUAVs UAVs higherthan thanininGroup GroupB,B,which whichmeans meansthat thatour method has better mission operation performance than the method in reference [16]. our method has better mission operation performance than the method in reference [16]. 100

The global average region revisited rate (%)

90 80 70

Group A: with the revisit mechanism (Our method) Group B: without the revisit mechanism (Ref.16)

60 50 40 30 20 10 0

0

5

10

15

20

25 Time /s

30

35

40

45

50

Figure 19. Comparison of the global average revisited rate (Scenario 1). Figure 19. Comparison of the global average revisited rate (Scenario 1).

Sensors 2018, 18, 1472

22 of 35

Sensors 2018, 18, x FOR PEER REVIEW

22 of 36

1 Group A: with the revisit mechanism (Our method) Group B: without the revisit mechanism (Ref.16)

Sensors 2018, 18, x FOR PEER REVIEW 0.9

22 of 36

The global average uncertainty

0.8

1

Group A: with the revisit mechanism (Our method) Group B: without the revisit mechanism (Ref.16)

0.7

0.9

0.6

The global average uncertainty

0.8

0.5

0.7

0.4

0.6 0.3

0.5 0.2

0.4

0.1

0.3

0

0.2

0

5

10

15

20

25 Time /s

30

35

40

45

50

0.1

Figure 20. Comparison of the global average uncertainty (Scenario 1).

Figure 20. Comparison of the global average uncertainty (Scenario 1). 0 0

5

10

15

20

25

30

35

5.2. Effect of Different Connectivity Maintenance Control Time /sStrategies

40

45

50

5.2. Effect of In Different Connectivity Maintenance Control Strategies Scenario 2, we set limited communication radius Rc = 1000 m and compared our proposed Figure 20. Comparison of the global average uncertainty (Scenario 1).

algorithm 2, with of communication reference [26]. Theradius essentialR difference of the two methods is that, In Scenario wethe setmethod limited c = 1000 m and compared our proposed inEffect our method, the Connectivity MST topology strategy is used to optimize the communication network topology, 5.2. of Different Maintenance Control Strategies algorithm with the method of reference [26]. The essential difference of the two methods is that, in our and only the communication links in the MST topology are maintained without violating the global method,connectivity the MST topology strategy iscommunication used to optimize the communication network topology, and only In Scenario 2, we set c = 1000 m and compared our proposed condition. Inlimited the method of reference radius [26], theRcommunication network topology is full algorithm with the means method of the reference [26]. essential difference of the two is that, the communication links in the MST topology areThe maintained without violating themethods global connectivity connected, which that links between each vehicle must be maintained during the mission in our method, the topology strategy used to optimize the communication network topology, condition. In the method of [26],isthe network topology isstrategies full connected, process. Thus, inMST order toreference test the influence of communication different connectivity maintenance control and only communication links ineach the MST topology are without violating theout: global on the the performance of mission operation, the following groups of experiments are carried which means that the links between vehicle musttwo bemaintained maintained during the mission process. connectivity condition. In the method of reference [26], the communication network topology is full Thus, in• order to A: test influence of different connectivity maintenance control strategies on the Group thethe MST topology strategy (the proposed method); connected, which means that the links between each vehicle must be maintained during the mission • Group B: the full connectedthe topology (the method of reference [26]). performance of mission operation, following two groups of experiments are carried out:

process. Thus, in order to test the influence of different connectivity maintenance control strategies

• •

on 5.2.1. the performance of mission operation, the following groups of experiments are carried out: Group A: the MST topology strategy proposed method); Group A: The Minimum Spanning(the Tree Topology two Group B: the connected topology (the ofmethod); reference • Group A: full the MST topology strategy (the method proposed The snapshots, Figures 21–23, respectively, show the search paths[26]). and communication topology • ofGroup B: the fullwhen connected topology method [26]). the whole UAVs the targets T1, T2(the , T3 are foundof in reference Group A. The communication topology of

5.2.1. Group A: The Minimum Treelines Topology the UAVs is denoted by theSpanning green dashed in these snapshots; it can be seen that the minimum 5.2.1. Group tree A: The Minimum Spanning Tree Topology spanning is used to optimize the topology of the communication network during the search

Theprocess, snapshots, Figures 21–23, respectively, show search and communication topology and only the communication links in the MST the topology arepaths maintained. The second smallest The UAVs snapshots, Figures 21–23, T respectively, show thein search paths and communication topology of the whole when the targets , T , T are found Group A. The communication topology of 2 communication 3 eigenvalue of the Laplacian matrix1of the topology is illustrated in Figure 24. It can of the whole UAVs when the targets T1, T2, T3 are found in Group A. The communication topology of the UAVs denoted by the greeneigenvalue dashed lines in these cannetwork be seenconnectivity that the minimum be is seen that second smallest is always largersnapshots; than zero, soit the is the UAVs is denoted by the green dashed lines in these snapshots; it can be seen that the minimum maintained duringtothe mission process. spanning tree is used optimize the topology of the communication network during the search spanning tree is used to optimize the topology of the communication network during the search process, and and onlyonly the the communication links topologyare are maintained. second smallest 1000 process, communication linksin inthe the MST MST topology maintained. TheThe second smallest eigenvalue of the Laplacian matrix ofofthe topology illustrated in Figure 800 eigenvalue of the Laplacian matrix thecommunication communication topology is is illustrated in Figure 24. It24. canIt can T2 600 be seen that second smallest eigenvalue is always larger than zero, so the network connectivity is be seen that second smallest eigenvalue is always larger than zero, so the network connectivity is 400 maintained during mission process. maintained during thethe mission process. 200 y/m

1000 0

800

-200

T2

600

-400 -600 -800

y/m

A3

A4 T3

T1

200

0 -1000

A2

A1

400

-1000

-500

0 x/m

-200

A

A

500

1000

A

4 3 10) (Group Figure 21.-400 t = 1.4 s, A1 found T1 in 2cell (10, A in Scenario 2).

A1

-600

-1000

T3

T1

-800 -1000

-500

0 x/m

500

1000

Figure 21. t = 1.4 s, A1 found T1 in cell (10, 10) (Group A in Scenario 2).

Figure 21. t = 1.4 s, A1 found T1 in cell (10, 10) (Group A in Scenario 2).

Sensors 2018, 18, 1472

23 of 35

Sensors 2018, 18, x FOR PEER REVIEW

23 of 36

Sensors 2018, 18, x FOR PEER REVIEW

23 of 36

Sensors 2018, 18, x FOR PEER REVIEW

23 of 36

1000 1000 800

1000 800 600

T2

800 600 400

T2

y/m y/m y/m

600 400 200

T2

A3 A3A2 A3 A

2

400 200 0 2000 -200

A1

0 -200 -400

A1

-200 -400 -600

A1

A2

-400 -600 -800

T1

A4

-600 -800 -1000

T1

A4

-800 -1000

-1000

T1

-1000

-500

T3

A4

-500

-1000

Figure 22. t = 8.9 -1000 s, A3 found T2 in -500

T3 T3

0 x/m 0 x/m cell 0(10, x/m

500

1000

500

1000

40) (Group A 1000 in Scenario 2). 500

Figure 22. t = 8.9 s, A3 found T2 in cell (10, 40) (Group A in Scenario 2). Figure 22. t = 8.9 s, A3 found T2 in cell (10, 40) (Group A in Scenario 2). 1000 Figure 22. t = 8.9 s, A3 found T2 in cell (10, 40) (Group A in Scenario 2). 1000 800 1000 800 600

T2

800 600 400

T2

y/m y/m y/m

600 400 200

A4

T2

400 200 0

A4

2000 -200

A4

0 -200 -400 -200 -400 -600

-600 -800 -1000 -800 -1000

T1

-1000

A3 T

A2

T1 -1000

AT1 3

A2

T1

-400 -600 -800

-500 -500

Figure 23.-1000 t = 19.4-1000 s, A3 found T3 in -500

0 x/m 0 cellx/m (40, 0 x/m

A1 A1

A2

A3T

500

A3

3

3

500

1000 1000

10) (Group A1000 in Scenario 2). 500

Figure 23. t = 19.4 s, A3 found T3 in cell (40, 10) (Group A in Scenario 2).

Figure 23. 23. t =t0.75 19.4 s,s,AA incell cell(40, (40,10) 10) (Group AScenario in Scenario 3 3found Figure = 19.4 found T T33 in (Group A in 2). 2). The second smallest eigenvalue The second smallest eigenvalue The second smallest eigenvalue

0.75 0.7 0.75 0.7 0.65 0.7 0.65 0.6 0.65 0.6 0.55 0.6 0.55 0.5 0.55 0.5 0.45 0.5 0.45 0.4 0.45 0.4 0.35 0.4 0 0.35 0 0.35The 24. 0

Figure

2

4

6

8

14

16

18

20

2

4

6

8

10 12 Time /s 10 12 Time /s second 2 4 smallest 6 8eigenvalue 10 12 Time /s

14

16

18

20

(Group 2). 14 16A in18Scenario 20

Figure 24. The second smallest eigenvalue (Group A in Scenario 2).

5.2.2. Group B: The Full Connected Topology Figure 24. The second smallest eigenvalue (Group A in Scenario 2). Figure The second smallest eigenvalue (Group A in Scenario 2). 5.2.2. Group B: The Full 24. Connected Topology The snapshots, Figures 25–27, respectively, show the search paths and communication topology 5.2.2. Group B: The Full Connected Topology of theThe whole UAVs when the25–27, targets T2, T1, T3 areshow found Grouppaths B. Asand shown in Figure 27,topology in order snapshots, Figures respectively, theinsearch communication 5.2.2.toGroup B: The FullFigures Connected Topology The snapshots, respectively, show thein search paths communication maintain aUAVs fully connected communication topology, the UAVs concentrated around left of the whole when the25–27, targets T2, T1, T3 are found Group B.are Asand shown in Figure 27,topology inthe order of the whole UAVs when the targets T 2 , T 1 , T 3 are found in Group B. As shown in Figure 27, in order half plane in the surveillance region. This causes the right half plane in the surveillance region that is to maintain a fully connected communication topology, the UAVs are concentrated around the left The snapshots, Figures 25–27, respectively, show the search paths and communication topology to maintain a fully connected communication topology, the UAVs are concentrated around the left not explored by the UAVs. The second smallest eigenvalue of the Laplacian matrix of the network half plane in the surveillance region. This causes the right half plane in the surveillance region that of the whole UAVs when the targets T2 , T1 , T3 are found in Group B. As shown in Figure 27, inisorder half plane in the region. Thissmallest causes the right halfofplane in the surveillance that is not explored by surveillance the UAVs. The second eigenvalue the Laplacian matrix ofregion the network to maintain a fully connected communication topology, the UAVs are concentrated around the left not explored by the UAVs. The second smallest eigenvalue of the Laplacian matrix of the network

half plane in the surveillance region. This causes the right half plane in the surveillance region that is not explored by the UAVs. The second smallest eigenvalue of the Laplacian matrix of the network

Sensors 2018, 18, 1472

24 of 35

Sensors 2018, 18, x FOR PEER REVIEW Sensors 2018, 18, x FOR PEER REVIEW Sensors 2018, 18, x FOR PEER REVIEW

24 of 36 24 of 36 24 of 36

y/m y/m y/m

topology can be be seen seen that that the the network network connectivity connectivity is topology is is illustrated illustrated in in Figure Figure 28. 28. It It can is maintained maintained during during topology is illustrated in Figure 28. It can be seen that the network connectivity is maintained during the mission process. the mission process. in Figure 28. It can be seen that the network connectivity is maintained during topology is illustrated the mission process. the mission process. 1000 1000 1000 800 800 800 600 600 600 400 400 400 200 200 2000 0 0 -200 -200 -200 -400 -400 -400 -600 -600 -600 -800 -800 -800 -1000 -1000 -1000

A4 A4 A4

A3 A3 A3

T2 T2 T2

A1 A1 A1

A2 A2 A2

T3 T3 T3

T1 T1 T1 -1000 -1000 -1000

-500 -500 -500

0 x/m 0 0 x/m x/m

500 500 500

1000 1000 1000

y/m y/m y/m

Figure 25. t = 9.2 s, A4 found T2 in cell (10, 40) (Group B in Scenario 2). Figure 25. 25. tt ==9.2 9.2s,s,AA44found foundTT2 2inincell cell(10, (10,40) 40)(Group (GroupBBininScenario Scenario2). 2). Figure Figure 25. t = 9.2 s, A4 found T2 in cell (10, 40) (Group B in Scenario 2). 1000 1000 1000 800 800 800 600 600 600 400 400 400 200 200 2000 0 0 -200 -200 -200 -400 -400 -400 -600 -600 -600 -800 -800 -800 -1000 -1000 -1000

T2 T2 T2

T1 A2 T T1 A2 1 A2 A1 A1 -1000 A1

-1000 -1000

A4 A4 A4 -500 -500 -500

T3 T3 T3

A3 A3 A3 0 x/m 0 0 x/m x/m

500 500 500

1000 1000 1000

y/m y/m y/m

Figure 26. t = 27.9, A2 found T1 in cell (10, 10) (Group B in Scenario 2). Figure 26. 1 in cell (10, 10) (Group B in Scenario 2). Figure 26. ttt ===27.9, 27.9,A A222found foundT cell(10, (10,10) 10)(Group (GroupBBininScenario Scenario2). 2). Figure 26. 27.9, A found TT11inincell 1000 1000 1000 800 800 800 600 600 600 400 400 400 200 200 2000 0 0 -200 -200 -200 -400 -400 -400 -600 -600 -600 -800 -800 -800 -1000 -1000 -1000

A1 A1 A1

-1000 -1000 -1000

-500 -500 -500

0 0 x/m 0 x/m x/m

A4 T 3 T A2 A A44 T3 A2 3 A2 500 500 500

A3 A3 A31000 1000 1000

Figure 27. t = 56.0 s, A4 found T3 in cell (40, 10) (Group B in Scenario 2). Figure 27. t = 56.0 s, A4 found T3 in cell (40, 10) (Group B in Scenario 2). Figure Figure 27. 27. tt == 56.0 56.0 s,s, A A44found foundTT33inincell cell(40, (40,10) 10)(Group (GroupBBininScenario Scenario2). 2).

Sensors 2018, 18, 1472

25 of 35

Sensors 2018, 18, x FOR PEER REVIEW

25 of 36

4

The second smallest eigenvalue

3.5

3

2.5

2

1.5

1

0

10

20

30 Time /s

40

50

60

Figure 28. The second smallest eigenvalue (Group B in Scenario 2).

Figure 28. The second smallest eigenvalue (Group B in Scenario 2).

Furthermore, in order verifythe theconnectivity connectivity maintenance scheme based on the topology Furthermore, in order toto verify maintenance scheme based onMST the MST topology optimization and efficaciously balance the coverage enhancement and the connectivity maintenance, optimization and efficaciously balance the coverage enhancement and the connectivity maintenance, and then improve the performance of mission operation, we analyze and compare the aggregated and then improve the performance of mission operation, we analyze and compare the aggregated coverage of the whole surveillance region and the global average uncertainty in Groups A and B, as coverage of the whole surveillance region and the global average uncertainty in Groups A and B, shown in Figures 29 and 30, respectively. as shown inScenario Figures 29PEER and 30, respectively. Sensors 18, x FOR REVIEW 26 of 36 In2018, 2, we defined the following aggregated coverage of the surveillance region for the Sensors 2018, 18, x FOR PEER REVIEW 26 of 36 UAVs to evaluate the region coverage capacity of the UAVs. 100

100 90

ϖk =

The The aggregated coverage (%) (%) aggregated coverage

80

The The global average uncertainty global average uncertainty

90 80

0.8 0.7

ωk Lx ⋅ Ly

× 100%

(34)

70 in which ϖ k denotes the aggregated coverage at time k. ωk indicates the aggregated number of the 70 60

cells that have been searched at least once by at least one UAV up to time k. 60 50 Then, we also use the global average uncertainty, which is defined in Equation (33), to evaluate 50 40 the performance of mission operation. 40 It can be seen that, in Group A, the aggregated coverage is higher than Group B (Figure 29), 30 and the convergence speed30 of the cognitive maps in the whole UAVs is higher than Group B 20 Group A: the better minimum spanning tree topology (Our method) (Figure 30), which means that20 our method has mission operation performance than the method 10 Group A: B: the the minimum full connected topology (Ref.26) (Our method) spanning tree topology in reference [26]. The reason10for theseGroup results is that the connectivity maintenance scheme based on Group B: the full connected topology (Ref.26) 0 0 10 20 30 40 50 60 70 80 90 100 the MST communication topology optimization relaxes the communication constraints and gives 0 Time /s 0 10 20 30 40 50 60 70 80 90 100 more freedom for the UAVs to search the more areas. However in Group B the UAVs tend to maintain Time /s Figure 29. Comparison of the aggregated coverage (Scenariomore 2). all the communication links with the other UAVs rather than exploring areas. Hence, the Figure 29.29. Comparison aggregatedcoverage coverage (Scenario Figure Comparisonof of the the aggregated (Scenario 2). 2). connectivity maintenance scheme based on the MST communication topology optimization can 1 efficaciously balance the coverage enhancement and thetreeconnectivity maintenance, which aim to Group A: the minimum spanning topology (Our method) 1 0.9 Group A: B: the the minimum full connected topology (Ref.26) (Our method) Group spanning tree topology maintain a connected topology for the network while minimizing the movement constraints on 0.9 Group B: the full connected topology (Ref.26) 0.8 the UAVs. 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0

0

10

20

30

40

0

10

20

30

40

50 Time /s 50 Time /s

60

70

80

90

100

60

70

80

90

100

Figure 30. Comparison of the global average uncertainty (Scenario 2). Figure 30. Comparison of the global average uncertainty (Scenario 2).

Figure 30. Comparison of the global average uncertainty (Scenario 2). 5.3. Effect of Varying Number of UAVs 5.3. Effect of Varying Number of UAVs In Scenario 3, defined we use different numberaggregated of UAVs to coverage test its influence on the average mission In Scenario 2, we the following of the surveillance region for the In Scenario 3, we use different number of UAVs test itsofinfluence average complete time (AMCT). In Monte-Carlo simulations, theto number targets is on M =the 3, while the mission number UAVscomplete to evaluate the region coverage capacity of thethe UAVs. of targets is M = 3, while the number simulations, of UAVs time is N (AMCT). = 5, 6 andIn7;Monte-Carlo thus, the simulations can benumber divided into 3 groups of experiments. The of UAVs of is experiments N = 5, 6 and was 7; thus, simulations can need be divided into 3 the groups of experiments. The number 100 the in each group. We to calculate AMCT of each group of number of experiments was 100 in each group. We need to calculate the AMCT of each group of experiments. The mission completion time is defined as the required mission execution time when experiments. The mission completion timeFor is defined as the required mission execution the global average uncertainty each experiment, the initial positions andtime the when initial η ≤ 0.01. the global average uncertainty ≤ 0.01. For each experiment, the initial positions and the initial η heading angles of the UAVs, and the initial positions of targets, are randomly generated. The

heading angles range of the isUAVs, andm. theThus, initial of targets, areisrandomly generated. The communication Rc = 4000 thepositions communication range large enough to maintain

Sensors 2018, 18, 1472

26 of 35

vk =

ωk × 100% L x · Ly

(34)

in which vk denotes the aggregated coverage at time k. ω k indicates the aggregated number of the cells that have been searched at least once by at least one UAV up to time k. Then, we also use the global average uncertainty, which is defined in Equation (33), to evaluate the performance of mission operation. It can be seen that, in Group A, the aggregated coverage is higher than Group B (Figure 29), and the convergence speed of the cognitive maps in the whole UAVs is higher than Group B (Figure 30), which means that our method has better mission operation performance than the method in reference [26]. The reason for these results is that the connectivity maintenance scheme based on the MST communication topology optimization relaxes the communication constraints and gives more freedom for the UAVs to search the more areas. However in Group B the UAVs tend to maintain all the communication links with the other UAVs rather than exploring more areas. Hence, the connectivity maintenance scheme based on the MST communication topology optimization can efficaciously balance the coverage enhancement and the connectivity maintenance, which aim to maintain a connected topology for the network while minimizing the movement constraints on the UAVs. 5.3. Effect of Varying Number of UAVs In Scenario 3, we use different number of UAVs to test its influence on the average mission complete time (AMCT). In Monte-Carlo simulations, the number of targets is M = 3, while the number of UAVs is N = 5, 6 and 7; thus, the simulations can be divided into 3 groups of experiments. The number of experiments was 100 in each group. We need to calculate the AMCT of each group of experiments. The mission completion time is defined as the required mission execution time when the global average uncertainty η ≤ 0.01. For each experiment, the initial positions and the initial heading angles of the UAVs, and the initial positions of targets, are randomly generated. The communication range is Rc = 4000 m. Thus, the communication range is large enough to maintain direct communication between each vertice so that the movement of the UAVs can be not restricted by the network connectivity constraint. The sensing radius is Rs = 60 m. The detection and false alarm probabilities are pd = 0.9 and pf = 0.3, respectively. Figure 31 shows the AMCT for different numbers of UAVs, and we can summarize that the larger the number of UAVs, the smaller the AMCT. This is because if the number of UAVs is larger, more cells in the surveillance region are detected by the UAVs, and hence the global average detection rate of the cell will be higher. According to Theorem 3, the convergence speed of the cognitive maps in the UAVs is higher, so the AMCT is smaller. Sensors 2018, 18, x FOR PEER REVIEW

27 of 36

40 35 30

AMCT /s

25 20 15 10 5 0

5

6 The number of UAVs N

7

Figure 31. Effect of varying number of UAVs on the AMCT (Scenario 3).

Figure 31. Effect of varying number of UAVs on the AMCT (Scenario 3). 5.4. Effect of Different Sensing Radius In Scenario 4, we set different values of sensing radius to test its influence on the AMCT. In Monte-Carlo simulations, the sensing radius is 20 m, 60 m, 100 m if the number of UAVs is kept as N = 5 and the number of targets is kept as M = 3. The communication range is Rc = 4000 m. The detection and false alarm probabilities are pd = 0.9 and pf = 0.3, respectively. In Scenario 4, the mission completion time is defined as the required mission execution time when the global average uncertainty η ≤ 0.1. For each experiment, the initial positions and the initial heading angles

5 0

Sensors 2018, 18, 1472

5

6 The number of UAVs N

7

Figure 31. Effect of varying number of UAVs on the AMCT (Scenario 3).

27 of 35

5.4. Effect Effect of of Different Different Sensing Sensing Radius Radius 5.4. In Scenario Scenario 4, 4, we we set set different different values In values of of sensing sensing radius radius to to test test its its influence influence on on the the AMCT. AMCT. In Monte-Carlo radius is 20 if the number of UAVs kept as In Monte-Carlo simulations, simulations,the thesensing sensing radius is m, 20 60 m,m, 60100 m, m 100 m if the number of isUAVs is N = 5 and the number of targets is kept as M = 3. The communication range is R c = 4000 m. kept as N = 5 and the number of targets is kept as M = 3. The communication range is Rc = 4000 m. The detection detection and and false false alarm alarm probabilities probabilities are are ppdd == 0.9 0.9 and and ppff == 0.3, 0.3, respectively. respectively. In The In Scenario Scenario 4, 4, the mission missioncompletion completiontime time is defined as required the required mission execution timethe when theaverage global the is defined as the mission execution time when global average uncertainty 0.1. For each experiment, thepositions initial positions theheading initial heading η ≤each uncertainty η ≤ 0.1. For experiment, the initial and theand initial angles angles of the of the UAVs, the positions initial positions of targets, are randomly generated. UAVs, and theand initial of targets, are randomly generated. AMCT for different different values values of of sensing sensing radius, radius, and and we we can can summarize summarize that that Figure 32 shows the AMCT the larger the sensing radius, the smaller the AMCT. This is because if the sensing radius radius is is larger, more cells cells in the surveillance region are detected by the UAVs, and hence hence the global average detected more UAVs, and rate of of the the cell cellwill willbe behigher. higher.According AccordingtotoTheorem Theorem3,3,the theconvergence convergence speed cognitive maps rate speed of of thethe cognitive maps in in the UAVs is higher, so the AMCT is smaller. the UAVs is higher, so the AMCT is smaller. 90 80 70

AMCT /s

60 50 40 30 20 10 0

20

60 The sensing radius Rs /m

100

Figure 32. 32. Effect Effect of of varying varying sensing sensing radius radius on on the the AMCT AMCT (Scenario (Scenario 4). 4). Figure

5.5. Effect of Detection and False Alarm Probabilities 5.5. Effect of Detection and False Alarm Probabilities In Scenario 5, we set different detection probabilities and different false alarm probabilities to In Scenario 5, we set different detection probabilities and different false alarm probabilities to test test their influence on the AMCT. In Monte-Carlo simulations, the detection probability pd is 0.6, 0.7 their influence on the AMCT. In Monte-Carlo simulations, the detection probability pd is 0.6, 0.7 and 0.9, and 0.9, while the false alarm probability pf is 0.2, 0.3 and 0.4. The number of targets is M = 3. while the false alarm probability pf is 0.2, 0.3 and 0.4. The number of targets is M = 3. The number of The number of UAVs is N = 5. The communication range is Rc = 4000 m. The sensing radius is UAVs is N = 5. The communication range is Rc = 4000 m. The sensing radius is Rs = 60 m. In Scenario 5, Rs = 60 m. In Scenario 5, the mission completion time is defined as the required mission execution the mission completion time is defined as the required mission execution time when the global average uncertainty η ≤ 0.01. For each experiment, the initial positions and the initial heading angles of the UAVs, and the initial positions of targets, are randomly generated. Figure 33 shows the AMCT for different detection probabilities and different false alarm probabilities, and we can summarize that, for a given detection probability pd , the smaller the false alarm probability pf , the smaller the AMCT. For a given false alarm probability pf , the larger the detection probability pd and the smaller the AMCT. According to Theorem 2, the better the performance of the sensor is, such as the larger the detection probability pd and the smaller the false alarm probability + or m− is smaller. In other words, the target pf , the minimum number of observations that require mavg avg existence status in each cell can be confirmed by fewer searches, and hence the AMCT is smaller.

probabilities, and we can summarize that, for a given detection probability pd, the smaller the false alarm probability pf, the smaller the AMCT. For a given false alarm probability pf, the larger the detection probability pd and the smaller the AMCT. According to Theorem 2, the better the performance of the sensor is, such as the larger the detection probability pd and the smaller the false + − or mavg is smaller. alarm probability pf, the minimum number of observations that require mavg Sensors 2018, 18, 1472 In other words, the target existence status in each cell can be confirmed by fewer searches, and hence 28 of 35

the AMCT is smaller. 75 pf = 0.2

70

pf = 0.3 pf = 0.4

65

AMCT /s)

60 55 50 45 40 35 30 0.6

0.7 The detection probability pd

0.9

Figure 33. Effect of various detection and false alarm probabilities on the AMCT (Scenario 5).

Figure 33. Effect of various detection and false alarm probabilities on the AMCT (Scenario 5). 5.6. Effect of Different Communication Range

5.6. Effect of In Different Communication Range Scenario 6, we set different communication ranges to test their influence on the AMCT. Monte-Carlo simulations, the communication range Rc is 800 m, 1200 m, 1500 and AMCT. In In Scenario 6, we set different communication ranges to m, test1000 their influence onmthe 1800 m, and keep the number of UAVs is N = 4; the number of targets as M = 3. The sensing radius is In Monte-Carlo simulations, the communication range Rc is 800 m, 1000 m, 1200 m, 1500 m and Rs = 60 m. The detection and false alarm probabilities are pd = 0.9 and pf = 0.3, respectively. For each 1800 m, experiment, and keep the of UAVs = 4; the number ofoftargets as M = the 3. The sensing radius is the number initial positions and is theNinitial heading angles the UAVs, and initial positions Rs = 60 m. The detection and false alarm probabilities are p = 0.9 and p = 0.3, respectively. For each of targets, are randomly generated. In Scenario 6, the mission completion d f time is defined as the required execution time the global average uncertainty experiment, the mission initial positions and when the initial heading angles of the UAVs, η ≤ 0.1.and the initial positions of Figure 34 shows the AMCT for different ranges, and time we canissummarize targets, are randomly generated. In Scenario 6, communication the mission completion defined asthat thethe required larger the communication range, the smaller the AMCT. This is because if the communication range mission execution time when the global average uncertainty η ≤ 0.1. is larger, the size of the connectivity constraint set εi,j,k is larger, and then the size of the allowable Figure 34 shows the AMCT for different communication ranges, and we can summarize that positions set ξi,k is larger, which gives more freedom for the UAVs without violating the network the larger the communication range, the smaller the AMCT. This is because if the communication connectivity constraint. In this case, the UAVs can search more areas, and which leads to the higher range isaverage larger,detected the size ofofthe εi,j,k is larger,speed and of then the size of the rate the connectivity cell. Accordingconstraint to Theorem set 3, the convergence the cognitive allowable positions set ξis is larger, which gives more freedom for the UAVs without violating the maps in the UAVs higher, so the AMCT is smaller. However, it is worth noting that when the i,k is large In enough, the AMCT is essentially unchanged. This is and because if theleads to networkcommunication connectivity range constraint. this case, the UAVs can search more areas, which communication range is large to maintain the direct communication links between every two of the the higher average detected rate enough of the cell. According to Theorem 3, the convergence speed vertices, the movements of the UAVs in the surveillance region are not bound by the communication cognitive maps in the UAVs is higher, so the AMCT is smaller. However, it is worth noting that range constraint. This means that the communication between UAVs can be seen as perfect, so that when the range is largerange enough, AMCT is essentially unchanged. This is because thecommunication influence of the communication can bethe ignored. if the communication range is large enough to maintain the direct communication links between every two vertices, the movements of the UAVs in the surveillance region are not bound by the communication range constraint. This means that the communication between UAVs can be seen as perfect, Sensors so that the influence of the communication range can be ignored. 2018, 18, x FOR PEER REVIEW 29 of 36 80 70 60

AMCT(s)

50 40 30 20 10 0

800

1000 1200 1500 The communication range Rc /m

1800

Figure 34. Effect of various communication ranges on the AMCT (Scenario 6).

Figure 34. Effect of various communication ranges on the AMCT (Scenario 6). It is also worth noting that, in Scenario 1, 3, 4 and 5, the communication range is large enough to maintain the direct communication between every two vertices so that each UAV can exchange the target probability maps with all the other UAVs. Thus, there is no deviation between each two individual target probability maps. However, in Scenario 6, due to limited communication range, there exits the deviation between each two individual target probability maps. Hence, in Scenario 6, on one hand, we use the global average uncertainty, which is defined in Equation (33), to evaluate

20 10 0

800

1000 1200 1500 The communication range Rc /m

Sensors 2018, 18, 1472

1800

29 of 35

Figure 34. Effect of various communication ranges on the AMCT (Scenario 6).

It is also worth noting that, in Scenario 1, 3, 4 and 5, the communication range is large enough It is also worth noting that, in Scenario 1, 3, 4 and 5, the communication range is large enough to maintain the direct communication between every two vertices so that each UAV can exchange to maintain the direct communication between every two vertices so that each UAV can exchange the the target probability maps with all the other UAVs. Thus, there is no deviation between each target probability maps with all the other UAVs. Thus, there is no deviation between each two two individual target probability maps. However, in Scenario 6, due to limited communication range, individual target probability maps. However, in Scenario 6, due to limited communication range, there exits the targetprobability probabilitymaps. maps. Hence, Scenario there exits thedeviation deviationbetween between each each two two individual individual target Hence, in in Scenario 6, 6, onon one hand, wewe use thethe global average one hand, use global averageuncertainty, uncertainty,which whichisisdefined definedininEquation Equation(33), (33),totoevaluate evaluatethe convergence performance of the target probability maps in the UAVs. On the other hand, the the convergence performance of the target probability maps in the UAVs. On the otherfollowing hand, global average uncertainty deviation is defined to evaluate consensus of the UAVs, the following global average uncertainty deviation is definedthe to evaluate theperformance consensus performance which implement the distributed and fusion scheme in fusion Equation (13) for merging. of the UAVs, which implementupdate the distributed update and scheme in map Equation (13) for map merging. N
1 ηi,c,k − η N ∆η k = (35) 1 ∑ ∑ k Δ η k = N ( L x × Ly ) i (35) ∈Ω η i , c , k − η k =1 c N ( L x × L y ) i =1 c∈ Ω

(

)

Figures 35 and 36 show the global average uncertainty and the global average uncertainty Figures 35 and 36 show the global average uncertainty and the global average uncertainty deviation for the different communication ranges, respectively. deviation for the different communication ranges, respectively. 1 Rc =800m

The global average uncertainty

0.9

Rc =1000m

0.8

Rc =1200m

0.7

Rc =1500m Rc =1800m

0.6 0.5 0.4 0.3 0.2 0.1 0

0

10

20

30

40

50 Time /s

60

70

80

90

100

Figure ThePEER global average uncertainty for the different communication ranges (Scenario 6). Sensors 2018, 18,35. x FOR REVIEW

Figure 35. The global average uncertainty for the different communication ranges (Scenario 6).30 of 36 0.5 Rc =800m

The global average uncertainty deviation

0.45

Rc =1000m

0.4

Rc =1200m

0.35

Rc =1500m Rc =1800m

0.3 0.25 0.2 0.15 0.1 0.05 0

0

10

20

30

40

50 Time /s

60

70

80

90

100

Figure Theglobal globalaverage averageuncertainty uncertainty deviation deviation for for the the different different communication communication ranges Figure 36. 36.The ranges (Scenario (Scenario 6). 6).

From Figure 35, it can be seen that the larger communication range, the faster the global average uncertainty decreases to 0. This is because the larger communication range gives more freedom for the UAVs without violating the network connectivity constraint. In this case, the UAVs can search more areas, which leads to the higher average detected rate of the cells. According to Theorem 3, the map convergence speed of the whole UAVs is higher, which means the performance of mission operation is better (this conclusion is also confirmed in Figure 34). From Figure 36, we can summarize that the larger the communication range, the faster the global

Sensors 2018, 18, 1472

30 of 35

From Figure 35, it can be seen that the larger communication range, the faster the global average uncertainty decreases to 0. This is because the larger communication range gives more freedom for the UAVs without violating the network connectivity constraint. In this case, the UAVs can search more areas, which leads to the higher average detected rate of the cells. According to Theorem 3, the map convergence speed of the whole UAVs is higher, which means the performance of mission operation is better (this conclusion is also confirmed in Figure 34). From Figure 36, we can summarize that the larger the communication range, the faster the global average deviation of the uncertainties decreases to 0, which means the consensus performance is better. This is because, the larger the communication range, the more communication neighbors of each UAV Ai . It means that more UAVs share their observation results with Ai . It is beneficial to eliminate deviations between the individual probability maps of the UAVs. 5.7. Comparison of the Two Map Update Methods To evaluate the effectiveness of our proposed distributed update and fusion scheme, two groups of experiments are carried out in Scenario 7.

• •

Group A: uncooperative map update. Each UAV only updates its own TPM according to its sensor observations. Group B: cooperative map merging. Each UAV first updates its own TPM according to its sensor observations, and then transmits the updated map to its neighbors for map fusion using our proposed update and fusion scheme in Equation (3).

We analyze and compare the global average uncertainty and the global average uncertainty deviation in Groups A and B, as shown in Figures 37 and 38, respectively. From Figure 37, it can be seen that the average uncertainty converges faster using our proposed distributed update and fusion scheme than by using the uncooperative map update method. This is because, if the neighboring UAVs exchange their current observations with Ai , Ai can get more observations each time than in the case without exchanging the observations, which increases the global average detected rates (mc ,k /(Nk)) over the covered cells. Specifically, in addition to the observations taken over the cells within its own sensing range, the UAV Ai can also get the observations taken over the cells outside its field of view (FoV) but inside the FoV of its neighbors. According to Theorem 3, the convergence speed of the cognitive maps in the UAVs is higher. The performance of mission operation with REVIEW a higher convergence speed. Sensors 2018, is 18,better x FOR PEER 31 of 36 1 Group A:Uncooperative map update Group B:Cooperative map merging

0.9

The global average uncertainty

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

10

20

30

40

50 Time /s

60

70

80

90

100

Figure Figure 37. 37. Comparison Comparisonof ofthe the global global average average uncertainties uncertainties (Scenario (Scenario 7). 7). 0.5

ty deviation

0.45 0.4 0.35

0.2 0.1 0

0

10

20

30

40

50 Time /s

60

70

80

90

100

Sensors 2018, 18, 1472

31 of 35

Figure 37. Comparison of the global average uncertainties (Scenario 7). 0.5

The global average uncertainty deviation

0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 Group A:Uncooperative map update Group B:Cooperative map merging

0.05 0

0

10

20

30

40

50 Time /s

60

70

80

90

100

Figure Comparison of the global average uncertainty deviations (Scenario Figure 38.38. Comparison of the global average uncertainty deviations (Scenario 7). 7).

From Figure 38, it can be seen that, implementing the distributed update and fusion scheme, From Figure 38, it can be seen that, implementing the distributed update and fusion scheme, the average uncertainty deviation can decrease faster to 0. This is because the deviation between the average uncertainty deviation can decrease faster to 0. This is because the deviation between individual probability maps can be eliminated by exchanging the TPM for map fusion. Eventually, individual probability maps can be eliminated by exchanging the TPM for map fusion. Eventually, all individual target probability maps can converge to the same one, which reflects the existence or all individual target probability maps can converge to the same one, which reflects the existence absence of the targets within each cell. However, in the uncooperative map update method, each or absence of the targets within each cell. However, in the uncooperative map update method, UAV only updates its own TPM according to its sensor observations. In this case, it is hard to each UAV only updates its own TPM according to its sensor observations. In this case, it is hard guarantee consensus among UAVs to maintain similar target probability maps and thus lead to to guarantee consensus among UAVs to maintain similar target probability maps and thus lead to mission performance degradations. mission performance degradations. 6. Conclusions 6. Conclusions This paper mainly studies cooperative search and coverage for a given bounded rectangle region This paper mainly studies cooperative search and coverage for a given bounded rectangle region by a team of UAVs with non-ideal sensors and limited communication ranges. The main contribution by a team of UAVs with non-ideal sensors and limited communication ranges. The main contribution of this paper is to develop a distributed cooperative search and coverage algorithm, which generates of this paper is to develop a distributed cooperative search and coverage algorithm, which generates paths to gather more information about the environment and find more targets. Following paths to gather more information about the environment and find more targets. Following conclusions conclusions can be obtained. can be obtained. 1. By integrating TPM, UM, and DPM, the cognitive map can effectively represent targets By integrating TPM, UM, and DPM, the cognitive map can effectively represent targets existence, 1. existence, uncertainty, and revisiting requirement of each cell in the surveillance region. Thus, uncertainty, and revisiting requirement of each cell in the surveillance region. Thus, the cognitive the cognitive map can serve as the UAV’s knowledge of the environment, effectively. map can serve as the UAV’s knowledge of the environment, effectively. Based on Bayesian rule and consensus theory, we design an update and fusion scheme of the 2. TPM. We prove that the designed update and fusion scheme can guarantee each one of the TPMs converges to the same one that reflects the true environment, and the convergence speed of the TMP is proportional to the average detected rate of the cells in the surveillance region. This conclusion can provide theoretical guidance for the controllable revisit mechanism. 3. A controllable revisit mechanism based on the digital pheromone is proposed to control the UAVs to revisit some important areas that have high target probabilities or have not been explored for a long time. The results of comparison simulations show that the controllable revisit mechanism could enhance the capacities of target capture and region coverage for the UAVs compared to the method that does not consider the controllable revisit mechanism. 4. In path planning process, the movement of UAVs is restricted by the potential fields to meet the requirements of avoiding collision and maintaining connectivity constraints. Moreover, using the minimum spanning tree (MST) topology optimization strategy, we can obtain the tradeoff between the search coverage enhancement and the connectivity maintenance. The results

Sensors 2018, 18, 1472

32 of 35

of comparison simulations show that removing the redundant communication links may relax the motion restriction of multi-UAVs and improve the efficiency of cooperative search and coverage operation. In future work, we will extend the cooperative search and coverage algorithm for moving targets and heterogeneous sensors. The impact of communication delays will also be considered, which will result in the deviation of individual cognitive maps. Author Contributions: Conceptualization, Z.L. and X.G.; Methodology, Z.L.; Software, Z.L.; Validation, Z.L., X.G. and X.F.; Formal Analysis, X.G.; Investigation, Z.L.; Resources, X.G.; Data Curation, Z.L.; Writing-Original Draft Preparation, Z.L.; Writing-Review & Editing, Z.L.; Visualization, Z.L.; Supervision, X.G.; Project Administration, X.G.; Funding Acquisition, X.G. Funding: The authors would like to express their acknowledgement for the support from the Fundamental Research Funds for the Central Universities under Grant No. 3102015ZY092. Conflicts of Interest: The authors declare no conflict of interest.

Appendix A. Proof of Theorem 1 Proof: Assume that, up to time k, the number of observations that have taken over cell c is mi,c,k , in which the number of Zi,c,k = 1 (i.e., “target detection”) is ai,c,k , the number of Zi,c,k = 0 (i.e., “non-target detection”) is (mi,c,k − ai,c,k ). From Equation (9), we obtain Qi,c,k = Qi,c,0 + ai,c,k ln

pf 1 − pf + (mi,c,k − ai,c,k ) ln pd 1 − pd

(A1)

If a target is present in cell c (i.e., ζ c = 1), Zi,c,k is a random variable based on the existence of targets, i.e., P(Zi,c,k = 1|ζ c = 1) = pd and P(Zi,c,k = 0|ζ c = 1) = 1 − pd . In addition, Zi,c,k for all k > 0 are independent and identically distributed (i.i.d.) random variables. Then, according to the law of large numbers, as mi,c,k → +∞, (ai,c,k /mi,c,k ) → pd . Thus, we get Qi,c,k a a Q p 1 − pf = i,c,0 + i,c,k ln f + (1 − i,c,k ) ln → mi,c,k mi,c,k mi,c,k pd mi,c,k 1 − pd

 pd ln

pf 1 − pf + (1 − pd ) ln pd 1 − pd



< 0 (A2)

Hence, Qi,c,k → −∞(i.e., pi,c,k → 1) as mi,c,k → +∞. In the same way, we can prove the conclusion for no target presenting in cell c (i.e., ζ c = 0). Appendix B. Proof of Theorem 3 Proof: Let γc,k = [Q1,c,k , . . . , QN,c,k ]T , V c,k = [v1,c,k , . . . , vN,c,k ]T , [W k ]i, j = ω i,j,k ; the distributed update and fusion rule in Equation (13) can be rewritten as γc,k = Wk (γc,k − 1 + Vc,k ) N

N

i =1

i =1

(A3)

Additionally, we define mc,k = ∑ mi,c,k and Qc,k = ∑ Qi,c,k . Assume that, up to time k, the number of observations that have taken over cell c by Ai is mi,c,k , in which the number of Zi,c,k = 1 (i.e., “target detection”) is ai,c,k , and the number of Zi,c,k = 0 (i.e., “non-target detection”) is (mi,c,k − ai,c,k ). Then, we get k

1− p

p

∑ υi,c,k = ai,c,k ln pdf + (mi,c,k − ai,c,k ) ln 1 − pdf

(A4)

l =1

Therefore, for the whole team of UAVs, we have N

Qc,k =

∑ Qi,c,k =

i =1

N

p

1− p

∑ Qi,c,0 + ac,k ln pdf + (mc,k − ac,k ) ln 1 − pdf

i =1

(A5)

Sensors 2018, 18, 1472

33 of 35

N

in which ac,k = ∑ ai,c,k is the total number of “target detection” observations obtained by all UAVs up i =1

to time k. According to Theorem 1, we conclude that, if a target is present in the cell c (i.e., ζ c = 1), as mc,k → +∞, Qc,k → −∞ and Qc,k p 1 − pf → pd ln f + (1 − pd ) ln mc,k pd 1 − pd

(A6)

Equation (A6) implies that, if each UAV can exchange the sensor observations with all the other UAVs, there is no deviation between each two individual target probability maps. All of the target probability maps converge to the same one, which reflects the targets’ true existence or absence in each cell of the search region. However, in our distributed update and fusion rule (Equation (13)), each UAV exchanges the target probability maps only with its neighboring UAVs. In this case, the deviation of the target probability maps exits. Next, we will prove that the deviation of the target probability maps is bounded by implementing the consensus protocol of Equation (13). At time k, the deviations between all the target probability maps γg,k and the global averaged probability map (((1/N)Qc,k )1) can be defined as ec,k = γc,k −

 1 T 1 Qc,k 1 = γc,k − 1 γc,k 1 N N

(A7)

in which 1 = [1] N×1 = [1, 1, . . . , 1]T . Therefore, according Equations (A3) and (A7), we have k

1 kec,k k = k ∏ Wt γc,0 − 1T N t =1

k

∏ Wt

t =1

!

k

k

1 γc,0 1 + ∑ ∏ Wt Vc,l − 1T N l =1 t = l

k

k

∑ ∏ Wt Vc,l

! 1k

(A8)

l =1 t = l

Due to 1T W k = 1T , W k 1 = 1 and [W k ]i,j ≥ 0 for all i and j, we get

kec,k k = k∆ 1, k + ∆2,k k ≤ k∆1,k k + k∆2,k k ∆1,k

k

 k 1 T = ∏ Wt γc,0 − 1 γc,0 1,∆2,k = ∑ N t =1 l =1

k

 1 T 1 V W V − ∏ t c,l N c,l 1 t=l

(A9) ! (A10)

According to the consensus theory, it can be easily proved that, given any (N × 1) vector θ and any (N × N) matrix W k that is associated with a connected topology G(k), we can find a number 0 < λ < 1 such that the following inequality holds.

k Wk θ −

1 T  1 T  1 θ 1k ≤ λ k θ − 1 θ 1k N N

(A11)

Therefore, we have

k∆2,k k ≤

k

∑

l =1

 1 T k∆ 1, k k ≤ λk kγc,0 − 1 γc,0 1k N !    k k  1 T 1 T k − l +1 k∏ Wt Vc,l − 1 Vc,l 1k ≤ ∑ λ kVc,l − 1 Vc,l 1k N N t=l l =1

(A12) (A13)

Substituting Equations (A12) and (A13) into Equation (A9), we get    k  1 T 1 T k − l +1 kec,k k ≤ λ kγc,0 − 1 γc,0 1k + ∑ λ kVc,l − 1 Vc,l 1k N N l =1 k

(A14)

According to Equation (9), it is easy to get  √ pf 1 T 1 − pf kVc,l − 1 Vc,l 1k < N ln − ln N pd 1 − pd

(A15)

Sensors 2018, 18, 1472

34 of 35

Substituting Equation (A15) into Equation (A14), we get !  √  pf λ (1 − λ k ) 1 − pf 1 T 1 γc,0 1k + N ln − ln kec,k k ≤ λ kγc,0 − N pd 1 − pd 1−λ k

(A16)

Thus, we have

√ ! pf 1 − pf λ N lim kec,k k = lim kec,k k < ln − ln mc,k →+∞ pd 1 − pd 1 − λ k →+∞

(A17)

Equation (A17) implies that the deviations between all the individual target probability maps and the globally averaged map are bounded. Thus, we have γc,k 1 → mc,k N



 Qc,k 1 mc,k

(A18)

Substituting Equation (A6) into Equation (A18), we get γc,k 1 → mc,k N



   Qc,k 1 − pf 1 p 1→ pd ln f + (1 − pd ) ln 1 mc,k N pd 1 − pd

(A19)

which implies   Qi,c,k 1 p 1 − pf → pd ln f + (1 − pd ) ln mc,k N pd 1 − pd

(A20)

In other words, if a target is present in the cell c (i.e., ζ c = 1), as mc,k → +∞, then Qi,c,k → −∞ (i.e., pi,c,k → 1) and (Qi,c,k /mc,kk ) → (pd /N)ln(pf /pd ) + ((1 − pd )/N)ln(1 − pf /1 − pd ). In the same way, we can prove the conclusion for no target presenting in cell c (i.e., ζ c = 0). References 1. 2. 3. 4.

5. 6. 7. 8. 9. 10. 11.

Fu, X.W.; Bi, H.Y.; Gao, X.G. Multi-UAVs cooperative localization algorithms with communication constraints. Math. Probl. Eng. 2017, 1943539. [CrossRef] Li, J.; Chen, J.; Wang, P.; Li, C. Sensor-oriented path planning for multiregion surveillance with a single lightweight UAV SAR. Sensors 2018, 18, 548. [CrossRef] [PubMed] Asif, K.; Evsen, Y.; Bernhard, R. Information exchange and decision making in micro aerial vehicle networks for cooperative search. IEEE Trans. Control Netw. Syst. 2015, 2, 335–347. [CrossRef] Lum, C.W.; Rysdy, R.T. Occupancy Based map searching using heterogeneous teams of autonomous vehicles. In Proceedings of the AIAA Guidance, Navigation, and Control Conference and Exhibit, Guidance, Navigation, and Control and Co-located Conferences, Keystone, CO, USA, 21–24 August 2006. Lum, C.W.; Vagners, J. A modular algorithm for exhaustive map searching using occupancy based maps. In Proceedings of the AIAA Infotech at Aerospace Conference and Exhibit, Seattle, WA, USA, 6–9 April 2009. Hata, A.Y.; Ramos, F.T.; Wolf, D.F. Monte carlo localization on Gaussian process occupancy maps for urban environments. IEEE Trans. Intell. Transp. Syst. 2017, 99, 1–10. [CrossRef] Asif, K.; Evsen, Y.; Bernhard, R. Information merging in multi-UAV cooperative search. In Proceedings of the IEEE Conference on Robotic & Automation (ICRA), Hong Kong, China, 31 May–7 June 2014. Zhang, M.; Song, J.; Huang, L.; Zhang, C. Distributed cooperative search with collision avoidance for a team of unmanned aerial vehicles using gradient optimization. J. Aerosp. Eng. 2016, 1, 1–11. [CrossRef] Kuhlman, M.J.; Otte, M.W.; Sofge, D.; Gupta, S.K. Multipass Target Search in Natural Environments. Sensors 2017, 17, 2514. [CrossRef] [PubMed] Lanillos, P.; Gan, S.K.; Besada-Portas, E.; Pajares, G.; Sukkarieh, S. Multi-UAV target search using decentralized gradient-based negotiation with expected observation. Inf. Sci. 2014, 282, 92–110. [CrossRef] Jin, Y.; Liao, Y.; Minai, A.A.; Polycarpou, M.M. Balancing search and target response in cooperative unmanned aerial vehicle (UAV) teams. IEEE Trans. Syst. Man Cybern. Part B Cybern. 2006, 36, 571–587. [CrossRef]

Sensors 2018, 18, 1472

12.

13. 14.

15. 16.

17. 18.

19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31.

32.

35 of 35

Galceran, E.; Campos, R.; Palomeras, N.; Ribas, D.; Carreras, M.; Ridao, P. Coverage path planning with real-time replanning and surface reconstruction for inspection of three-dimensional underwater structures using autonomous underwater vehicles. J. Field Robot. 2015, 7, 952–983. [CrossRef] Zhong, M.; Cassandras, C. Distributed coverage control and data collection with mobile sensor networks. IEEE Trans. Autom. Control 2011, 56, 2445–2455. [CrossRef] Millet, T.; Casbeer, D.; Mercker, T.; Bishop, J. Multi-agent Decentralized search of a probability map with communication constraints. In Proceedings of the AIAA Guidance, Navigation, and Control Conference, Toronto, ON, Canada, 19–22 August 2013. Yang, Y.L.; Polycarpou, M.; Minai, A.A. Muti-UAV cooperative search using an opportunistic learning method. J. Dyn. Syst. Meas. Control 2007, 129, 716–728. [CrossRef] Yang, Y.; Minai, A.A.; Polycarpou, M.M. Decentralized cooperative search by networked UAVs in an uncertain environment. In Proceedings of the American Control Conference, Boston, MA, USA, 30 June–2 July 2004. York, G.; Pack, D.J. Ground target detection using cooperative unmanned aerial systems. J. Intell. Robot. Syst. 2012, 65, 473–478. [CrossRef] Yang, F.; Ji, X.; Yang, C.; Li, J.; Li, B. Cooperative search of UAV swarm based on improved ant colony algorithm in uncertain environment. In Proceedings of the IEEE International Conference on Unmanned Systems, Beijing, China, 27–29 October 2017. Sujit, P.B.; Beard, R. Multiple UAV exploration of an unknown region. Ann. Math. Artif. Intell. 2008, 52, 335–366. [CrossRef] Farid, S.; Mostafa, M.; Zhang, Y.M.; Rabbath, C.A.; Zhang, Y.M. Cooperative multi-vehicle search and coverage problem in an uncertain environment. Unmanned Syst. 2015, 3, 35–47. [CrossRef] Pugliese, L.D.P.; Guerriero, F.; Zorbas, D.; Razafindralambo, T. Modelling the mobile target covering problem using flying drones. Opt. Lett. 2016, 5, 1021–1052. [CrossRef] Cortés, J.; Martínez, S.; Karatas, T.; Bullo, F. Coverage control for mobile sensing networks. IEEE Trans. Robot. Autom. 2004, 20, 243–255. [CrossRef] Panagou, D.; Stipanovi´c, D.M.; Voulgaris, P.G. Distributed dynamic coverage and avoidance control under anisotropic sensing. IEEE Trans. Control Netw. Syst. 2017, 4, 850–862. [CrossRef] Riehl, J.R.; Collins, G.E.; Hespanha, J.P. Cooperative search by UAV teams: A model predictive approach using dynamic graphs. IEEE Trans. Aerosp. Electron. Syst. 2011, 47, 2637–2656. [CrossRef] Di, B.; Zhou, R.; Duan, H.B. Potential field based receding horizon motion planning for centrality-aware multiple UAV cooperative surveillance. Aerosp. Sci. Technol. 2015, 46, 386–397. [CrossRef] Ru, C.J.; Qi, X.M.; Guan, X.N. Distributed cooperative search control method of multiple UAVs for moving target. Int. J. Aerosp. Eng. 2015, 2015, 317953. [CrossRef] Yang, P.; Freeman, R.A.; Gordon, G.J.; Lynch, K.M.; Srinivasa, S.S.; Sukthankar, R. Decentralized estimation and control of graph connectivity for mobile sensor networks. Automatica 2010, 46, 390–396. [CrossRef] Zavlanos, M.M.; Pappas, G.J. Potential Fields for Maintaining Connectivity of Mobile Networks. IEEE Trans. Robot. 2007, 23, 812–816. [CrossRef] Yang, Y.L. Cooperative Search by Uninhabited Air Vehicles in Dynamic Environment. Ph.D. Thesis, University of Cincinnati, Cincinnati, OH, USA, 2005. Ando, H.; Oasa, Y.; Suzuki, I.; Yamashita, M. Distributed memoryless point convergence algorithm for mobile robots with limited visibility. IEEE Trans. Robot. Autom. 1999, 15, 818–828. [CrossRef] Soleymani, T.; Garone, E.; Dorigo, M. Distributed constrained connectivity control for proximity networks based on a receding horizon scheme. In Proceedings of the American Control Conference, Chicago, IL, USA, 10–12 June 2015; pp. 1369–1374. Gallager, R.G.; Humblet, P.A.; Spira, P.M. A distributed algorithm for minimum-weight spanning trees. ACM Trans. Programm. Lang. Syst. 1983, 5, 66–77. [CrossRef] © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).