Probabilistic Double-Distance Algorithm of Search after Static or

2010 IEEE 26-th Convention of Electrical and Electronics Engineers in Israel

Probabilistic Double-Distance Algorithm of Search after Static or Moving Target by Autonomous Mobile Agent Eugene Kagan

Gal Goren

Irad Ben-Gal

Dept. Industrial Engineering Tel-Aviv University, Israel

Dept. Mechanical Engineering Tel-Aviv University, Israel

Dept. Industrial Engineering Tel-Aviv University, Israel

 Abstract—We propose a real-time algorithm of search and path planning after a static or a moving target in a discrete probability space. The search is conducted by an autonomous mobile agent that is given an initial probability distribution of the target's location, and at each search step obtains information regarding target's location in the agent's local neighborhood. The suggested algorithm implements a decision-making procedure of a probabilistic version of local search with estimated global distances and results in agent's path over the domain. The suggested algorithm finds efficiently both static and moving targets, as well as targets that change their movement patterns during the search. Additional information regarding the target locations, which is unknown at the beginning of the search, can be integrated in the search in real-time, as well. It is found that for the search after a static target, the algorithm actions depend on the global estimation at all stages of the search, while for the search after a moving target the global estimations mostly affect the initial search steps. Preliminary analysis shows that for the search after a static target the obtained average number of steps is close to optimal, while for the Markovian target the average number of steps is at least in the bounds that are provided by known search methods. Index terms—Search and screening, static and moving target, autonomous mobile agent

1. INTRODUCTION Consider a target moving in a bounded discrete domain and a mobile agent that searches the target. The agent starts with an initial probability distribution of the target's possible location over the domain. At each search step the agent obtains information regarding target's location in the agent's local neighborhood, and terminates the search when the target is located in this neighborhood. The target movements are governed by a definite Markov process that is known to the agent, while the target is not informed about the agent's behavior. The goal of the search agent is to find the target in minimum number of steps. In the presented formulation, the problem of search after static or mobile target was originated by Koopman [14] during the Second World War in response to the German submarine threat in the Atlantic [20]. The initial studies of the problem

978-1-4244-8682-3/10/$26.00 ©2010 IEEE

dealt with the probabilistic search for a static target or a target that moves relatively slow, while in the last decades the main interest is focused on search for targets with comparable velocity to that of the search agent, where the location probabilities of target’s change over time [5], [8], [21]. There are two main approaches of the considered search problem. Following the first approach, an optimal path of the agent is created off-line by use of global optimization methods. It is further assumed that the search effort [14] is infinitely divisible over the points of the domain and that the search period is defined and finite. The path-planning is based on considerations of optimal allocations that represent optimal distributions of search efforts over the domain with respect to the location probabilities and the search agent abilities [4]. For cases of search in continuous time and space the existence of optimal allocations has been proved, and optimal paths of the agent were obtained [19]. Regarding discrete time and space search, existence of optimal allocations were proven in the most general cases, while the algorithms that generate such allocations were obtained only for a search after a static target [1], [19]. For the search after a moving target, optimal algorithms have been built for the concave detection probabilities function [2] and for any detection probability function and some necessary (but not sufficient) conditions of optimality [22], [23]. In variants of the above problem, methods of partially observable Markov decision processes were applied and resulted in optimal allocations when the evaluation function is piecewise linear and convex [7] and when dynamic restrictions on search efforts are applied [18]. The second approach to the above problem addresses a realtime search, such that the path-planning is generated during the search process. Decision-making regarding each next step of the search agent is conducted on the basis of the obtained information, and after choosing the next step, the agent updates the available information about target’s locations. Such approach follows the line of A* algorithms and local optimization for the search over graphs or grid domains. The basic real-time A* algorithm results in a path of the search agent in the graph, and implements a search after a static target [15]. Later, a similar approach was applied in the moving-target search algorithm [9]. For the decision-making, both algorithms use local distances and heuristic distance

000160

estimations without a utilization of probabilistic information regarding target’s location. Further studies of the real-time algorithms and methods of learning lead to a number of different procedures, some of which implemented heuristic learning methods of target location probabilities [3], [17]. A certain progress in the unification of off-line and realtime search was obtained by the use of information theory methods and application of real-time search procedures with information metrics [10], [11]. The obtained algorithms used admissible informational distances both for local decisionmaking and as global distance estimations and demonstrated Huffman-like optimal and near-optimal results in the search after static and moving targets. The decision-making in these algorithms is conducted by the use of partitions of the considered domain on the basis of the target’s location probabilities. In further developments of these informational algorithms, a topology of the domain was specified [12] and an optimal algorithm of search after static target by a number of cooperative and non-cooperative searchers was found [13]. Although the significant advancement that has been achieved in the development of real-time search algorithms and the considerable progress in the studies of optimal allocating algorithms, no effective procedures for real-time allocating and on-line path-planning have been reported so far. The objective of this work is to develop and to analyze a real-time algorithm of search after static and moving targets that implements methods of allocating and of on-line pathplanning. The suggested algorithm is based on a probabilistic version of a local search with estimated global distances. Decision-making at each step of the search applies a probabilistic distance estimation that utilizes distance estimations and the characteristics of target's movement. The algorithm is applicable for search after static or moving targets and for the search after a target that changes its movement pattern during the search process. Additionally, obtained information regarding the target locations, which was unknown to the agent at the beginning of the search, can be taken into account in real-time, as well. 2. PROBLEM FORMULATION AND BACKGROUND Let  = { ,  , … ,  } be a set that represents a rectangle geographical domain of the size ( ,  ),  =    . Each point  ,  = 1,2, … , , represents a cell, in which the target can be located at any time moment = 0, 1, 2, …, and is called location. For simplicity, we index the points of X in linear order such that for the Cartesian coordinates (  ,  ) of the point  , its index is given by  = (   1) +  , where  = 1, … ,  and  = 1, … ,  . Assume that at each time moment t there is a defined probability mass function  :   [0, 1] that specifies the probabilities  ( ) = Pr{  =  },  = 1, 2, … , , of target's location in the points   of the set X, where    denotes the target's location at time moment t. Probabilities  ( ) are called location probabilities, and for any t it holds true that 0   ( )  1 and    ( ) = 1. The set X with the defined probability mass function  is the sample space.

The target's movement is governed by a discrete Markov process with transition probabilities matrix  =   × ,

where   = Pr  =  |  =   and    = 1 for each  = 1, 2, . . , . If matrix  is a unit matrix, then the target is static, otherwise, we say that it is moving. Given target location probabilities  (),  , at time moment t, location probabilities at the next time moment + 1 are calculated as  ( ) =     ( ) ,  = 1, 2, . . , . The search agent moves over the sample space and looks for a target by testing a neighboring observed area    of a certain radius  > 0. The ability of the agent to detect the target in the observed area  is defined by a detection function M that, for any observed area  , specifies a probability !( , ") of detecting the target in the area  given that the target is located in  ,    , and the agent applies on  a search effort N [14], [19]. In this paper, we assume that for any search effort "

[#0, $)#, the probability !( , ") = !( ) is defined as an indicator %(  ,  ) of the area   , i.e., !( ) = 1 if    and !( ) = 0 otherwise. In addition, we assume that for all time moments t the observed areas  have the same size. The search process is conducted as follows [10], [11]. The search agent deals with the estimated location probabilities & (  ), that for the initial time moment = 0 are equal to the initial location probabilities &' ( ) = ' ( ),  ,  = 1, 2, … , . At each time moment t, the agent chooses an observed area    and observes it. If an observation result in !( ) = 1, the search terminates. Otherwise, the agent updates estimated location probabilities &  ( ) over the sample space X according to the Bayes rule. The updated probabilities are called observed probabilities and are denoted by *  ( ). To the observed probabilities, the agent applies some known or estimated rule of the target movement and obtains the estimated location probabilities & ( ),  = 1, 2, … , , for the next time moment + 1. E.g., if the target is moving according to a Markov process with a transition probabilities matrix   × , then the estimated location probabilities &  ( ) for the next time moment + 1 are defined as & ( ) =    *  ( ),  = 1, 2, . . , . Given the estimated probabilities &  (),  , the search agent applies a decision making regarding the next observed area    to select and the process continues. The goal is to find a policy for choosing the areas    such that the search procedure terminates in a minimal average (over possible target moves) number of steps, while the centers of the areas  form a continuous trajectory over the sample space X with respect to a Manhattan metric. An existence of such policy in the most general form and the convergence of the search procedure are provided by the A*-type algorithms [9], [15] and their informational successors [10], [11], while the existence of optimal allocations are supported by Washburn theorems [23]. Here, we present a real-time algorithm that, as it follows from simulative results, creates the required trajectory of the agent.

000161

3. DOUBLE-DISTANCE SEARCH ALGORITHM As indicated above, the suggested algorithm is based on a probabilistic version of local search with estimated global distances, while decision-making utilizes distance estimations and the characteristics of target's movement. We assume that at each time moment t, both the search agent and the target can apply one of the following movements: - = {/ , / , /3 , /4 , /5 }, where / stands for “move north”, / - “move south”, /3 - “move east”, /4 “move west” and /5 - “stay in the current point”. These are the simplest movements that correspond to the Manhattan metric; for other metrics additional movements can be considered. The target’s choice of the movements is conducted according to its transition probabilities matrix U, while the agent’s choice follows the decision that is obtained according to the result !( ) of the observation of the area  around the agent. The outline of the suggested search algorithm includes the following steps. Given initial target location probabilities ' ( ),  = 1, 2, … , , and transition probabilities matrix U, 1. The search agent starts with estimated probabilities &' (  ) = ' ( ),  = 1, 2, … , , and an initial observed area ' . 2. If !(' ) = 1, i.e. the target is found and the process terminates. Otherwise, the following steps are conducted. 3. While !( ) 6 1, do 3.1. Calculate probabilities *  ( ),  = 1, 2, … , : For all  do: if   then set & ( ) = 0. For all  do: set *  (  ) = & ( )7  &  ( ). 3.2. Calculate probabilities &  ( ),  = 1, 2, … , : For all  do: set &  ( ) =    *  ( ). 3.3. Calculate weights 89 for the movements /9 -: For all /9 do: set 89 = ;