A Study of Terrain Coverage Models Ronaldo Menezes Computer Sciences Florida Tech Melbourne, Florida, USA
[email protected]
ABSTRACT The ability to cover a terrain has numerous applications to our everyday lives: from spell checking to Web crawling, from surveillance to automated vacuuming. Indeed, these applications require different approaches to solving the coverage problem because they are based on different ways of evaluating the goodness of the coverage; a good terrain coverage to vacuuming may not be desirable in Web crawling. To better understand the issues, this paper divides the terrain coverage problem into two classes (temporal and spatial) and discusses the performance of standard and self-organized algorithms as means of solving both classes of coverage problem. In order to perform the comparison the paper introduces suitable metrics to evaluate the goodness of a coverage (temporal or spatial).
Categories and Subject Descriptors I.2.11 [Artificial Intelligence]: Problem Solving, Control Methods, and Search —Plan execution, Formation, and Generation; I.2.9 [Artificial Intelligence]: Robotics—Workcell Organization and Planning; I.6.4 [Simulation and Modeling]: Model Validation and Analysis—Metrics
Harry Bullen Computer Sciences Florida Tech Melbourne, Florida, USA
[email protected]
that without a terrain coverage algorithm would cause you more inconvenience than give you peace of mind. Ergo it is not surprising that the study of terrain coverage is an active area of research. The term terrain coverage on its own does not convey well what task need to be performed; there are many variations of terrain coverage. The task becomes clear after a metric is defined, such as the minimization of unvisited cells in the terrain, the minimization of the time between visits to a particular cell, the pattern of the coverage (when using multiple agents), and others. One variation of the terrain coverage that has been neglected in the literature is what this paper names temporal terrain coverage which is characterized by the continuous covering of the terrain; meaning that the task is not to cover the terrain only once (all locations being visited). This may be necessary due the terrain itself being dynamic or constantly changing. At the time of writing this article, the authors were not aware of any solution dealing directly to what is called here temporal terrain coverage. In this paper we describe several algorithms for terrain coverage available in literature and compare them using well-defined proposed metrics. The results demonstrate the capability of a general purpose self-organized approach to terrain coverage to perform at comparable levels to the best deterministic algorithms.
Keywords Terrain Coverage, Autonomous Agents
1. INTRODUCTION One cannot deny that the world we live in has been and is continuously being shaped by wars, conflicts, battles, and struggles. One characteristic that appears to be important in any of these issues is the ability to know the battlefield. Lack of knowledge about the battlefield (a terrain) have, thorough history, made the difference between victory or defeat. This knowledge of terrains is important beyond battlefield scenarios. When discussing for instance the issue of target location, one can associate the problem of terrain coverage to any animal society that needs to locate food, water or other needed supplies. Even in our everyday lives the ability to cover a terrain is likely to important. For instance you may own an automatic vacuum cleaner
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC’08 March 16-20, 2008, Fortaleza, Cear´a, Brazil Copyright 2008 ACM 978-1-59593-753-7/08/0003 ...$5.00.
2.
TERRAIN COVERAGE
As pointed out by VanderHeide and Rao [7] there is quite a bit of confusion about the definition of terrain coverage since it relates to path planning and terrain model acquisition. In a nutshell, one can say that terrain coverage refers to the problem of covering as much surface area in a defined terrain as possible. This is different from path planning which aims at building an optimum path between points A and B according to pre-defined criteria. It is also different from terrain model acquisition because terrain coverage requires the agents to go over all of the terrain in a bounded area while terrain model acquisition requires the area to be reconstructed but may not require the agent to move in all of the terrain. In the context of this paper we focus on two aspects of terrain coverage: temporal and spatial. Although both have the purpose of achieving a good coverage of a terrain, “goodness” is seen differently. Temporal coverage relates mostly to the amount of time between cell visits—if we look at the time between visits for all the cells, we should not have much differences between them. This means that the standard deviation of the time between visits to each cell should be low. In Figure 1 we depict three distinct scenarios of temporal terrain coverage. Each row represents the evolution of visits to a cell in the terrain. Hence one entire column represents the status of all the cells in the terrain at a time t. Figure 1 therefore show the evolution of visits to a terrain over time. In the figures, we mark the maximum
1964
space (cells to be covered)
space (cells to be covered)
time (number of steps in the execution of the terrain coverage)
space (cells to be covered)
time (number of steps in the execution of the terrain coverage)
time (number of steps in the execution of the terrain coverage)
Figure 1: Above we have two different examples of temporal coverage. The arrow ( a cell remains unvisited. ). Note that in time a cell remains unvisited using an arrow ( Figures 1(top) cells remain unvisited for quite a long time while in Figure 1(bottom) the time is quite shorter—more desirable. Spatial coverage relates only to the number of times a cell is visited, independently of the time distribution of the visits. Using the examples in Figure 1 we can say that spatially, all three coverages are similar (one cannot be said to be better than the others). This is due to the fact that all cells are, at the end of the coverage execution, visited the same number of times (12 times). However, Figure 2 shows two other possibilities of spatial coverage differing significantly. the Figure 2(right) is better than the other (on the left) when it comes to spatial coverage. The reason is that in the example of Figure 2(right) all cells are visited 10 times at the end of the coverage. Contrast this with Figure 2(left), where some cells are visited 50 times while others are never visited. In this paper is to evaluate the best-known models for terrain coverage with regards to space and time; we demonstrate the power of a self-organized approach to coverage in relation to these standard approaches. In order to do so, we introduce in Section 4.1 a few metrics that enables a clear comparison of the models described in Section 3.
2.1
Applications of Terrain Coverage
Terrain coverage has many well-known applications beyond the obvious cover every position of a terrain. The term terrain coverage is commonly used whenever the objective is to interact with all positions/locations of a given terrain. We refer to this approach as spatial terrain coverage. There are many situations where spatial terrain coverage is useful, for example checking a document for spelling or grammatical errors would be a trivial example of spatial coverage. Other typical examples include vacuum cleaning, painting, and harvesting. These are all examples of spatial coverage because, to complete the task, each location (cell) has to be visited a fixed number of times independently of the periodicity of the visits. However, there are many other situations where it is important to have a location in the terrain visited repeatedly during a long period of time—we refer to this here as temporal coverage. Temporal coverage should be thought of as pertaining to situations where it would be desirable, but impractical, to have an agent at each location of the terrain in every time interval—hence the existing agents
) shows the maximum amount of time that
need to be moving constantly to maximize their value. The most common situation is surveillance; while it might be possible to cover a building with cameras, it would be very difficult to cover a city. In this situation autonomous robots could move through the city providing information about the location they visit. In this surveillance example, one could have the robots feeding a database with information about what is taking place in their location; the information could then be used to distribute the police force in the city. Note that the dynamism of the city requires robots to return to previously visited locations to re-check the status of that location. The temporal coverage in this case would aim at minimizing the time between re-visits so that areas are not left unvisited for a long period of time. Similar to surveillance would be exploration, perhaps of disaster areas such as the one that took place in New Orleans, USA with the hurricane Katrina (in 2005), where many small rovers would work to collect data. Given the dynamism of the area, rovers would be required to return to previously visited areas. Another interesting application of temporal coverage is crawling the Internet, where the agents (spiders) retrieve web pages for use elsewhere, typically searching. In this case, it is important for the search engine to have an up-to-date copy of every page. Some may think of vacuuming or sweeping an area as a type of temporal coverage. From our point of view this is not the case. In vacuuming or sweeping, floors do get dirty again, but the rate in which this happens does not make the task dynamic enough to require temporal coverage. In our terminology, vacuuming or sweeping would be more correctly categorized as a type of spatial terrain coverage. A room would need the spatial terrain coverage to be repeated at regular intervals but this should be confused with the constant temporal coverage needed in such situations as surveillance or exploration. One important concept to note is that in each of these cases, terrain coverage is simply a means to an end. As a result, what works well in one situation may not work as well in others. When exploring disaster areas, the success of the model would be measured by how much valuable data is retrieved. Conversely, in the surveillance situation, success would be measured by reduction in crime or increases in alleged criminals apprehended.
3. 1965
MODELS OF TERRAIN COVERAGE
space (cells to be covered)
space (cells to be covered)
time (number of steps in the execution of the terrain coverage)
time (number of steps in the execution of the terrain coverage)
Figure 2: Examples of distinct configurations of spatial coverages. In this evaluation study we have chosen models that are significant in the field of terrain coverage. The Alarm model is the only model that is proposed as a self-organized approached (inspired by ant’s alarm pheromone). Yet, many models have self-organized characteristics in their description (e.g. Recency). Whenever applicable, we discuss modifications that can be made to the other models to make them more self-organized. Some of the models that follow were proposed for specific situations of terrain coverage, but given their wide use in the literature we have included them here. There are many other important papers that were left out due to space limitations.
3.1
Node Counting
Node counting has been suggested as a simple, intuitive, and effective model. It is also used as starting point for more complex models. Here, each time an agent visits a cell it increases a counter that records the number of times the cell has been visited. The agents then deterministically move to the neighboring cell that has been visited the fewest times. This has been shown to quickly cover an area [6, 2]. This model can be modified to work probabilistically; agents can choose the next neighbor to move based on a roulette function where every neighbor carries a probability proportional to its counter. Another possible modification is to cause the count of visits in each state to “decay” over time. This makes the model have preference for states that have not been visited recently over ones that have. According to Thrun [6], this makes the model perform similarly to the recency model described in the next section. Counters can also be used to influence neighboring cells via a mechanism of dispersion of the counter.
3.2
Recency
Recency is a model where the decision of where to move is based on how recently the cells were visited. Note that this is like Node Counting but rather than spatial counting, the algorithm perform temporal counting. This can be implemented setting a value to cells that are visited. Over time this value decays at a fixed rate so the agents can tell how long ago a location was visited. This is very similar to pheromone-based models such as the Alarm model described in Section 3.5—the decaying of the value relates to pheromone evaporation. This model was used by Thrun [6] to test exploration in reinforcement learning and he found it performed well against node counting. In fact, he found that during the first pass it behaved the same as node counting. It is suited to be used for temporal coverage, because it focuses on how frequently an area is covered. And if all areas are covered equally often, they should also be covered an equal number of times, giving recency some of the same benefits as node counting.
3.3
Learning Real Time A*
The Learning Real Time A* (LRTA*) [3] is a derivative of the A* search designed for online searching. This model works like the A* search, by using a non-underestimating heuristic to speed up searching. Like A*, each location has a heuristic estimating the distance to the goal, however, the LRTA* search improves these heuristics by updating them with new information during the search. A heuristic can be updated by changing it to the value of the next location plus the cost to travel there if this gives a higher, and thus more accurate, value. LRTA* was adapted for use in terrain coverage by observing that when searching a terrain for a goal with an unknown location, the search agent would in the worst case have to cover the terrain. So we adapted LRTA* to perform terrain coverage, by simply removing the goal cell from the environment; the agents were allowed to run as normal. Koenig et al. [2] provide an excellent source of comparison between Node Counting and LRTA*. The models are compared under various conditions. However the main metric was time to first coverage—time taken to have all the cells in the terrain visited at least once. This metric is not used here because it has very restricted applications.
3.4
Online Mapping Algorithm
Thrun [6] has proposed a probabilistic model for online mapping (henceforth called OMA). The model can be considered a sort of a cross between LRTA* and node counting. It uses the LRTA* update rule, but always increases the value of the pheromone (count of visits) by at least one. This ensures that every time the cell is visited, it becomes less attractive to other agents. As such, it is expected that this model will have qualities of both LRTA* and node counting. The OMA presented here is an adaptation of the original proposed by Thrun. The reason is that the original model’s goal is mapping which impose requirements to the model (such as the robots knowing all other robots position in the terrain) that are not necessary in terrain coverage.
3.5
Alarm
The Alarm model [5] is inspired by ant-colony optimization[1], where ants search for an optimal solution by sending many agents through a terrain who lay down pheromone based on the quality of the solution they found. The other agents then chose their path based on the pheromones left behind on the terrain. This results in all the agents converging to one of the shortest paths. For the Alarm model the pheromone repels the agents in order to spread them on the terrain (get maximum coverage); this mimics the alarm pheromone behavior common in many species of ants [4]. The approach also assumes the possibility of food being present in the terrain. While pheromone repels the ants, the food attracts them to a cell. So every time a position is visited, its pheromone count is incremented and the food count decremented. The food
1966
value allows for some locations to be more important than others, which in real examples may be very important. For example, in a surveillance system the amount of food might be proportional to the crime rate, while in a web crawler it might be proportional to the site’s popularity. In our results we made no use of the food being differentiated per cell; each cell was simply seeded with the same amount food units (100 in all experiments). Note that the existence of two factors influencing the agents stochastic choice, allows the Alarm model to be more generic. If food is disregarded the models becomes similar to a stochastic version of Node Counting. If pheromone is disregarded, the model behaves similarly to the Recency model. True self-organization comes from the interplay of these two factors.
4.
EXPERIMENTAL RESULTS
We have implemented a simulation framework in NetLogo, a Java based derivative of logo specifically designed for multi-agent simulations [8]. NetLogo has a very simple language structure that is basically an extension of the Logo functional language with constructs to support agents and concurrency. In NetLogo, the simulation is made up of a grid of patches (cells) and a collection of agents. Furthermore, each patch and agent has its own memory containing its own set of variables. For this simulation, the edges of the world were allowed to wrap around, forming a torus. The world was divided into a grid of patches each storing the number of visits, and a counter (pheromone) of how long it had been unvisited and the time of the most recent visit. All the agents in our framework operated only on the local area of the patches. Furthermore, a large number of metrics are extracted about the agents’ activities including the unvisited time, the revisited, the total number of visits, the minimum distance from each agent to another agent and the minimum distance from each patch to an agent (these metrics are explained in Section 4.1). The results obtained were consistent for many different world sizes and number of agents. However, for consistency, in this paper, all results were only for worlds 51 by 51 cells in size populated by 48 ants initially distributed randomly across the grid. Figure 3 depicts the end of a run in an environment. The simulator has the ability to depict any of the counters (visits, pheromones, food, etc).
Figure 3: Execution environment for all terrain coverage algorithms. Left shows node counting values whereas the right shows pheromone levels. On the left the darker the shade the higher the counter. On the right, the lighter the shade the stronger the pheromone.
4.1
coverage. This is applicable to areas where the number of visits to each cell is important such as painting and harvesting. The goal is to have all the cells visited the same number of times.
Performance Evaluation
The first metric in our evaluation is the distribution of coverage, or the number of times each cell was visited—a metric of spatial
Figure 4: Results for spatial coverage. Graphs relate to the total number of visits in each cell. Figure 4 show the results for spatial coverage. One can clearly see from the results of the standard deviation of the number of visits (bottom) that both Alarm and Node Counting perform significantly better than the other models. More importantly, the result demonstrate the ability of a self-organized terrain coverage model (Alarm) to perform as well as a deterministic approach (Node Counting). From the same result we see that the LRTA* model does not do well in spatial terrain coverage, which is likely to be the result of LRTA* being designed for target location rather than coverage. Another metric related to spatial coverage is the distance of the agents to each other—a kind of spatial distribution of the agents in the terrain. If two or more agents are closely grouped together it is easier for them to visit a cell that has recently been visited by one of the other agents. Furthermore, because they are close together there must be some other location that is lacking agents, creating a blind spot. The metric used here is the distance of each agent to the nearest agent. So while there is no inherent requirement, is seems logical that in a good model agents would tend to stay away from each other. We have performed experiments (not shown in the paper) and both the Alarm and the Recency models performed better than the others. The primary metric used in temporal coverage is the time between visits and the length of time since each location has been visited. These two are closely related. First, the time since a location has been visited is called unvisited, and it is the main metric used in the Recency model. This measurement should be as low as possible indicating all locations are getting attention frequently. Also, the value of the cell with the longest (max) amount of time since it has been visited, tells valuable information about the uniformity of the agents. Figure 5 shows the performance of the models with regards to time since the cells have been visited (the amount of time a cell remains unvisited). The Recency model is clearly better than all the others. Recency is a model designed to solve this problem. All the other models
1967
Figure 5: Results describing the performance of the models for temporal coverage. Here the time since the last visit is depicted showing the efficiency of the Recency model.
perform at the same level with a slight advantage to Alarm and Node Counting over OMA and LTRA*. As argued before, Alarm can be made better by disregarding pheromones. This experiment is not displayed in the paper. The last metric we used to evaluate the models is the time between visits or revisits. In theory the revisits time should approximate the number of cells divided by the number of agents. This is slightly different from unvisits because it evaluates the ability of agents to return to previously visited cells—the sensitivity to cells that have been visited. Figure 6 confirms the superiority of the Recency model. In summary, our studies show that Node Counting and Alarm perform very well in spatial terrain coverage while Recency is the clear winner in terms of temporal coverage. Although Alarm can be perform better than the one described we shown only results where pheromone is considered. The diminishing of pheromone influence in the Alarm model is counter-intuitive since it was proposed to benefit from the use of pheromones.
5. CONCLUSION In this paper we described five models of terrain coverage and evaluated then according to three different metrics. Terrain coverage refers to the ability of agents to cover the whole of a terrain. Most models look at time to get first coverage (all cells visited at least once), or time to locate target in a terrain. However there are applications of terrain coverage in which one needs to look at spatial coverage (distribution of visits in cells in a fixed amount of time) and temporal coverage (the pattern of visits over time). Our results demonstrated that Alarm and Node Counting perform very well in spatial coverage while the Recency model is the best at temporal coverage. Other more specialized models (originally designed with the purpose of locating targets in terrains) such as OMA and LTRA* did not perform well in these settings. Our study should lead to further research on the design of models that are specifically designed to temporal coverage. Ideally, we
Figure 6: Time between visits. The Max shows the cell in all the simulator that has the longest time between visits. would like to extend the Alarm model to do as well as Recency because models such as Alarm (inspired by Ant Colony Optimization) are very scalable. Last, we needs to look at the sensitivity of these algorithms to the number of agents since this number has a clear relation scalability and robustness of these approaches. We are currently working on the extension of our simulation tool to take be able to evaluate this effect.
6.
REFERENCES
[1] M. Dorigo, V. Maniezzo, and A. Colorni. The ant system: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, 26(1):29–41, 1996. [2] S. Koenig and Y. Liu. Terrain coverage with ant robots: A simulation study. In AGENTS ’01: Proceedings of the fifth international conference on Autonomous agents, pages 600–607, New York, NY, USA, 2001. ACM Press. [3] R. Korf. Real-time heuristic search. Artificial Intelligence, 42(2-3):189–211, 1990. [4] C. Lloyd. The alarm pheromones of social insects: A review. Technical report, Colorado State University, 2003. [5] R. Menezes, F. Martins, F. E. Vieira, R. Silva, and M. Braga. A model for terrain coverage inspired by ant’s alarm pheronome. In SAC ’07: Proceedings of the 2007 ACM Symposium on Applied Computing, pages 728–732, New York, NY, USA, 2007. ACM Press. [6] S. Thrun. A probabilistic on-line mapping algorithm for teams of mobile robots. International Journal of Robotics Research, 20(5):335–363, 2001. [7] J. R. VanderHeide and N. S. V. Rao. Terrain coverage of an unknown room by an autonomous mobile robot. Technical report, Oak Ridge National Laboratory, Tennessee, United States, 1995. [8] U. Wilensky. Netlogo. Technical report, Center for Connected Learning and Computer-Based Modeling, Northwestern University, 1999. http://ccl.northwestern.edu/netlogo.
1968
