Decentralized Cooperative Auction for Multiple Agent Task Allocation Using Synchronized Random Number Generators D. Palmer, M. Kirschenbaum, J. Murton, K. Zajac Department of Math and Computer Science John Carroll University University Heights, OH 44118 USA
M. Kovacina, R. Vaidyanathan Orbital Research, Inc. Highland Heights, OH 44143, USA contact author:
[email protected]
Abstract − A collection of agents, faced with multiple tasks to perform, must effectively map agents to tasks in order to perform the tasks quickly with limited wasted resources. We propose a decentralized control algorithm based on synchronized random number generators to enact a cooperative task auction among the agents. The algorithm finds probabilistically reasonable solutions in a few rounds of bidding. Additionally, as the length of the auction increases, the expectation of a better solution increases. This algorithm is not intended to find the optimal solution; it finds a “good” solution with less computation and communication. 1. PROBLEM OF DYNAMIC TASK ALLOCATION For a collection of n homogeneous agents that must perform m interchangeable tasks at differing costs to each agent depending on initial conditions, there are centralized, polynomial time algorithms for computing optimal one-toone mappings when n equals m. For example, the Hungarian method [4], uses the simplex method to solve the symmetric n-agents-to-n-tasks problem in O(n3). To implement these strategies in a distributed fashion across a collection of agents requires prohibitive amounts of communication [7]. If n and m are allowed to vary independently and agents are allowed to perform multiple tasks (or no tasks), the complexity of finding the optimal solution is np-hard. Even the simplest case degenerates into the traveling salesman problem when one agent is scheduled to perform all the tasks. Yet, task assignment is a fundamental action of any autonomous, decentralized system, and therefore must be done, even under these computationally unfavorable conditions [1,2].
In particular, each agent must know its own task cost table. The auction seeks a reasonable solution in a reasonable amount of time. This approach exhibits several desirable features including: the cost is linear in the number of agents or tasks (whichever is larger), it generates usable solutions every iteration, it probabilistically produces improved solutions with increased computation time, and requires only a limited number of broadcast communications based on synchronized random number generators. The basic principle of CAST is a time-limited, probabilistic search of the full permutation space that yields good solutions with comparatively small amounts of computation and communication. The steps of this approach are as follows: 1. 2. 3.
4.
5.
2. CAST DECENTRALIZED AUCTION ALGORITHM 6. Our approach for solving the dynamic task-assignment problem is to enact an auction among the agents. The Cooperative Assignment of Simultaneous Tasks (CAST) auction is a distributed algorithm for matching agents with tasks using a cooperative bidding approach. Unlike systems that use Contract Nets i.e. MURDOCH, there is no competition among the agents during the task allocation process [3]. (Hence our non-standard use of the term "bidding", which actually means "selecting" here). An agent only bids on a task when it is its turn to do so. This cooperation among agents is the key ingredient for limiting agent communication during the auction. Each agent uses a greedy algorithm to choose its bid but typical disadvantages are avoided through randomization. The algorithm begins with each agent knowing information about itself and all possible tasks in the auction.
All agents compute their cost tables for the tasks independently, in parallel. Each agent computes an ordered choice list based on task cost. All agents have a pre-determined bid order based on synchronized pseudo-random number generators. Subsequent bid orders are globally known without any communication. The computed first bidder selects their first choice from their cost table and broadcasts their task selection and cost to all agents. Subsequent bidders select their best task from their remaining choices. This continues until all tasks have been selected. Each agent records the total cost. Repeat steps 3 and 4 with a new random bid order. If the total cost is more than previous mapping the new mapping is dropped, otherwise it supplants the existing mapping. The process repeats as time and resource constraints allow, with the lowest cost mapping retained.
The tractable version of this approach has each participant bidding only once creating a one-to-one mapping of agents to tasks. The full potential of this algorithm is realized when these restrictions are relaxed. This introduces the possibility of even lower cost solutions by allowing mappings in which one agent can perform as few as zero or as many as all the tasks. We introduce two techniques for allowing agents to either make multiple bids or remove themselves (either temporarily or permanently) from the bidding process for the overall good. How the "next" bidder is chosen and whether they handle multiple tasks is the core of the algorithm and highly dependent on the particulars of each
scenario. Factors going into this choice include characteristics of individual sorted cost tables, for example very small or very large gaps in the cost of adjacent options. These techniques are called: in-range re-bidding (for small gaps), and multi-skip gap bidding (for large gaps). To make the description of these techniques more intuitive, this paper focuses on a two-dimensional space populated with agents and fixed locations. (See figures 3a-3f). The tasks consist of visiting the fixed locations and the associated costs are proportional to the distance between the agent and the location. The in-range re-bidding technique allows an agent who has just bid, to immediately bid again. When an agent selects a location it also takes responsibility for all other nearby unvisited locations. If an agent selects location A, it computes a fixed percentage of the distance it needs to travel to get to A. If the cost from A to a another location, B is within the calculated percentage, then the agent will also visit location B. The agent keeps bidding until there are no unvisited locations within its range. The multi-skip gap technique helps agents determine when they should remove themselves (even temporarily) from the auction. Agents bidding towards the end of a round are often left with only very costly choices. In these instances, it is better for the overall solution to have these agents not bid at all. To implement this each agent sorts the costs in its cost table into ascending order and calculates the largest gap between consecutive costs. The difference between the best and worst costs (the entire cost range) is used to compute the ratio of the gap to the range. If it is that agent’s turn to bid and if its best remaining choice is on the undesirable side of the gap, the agent will skip a bid. This allows for other agents with better costs an opportunity to bid. To avoid perpetual bid skipping when no agents have any good choices left, agents only skip a few bids before being required to accept a costly location. The number of skipped bids for each agent is proportional to the computed ratio of its gap size to its total range. 3. BEHAVIOR EXPECTATION Due to the intractability of the general assignment problem, this paper first considers the simpler, one-to-one mapping scenario. This restriction makes it possible to compare the auction results with an optimal solution. (Note that the cost relationships between agents and tasks remain fixed throughout the task selection process). The effect of the CAST auction on three representative 10-agents-to-10-tasks scenarios is shown in Figures 1a, 1b and 1c. The graphs show the results for all possible mappings (exhaustive search) and all possible bidding orders. The results are a histogram that clearly shows the auction's beneficial effects. The best-case scenario for the CAST auction is when no two agents have the same first choice task. Under these conditions, any arbitrary bid order will produce the optimal solution. Figure 1a shows a scenario close to this situation, the graph shows the superior performance of the CAST auction. While this type of scenario is not expected, it is a case that CAST handles extremely well even for arbitrarily
large n. Optimal algorithms cannot take any shortcuts in this situation, so they incur the full computational cost to handle this case. The worst-case scenario for CAST is when the agents and tasks are arranged such that all the agents have exactly the same ordering for all their choices but the agents are not co-located. In this case, CAST provides no benefit over a randomly selected mapping. In fact, since their choices are all the same, the random bid order uniquely defines a random mapping from agents to tasks. Figure 1c shows a graph that is close in behavior to the worst case. Note that if the agents are not spread out, this worst case quickly turns into an ideal case. If all the agents are co-located, then every mapping of agents to tasks produces the same result -- the optimal solution. (Note: this also occurs if all the tasks are co-located). A typical case scenario has agents and tasks randomly distributed. In this scenario, we expect the CAST auction to provide a useful, realistic benefit. Figure 1b shows this behavior. What can be seen from Figures 1a and 1b is that essentially, by selecting a random bid order and then filtering that through a list of choices for each agent, the algorithm has the effect of taking the apparently normal distribution of random mappings and compressing them in the direction of the optimal solution. The amount of communication needed for the CAST auction is no more than what would be required for each agent to randomly select a target to go to and broadcast that information to all other agents. Due to the probabilistic nature of the algorithm, a bounded runtime for computing the optimal solution is not possible. However, a balance must be maintained between the desire for more rounds to produce a statistically good solution and the cost of additional communication. Finally, note that agents visiting locations in 2D space is simply a convenient way to visualize the auction. A more abstract conceptualization could be list processing. An example task could be removing duplicates whose cost depends on several components. If an agent already has a sorted copy of the list, then their cost to remove duplicates would be lower than agents’ without copies or with unsorted copies. 4. STATISTICAL JUSTIFICATION To evaluate the CAST auction as a viable method, we want to compare its results to the optimal solution. Unfortunately, that computation is np-hard when agents can handle multiple tasks. Instead we do extensive analysis for one-to-one mappings, which has an easily computable optimal solution, and consider several representative examples for the np-hard case. The availability of a fast solution allows semi-automated performance evaluation of numerous scenarios (1200 randomly generated scenarios of 10 agents and 10 tasks). For the more general solution, 25 sets of initial conditions were designed to represent the range of all possibilities. These initial conditions were studied and used to evaluate the effectiveness of CAST. The remainder of this section refers to the one-to-one CAST.
Ideal CAST Auction Behavior
300000 200000
700
Exhaustive Search Auction
120000 100000 80000 60000 40000 20000
100000
Frequency of Cost
Frequency of Cost
400000
140000
Frequency of Cost
Exhaustive Search Auction
500000
Pathological CAST Auction Behavior
Typical CAST Auction Behavior
600000
Exhaustive Search
600
Auction
500 400 300 200 100
Cost
Cost
530
516
502
488
474
460
446
432
418
404
390
376
362
348
320
334
0
743
710
677
644
611
578
545
512
479
446
413
380
347
314
826
785
744
703
662
621
580
539
498
457
416
375
334
293
252
211
170
123
0
281
237
0
Costs
Fig.1 a,b,c Comparison of CAST Dynamic, Cooperative Task Allocation and Exhaustive Search of Complete Solution Space
In our simulations, the auction demonstrated very good behavior most of the time. This is expected since we are dealing with a distributed, randomized algorithm. In this section we give precise meaning to the phrases “very good” and “most of the time”. For an intuitive impression consider Figure 2, which shows the complete results for 1,200 randomly generated 10 agents to 10 task auctions. For each initial condition, all 10! (3,628,800) possible bid orders were tried and compared against the optimal solution. The results indicate the number of bid orders that give good results. For example, of the 1,200 configurations, approximately 850 of them (70%) have a 50% probability of giving a mapping within 20% of optimal on the first round of bidding! Likewise, approximately 200 bid orders (~16%) have a 50% probability of getting within 10% of optimal on the first attempt. Obviously, as the number of rounds increase, the chances of randomly selecting a desirable bid order increases. The rest of this section presents a more rigorous analysis of this data. Suppose we have a cost table representing a particular agent to task assignment problem. Let pr = the proportion of all auctions that produce a cost within some percentage say r of the optimal cost. For example, suppose 75,000 bidding orders yields a cost within 10% of the optimal solution in a 10 agents to 10 tasks assignment problem. Since the total number of bidding orders is 10! = 3,628,800 we have pr = 75,000/3,628,800 ~ .2. The probability of an arbitrary auction having a cost at least r percent more than the optimal cost is (1- pr). Hence, the probability of performing n auctions where all of them have a cost at least r percent more than the optimal cost is (1- pr)n or the probability of performing n auctions where at least one of them is within r percent of the optimal cost is 1 - (1- pr)n. Therefore, if we run the auction n times and want to be within r percent of the optimal solution at least α percent of the time we must have: 1 - (1- pr)n > α ⇒ pr > 1 - (1-α) 1/n. Given the number of runs of an auction, n, and the desired probability α of obtaining at least one of the n auctions to be within r percent of the optimal solution, the above formula may be used to calculate the necessary proportion of successes required. i.e. for r = .10 and various values of n and α, Table 1 gives the necessary proportion pr of success. For example, if r = .10, n = 15 and suppose we know that 80% of all 10 agents to 10 tasks assignment cost tables have pr >= .2644, meaning at least 26.44% of the solutions are within 10% of the optimal solution, then using line 4 in the table we could say the following: For 80% of all 10
agent to 10 task assignment problems when the auction is run 15 times, its minimum cost will be within 10% of the optimal solution 99% of the time. Unfortunately, given an assignment problem, and a value for r, it isn’t feasible to determine what percent of all assignment problems have pr >= .2644. For a given value of r, our strategy is to estimate p, the percent of assignment problems with pr >= .2644, by drawing a sample of size 1,200 from all possible assignment problems and computing the proportion pˆ of success for r = .10, .15 and .20. For each value of r a confidence interval for the true proportion p of all 10 agents to 10 tasks assignment problems that have a success rate pr >= .2644 can be computed. A 99.9% confidence interval for p is pˆ ± 3.291 pˆ (1 − pˆ ) / 1200
. In our current example, it turns out that pˆ = .4208 yields a confidence interval of [.3739,.4677]. Instead of the 80% used above it can only be expected that 37.39% of all assignment problems have pr >= .2644. Therefore what can be said is that we are 99.9% confident that for at least 37.39% of all 10 agents to 10 tasks assignment problems, if we have 15 runs of the auction, then the minimum cost will be within 10% of the optimal solution 99% of the time. On the other hand, the values n = 20 and r = 20%, where success means pr >= .1088, provide a much stronger scenario for the auction. The data collected obtained .995 for pˆ producing the confidence interval [.9883,.1001]. Using line 9 in the table, we are 99.9% confident that the method used to produce the interval will capture the true proportion p, i.e. .9883
