crossings over them. The difficulty is that R1 has 2 bridges. (railroad and highway) but only 1 F-15 and R2 has 3 bridges. (railroad, highway and local road) and ...
Resource Coordination in Single Agent and Multiagent Systems Gifty Edwin and Michael T. Cox Department of Computer Science and Engineering College of Engineering & CS Wright State University Dayton, OH 45435-0001 {egifty;mcox}@cs.wright.edu
Abstract An intelligent multiagent system includes a number of software agents interacting to solve a problem. Various agents are responsible for planning different tasks. These tasks must be coordinated to solve complex problems. A number of reasons exist for which coordination among the agents is necessary, and numerous issues have to be tackled to achieve efficient coordination. The focus of this paper is resource coordination. An agent requests a resource from another agent when it does not have enough resources to construct a complete plan. We present a cost-benefit analysis that can be performed to determine if the requesting agent receives the requested resource. We present empirical results that demonstrate the relative goal satisfaction achieved in the PRODIGY planner with and without resource coordination. We have implemented a preliminary multiagent system called COMAS (CoOrdination in MultiAgent Systems) that implements resource coordination.
1: Introduction In multiagent systems (MAS) several autonomous agents interact to solve complex tasks. Each MAS agent has its own goal. These goals must be coordinated to achieve better and more efficient overall plans. Another important characteristic of MAS is that the environment is no longer static. As opposed to single agent systems, the dynamics of the environment are affected by the actions of other agents in that environment. An example domain where MAS could be used is a system planning in a military scenario. Different agents plan for different goals such as deploying units, destroying enemy units and securing rivers. A number of reasons exist for using MAS. The first and main reason for using MAS is that it helps us understand teamwork and social behavior in a human organization. Secondly, a planner with multiple goals needs the information of the resources available and the states associated with each of its goals. Since there is a deluge of
information, handling it becomes difficult. The third reason is that due to inherent parallelism in MAS, the time required for a multiagent system to come up with a solution maybe less than the time required by a single planner with the same goal. There is considerable delay in order to achieve efficient coordination. Scalability, robustness and nature of the domains are some other reasons for using MAS. Our research concentrates on two types of coordination, namely resource coordination and goal interaction coordination [4]. Resource coordination is concerned with the intelligent allocation of resources among the different agents and goals. An agent that is planning may find that it cannot achieve one of its goals because of a lack of a resource. The problem faced here is how to determine if an agent needs a resource. If so, where does it get that resource? This agent can request the resource from another agent or reassign a resource from one of its other goals, if there are any. An agent can also get resources from the environment if new resources become available. Another problem is, what happens if the resource request is not granted? We propose that if a resource request is not granted, the goal is minimally altered. For example, if the city cannot be captured, then capture the local airbase. Intelligent resource allocation among the agents in multiagent systems results in better and efficient overall plans for the individual agents. This is mainly because of higher goal satisfaction that is achieved with resource coordination. Resource coordination is required not only in multiagent systems, single agent systems with multiple goals need resource coordination. In this case, a resource is reassigned from one of its other goals to the goal that needs a resource. Section 2 describes the resource coordination. It describes single agent resource contention and multiagent resource contention. It also describes a cost-benefit analysis that can be done to determine if a resource request is to be satisfied. Section 3 describes the implementation of a preliminary system called COMAS [4]. Section 4 describes the empirical results of COMAS planner with and without resource coordination. Finally, section 5 is the conclusion.
2: Resource Coordination While planning, an agent may find that it cannot achieve a goal because of a lack of resources. To handle such a situation, an agent has at least two competing options over plan failure. First, this agent can request a resource from another agent or transfer a resource in service of another goal if any exist. It could also get a resource from the environment, if new resources become available. If the agent gets the resource, it can achieve the originally intended goal. Second, the agent may change its goal to a similar one that is achievable with the resources it already has. The main reason for resource coordination, as mentioned earlier, is to achieve better and more efficient overall plans. Resource coordination is required not only in multiagent systems. Single agent systems with multiple goals also need resource coordination (i. e. reallocation). This section is divided into 4 subdivisions. In the first subdivision, the concept of a goal transformation and the necessity for it is discussed. The second subdivision details the cost-benefit analysis that is used to determine the resource allocation. The third subdivision gives an example of a resource contention problem and describes how efficient resource allotment can result in better plans. The fourth subdivision discusses resource coordination as achieved in a single agent system. The final subdivision deals with resource allocation in a multiagent system.
2.1: Goal Transformations
“river-impassable” to the goal “restricts-movement-acrossriver”.
2.2: Resource Contention Problem Consider the following problem (Bridges Problem) from the air campaign-planning domain taken from [3]. The mission is to make two rivers, R1 and R2, impassable by removing the crossings over them. The difficulty is that R1 has 2 bridges (railroad and highway) but only 1 F-15 and R2 has 3 bridges (railroad, highway and local road) and 2 F-15s. Figure 1 shows the situation. Thus, 3 F-15s constitute insufficient resources to achieve the overall goals. This is insufficient due to the following assumptions. First, an F-15 fighter can destroy only 1 bridge and damage any number of additional bridges. Second, a destroyed bridge has 100% reduction in transportation over it, whereas, a damaged bridge has φ% reduction in transportation over it. In this example, we consider the following question. If an extra F-15 becomes available, where should it be assigned? Assuming φ is 50% and goal satisfaction is defined as the total reduction in transportation capacity over the whole river. If the F-15 is assigned to R1, we get 100% (both the bridges across the R1 are destroyed) goal satisfaction for R1 and 83% (2 bridges are destroyed and 1 bridge is damaged, 250/300 *100) goal satisfaction for R2. If it is assigned to R2, we get 75% goal satisfaction for R1 and 100% goal satisfaction for R2. The assignment of the extra F-15 to R1 is therefore preferable because the overall goal satisfaction is optimized.
In a dynamic environment, the world changes during planning either by the actions of agents or by exogenous events. Therefore, goals may become obsolete. For example, a goal of securing city1 may become obsolete, if all enemy units move to city2. The planner, being aware of the world changes, minimally alters its goal to adapt to the change. That is the goal is changed to securing city2. A goal transformation [3] is defined as a movement of a goal in a goal space and implements the concept of goal change. The two reasons for using goal transformations are as follows. First, the planner may find that it does not have enough resources to achieve its goal. This use of a goal transformation is relevant to resource coordination. Second, the planner may sense changes in the environment that dictate an adjustment in the planning process. The relevance of this second type becomes clear in the next section. The advantage of using goal transformations in planning is obvious when no plan is possible for the original goal. In this case, a minimal goal shift is clearly warranted. An example of a goal transformation is an operationalization. Replace a vague goal of making a river impassable with concrete goals to destroy each bridge across it. An example of an erosion transformation would be to change the goal predicate from
Figure 1: Resource contention problem.
2.3: Cost – Benefit Analysis The resource contention problem examines the general problem of where to assign such extra resources, if they become available. In this subsection we consider an agent that requests a resource from another agent rather than the case where the resource becomes available exogenously. We present a simple cost-benefit analysis that can be performed to determine whether the resource transfer can be made, or the requesting agent changes its goal, because it does not have enough resources. There are ‘N’ number of agents, each agent making a request for a resource from N-1 other agents. The Cost-Benefit analysis from the point of view of requesting agent ‘A’ is stated as • The requesting agent’s Goal Benefit is the percentage ratio of improvement in performance to performance with resource. The improvement in performance is the difference between performance with resource and without resource. • Requesting agent’s Goal Cost is value of the loss of the requested Agent (or the loss incurred with respect to another goal when reassigning its own resources). • If ((Goal Benefit minus Goal Cost) is greater than 0), then grant resource request (or make the reassignment), if not, then perform transformation (e.g., goal erosion) on the requesting goal. We call the benefit minus cost value Utility. The utility value is calculated using a simple formula. Given: • φ = transformation factor (a measure of goal satisfaction in a partially achieved goal). It is assigned a value between 1 and 1001. • nfA = number of fully achieved goals of the requesting agent • npA = number of partially achieved goals of the requesting agent. • PA = priority assigned to the requesting agent’s goals.
Given: • nfi = number of fully achieved goals of the requested agents • npi = number of partially achieved goals of the requested agents. • 0 < i < number of agents, i A. • Pi = priority assigned to ith requested agent. = min ∪i = 1..n, i A Pi * {(100 nfi + φnpi) – [100(nfi -1)+ φ(npi + 1)]} * 100 100nfi + φnpi Assuming all the priorities are equal and by using simple mathematics this equation can be reduced to 100 - φ 100nfA + φnpA which is the minimum difference between the goal satisfaction if the resource request is not granted and the goal satisfaction possible if the resource request was granted. In the Bridges Problem, assume river R1 has 1 resource and R2 has 3 resources. If R1 makes a request for a resource, the cost-benefit analysis is done as follows, assuming the priorities are 1. nfA = 1, npA = 1, nfi = 3, npA = 0, φ = 50, PA = 1, P1 = 1. Therefore, Goal Benefit = [100 * (1 + 1) + (50 * 0)] – [100 * 1 + 50 * 1] / [100 * (1 + 1) + (50 * 0)] * 100 = (200 –150) / 200 * 100 = 25 Goal Cost
Goal Benefit = PA *{[(100(nfA + 1) + φ (npA– 1)] – (100nfA +φ npA)}*100 100(nfA + 1) +φ(npA – 1) Assuming all the priorities are equal and by using simple mathematics this equation can be reduced to 100 - φ 100nfA + φnpA + 100 - φ
Utility
= [(100 * 3) + (50 * 0)] – [(100 * 2) + (50 * 1)] / [(100 * 3) + (50 * 0)] *100 = (300 – 250) /300 * 100 = 16.66 = 25 – 16.66 = 8.33
The previous section was about multiagent choices. The next section examines the choices in a single agent system.
2.4: Single Agent Resource Contention We have solved the Bridges problem using a single copy of the PRODIGY planning and learning architecture [1][6] agent 1
50 in the Bridges Problem because of the assumption that damaged bridges are reduced in transportation capacity by 50 %.
whose goals are to make R1 and R2 impassable. PRODIGY was designed and implemented at Carnegie Mellon University. It employs a nonlinear state-space planner and follows a means-ends analysis backward-chaining search procedure that reasons about both multiple goals and multiple alternative operators from its domain theory appropriate for achieving such goals. A domain theory is composed of a hierarchy of object classes and a suite of operators and inference rules that change the state of the objects. A planning problem is represented by an initial state (objects and propositions about the objects) and a set of goal expressions to achieve. During search for a solution the plans considered are called incomplete plans. In PRODIGY, an incomplete plan consists of two parts, the head-plan and the tail-plan. The head-plan is achieved by applying a sequence of operators to the initial state. The tail-plan is built by a partial-order backward-chaining algorithm, which starts with a goal statement and adds operators, one by one to achieve preconditions of other operators, that are not true in the current state. an example of backward chaining is that to deploy an air unit to a forward base, base must be secured. There is a gap between the head and the tail. The purpose of planning is to bridge the gap. Planning decisions consist of choosing a goal from a set of pending goals, choosing an operator (or inference rule) to achieve a particular goal, choosing a variable binding for a given operator, and deciding whether to commit to a possible plan ordering and to get a new planning state or to continue subgoaling for unachieved goals. Different choices give rise to alternative ways of exploring the search space. The search is guided by what are called the control rules. The problem that was given to PRODIGY was to make rivers R1 and R2 impassable by destroying the bridges across them. River R1 has two bridges and one F-15 fighter assigned to it. River R2 has three bridges and three F-15 fighters. This problem is different from the resource contention example in which, R1 had 1 F-15 and R2 had 2 F-15s assigned respectively. Moreover, previously the resource becomes available during the course of planning. In this case, during planning, PRODIGY finds that it does not have a resource to achieve one of its goals, in this case R1, then, it deliberates as to changing its goals or reassigning resources. During planning, we find a point where we know that to achieve the pursuing goal a resource is required. We can find this point using control rules that are part of PRODIGY. Now the CostBenefit analysis is performed to determine if the requesting goal gets the resource. As mentioned, the resource could be from the environment. The cost-benefit calculation helps determine if the extra resource is assigned to this goal or to the other goal that needs it more. If there are no new resources and the cost-benefit analysis determines that the requesting goal must have the resource then, a resource transfer is made from one goal to the one that needs it most. Figure 2 shows the decision point where the Cost – Benefit analysis is done. R1 gets the resource and therefore achieves the intended goal. R2 had to lower goal expectations as it does not have enough resources to achieve the intended goal.
Figure 2: The decision point, to grant a resource request or perform goal transformation.
2.5: Multiagent Resource Contention Planning for the Bridges Problem using a single agent system has all the disadvantages of using a single agent system where multiagent systems can be used amicably. To solve the Bridges Problem using multiagent systems, we assign the goal of making R1 impassable to one agent (A1) and the goal to make R2 impassable to another (A2). To understand the resource contention in multiagent systems, consider the same problem solved in the case of single agent systems, that is R1 has 2 bridges and 1 F-15 and R2 has 3 bridges and 3 F-15s. During planning A1 finds that it needs an F-15 to achieve its goal. It makes a request for an F-15 from A2. As mentioned earlier, there is a cost associated with transferring a resource, which is the servicing agent’s (A2’s) loss (Value of achieving goal less value of achieving the transformed goal) this cost is sent to A1, which in turn does the Cost-Benefit analysis to see if the utility value is greater than zero. Because the utility is greater than zero, the agents agree for a resource transfer. After the transfer, if A2 does not have enough resources it may perform a goal transformation or may request a resource from another agent in the environment. If the utility lesser than zero, A2 does not let go of its resource, A1 may either lower goal expectations or may request for the resource from another agent if there is one. In this case, the utility value is greater than zero; therefore, the resource is transferred from A2 to A1.
3: COMAS (CoOrdination in MultiAgent Systems) We have implemented a preliminary Multiagent system called COMAS (CoOrdination in MultiAgent Systems), which
implements resource coordination. COMAS have ‘N’ number of agents. We have used a version of PRODIGY called Prodigy/Agent [4]1 to represent multiple planning agents. Each agent can make a resource request to the others if it needs a resource so as to achieve resource coordination. Each agent passes the state change information to the other agents after it has performed an action to achieve goal interaction coordination. Figure 4 shows the virtual communication for resource requests in COMAS. Here Agent1 makes a resource request to Agent2. The cost-benefit analysis is performed to see if the benefit it could get if it gets the resource is greater than the cost. Agent2 responds by sending the cost associated with that resource. If so, Agent1 sends accept message else sends a reject message. If Agent2 receives accept message, it gives up its resource and performs a goal transformation, if it cannot reach its goal. If Agent1 does not send an accept message it does not get the resource and, in turn, does a goal transformation.
Prodigy/Agent1
Figure 4: Resource coordination as achieved in COMAS Figure 5 shows COMAS solving the Bridges Problem. The goal is to make river R1 and river R2 impassable. Here the R1 has 1 F-15 assigned to it and needs 1 more to achieve its goal of making R1 impassable and R2 has 3 F-15s assigned to it.
1
The Prodigy/Agent software has taken the Prodigy4.0 planning architecture and combined it with a wrapper so that it behaves as an independent agent performing a plan server function. It uses the KQML communication language to build communication protocols that support requests for plans (over sockets) to achieve input goals. The software is publicly available at www.cs.wright.edu/~mcox/ProdigyAgent. Further details are available in Cox, Edwin, Balasubramanian, and Elahi (2001).
Prodigy/Agent2
Figure 5: An example of COMAS solving a resource contention problem.
4: Evaluation We conducted an experiment to compare the relative performance of COMAS in terms of overall goal satisfaction with and without resource coordination. We allowed the COMAS system to run first on problem files with resource coordination. Then ran the PRODIGY system to run on problems without resource coordination. We calculated the overall goal satisfaction in each case. The overall goal satisfaction is defined as the total reduction in transportation capacity over the all the rivers. In this experiment, the goals to be achieved are kept constant at ten (i. e., 10 rivers to make impassable) and the number of bridges across the rivers is randomized to be between one and three. To measure the relative performance with and without resource coordination, the number of resources available was varied from 1 to a maximum of 30 and the corresponding goal satisfaction with and without resource coordination is calculated. Each time, the number of available resources is incremented, ten different problems are generated, that is, the number of bridges over the rivers is varied and the number of resources held constant. The final goal satisfaction achieved with the available resources is the average of the goal satisfaction from the ten different problems. The total planning problems therefore amount to 300. As specified in section 2, the following assumptions are made: First, an F-15 can destroy one bridge and damage all the other bridges. Second, the destroyed bridge is assigned 100% reduction in transportation capacity, and third a damaged bridge is assigned 50% reduction in transportation capacity. Best Case With Resource Coordination Without Resource Coordination Worst Case 100 90 80
Utility Value
70 60 50
The graph in figure 6 shows COMAS system with and without resource coordination. First, COMAS was run on 300 planning problems without resource coordination. In this case, the resources were assigned based on the first come first served principle, which is the order of the goals in this case. Second, COMAS was then run with resource coordination on the same set of planning problems. We calculated the goal satisfaction for each of the problems. The way resource coordination achieved was through a Java application that took the planning problem, parsed through it, and made intelligent resource assignment to the goals in this case to the rivers. We then ran Prodigy/Agents on these processed planning problems and calculated the goal satisfaction achieved. The figure 6 also shows plots for worst case and best case scenarios. The best-case scenario occurs when there are 10 rivers, and each river has just one bridge across it. When the number of resources is varied from 1 to 30, the first 10 assignments results in the best assignment and when the number of resources is 10 all the rivers are made impassable and therefore even if the available resources are increased the goal satisfaction is not increased. The worst-case scenario occurs when there are 10 rivers and each river has 3 bridges across. The total number of bridges is 30 and the number of resources required is 30. In this case, any combinations of resource assignment result in the same goal satisfaction. For relative goal satisfaction achieved in PRODIGY with and without goal transformation see [3].
5: Conclusion The focus of this paper is on resource coordination in multiagent systems. From our research, we conclude resource coordination achieve efficient and better plans. The costbenefit analysis that we introduced is efficient, but in real world planning, it should include other parameters like individual sub goal priorities and transportation costs in the cost-benefit function. Although the cost-benefit function is simple, it demonstrates that an intelligent choice of whether to get a resource from another agent or to change the goal of an agent will lead to higher quality plans that have a higher overall goal satisfaction. The cost-benefit analysis provides a way to determine if a resource request can be granted.
40 30 20 10
No of Resources
Figure 6: Shows the relative performance with and without resource coordination.
29
27
25
23
21
19
17
15
13
11
9
7
5
3
1
0
When the requesting agent does not get the requested resource, and the originally intended goal is not possible because of that, the agent can achieve a partial goal by using the concept of goal transformation. Using an efficient resource allocation algorithm results in achieving a higher overall goal satisfaction on average. The experimental results in the previous sections confirm this. An improvement in the overall goal satisfaction is certainly warranted in real world planning. It also is better to involve the human while making these decisions in the loop. A mixed-initiative approach, where the
decision making capabilities of the human agent is taken into account, is thus a better way for making these critical decisions. By using the cost-benefit analysis, the user can make better decisions than making decisions without the costbenefit analysis.
6: Acknowledgements This research is funded by Dayton Area Graduate Studies Institute (DAGSI) under grant #HE-WSU-99-09, and the Air Force Research Laboratory (AFRL) under grant number F49620-99-1-0244 and by the Information Technology Research Institute (ITRI). We thank Boris Kerkez, Greg Kramer and Mamun Elahi for comments on a draft of this paper.
7: References [1] Carbonell, J. G., Blythe, J., Etzioni, O., Gil, Y., Joseph, R., Kahn, D., Knoblock, C., Minton, S., Pérez, A., Reilly, S., Veloso, M. M., and Wang, X. 1992. PRODIGY4.0: The Manual and Tutorial, Technical Report, CMU-CS-92-150, Computer Science Dept., Carnegie Mellon University. [2] Cox, M. T., Edwin, G., Balasubramanian, K., & Elahi, M. 2001. MultiAgent Goal Transformation and Mixed-Initiative Planning using Prodigy/Agent. Proceedings of the 5th World Multiconference on Systemics, Cybernetics and Informatics (SCI 2001). [3] Cox, M. T., & Veloso, M. M. 1998. Goal Transformations in Continual Planning. In M. desJardins (Ed.), Proceedings of the 1998 AAAI Fall Symposium on Distributed Continual Planning, 23-30. Menlo Park, CA: AAAI Press / The MIT Press. [4] Edwin, G. & Cox, M. T. 2001. COMAS: CoOrdination in MultiAgent Systems. Proceedings of the 5th World Multiconference on Systemics, Cybernetics and Informatics (SCI 2001). [5] Veloso, M. M., Pollack, M. E., and Cox, M. T. 1998. Rationale-Based Monitoring For Continuous Planning in Dynamic Environments. In R. Simmons, M. Veloso, and S. Smith, eds., Proceedings of the Fourth International Conference on Artificial Intelligence Planning Systems, 171-179. Menlo Park, AAAI Press. [6] Veloso, M. M., Carbonell, J., Perez, A., Borrajo, D., Fink, E., & Blythe, J. 1995. Integrating planning and learning The PRODIGY architecture. Journal of Theoretical and Experimental Artificial Intelligence. 7(1): 81-120.
