Different agents plan for different .... to know of the state change has to either plan to achieve ... behaves as an independent agent performing a plan server.
COMAS: CoOrdination in MultiAgent Systems Gifty Edwin and Michael T. Cox Department of Computer Science and Engineering College of Engineering & CS Wright State University Dayton, OH 45435-0001 {egifty; mcox}@cs.wright.edu ABSTRACT A multiagent system is one in which multiple autonomous agents interact to solve complex tasks. Each agent in a multiagent system has its own set of goals. The central problem addressed in this paper is coordination in multiagent systems. Coordination is a process by which, a system of agents work together harmoniously. The focus of this paper is on two types of coordination, resource coordination and goal interaction coordination and its implementation in a system called COMAS. Keywords: Resource coordination, Cost-Benefit analysis, Goal transformation, goal interaction coordination, COMAS.
INTRODUCTION A multiagent system is one in which multiple autonomous agents interact to solve complex tasks. Each agent in a multiagent system has its own set of goals. An example domain where a multiagent system could be used is planning in a military scenario. Different agents plan for different goals, such as having units deployed, having enemy units destroyed and having rivers secured. The central problem addressed in this paper is coordination in multiagent systems. Coordination in multiagent systems is defined as a process by which, a system of agents work together harmoniously. The focus of this paper is on two types of coordination, resource coordination and goal interaction coordination. Resource coordination is concerned with the intelligent allocation of resources among the different agents and goals. An agent that is planning may find that it cannot achieve one of its goals because of a lack of a resource. The problem faced here is how to determine if an agent needs a resource. If so, where does it get that resource? This agent can request the resource from another agent or reassign a resource from one of its other goals, if there are any. An agent can also get resources from the environment if new resources become available. Another problem is what happens if the resource request is not granted. We propose that if a resource request is not granted, then lower the expectations of the goal that needs the resource [4]. That is the goal is minimally altered. For
example, if a city cannot be captured, then capture a local airbase. Intelligent resource allocation among the agents in multiagent systems results in better and efficient overall plans for the individual agents. This is mainly because of higher goal satisfaction that is achieved with resource coordination. Resource coordination is not only required in multiagent systems, but also single agent systems with multiple goals, need resource coordination. In this case, a resource is reassigned from one of its other goals to the goal that needs a resource. In single agent systems, the environment can be either static or dynamic. The environment is said to be static, if during planning the world states do not change. An environment is dynamic if during planning the world state change. Planners like PRODIGY [1] have the capability to plan in dynamic environment [5]. In a multiagent system, the environment can never be static. The dynamics of the environment are affected by the actions of the other agents in the environment. Goal interaction coordination is necessary when a state change made by a team member on the environment interferes with the planning of the other members. Section 2 describes resource coordination in brief. For further details on resource coordination see [4]. Section 3 describes the goal interaction coordination. Section 4 deals with a multiagent system called the COMAS, developed here at Wright State University. This section explains some of the implementation details of COMAS. Section 5 is the conclusion.
RESOURCE COORDINATION As a solution to the resource coordination problem, we present a cost-benefit analysis [4] that can be performed to determine if the requesting agent can be granted the requested resource or the agent's goal that needs the resource is altered minimally, a type of goal transformation [3] called goal erosion, in case the request cannot be granted. As mentioned earlier, the requesting agent can get the resource from another agent or a
reassignment of resources can be made between its goals. It can also get it from the environment if resources become available. The cost-benefit analyses for obtaining a resource from another agent or a reassignment of a resource between the agent's goal are the same. To state it in simple terms, it gives a utility value, which is equal to the benefit that the agent or goal gains if it gets the resource minus the cost of getting the resource. Depending on the utility value, a decision is made whether or not to grant a resource request. Resource request is granted if the utility is greater than zero. If the utility is less than zero, the resource request is not granted. It is done slightly different in the case where the new resources become available in the environment. In this case, we calculate the difference between the goal satisfaction if an agent gets a new resource (GA) and goal satisfaction without the resource (GT). The agent that has the maximum difference between (GA) and (GT) is assigned the resource. We have implemented thee resource coordination in a system called COMAS which uses Prodigy/Agents as individual agents
Consider the classic "blocks world" domain, there are four blocks A, B, C and D. If the goal is to Stack C on D, B on C and A on B. If one agent is working to achieve C on D and the other to achieve the overall goal. Figure 1 shows the scenario. Instead of the second agent planning for C on D, it is sufficient if we make this state available for the other agent. The main reason for the need for goalinteraction coordination is to achieve better and more efficient plans. Redundant plan steps are prevented with this type of coordination because agents are aware of what the other has already achieved. All this happens when there is positive interaction. There could be negative interaction when an agent changes a state that is needed by another. For the scenario in figure 2, if A1 clears block B by putting E on C. A2 has to be aware of this change and plan accordingly.
GOAL INTERACTION COORDINATION
Figure 2: Negative goal interaction scenario
In a multiagent system, the environment is never static. The dynamics of the environment are affected by the actions of the other agents in the environment. The problem is how to make the agents in a multiagent system aware of the actions of other agents in the environment. Goal interaction coordination is necessary when a state change made by a team member on the environment interferes with the planning of the other members. Consider an agent that changes State1 to State2. This change can be favorable to another agent if it needed State2. This is because the other agent does not have to plan to achieve State2. This change from State1 to State2 can adversely affect the planning of another agent if that agent needed State1. Now the other agent when it comes to know of the state change has to either plan to achieve State1 or has to adjust its planning decisions.
As mentioned the problem that is faced in goal interaction coordination is how to make the agents aware of the actions of other agents on the environment. In other words, we should make agents aware of the state changes made to the environment by other agents. There are two ways the other agents can be made aware of the state changes made by an agent. First, when an agent makes the change, it takes the responsibility of broadcasting the changes to the other agents. Second, when an agent makes a change, it writes those changes to a virtual blackboard, from which the other agents can read. An agent reacts to only those changes that interfere with its planning. We have implemented goal interaction in COMAS. Two primary changes are made to adapt the rationale-based monitoring of PRODIGY [5][6] that is used in Prodigy/Agent so that it can be used in a multiagent environment. First whenever there is a state change, it is broadcast to the other agents. Second during sensing, the states changed by one agent are received by the other agents and their corresponding monitors are fired. When a monitor is fired, the plan is altered accordingly.
Figure 1: Positive goal interaction scenario
Figure 3 shows an example of goal interaction. Here there are two agents; one (PRODIGY1) has the goal to make two rivers R1 and R2 impassable and the other (PRODIGY2) has the goal to make R2 impassable. Due to positive goal interaction PRODIGY1 does not have to
plan to make R2 impassable as it is achieved by PRODIGY2.
Figure 3: Example of goal interaction
COMAS (CoOrdination in MultiAgent Systems) We have implemented a preliminary Multiagent system called COMAS that implements resource coordination as well as goal interaction coordination. COMAS can have multiple planning agents. We have used a version of PRODIGY called Prodigy/Agent [2]1 to represent multiple planning agents. Each agent can make a resource request to another if it needs a resource to achieve resource coordination. Additionally, each agent passes the state change information to the other agent after it has performed an action to achieve goal interaction coordination.
Figure 4: COMAS system 1
The Prodigy/Agent software has taken the Prodigy4.0 planning architecture and combined it with a wrapper so that it behaves as an independent agent performing a plan server function. . It uses the KQML communication language to build communication protocols that support requests for plans (over sockets) to achieve input goals. The software is publicly available at www.cs.wright.edu/~mcox/ProdigyAgent. Further details are available in [2].
Resource coordination implementation in COMAS We have implemented the resource coordination in COMAS using a Client/Server architecture. We have implemented the server (called the resource coordinator) in Java. Prodigy/Agents are the clients in this case. The clients are connected to the server through sockets. The server side coding is in Java and the clients are coded in Allegro Common LISP version 6.0. The resource coordinator is a multithreaded server and can accept any number of clients. The server has the data regarding the number of goals and the number of resources associated with each Prodigy/Agent. There are several reasons for implementing the server in Java. First, the concurrency patterns provided in Java allow efficient and easy to develop constructs for protecting the critical sections of the server, in this case details of all the clients. Second, an object-oriented approach to a server architecture allows efficient reuse and reliability of the code. Third, Java provides easy to develop graphic user interface. When a client needs a resource to achieve one of its goals, it makes a request to the server. The server now performs the cost-benefit analysis. The server obtains the lowest cost for that resource. This cost is the cost for obtaining the required resource from the agent with the least cost from among all the agents that have that resource. This cost is compared to the benefit of the requesting agent. If the benefit minus cost value (utility value) is greater than zero, then a resource transfer is made to the client.
The clients in COMAS, the Prodigy/Agent, are coded in LISP and the communication protocol between the resource coordinator and Prodigy/Agent are written in LISP. The clients, as mentioned earlier, make a request for a resource to the server, when they fall short of resources. Depending on the utility value, the resource is obtained or the client does goal erosion that is it lowers its goal expectations. Lowering goal expectations and achieving a plan that may not achieve the originally intended goal is favorable than not having a plan at all. Algorithm to handle Resource Requests on the Server side 1. Wait for requests from the clients. 2. Upon receiving a request • Perform cost-benefit analysis (see [3]) • Obtain the cost, which is the minimum from all agents. • Calculate the benefit for the requesting agent • Calculate utility is equal to the benefit minus cost • If (utility > 0) assign resource else lower the client’s goal. 3 . If all clients have finished planning quit else go to step 1.
Algorithm to handle Resource Requests on the Client side 1.
Send request to the Resource coordinator if more resources are needed. 2. Wait for reply. 3. Receive the utility value. 4 . If (utility > 0) pursue the intended goal else lower the goal expectations. 5 . If finished planning send “quit” to the server else go to step 1. Figure 5 shows COMAS solving the Bridges Problem [3] [4]. The goal is to make river R1 and river R2 impassable. Here the R1 has 1 F-15 assigned to it and needs 1 more to achieve its goal of making R1 impassable and R2 has 3 F-15s assigned to it. In the figure due to resource coordination, an F-15 from Prodigy/Agent2 was transferred to Prodigy/Agent1. Figure 6 shows the virtual communication occurs between the requesting agent and the agent that has the least cost for the requested resource.
Figure 5: An example of COMAS solving a resource contention problem.
agents through a :state-change interrupt3 via the socket connections. Interrupt handlers use PRODIGY control rules to force a jump in the search tree to the node that represents the original decision. Appropriate plan adaptations can then be performed for either positive or negative goal interactions.
CONCLUSIONS Figure 6: Virtual communication as in COMAS
Goal interaction implementation in COMAS In multiagent systems, the dynamics of the environment are constantly changed due to the actions of other agents in the environment. Two primary changes are made to adapt the old rationale-based monitoring of PRODIGY so it can be used in a multiagent environment. First, whenever there is a state change, it is broadcast to other agents. Second, during sensing, the states changed by one agent are received by the other agent and their corresponding monitors are fired. When a monitor is fired the corresponding plan adaptation is done. Rational-based planning monitors were first developed in [5]. The monitors are used to notice state changes that contradict planning commitments made in previous planning decisions. For example open preconditions on planning operators require the planner to plan further to achieve the precondition. If the rationale to subgoal on an operator precondition is that the precondition state is false, a plan-based subgoal monitor[5] is created to watch for changes in the world that indicate the state became true. If the state does become true, then the planner must prune that part of the plan related to achieving the precondition. In this way, monitors are surveying only those states of the world that bear directly on the reasons used to make previous planning decisions.2 In [5] the monitors were used to detect changes in the world due to exogenous events such as changing weather conditions. In a multiagent environment, change may also arise from planning decisions from other agents. We therefore modified the monitor implementation to establish socket connection between all agents working in a team. As each operator is applied to the current state in a forward-chaining search decision, the states added and deleted to the individual agent’s world are sent to other
2
[5] describes a number of additional rationale-based plan monitors that are implemented in this system.
For our research, we conclude that every multiagent system should confront two types of coordination namely: resource coordination and goal interaction coordination. As mentioned previously, with resource coordination, the plans for the individual agents result in a higher goal satisfaction than, if there was no resource coordination. Goal interaction coordination enables each agent in the environment to be aware of the actions of the other agents in the environment. This helps the system of agents to act together harmoniously. We would like to implement the concept of negotiation [7] among the agents in MAS before a state change is made to the environment by an agent. This negotiation will determine if this state change is beneficial to the overall system. If it is beneficial then the state change is agreed by the other agents. If it is not beneficial, the agent that wants to make the state change is not allowed to do so and has to consider planning again for the action that caused the state change. This negotiation will result in a more efficient overall plan and individual plans more agreeable to the individual agents. Negotiation compromises speed for quality because there is an overhead during negotiation before a compromise is made.
ACKNOWLEDGEMENTS This paper is supported by the Dayton Area Graduate Studies Institute (DAGSI) under grant #HE-WSU-99-09, by the Air Force Office of Scientific Research(AFOSR) under grant number F49620-99-1-0244, by a grant from the Information Technology Research Institute (ITRI),and by Wright State University (Research Challenge: Young Investigator grant).
3
The PRODIGY interrupt system is described in section 8.2 of [1].
REFERENCES [1] Carbonell, J. G., Blythe, J., Etzioni, O., Gil, Y., Joseph, R., Kahn, D., Knoblock, C., Minton, S., Pérez, A., Reilly, S., Veloso, M. M., and Wang, X.. PRODIGY4.0: The Manual and Tutorial, Technical Report, CMU_CS_92_150, Computer Science Dept., Carnegie Mellon University. 1992. [2] Cox, M. T., Edwin, G., Balasubramanian, K., & Elahi, M. MultiAgent Goal Transformation and Mixed-Initiative Planning using Prodigy/Agent. In Proceedings of the 5th World Multiconference on Systemics, Cybernetics and Informatics (SCI 2001). (In press). 2001. [3] Cox, M. T., & Veloso, M. M. Goal Transformations in Continual Planning. In M. desJardins (Ed.), Proceedings of the 1998 AAAI Fall Symposium on Distributed Continual Planning, 23-30. Menlo Park, CA: AAAI Press / The MIT Press. 1998. [4] Edwin, G., & Cox, M. T. Resource Coordination in Single and Multiagent Systems. Submitted to 6th European Conference on Planning. 2001.
[5] Veloso, M. M., Pollack, M. E., and Cox, M. T. Rationale_Based Monitoring For Continuous Planning in Dynamic Environments. In R. Simmons, M. Veloso, and S. Smith, eds., Proceedings of the Fourth International Conference on Artificial Intelligence Planning Systems, 171_179. Menlo Park, AAAI Press. 1998. [6] Veloso, M. M., Carbonell, J., Perez, A., Borrajo, D., Fink, E., & Blythe, J. Integrating planning and learning The PRODIGY architecture. Journal of Theoretical and Experimental Artificial Intelligence. 7(1): 81-120. 1995 [7] Huhns, M. N., & Larry M. Stephens, L. M. Multiagent Systems and Societies of Agents. In G. Weiss (Ed.), Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence. 79-120. Cambridge, MA: The MIT press. 1999.
