Dynamic Interaction Protocol Load in Multi-agent System Collaboration

Dynamic Interaction Protocol Load in Multi-agent System Collaboration Maoguang Wang1,2,3 , Zhongzhi Shi1, Jiewen Luo1,3, Jun Hu1, Fen Lin1,3 1

Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100080, China 2 School of Computer Science, China University of Mining and Technology, Xuzhou 221008, China 3 Graduate School of the Chinese Academy of Sciences, Beijing, 100039, China {wangmg,shizz,luojw, huj, linf}@ics.ict.ac.cn

Abstract. Multi-agent system(MAS) has more advantages because the agents can collaborate to complete complex tasks. However for the interactions are more complicated and even uncontrolled, industry find it difficult to resolve the multi-agent collaborations. This paper focuses on the interaction protocol in MAS. Firstly the interaction ontology library comprising role ontology, action ontology and message ontology is introduced into the semantic interactions. Then the representation of interaction protocol is presented, which expresses an interaction protocol as process description and decision-making rules. Then we propose a three-layer agent architecture that contains knowledge layer, collaboration control layer and communication layer. We embed a protocol interpreter and loader into the collaboration control layer to deal with the interactions between agents. Finally dynamic interaction protocol load in MAS are discussed, which load the interaction protocol from protocol library conveniently and the code will be generated automatically. Then we give an experiment to verify our resolution.

1 Introduction and Motivation Multi-agent system mainly studies the interoperability of the related agents and their negotiation[1] and collaboration. Because its ability to resolve problems is more superior to a single agent, the multi-agent system is becoming more and more concerned. In MAS, agents can perform various kinds of behaviors and conforms to the special kind of interaction protocol. Considering the interaction complexity, we should ensure the agent interactions comply with the relevant protocol specifications. If the MAS is confined to a close environment, we can predict and adopt the specified protocols. However to adapt to the open and dynamic environment, the MAS must consider the agent dynamic changes including role changes, protocol changes and so on.

When we design MAS, the semantic issues have to be resolved including who participate in the interaction, what message to communicate and which actions should take place etc. Therefore this paper firstly introduces the ontology library to the interaction protocol according to the participant role, the communication message and the actions. Furthermore, to adapt themselves to the dynamic changing environment the agents should adopt various protocols to interact with other agents. For instance, if one agent migrates from the contract-net environment to the auction one it must abide by the new ‘game rules’. If the agent is restricted to the fixed protocol, its collaboration with other agents is also confined to the limited environment. And we focus on the relevant issues of loading interaction protocols dynamically. James J. Odell, H. Van Dyke Parunak, et al. [2] analyzes and classifies the various kinds of role changes over time that may occur using dynamic activation and dynamic classification methods, and shows how this analysis is useful in developing a more formal description of the application. However it did not consider the protocols load while changing roles. Javier Vazquez-Salceda, Virginia Dignum, et al. [3] proposes a framework for modeling agent organizations, OMNI, both the norms that regulate interaction between agents, as well as the contextual meaning of those interactions are important aspects when specifying the organizational structure, but lack of detailed considerations of dynamic protocol changes due to the context changes. The rest of this paper is organized as follows: Section 2 analyzes the semantic issues in the multi-agent interaction protocol. Section 3 studies the dynamic protocol load in MAS based on the protocol process description and decision rules, ultimately it gives an experiment to verify our resolution using VAStudio. Section 4 summarizes the paper and discusses future work.

2 Semantic Interaction Protocol In MAS one interaction is a process of exchanging a sequence of messages, and each message should be handled in an explicit way. The related agents should predict what kind of message would be received and how to deal with it. Due to lack of sufficient semantic support, the agents may sometimes misunderstand the communication contents. For instance, when an agent asks another agent for a ‘locate the Wall Street’ message, the receiver should know ‘Wall Street’ is a financial center of America, not a street name or anything else. It requires that the exchanged message uses the same message representation, or establishes the ontologies[4]. Thus an agent can work out where and how to plug in to the protocol its own code for message processing and for domain-specific decision making, and begin using that protocol to interact with other agents[5]. To ensure the mutual understanding, we introduce the ontology library to the interaction protocol. The ontology library includes three basic kinds of ontology: role ontology, message ontology and action ontology, which contain the relevant concepts, the instances, the relations, the properties and the actions needed to support the agent interaction. Moreover the basic ontology can be added to the predefined ontology if needed. We use a 4-tuple to describe the interaction ontology, Ir=,

where the element represents sender, action, message content and receiver respectively. For S and R, they usually represent the specified roles. If we build the role ontology—supervisor role and student role, the receiver should include the professor and researcher automatically when the PhD student requests the supervisor. Action ontology includes a set of classes that are commonly part of the related ontology, such as action, trueproposition, result and so on. We use a 6-tuple to formal action ontology. Ac=. Action class is used to represent actions. It has a couple of methods to set/get the AID of the actor (i.e. the agent who should perform the action) and the action itself (e.g. Register/Deregister/Modify)[6]. Furthermore, the class should define a set of constants used to refer to the FIPA performatives, such as REQUEST, INFORM, PROPOSE and so on. Figure 1 illustrates the interaction ontology[7].

Fig. 1. The Interaction Ontology

The message is composed of communication-act and content: message ::= communication-act(param1, param2, ……). Communication-act defines the type of the message. Message content is a set of variables. And both the sender and the receiver can understand the values of the variables. According to FIPA specifications, ACL is used to represent exchanged messages. It contains a set of attributes defined by the FIPA specifications. For the INFORM-meta-protocol, it can be represented as follows: (INFORM :sender agent1 :receiver agent2 :content(price good0 100) :language sl :ontology English-auction ) After adopting ontology library, we can define it as follows. (INFORM :action inform :sender agent1 :receiver agent2

:content(price good0 100) :language sl :ontology Action, Role, English-auction ) Based on that, we can further add the ontology library to the agent collaboration framework.

3 Dynamic Interaction Protocol Load in MAS In the dynamic environment, the agents should not only be assigned roles and conform to the specified protocols, but also change their roles and behaviors according to the environment changes. For instance, the agent using the FIPA-Contract-Net-Protocol should load FIPA-English-Auction-Protocol dynamically according to the changing environment. We can formal the MAS collaboration model[8] as 5-tuple M=, where Ag={A1, A2, … ,An} is the set of agents which participate in the collaboration, G represents the collaboration goal that is produced under the specified circumstances, P represents collaboration plan whose key problem is to produce the society collaboration structure and define the required roles, the roles relationships and their interaction protocols, T is the set of collaboration members, Ag∈T, S is the collaboration scheme, S∈P. Apparently P is the critical element and we focus on the dynamic changing roles and protocols.

3.1 Agent Architecture To support the representation of the interaction protocol, we propose a three-layer agent architecture based on the BDI architecture. BDI agent architecture is widely used, but it is relatively difficult to implement interaction protocols. And we embed a protocol interpreter and loader in the BDI agent to control the message communication and protocol load process. When receiving a message, the protocol interpreter and loader updates the belief, asks agent make decisions if needed, and sends reply according to the protocol description stored in the protocol process description library. New protocol description can be added to the library, and existing protocol descriptions can be updated, so agent has the ability to use the protocols flexibly. In order to handle the protocols conveniently, we insert a collaboration control layer between the knowledge layer and the communication layer, in which a protocol interpreter and loader, a protocol process description library, ontology library and a decision rule library, are embedded. The architecture is shown in figure 2.

Protocol interpreter and loader is the core of the collaboration control layer. When the environment changes, the protocol interpreter and loader can load the protocol based on the initialization of the collaboration environment, looks up the protocol process description library and ontology library, and executes the operations including initializing the loading process, adding fact, making decisions, sending messages referring to ontology library and so on.

3.2 Protocol Process Description Complex collaboration comprises a lot of interaction processes including what kind of message should be received, how to handle it and so on. Based on the ask-answer manner and event-driven process in the interaction, we represent the protocol process using BNF syntax as follows: ::= { } ::= ::= | ::= { } ::= | ::= on do :: = | ::= () ::= ::= | ::= ::= ::= { } ::= | ::= | |

::= if () {statements} | if (< expression >) {statements} else {statements} | goto() | exit ::= and(expression, expression) | or(expression, expression) | not(expression) | decide() | ::= assert() | send(,[ ]) ::= start(,) | stop() ::= | ::= ::= any word consisting of only letters and numbers (but starting with a letter). ::= any first-order logic predicate. Actor is a participant of the conversation. An event handling process has the form of ‘on event do procedure’, where event is the trigger and procedure is the handling process. There are two basic kinds of events that can be used as the trigger: message and timer. Message represents the event of receiving signal message. Timer is a timer event. Some protocols need timers, such as the timeout of the proposal in the FIPA-English-Auction-Protocol. Procedure defines the event handling process and they should be understood and executed by agents, so a set of basic operations are defined, which can be executed by the protocol interpreter and loader. The basic operations are defined as follows: 1) Flow control: z Condition judgement: if(condition){…}[else {…}]; z Jump: goto(Event); z End conversation: exit; 2) Interface to the knowledge layer: z Add fact: assert(Proposition), where Proposition is a fact description in first order predicate logic form; z Decision: decide(Action), agent needs to decide whether execute the Action; 3) Interface to the communication layer: z Send and Receive message: send_receive([Sender], Message, [Receiver]), where Sender and Receivers are the set of agents that should send or receive the Message; 4) Timer operation: z Start timer: start(Timer, Timeout)； z Stop timer: stop(Timer)； 5) Logic operator: z A and B：and(A, B)； z A or B：or(A, B)；

z Not A：not(A)； For example, the handling procedure of cfp message in the FIPA English Auction Protocol is defined as follows, where x, y, o, p, n represent the sender agent, the receiver agent, the object, the price and the turn respectively: on cfp(x, y, o, pn, n) do { assert(cfp(x, y, o, pn, n)); if (decide(not-understand(x, y, o, pn, n))) { send_receive(not-understand((x, y, o, pn, n), [x]) } else if (decide(propose(x, y, o, pn, n))) { Send_receive(propose(x, y, o, pn, n), [x]);}}

3.3 Decision rule Process descriptions and decision rules are the knowledge for interaction protocols. Decision rule tells agent how to take actions. Agent also needs to consider the collaboration policy in order to gain the most benefits when making decisions, such as the increment of the price in each turn in English Auction Protocol. This paper only discusses the necessary limits on the communication act in decision-making, and detailed collaboration policy will be discussed in our future work. Decision rule has the following form: Decision ← Precondition. Decision is the result of the decision, which usually is a handling process. Precondition—a logic expression representing the agent initializing condition. For instance, the decision rules of English Auction Protocol are as follows: Initiator: //decision rules of the initiator z accept-proposal(x,y,o,pn,n)←propose(x,y,o,pn,n) ∧ ¬(∃Y)accept-proposal(x,Y,o, pn,n); //accept the proposal z reject-proposal(x,y,o,pn,n)←propose(x,y,o,pn,n) ∧ (∃Y)accept-proposal(x,Y,o, pn,n); //reject the proposal z deal(x,y,o,pn-1,n)←¬(∃Y)propose(x,Y,o,pn,n) ∧ accept-proposal(x,y,o,pn-1,n-1) ∧ larger(pn-1,min_price(x,o)); //if no proposal appears in the current turn, and the price in last turn is higher than the reservation price, then deal z cfp(x,z,o,pn,n)←(∃Y)(propose(x,Y,o,pn-1,n-1) ∧ larger(pn,pn-1)) ∨ start(x,o,pn,n); //if a proposal exists in last turn or conversation just starts, then start a new turn to call for bids and the price is higher than that in last one Participant: //decision rules of the participants z propose(x,y,o,pn,n)←cfp(x,y,o,pn,n) ∧ larger(max_price(y,o),pn) ∧ ((accept-proposal(x,y,o,pn-1,n-1)∉belief-set) ∨ larger(min_price(x,o),pn-1)) ； //if the price in current turn is lower than the reservation price and its proposal in last turn is rejected or the price in current turn is lower than the reservation price, then propose z not-understand(o)←cfp(x,y,o,pn,n)∧(deal(x,y,o,pn)∉goal-set)；//if no desire to buy goods o, then reply with ‘not-understand’ The meanings of the predicates are as follows: deal(x,y,o,p): agent x and y make a deal that x sells y o at price p. larger(a,b): a>b.

start(x,o,p,n): agent x starts an auction to sell goods o; the initial price is p; n represents the turn of the bids and only can be 1. The meanings of the variables are: min_price(x,o): the lowest price that agent x can accept to sell o. max_price(y,o): the highest price that agent y can accept to buy o. belief-set: the set of beliefs of the agent. goal-set: the set of goals of the agent.

3.4 Experiment Multi-agent platform MAGE developed by out lab is a multi-agent development and running environment, which is based on the intelligent and multi-agent technologies, utilize the agent-oriented software engineering to analyze, design and deploy the multi-agent system. As a distributed development platform, the MAGE environment comprise three tools: Modeling Tool for Multi-Agent System (MTMAS) supports the domain analysis and design including AUML modeling tool; Visual Agent Studio(VAStudio) is a multi-agent development platform; MAGE deployer provides the deployment and running environment for the multi-agent system. VAStudio[9] is a visual agent development platform and provides the practical agent development processes, which also can encapsulate external components to agents. After that, the agents can be deployed to the MAGE[10] platform. When developing agent, VAStudio provides the wizard to facilitate the development. When developing a collaborative multi-agent system, the behaviors including FSMBehaviour, ParallelBehaviour, SequentialBehaviour etc. describe the agent capabilities. The agent adding the needed behaviors is an autonomous software entity. The related agents construct a virtual agent society, which can collaborate to resolve the appointed tasks according to their roles and priorities. VAStudio have built a protocol library, which includes most of the used protocols, such as FIPA-Contract-Net-Protocol, Negotiation-Protocol, Auction-Protocol, the related Meta-Protocols and so on. For the deduction and collaboration, the library also provides: z The decision rules to control agent interactions, including when and how to interact with other agents. Likewise, the plan and collaboration are under the control of the agent knowledge base including its available resources, its abilities, the organization relationship with other agents and its available collaboration polices etc. So except the predefined protocols, the library should also provide: z The predefined organization relations, which mainly include the relations of superior-subordinate, co-worker and peer-peer relationship etc. z The ontology library and the database for describing the agent resources and abilities. Considering the common senses and the capabilities, the agent should take on the simple or composite behaviors once it joins the MAS. If the agent society is completing the concrete task such as auction or vote, the agent should be assigned the specific protocol dynamically. Thus the society can continue to process the task uninterruptedly.

From the collaboration model M, where plan can be represented as 4-tuple P=. In our experiment, once we design the society and specify the roles and protocols, the relationship can be derived. Also we specify the agent ontology for their mutual understanding. Now we set up a FIPA-Contract-Net environment, firstly we develop 3 agents:a1,a2,a3, a1 is the initiator, a2 and a3 are the responders, then add them to the society s1. Now these agents only have basic behaviors. VAStudio just specify the role and FIPA-Contract-Net-Protocol to them respectively. Figure 3 shows the agent collaboration roles and protocols loading. Fig. 3. Agent Collaboration Roles and Protocols Once we specify the roles and protocols, VAStudio can add the corresponding code to the method automatically. Figure 4 illustrates the generating code.

Fig. 4. The Dynamic Generating Code

Now suppose that the agent a4 move from another auction society s2 to s1 society. VAStudio can load the FIPA-Contract-Net-Protocol and assign the responder role to a4 agent. The process is showed in figure 5.

Fig. 5. The New Agents Role and Protocol Load

The fundamental generating codes are as follows: protected void setup() { //FIPA_CONTRACT_NET,ROLE_RESPONDER addBehaviour( new FipaContractNetResponderBehaviour(this){ final String whatToDo = "Goods0"; public ACLMessage handleCfpMessage(ACLMessage cfp){ ACLMessage msgReply = cfp.createReply(); msgReply.setSender(getAID()); msgReply.addReceiver(cfp.getSender()); String cfpContent = cfp.getContent(); if(whatToDo.equalsIgnoreCase(cfpContent)){ msgReply.setPerformative(ACLMessage.PROPOSE); msgReply.setContent(""); }else{ msgReply.setPerformative(ACLMessage.NOT_UNDERSTOOD); msgReply.setContent("not-understand");} return msgReply;} public ACLMessage handleAcceptProposalMessage(ACLMessage msg){ACLMessage msgReply = msg.createReply(); msgReply.setSender(getAID());

msgReply.addReceiver(msg.getSender()); msgReply.setPerformative(ACLMessage.INFORM); msgReply.setContent("Bargain On!"); return msgReply;} public void handleRejectProposalMessage(ACLMessage msg){} public void handleOtherMessages(ACLMessage msg){}} ); //FIPA_CONTRACT_NET,ROLE_RESP End} Further formal verification can be discussed[11]. After that VAStudio can deploy these agents to MAGE platform and test the task running[10].

4 Conclusion and Future Work The purpose of this work is to propose a methodology, encompassing together interaction protocol ontology and dynamic interaction protocol load in MAS. It has been demonstrated with the role changes and protocols load in VAStudio. Complex policy to control the agent loading process and collaborations is being studied. Also the efficient methodologies of formal ontology and automatic protocol verification should be further developed in VAStudio.

Acknowledgements. This work is supported by the National High-Tech Programme of China (Grant No. 2003AA115220), the National Basic Research and Development Plan of China (Grant No. 2003CB317000) and China University of Mining and Technology (Grant No. 0D4489).

References 1. OhByung Kwon: Multi-agent system approach to context-aware coordinated web services under general market mechanism, 2004. 2. James J. Odell, H. Van Dyke Parunak, Sven Brueckner, John Sauter: Temporal Aspects of Dynamic Role Assignment. AOSE 2003: pp. 201-213. 3. Javier Vazquez-Salceda, Virginia Dignum and Frank Dignum: Organizing Multiagent Systems. Institute of information and computing sciences, Utrecht university, technical report UU-CS-2004-015, http://www.cs.uu.nl. 4. Nicola Guarino(ed.): Formal Ontology in Information Systems. Proceedings of FOIS’98, Trento, Italy, 6-8 June 1998. Amsterdam, IOS Press, pp. 3-15. 5. Stephen Cranefield, Martin Purvis, Mariusz Nowostawski and Peter Hwang: Ontologies for Interaction Protocols. Proceedings of the Workshop on Ontologies in Agent Systems, 1st International Joint Conference on Autonomous Agents and Multi-Agent Systems, S. Cranefield, T. Finin and S. Willmott (eds.), CEUR Publications, KUB Tilburg University, Tilburg, The Netherlands (2003). 6. JADE PROGRAMMER’S GUIDE. http://jade.tilab.com/,2005.

7. Yuh-Jong Hu, Se-Ting Chen, Min-Huei Yang: Agent Communication Protocols for Authentication Authorization and Delegation on the Semantic Web. WWW2004, New York, NY USA. 8. Shi Zhong zhi. Intelligent Agent and Its Applications. Press of Science. 2001. 9. VAStudio. http://www.intsci.ac.cn, 2005. 10. Zhongzhi Shi, Haijun Zhang, Yong Cheng, Yuncheng Jiang, Qiujian Sheng, Zhikung Zhao: MAGE: An Agent-Oriented Programming Environment. IEEE ICCI 2004: 250-257 11. Matias Alvarado, Leonid Sheremetov, Ernesto Gernam, and Erick Alva: Logic of Interaction for Multiagent Systems. MICAI 2002, LANI 2313, pp.387-400, 2002.