Design Pattern for Self-Organization Multi-agent Systems Based on

2011 International Joint Conference of IEEE TrustCom-11/IEEE ICESS-11/FCST-11

Design Pattern for Self-Organization Multi-agent Systems based on Policy Yi Guo, Xinjun Mao, Cuiyun Hu School of Computer National University of Defense Technology Changsha, Hunan, China [email protected], [email protected], [email protected]

the environment and the changing user requirements. The reason is that self-organization mechanisms for the system often have been decided during the design time, as a result the behaviors of agents can not be dynamically adjust during the run time. For example, stigmergy [5] is a well known self-organization mechanism which is inspired from ant colony. However when the ants are deposited in an environment with pungent odor, the stigmergy phenomena probably will not take place because the odor in the environment results in that ants can not scent the pheromone, and therefore they can not walk alone with the pheromone. To address the above problem, we propose a new selforganization design method inspired from the policy in the human society. Policy is an effective way to adjust complex systems like human society. They typically take various forms like law, norm, etc. They are helpful to keep the whole organization or society in an order state as well as restrict the behaviors of agents in the organization or society. We believe that policy can act as an effective way to address the above problems. First, policy per se is a global restriction for the whole system that every agent must obey it. Second, policy can be dynamically adjusted during the run-time, which can be used for handling the unpredictable situations. Thus, understanding the principles behind the policy mechanism as well as their computational adaptation is of great importance to engineering effective, self-organization multi-agent systems. In order to engineer self-organization MAS based on policy principle, it is important to capture the principles in a structured design pattern that can be instantiated and used according to specific needs. Therefore, in this paper we regard the policy design method as design pattern to better support the development of self-organization systems. In section 2, we introduce the principles of policy. Section 3 presents the design pattern with practice and knowledge about exemplary instantiations of the pattern. Related works are discussed in section 4 and conclusions are made in section 5.

Abstract—Self-organization is an important characteristic for multi-agent systems. However developing self-organization multi-agent systems in a repeated and effective way is still a great challenge. It is well known that design patterns are the practice and knowledge about solutions for recurring problems, and reusing design pattern can significantly improve the software development quality and efficiency. On the other hand, current self-organization applications are still lack of effective way to handle the unpredicted environment and changing user requirements. In this paper, we take policy as the basic principle for self-organization multi-agent systems to solve this problem, and propose a policy based design pattern for the self-organization multi-agent systems. Keywords-component; Design Organization;Multi-agent System

I.

Pattern;

Self-

INTRODUCTION

Current and emerging systems are characterized with many properties like openness, autonomy, locality in control and interactions, self-organization, emergency, etc. Typically, they consist of a number of elements that are autonomous in behavior and local in their interactions and decentralized in control. In the literature, this kind of systems is commonly referred to as self-organization systems [1]. An essential key issue for the autonomy of these systems is their capability of effective, spatial and/or temporal selforganizing for purposes such as resource allocation, load balancing, spatial distribution, dynamic clustering, group formation, etc. To make a better solution for the issue, recently self-organization multi-agent systems have drawn a lot of attentions from industry and academic communities [2] [3]. There are several important characteristics of selforganization systems, such as autonomous entities, local interaction, decentralized control, emergence. These characteristics bring great obstacles for the development of such systems. In the literature of self-organizing MAS (Multi-Agent Systems), a key issue is how to design selforganization mechanism to specify and implement selforganization systems and obtain the expected global characteristics of the systems. Current self-organization mechanisms are mainly inspired by bio-system, such as foraging, nest building, brood sorting, web weaving, schooling, herding etc [4]. However, we argue that existing researches on self-organization mechanisms have an important deficiency: current self-organization MAS is lacks of appropriate way to handle the unpredictable situations like 978-0-7695-4600-1/11 $26.00 © 2011 IEEE DOI 10.1109/TrustCom.2011.218

II.

PRINCIPLES OF POLICY FOR SELF-ORGANIZATION MAS

Policy is a kind of information that affecting the behavior of agents within the policy domain. The concept of policy is not new in the literature of MAS, instead the related concepts are norms, rules that are proposed in some researches [6]. However, these methods can not address the dynamic requirements of self-organization systems. When normbased MAS are developed, the system’s objectives usually

1572

should be pre-determined, so they can not completely support the applications whose objectives should be modified frequently at run-time. The ideas of policy-based management are useful to deal with this problem. According to [7][8], policy is a means to dynamically regulate the behavior of system components. While sharing much in common with norm-based approaches, policy-based perspectives differ in subtle ways. Feltovich pointed out that norms are designed offline, while policy-based approaches support dynamic runtime policy changes, and not merely static configurations determined in advance [9]. In general, the policy focuses on regulating behaviors of agents. Typically, policy can be classified as two categories: authorization and obligation. The former defines what agents are permitted or forbidden to do, while the latter specifics what agents should do. The behaviors of each agent are the essential for self-organization process of the whole MAS. However, current researches on norms, rules just take into account the micro-level such as agent interaction, negotiation, and rarely consider the macro-level like selforganization, emergent. Nevertheless, we believe that there is a close relationship between policy and self-organization. We could see many cases in human society that with the policies promulgation, and some self-organization processes could be seen obviously. For example, in economic domain a promulgation of a new policy could result in drastic economic activities which may lead some economic phenomena like stock slump, economic growth. Therefore, there is a close relationship between policies and selforganization of systems in the policy based multi-agent systems. Especially, we believe that the following three characteristics of policies are related with the selforganization of multi-agent systems: „ Policy can affect the behaviors of agents. „ Policy aims at the system macro-level requirements, i.e. the policies are not used for limiting the behaviors of agent only, but for realizing the goals of the whole systems through affect the behaviors of agent in it. „ Policy can be dynamically established or abolished during at run-time of system. The first two characteristics show that policy as the system restrictions not only can affect the behaviors of agents, but also can realize the self-organization process of systems. The last characteristic shows that policies can solve the problems discussed in section 1. Therefore, we can take policy as design method to realize self-organization multiagent systems to meet the problem we described in section 1. However, this new method also brings some challenges in developing self-organization systems. In the next section, we will propose this method as a design pattern to support the development of self-organization multi-agent systems. III.

to a recurrent problem in a specific domain which could be solved by policy based design method appropriate. The use of design patterns offers several advantages, such as, reducing design-time by exploiting off-the-shelf solutions, and promoting collaboration by providing a shared language. Specifically, in the case of self-organization systems, patterns play a key role in driving the designer choices among the chaotic behaviors displayed by complex systems. In this paper, we adopt the pattern scheme which was used by [13]. The reason is some items like Feedback Loop, Locality in [11]or [12] are more suitable for the decentralized coordination self-organization mechanisms like Evaporation, Aggregation, etc. Table 1 show the item of the pattern. For the reason of Name and Aliases are obvious (the name is “policy driven self-organization” and no aliases), in the rest of this section, we begin with the Problem item. Table 1. Item of pattern Name Aliases Problem Forces Dependencies Entities Implementation Example Known Uses See Also

The name of the pattern Alternative names The problem solved by the pattern Trade-offs in system dynamics Assumeing and environmental requirement Entities participating to the pattern Hints on implementation An example of usage Existing applications using the pattern References to other patterns

A. Problem This pattern suit to construct the self-organization MAS which has the following properties: „ Decentralized control: The system does not have a centralized control node and each agent in this system can choose behaviors by itself. „ Variety of user requirements or environment: The user requirements (environment) are often changing and the system needs different self-organization processes to adapt to these changes. „ Heterogeneity agents: Heterogeneous agents with different capabilities regarding their mobility, ability to communicate, or functionality. The system users may ignore the difference among the heterogeneous agents and just constitute the policy according with the requirements. „ Openness and scalability: Autonomous agents may leave or join the system at any time and any location without affecting the overall performance negatively. They only need to obey the policy when they are in the system. B. Force „ Centralization vs. decentralization: Comparing with centralized approach, decentralized approach usually causes issues of communication and coordination overhead. One solution is to make every agent has almost global knowledge about the system state. However, the global state usually can not be obtained without a further assumptions or restrictions. An

DESIGN PATTERN FOR POLICY BASED SELFORGANIZATION MAS

Some design pattern for self-organization systems based on different mechanisms have been presented by some researchers, for example Tom Dewolf [11] and Gardelli [12]. A design pattern based on policy provides a reusable solution

1573

„

When the system is running, the duties and rights of each role are specified in terms of policies, and the policies may be modified, then the objectives of the role are indirectly fulfilled by role enacting agent, thus the system is able to timely response to the variation of dynamic requirements and environment through self-organization. Moreover, the behaviors of agents in the system can be influenced by the specified policies and agent’s decision making is made as consistent as possible with the system requirements.

alternative approach is to make the information to control the system intrinsically distributed. Moreover, in dynamic environments a decentralized approach has no bottleneck or single point of failure. Policies vs. behaviors: The behaviors of an agent indicate how the agent responds in specific scenario. However, according to the system’s requirements, if there are some actions need to be performed by an agent in some scenario, we make these actions either as the agent’s behaviors or as policies. If we make these actions as behaviors of agents, agents will not compute the policy and the system’s running efficiency will be reinforced. However, in such way, agents may require complex reasoning algorithms and communication capabilities, which will make the system less flexible. While making these actions as policies is more flexible, but more policies in the system will reduce the system’s running efficiency.

E. Implementation The other focus of the pattern proposed here is the system implementation architecture. For a more detailed structure, this section is subdivided into two subsections: Structure of agent describes the agent structure and the process that policy affecting the behaviors of agent. Implementation architecture describes the implementation framework of the whole system. 1) Structure of agent We propose an agent structure based on BDI structure [10]. It is composed of an Interface module, a State module and an Execution Module. The interface module consists of as follows: „ Event Engine is responsible for receiving deal with various events, for example environment events, role events, policy events and etc. each environment event is about the changes occurred in the agent’s environment. Role event is about either agent enacting or quitting a specific role. Policy event is the event about the policy deploying or abolishing. „ Belief Update component updates beliefs according to the events and current beliefs. „ Policy Engine evaluates policies according to the policy events and current state module, updates current policy belief and generates policy update events. „ Goal maintenance component receives belief update events and policy update events, treats with possible conflicts among the current belief, and produces goals. „ Plan Engine conducts means-end reasoning according to current goals and available plans in Plan Library. The State module is composed of Intention Set, Goal Set and Beliefs, in which beliefs are divided into policies and other beliefs. The Execution module contains an Execution Engine and actions of agent. Figure 1 shows the agent architecture. The process that policies affect the behaviors of agent is described as follows: „ The Event Engine observes the world and sends the events to the Belief update component and Policy engine respectively. „ The Belief Update component updates beliefs according to current beliefs and the events. „ According to current beliefs and related events, the Policy Engine evaluates policies and generates policy updates events and sends the events to Event Engine. „ After the Event Engine receives the update events, it sends them to the Goal Maintenance component.

C. Dependencies The problem in hand demands an autonomous solution that requires the decentralized coordination but global restrictions of multiple homogeneous and/or heterogeneous autonomous agents in order to achieve a globally system’s goal. The global restrictions could be changed at run-time. The agents in multi-agent system which have global policies should be: „ able to know what policy exists in the system. „ able to adopt policy implying on its own decisions and behaviors. „ able to deliberatively follow those policy in the agent’s behaviors. „ able to deliberatively violate a policy in case of conflicts with other policies or, for example, with more important personal goals. Of course, such an agent can also accidentally violate a policy (either because it ignores or does not recognize it, or because its behaviors does not correspond to its intentions). Moreover, to meet the dynamic requirements change and variety of environment, the policy may be dynamic and the changed policies may be perceived by agents. D. Entities The entities in policy based self-organization multi-agent systems are agents. Especially, we use the concept of role as the agent class [14]. Role is the abstract description of the right, duty, behaviors of the agents. The role in our pattern is different from the traditional one. The duties and rights of a role are specified by the policies that can be described at runtime. The policies are described by policy ontology [8]. They define the rights and the duties of a role. Agents can enact roles. During the process of self-organization, the agent will enact or quit a role with its desire and policy restriction. When an agent enacts a role, the rights of the role will be owned by the agent as will as the agent will responsible for the duties of the role. On the contrary, if an agent quits a role, it will lose the rights of the role and does not need to responsible for the duties of the role.

1574

„ „

„

high-level policy, transforming them into low-level implementable policies, and distributing these policies to underlying agents who are in charge of enforcing policies. Since roles of an agent and their rights and duties are comparatively static and the agents who undertake the roles are dynamic, the policy specification is role-oriented. Before policy distributor distributes policies, role-based policies are transformed to specific implementable policies through role assignment. There are five components in policy specification and deployment layer, including generic policy ontology, application specific ontology, policy specification service, policy repository, and policy distributor. The generic policy ontology defines the common ontology about subject, target, behavior, constraint, and policy. Application specific ontology is ontology about multi-agent system and domain concepts [19]. With these two ontologies, human administrator establishes polices through policy specification service. Policy Distributor takes charge of policy distribution.

The Goal Maintenance component receives the update events and treats with conflicts among them, and sends them to the Goal Set. The Plan Engine looply checks the Goal Set, takes out a goal, and generates a plan instance according to current beliefs and available plans in the Plan Library, then pushes the plan instance to an existing or new intention stack, according to whether or not the event is a sub-goal. The Plan Engine selects an intention stack, takes the topside plan instance, and drives the Execution Engine executing the next step of this plan instance.

F. Example We take a service-oriented application system as an example. Service-oriented applications have been researched for many years. However, the existing solutions for management of services are application-specific and can not flexibly adjust at run-time according to the high-level policy. In service-oriented applications, a main problem is service matching. There are often many different types of service providers and consumers in the system and how to match them in an effective way is a challenge. Moreover, the matched provider and consumer are not always unalterable. The requirement of service consumer will change during the run time. How to realize this kind of system is a challenge in current work. We consider this kind of system is suit with our pattern because it is hard to match the services by use of centralized control and the system requirements maybe changed during runtime. These properties are accord with the Problem item in our pattern. By our pattern, service providers and consumers are encapsulated as agent in the system and there is another kind of agents which take charge of the matching of the providers and consumers agents. There are two kind of role in the system: Task Agent (TA) role, and Management agent (MA) role. Service provider agents and service consumer agents enact the TA and matching agents enact the MA. Agent organizations are a group of agent which have one MA and a set of TA. TAs in the same organization must have same service (provide or consume) type so as to match each other and they can join one or more agent organizations for services transaction, interoperation or cooperation according policy. MA that takes charge of recruiting member agents, organizing cooperation of member agents according to recipes (static plan templates) defined in advance. MA or individual member agents locally enforce the policies from policy distributor. The distributor distributes policies according to subjects or targets of policies, with the support of organization information service (OIS). OIS manages the information of agents, including agents’ ID, potential

Figure 1. Agent implementation architecture

2) Implementation architecture The key issue in the policy-based self-organization multiagent systems is to specify the appropriate policy that can make the system exhibit the expecting self-organization process [8] . We need to construct a specific facility which can be used for deploying the policy by system user. Figure 2 shows the architecture of policy- based framework.

Figure 2 Implementation architecture of policy-based self-organization MAS

On top of agent layer, there is a policy specification and deployment layer. This layer mainly deals with representing

1575

activities, properties, locations, etc. Each agent has a local policy depository, a policy engine, and an agent mental state monitor. 1) Scenario 1: Organization Formation Supposing there is two kind of type of services (services 1 and services 2) provider and consumer agents in the system. MA 1 and MA 2 represents different service type matching agent, i.e. they need to recruit service consumer agent and service provider agent in its organization to realize the service matching according with its service type (see (a) of figure 3). We could constitute two polices to TA and MA respectively. „ TA: If the TA does not belong to any organization, the TA should request a MA at random. Then if the MA has the same type service with the TA, TA will register itself in this organization. But if the service type is different from the MA, the TA conforms to the MA’s recommending and register to the recommending MA. „ MA: If the MA is requested by a TA for registering, the MA compares its service type with the TA. If they have same type of services, the MA accepts the register request. If the type is not matched, the MA recommends the TA to the other MA which has the same service type with the TA. In this scenario, we assume that each MA knows the service type of others. Through the constituting of the two policies, the different services consumer agents and services provider agents are classified in different organization according to their service type. Especially, we distinguish different TA in different organization from TA 1 and TA 2. (see (b) of figure 3). 2) Scenario 2: change of services requirement Based on scenario 1, with the running of the system, a new user requirement is coming. According to the need of new requirement, service 1 consumer agent also needs the service 2. Therefore, we need to combine organization 2 with organization 1, i.e. the TA of organization 1 need to become the member of both organization1 and 2. We could constitute two polices to MA1 and MA2: „ MA1: MA1 check its OIS, if there still has TA which is not the member of organization 2, recommend this TA to MA2, and do not log-out its identity in organization 1. „ MA2: If there have service 1 consumer agent request for registering, accept the request. (see (c) of figure 3)

(a) Initialization

(b)

Scenario 1

(c) Scenario 2 Figure 2. Example of The Pattern

G. Konwn Use Examples of the policy pattern can be found in some various solutions: „ Autonomous Computing: As a method of management for autonomous system, PBM (Policy Based Management) have been used for the management of autonomic network and distributed systems, especially in encrypt, authorization access, resource control etc. [15] –[17]show these research works. „ Grid Computing: In Grid computing, policy was used for aggregation of computing resource. [18] realized a policy based method to drive the grid agent to offer their computing resource for the computing request. „ Web Service: The integration of web services and intelligent agents is promised for automated service discovery, negotiation, and cooperation. [19] models an ontology-based, policy-driven control framework that is suitable to supervise the dynamic agents according to high-level policies. H. See Also The policy based mechanism is a high-level mechanism that many self-organization mechanisms like Evaporation, Aggregation, and Diffusion [12] can be abstract as policy formation.

1576

IV.

RELATED WORKS

Although design patterns for the self-organization multiagent systems are presented for a while, relevant research works are yet rare. Gardelli [12] and Wolf [11]describe basic low level patterns common to various biological systems, such as replication, collective sort, evaporation, aggregation, and diffusion. On a similar level of abstraction, Babaoglu [20] describes further patterns of biological coordination, including plain diffusion, replication, stigmergy, and reaction-diffusion. A proper combination of some of these patterns may produce more complex patterns for selforganization emergent systems. These works simply describe the self-organization mechanisms and do not give implementation. In [21] an organic design pattern is presented for self-* systems that consist of a set of independent agents interacting with each other and where reconfiguration/ adaptation can be expressed as a reallocation of roles. However, a structured description of pattern is missing. V.

[8]

[9]

[10]

[11]

[12]

CONCLUSIONS

Engineering self-organization multi-agent systems should explicitly consider ways to coordinate multiple autonomous agents to achieve desired macroscopic properties. In this paper, we present a design pattern that facilitates a global restriction and dynamic adjustment the behaviors of agents by means of policy. The pattern captures the principles behind the policy promulgation in human society and describes them structurally to be used in a systematical engineering process. The policy pattern makes essential improvements to existing self-organization mechanism in terms of global restriction and dynamic adjustment of behaviors of agents. The known uses of corresponding design patterns in general are spares yet. The policy pattern thus can be considered to improvement in current work. REFERENCES [1] Jelasity M., O.Babaoglu, and.Laddaga R. “Introduction: selfmanagement through self-organization”. IEEE Intelligent Systems, Vol. 21, No. 2, 2006.pp 8-9. [2] Giovanna D.M.S., Marie P. G., Anthony K. “Selforganisation and emergence in MAS: an overview”. Informatics Vol.20, 2006, pp45-54,. [3] Jonathan M.E.Gabbai, Hujun Y. Wright W.A. “Selforganization, emergence and multi-agent systems”. International Conference on Neural Networks and Brain, 2005 pp. 13-15. IEEE Press. [4] Giovanna D. M. S., Noria F., Salima H., Anthony K., Soraya K. M., Omer F. R., Mihaela U.,etl. “Self-organisation: paradigms and applications”. J.G.Carbonell (edit.) LNAI, vol. 4128, 2006, pp. 1148-1158. Springer, Heidelberg. [5] Camazine S., Deneubourg J., Frank R.N., Sneyd J., Theraulaz G., Bonabeau E. “Self-organization in biological systems”. 2001,Princeton University Press, USA. [6] Dignum F. “Autonomous agents with norms”. Artificial Intelligence and Law, 1999,Vol. 7,pp 69-79. [7] Tonti G., Bradshaw J.M., et al. “Semantic web languages for

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

1577

policy representation and reasoning: a comparison of KAoS, Rei, and ponder”. Proc. Second International Semantic Web Conference. LNCS Springer, Heidelberg Vol.2870, 2003, pp 419-437. Beishui Liao, et al. “Ontology-based conceptual modeling of policy-driven control framework: oriented to multi-agent system for web services management”. In Proceedings of AWCC 2004, Springer-Verlag, 2004, pp. 246-356. Feltovich, et al. Order and KAoS. “Using policy to represent agent cultures”. In Proceedings of the Autonomous and Adaptive Multi-Agent Systems ( AAMAS 03) Workshop on Humans and Multi-Agent Systems. Australia. 2003. Beishui Liao, Huaxin Huang, JI Gao. “An extended BDI agent with policies and contracts”. PRIMA 2006. LNAI, Spring-Verlag, Vol 4088, 2006, pp 94-104. Tom D. W. and T. Holvoet. “A catalogue of decentralised coordination mechanisms for designing self-organising emergent applications”. Department of Computer Science, K.U.Leuven, Belgium. Report CW 458, 2006. L. Gardelli, M. Violi, and A. Omicini. “design patterns for self-organizing multiagent systems”. In Proceedings fo CEEMAS 2007, Leipzig, German, LNCS, Springer-Verlag Vol.4696. 2007, pp 123-132. J.Lind. “Patterns in agent-oriented software engineering”. 3rd International workshop on Agent Oriented Software Engineering, Springer-Verlag, 2002,pp. 23-34. Xinjun Mao, Eric Yu. “Organization and social concepts in agent oriented software engineering”. In Agent-Oriented Software Engineering 2004, Springer-Verlag, Berlin 2004 ,pp 1-15. Chadha, R. “Policy-based mobile Ad Hoc network management”. Proceedings of the 5th IEEE International Workshop on Policies for Distributed Systems and Networks, IEEE Press, New York, 2006,pp. 181--194. Uttamchandani, S., Alvarez, G.A. and Agha, G.: DecisionQos. “An adaptive, self-evolving Qos arbitration module for storage system”. Proceedings of the 5th International Workshop on Policies for Distributed Systems and Networks, 2006. Firozabadi, B. S.A. “Framework for contractual resource sharing in coalitions”. Proceedings of the 5th IEEE International Workshop on Policies for Distributed Systems and Networks, 2006. Bei-shui Liao and Ji Gao. “An automatic policy refinement mechanism for policy-driven grid service systems”. Grid and Cooperative Computing 2005, LNCS Springer-Verlag, Vol. 3795, 2005, pp 166-171. Beishui Liao, Ji Gao Jun Hu and Jiujun Chen. “Ontologybased conceptual modeling of policy-driven control framework: oriented to multi-agent system for web services management”. AWCC 2004, LNCS Springer-Verlag Vol. 3309, 2004, pp 346-356. O. Babaoglu, G. Canright, A. Deutsch, G. A. D. Caro. F. Ducatelle, L. M. Gambardella, N. Ganguly, M. Jelasity. “Design patters from biology for distributed computing”. ACM Transactions on Autonomous and Adaptive Systems. Vol. 1, 2006, No. 1, 36-66. M. Gudemann, F. Nafz, F. Ortmeier, H. Seebach, and W.Reif. “A specification and construction paradigm for organic computing systems”. In Proccedings of Self-Adaptive andSelf-Organization Systems 2008 (SASO 2008), IEEE Computer Society, New York, 2008 pp.128-134.