Specifying Laws in Open Multi-Agent Systems - CiteSeerX

Specifying Laws in Open Multi-Agent Systems Rodrigo Paes1 , Gustavo Carvalho1 , Carlos Lucena1 , Paulo Alencar2 , Hyggo Almeida3 , and Viviane Silva1 1

Puc-Rio, Rio de Janeiro, Brazil {rbp, guga, lucena, viviane}@inf.puc-rio.br 2 University of Waterloo, Waterloo, Canada [email protected] 3 UFCG, Campina Grande, Brazil [email protected]

Abstract. Laws have been used to deal with theoretical and practical issues regarding notions of openness and unpredictably of open multiagent systems. In order to achieve a higher degree of predictability and confidence, practical applications of distributed agents need to describe the interaction structure of the system, and this interaction structure must be regulated. Governing agent interactions can lead to higher degree of system trust. This paper uses the notion of interaction laws as abstraction to regulate agents’ interaction. In this paper, we introduce a conceptual model for developing laws in open multi-agent systems. This model is composed of static, dynamic and formal definitions. We also depict a declarative language used for supporting the conceptual model. Moreover, we show a software implementation that allows the enforcement of laws through the interception of agents’ interaction. Finally, the conceptual model is applied in a case study in the domain of flight ticket trading.

1

Introduction

Laws are rules governing the behavior of a system [1]. Such kind of approach has gained increasing importance in the field of distributed systems [2, 3]. These systems are frequently composed of components that can enter and leave from the system at any time; and they can have been developed using different languages and architectures. In this sense, those systems have much of the features attributed to open multi-agent systems (open MAS) [4, 5], and we use the agent abstraction to refer to this sort of distributed system. The global behavior of open MAS emerges from the interaction of the agents, and therefore, it is not totally predictable. Although this kind of emergent behavior is acceptable or even desirable, in some applications unexpected behavior may lead to system faults [6]. It is particularly true in domains such as electronic commerce, supply chain and health care.

2

Rodrigo Paes et al.

To deal with this scenario, distributed software systems are viewed as a set of communicating agents whose interaction is based on precise specifications named laws. The conventional approach to implement laws is to hard-code them into all members of a multi-agent system. Separating the law from the agents facilitates the dynamical change of the distributed system’s behavior, and enables reuse of experts’ knowledge. In this paper, we propose a conceptual model used to specify laws in open multi-agent systems, where laws [7] are used as abstraction to specify policies that regulate the agents’ interactions. This model includes many useful elements for regulating the interaction among agents and it also considers time aspects to specify laws. Moreover, it is introduced to support our approach a software middleware for governing laws. Such middleware is able to mediate agent’s interaction and verify if the interactions are fulfilling the laws. The specification of laws is performed using a declarative language. The remainder of this paper is organized as follows. Section 2 describes the law governed approach. In Section 3, we present the conceptual model including its static, dynamic and formal definitions. In Section 3.4, we present a declarative language used for representing the concepts introduced in Section 3. We also explain the software developed to support our approach in Section 4. Section 5 illustrates the applicability of this proposal in a case study. Some related works are discussed in Section 6. Finally, discussions are presented in Section 7.

2

The Law Governed Approach

The law governed approach is used to build open multi-agent systems where a certain degree of control over agents’ behavior is desirable. Although no agreement exists on the exact definition of agent, there is a consensus about the importance of at least two characteristics: agents are autonomous[4] and have social ability[5]. Autonomy is the agent’s ability to (i) act without external intervention, (ii) say yes or no in response to another agent request, and (iii) initiate an action selfmotivated [8]. Social ability is the ability to interact with other agents through some communication language [9]. Governing interactions affects the agent’s autonomy, because it regulates agent’s actions by laws. So, laws can be seen as a boundary that establishes limits for agent’s autonomy. Laws act as a regulatory system controlling the behavior of agents. In an open MAS, the internal aspects of agents are inaccessible. The only accessible information about agents is their observable behavior from the exchange of agents’ messages. We argue that a law governed mechanism should intercept those messages, and then, enforce the desired system behavior. This mechanism acts as a mediator among agents. This mediator is not necessarily neither one unique entity or centralized. If the mediator is centralized, some degree of scalability can be achieved using a pool of mediators. However, in this case, we still having a unique point of failure. Another solution is to think of mediator as many decentralized

Governing Open Multi-Agent Systems

3

entities that monitor the system execution. In this sense, depending on how mediators are implemented (centralized or decentralized), higher or lower level of scalability can be achieved. The mediator idea was already reported, with minor variations, in a number of publications such as controllers [10], governors [11], and inter agents [12]. The mechanism is responsible for providing the basic communication components and the interoperability concerns. It could be necessary to extend this basic infrastructure to attend system requirements. Our law governed mechanism life cycle consists of three high-level activities (Figure 1): (i) interception; (ii) enforcement; and (iii) redirection. In the first stage, the mechanism intercepts the messages exchanged between agents. Then, the mechanism interprets the law specifications and check if the message complies with the laws. If so, the mechanism redirects the message to the real addressee. In this sense, all the observable behavior of agents can be controlled and the laws can be enforced. Agent A

Agent B

Communication

1

Interception

3

Redirection

Law Enforcement Mechanism 2

Enforcement

Fig. 1. Law Governed Interaction Model

3

Conceptual Model

To further discuss the law governed approach, it is necessary to introduce the elements that compose our approach. In this sense, this section aims to provide the conceptual base describing a Structural Model, Dynamic Model, the formalization of relationships among elements, and an example of a specification language based on these concepts. 3.1

Structural Model

We propose a conceptual model for laws (Figure 2) and represent it by using the UML notation. This diagram shows the laws elements and their relationships. The starting point of this model is the LawOrganization element. The LawOrganization element is not the multi-agent organization itself; indeed, it represents the laws of a multi-agent organization. This element contains Scenes, Norms and Clocks. Scenes specify agents and agent roles that may interact while the scene is running. By agent, we mean any autonomous, interactive entity that communicates through message exchanges. These agents usually play roles, which is

4

Rodrigo Paes et al. entrance/creation

LawOrganization

Scene

0..*

Agent

0..*

sends

Message -template: String

0..*

1..*

0..*

Clock

State 1..*

-type: String -tick-period: long

0..*

requiredNorms 0..* 0..*

Role

entrance/creation

Norm 0..*

-type: String -label: String 1 to

0..* 1

Protocol

-owner: Agent or Role

1

plays

message-ref

1 from

Transition 0..*

activeNorms

Fig. 2. Law Conceptual Model

an abstraction of recurrent agent behavior, linked to an interaction context in a scene. Moreover, scenes are composed of Protocols, Clocks, and Norms. It means that these three elements share a common interaction context through the scenes. Every scene specifies one protocol. Because protocols define the interaction among the agents, different protocols should be specified in different scenes. Protocols represent the interaction patterns between the agents; they define the valid interactions that agents can have. We represent protocols as nondeterministic automatons, where states represent snapshots of the protocol execution and transitions are the link between the states. Finite automatons have a set of symbols that they are able to process. In the context of protocol specification, those symbols are messages exchanged between the agents, and these messages might fire transitions. Each transition has an attribute called messagePattern. It means that the transition only fires if the received message matches the specified pattern in the attribute messagePattern. Norms prescribe how the agents are allowed or must behave and what their rights are. In other words, they prescribe the permissions, obligations and prohibitions of agents. As a consequence of the relationship between norms and transitions, it is possible to specify which norms must be active for firing a transition. For instance, a transition should only fire if the sender agent has a specific permission. Many scenes might compose a law, and each scene defines a specific execution context. That context defines scopes where norms, clocks and protocols are valid. It means that active norms are only visible in the context where they were created. There are two levels of contexts: organization and scene context. Norms specified inside the organization context (LawOrganization element) are visible to all scenes, and norms specified inside a scene are only visible to that scene. Time Aspect in Laws Time aspects of a system are relevant issues in the context of many classes of important systems, such as real-time, simulators,

Governing Open Multi-Agent Systems

5

electronic commerce, and interactive systems. To the best of our knowledge, other law governed approaches [13–15] have not addressed time aspects in law specification. We argue that laws must be time sensitive. In other words, although an agent may be able to perform an action a1 at time t1 , it might not be able to perform the same action at time t2 (t1 < t2 ). Our conceptual model provides the Clock element acting as a timing aspect. The clock element is able to generate events at specific time intervals. For example, a clock allows the activation of norms after a certain amount of time has elapsed. A transition firing or a norm activation might also activate clocks. A new clock instance is created each time a clock is activated. It means that the clock will start counting the time from that beginning and independently of other active clocks. Once activated, a clock can generate clock-tick events or deactivate itself. Besides, it is allowed to have many simultaneous clocks. 3.2

Dynamic Model

In this section, we describe the behavior of the elements that compose a law specification. Events are the basis of the communication among law elements, that is, law elements dynamically relate with other elements through event notifications. Basically, we can understand the dynamic of the elements as a chain of causes and consequences, where an event can activate a law element; this law element could generate other events and so on. For example, the arrival of a message activates a transition, which activates a clock, which generates a clock tick, and the clock tick activates a norm. Figure 3 shows all the elements which are able to generate and sense events.

LawElement

Transition

Clock

Norm

generates/senses

Protocol

Scene

Event

Message

Fig. 3. Event Generators

There is a close relationship between law elements and events. Events are generated during the life cycle of law elements. Clocks generate a clock activation event when the clock starts working; clock tick events when the time specified in the clock specification is elapsed; and deactivation events when the clock stops working. Norms also generate activation and deactivation events; however, it is important to notice that their activation event are norm activation events.

6

Rodrigo Paes et al.

Transitions generate only transition activation events. Transitions are part of protocols, and therefore, a transition for a message arrival will only be notified if the current protocol execution state reaches the transition. Furthermore, a transition activation event is fired only if two other conditions are satisfied: the message that originated the message arrival event must satisfy the message pattern; and all the required norms must be enabled. Scenes generate scene completion events when their protocol reaches a final state. If a final state is reached many times during a scene execution, many scene completion events are generated. 3.3

Formal Law Structure Definition

Our law governed conceptual framework is formally defined as below. This formalization is mapped to the XMLaw language in the next section (Section 3.4). For example, the Org element is mapped to the XML tag < LawOrganization >, the Scni is mapped to the XML tag < Scene >, and so on. – Org = hScn, N i, where Org stands for organization, Scn is a finite set of scenes, N is a finite set of norms and C is a finite set of clocks; – Scni = hR, P, C, N, κ, Agi, where: Scni ∈ Scn, i ∈ N; R stands for the operator requires meaning a finite set of all the norms that should be active in order to the scene begin, P stands for protocol, C is a finite set of clocks, N is a finite set of norms, κ is a finite set of roles that interact in this scene, and Ag is a finite set of agents that interact in this scene; – κi = hidi, where where κi ∈ κ, i ∈ N, and id is a identifier for the role; – Agi = hidi, where Agi ∈ Ag, i ∈ N, and id is a identifier for the agent; – P = hS, T i, where S is a set of states and T is a set of transitions; – Si , where Si ∈ S, and i ∈ N; – Ti = hF, D, τ, Ai, where Ti ∈ T , i ∈ N, F stands for source state, D stands for destination state, F ∈ S, D ∈ S, τ is a message template, and A is a set of norms that must be active; – Ci = hδ, π, α, θi, where Ci ∈ C, i ∈ N, δ stands for clock type, π stands for clock period, α represents the events that activate the clock, and θ are the events that deactivate the clock; – Ai = hρi, where Ai ∈ A, i ∈ N, and ρ stands for references to norms; – Ri = hρi, where Ri ∈ R, i ∈ N, and ρ stands for references to norms; – Ni = hΘ, Γ, Ωi, where Ni ∈ N , i ∈ N, Θ stands for a finite set of obligations, Γ stands for a finite set of permissions, and Ω stands for a finite set of prohibitions; – Θi = hα, ξi, where Θi ∈ Θ, i ∈ N, α represents a finite set of elements that activate the obligation, and ξ is a finite set of events that deactivate the obligation; – Γi = hα, ξi, where Γi ∈ Γ , i ∈ N, α represents a finite set of elements that activate the permission, and ξ is a finite set of events that deactivate the permission;

Governing Open Multi-Agent Systems

7

– Ωi = hα, ξi, where Ωi ∈ Ω, i ∈ N, α represents a finite set of elements that activate the prohibition, and ξ is a finite set of events that deactivate the prohibition; – E = hnamei, where E stands for event, and name is the name which identifies the event. 3.4

Textual Law Representation: XMLaw

Based on our conceptual model and on its formal definition, we developed a declarative language for law specification. That language is based on XML. In this section, we explain the main elements of the language and provide some explanations and comments on how to use them. LawOrganization : the laws of an organization are specified inside the LawOrganization element. The id attribute is used to uniquely identify this element. The name attribute allows humans to identify the organization more easily because it is a description. Furthermore, the organization has the Scenes element that represents all the possible scenes of an organization, as well as the set of norms. 1 2 3 4 5 6

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

. . . . . . . . . < !−− A l l p o s s i b l e messages p a t t e r n s −−>

< !−− A l l t h e norms f o r t h i s s c e n e −−>

8

Rodrigo Paes et al.

Requires : the Requires element contains a reference to some norms specified inside the Norms element in the organization. It does not make sense to use a reference to some norm inside the scene itself because the Requires element is a precondition to start the scene execution, but if the norm belongs to the Norms element inside the scene specification, the norm can be only activated during the scene execution. Therefore, the scene will never start. 1 2 3



Messages : the Messages element specifies all the templates of messages used in the scene. This template comprises the fields performative; sender, which specifies both the agent that is sending the message and its respective role; receiver, which defines both the addressee agent and its role; and finally the content of the message being transmitted. The protocols use these templates to specify transitions. 1 2 3 4 5



Protocol : the Protocol element is a type of non-deterministic automaton, and it is composed of only two attributes: a set of states and a set of transitions. 1 2 3 4

. . . . . .

State : the id attribute allows the law developer to make references to the state. In this way, transitions can connect one state to another. Moreover, a state has a label and can be the initial or the final state of the protocol. 1 2



Governing Open Multi-Agent Systems

9

Transition : transitions connect two states. A transition is activated when the protocol is at the state from and if a message that matches the pattern specified by aMessageId arrives. It is important to notice that the attributes from and to are references to states specified in the States element, and the message-ref is also a reference to a message pattern specified in the Messages element. Furthermore, transitions can also have the ActiveNorms element representing all the norms that must be active in order to the transition been activated. 1 2 3 4 5 6



Clock : clocks can be of two types- regular or periodic. Regular means that it will generate only one clock-tick after a tick-period of milliseconds have been elapsed. Periodic clocks generate clock-ticks at the interval specified at the tickperiod attribute. Clocks are activated and deactivated by law elements. The Activation and Deactivation elements represent these activations and deactivation conditions. They both have reference to other law elements, and they have two attributes: ref and event-type. The ref is a reference to an id attribute and the event-type is the type of event generated by the referenced element. It is necessary because some elements generate more than one type of event, then; with the event type attribute it is possible to specify exactly what event should start the clock. 1 2 3 4 5 6 7 8



Norm : norms can contain three types of elements - Permission, Obligation and Prohibition. In the presented language, the structure of these three elements is equal. Each type of norm contains activation and deactivation conditions. These conditions have the same semantic of the clock’s conditions. 1 2 3 4 5 6 7 8 9 10 11 12

. . .

. . .

10

Rodrigo Paes et al.

4

Software Support

Development of multi-agent systems requires appropriated techniques, models and tools [16]. Tools, in particular, play an important role in the development scenario helping to build multi-agent systems faster and less error prone. Recognizing the importance of these aspects, we developed an object-oriented framework and a monitoring tool supporting the law governed approach. In the proposed framework, agent developers have access to the API (Application Program Interface) for Law Governed which provides a basic agent communication component and other facilities. They can use this API to create their agents and interact with the law governed mechanism. Law developers can extend our framework to adapt it to their needs. Each message format must have a message pattern recognizer associated with it. Declaration of message patterns is necessary for protocol specification. Our framework provides a prolog like message pattern. It means that agents can interact with each other using prolog as the content language. Another framework’s hotspot is the communication mechanism. Currently, it is offered a default implementation for the communication this hot-spot based on the Jade platform [17]. In this way, the Jade platform is used just as a message transport layer among the agents. However, developers are also free to use any other communication infrastructure, implementing the communication interface provided by the framework. A monitoring tool provides graphical support for monitoring of laws’ execution. In this tool, it is possible to visualize all the scenes in execution, which norms and clocks are active, and also what is the current protocol execution’s state.

5

Case Study

Nowadays large airports do more than just serving as a place for airplanes landing and taking off. Aiming to increase the customer satisfaction, the fictitious airport of this case study provides an Internet based system where users can buy flight tickets. This system is based on an auction mechanism, and the airline companies placed in the airport dependencies integrate their own systems to it. The airport system should provide a good level of trust in the sense that the costumers should be protected against malicious behavior of airline companies and airline companies should be also protected against malicious users. However, the airline companies’ systems are not available to the airport development team, and therefore, they cannot blindly trust in the airline companies systems’ behavior. In this example, the whole process consists of three stages: negotiation, payment, and ticket emission. First, the customer exposes its flight preferences, such as destination and cabin type, and waits for the airline company’s bids. After that, the costumer chooses one of the bids. The second stage starts when the negotiation stage ends. The costumer should follow the required procedures and

Governing Open Multi-Agent Systems

11

carry the payment out. In the last stage, the airline company emits an electronic flight ticket to the costumer. 5.1

Applying the Conceptual Model

According to the scenario description, we can identify two participant roles: customer and airline company. During the negotiation, customers may get irritated if the airline companies take too long for providing their bids. In that way, we propose that 10 seconds is the maximum time for airline companies proposing their bids. Furthermore, if an airline company is negotiating a flight ticket with a customer, then it cannot sell the same ticket for another costumer. On the other hand, the airline companies can also have financial losses in cases where the user takes too long to choose among the bids, so, the user should also have a time-out to decide, for instance, 5 minutes. In this case study, we have a negotiation phase, a payment phase and we conclude the process with a ticket emission phase. Each phase corresponds to a scene. Each scene is analyzed and designed. We have made two simplifications on the problem. First, it is described only part of the negotiation scene, and second, it is considered the negotiation process between one agent and one airline company. The scene begins when a costumer sends a request of flight ticket proposal to the airline company through a request message. In this point, the airline company has to answer within 10 seconds; otherwise, the costumer can send a cancel message and leaves the scene. Once the airline company accepted the request on time, the customer has 5 minutes to decide about, and to send an accept-proposal or a reject-proposal message. After that, the airline company informs if the sale was well succeed or if some problem happened. That sequence of interaction is the basis of the protocol element. We specify this protocol using declarative law language (Listing 1.1). In the line 1, we specify the organization element. The line 2 delimits the beginning of the negotiation scene specification. This scene contains a protocol specification (lines 07 to 21). This protocol contains three states (lines 08 to 12) and two transitions (lines 13 to 20). The transition t1 links the states s0 and s1 and it is activated by the request message, specified in the line 4. When this transition is activated, the clock specified in the line 23 is also activated. This occurs because the activation attribute in the line 25 has a reference to the transition t1. This clock addresses the non-functional requirement response-time-1. It counts 10 seconds and fires a clock-tick event. The permission cancel-request (line 30) has specified in its activation condition (line 32) that clock-ticks events originated by the clock The response-time-1 activate this transition. The transition t2 (line 15) links the state s1 to the state s3, and it is activated by the cancel message - m2 (line 5). However, the norm cancelrequest must be active for enabling this transition (line 17). Although we do not specify the whole scene, the remaining specification is similar to the provided specification.

12

Rodrigo Paes et al.

The execution of the laws is monitored and this monitoring helps to achieve a better understanding of the system. For instance, in this case study, the monitoring can identify that many customers take more than the specified 5 minutes to decide about the bid. This might indicate that the initial time is not sufficient or that the customers are having difficulties to use the system. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

Listing 1.1. Law Description

6

Related Work

In Esteva [11] approach, scenes are similar to the protocol elements proposed in this paper. Both Esteva scenes and protocol elements specify the interaction protocol using a global view of the interaction. It means that all the interaction among the agents is specified in only one protocol in opposite to individual agent views, where many partial views of the protocol (one of each agent) are specified. The time aspect is represented in Esteva approach as timeouts. Timeouts allow

Governing Open Multi-Agent Systems

13

activating transitions after a given number of time units passed since a state was reached. On the other hand, due to our event model, the clock element proposed in this paper can both activate and deactivate not only transitions, but also other clocks and norms. Connecting clocks to norms allows a more expressive normative behavior; norms become time sensitive elements. OMNI [18] is a framework for modeling agent organizations. This framework is composed of three dimensions: normative, organizational and ontological. These dimensions aim to cover from analysis to implementation of agent organizations. In the normative dimension, developers specify the mechanisms of social order, in terms of common norms and rules, which members are expected to adhere to. The organizational dimension describes the structure of an organization. The ontological dimension defines environment and contextual relations and communication aspects in organizations. In addition, each one of these dimensions can be viewed in three abstraction levels: abstract, concrete and implementation. In the abstract level, the general organization goals are defined in a high level of abstraction. It also contains the definition of the ontology of the model itself. Based on the abstract level, norms, rules, roles, interaction protocols and concrete ontological concepts are defined. The implementation level assumes a given multi-agent architecture as basis for the implementation of the organizational model, and also mechanisms for role enactment and norm enforcement. Minsky [19, 20] proposes a coordination and control mechanism called law governed interaction (LGI). This mechanism is based in two basic principles: the local nature of the LGI laws and a decentralization of law enforcement. The local nature of LGI laws means that a law can regulate explicitly only local events at individual home agents, where home agent is the agent being regulated by the laws; the ruling for an event e can depend only on e itself, and on the local home agent’s context; and the ruling for an event can mandate only local operations to be carried out at the home agent. On the other hand, the decentralization of law enforcement is an architectural decision argued as necessary for achieving scalability. Furthermore, it provides a language to specify laws and it is concerned with architectural decisions to achieve a high degree of robustness. In contrast, our approach provides a explicit conceptual model and focuses on different concepts such as Scenes, Norms and Clocks.

7

Conclusion

In this paper, we presented a conceptual model for specifying laws in open MAS. This model includes static, dynamic and formal definitions and integrates concepts of norms, interaction protocols, and time, using a flexible event based dynamical model. We also presented a declarative language, XMLaw, used for supporting the conceptual model. XMLaw is based on XML and was designed to be simple and even expressive. We briefly described a software implementation that allows the use of laws through the interception of agents’ interaction. Finally, the approach was exemplified through a case study.

14

Rodrigo Paes et al.

The identification of a group of static definitions permits the discussion of concepts regarding their structural semantics and their relationships. The identification of events as dynamic elements has facilitated the integration of static concepts, because events reify the relationships described in the structural definitions. However, we know that this conceptual model is still under construction. We aim to improve this version by implementing more experiments and by reflecting elements gathered from other related work. Despite the contributions of this work, there are many further research topics such as role assignment [21], and the integration of deliberative normative agent architectures [22] with our approach. The former is concerned with which agent is able to play certain organizational roles, and when they can play the role. In this paper, it is assumed that roles were already assigned and the information related to the agent role contained in the messages is trustful. A future work is the development of tools that help the specification and validation of laws in a graphical way. Furthermore, a proposal to improve this approach by dealing with laws in different abstraction levels is under development. Besides, we are researching how to represent this conceptual model using an approach based on ontology to easier the interpretation of laws by software agents. Another important issue that we are researching is the relation between laws and contracts. We claim that the proposed approach is suitable to deal with the unpredictability of open systems by taming agents’ autonomy through the enforcement of interaction laws.

References 1. Sloman, M.: Policy driven management for distributed systems. J. Networks Syst. Manage. 2 (1994) 2. Ao, X., Minsky, N., Nguyen, T.D.: Hierarchical policy specification language, and enforcement mechanism, for governing digital enterprises. In: 3rd International Workshop on Policies for Distributed Systems and Networks (POLICY’02). (2002) 3. Feeney, K.C., Lewis, D., Wade, V.P.: Policy based management for internet communities. In: POLICY ’04: Proceedings of the 5th International Workshop on Policies for Distributed Systems and Networks (POLICY’04), IEEE Computer Society (2004) 4. Jennings, N.R.: Agent-Oriented Software Engineering. In: Proceedings of the 9th European Workshop on Modelling Autonomous Agents in a Multi-Agent World. Volume 1647., Springer-Verlag (1999) 1–7 5. Woolridge, M.: Introduction to Multiagent Systems. John Wiley & Sons, Inc. (2001) 6. Sommerville, I.: Software Engineering. 7 edn. Addison-Wesley (2004) 7. Jones, A.J., Sergot, M.: On the characterisation of law and computer systems: The normative systems perspective. In Meyer, J.J., Wieringa, R., eds.: Deontic Logic in Computer Science: Normative System Specification. John Wiley and Sons, Chicester (1993) 275–307 8. Huhns, M.N., Singh, M.P.: Agents and multiagent systems: Themes, approaches, and challenges. In Huhns, M.N., Singh, M.P., eds.: Readings in Agents. Morgan Kaufmann (1997) 1–23

Governing Open Multi-Agent Systems

15

9. Wooldridge, M.J., Jennings, N.R.: Intelligent agents: Theory and practice. The Knowledge Engineering Review 10 (1995) 115–152 10. Murata, T., Minsky, N.H.: On monitoring and steering in large-scale multi-agent systems. In: Selmas’03 2nd International Workshop on Software Engineering for Large-Scale Multi-Agent Systems, Portland, Oregon (2003) 11. Esteva, M.: Electronic Institutions: from specification to development. PhD thesis, Institut d’Investigaci en Intel.ligncia Artificial, Catalonia - Spain (2003) 12. Martn, F.J., Plaza, E., Rodriguez-Aguilar, J.A.: An infrastructure for agent-based systems: an interagent approach. International Journal of Intelligent Systems (IJIS) (1999) 217–240 13. Castelfranchi, C., Dignum, F., Jonker, C.M., Treur, J.: Deliberative normative agents: Principles and architecture. In: Agent Theories, Architectures, and Languages. (1999) 364–378 14. Dignum, F.: Abstract norms and electronic institutions. In: Proceedings of International Workshop on Regulated Agent-Based Social Systems: Theories and Applications (RASTA’02), at AAMAS, Bologna, Italy (2002) 15. Vazquez-Salceda, J., Dignum, F.: Modelling electronic organizations. In: MultiAgent Systems and Applications III. Springer Verlag, Berlim (2003) 16. Choren, R., Garcia, A., Lucena, C., Griss, M., Kung, D., Minsky, N., Romanovsky, A., Castro, J., de Lemos, R., Weyns, D.: Software engineering for large-scale multiagent systems - selmas 2004: workshop report. SIGSOFT Softw. Eng. Notes 29 (2004) 1–10 17. Bellifemine, F., Poggi, A., Rimassa, G.: Jade: a fipa2000 compliant agent development environment. In: Proceedings of the fifth international conference on Autonomous agents, ACM Press (2001) 216–217 18. Vzquez-Salceda, J., Dignum, V., Dignum, F.: Organizing multiagent systems. Technical report, Institute of Information & Computing Sciences (2004) 19. Minsky, N.H., Murata, T.: On manageability and robustness of open multi-agent systems. In Lucena, C., Garcia, A., Romanovsky, A., Castro, J., Alencar, P., eds.: Software Engineering for Multi-Agent Systems II: Research Issues and Practical Application. Volume 2940/2004. Springer-Verlag Heidelberg (2004) 189–206 20. Minsky, N.H., Ungureanu, V.: Law-governed interaction: a coordination and control mechanism for heterogeneous distributed systems. ACM Trans. Softw. Eng. Methodol. 9 (2000) 273–305 21. Dastani, M., Dignum, V., Dignum, F.: Role-assignment in open agent societies. In: Proceedings of the second international joint conference on Autonomous agents and multiagent systems, ACM Press (2003) 489–496 22. Kollingbaum, M.J., Norman, T.J.: Norm adoption in the noa agent architecture. In: Proceedings of the second international joint conference on Autonomous agents and multiagent systems, ACM Press (2003) 1038–1039