Controlled Experimentation with Agents—Models and Implementations

Controlled Experimentation with Agents — Models and Implementations Mathias R¨ohl and Adelinde M. Uhrmacher University of Rostock, Department of Computer Science and Electrical Engineering, Albert-Einstein-Str. 21, D-18059 Rostock, Germany phone: +49 381 498 3329 {mroehl,lin}@informatik.uni-rostock.de

Abstract. The deployment of multi-agent systems demands for justified confidence into their behaviour, both with respect to correct results of computations and with respect to timeliness thereof. Depending on the stage of the development process different mechanisms and abstractions are needed to facilitate the rigorous evaluation of interacting agents. We propose a modelling and simulation framework for supporting the development process of multi-agent systems; from specification and implementation. Therefore, experimental set-ups are needed that allow an incremental refinement of agents while providing rigorous observation facilities. The benefit of using modelling and simulation for evaluating cooperative agents is illustrated using a simple example based on the Contract Net Protocol.

1

Introduction

Approaches for developing agents have matured to a broad spectrum of methods. Design methodologies support the development of agents by an agent-oriented development process [3] or providing modelling languages enriched with agent concepts [17]. Formal analysis of agents based on logics are particularly aimed at rational agents and pruned for reasoning about changing beliefs [24]. Other approaches suggest to develop communities of agents by defining social norms and regulations [19]. In contrast to static techniques, testing activities are dynamic analysis methods that require the execution of software. Testing concentrates on the validation of an implementation against a specification and thereby complements software design and static analysis. Developing agents is of an intrinsically experimental and exploratory nature: “the development of any agent system — however trivial — is essentially a process of experimentation” [28]. Surprisingly, only little work has been done so far on developing methods for testing agents [5]. The current methodologies and tools for MAS concentrate on supporting the design and implementation of agents, leaving a gap between specifications and implementations [10]. For exploiting the potential of MAS formal design methodologies and methods for observing and evaluating emergent behaviour have to be brought together

2

Mathias R¨ ohl and Adelinde M. Uhrmacher

by a rigorous experimental approach allowing for consistent observation of MAS [15]. Pursuing a simulation-based approach for evaluating agents, our work is addressing this deficiency.

2

Experimentation with Multi-Agent Systems

To construct agents essentially means to develop software that is able to successfully accomplish specified tasks in a certain environment [27]. Consequently, verification of agent behaviour has to take into account the conditions under which the agent is intended to work correctly, i.e., testing has to treat agents as systems that frequently interact with their environment. The usage of a virtual environment in contrast to the real environment typically reduces costs and efforts and allows to test a system’s behaviour in “rare event situations”. Virtual environments are easier to observe and to control, and probe effects are easier to manage. “For software engineers, virtual environments offer a powerful means of integration and systems testing” [12]. Environment models can be used to generate the different test cases dynamically during simulation, including specific interaction patterns and time constraints [21]. As testing cannot show the absence of faults but only their presence [16], the validity of the environmental models will be crucial, independently whether abstract models of agents are experimentally evaluated [26], single agent modules are embedded for testing [14], or entire agent systems are plugged into the virtual environment [18]. Simulation is an experiment performed on a model and “a model M of a system S and an experiment E is anything to which E can be applied in order to answer questions about S” [13]. This definition of model emphasises that a model is not developed for a system “per se”, but always for a combination of a system to be analysed and questions to be asked. According to this definition, multiple objectives require multiple models. The concept of experimental frame has been introduced to model explicitly experimental assumptions and system’s requirements [30]. Developing an experimental frame for testing agents requires a modelling formalism that is sufficiently expressive to model complex dynamic environments for agents and able to express different kinds of timing requirements. 2.1

Discrete-Event Modelling and Simulation

Devs (Discrete Event System Specification) is one of the formal approaches of discrete event modelling and simulation stemming from general systems theory [29]. It provides a powerful basis for modelling test settings because it is able to encode many other modelling formalisms like statecharts and petri nets [25], e.g. Giambiasi et al. made concrete use of this capability by transforming timed input/output automata specifications into a Devs simulation model [9]. Devs distinguishes between atomic and coupled models. An atomic model is described by a set of input ports, a state set, a set of output ports, an internal and external transition function, an output function, and a time advance function.

Controlled Experimentation with Agents — Models and Implementations

3

The internal transition function dictates state transitions due to internal events, the time of which is determined by the time advance function. At an internal event, the model produces output. The external transition function is triggered by external inputs. Networks (alias coupled models) support the hierarchical, modular construction of models. Its interface equals that of an atomic model. It is described by a set of component models, which may be atomic or coupled, and by the couplings that exist among the components, and between the components and its own input and output ports. In simulation, we distinguish between physical time, simulation time, and wall clock time. Whereas simulation time and physical time are connected by a semantic relation, e.g. one tick in simulation time refers to one minute in physical time, wall clock time is not necessarily related to either of both. Execution is normally done in an unpaced mode, which means, that simulation time does not elapse in relation to wall clock time but jumps as fast as possible from one event to the next, neglecting the simulation time (and thus the represented physical time) that lies inbetween [8]. Figure 1 shows part of an execution of two models in both simulation time and wall clock time. While the wall clock time continuously progresses simulation time increases at distinct time points according to scheduled events. The production of output by Model1, the reception of the according input by Model2, the internal transition of Model1, and its determination of next internal event are all realised at simulation time tsim . In the depicted i sim + m, case the value of the next simulation time is calculated by tsim i+1 = ti ≥0 where k, m ∈ R and m < k.

Fig. 1. Execution Fragment of two dynDevs Models

2.2

Modelling and Simulation of Multi-Agent Systems

The agent metaphor promotes the design of systems as consisting of entities which concurrently act and interact, and whose configuration and environment

4

Mathias R¨ ohl and Adelinde M. Uhrmacher

A dynDevs Model is a tuple hX, Y, m0 , M(m0 )i, where – X, Y are structured sets of inputs and outputs – m0 ∈ M(m0 ) is the initial model incarnation – M(m0 ) = {m | (∃mi ∈ M(m0 ). n = ρα (si , xip )) ∨ n = m0 } is the least set of model incarnations mi = hS, s0 , Xp , Yp , δext , δint , ρα , λ, tm, tai, with S the set of states s0 ∈ S the initial state Xp the set of peripheral input ports Yp the set of peripheral output ports δext : Q × X × Xp → S × Yp the external transition function, where Q = {(s, e) : s ∈ S, 0 ≤ e ≤ ta(s)} δint : S × Xp → S × Yp the internal transition function ρα : S × Xp → M(m0 ) the model transition function λ : S × Xp → Y the output function tm : R≥0 → R≥0 ∪ {∞} the time model function ta : S × Xp → R≥0 ∪ {∞} the time advance function Fig. 2. Syntax of the dynDevs formalism, extended with peripheral ports and the time model

is frequently changing. Variable structure models are a prerequisite to specify and analyse such systems. To support the modelling and simulation of agents that dynamically adapt their interaction, composition, and behaviour pattern the formalism dynDevs has been developed [22], which adds reflection to the Devs formalism. An atomic model in dynDevs (see Figure 2) is defined as a set of models that inherit state set, transition functions, output and time advance function from Devs atomic models. The reflectivity is introduced by the model transition (ρ α ) which maps the current state of a model into a set of models to which the model belongs. Thereby, sequences of models are produced. A model can change its own state and its behavior pattern, i.e. its transition, output, and time advance function, during simulation. As the coupled model holds the information about composition and interaction between components, a change of composition or interaction, even though induced by an atomic model, takes effect at the level of the coupled model. Therefore, coupled dynDevs models are introduced, that are the means for modelling adaptable system structures. For a more detailed discussion of the formalism and the definition of semantics of dynDevs models, see [22]. To test planning and commitment strategies of agents [20], dynDevs models have been equipped with peripheral ports (Xp , Yp ), which are now used to support the interaction of atomic models with external processes in general. The classical ports of Devs models collect and offer events that are produced by models. In addition, the peripheral ports allow dynDevs-models to communicate with processes that are external to the simulation. Thereby, not the entire simulation system as one black box interacts with external agents, but

Controlled Experimentation with Agents — Models and Implementations

5

each single model can function as an interface to external processes. The transition functions and the λ-function of the interface model describe how incoming data are transformed into data that can be launched into the simulation. The δint and δext function are responsible for the transformation of simulation data into data usable by the externally running software. The role of the ta-function is typically to model the pro-activeness of an entity, as it triggers internal events by the flow of time. In the case of externally running software, “pro-activeness” from the point of view of the simulation is triggered by incoming data, the time of which is determined by the time model. The simulation system uses the time model function tm to translate the resource consumption of the externally running software into simulation time. External processes are invoked by the simulation system and the information put into the peripheral output ports are forwarded to them. After the external computation has finished, the results of this invocation arrives at the peripheral input ports of the interface model at a simulation time which is determined by the time model. The time model might be constant which indicates that timing requirements are of no interest. Generally, it relates resource consumption to simulation time.

3

An Example — Evaluating Collaborative Agents

Depending on the actual phase in developing agents, specifications might be tested, single modules, or entire agents. In the following we will illustrate how simulation can be used to facilitate controlled experimentation within virtual environments during different phases of the development. Based on a simple, however extensible experimental set-up, which accompanying the development of problem solving agents shall help to answer the question: How well is are certain models or implementations of collaborative agents suited for solving a global task? The example comprises multiple agents with individual bounded capabilities. For coordination they use the Contract Net Protocol [7]. Tasks appear frequently in the environment. Only one of the agents is notified. To solve a task agents use deliberation to select appropriate action sequences. If the agent notices that he is not capable of performing all necessary actions by himself he may launch a call for proposals into the network of agents. Receivers of a cfp respond either with a proposal or by refusing the call. The initiator of the cfp may either reject or accept a proposal. Depending on the current task each agent may be a initiator or a participant of a contract. For reasons of simplicity we make several assumptions. All agents are benevolent. (Sub)tasks committed to are really solvable by the responsible agent. Tasks are decomposable into a set of sub-tasks with no causal inter-relationships. Only one level of delegation is sufficient for solving the task. At each time point only one task is launched into the network of agents. However, due to the time needed for solving tasks the agents might at different tasks at a time. Engineering agents for such a scenario will at least require to i) formalise the requirements for the agents ii) design a prototype and evaluate it iii) implement

6

Mathias R¨ ohl and Adelinde M. Uhrmacher

Fig. 3. Cooperative problem solving using the contract net protocol

the agent and test the implementation against the requirements. These development phases can be naturally supported by modelling and simulation. We will illustrate this, by starting to construct an experimental frame that formalises the conditions under which the agents are expected to work and thus, define the success or failure of the agent community in solving the task. The next step will be modelling a prototype of an agent for this scenario. First everything will be modelled within the simulation system. Arriving at a point, where the modelled multi-agent system works as it is supposed to do we can start to successively add implementations to be tested into the scenario. This is illustrated by upgrading the agent model with an implementation of the most critical functional part of the agent: the planning component. Time models are used for the execution of tests. 3.1

Specification of an Experimental Frame

Our first step is the modelling of an experimental set-up according to the informal description above. At the level of experimental frame all agents are treated as black boxes, which interact via communication channels with the task environment. This abstraction is directly supported by the modelling formalism by means of coupled models (cf. Section 2.2). The Environment is modelled as an atomic model responsible for generating new tasks every 10 time units and evaluate actions of the agents. If not all actions necessary for solving the tasks were received by the environment within these 10 time units the agents failed. For each task the environment selects a receiver agent ∈ {A1, A2, . . . , An} randomly and forwards the task to this agent. The agents are able to broadcast messages to all other agents by using the toAllport, or directly communicate with a single agent using a private channel, e.g. Agent1 may use the port toA2 to contact Agent2. Communication is not cost free, so currently a randomised delay is associated with each communication activity. An according dynDevs model for n agents could be specified as Environment = hX, Y, m0 , M(m0 )i, where – – – –

X = {fromA1 , . . . , fromAn}, Y = {toA1 , . . . , toAn} M(m0 ) = {m0 }, with S = (V, 2Action × Task × String), V = {executedActions, curTask , receiver } s0 = (executedActions=∅, curTask=new Task(), receiver=random(n))

Controlled Experimentation with Agents — Models and Implementations

7

Fig. 4. Experimental Frame for Evaluating a Group of Problem Solving Agents

– Xp = Y p = ∅ – δ ext(s, e, x, x p) = {s.executedActions.add((Action) x); } – δ int(s, x p) = { s.evaluate(s.curTask, s.executedActions); s.curTask = new Task(); s.receiver = random(n); } – λ(s, x p) = { send("to"+s.receiver, s.curTask);} – ta(s, x p) = {return 10.0;}

– ρα (s, xp ) = m0 (no variable structures needed) – tm(s, xp ) = 0 Please note, that transition functions of atomic dynDevs models can be arbitrary functions. Within the simulation system James [22] these functions can be specified using the Java programming language. 3.2

A Sample Agent

Having formalised the experimental frame for our agent(s), the next step is to model a prototypical agent. We omit the entire formal definition of the agent model. An intuitive, statechartlike representation of the dynamics of the agent is given in Figure 5. The dashed arrows represent internal transitions and the solid lines represent external transitions. The after-statements comprise the time advance function and the sendstatements the output function. Same as the environment no variable structures are needed. And still no peripheral ports are in use. In idle the agent is waiting for tasks appearing in its environment. In phase wait4proposals the initiator of a contract waits for proposals, while in phase

8

Mathias R¨ ohl and Adelinde M. Uhrmacher

Fig. 5. Behaviour of a sample agent

wait4contract an agents waits for rejection or acceptance of a proposal. The phases deliberateT, deliberateS, act are the only ones where simulation time is consumed by the agent. During deliberateT the agent tries to generate a plan for successfully accomplishing the announced task. The set of actions he is not able to execute are interpreted as sub-tasks that he may delegate to other agents. Consequently, he has to launch call for proposal for these sub-tasks. Agents receiving a cfp have to turn themselves to deliberation (phase deliberateS) for deciding how to achieve sub-tasks. The agents have to dynamically generate plans for solving a task. But the task decomposition takes place within the model transition (in phases deliberateT and deliberateS). Because of the fixed ta-value, execution (cf. Figure 1) of the experiment abstracts resources consumed for task selection to a constant amount of time. Many extension of this simple scenario can be imagined. E.g. if an agent is busy solving one task and receives another call for proposal it might decide to clone itself and to launch the new agent into the network, hoping that free capacities for calculation are somewhere available in the network. Depending on the degree of specialisation of agents this strategy might improve the performance of the multi-agent system. 3.3

From Models to Implementations

Simulation is perfectly suited for testing models and thereby facilitates the evaluation of agent specifications. When models have matured and parts of the agent

Controlled Experimentation with Agents — Models and Implementations

9

become implemented, the focus of analysis moves from models to implementations. Having constructed a logically valid model, which means that the agents really succeed in the experimental frame, the implementation can move into the focus of interest. Critical parts of the agents are no longer mere models but implemented software and thereby external to the simulation. If the implementation does not posses an own clock, a synchronous interaction of simulation and the external software will prove beneficial, as it gives the simulation system full control of the experiment. Suppose the development group responsible for implementing the deliberation comes up with a run-time optimised implementation Planner that implements the following interface public interface IPlanner { Vector plan(Vector actions, KnowledgeBase kb, Task t) (); }

The implementation should now be used instead of the prototype-code within the transition functions: δ ext(s, e, x, x p) = { ... if(s.phase.equals("idle") && x instanceof Task && portOf(x).equals("fromEnv")) { putIntoYp("Planner", "plan", s.actions, s.kb, x); s.phase="deliberateT"; } ... }

Java reflection will be used to instantiate the Planner and invoke its plan()method with the task received from the environment, the set of possible actions, and knowledge relevant for planning stored in the knowledge base. Now, not only the correctness of the generated plan is evaluated but the execution performance as well. The simulation system and software interact in a synchronous manner. The peripheral input ports are filled by the external software at a time determined by the time model which relates resource consumption and simulation time, e.g., tsim = tsim + tm(∆twc ) (cf. Figure 6). o i Using the time model different types of temporal abstraction can be realised between wall clock time, consumed resources on the one hand and simulation time on the other hand. Most often the consumed wall clock time is used as a resource, as it is easily accessible. However, its usage endangers the repeatability of simulation runs and implicitly introduces uncertainties due to hardware configuration and current work load. A kind of distortion might be induced by the used hardware configuration and the work load during testing. To avoid this type of uncertainties other approaches to define time models exist, that require more efforts and depend on the type of implementation, the language used, or the underlying operating system. For some planning systems the number of expanded

10

Mathias R¨ ohl and Adelinde M. Uhrmacher

Fig. 6. Synchronous interaction with an externally running Planner instance

nodes might be a suitable measure for the consumed resources, some languages facilitate the overloading of operators [1], and sometimes a performance counter on top of the operation system may be used [2].

4

Discussion and Future Work

dynDevs provides a general framework for discrete event simulation and is firmly rooted in systems theory. Models are defined as time-triggered, composite, and reflective automata. The operational semantic is clearly defined by abstract simulators. Extended by peripheral ports that allow to specify how models interact with external processes dynDevs supports the re-use and successive refinement of experimental settings from specification to implementation. Based on the formalism dynDEVS the simulation system James has been implemented [22]. Due to its variable structure facilities it is cut out for the modelling and simulation of open dynamic systems. The flexibility of the simulation system has been and is being put to test in such diverse areas as cell biology [4], testing of mobile agents [23], and testing of software aimed at running on mobile robots [11]. Furhtermore, James has been applied to analyse economic and demographic dynamics in pre-modern European towns in the aftermath of mortality crises [6]. To illustrate basic principles in moving from models to implementations we presented an abstract MAS scenario comprising an experimental frame which launches tasks into the network of agents, and randomised delays which are associated with the communication between agents. In spite of the simplicity of the scenario, an effective and timely accomplishment of tasks cannot be foretold by a static analysis. The presented scenario assigns work load, efficiency of single agents, and effort required for accomplishing sub-task arbitrarily. Therefore,

Controlled Experimentation with Agents — Models and Implementations

11

future work will be directed towards refining agents and environment to support the testing of a concrete agent application. The presented framework can than be used to analyse the number and efficiency of single agents, time complexity of single tasks, and the frequency with which tasks are launched into the network and their effect on successfully accomplishing tasks.

References 1. S.D. Anderson. Simulation of multiple time-pressured agents. In Proc. of the Wintersimulation Conference, WSC’97, Atlanta, 1997. 2. Shirley Browne, Jack Dongarra, N. Garner, George S. Ho, and Phil Mucci. A portable programming interface for performance evaluation on modern processors. The International Journal of High Performance Computing Applications, 14(3):189–204, 2000. 3. Jaelson Castro, Manuel Kolp, and John Mylopoulos. Towards requirementsdriven information systems engineering: the Tropos project. Information Systems, 27(6):365–389, September 2002. 4. Daniela Dagenring, Mathias R¨ ohl, and Adelinde M. Uhrmacher. Discrete event, multi-level simulation of metabolite channeling. BioSystems, 75(1-3):29–41, 2004. 5. Khanh Hoa Dam and Michael Winikoff. Comparing agent-oriented methodologies. In Proceedings of the Fifth International Bi-Conference Workshop on AgentOriented Information Systems, Melbourne, 2003. 6. U.-C. Ewert, M. R¨ ohl, and A.M. Uhrmacher. Agent Based Computational Demography, chapter What good are deliberative interventions in large scale disasters? Exploring the consequences of crisis managment in pre-modern towns with agentoriented simulation. Physica Verlag (Springer), 2003. 7. FIPA Contract Net Interaction Protocol Specification. http://www.fipa.org/specs/fipa00029, Dec. 3rd 2002. 8. Richard M. Fujimoto. Parallel and Distributed Simulation Systems. John Wiley and Sons, 2000. 9. Norbert Giambiasi, Jean-Luc Paillet, and Fr´ed´eric Chˆ ane. From timed automata to DEVS models. In S. Chick, P.J. S´ anchez, D. Ferrin, and D.J. Morrice, editors, Proceedings of the 2003 Winter Simulation Conference, pages 923–931, New Orleans, USA, 2003. 10. Vincent Hilaire, Abder Koukam, Pablo Gruer, and Jean-Pierre M¨ uller. Formal specification and prototyping of multi-agent systems. In ESAW 2000, volume 1972 of Lecture Notes in Artificial Intelligence, pages 114–127. Springer Verlag, 2000. 11. Jan Himmelspach, Mathias R¨ ohl, and A.M. Uhrmacher. Simulation for Testing Software Agents – An Exploration Based on James. In S. Chick, P.J. S´ anchez, D. Ferrin, and D.J. Morrice, editors, Proceedings of the 2003 Winter Simulation Conference, pages 799–807, New Orleans, USA, 2003. 12. Robyn Lutz. Software engineering for safety: A roadmap. In A. Finkelstein, editor, ICSE - Future of SE Track, pages 213–224. ACM Press, 2000. 13. Marvin Minsky. Models, minds, machines. In Proc. IFIP Congress, pages 45–49, 1965. 14. T. Montgomery and E. Durfee. Using MICE to Study Intelligent Dynamic Coordination. In Second International Conference on Tools for Artificial Intelligence, pages 438–444, Washington, DC, 1990. Institute of Electrical and Electronics Engineers.

12

Mathias R¨ ohl and Adelinde M. Uhrmacher

15. Gianluca Moro and Mirko Viroli. On observing and constraining active systems. In ESAW 2000, volume 1972 of Lecture Notes in Artificial Intelligence, pages 34–51. Springer Verlag, 2000. 16. G.J. Myers. The Art of Software Testing. John Wiley & Sons, Inc., 1979. 17. J. Odell, H.V.D. Parunak, M. Fleischer, and S. Brueckner. Agent UML: A formalism for specifying multiagent software systems. In F. Giunchiglia, J. Odell, and G. Weiss, editors, Agent-Oriented Software Engineering III, volume 2585 of Lecture Notes in Computer Science, pages 16–31. Springer, 2003. 18. M.E. Pollack. Planning in Dynamic Environments: The DIPART System. In A. Tate, editor, Advanced Planning Technology. AAAI, 1996. 19. Mark Ryan and Pierre-Yves Schobbens. Agents and roles: Refinement in alternating-time temporal logic. In J.J. Meyer and M. Tambe, editors, Intelligent Agents VIII: Agent Theories, Architectures, and Languages, volume 2333 of Lecture Notes in Artificial Intelligence, pages 100–114. Springer-Verlag, 2002. 20. Bernd Schattenberg and Adelinde M. Uhrmacher. Planning agents in James. Proceedings of the IEEE, 89(2):158–173, February 2001. 21. W. Sch¨ utz. The testability of distributed real-time systems. Kluwer Academic Publishers, Boston / Dordrecht / London, 1993. 22. A.M. Uhrmacher. Dynamic Structures in Modeling and Simulation - a Reflective Approach. ACM Transactions on Modeling and Simulation, 11(2):206–232, 2001. 23. A.M. Uhrmacher, M. R¨ ohl, and B Kullick. The role of reflection in simulating and testing agents: An exploration based on the simulation system james. Applied Artificial Intelligence, 16(9-10):795–811, 2002. 24. Wiebe van der Hoek and Michael Wooldrige. Towards a logic of rational agency. Journal of Autonomous Agents and Multi-Agent Systems, 11(2):133–157, 2003. 25. Hans Vangheluwe. DEVS as a common denominator for multi-formalism hybrid system modeling. In Proceedings of the IEEE International Symposium on Computer Aided Control System Design, pages 129–134, Anchorage, Alaska, September 2000. 26. D. Wolpert and J.W. Lawson. Designing agent collectives for systems with markovian dynamics. In AAMAS 2002: Autonomous Agents and Multi-Agent Systems, 2002. 27. Michael Wooldridge. The computational complexity of agent design problems. In Proceedings of the Fourth International Conference of Multi-Agent Systems (ICMAS-2000), pages 341–348, Boston, 2000. 28. Michael Wooldridge and Nicholas R. Jennings. Pitfalls of agent-oriented development. In Proceedings of the 2nd International Conference on Autonomous Agents, pages 385–391, 1998. 29. Bernard P. Zeigler. Theory of Modelling and Simulation. John Wiley, New York, 1976. 30. Bernard P. Zeigler. Multifacetted Modelling and Discrete Event Simulation. Academic Press, London, 1984.