Bringing Discrete Event Simulation Concepts into Multi-Agent Systems Daniele Gianni Dept. of Computer Science University of Rome TorVergata Via del Politecnico, 1, I-00133, Rome, Italy
[email protected] Abstract Agent-based Modelling (ABM) is a very efficacious conceptualisation paradigm that easily allows the representation of very complex physical systems composed of autonomous, possibly intelligent, interacting entities. For this reason, Agent-based systems (ABS) have been incorporated into simulation as a mean of effectively increasing the realism through adaptation and learning abilities. The integration between ABM and simulation, namely Agent-based Modelling and Simulation (ABMS), however, can be further exploited by modelling the simulation system as a Multi Agent System (MAS). To achieve this, the first step is to provide ABSs with discrete event simulation (DES) concepts (simulation time and services) and an interaction protocol. In this paper, we present a formulation of DES in terms of MAS through the design of software framework, named SimJADE, for the ABMS.
1. Introduction The Agent-based Modelling (ABM) paradigm is based on the concept of autonomous interacting components, named agents, which maintain control over their status and can carry out specific tasks [1]. For their autonomy, agents can easily incorporate intelligence (e.g.: adaptation, reasoning and learning mechanisms), and therefore become intelligent agents [1]. Simulation systems, which often deal with representation of human behaviour in synthetic environments, started integrating agent-based technologies in simulated worlds to increase the realism in the representation of such behaviours [2]. Although such integration was further developed and exploited in this direction, its scope can be broadened by modelling Discrete Event Simulation (DES) systems through Multi Agent System (MAS) paradigm. The most adopted DES paradigm, process interaction [3], indeed presents many affinities with MAS. It is based on independent simulation entities that communicate and synchronize their logical time to carry out the simulation according to the properties of causality and reproducibility. In this paper, we present a MAS-based modelling of DES systems through the design of SimJADE, an Agent-based Modelling & Simulation (ABMS)
framework. SimJADE is built on top of the popular JADE framework [4], and is designed according to modern software engineering techniques [5]. For its structure, SimJADE also eases the development of ABMS systems because conventional JADE components (messages and behaviours) can be effortless plugged into the simulated world. The paper is organized as follows: a related works section highlights the differences with other state-of-art contributions, and then a brief overview of the JADE framework follows. The framework SimJADE is thus presented, and a brief example concludes the paper.
2. Related Works There are several valuable contributions that introduce frameworks for the ABMS. To be mentioned are JADE-HLA [6], SIM_AGENT [7], and the Time-Extension for MAS [8]. JADE-HLA, which is built on JADE and on the distributed simulation standard High Level Architecture (HLA) [9], differs from this work in four main points. First of all, SimJADE adopts a general DES modelling, and therefore is not related to specific technologies. Secondly, SimJADE implements an agent-based conceptualisation of DES systems; thirdly, SimJADE maintains a uniform view with the JADE design outline, and therefore makes ABMS easier for JADE developers. JADEHLA, however, enables the agent-based distributed simulation on HLA. Similarly, SIM_AGENT provides a framework to develop ABMS systems. It differs from SimJADE because it does not formulate the simulation in terms of MAS, and binds the reasoning, planning, etc. mechanisms to the framework. Differently, SimJADE deals only with the issues related to the simulation, and therefore allows the use of JADEcompliant frameworks currently available (e.g.: Jess rule engine [10], JADEx [11], etc.). The Time-Extention shares with SimJADE the partial objective of bringing the simulation time concept into MAS; however, there are considerable differences. First of all, SimJADE has a wider scope. It presents a formulation of DES systems as MASs. Secondly, the Time-Extension uses innovative aspect-oriented methodologies [12] to bring duration semantic in MAS. Differently, SimJADE shows how
conventional object-oriented techniques can effectively support this through the mere application of the Decorator Pattern concept [13]. Thirdly, the Time-Extension mechanism introduces some discontinuities between MAS and simulated MAS. They are due to the use of the aspect oriented technology, which is specifically used to produce the simulated MAS starting from the MAS, and to the non-encapsulation of the implicit wall-clock time concept that agents have.
3. JADE JADE [4] is a popular Java-based framework for the implementation of MASs. Its base element is the agent, which maintains an internal state and whose dynamic is described through a set of pluggable behaviours. The behaviour is a sequence of internal operation and communication acts with other agents, and it is possibly composed of sub-behaviours according to several composition structures (parallel, serial, etc.). The most fundamental JADE aspect is the communication [14], which is carried out according to FIPA specifications [15] through an asynchronous mail-box based mechanism. As FIPA defines, JADE messages are composed of the following attributes: sender, list of recipients, performative action, content, content language reference, content ontology reference, and a set of minor fields to control concurrent conversations. Besides attributes of immediate understanding, the message contains a performative action attribute, and two references to the content coding language and to the shared ontology, which needs further details. The performative action attribute specifies the type of communication, which has been classified by FIPA into twenty-five different communicative acts. For example, it can be of value REQUEST when the sender agent asks for a service request to the recipient agents, or can be of value INFORM in the case of “notification” of state change. Concerning the reference attributes to the content language and content ontology, they provide the recipient agents of, respectively, the information needed to decode and interpret the semantic of the content field. JADE ontologies are in turn to be built on top of the basic ontology, which provides basic concepts for primitive data types, and can define three types of elements [14]: predicates, concepts, and actions, which are a specialization of concepts. Predicates represent facts in the modelled world, and can be true or false. Concepts represent complex data structures, which are composed of standard simple types like String, Integer, etc.; and, finally, actions define special concepts that are internally associated to actions that the agents can perform.
4. SimJADE SimJADE is a Java framework for the implementation of ABMS. It is built on top of JADE, and therefore inherits all the compliances with the FIPA specifications [15]. The main ideas behind SimJADE are two. The first is to provide an agent-based implementation for DES systems; and the second is to enable ABS developers to ABMS with the simply plugging their agents’ models into SimJADE. SimJADE is composed of: • A simulation ontology • A set of simulation agents • A set of simulation behaviours • A set of simulation event handlers. The simulation ontology, named DES-Ontology, defines the DES concepts (simulation time) and actions (DES and simulation life cycle management services) that are used as semantic base for the communications among the simulation agents. The simulation agent society is composed of two types of agents: simulation entity and simulation engine. The simulation entity agent encapsulates the simulation logic, i.e. the sequence of state evolution and DES service requests; whereas the simulation engine agent, which is unique within the society, has the role of requests collector and simulation coordinator. The simulation behaviours can be classified into three groups, for the implementation of the actions behaviours, for the implementation of entity and engine basic behaviours, and the last for the encapsulation of JADE standard behaviours in the simulated environment. The simulation event handlers are the conventional routines to be processed for each of the simulation service event (e.g., sendEvent, wakeup, etc.). These handlers are in close analogy with the DES framework SimJ [16] [17].
4.1 DES-Ontology The DES-Ontology inherits basic concepts and actions from the JADE standard ontology [14] while introducing new ones for the specific DES domain. These concepts and actions can be easily inferred by the DES domain. The concepts are related to the simulation time, while the actions are related to the simulation interaction between simulation entity and simulation engine. For the concepts, the DES-ontology defines two different representations of the simulation time: AbsoluteSimulationTime, for absolute values of the simulation time; and RelativeSimulationTime, for relative values of the simulation time, with “relative” having default semantic “respect to the current time”. Although the two concepts are connected through the transformation = + , the definition of a relative
time concept is included in the ontology because it is common practice to use it as parameter type in several DES services. For the actions, the ontology defines two groups of items: the simulation management services and the DES services. Belonging to the first category are the actions to manage the simulation life cycle: • Register agent: to request the permission to join the simulation society; • Registration successful: to acknowledge the permission in response to a Register Agent request; • Remove agent: to resign the society; • Simulation end: to inform that the society objective has been reached. Whereas the second group includes: • Conditional hold time: to request an hold for a given simulated time unless any even notification before it; • Hold time: to request an unconditional hold for a specified simulated time; • Notify time: to inform that the specified time has been reached; • Notify message: to inform that the specified event was requested to scheduled for the receiving agent, at the current time; • Send message: to request the delivery of the specified event at the specified time to another simulation entity agent; • Wait message: to request to be wake up when a simulation message is to be notified. In the first group, actions register agent and remove agent, which are both of performative type REQUEST, have no attributes since the action object, i.e. the name of agent requesting the action, can be inferred from the message envelop. To conclude the first group, actions registration successful and simulation end, which are both of performative type INFORM, bring with them an instance of AbsoluteSimulationTime that respectively indicates the simulation time from which the agent has to start, and the simulation end time. In the second group, conditional hold time and hold time, which are both of performative type REQUEST, are characterized by a relative simulation time that specifies the simulation time for which they want to be put in sleep state. Notify time, which is of performative type INFORM, informs the receiving agent of the absolute simulation time reached. Similarly the notify message, which instead notifies a message, and thus is described by four attributes: sender agent, recipient agent, message and time; all of type String, except the time, which is of type AbsoluteSimulationTime. The action send message plays the opposite role of the Notify Message, and therefore is described by
the same attributes, but with performative type REQUEST. In the specific case, to maintain a logical uniformity with the common practice in DES, the time is taken of type RelativeSimulationTime. Finally, Wait Message, which is of performative type REQUEST, informs the engine that the sender agent is blocking and waiting for new messages.
4.2 Simulation Agents The simulation agents’ society is composed of two types of agents: simulation entity and simulation engine. The simulation entity agents incorporate the simulation logic through custom simulation behaviours; whereas the simulation engine agent is in charge of coordinating the society, and therefore includes a list of the simulation events and a record of the society composition. The simulation entity agents have a life cycle defined by the state diagram in Figure 1, which shows the simulation states and therefore has to be considered on top of the standard JADE agent state diagram [14].
Figure 1 Simulation entity agent's state diagram
The diagram presents characteristics very similar to the equivalent diagram for conventional simulation entities in any DES framework, and therefore such details are not discussed further here, an example of that can be found in [16] [17]. The two diagrams, however, differ for the presence of the initial Waiting for Registration Acknowledge state, during which the simulation engine collects the registration requests and checks when the society is ready to start the simulation. This difference is due to the decentralised nature of the MAS modelling. In fact, in conventional DES frameworks, the registration phase is hard-coded in the simulator at the entities instantiation time through a local method call, and therefore such state is not present in the dynamic evolution of the entity. To realize the above dynamic, the entity’s behaviour is configured as a serial composition of a RegisterAgentBehaviour (see next section) and an EntityMainCycleBehaviour, with the latter to be configured according to the model specifications. Since EntityMainCycleBehaviour should be allow the easy plugging of any conventional JADE behaviour, the entity agent interface must be consistent with the JADE agent interface. To achieve this, the simulation entity agent must therefore
invoke the simulation actions conditional hold time, hold time, send event, and wait event within the JADE standard methods blockingReceive(millisecs), doWait(), send(), and blockingReceive(), respectively. In a similar way, the simulation engine agent can be described by the state diagram in Figure 2.
numberOfRunningAgents--; schedule wakeUpHandler; store agentName in conditionalList; break; RegisterAgent: numberOfRunningAgents++; registerAgent; start RegistrationSuccessfulBehaviour; } // end case } // end while If (eventsList.size() > 0) { nextEvent = eventsList.remove(0); nextEvent.process(); } else { setSimulationEnd(); }
Figure 2 SimulationEngineAgent state diagram
The engine agent behaviour is organized in three sequential phases, each implemented as different sub-behaviour. The first phase is the registration phase that cares of synchronizing the start-up phase through the Waiting for Registration Requests and Confirm Registration Successful states. In this phase, the engine accepts incoming register agent requests while checking whether the simulation society becomes complete. Once the society is completed, the engine notifies the registration successful to all the registered agents. Such states are not present in conventional DES framework because the entities registration is performed through static invocation of local methods at simulator coding time. Similarly to the entity agent, the presence of these two states comes from the inherent decentralised nature of the system. After completing this phase, the engine proceeds to the second phase, EngineMainCycleBehaviour, which includes the states Waiting for Simulation Requests and Processing Internal Event. This phase is defined by the algorithm detailed below: While (numberOfRunningAgents > 0) { wait for a simulation message; Case of: SendEvent: schedule sendEventHandler; break; HoldTime: numberOfRunningAgents--; schedule wakeUpHandler; break; WaitEvent: numberOfRunningAgents--; break; RemoveAgent: numberOfRunningAgents--; start RemoveAgentBehaviour; break; ConditionalWaitEvent:
The algorithm is composed of two main blocks: a while-block for the requests collection at constant simulation time, and an if-then-else block for the processing of the next scheduled event and, therefore, for the simulation time advancement. The algorithm is based on two assumptions. The first is that the cardinality of the simulation society is known from the previous phase; whereas the second is that the agents requesting hold time, conditional hold or wait event simulation services block. While blocking, they do not perform further requests until they receive proper simulation notifications. The second assumption defines the interaction protocol between entity and engine agents, and guarantees that in the second block the actual number of running agents is zero. Such condition is locally verified through the value of the local variable numberOfRunningAgents. The first block repeats while there are running agents in the society. In such block, the activities follow a sequential wait-and-serve cycle, in which the requests are processed by properly updating the numberOfRunningAgents variable and by performing the respective action: either the scheduling of a new event handler in the list or the activation of simulation service behaviour. For example, upon receiving a send event request, the engine schedules a new local SendEventHandler with all the proper data (recipient, time, message, etc.). Similarly for wait event requests, the engine only counts that the requesting agent is now blocked and will not perform other requests until a local event unblocks it again. The algorithm also deals with the dynamicity of the agent society composition by processing register agent and remove agent requests accordingly. Once the simulation end event is reached, the engine stops and the EngineMainCycleBehaviour terminates. The engine proceeds to the last phase, which includes the Simulation End Notification state. In this phase, the engine notifies a simulation end message to the entire society before concluding its life cycle and removing from the agent container.
4.3 Simulation Behaviours The behaviours within SimJADE can be grouped into three distinct categories: 1. Simulation service behaviours: to implement each of the actions defined in the ontology; 2. Simulation life cycle behaviours: to regulate the simulation entity and simulation engine’s life cycle phases; 3. Model behaviours: a set of empty behaviours to allow SimJADE users to plug their conventional MAS behaviours into the simulated agents. The behaviours in the first group are: 1. ConditionalHoldTimeBehaviour 2. HoldTimeBehaviour 3. NotifyTimeBehaviour 4. NotifyMessageBehaviour 5. RegisterAgentBehaviour 6. RegistrationSuccessfulBehaviour 7. RemoveAgentBehaviour 8. SendEventBehaviour 9. SimulationEndBehaviour 10. WaitEventBehaviour. Where behaviours 1, 2, 5, 7, 8 and 10 are invoked by simulation entity agents when the relative action is requested; and the remaining 3, 4, 6, and 8 are used by the simulation engine agent when the simulation reaches a particular state (e.g.: simulation society completed) or when the respective action is requested for the processing of a simulation event (e.g. notify time, etc.). While the behaviours invoked by the simulation engine agent are all of non-blocking type, i.e. the agent does not block for a notification, the behaviours used by the simulation entity agent can be of blocking or non-blocking type. The blocking behaviours are: 1. ConditionalHoldTimeBehaviour 2. HoldTimeBehaviour 3. RegisterAgentBehaviour 4. WaitEventBehaviour and their respective expected notifications are: 1. Notify time, notify event or simulation end; 2. Notify time or simulation end; 3. Registration successful; 4. Notify event or simulation end. The remaining behaviours, SendMessageBehaviour and RemoveAgentBehaviour, are non-blocking because of the reliable communication channel among the agents. The second group includes the ad-hoc behaviours for the management of entity and engine agents’ life cycle phases, which were anticipated to the previous section. The third group contains the basic components for encapsulating JADE standard behaviours into
SimJADE by setting time duration for it. It is composed of a general SimulationBehaviour, and a SimulationInternalBehaviour. The former is to be used with any standard JADE behaviour that includes interactions with other agents, e.g. send event, wait event etc. For the presence of communicative acts, the behaviour duration can not be explicitly set. Differently, the latter is basically a two-branch parallel behaviour that allows setting time duration to any internal (i.e not including any communicative acts) JADE behaviour. The first branch contains a HoldTimeBehaviour, while the second branch is available to plug any internal behaviour.
4.4 Simulation Event Handlers The handlers are the routines to be performed upon the processing of a simulation event. In DES, events can be of four types: • Conditional wake up • Wake up • Send event • Simulation end And therefore four handlers are to be defined. Their definition proceeds similarly to what happens in conventional DES frameworks but considering the decentralised nature of the interactions. The conditional wake up and the wake up handlers have the same handler, which consists of the invocation of a NotifyTimeBehaviour, and the count of a more running agent in the society. Differently from wake up, however, conditional wake up handlers might be removed from the event list if a send event handler is scheduled for the requesting agent at a previous simulation time. The send event handler, which performs an event notification, is defined with the invocation of a NotifyMessageBehaviour and the count a running agent more in the society. This handler also checks whether there is a conditional wake up event scheduled for the same agent, and in case removes it. The simulation end handler performs only the activation of a SimulationEndBehaviour, which in turns informs all the agents that the simulation end has been reached.
5. Example Application A good example for the application of SimJADE is the simulation of a manufacturing system. For the sake of simplicity, let us consider a simplified system where workers move around the factory premises in order to reach the machines they need to use. In such a scenario, the fundamental aspect is represented by the movement of the workers. A possible space modelling for this system is represented by a graph which nodes model the possible positions, and which edges model the possible movements between two positions that the workers can perform. The nodes
also constitute physical resources that can exclusively be used by only a worker per time, whereas the edges can simultaneously be walked by more workers at the same time. A possible MAS modelling of such system could include two types of agents: a ResourceManager agent, which coordinates the access to the physical points; and a Worker agent, which wants to reach a machine. A synthetic sequence diagram for two worker agents and a resource manager is shown in Figure 3. The workers inform the ResourceManager about when they reach the node and then wait an authorization event. The ResourceManager authorizes the movement when the required node is free and delays authorizations when the node is busy. Upon the completion of a movement, the ResourceManager activates enqueued workers. The diagram does not include the engine agent because it is not visible at the system modelling level. The simulator can then be quickly implemented by the above specification of the MAS system, which indeed does not contain any reference to execution context, either simulated or real-world.
8. References [1]
[2]
[3]
[4] [5] [6]
[7]
[8]
[9]
[10] [11] Figure 3 Example sequence diagram
6. Conclusions [12]
In this paper, an Agent-based formulation of Discrete Event Simulation (DES) systems is presented through the design of a software framework named SimJADE. SimJADE introduces DES concepts and behaviours that can contain conventional JADE messages and behaviours and therefore agent-based system developers can easily produce agent-based simulator assembling off-the-shelf components and developing ad-hoc ones. A simplistic example in the manufacturing domain is also presented to show the potential application to more complex systems. A distributed version is being developed to support larger scale scenarios.
7. Acknowledgements This work was partially supported by funds from the FIRB project “Software frameworks and technologies for the development and maintenance of open-source distributed simulation code” and by the CERTIA Research Center.
[13]
[14]
[15] [16]
[17]
N.R. Jennings, and M. Wooldridge, “Application of Intelligent Agents”, Agent technology: foundations, applications, and markets, Springer-Verlag, 1998, pp. 3 – 28. E. Gelenbe, E. Seref, and Z. Xu, “Simulation with Learning Agents”, Proceedings of the IEEE, vol. 89, n. 2, February, 2001, pp. 148 – 157. Richard E. Nance, “The time and state relationships in simulation modeling”, Communications of the ACM, vol. 24, n. 4, April 1981, pp. 173-179. JADE project home, http://jade.tilab.it,Telecom Italia I. Sommerville, Software Engineering, 7th ed., Addison Wesley (2007). F. Wang, S.J. Turner, and L. Wang, “Agent Communication in Distributed Simulations”, Proceedings of the Multi-Agent and Multi-AgentBased Simulation (MABS 2004), Springer-Verlag, LNAI 3415, 2005, pp. 11–24. A. Sloman and B. Logan, “Building cognitively rich agents using the SIM_Agent toolkit”, Communication of the ACM, vol. 42, n. 3, 1999, pp. 71 – 77. A. Helleboogh, T. Holvoet, D. Weyns, and Y. Berbers, “Extending Time Management Support for Multi-agent Systems”. Proceedings of the 2004 Workshop on Multi Agent Simulation and Multi Agent-based Systems, LNCS 3415/2005, Springer Verlag, 2004, pp. 37 – 48. IEEE 1516, Standard for Modeling and Simulation (M&S) High Level Architecture (HLA) – Framework and Rules. Jess Project http://www.jessrules.com. A. Pokahr, L. Braubach, and W. Lamersdorf, “JADEx: Implementing a BDI-Infrastructure for JADE Agents”, EXP - In Search of Innovation (Special Issue on JADE), vol 3, n. 3, Telecom Italia Lab, Turin, Italy, 2003, pp. 76-85. G. Kiczales, J. Lamping, A. Menhdhekar, C. Maeda, C. Lopes, J. Loingtier, and J. Irwin, “Aspect-oriented programming”, Proceedings of the European Conference on Object-Oriented Programming, Vol. 1241, Springer-Verlag, 1997, pp. 220 – 242. E. Gamma, R. Helm, R. Johnson, J. Vlissides, Design Patterns: Elements of Reusable ObjectOriented Software, Addison Wesley (2000). F. Bellifemine, G. Caire, and D. Greenwood, “Developing Multi-Agent Systems with JADE”, Wiley (2007). FIPA Specification, http://www.fipa.org. A. D’Ambrogio, D. Gianni, and G. Iazeolla, “SimJ: a Framework to Distributed Simulators”, Proceedings of the 2006 Summer Computer Simulation Conference (SCSC06), Calgary, Canada, 2006, pp. 149 – 156. D. Gianni and A. D’Ambrogio, “A Language to Enable Distributed Simulation of Extended Queueing Networks”, Journal of Computer, Vol. 2, N. 4, July, 2007, Academy Publisher, pp. 76 – 86.
