Berbers}@cs.kuleuven.be. Keywords. Action model, automated guided vehicles, multi-agent systems. Abstract. Events fall short to simulate actions of multi-agent ...
SIMULATING ACTIONS IN DYNAMIC ENVIRONMENTS Alexander Helleboogh, Tom Holvoet and Yolande Berbers DistriNet, Department of Computer Science K.U.Leuven, Belgium {Alexander.Helleboogh,Tom.Holvoet,Yolande.Berbers}@cs.kuleuven.be
Keywords Action model, automated guided vehicles, multi-agent systems
Abstract Events fall short to simulate actions of multi-agent systems (MASs). Agents act in dynamic environments, and the characteristics of actions in dynamic environments differ significantly from the properties of events. In this paper, we propose a conceptual model for actions in dynamic environments, based on the concepts influence, activity and effect. The model is especially important for simulating industrial multi-agent systems as it explicitly supports three essential characteristics of actions in dynamic environments: (1) the environment changes while an agent is acting upon it, (2) agents do not control the actual outcome of their actions, and (3) agents do not control the time instant their actions complete. The model enables a developer to incorporate a suitable amount of dynamism with respect to the actions in the simulation in an explicit and systematic way. This allows simulating scenarios where actions do not yield their intended result. We illustrate the model using a scenario in an automated warehouse transportation system and describe the architecture of a prototype that supports the model.
1 INTRODUCTION We are currently involved in a research project, called Egemin Modular Controls Concept (EM C 2 ) in cooperation with Egemin, an industrial expert in automating warehouse transportation systems using automated guided vehicles (AGVs). An automated warehouse transportation system is an industrial transport system using several automated guided vehicles (AGVs). An AGV is an unmanned, computer-controlled transportation vehicle using a battery as its energy source. AGVs have to perform transportation tasks. A transportation task consists of picking up a load at a particular spot in the warehouse and bringing it to its destination. To move from one location to another, AGVs use a complex, graph-like network of predefined road segments intersecting at crossroads. The AGVs are equipped with a low-level controller that offers a number of high-level actions that can be used to instruct the AGV. A move action instructs an AGV positioned at a particular node to move towards an adjacent node over a specified road segment, crossing all intersections on the segment. A pick action instructs the AGV to pick up a load at the end of a specified road segment. A put action puts down the load at the end of the specified segment. Finally, a send action instructs the AGV
to transmit a given message using on-board wireless communication infrastructure. The physical environment in which the agents are situated, is highly dynamic. A dynamic environment is an environment that changes continuously, even while the agents are acting upon it. Examples of dynamism are road segments that get congested or blocked by other AGVs, new goods that arrive, other AGVs that enter or leave the system, etc. Because of the high degree of dynamism in a warehouse environment, customers demand increasing flexibility of the transportation system. AGVs should adapt their behavior to the changing circumstances of the environment they are situated in. This requirement to increase flexibility imposes major difficulties on the centralized planner that is currently used to steer all AGVs. The goal of the joint project with our team, is to increase flexibility by decentralizing system control, and delegating autonomy of to agents controlling individual AGVs (Weyns et al. 2005). MASs are used in industrial applications for modelling flexible behaviour of the system in often unpredictable situations. Such situations typically result from scenarios where actions do not yield their intended result. Simulation is crucial for testing (Himmelspach et al. 2003; Riley and Riley 2003) the behavior of the system in unpredictable scenarios without the risk of damaging the AGVs. However, supporting such scenarios in a simulation is difficult, because actions in dynamic environments have three important characteristics. (1) Actions do not happen instantaneously, but each action takes a particular amount of time during which parallel actions and changes in the environment may occur. For example, a particular pick action of an AGV could take 23 seconds, corresponding to the time an AGV needs to drive towards the end of the specified segment, position itself with respect to the load and lift up the load, whereas a send action could only take 450 milliseconds. (2) The actual outcome of an action is not controlled by the agent, as the action may be affected by other actions. For example, the result of a move action performed by an agent, may be affected by the move of another agent if both road segments intersect. (3) The moment in time an action completes is not controlled by the agent. For example, the moment in time a pick action completes, depends on the length of the road segment and the amount of repositioning the AGV needs with respect to its load. It is essential that the characteristics of actions in dynamic environments are explicitly supported in the simulation. This is necessary to test whether the agents are flexible enough to achieve their goals by means of actions of which the precise outcome as well as the completion time are unpredictable. If the characteristics of actions in dynamic environments remain unsupported in a simulation, the flexibility that is required of agents in industrial applications cannot be tested in an adequate manner.
This is a first step in our attempt to develop an integrated model to support actions in dynamic environments. The benefit is an increased realism to simulate MASs in dynamic environments. The added value of the model is that it provides explicit support to test the agents in dynamic scenarios before the MAS is deployed in the real world. This paper is organized as follows. In Sect.2, we elaborate on the problem of simulating actions in dynamic environments. Section 3 unfolds the model using a scenario of moving AGVs, and describes the architecture of the prototype that supports the model. We draw conclusions and look forward to future work in Sect. 4.
stop event (Anderson 1997). These approaches exploit the original event concept in an artificial and ad hoc way. As such, these tricks confuse and mislead the developer by forcing him/her to think actions in terms of fabricated event representations. In this paper, we propose a conceptual model aimed at explicitly supporting the characteristics of actions in dynamic environments (see Fig.1). The model enables a developer to incorporate a suitable amount of dynamism with respect to the actions in the simulation in an explicit and systematic way. Therefore, the model comprises several concepts to represent the characteristics of actions in dynamic environments. In this paper, we describe each of these concepts in detail and point out how they can be used to represent the characteristics of actions in dynamic environments.
2 PROBLEM STATEMENT e A
Multi-agent simulation has a strong background in distributed discrete-event simulation. This is reflected in the fact that stateof-the-art MAS simulation tools usually represent actions of agents by means of events (Logan and Theodoropoulos 2001; Uhrmacher and Schattenberg 1998; Riley 2003; Vincent et al. 2001). An event is defined as a change in the state of the simulation that occurs instantaneously at a specific point in simulation time (Schriber and Brunner 1999). Events are interesting for running a simulation as they represent an order of state changes to be applied in the simulated environment. However, the properties of events differ significantly from the characteristics of actions in dynamic environments (mentioned in Sect.1). First, each action of an agent takes a particular, non-negligible amount of time. In contrast, events happen instantaneously at a specific point in time. Second, agents do not control the outcome of their actions, as the outcome can be affected by other actions happening in the meanwhile. However, each event is a state change and as such defines an action in terms of its outcome. Third, the moment in time an action completes, is not controlled by the agent. In contrast, each event encapsulates the moment in time the state changes are applied. Fourth, the granularity of actions is fixed and defined a priori at application level. In contrast, the granularity of events is variable and depends upon the accuracy required from a simulation. For example, a single move action in the AGV application always corresponds to the distance of the specified segment, independent of whether the required accuracy to determine collisions in a simulation is 10 millimeters of 5 meters. From this comparison, it is clear that a straightforward mapping of actions to events overlooks a number of essential characteristics of actions in dynamic environments. This oversimplification endangers testing the flexibility of a MAS in a realistic way. As such, there is a lack of support to model the characteristics of actions in dynamic environments in an explicit manner. Currently, enforcing these characteristics in a simulation has to be tackled by the developer without any support. This typically results in ad hoc solutions that are developed from scratch and fail to address the core of the problem. For example, representing an action as a single event which is scheduled with a particular delay representing the duration of an action (Riley 2003; Vincent et al. 2001; Uhrmacher and Schattenberg 1998). Another example is considering an action as a combination of a start and a
n
t
e
g
A
c
t
i
o
n
t
g
n
c
A
o
d
t
i
o
n
A
e
l
f
o
r
a
c
t
i
o
n
s
i
n
d
y
n
a
m
i
c
M
e
n
v
E
S
i
m
u
l
i
v
a
t
e
e
r
o
n
d
n
m
e
n
t
s
t
E
E
n
v
i
r
o
n
m
v
e
e
n
n
t
t
Figure 1: A model to bridge the gap between actions and events.
3
THE INFLUENCE-ACTIVITY-EFFECT MODEL
An important model to tackle some of the fundamental problems of events to represent actions, is the influence-reaction model of Ferber et al. (Ferber and M¨uller 1996). This model was later extended by Weyns et al. (Weyns and Holvoet 2004). In essence, the influence-reaction model makes an explicit distinction between what an agent wants to perform, and what actually happens. The influence-reaction model represents an action by two different concepts. Influences come from inside the agent and are attempts to modify the state of affairs in the environment. On the other hand, reactions represent state changes, and are produced in the environment by combining the influences. In the influence-reaction model, the original attempt of the action is decoupled from its actual outcome, which is essential in dynamic environments. However, in contrast to events, there is no explicit notion of time in the influence-reaction model. The influencereaction model considers the evolution of a MAS in a sequence of cycles. In each cycle, all agents are forced to perform influences synchronously, after which the reactions to all influences are calculated. As such, the influence-reaction model does not allow the modeler to take into account the timing of each individual action. The time of a cycle from influence to reaction is
the same for all actions. Consequently, the time it takes an AGV to perform a send action would be the same as the time to perform a pick action. We now elaborate on the influence-activity-effect model for actions in dynamic environments. By means of a scenario which is easy to trace, we illustrate the model and explain each of its concepts in detail. The scenario is depicted in Fig.2 and consists of two AGVs moving in a dynamic environment. Agent A instructs its AGV to move towards position A’. One second after agent A started to move, agent B instructs its AGV to move towards position B’. Although the road ahead is clear for both agent A and B at the time they initiate their respective move actions, it is obvious that both actions can interact. The reason for this is that changes occur in the environment while an AGV is moving. In this case the change is caused by the movement of the other AGV. In the scenario, a collision will occur, resulting in different outcomes for both actions than the ones intended by the agents. Note that by representing both move actions as a simple event, i.e. as an instantaneous state change from A to A’ and from B to B’ respectively, typically a collision never occurs. B
An influence may have no consequences whatsoever. It possibly not even starts an action or causes anything to happen. For example, at a time instant an AGV is still moving, the agent can perform an influence to indicate it wants to start a different move action. The influence is just an indication that the agent has decided to do another move action at that particular time. However, in this case the influence causes nothing to happen, as the AGV controller discards the new action in case the AGV is already moving. An agent is allowed to perform several influences at the same time instant. For example, at a particular time instant, an agent can indicate it wants to start performing a move action as well as to start sending a message. As an example, consider Fig.3. I-1 represents an influence generated by agent A to indicate it wants to perform a send action at time instant 1. I-2 represents a second influence generated by agent A at time instant 1 indicating a move action over segment X. Finally, I-3 represents an influence generated by agent B at time instant 2 indicating a move action over segment Y.
'
A
A
A
'
B
B
Figure 2: A scenario of 2 moving AGVs.
I
!
I
!
1
2
I
!
3
T
3.1 Influences We extend the influence concept of the influence-reaction model with an explicit notion of time. An influence is defined as the agent’s attempt to start an action at a particular time instant. Agents can only affect the environment by means of generating influences. An influence allows an agent to control two aspects. First, each agent can choose the action it would like to initiate. For example, an agent can opt for a particular move action between all possible moves over adjacent road segments, or for a send or a pick action. Second, an agent can also decide upon the time instant it initiates the action. For example, an agent can decide to wait a little longer before initiating a move action, so as to allow other AGVs to cross an intersection. When using simulation for testing purposes, the agents under consideration could perform any possible influence. It is always possible that a malicious or buggy agent performs an influence that corresponds to an illegal action. Preventing illegal actions has to be done in a way that is transparent for the agents, as a malfunctioning agent can otherwise compromise the consistency of the simulation. The agents cannot be expected to adhere certain constraints with respect to the possible influences they are allowed to perform. Consequently, the model has to accept any possible influence performed by the agents. This does not mean that an agent is allowed to perform any possible action, it only states that an agent is allowed to try to perform any possible action.
1
2
3
4
5
i
m
e
(
s
)
6
Figure 3: An example of 2 AGVs generating 3 influences.
3.2 Activities and Influence Laws An activity is an explicit representation of an action that is in progress of happening. An action is in progress of happening after the time instant it has been started, and before the time instant it completes. As such, an activity allows a developer to reify the act itself of performing an action, decoupled from the causes that trigger the action, and decoupled from all state changes that correspond to the actual outcome realized by the action. For example, a move activity represents the act of moving of an AGV over a particular distance. The move activity is clearly separated from its cause, which could be an influence performed by the agent instructing the AGV to move, or another move activity of a different AGV that pushes the AGV forward. The move activity is also decoupled from the possible outcome it realizes, which could be a change the position of the AGV combined with a decrease of the energy level of the battery, or a loss of functionality due to damage in a collision. An important characteristic of each activity is its duration. In contrast to an influence that happens at a particular time instant, the act of performing an action happens over a particular time
interval. As such, different activities happen over different time intervals. For example, the time interval of a move activity over a lengthy road segment is longer than the time interval of a send activity of a single message or a move activity over a shorter road segment. Therefore, each activity incorporates the time interval during which the corresponding action is in progress of happening. Agents cannot directly generate or control activities. Agents can only perform influences. Whether an activity results from an influence is determined by influence laws. An influence law is a rule that specifies for an influence which activity is caused by it, if any. For example, an agent performs an influence at a particular time instant to start a move action. The influence law then checks whether the AGV is not broken and whether its battery is not empty, and derives a corresponding move activity. In Fig.4, the activities are depicted that result from applying the influence laws on the influences in Fig.3. A-1 represents a send activity performed by agent A over time interval (1,2), caused by the influence I-1. A-2 represents a move activity on segment X performed by agent A over time interval (1,5), caused by the influence I-2. A-3 represents a move activity on segment Y performed by agent B over time interval (2,6), caused by the influence I-3. A-2 and A-3 are visualized at the top of Fig.4 by depicting virtual positions.
law only transforms activities, and as such does not apply state changes in the simulated environment. The outcome of a law is expressed in terms of activities only. We illustrate this by means of the example in Fig.4. Suppose we have an activity law that checks if collisions occur between several move activities, and that this law is applied to activity A-2. All activities of which time intervals overlap with the time interval (1,5) of activity A-2 are also considered. Both A-1 and A-3 satisfy that condition. The law then checks for collisions between the three activities. A collision is detected between activity A-2 and A-3, as they represent two AGVs moving over a crossroad at the same time. The law could then transform the set of three activities of Fig.4 into a set of five activities depicted in Fig.5. Activity A-2 and A-3 have been modified and now represent the move activities of agent A and agent B until the moment of their collision at time instant 3.6. Activities A-4 and A-5 have been added by the law and represent the remaining movement of agent A and agent B after the collision. 4
A
2
2
A
P
A
A
S
3
B
A
A
A
F
F
V
5
1
2
A
F
2
A
F
4
A
A
F
3
A
A
