and âQuake 3 Arenaâ [14] to guide the behaviour of non-human characters. When dealing with a physical environment, a distributed storage capability should be ...
Gradient Field Based Order Assignment in AGV Systems Wannes Schols Nelis Bouck´e Danny Weyns Tom Holvoet Report CW 425, September 2005
Katholieke Universiteit Leuven Department of Computer Science Celestijnenlaan 200A – B-3001 Heverlee (Belgium)
Gradient Field Based Order Assignment in AGV Systems Wannes Schols Nelis Bouck´e Danny Weyns Tom Holvoet Report CW 425, September 2005 Department of Computer Science, K.U.Leuven Abstract This research examines the feasibility of using a field-based approach, within the multi-agent system (MAS) paradigm, to achieve transport assignment in an automatic guided vehicle (AGV) system. In a field-based approach, transports emit fields into the environment. The AGV’s behavior consists of continuously combining received fields and following the gradient of these combined fields. This will guide towards pick locations of transports, much like a ball rolls towards a valley in a (continuously changing) mountainous landscape. To avoid multiple AGVs driving towards the same transport, AGVs emit repulsive fields. The AGVs continuously reconsider the situation of the environment and transport assignment is delayed until a pickup, which benefits the flexibility of the system. The field-based approach is implemented in the AGV simulator developed by the AgentWise task force and its performance is examined. Experiments indicate that the field-based approach outperforms a Contract Net based protocol on various performance measures, such as the average time a transport has to wait for execution and the throughput, and in various circumstances, such as a varying number of AGVs and transports in the system. However, limitations of the field-based approach include AGVs driving superfluous distance and an unequal distribution of wait times across different pick locations. Also, it is not feasible to control, predict or reason about the emergent system-wide behavior or to provide guaranteed reaction times. Additionally, the tuning of the parameters of the approach often proved to require a time consuming trail-and-error approach. Finally, it remains to be seen if the large amount of messages that are transmitted in the approach can be supported by a real-world AGV-system. Keywords : gradient field, task assignment, situated multi-agent systems. CR Subject Classification : C.2.4 AMS(MOS) Classification : Primary : 68W15, Secondary : 68M14.
Contents 1 Introduction
1
2 Multi-Agent Systems and the application
4
2.1
2.2
Multi-Agent Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
2.1.1
Motivation of MAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
2.1.2
Definition of MAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
The AGV-system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
2.2.1
The real-world AGV-system . . . . . . . . . . . . . . . . . . . . . . . . .
7
2.2.2
The simulated AGV-system . . . . . . . . . . . . . . . . . . . . . . . . . 10
3 Related work
13
3.1
Field-based techniques in MASs . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2
Task assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4 Using a gradient field-based approach for transport assignment
18
4.1
High-level overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.2
Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.3
Behaviour of agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.4
4.3.1
AGV- and transport-agent: Communication . . . . . . . . . . . . . . . . 22
4.3.2
Transport-agent: Calculation of priority and field-range . . . . . . . . . . 25
4.3.3
AGV-agent: Filling the look-ahead buffer . . . . . . . . . . . . . . . . . . 27
4.3.4
AGV-agent: Performing an action . . . . . . . . . . . . . . . . . . . . . . 30
4.3.5
AGV-agent: Behaviour after picking up the packet of a transport . . . . 31
Construction of the calculated-field . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.4.1
Single AGV-agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
ii
4.5
4.4.2
Multiple AGV-agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.4.3
Special case: transport on target station . . . . . . . . . . . . . . . . . . 36
Remark about straight-line and logical distances . . . . . . . . . . . . . . . . . . 37
5 Implementation
39
5.1
Design of the existing AGV simulator . . . . . . . . . . . . . . . . . . . . . . . . 39
5.2
Positioning of our implementation within the AGV simulator . . . . . . . . . . . 40
5.3
Adaptations to the Graphical User Interface . . . . . . . . . . . . . . . . . . . . 42
6 Results and discussion
44
6.1
The layout used for testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6.2
Comparison with a simple reference algorithm . . . . . . . . . . . . . . . . . . . 44
6.3
6.2.1
Overview of the reference algorithm . . . . . . . . . . . . . . . . . . . . . 45
6.2.2
The parameters and their default values . . . . . . . . . . . . . . . . . . 46
6.2.3
Comparison using the default parameters . . . . . . . . . . . . . . . . . . 47
6.2.4
Comparison under varying number of AGVs and transports . . . . . . . 49
6.2.5
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
The effects of various parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 51
7 Conclusion 7.1
7.2
57
Discussion of the approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 7.1.1
Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
7.1.2
Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Applicability beyond the AGV-simulator . . . . . . . . . . . . . . . . . . . . . . 59
A Realistic AGV-system layout
61
B Class diagrams
62
C Interaction diagrams
64
iii
Chapter 1 Introduction The subject of this thesis is transport assignment in an Automated Guided Vehicle (AGV) system, through the use of a field-based approach within the multi-agent system (MAS) paradigm. Context Applying a MAS [1, 2] means that the control of the system is distributed over numerous agents, which are located in an environment. The agents interact with each other and with the environment to achieve the system goal. An agent operates proactive, autonomous, and reacts readily to changes in the environment. There is an upcoming use of MASs as a paradigm to develop self-organizing distributed systems, with the most important being flexibility, adaptivity and openness. Recently, techniques based on physical, mechanical properties have emerged in the MAS context. In particular, field-based techniques [1, 3], where fields lead agents to their goals, have proven effective in applications in a metric space. The application central in this thesis is an AGV-system. An AGV-system is a fully automated industrial transport system that makes use of numerous AGVs. An AGV is a battery powered, computer controlled, unmanned vehicle that is capable of transporting goods in an industrial environment. The AGVs are restricted to follow predefined paths, on which they navigate by using sensors and stationary beacons. Transports, which consist of picking up a packet on a certain location (the ’pick location’) and transporting it to certain destination (the ’drop location’), are generated by an external system and have to be assigned to the AGVs. A traditional way to control an AGV-system is through a central server [4]. This server maintains all relevant information, which is used to to decide which AGV has to do what and which route is to be taken. By applying the MAS paradigm to an AGV-system, the control is distributed over the different AGVs, who cooperate to achieve the goals of the system. Substantial research has been done in this area, eg. EMC2 , an industrial project between Distrinet and Egemin (a company that develops advanced and automated systems for existing industrial environments) [4]. This thesis focusses on a simulation [5] of an AGV-system, developed by the AgentWise workgroup. The differences between the simulation and a real-world, physical AGV-system will be considered in this thesis.
1
Problem statement In an AGV-system, transports are dynamically created when needed and the transport stream is irregular and unpredictable. The problem this thesis focusses on is assigning the transports to AGVs in an ’satisfactory’ way. It’s hard to define ’satisfactory’, as various performance measures can be associated with transport assignment, eg. the average time a transport has to wait before it is executed, the throughput, the average distance AGVs have to cover to execute a transport, etc. Goals The main goal of this thesis is to examine the feasibility of using a field-based approach (within the MAS paradigm) to achieve transport assignment in an AGV-system. In a field-based approach, transports emit attractive fields that guide AGVs towards them. We aim to compare the advantages of the approach to the limitations. An expected advantage is flexibility, as a result of delaying transport assignment until an AGV actually picks up the packet of the transport. Other expected advantages include, robustness and openness, as a result of not using strict protocols. However, difficulties are expected related to the realization of fields in an AGV-system, for example, a high amount of messages might be needed. Also, it is expected that the emergent behaviour will be hard to control and reason about. To achieve this main goal, the following sub-goals are defined. Firstly, a field-based approach will be developed and implemented in an AGV simulator [5]. This software already contains a MAS-framework, which allows us to focus on the field-based behaviour of agents. A second sub-goal is to measure the performance of the approach. Various performance measures will be used, including the average time a transport has to wait before it is executed, the throughput, the average distance AGVs have to cover to execute a transport and the number of messages that are transmitted. To give the performance measurements more meaning, a reference algorithm for transport assignment will be implemented. The Contract Net protocol [1] is chosen, since it is well known and easy to implement. Additionally, the effects of various parameters and circumstances (such as the number of AGVs) on the performance of the field-based approach will be examined. Lastly, although this thesis focusses on a simulated AGV-system, the real-world counterpart is kept in mind throughout this thesis. We hope that our approach can be applied in a real-life AGV-system without major adaptations. We will also investigate how our approach can be combined with concepts other than transport assignment. Overview In chapter 2, the context of this thesis is described in more detail. Firstly, a MAS and its properties are discussed. Next, a typical real-word AGV-system is described, followed by the simulation [5] that is used in this thesis. The differences between the simulation and a real-word AGV-system are discussed. Next, an overview of the related work on field-based techniques and task assignment is given in chapter 3. Chapter 4 discusses the developed field-based approach for transport assignment in detail.
2
Chapter 5 discusses the implementation of the field-based approach in the existing MASframework of the AGV simulator [5]. Also discussed are the adaptations made to the GUI of the AGV simulator, in order to visualize the field-based concepts. In chapter 6 the performance of the field-based approach is examined by comparing various performance measurements with a Contract Net protocol. The effect of various parameters and circumstances (such as the number of AGVs) is also examined. From these measurements, the advantages and limitations of the field-based approach are pointed out. A general conclusion of this thesis is given in chapter 7.
3
Chapter 2 Multi-Agent Systems and the application This chapter discusses the context of this thesis in more detail. First, Multi-Agent Systems (MASs) are discussed. Next, the application this thesis focusses on, the AGV-system, is described in more detail. The differences between a real-world and simulated AGV-system are highlighted.
2.1
Multi-Agent Systems
The general ideas about MAS are based on the course notes from Prof. T. Holvoet [6], a book from J. Ferber [1], a book from M. Wooldridge [2], a Master Thesis from Nelis Bouck´e [4], a paper from Danny Weyns and Prof. T. Holvoet [7] and own ideas.
2.1.1
Motivation of MAS
During the last decades, a shift in focus has occurred in the computer world. Applicationdomains and systems have become more distributed. The reasons for this distribution include application-domains that involve physical distribution (eg. traffic control, telecommunication networks,...), the emerging trends in networking (eg. mainframes are replaced by networks of many interconnected PC’s, pervasive computing,...), problems that are too complex to be handled by a single, centralized system and systems that are naturally modelled as multiple interacting entities. A classical way to tackle distributed algorithms is by controlling them from a central point. Although this approach is simple and easy to maintain, the single point of control may become a bottleneck and single-point-of-failure. Decentralization is an upcoming trend which aims to deal with the scalability issues of a centralized system. MASs are particulary suited for modelling decentralized systems, where the control is distributed over multiple agents. The agents are autonomous entities that are embedded in an environment and that cooperate with each other in order to reach a common goal. The intelligence of a MAS emerges from the (local) interactions among agents themselves and between agents and the environment. A MAS is suitable for systems in a complex, dynamic environment that is only partially and locally perceivable. 4
Although a MAS can achieve flexibility, adaptivity, openness and robustness, there are some consequences of applying a MAS one should keep in mind. Firstly, the global system behaviour emerges in a complex, dynamic and often non-deterministic way from the actions and interactions of the individual agents. This makes it practically infeasible to control, debug, predict or proof formally the correctness of the system-wide behaviour, although an approach that allows to analyze and assess trends in system-wide behaviour has been proposed [8]. Another problem with MASs is that they might require a lot of communication, placing a burden on the communication-infrastructure. Finally, the MAS-approach isn’t fit for every problem, it’s no ’silver-bullet’. Next, a MAS is defined in more detail.
2.1.2
Definition of MAS
A MAS can be defined as the combination of the following aspects: • an environment • agents, with the following capabilities: – actions (performed in the environment) – perceptions (coming from the environment) – sending messages to other agents First, the agents are described in more detail, followed by the environment. Agents Agents are autonomous entities that are situated in the environment and that cooperate with each other in order to meet the system goal [2, 1]. In this thesis, a specific class of agents is used, namely situated agents [7]. Specific to a situated MAS is the emphasis on the fact that an agent is not an unrelated entity, but is explicitly situated in the environment. Decisions of an agent are based on the current situation in which the agent resides. Figure 2.1 shows a high-level view of the agent and the environment. The agent perceives the environment locally and performs actions in the local environment. Important to note is that an action is merely an attempt to produce a certain effect in the environment, which is called an influence [1]. The effect of an influence is non-deterministic. That is, an agent cannot rely on a predefined outcome of an action. The action may fail or have a different outcome.
Figure 2.1: High-level view of the agent and the environment.
5
This realistic limitation is one of the reasons an agents need to be flexible. Other important properties of agents include: • reactivity: An agent reacts readily to stimuli, coming from the environment or from received messages. • autonomy: An agent acts on its own, it doesn’t need interaction from users. • proactive-ness: An agent doesn’t just respond to stimuli, it also initiates appropriate actions to achieve its goals • social abilities: An agent can cooperate with other agents, coordinate its actions and communicate. This thesis focusses on situated agents. Some other types of agents include [6, 1]: • Deductive reasoning agents: These agents are based on ideas from symbolic AI. An agent maintains a symbolic description of knowledge (eg. the environment, its behaviour, its goals, ...) Decisions are then made by manipulating those symbols, eg. by theorem proving. • Practical reasoning agents: Whereas deductive reasoning agents reason towards knowledge, practical reasoning agents reason towards actions. First, the agent decides what state of affairs it wants to achieve (intentions). Then, it decides how it wants to achieves this state of affairs (plans). • Reactive agents: A reactive agent selects an action based only on the current perceptions. The agents tempt to be very simple, lacking internal state. • Hybrid agents: These agents combine concepts from reasoning and reactive agents. Environment In a MAS, agents and the environment are complementary entities. The interactions between the two constitute the intelligence of the MAS. The environment may contain objects, which can be manipulated by agents. A realistic environment has the following properties: • only locally and partially observable: Agents can only perceive their immediate surroundings and these perceptions may be incomplete. • non-deterministic: Actions in the environment can fail or have an unexpected effect. • dynamic: Besides agents acting on the environment, the environment is under constant change (eg. moving objects) • provides a communication-infrastructure: The communication-infrastructure accepts a message from a sending agent and delivers it at the receiving agent(s). Note that communication is not necessarily deterministic: message deliverance may not be guaranteed.
6
Figure 2.2: Model of the environment that is used in this thesis.
The environment can either be physical or implemented in software. In this thesis, an existing model of the environment is used [1, 7, 9, 10], as shown figure 2.2. The communication-infrastructure provides message deliverance. The collector collects the influences, produced by agents when performing an action. The reactor takes as inputs a collection of recent influences, the current state of the environment and the laws of the modelled MAS to calculate the effects of the influences. The effector realizes these effects by updating the state of the environment and making new perceptions available to the agents. The environment will not be discussed in greater detail, as the focus of this thesis is on the behaviour of agents: how agents relate inputs (perceptions, messages) to outputs (actions, messages).
2.2
The AGV-system
The application this thesis focusses on is a simulation of an Automated Guided Vehicle (AGV) system. Subsection 2.2.1 describes a typical real-world AGV-system. Next, the simulated AGV-system that is used in this thesis is discussed in subsection 2.2.2.
2.2.1
The real-world AGV-system
An AGV-system is a fully automated industrial transport-system, where multiple AGVs (Automated Guided Vehicle) are used to transport packets (such as materials, products,...) in an environment such as a warehouse. This subsection describes a real-world AGV-system as it is used by Egemin [11]. This is a company that develops advanced and automated systems for existing industrial environments. The system is described in different parts: the map of the environment, the transports and the AGVs. Next, a concrete transport example is given and the essential responsibilities of the system are defined.
7
Figure 2.3: An abstract illustration of the map of the environment.
The map of the environment AGVs are confined to certain routes in the environment, which are defined in the map of the environment, also called the layout. The map is a network of stations that are connected by segments, as shown in figure 2.3. A station is a predefined point on the map that can hold at most one AGV at a time. Stations are needed because the length of a segment is limited. Stations can also hold special locations, such as a parking place for AGVs, a place where AGVs can reload their battery, a pick or drop location for transports,... A segment defines in which direction it can be used (bi-directional is possible) and whether an AGV must drive forwards or backwards on it. Note that the map in figure 2.3 is a simplified map. A realistic map contains additional information such as speed limits, mutexes (indicating that only one AGV can enter a pair of segments at a time),... An example of a realistic map is given in appendix A. Transports The goal of the AGV-system is to let AGVs transport packets. A transport is created by an external system, eg. a management-system of a warehouse. It is the order to transport a packet, located on a certain station (the ’pick location’), to a certain target station (the ’drop location’). Each transport has a priority from 1 to 10, with 10 being the most important. A transport’s life-cycle starts when it enters the system. The transport is started when an AGV picks up the packet and is terminated when that AGV drops the packet at the drop location. AGVs An AGV is an unmanned, computer controlled vehicle, capable of picking up packets, driving them around and dropping them. An AGV is confined to driving on the network of segments and stations. An AGV navigates by storing an internal map of the environment and using internal and external sensors, in combination with stationary beacons, to correct its position. The low-level control is outside the scope of this thesis; it suffices to say that an AGV is capable of following the predefined paths of the map. To allow smooth and efficient movement, an AGV has to make routing-decisions over a certain distance in advance. This distance, called the look-ahead, is not a straight-line distance, but is measured according to the network of stations and segments. The look-ahead buffer associated with an AGV is an ordered array that holds these routing-decisions. It represents a path that the AGV will follow with certainty, unless something catastrophical (such as a crash) happens. 8
Figure 2.4: An AGV and its associated look-ahead buffer. The red line represents the reach of the look-ahead.
Figure 2.4 shows an AGV and its associated look-ahead buffer. Since its already determined that the AGV will drive straight ahead to station B when arriving on station A, a higher driving speed can be obtained. An AGV is powered by a battery, which can be reloaded at designated locations. AGVs can communicate by means of a wireless LAN-network. Responsibilities A typical scenario in the AGV-system is as follows: 1. A transport is created by an external system. The responsibility of the system begins. 2. The system selects an AGV to execute the transport. 3. The AGV drives towards the the pick location associated with the transport and picks up the packet. 4. The AGV drives towards the drop location associated with the transport and drops the packet. 5. The transport is completed. The responsibility of the system ends. In the light of this scenario, the essential responsibilities of the system are defined. The first responsibility is assigning transports to AGVs in an (near-) optimal way. The second responsibility is that an AGV has to reload his battery in time. Care must be taken that an AGV has enough energy to execute an assigned transport. The third responsibility is the routing of AGVs across the map. The final essential responsibility of the system are avoiding deadlocks (where multiple AGVs are blocking each other indefinitely) and collisions. The focus of this thesis on achieving transport assignment, by applying a field-based approach (within the MAS paradigm) (see chapter 4). This approach also achieves routing, although each AGV is required to have a router (an existing router is used). 9
2.2.2
The simulated AGV-system
The application this thesis focusses on is not a real-world AGV-system, but a simulation of an AVG-system. In particular, the AGV simulator developed by the AgentWise workgroup [5] is used. The AGV simulator contains a framework for MASs. The environment, actions, perceptions and communication are already modelled and implemented. The user has to implement the actual behaviour of the agents and of the external system that generates the transports. Chapter 5 discusses internal structure of the AGV simulator in more detail, as well as how the field-based approach, described in chapter 4, is implemented in the AGV simulator. Next, the Graphical User Interface (GUI) of the AGV simulator is described. After this, the differences between the AGV simulator and the real-world AGV system are discussed. The Graphical User Interface The GUI of the simulator is shown in figure 2.5. In the top of the screen, buttons are provided for starting, stopping and pausing the simulation, as well as zooming in and out. Also, in the upper right, a layout of the environment can be chosen. Layouts can be built by the user in XML-format. On the right, checkboxes can be used to turn various graphical representations on and off. The GUI shows a graphical representation of the running simulation. AGVs, segments and stations are depicted. The arrows on the segments represent the allowed driving direction. Pick locations are represented by an arrow pointing inwards. An arrow pointing outwards represents a drop location. In the figure, the AGV in the middle is carrying a packet. Furthermore, communication between AGVs is shown. The transparent red circles show that an AGV is broadcasting in a certain range. Message reception is depicted by a little envelope. Unicasts are also shown by drawing a line between two AGVs (not visible in the figure). The bottom part of the screen shows an overview of the transports (the older term ’order’ is used in the AGV simulator) currently in the system, including their priority, pick and drop location and their creation time. Differences from a real-world AGV-system The AGV simulator makes some simplifications and abstractions in relation to the real-world AGV-system. The impact these simplifications and abstractions have on our approach (described in chapter 4) is discussed. • In the simulator, the AGVs drive at a constant speed : they reach their top speed immediately and can come to a halt immediately. This means that in the simulator, the use of a look-ahead buffer has no advantage in terms of smoother and more efficient driving. However, to minimize the impact of this simplification, we implemented a look-ahead buffer in our approach. • The segments in the simulator are always straight, as opposed to the real-word segments, who are often curved. Additionally, the segments in the simulator don’t specify whether AGVs have to drive forwards or backwards on them; AGVs always drive forwards in the AGV simulator. These simplifications have a small impact on our approach, since 10
Figure 2.5: The GUI of the AGV simulator.
decisions are made on a station-by-station basis; how (which specific trajectory and which driving direction) an AGV drives between two adjacent stations is of less importance. • In the simulator, AGVs communicate in a simplified way. A sent message is immediately delivered to the receiving AGV. This is an abstraction from real-world communication, where messages have to traverse a communication-infrastructure, which means messages can be delayed, lost, or arrive in a different order. However, since no rigorous communication protocols are used in our approach, the impact of an occasional lost, delayed or unordered message would be small. In the simulation, an unlimited amount of bandwidth is available, which is certainly not the case in a real-world AGV-system. Since our approach requires a considerable amount of messages to be transmitted (as shown in chapter 6), it remains to be seen if our approach can be effectively used in a real-world AGV-system with typical bandwidth limitations. • In the AGV simulator, no concept of batteries is used: the AGVs can keep driving forever without reloading their batteries. Since reloading batteries is not in the scope of this thesis, this simplification is acceptable for our approach. Chapter 7 discusses how the reloading of batteries could be combined with our approach. • A simple and unrealistic collision avoidance system is used in the simulator. AGVs lock the station they are driving to and release the lock of a station when they arrive on the next station. A station can only be locked by one AGV at a time. When an AGV wants to drive to a station that is already locked by another AGV, it has to wait until the lock is released. 11
(a)
(b)
Figure 2.6: Collision can still occur with the collision avoidance system of the AGV simulator.
This mechanism is unrealistic in the sense that AGVs have a global knowledge of which stations are locked. Realistically, AGVs only have knowledge about stations locked by them self and this knowledge has to be exchanged between AGVs. The collision avoidance system is also too strict, delaying AGVs unnecessarily long. Also, although this mechanism avoids collisions in most cases, AGVs can still collide when the layout contains crossing segments. Figure 2.6 illustrates this. AGV 1 wants to drive to station B and locks it. AGV 2 wants to drive to station A and locks it (figure 2.6-a). Since both locks are granted, both AGV start driving and collision occurs (figure 2.6-b). However, this collision avoidance system is sufficient for this thesis, since collision avoidance is not in the scope of this thesis. Chapter 7 discusses how a more realistic and effective collision avoidance system could be combined with our approach. • In the simulator, no deadlock avoidance is applied. A deadlock is the event where AGVs block each other indefinitely. Again, note that deadlock avoidance is not in the scope of this thesis. Chapter 7 discusses how a deadlock avoidance system could be combined with our approach. Since this thesis focusses on transport assignment in the AGV simulator, the simplification and abstractions of the simulator mentioned above have no large impact. However, when implementing our approach in a real-world AGV-system, the impact of the various simplifications and abstraction have to be considered. Chapter 7 discusses the feasibility of implementing our approach in a real-world AGV-system in more detail. The outline of this thesis is as follows. Chapter 3 discusses field-based techniques in MASs and gives an overview of the research on task assignment. Chapter 4 discusses our field-based approach for transport assignment (which qualifies as task assignment) in an AGV-system. As mentioned before, the focus is not on a real-world AGV-system, but on the AGV-simulator. How this approach is implemented in the AGV-simulator is illustrated in chapter 5. Chapter 6 gives a discussion on the performance of our approach, highlighting some advantages and problems. Finally, a general conclusion is given in chapter 7.
12
Chapter 3 Related work In this chapter, two research domains central to this thesis are discussed. Section 3.1 discusses field-based techniques in MASs. Section 3.2 gives an overview of the research on task assignment. Transport assignment in an AGV-system is a specific example of task assignment.
3.1
Field-based techniques in MASs
Situations concerning the movement of agents in a metric space can be approached in two ways [1]. In the first approach, agents maintain an internal representation of the environment and plan their movement through symbolic and logical analysis. In a second, more recent approach, the behaviour of agents is treated as governed by the laws of physics and mechanics. More specifically, techniques based on fields have proven to be interesting in various applications that involve movement of agents in a metric space [1, 3, 12, 13, 14, 15]. In this approach, objects in the environment produce fields, which are propagated in the environment in a certain range. These fields have at each position in the metric space a certain value, forming potential fields. These values are typically proportional to the distance from the source of the field. Agents receive these fields and combine them in a certain way. The behaviour of an agent then simply consists of following the combined potential field ’downhill’, that is, they follow the direction of
Figure 3.1: The agent has to reach his goal while avoiding obstacles. A field-based approach is used [1].
13
(a)
(b)
Figure 3.2: Guards patrolling a museum want to keep a fixed distance between each other as they move around [3, 6]. For this, the guard-agent emits the field shown in (a). (b) shows the resulting pattern of 4 guards.
the gradient of the field. As an example, suppose an agent has to move to a goal while avoiding obstacles in the environment [1]. The obstacles emit repulsive fields, while the goal emits an attractive field. The agent combines the fields by adding the values of the attractive and repulsive field. The combined field can be visualized as a landscape, with the goal representing a valley and the obstacles mountains, as shown in figure 3.1. This combined field is then followed ’downhill’, which can be viewed as a ball rolling down the slopes of the landscape towards the goal. The field-based approach readily extends to task allocation, where each task emits an attractive field and agents follow the strongest field. Collision avoidance between agents can be achieved by letting the agents emit repulsive fields. The field-based technique has also been applied in systems that are inspired by biological phenomena. For example, birds in a flock all fly in the same direction and avoid collision by keeping a fixed distance between each other. An example application for this idea are guards who have to patrol a museum, keeping a certain distance between each other to cover more ground [3, 6]. This can be achieved by letting the guard-agents emit the field shown in figure 3.2-a. Figure 3.2-b shows the resulting pattern of 4 guards. A variation on the field-based approach where agents construct a field in their direct neighborhood, is applied to achieve routing and deadlock-avoidance in a simplified AGV-system [12]. Another variation is described in [13], where the MMASS (Multilayered Multi Agent Situated System) formal model for multi-agent coordination is used. In this model, the environment is represented as a multi-layered graph in which the agents can spread abstract fields. In the standard field-based approach, agents combine perceived fields and are constantly guided by the fields, while in MMASS fields are considered independent from each other and are exploited only to trigger one-shot reactions. The field-based approach is an elegant and effective way to tackle problems situated in a metric environment. The agents can be kept simple, relying on the environment (more specifically fields in the environment) to achieve the system objective. Since no rigorous protocols are used, robustness and openness (agents can be added or removed from without any extra burden) are achieved. The approach also benefits from adaptivity: the landscape of fields is readily adjusted to changes in the environment. Also, the fields can be interpreted and combined in different 14
(a)
(b)
Figure 3.3: (a)The field emitted by room A. (b)The field representing the number of tourists in different rooms. [3]
ways by different agents (eg. a location may be a goal for one agent, while another agent needs to avoid it). However, the field-based approach also suffers from some limitations. Firstly, the approach is inapplicable when the situation cannot be represented as a metric space. Secondly, local minima can arise. For example, consider again figure 3.1. If one of the obstacles is concave, the agent can become stuck at a local minima. This can be remedied by letting the attractive field of the goal diffuse around the obstacles, instead of using straight-line distances. For this to work however, the environment has to be digitized (broken down into a set of cell of equal size). The field can then be diffused breadth-first from cell to cell, each time increasing the value of the field. Another problem is that the approach is hard to combine with, for example, intentions and planning. Finally, it’s hard to predict or reason about the overall evolution of the system; there is an emergent global behaviour. An important issue in using the field-based approach is how the fields are maintained and distributed within the environment. The environment should have a distributed storage infrastructure to store the different values of fields in different (digitized) positions in the environment. When dealing with a software environment, this requirement is automatically achieved. For example, the field-based approach has been successfully used in games such as ”The Sims” [16] and ”Quake 3 Arena” [14] to guide the behaviour of non-human characters. When dealing with a physical environment, a distributed storage capability should be physically supported by the environment. For example, in [3], the field-based approach is applied in guiding tourists in a museum. The tourists are provided with a software agent running on some wireless handheld device, giving suggestions on how and where to move. A computer network with a topology that mimics the topology of the museum plan is embedded in the museum walls. The hosts in the network are associated to each room and are capable of communicating with each other and the mobile devices in their proximity. Fields can be injected at a host, after which they will be diffused hop-by-hop across the network, modulating the values of the fields as necessary. The latter can be achieved by running middleware, such a TOTA (’Tuples On The Air’) [15]. As an example, a tourist that wants to go to a certain room, while avoiding large crowds, can follow a field ’downhill’, composed of a room-field, shown in figure 3.3-a, and a crowd-field, shown in figure figure 3.3-b. 15
Fields can also be realized by broadcasting information messages, which contain the position of the source of the field. Agents receive these messages and calculate the value of the field on their position. Notice that an increased burden is placed on the agents. This approach is applied in this thesis, as described in chapter 4, since a typical AGV-system (section 2.2) supports no distributed storage infrastructure.
3.2
Task assignment
Efficiently assigning tasks to individual entities in a decentralized system (eg. a MAS) is a common, but certainly not a simple problem [1]. A specific example is the assigning of transports to AGVs in an AGV-system. This thesis makes use of field-based techniques to achieve task assignment. Here, other approaches for task assignment are discussed. It is not our intent to provide a full overview, but rather to provide some typical examples. One way to tackle task assignment is by using a central entity (eg. an agent, a server,...) with global knowledge about all entities (eg. agents) to decide which entity is to be assigned to a task [1]. For example, [4] describes a centralized AGV-system, where a central server with global knowledge about the environment assigns transports to AGVs and decides how AGVs have to drive. Stigmergy [1, 6, 17] is an approach that is reactive and environment-driven, just like the fieldbased approach. This technique is inspired by the behavior of biological insects like ants, who drop and sense (by smelling) pheromones in the environment. For example, ants achieve food foraging by dropping pheromones when returning to the base with food. Other ants can then follow this pheromone trail to the food source. When applied in MASs, pheromones can only be used in case of a software environment or when there is an infrastructure for simulating the dropping of pheromones. An example of the latter is the use of a virtual environment in AGV-systems, as proposed in [4]. In [18], an approach is described for allocating tasks that are dynamically perceived from the environment and can be tied by execution constraints. An example is a collection of robots that drive around and perceive objects in the environment. The task associated with an object is picking it up and driving it to a certain location. However, two robots need to cooperate to pick up the object. Two roles are associated with the task: a helper robot and a collector robot. The approach is based on tokens, which are associated with the roles of a task. The tokens can be passed between agents and whoever contains a token is assigned to the task in the associated role. An other way to achieve task assignment is through auction like mechanisms, eg. the widely known and extensively used Contract Net (CNET) protocol [1]. CNET is based on the notion of call for bids on markets, where ’managers’ request for bids and ’bidders’ bid to perform the task. In a first step, the manager sends a description of the task to perform to all the bidders. In a second step, the bidders draw up a proposal based on the description of the task and send it to the manager. In a third step, after the manager has received a proposal from all bidders or after a deadline has expired, the manager evaluates the received proposals and assigns the best bidder the task by sending him a message.
16
In the original CNET protocol, a bidder that has placed a bid for a task (step 2), cannot place a bid for a new task before the protocol with the first task has ended. In [19], an improvement to the CNET protocol is proposed that allows a bidder to place bids for multiple unassigned tasks. This is achieved by giving the bidder the option to refuse the task in step 3 of the original protocol. The manager will then award the contract to the second best bidder, who can again accept it or refuse it, etc. The CNET protocol can also be expanded to allow for cascading task assignment, where the receiver of a task proposal starts one or more new CNET protocols to delegate the task or parts of the task to other agents [19]. For example, a system of holonic agents can be used to improve efficiency when dealing with complex tasks that require the collaboration of several agents [20]. A holonic agent consists of several agents but interacts with agents outside the holon as a single agent. The CNET protocol suffers from low flexibility when dealing with tasks that can not immediately be started upon the allocation. Consider for example the CNET protocol applied to transport assignment in an AGV-system. When a new transport enters the system, a suitable AGV is selected through the CNET protocol. An AGV is thus assigned to the transport as soon as the transport enters the system. Suppose that while the assigned AGV is driving towards the pick location of the transport, a new transport appears that is more suitable to the AGV (eg. it is closer or has a higher priority). The AGV will not be able to execute the second transport, since it is already assigned to the original transport. Thus, the CNET protocol achieves an immediate and permanent (once a task is assigned to an agent, the agent can not consider other tasks until the original task is finished) task assignment, preventing the system to react flexibly and adaptively when dealing with tasks with delayed start. In [21], an approach for flexible and decentralized task allocation is described that deals with the problem mentioned above. The approach consists of a continuous negotiation protocol where the situation and the allocation of tasks is continuously reconsidered until the task is actually started. The next chapter describes an approach that achieves transport assignment in an AGV-system by using field-based techniques. Similar to the approach in [21], assignment is delayed until the AGV actually reaches the pick location of the transport, allowing the system to react flexibly and adaptively to new transports entering the system.
17
Chapter 4 Using a gradient field-based approach for transport assignment In this chapter, a gradient field-based approach for transport assignment in an AGV-system is explained. The approach is build into an existing, generic MAS model [1, 7, 9, 10]. Inspiration is drawn form field-based techniques in MASs [1, 3, 12, 13, 14, 15], described in section 3.1. Section 4.1 gives a high-level overview of the approach. Section 4.2 discusses the architecture of the MAS adapting the field based approach. In section 4.3, the different modules of the architecture and how they interact are discussed in detail. The construction of the calculatedfield is an important part of the approach and is discussed in a separate section, section 4.4. Finally, a remark is made about the concepts of straight-line and logical distance in section 4.5. The approach described in this chapter focusses on the AGV-simulator (see subsection 2.2.2). However, a real-life AGV system is kept in mind; we hope the approach is applicable to a real-life AGV-system without major adaptations.
4.1
High-level overview
The developed approach achieves transport assignment by making use of fields. The approach is described on a high-level by means of the scenario in figure 4.1: • AGV- and transport-agents: The transport assignment is achieved by the interaction between two kinds of agents: AGV-agents and transport-agents. The AGV-agents correspond to the physical vehicles and the transport-agents correspond to unassigned transports (transports whose associated package is not yet yet picked up by an AGV). This means that whenever a new transport enters the system, a corresponding transportagent is created. The lifetime of the transport-agents ends when the associated package is picked up. On which physical computer system(s) the transport-agents reside is deliberately left open, the only important issue is that the agents operate independently and by this can be distributed across several computer systems. A transport agent conceptually resides at the pick location of the associated transport. • Agents emit fields: Both AGV- and transport-agents emit fields, called AGV- and transport-fields, as shown in figure 4.1-a. Fields have a certain range and contain information about the sending agent. 18
(a)
(b)
(c)
Figure 4.1: Example scenario of the developed approach. The full circles represent transportfields, the dotted circles AGV-fields. The fields are not drawn in (b) and (c) for clarity.
19
• AGV-agents store received fields: At a given time, an AGV-agent can lie in the range of a number of fields. The AGV-agent stores the data contained in these fields in a field-cache. A cache-entry contains the source of the received field (an ID), the most recent data contained in that field and a freshness. The freshness is a measure of how up-to-date the cached data is. Figure 4.1 shows the field-cache of AGV A through the scenario. • AGV-agents construct calculated-fields to decide their movement: An AGVagent constructs a calculated-field to decide in which direction to drive from a station. A calculated-field is a combination of the received fields, which are stored in the field-cache. The lower the freshness of a cache-entry, the lower the influence of the associated field on the calculated-field. The calculated-field is constructed locally and temporarily on a station and contains values for each outgoing segment. An AGV-agent follows the calculated-field in the direction of the smallest value. This can be considered as following the calculated-field ’downhill’, in the direction of the gradient. Transport-fields have an attractive influence on the calculated-field, which results in AGVs driving towards unassigned transports. However, it is undesirable that many AGVs drive to the same transport, since they will obstruct each other and travel superfluous distance. To remedy this, AGV-fields have a repulsive influence. In figure 4.1-b, AGV A constructs a calculated-field on a station. Although transport 1 is closer, the calculated-field will guide AGV A towards transport 2. This is the result of the repulsive effect of AGV B. Notice that it would have been ineffective for AGV A to drive towards transport 1, since AGV B is closer and will probably reach the transport first. • Transport assignment occurs at pick up: Transport assignment is delayed until an AGV actually reaches the pick location and picks up the package. This results in a greater flexibility with respect to transport assignment, in relation to for example when the CNET protocol is used, which assigns a transport as soon as it enters the system (see section 3.2). By delaying transport assignment, the field-based approach can cope with unforseen situation like transports which suddenly popup, as illustrated in figure 4.1-c. While AGV A is driving towards transport 2, a new transport (transport 4) appears close to AGV A. Since no transport has been assigned to AGV A yet, it can drive towards the closer transport 4.
4.2
Architecture
In order to develop the MAS based on the described field-based approach, we start from an existing, generic model for the environment, perception, communication and actions in a situated MAS [1, 7, 9, 10], see chapter 2. This chapter focusses on the development of the behaviour of the AGV- and transport-agent inside this generic model. The architecture of the AGV- and transport-agent are shown in figure 4.2. The different components that we developed are quickly introduced below; a more in-depth discussion is given in the next section.
20
Figure 4.2: The architecture of the AGV- and transport-agent.
• Field-cache: This module of the AGV-agent stores the information of fields received from other AGV-agents and transport-agents in cache-entries. A freshness is associated with each cache-entry, which is a measurement of how up-to-date the entry is. • Router: This module contains a global map of the stations and segments. The router is used by the AGV-agent to calculate paths and distances from one station to another. We used the existing router in the AGV simulator [5]. This is a static router that uses the A*-algorithm. This means that dynamic concepts such as traffic distribution are not taken into account. However, the approach discussed in this chapter is fully compatible with a dynamic router and it would be interesting to examine how the performance would be enhanced by using a dynamic router. • Field-calculator: This module constructs a calculated-field locally and temporarily on a target station by combining the received fields, which are stored in the field-cache. The higher the freshness of a cache-entry, the more influence the field associated with the cache-entry will have on the construction of the calculated-field. Thus, although still used, less importance is given to out-dated information. The field-calculator makes use of the router to calculate the values of the calculated-field on different positions. The gradient of the calculated-field is used as driving direction on the target station. • Look-ahead buffer: As explained in section 2.2, an AGV has to make routing-decisions over a certain distance in advance to allow smooth and efficient movement. This distance, called the look-ahead, is not a straight-line distance, but is measured according to the network of stations and segments. The look-ahead buffer of an AGV-agent is an ordered array that holds these routing-decisions. It represents a path that the AGV will follow with certainty, unless something catastrophical (such as a crash) happens. • Behaviour-module: This module is the thread of execution of an agent. It keeps performing a procedure with a constant frequency. The behaviour-module of an AGVagent also maintains the current most important transport and the distance to it. These values are used to achieve the repulsive influence of AGV-agents, as described in section 4.4. The behaviour-module of a transport-agent maintains, among other things, the current priority of the transport. 21
The next section discusses the behaviour of the agents and the different developed modules in the architecture in more detail.
4.3
Behaviour of agents
As mentioned in the previous section, the behaviour-module of the AGV- and transport-agent repeats a procedure with a constant frequency. The general outline of this procedure for the AGV-agent is the following: repeat with constant frequency { 1. Do communication 1.1 Send out field 1.2 Accept received fields and enter them in the field-cache 2. Fill the look-ahead buffer (if needed) 3. Perform an action } For the transport-agent, the behaviour-procedure is as follows: repeat with constant frequency { 1. Do communication 1.1 Send out field 1.2 Accept received messages 2. Calculate priority and field-range } The steps of the above procedures will be discussed in the following order (for convenience): first the communication for both AGV- and transport-agent, then the rest of the behaviour of the transport-agent, followed by the rest of the behaviour of the AGV-agent. The described behaviour is related to transport assignment, that is, AGV-agents are not holding a package associated with a transport. At the end of this section, the behaviour of AGV-agents after they’ve picked up package associated with a transport is discussed.
4.3.1
AGV- and transport-agent: Communication
The communication of AGV- and transport agents is explained by considering the scenario in figure 4.3. Three parts can be identified in the communication of agents: • Agents emit fields: Both AGV- and transport-agents emit fields. Because a typical AGV-environment doesn’t support a distributed storage infrastructure (see section 3.1), these fields are not being physically distributed in the environment with values changing through the environment (eg. like in figure 3.3-a). Instead, the emitting of a field is realized by transmitting a message with position-independent data in each iteration of the behaviour-procedure. Such a message is called a field-message and is sent to all agents that lie in a circle around the sender of the field. The radius of this circle is called the field-range, which is constant and the same for all AGV-agents and growing 22
for transport-agents. Figure 4.3-a shows a scenario with different agents with different field-ranges. Figure 4.3-b shows how the fields are realized by transmitting field-messages (ignore the dotted arrows for now). The data in an AGV field-message consists of the ID of the AGV, the current most important transport of the AGV and the distance to it. Transport field-messages consist of the ID of the transport, the ID of the pick location of the transport and the priority of the transport. An important remark is at place. The range of fields is determined by straight-line distances. However, when an AGV combines the received fields into a calculated-field, distances measured according to the network of stations and segments are used, as explained in section 4.4. These distances are calculated by using the router. In section 4.5, a consequence of using straight-line distances for the field-range is discussed. • AGV-agents receive and store fields: Only AGV-agents make use of the information contained in the fields by storing it in their field-cache. Transport-agents simply ignore received field-messages. Figure 4.4-a, which shows the communication of AGV A from figure 4.3, illustrates this. The AGV-agent sends out its field-message, while (asynchronously) receiving field-messages from agents in whose field-range the AGV-agent lies (only one received field-message is shown for clarity). The received field-messages are stored in the field-cache, overwriting older cache-entries originating from the same field. The age of a cache-entry indicates when the last message of a field was received. The higher the age, the lower the freshness of a cache-entry. The freshness is a measure for how up-to-date
(a)
(b)
Figure 4.3: Example scenario of agent communication. (a)Agents emit fields. (b)This is realized by transmitting field-messages.
23
(a)
(b)
Figure 4.4: (a)Communication performed by AGV-agent A from figure 4.3. Only the relevant modules are shown. (b)Communication performed by transport-agent 1 from figure 4.3
a cache entry is. A typical freshness-function is shown in 4.5. When the freshness of a cache-entry drops below a threshold, it is deleted. The information of received fields is always accessed through the field-cache. The higher the freshness, the more importance is given to the information. • AGV-agents send acknowledge-messages to their most important transportagent: Upon receiving an transport field-message, an AGV-agent checks whether its current most important transport matches the transport of the field-message. If this is the case, the AGV-agent sends an acknowledge-message to the transport-agent, indicating that it is interested in the transport, as shown in figure 4.3-b. A transport-agent maintains the number of interested AGV-agents in its field-range by counting the received acknowledge-messages. This is illustrated in figure 4.4-b, where the communication of transport-agent 1 from figure 4.3 is shown. Since AGVs A and B both lie in range and
24
Figure 4.5: Typical freshness values in function of the age of cache-entry.
since they both have transport 1 as most important transport, two acknowledge-messages are received. A transport-agent uses the number of interested AGV-agents to calculate its field-range, as explained next.
4.3.2
Transport-agent: Calculation of priority and field-range
Besides doing communication, the behaviour of a transport-agent consists of two parts: calculating the priority and calculating the field-range. Calculating the priority Each transport is created with a certain priority. To avoid that a transport has to wait a long time before it gets picked up (starvation), the priority grows over time, as shown in figure 4.6. A transport remains at its initial priority for a time T that is based on the average time an AGV needs to cover the full length of the environment. Then, at intervals of T/2, the priority is increased until the maximum priority (10) is reached.
Figure 4.6: Typical priority values in function of time for a transport with initial priority 7.
25
Calculating the field-range While the field-range of AGV-agents is constant and equal for every AGV-agent, the field-range of transport-agents is variable: it grows over time. The calculation of the field-range, which depends on the number of interested AGVs (AGVs whose most important transport is this transport, see previous subsection) in the current field-range, is explained by considering the scenario in figure 4.7. • (a): Initially, the field-range of a transport-agent is small (typically smaller than the field-range of AGV-agents). While there are no interested AGVs in the field-range, the field-range grows quickly: the transport-agent attempts to quickly find a candidate AGV. • (b): When an interested AGV falls in the field-range, the field-range will grow much slower. • (c): As more interested AGVs fall in the field-range, the speed at which the field-range grows drops accordingly (it is inversely proportional to the number of interested AGVs in range). This reflects that the more interested AGVs are in the field-range, the smaller the probability that a better candidate for the transport is going to be found outside the field-range and thus the smaller the need to expand the field-range. Some remarks are required. Firstly, the reason for using the described approach, as opposed to just using a large, non-growing field-range, is related to performance. The bigger the fieldrange, the more field-messages need to be sent, since more agents fall in range. This burdens the communication-infrastructure as well as the sending and receiving agents. Additionally, the more received fields, the more computationally expensive the construction of the calculated-field (see section 4.4). The described approach aims at keeping the field-range as low as possible, therefore keeping performance as high as possible. The second remark stresses the importance of considering only interested AGVs in range. Consider figure 4.8, where AGV-agent A is in range of transport-agents 1, 2 and 3 and has transport 1 as most important transport. This causes the field-range of transport-agent 1 to grow slowly and the field-ranges of transport-agent 2 and 3 to grow fast. If the growth speed would have been based on the number of AGVs (not necessarily interested AGVs) in range,
(a)
(b)
(c)
Figure 4.7: The field-range of an transport-agent grows over time. The more interested AGVs in range, the slower the growth. The length of the arrows represents the growth speed.
26
Figure 4.8: AGV-agent A is in range of transport-agents 1, 2 and 3 and has transport 1 as most important transport. This causes the field-range of transport-agent 1 to grow slowly and the field-ranges of transport-agent 2 and 3 to grow fast.
transport-agent 2 and 3 would have grown needlessly slow. This slow growth would have kept transport-agents 2 and 3 waiting for a long time, while there are good candidates handle the transports (AGV-agents B and C).
4.3.3
AGV-agent: Filling the look-ahead buffer
The look-ahead buffer of an AGV-agent holds an ordered list of routing-decisions for stations that lie in the look-ahead, constituting a path that the AGV will follow and allowing smooth and efficient movement. The management of the look-ahead buffer is explained by considering the example in figure 4.9: • (a): An entry in the look-ahead buffer contains a routing-decision, in the form of a station to drive to, and the distance remaining towards that station. At any time, the first entry contains the end-station of the segment the AGV is driving on. The distance kept in the first entry is the distance between the AGV and the associated station. The distance kept in the other entries is the distance of the incoming segment. • (b): When a new station (station C) falls in the reach of the look-ahead, a new routingdecision is made for that station. A new entry containing that routing-decision is added to the back of the look-ahead buffer. • (c): When the AGV arrives on a station (station A), the first entry of the look-ahead buffer can be removed. The station to drive to is found in the new first entry. In each iteration of the behaviour-procedure of an AGV-agent, it is checked whether a new station falls in the look-ahead and if this is the case, a new entry is added to the look-ahead buffer. The steps that are taken to achieve this are illustrated in figure 4.10 and are discussed below: • 1: The behaviour-module checks whether a new station has fallen in the look-ahead. The station to check is always the last station stored in the look-ahead buffer (or when the 27
(a)
(b)
(c)
Figure 4.9: The management of the look-ahead buffer. The red line represents the reach of the look-ahead.
look-ahead buffer is empty, the station the AGV is standing on). To check whether it falls in the look-ahead, all the distances kept in the look-ahead buffer are added. If this sum is greater than the look-ahead, the new station doesn’t fall in the look-ahead. For example, consider the case in figure 4.9-a. The sum of distances is 5.2 (1.0 + 2.2 + 2.0), which is greater than the look-ahead (5.0): station C doesn’t fall in the look-ahead yet. • 2: When a new station falls in the look-ahead, a routing decision has to be made for it. This is done by letting the field-calculator construct a calculated-field on the station (called the target station). The calculated-field is a combination of the received fields, stored in the field-cache. The details of the construction of the calculated-field is explained in section 4.4. The calculated-field has an entry for all directions leaving the target station. Each entry in the calculated-field contains a value and the transport that most contributed to that value.
28
Figure 4.10: The steps taken to fill the look-ahead buffer if necessary. Only the relevant modules are shown.
• 3: The behaviour-module chooses the direction associated with the smallest value of the calculated-field as the chosen direction. This can be viewed as choosing the direction of the gradient of the calculated-field. • 4: The most important transport of the AGV is set to the transport associated with the chosen direction. The distance to the most important transport is calculated by the router and stored. • 5: Finally, the chosen direction, along with the distance of the incoming segment (calculated by the router), is added at the back of the look-ahead buffer. Some remarks are required. Firstly, notice that by applying the above procedure, the most important transport of an AGV is defined as the transport that most contributed to the most recently made routing-decision. Also note that the calculated-field is only constructed locally and temporally; it is deleted once the chosen direction is extracted from it. Secondly, when there are no transport cache-entries in the field-cache, the calculated-field will be empty. In this case, the most important transport is set to ’none’ and no entry is added to the look-ahead buffer. The result of this is that an AGV that receives no transport-fields will be idle (not moving) and will have no most important transport. Thirdly, if the target station is also the pick location of a transport, the calculated-field will also contain an entry associated with the ’hold-position’-direction, representing that the AGV has to remain positioned on the target-station. If the ’hold-position’-direction is selected as chosen direction, no entry is added to the look-ahead buffer. This allows the look-ahead buffer to become empty when an AGV arrives at the pick location of a transport. Finally, since more than one new station can fall in the look-ahead in one iteration of the behaviour-procedure, the above procedure is repeated until no new station falls in the lookahead.
29
4.3.4
AGV-agent: Performing an action
The last step in the procedure of the behaviour-module of the AGV-agent consists of performing an action. Depending on the current situation, the AGV-agent selects one of the following actions (see figure 4.11):
Figure 4.11: An AGV-agent performs an action depending on the current situation.
• When the AGV is not on a station (detected by the perception-module, see figure 4.2), it is on a segment. The AGV drives a certain distance towards the end-station of the current segment (an AGV can only drive a certain distance in each iteration of the behaviourmodule). • When the AGV is on a station and its look-ahead buffer is not empty, the following steps are taken. An example of this case is shown in figure 4.9-c. Since the AGV arrived on a station, the first entry of the look-ahead buffer is obsolete and is removed. After this, the next station to drive to is found in the new first entry of the look-ahead buffer and the AGV drives a certain distance towards this station. Since the AGV is now on a segment, the AGV will continue to drive towards the end-station of the segment in the following iterations of the behaviour-procedure. • When the AGV is on a station and the look-ahead buffer is empty, the AGV comes to a halt. The perception-module is used to check whether the packet of a transport is located on the current station. If this is the case, the AGV picks it up: the transport is assigned to this AGV. If there is no packet of a transport on the current station, the AGV simply remains positioned on the station (it is idle). Some remarks are required. Firstly, in the cases where the AGV moves (the first two cases), the distance in the first look-ahead buffer entry, which is the distance between the AGV and the end-station of the current segment, has to be updated. Figures 4.9-a and -b show an example of this, where the distance is updated from 1.0 to 0.8. The same goes for the distance to the most important transport. When updating this distance, it is taken into account that the AGV has to follow the path stored in the look-ahead buffer. 30
Figure 4.12: The steps taken to fill the look-ahead buffer if necessary, in the case that the AGV is executing a transport. Only the relevant modules are shown.
Secondly, note that an AGV can only pick-up the packet of a transport when the look-ahead buffer is empty. When the look-ahead buffer is not empty, the AGV may not be able to stop to pick up the packet. For example, if the AGV is driving over a station when suddenly a transport appears on it, the AGV may be going too fast to suddenly stop.
4.3.5
AGV-agent: Behaviour after picking up the packet of a transport
When an AGV picks up the packet associated with a transport, the transport is assigned to the AGV. This means the AGV has to drive to the associated drop location and drop the packet. To achieve this behaviour, the behaviour-procedure (outlined in the beginning of this section) is adapted in the following way: • 1. Do communication: This step is dropped. An AGV carrying a packet of a transport has no need to receive fields, because it doesn’t need to (and can’t) handle another transport. Also, other AGVs have no need to receive a field from an AGV already executing a transport, because it is not competing for transport assignment. • 2. Fill the look-ahead buffer (if needed): This step is altered from the procedure in figure 4.10 to the one in 4.12. The following steps are taken: – 1: First, it is checked whether a new station falls in the look-ahead, in the same way as before. If this is the case, the procedure continues with step 2. – 2: The router is used to calculate the path from the new station in the look-ahead towards the drop location associated with the transport. – 3: Finally, the first station of this route, together with the length of the incoming segment, is added at the back of the look-ahead buffer. When the calculated route is empty, no entry is added, allowing the look-ahead buffer to become empty when the AGV arrives on the drop location.
31
• 3. Perform an action: This step remains exactly the same as before (see figure 4.11) with one exception: dropping a packet of a transport is considered instead of picking it up. Once an AGV drops the packet of the transport on the associated drop location, the standard behaviour is applied again. The next section describes the constructing of the calculated-field in more detail.
4.4
Construction of the calculated-field
This section explains how the field-calculator constructs the calculated-field on a target station. First, the case of a single AGV-agent is handled. Next, the influence of other AGV-agents is discussed. Finally, the special case where a packet of a transport is present on the target station is handled.
4.4.1
Single AGV-agent
The construction of the calculated-field is explained by considering the example in figure 4.13. The field-calculator of AGV-agent A constructs a calculated-field on the target station X. For this, the router and field-cache are used. The field-cache of AGV-agent A in this example is shown in 4.14-a. As the field-cache shows, AGV-agent A lies in the field-range of transportagents 1, 2 and 3. The freshness of cache-entries in the examples in this section is always taken as 1. The received fields are combined into a calculated-field through a number of steps: • step 1: distance: For each transport entry in the field-cache, the distance from the target station (station X) towards the pick locations of the transport, taking a certain direction on the target station, is calculated by the router. These distances are measured according to the network of segments and stations and used to create a partial-field for each received transport-field. A partial-field is associated with a transport and contains a value for each direction from the target station. The lower a value, the more favorable a certain direction/transport combination. In figure 4.13, the partial-fields have the same color as their associated transports. Taking Fi,j as the value of the partial-field of transport i in the direction j, we get: Fi,j = distance from target station to transport i taking direction j This is illustrated in figure 4.13-a. • step 2: priority and freshness: Next, the priority and freshness of the transport cacheentries are taken into account. The higher the priority, the more importance should be given to the transport. The lower the freshness, the less up-to-date the cache-entry and the less importance should be given to it. This is achieved by adjusting the values of the partial-fields as follows: Fi,j = Fi,j + Fi,j ∗ [
paramP rior + (1 − transporti .f resh) ∗ paramF resh] transporti .prior
32
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.13: AGV-agent A constructs a calculated-field on station X. See text for details.
33
(a)
(b)
Figure 4.14: (a)The field-cache of AGV-agent A in figures 4.13-a to -d. (b)The field-cache of AGV-agent A in figures 4.13-e to -f.
The values transporti .prior and transporti .f resh are the priority and freshness of the cache-entry associated with transport i. paramPrior and paramF resh are parameters that reflect the amount of influence the priority and freshness of transport cacheentries have on the calculated-field. The above formula is applied in figure 4.13-b, where paramP rior and paramF resh are 1. • step 3: turn around: Turning around is a very time consuming action for AGVs. That’s why an extra penalty is given to the direction from where the AGV approaches the target station: if (j = direction from where the AGV approaches the target station): Fi,j = Fi,j + paramT urnAround The above formula is applied in figure 4.13-c, where a paramT urnAround of 30 is used. • step 4: clusters: Finally, the different partial-fields are combined into the calculatedfield, which has a value for each outgoing direction from the target station. The most simple and straightforward way is to take the minimal value in each direction. However, a method is used that gives an extra advantage to clusters of transports (a group of transports that lie close together). The reason for this is explained by considering figure 4.15. Transports 2, 3 and 4 form a cluster. Suppose that AGV A drives towards transport 3 (figure 4.15-a), but AGV C arrives at the transport before AGV A. Because AGV A also drove in the (general) direction of transport 2 and 4, there is a good probability that AGV A can execute one of these transports (figure 4.15-b). So, the travelled distance of AGV A was not in vain. Now suppose AGV A drives towards transport 1 (figure 4.15-c) and AGV B arrives at the transport before AGV A. In this case, the travelled distance was in vain and AGV A will have to turn around to drive to one of the other three transports, that are now even further away (figure 4.15-d). This example illustrates why more importance is given to clusters of transports. The following formula achieves this, with Fj de value of the calculated-field in direction j and Fmin,j the minimal value of the partial-fields in direction j: Fj = Fmin,j + Fmin,j ∗ (
34
paramCluster ) P 1.0 + i6=min Fmin,j Fi,j
(a)
(b)
(c)
(d)
Figure 4.15: Clusters of transports are attractive for AGV-agents.
The parameter paramCluster controls the influence of clusters of transports. Besides a value for each direction (Fj ), the calculated-field also contains for each direction the transport that most influenced the associated value. For direction j, this is the transport associated with Fmin,j . The calculated-field is shown in figure 4.15-d, where the color of a value represents the transport that most influenced it. A paramCluster of 0.5 is used. As explained in section 4.3, the direction of the calculated-field with the smallest value, which can be viewed as the gradient, is used as the routing-decision on the target-station. The direction is stored in the look-ahead buffer and the transport associated with the direction becomes the new most important transport of the AGV-agent. In the example, the most important transport is set to transport 1 and AGV-agent A will drive upwards on station X.
4.4.2
Multiple AGV-agents
When multiple AGVs drive towards the same transport (that is, the pick location of the transport), the AGV that first arrives at the transport will be assigned to it. The other AGVs wasted resources, such as time and energy, by driving towards the transport. To prevent this, AGV-agents send out a repulsive field. However, a problem arises when this repulsion is direct, as illustrated by figure 4.16. AGV A is repelled by AGV B, which causes AGV A to drive away from transport 1. However, AGV A is also driven away from transport 2. This is undesirable, because AGV A is a suitable candidate to execute transport 2. This problem is solved by letting an AGV repel other AGVs in an indirect way, through its most important transport. This is done as follows. Consider again figure 4.16 and suppose the most important transport of AGV B is transport 1. Because transport 1 is the most important order of AGV B and because AGV B lies closer to transport 1 than AGV A lies to transport 1, transport 1 will have a less attractive influence on AGV A. This results in AGV A being most attracted by transport 2. AGV A won’t turn around and will drive towards transport 2, as opposed to the case where direct repulsion is used. More specifically, the effect of other AGV-agents on the calculated-field is calculated in an extra 35
Figure 4.16: AGV A is repelled by AGV B, which causes AGV A to drive away from transport 1 and 2.
step after step 2 from the last subsection. • step 2 extra: other AGV-agents: Let AGVA = AGV constructing its calculated-field Let AGVB = AGV in range of AGVA Let transporti = most important transport of AGVB Let AGVA .dist = distance from AGVA to transporti Let AGVB .dist = distance from AGVB to transporti Let AGVB .f resh = freshness of cache-entry associated with AGVB in field-cache of AGVA If (AGVA has a received field from transport i) and (AGVA .dist > AGVB .dist): Fi,j = Fi,j + Fi,j ∗ [
AGVA .dist ∗ (AGVB .f resh) ∗ paramAGV ] AGVB .dist
AGVA calculates AGVA .dist by using its router and finds AGVB .dist in the cache-entry associated with AGVB . Both distances are measured according to the map of stations and segments. paramAGV is a parameter that reflects the amount of influence other AGVs have on the construction of the calculated-field. Notice that a penalty is given to the partial-field of transporti proportional to the ratio of AGVA .dist and AGVB .dist: the bigger this ratio, the smaller the probability that AGVA will reach transporti first and the bigger the penalty. The result of step 2 extra is shown in figure 4.13-e and the field-cache of AGV-agent A is shown in 4.14-b, which shows that AGV-agent A lies in the field-range of transport 1, 2 and 3 and AGV-agent B. paramAGV is taken as 1. The calculated-field, obtained after applying step 3 and 4, is shown in figure 4.13-f. The routing-decision taken on station X is the downwards direction and the most important transport is set to transport 2. Compare this with the result in figure 4.13-d: AGV-agent B made AGV-agent A drive towards transport 2 instead of transport 1.
4.4.3
Special case: transport on target station
When an the pick location of a transport is located on the target station itself, it makes no sense to consider that transport at the different outgoing segments from the target station. The transport on the target station is handled separately during the construction of the calculatedfield, as shown in figure 4.17-a. The distance from the target station to transport 4 is set to a minimal value of 1. The value of 0 is not chosen because the value would remain 0 through the construction of the calculated-field. 36
(a)
(b)
Figure 4.17: AGV-agent A constructs a calculated-field on station X, where transport 4 is positioned. See text for details.
The final calculated-field is shown in figure 4.17-b. Notice that steps 3 and 4 (see previous subsections) are not applicable to the transport on the target station. The most important transport is set to the transport 4 and the routing-decision for station X is the ’hold-position’direction. As mentioned in subsection 4.3.3, this routing-decision is not entered in the lookahead buffer, allowing the look-ahead buffer to become empty by the time AGV-agent A arrives on station X. This will allow AGV-agent A to pick up the packet of transport 4 on station X.
4.5
Remark about straight-line and logical distances
Because a typical AGV-environment doesn’t support a distributed storage infrastructure (see section 3.1), the fields sent out by agents are not being physically distributed in the environment with values changing through the environment (eg. like in figure 3.3-a). Instead, the emitting of a field is realized by repeatedly broadcasting messages with data that is the same on every position in the range of the field. A consequence of this is illustrated in figure 4.18. AGV A falls in the range of the transportfield and AGV B does not, since AGV A is closer to the transport in straight-line distance than AGV B. This results in the transport attracting AGV A, but not AGV B. However, the logical distance (the distance measured according to the network of stations and segments) between AGV A and the transport is higher than the logical distance between AGV B and the transport. Thus, although AGV B is a better candidate for the transport than AGV A, only AGV A is attracted towards the transport. Since the range of the transport-field grows, AGV B will eventually also fall in range of the transport-field. This limits the impact of this consequence on the performance of the system. Again, it’s important to note that during the construction of the calculated-field, logical distances, as opposed to straight-line distances, are used, as explained in the previous section. The next chapter explains how the approach discussed in this chapter is implemented in an AGV-simulator.
37
Figure 4.18: The range of a field is determined by straight-line distance. The lines show the shortest paths from the AGVs to the transport.
38
Chapter 5 Implementation This chapter focusses on the implementation of the approach discussed in chapter 4. For the implementation, the AGV simulator developed by the AgentWise workgroup [5] is used, which was introduced in subsection 2.2.2. Section 5.1 discusses the design of the existing AGV simulator. Next, the implementation of our approach in the AGV simulator is discussed in section 5.2. Finally, section 5.3 discusses the adaptations made to the Graphical User Interface (GUI) of the AGV simulator in order to visualize the concepts of our field-based approach. Note that in the AGV simulator, the older term ’order’ is used instead of ’transport’.
5.1
Design of the existing AGV simulator
This section gives a high-level overview of the internal structure of the AGV simulator. Figure 5.1 shows a partial class diagram of the AGV simulator. Note that this is in no way a complete class diagram; only the parts relevant to our implementation are shown. The grey classes represent the access-points for our implementation. For a more detailed discussion, see [5], where the code with java-doc can be downloaded. An AGV uses the PerceptionModule to actively perceive the environment (eg. the segment the is currently located on, ...), the CommunicationModule to transmit and receive messages and the ActionExecutionModule to perform actions in the environment (eg. picking up a packet, ...). All the interaction between the AGV and the environment happens through an interface to the environment, the EnvironmentAccesPoint. For the environment and the agent-environment interaction, the collector-reactor-effector model is used [1, 7, 9, 10], as described in subsection 2.1.2. The AGV has access to a RoutingModule, which is used to calculate paths and distances in the layout. For this, an internal representation of the layout is used (InformationMap). The AGV is a subclass of ActiveEntity, which has a step method. The step method is repeatedly called and represents one step in the simulation of the entity. The step method of an AGV calls the step method in its BehaviourModule. By creating a subclass of BehaviourModule and overriding the step method, the user can implement the desired behaviour of agents. The OrderLocationManager, another subclass of ActiveEntity, represents the external system that is responsible for managing the orders. By creating a subclass of OrderLocationManager 39
Figure 5.1: Partial class diagram of the AGV simulator. The grey classes represent the accesspoints for our implementation.
and overriding the step method, the user can implement the desired behaviour related to the creation and removal of orders. A time management system is applied to ensure that a simulation runs at a consistent logical time [22], by allowing the MAS developer yo express timing requirements that reflect the semantics of the MAS. For example, if it is defined that picking up a packet should take ten times as long as driving 1 meter, an AGV can drive 10 meters while another AGV can pickup a packet.
5.2
Positioning of our implementation within the AGV simulator
This section explains how our implementation of the field-based approach fits in the AGV simulator. Only a brief discussion of our implementation is given here, because the concepts and details of the approach were already discussed in chapter 4. For more detailed information on our implementation, see appendix B for class diagrams and appendix C for interaction diagrams. Our implementation fits in the simulator at three access-points (shown in gray in figure 5.1): • As shown in figure 5.2, we made a subclass of BehaviourModule, FieldBasedBehaviourModule. The step method of this class implements the behaviour-procedure of AGV-agents discussed in chapter 4. To achieve this, the class has an associated LookAheadBufferManager, which contains the look-ahead buffer, in the form of an array of LookAhead40
Figure 5.2: The classes associated with the behaviour of AGVs.
Figure 5.3: The classes associated with the behaviour of transport-agents (order-agents).
BufferEntries, and provides methods for accessing and manipulating it. The FieldBasedOrderAssignmentModule is used for all the field-related operations. It maintains the most important order and distance to it and has references to a CacheManager ant a FieldCalulator. The CacheManager maintains the field-cache, in the form of an array of CacheEntries and provides methods for accessing and manipulating it. The FieldCalulator makes use of the RoutingModule and CacheManger to construct calculated-fields. • As shown in figure 5.3, we made a subclass of OrderLocationManager, FieldBasedOrderLocationManager. The class maintains a list of OrderAgents, associated with orders that are not yet picked up. Since the AGV simulator doesn’t contain a class for general agents
41
(it only contains an AGV class), we created the OrderAgent class from scratch. It has reference to a OrderAgentBehaviourModule, which contains the behaviour locic of the OrderAgent, and to a OrderAgentCommunicationModule, which is used to manage the communication of the OrderAgent. The step method of the FieldBasedOrderLocationManager executes the following steps: – 1: For each pick location that has no order yet, a new order is placed on it with a given probability. This probability is a parameter called the creationPercentage. – 2: For each drop location that holds an order, the order is removed with a given probability. This probability is a parameter called the deletionPercentage and is usually taken close to 1. – 3: The step method of each OrderAgent is called. This method implements the behaviour-procedure of transport-agents discussed in chapter 4. • We made a subclass of AbstractCommunicationInfrastructure, VariableRangeInfrastructure. As mentioned in chapter 4, the emission of fields is realized by repeated broadcasts over varying distances. In VariableRangeInfrastructure, messages contain the range they are to be transmitted over, thus allowing broadcasts over varying distances.
5.3
Adaptations to the Graphical User Interface
Figure 5.4: The GUI is extended to visualize the concepts of our approach. The bottom- and right-panel are not shown for clarity. In order to visualize the concepts of our field-based approach, we made the following extensions to the GUI of the simulator: 42
Figure 5.5: The stations in the look-ahead buffer are colored green. When a calculated-field is constructed, its values are shown temporarily.
• Fields are visualized by circles centered around the sender. As shown in figure 5.4, AGVfields are red and order-fields blue. • The user can use the mouse to click on any AGV. The following information about the clicked AGV is displayed on the left of the screen (see figure 5.4): – The ID of the clicked AGV. – The ID of, the location of and the distance to the most important order. – The field-cache. • As illustrated in figure 5.4 and 5.5, the stations in the look-ahead buffer of the clicked AGV are colored green. In case the AGV is carrying no order, the pick location of its most important order is also colored green. In the case the AGV is carrying an order, the associated drop location is colored green. • When a calculated-field is constructed by the clicked AGV, its values are shown temporarily. This is shown in figure 5.5, where the AGV constructs a calculated-field on station ’s10’. Since the direction towards station ’s11’ is associated with the lowest value, station ’s11’ is added to the look-ahead buffer.
In the next chapter the performance of the implemented field-based approach is investigated.
43
Chapter 6 Results and discussion This chapter discusses the performance of the implemented approach under varying circumstances and parameters. By analyzing the results, the advantages as well as the problems of the approach are pointed out. The layout used for obtaining the results is described in section 6.1. To make the performance measurements more meaningful, the implemented approach is compared to a simple reference algorithm, as discussed in section 6.2. Finally, the effects of various parameters of the implemented approach are investigated in section 6.3.
6.1
The layout used for testing
In order to measure the performance of our approach in a realistic environment, a layout based on a real-life layout (shown in appendix A), was constructed in the XML-format of the simulator [5], see figure 6.1. The layout counts 56 pick and 50 drop locations. The average length of a segment and a connected end-station is 8 meters. Some abstractions and adaptations were made in relation to the real-life layout: • In the real-life layout, transport locations are stacked on long segments. Also, each transport location functions as both a pick and drop location. In the simulator layout the groups of transport locations were replaced by one drop location connected to one pick location, as the zoom-in shows in figure 6.1. • The lower six pick locations (pick location 1 to 6) have no connected drop location, as indicated in figure 6.1. This was done to examine starvation, as no AGVs come near those pick locations when handling other transports. • Various crossing segments were left out to avoid collision. Also some bidirectional segments were dropped, because they are prone to deadlocks. Note that collision and deadlock avoidance are not in the scope of this thesis.
6.2
Comparison with a simple reference algorithm
To obtain a reference point for the performance measurements of the field-based approach, a simple reference algorithm for transport assignment was developed and implemented (in the 44
Figure 6.1: The layout used for testing.
simulator [5]). First, this algorithm is explained. Next, the parameters of the field-based approach and the reference algorithm, together with their default values, are discussed. Next, the comparison between the field-based and the reference algorithm is made using the default parameters. After this, the effects of changing the number of AGVs and transports are examined. Finally, a conclusion is drawn form these measurements.
6.2.1
Overview of the reference algorithm
The implemented reference algorithm is based on the Contract Net (CNET) protocol [1], as discussed in section 3.2. Transport assignment is achieved in the following steps, as shown in figure 6.2: • (a): The transport broadcasts a request-message to all AGVs, indicating that it seeks an AGV for assignment. The request-message contains the ID and the position of the transport. • (b): Each AGV replies upon receiving the request-message. Two cases can be distinguished: – When the AGV is currently not assigned to another transport, it replies with its ID and the distance from the AGV to the transport. This distance is calculated according to the map of stations and segments. – When the AGV is already assigned to another transport (or can’t handle the transport for another reason), it replies with a decline-message. 45
(a)
(b)
(c)
(d)
Figure 6.2: The reference algorithm for transport assignment.
• (c): The transport compares the received distances and sends an assignment-message to the the AGV associated with the smallest distance. • (d): Upon receiving an assignment-message, the AGV is effectively assigned to the transport. The AGV will drive towards the transport to pick it up. Next, the AGV will drive towards the drop location associated with the transport to drop the transport. When a transport receives a decline-message (step (b)) from every AGV, it is put in a waiting status. All waiting transports are sorted by priority. The transport with the highest priority tries to achieve assignment first (by executing the above stepts). When this transports gets assigned to an AGV, the transport with the second highest priority is handled, etc. The priority of transports grows over time, in the same way as in the field-based approach (see subsection 4.3.2). As mentioned in section 3.2, the CNET protocol achieves an immediate and permanent assignment, which results in a lower flexibility. The reference algorithm should not be considered a full-fledged algorithm for transport assignment; it is only used for comparison.
6.2.2
The parameters and their default values
Below the most important parameters of the field-based approach are discussed. The first three parameters also apply to the reference algorithm. The default values are given between the parentheses. In the rest of this chapter, the default parameters are used unless mentioned otherwise. 46
Figure 6.3: Different performance measurements for the field-based approach and the reference algorithm using the default parameters.
• number of AGVs (14): The number of AGVs to use. • creationPercentage (0.00007): The probability that a new transport is created on an empty pick location in each time step (one execution of the behaviour-procedure (see section 4.3) takes one time step on average). • deletionPercentage (0.9): The probability that a transport is deleted on a drop location in each time step. • paramPrior (5.0): Reflects the amount of influence the priority of transports have. See section 4.4. • paramFresh (1.0): Reflects the amount of influence the freshness of cache-entries have. See section 4.4. • paramAGV (1.0): Reflects the degree of repulsion AGVs have on other AGVs. See section 4.4. • paramTurnAround (5.0): The penalty for an AGV to turn around. See section 4.4. • paramCluster (1.0): Reflects the amount of influence clusters of transports have. See section 4.4. • AGV field-range (25 meters): The range of the field sent out by AGVs. • transport field-range (5 meters): The initial range of the field sent out by transports. • look-ahead (10 meters): The distance in which routing-decisions have to be made in advance. Furthermore, the priorities and the drop locations associated with transports are chosen randomly.
6.2.3
Comparison using the default parameters
Figure 6.3 shows a comparison of the field-based approach with the reference algorithm, using the default parameters. All results are obtained by averaging over multiple runs. The following performance measurements were used:
47
Figure 6.4: Average wait time for transports grouped by pick location.
Figure 6.5: Average number of transports waiting over time.
• average transport wait time: The average number of time steps a transport has to wait before its associated package is picked up. This value is 138 percent higher for the reference algorithm, indicating a worse overall performance. However, when the transports are grouped by pick location, the reference algorithm achieves a more equal distribution, as shown in figure 6.4. The field-based approach achieves higher wait times for pick locations 1 to 5. Those are positioned far away from the main traffic distribution, because there are no drop locations nearby (see figure 6.1). Thus, the field-based approach suffers from a light form of starvation: remote transports may have to wait a longer time for pickup. The growing of the priority of transports aims to remedy or limit starvation and is discussed in the next section. • average number of transports waiting: The average number of transports that are not yet picked up is 79 percent higher for the reference algorithm. As figure 6.5 shows, the AGVs can better satisfy the transport needs when the field-based approach is used. • average distance ratio: The distance ratio is defined as follows. Call the start-station the station an AGV is positioned on when it first enters the system or when it has just completed a transport. The AGV then drives around until it picks up a transport. The distance ratio is the ratio of the driven distance to the optimal distance between the start-station and the transport. 48
The distance ratio is always at the minimal value of 1 in the reference algorithm. In the field-based algorithm however, the average distance ratio is 1.15, indicating that an AGV covers 15 percent more distance than the optimal distance on average. Given the flexibility of the field-based approach, this value can be considered low. • average number of new transports considered: The average number of transports an AGV considers its most important transport between two successive transport assignments. In the reference algorithm, the minimal value of 1 is achieved. In the field-based approach, an average of 1.9 is achieved. • percentage of time steps idle: The percentage of time steps where AGVs don’t perform an action. In the field-based approach, an AGV is almost always in the range of at least one transport-field and thus will only be idle sporadically (0.3 percent). • average distance travelled per transport: This is the average distance all the AGVs have covered to handle one transport. This value is obtained by dividing the total distance covered by all AGVs by the number of transports handled. The value is considerably higher for the reference algorithm (34 percent). • average number of transports done in 100 time steps: This can be considered the throughput of the system. The reference algorithm scores 27 percent worse than the field-based approach. • average number of messages transmitted in 100 time steps: In the field-based approach, fields are realized by repeatedly broadcasting messages. This results in a high number of message transmissions (an average of 1036 each 100 steps).
6.2.4
Comparison under varying number of AGVs and transports
The effects of using different numbers of AGVs and transports on the performance of both algorithms were examined. In particular, the ratio of the number of AGVs over the average number of transports, the AGV-transport ratio, is of importance. Changing the number of average transports is achieved by changing the parameter creationPercentage.
Figure 6.6: Average wait time for different ratios of the number of AGVs over the creationPercentage.
49
Figure 6.7: Average distance travelled for a transport, for different ratios of the number of AGVs over the creationPercentage.
As the AGV-transport ratio increases, the average wait time of transports decreases for both algorithms, since the AGVs have to drive an increasingly shorter distance to a transport. However, this also results in the various concepts of the field-based approach, such as the repulsion of AGVs, being less exploited. Thus, the difference between the two algorithms decreases as the AGV-transport ratio increases, as figure 6.6 shows. Figure 6.7 shows the average distance travelled by all the AGVs per transport. At lower AGV-transport ratios, this value is lower for the field-based approach. However, as the AGVtransport ratio increases, the average total distance travelled per transport grows for the fieldbased approach and becomes higher than for the reference algorithm. Figure 6.8, which shows a scenario with a high AGV-transport ratio, illustrates the reason for this. • (a): An new transport enters the system, attracting all the AGVs. Note that the repulsive influence of AGVs has no effect: it makes the transport less attractive for other AGVs,
(a)
(b)
(c)
(d)
Figure 6.8: When the AGV-transport ratio is high, AGVs drive superfluous distance in the field-based approach.
50
Figure 6.9: Average waiting time for different numbers of AGVs and creatationPercentages, keeping their ratio fixed at 200000.
but since there is only one transport, it is the most important transport for all the AGVs. • (b): AGV A is assigned to the transport. All the other AGVs have covered superfluous distance. • (c): A new transport enters the system and the all the unassigned AGVs are attracted by it. • (d): AGV E is assigned to the transport. The other AGVs have again covered distance in vain. It is interesting to examine the effect of varying the number of AGVs and creationPercentage, while keeping their ratio fixed. As figure 6.9 shows, the more AGVs and transports are used, the better the advantages of the field-based approach are exploited.
6.2.5
Conclusion
The reference algorithm is outperformed by the more flexible field-based approach. The more AGVs and transports are present in the system, the more the potential of the field-based approach can be exploited. However, the field-based approach requires a lot of messages to be transmitted, placing a burden on the communication-infrastructure. Also, when the ratio of AGVs over transport is high, the AGVs tend to drive a lot of superfluous distance, which is a waste of energy.
6.3
The effects of various parameters
In this section the effects of various parameters are examined by changing their values. The results are expressed as percent difference from the result obtained with the default parameter values. 51
The effects of the parameter ’paramPrior’
Figure 6.10: The average normalized wait time for the different priority values. The parameter paramPrior was introduced to control the impact of the priority of transports. Figure 6.10, which depicts the average normalized wait time, shows that this goal is achieved. This value is obtained as follows. First, for each priority (from 1 to 10), the wait times of transports with that priority value are averaged. Then, those averages are normalized by dividing them by the biggest average. When paramPrioity is 0, priorities are not taken into account and the curve becomes near horizontal. As paramPriority increases, the priority of transports becomes more important and the curve becomes increasingly steeper. However, note that the difference in average normalized wait time becomes smaller as the priority increases. The reason for this is that the ratio of successive priorities becomes smaller with growing priority, eg. 2 is 2 times greater than 1, 3 is only 1.5 times greater than 2, etc. It would be interesting as future work to investigate how a more linear influence of priority can be achieved. Next, the effect of paramPrior on the global performance of the system was examined. As figure 6.11 shows, the average distance travelled per transport increases as paramPrior grows. The
Figure 6.11: Average wait time and average distance travelled per transport for changing values of paramPrior (in percent difference from default paramPrior of 5).
52
Figure 6.12: Average wait time for transports grouped by pick location.
reason for this is that as paramPrior grows, the priority of transports becomes more important in relation to their distance and AGVs will drive to more remote (but high priority) transports. The average wait time of transports follows this trend, except at paramPrior 0, as shown in figure 6.11. At paramPrior 0, priorities are not taken into account and thus the growth of priorities (to prevent starvation, see subsection 4.3.2) has no effect. This results in remote transports having to wait a long time (starvation), which explains the peak of average wait time at paramPrior 0. As paramPrior increase, the distribution of the wait time of transports among pick locations becomes more equal, as shown in figure 6.12. In conclusion, a trade-off has to be made between the effect of priority (figure 6.10) and average wait time (figure 6.11). A paramPrior of 3 is a suitable value in this particular situation. The effects of the parameter ’paramCluster’ The higher paramCluster, the stronger AGVs are attracted by clusters of transports, which should increase performance (see section 4.4). As shown in figure 6.13, increasing paramCluster from zero results in an increasingly better average wait time for transports. However, if paramCluster is taken too high, the clusters have a dominant effect on AGVs. This causes AGVs to drive to clusters of transports, no matter the distance. Isolated transports are largely ignored, which causes the average wait time to grow, as shown in figure 6.13. Additionally, experiments with high values of paramCluster (greater than 10) showed that all AGVs were attracted towards the middle of the layout, which resulted in a high rate of deadlocks. The effects of the parameter ’paramAGV’ The most important transport of an AGV is made less attractive for other AGVs in range (see section 4.4). The strength of this repulsive influence is controlled by the parameter paramAGV. 53
Figure 6.13: Average wait time for changing values of paramCluster (in percent difference from default paramCluster of 1).
This repulsive influence improves the performance of the system, as shown in figure 6.14. Again, the parameter should be carefully tuned: when the parameter is too high, the performance of the system drops.
Figure 6.14: Average wait time for changing values of paramAGV (in percent difference from default paramAGV of 1).
The effects of the field-range The AGV field-range controls the size of the fields emitted by AGVs. The greater the AGV field-range, the greater the environmental awareness of AGVs, since more distant AGVs are detected. As figure 6.15 shows, increasing the AGV field-range from 0 results in a lower average wait time. However, if the AGV field-range is too large, performance will decrease. It is not beneficial to be infuenced by AGVs that are a substantial distance away. This is in line with the property of a MAS that an agent can (and should) perceive its environment only locally. 54
Figure 6.15: Average wait time for changing values of the AGV field-range (in percent difference from default AGV field-range of 25 meters).
Figure 6.16: Number of transmitted messages for changing values of the AGV field-range (in percent difference from default AGV field-range of 25 meters).
Another problem with higher AGV-field ranges is that the number of transmitted messages increases dramatically, as more AGVs are in range of an emitted AGV-field. Figure 6.16 illustrates this. Also, as the AGV-field range increases, the construction of the calculated-field becomes more computationally intensive (since more AGV-fields are received, see section 4.4). Similar results were obtained for the initial transport field-range. The effects of the look-ahead The look-ahead is the distance over which AGVs have to make routing-decisions in advance. The higher the look-ahead, the slower AGVs can react to changes in the environment (eg. a new transports entering the system). This results in longer average wait times for transports, 55
Figure 6.17: Average wait time for changing values of the look-ahead (in percent difference from default look-ahead of 10 meters).
as shown in figure 6.17. However, as mentioned in section 2.2, a look-ahead is required in realistic AGV-systems to allow smooth and efficient movement. Conclusion The field-based approach uses a lot of parameters to adjust its behaviour. For example, if it is noticed that high priority transports have to wait nearly as long as lower priority transports, the parameter paramPrior can be increased. It is however far from intuitive to grasp how the different parameters effect the emergent behaviour of the MAS. In this section, the effects of some parameters were examined, using a particular layout and in combination with particular values for the other parameters. While this gives a general indication of the effects of various parameters, the results may differ in other situations. The tuning of the parameters is a serious burden and it would be interesting as future work to automate this. Two approaches can be distinguished. Firstly, the parameters could be calculated in advance from properties of the AGV-system (eg. the size of the layout, the number of pick and drop locations, the average length of a segment,...). Another approach would be to adjust the parameters online, applying a learning technique. Note however that the field-based approach is expected to perform reasonably without tuning the parameters for a certain situation (eg. the layout, the number of AGVs,...). For example, it is expected that the field-based approach will outperform the reference algorithm on most layouts by simply using the default parameters given in this chapter.
56
Chapter 7 Conclusion In order to investigate the feasibility of using field-based techniques within the MAS paradigm to achieve transport assignment in an AGV-system, the following steps were taken: 1. We developed a field-based approach for transport assignment, drawing inspiration from existing field-based techniques and starting from an existing, generic model for situated MASs. 2. We implemented the field-based approach in the AGV simulator developed by the AgentWise workgroup [5]. The AGV-simulator already contains a MAS framework; we implemented the actual behaviour of agents and the necessary field-based concepts. 3. We investigated the performance of our approach by making various performance measurements, such as the average time a transport has to wait before it is picked up and the average distance the AGVs drive to execute one transport. To obtain a reference point for the performance measurements, we implemented a transport assignment algorithm based on the Contract Net (CNET) protocol. The two algorithms were compared under a varying number of AGVs and transports. We also investigated the effect of the various parameters of the field-based approach on the performance.
7.1
Discussion of the approach
Our approach achieves transport assignment by associating agents with both the AGVs (AGVagents) and the pick location of unassigned transports (transport-agents). Transport-agents emit attractive fields into the environment. These fields have a growing range and contain information about the transport. AGVs continuously combine received fields into a single field, the calculated-field. The AGV’s behaviour consists of continuously following the gradient of the dynamic calculated-field. This will guide the AGVs towards pick locations of transports, much like a ball rolls towards a valley in a (continuously changing) mountainous landscape. To avoid multiple AGVs driving towards the same transport, AGVs emit repulsive fields.
7.1.1
Advantages
The AGVs are continuously guided by the dynamic fields in the environment, thereby continuously reconsidering the situation of the environment and delaying transport assignment until 57
a pick location of a transport is reached. This benefits the flexibility of transport assignment: AGVs readily adapt to new situations in the environment, such as a new transport entering the system. This is in contrast to for example the CNET protocol, where an AGV is assigned to a transport as soon as the transport enters the system. On its way towards the pick location of the transport, the assigned AGV cannot react to new, perhaps more favorable transports. Experiments indicate that the more flexible field-based approach outperformed a CNET based protocol on a particular layout in measurements such as the average time a transport has to wait for pick up and the throughput of transports, using a varying number of AGVs and transports. We expect these result also apply to other layouts and circumstances. Experiments also indicate that the more AGVs and transports are present in the system, the more the potential of the field-based approach can be exploited. In the field-based approach, no complex communication protocols are used; the AGVs are simply guided by the fields in the environment. This would result in an increased robustness when the approach is applied in a real-world AGV-system. For example, delayed or lost messages would not have a large impact, as no protocols are disrupted. We also expect that the lack of communication protocols will benefit openness when the approach is applied in a real-world AGV-system: AGVs could leave or enter the system without any additional consequences. In the field-based approach, the priority of transports is integrated in a natural way. Both the distance and the priority influence the attractive strength a transport has on AGVs. Additionally, the user can adapt the impact of priorities through a parameter.
7.1.2
Limitations
Due to the nature of the field-based approach, i.e. AGVs being guided by fields in the environment, it is not feasible to control, predict or reason about the emergent system-wide behaviour or to provide guaranteed reaction times. Although we have applied the field-based approach with success in several layouts and circumstances, less favorable results might be achieved in other cases. The field-based approach uses numerous parameters to control the behaviour of AGVs. It is however far from intuitive to grasp how the different parameters effect the emergent behaviour of the system. We have experienced that tuning the parameters often requires a time consuming trail-and-error approach. However, experience indicates that the field-based approach performs reasonably without tuning the parameters for a certain situation (eg. the layout, the number of AGVs,...). It would be interesting as future work to (partially) automate the parameter tuning. Two approaches come to mind. Firstly, the parameters could be derived in advance from properties of the AGV-system (eg. the size of the layout, the number of pick and drop locations, the average length of a segment,...). Another approach would be to adjust the parameters online, applying a learning technique [?]. Experiments indicate that the effect of the priority of transports is not linear ; that is, the additional attractive influence of priorities gets smaller as the priority increases (eg. a transport 58
of priority 2 is much more attractive than a transport of priority 1, while a transport of priority 10 is only marginally more attractive than a transport of priority 9). It would be interesting to investigate how the effect of priority can be made more linear. The approach can result in AGVs driving superfluous distance, especially when there is an excess of AGVs in relation to the average number of transports. In this case, a transport that enters the system tends to attract multiple AGVs, while only one AGV can actually execute the transport. The other AGVs will have wasted resources (eg. energy, time,...) by driving towards the transport. Experiments have indicated that the average distance driven to execute one transport (measured by dividing the total number of executed transports by the total distance driven by all the AGVs) was lower for the field-based approach than for a CNET based algorithm, when the ratio of the number of AGVs over the average number of transports was low. However, for higher ratios of the number of AGVs over the average number of transports, the trend reversed: the field-based approach achieved a higher average distance driven to execute one transport. However, in any case, the average time a transport has to wait before it is executed was lower for the field-based approach than for the CNET based algorithm. Another problem we encountered was a light form of starvation. Transports with pick locations that are far away from the main traffic distribution might have to wait a long time before they are executed. The cause of this is that most of the time, AGVs will be attracted by other, more favourable transports. Our approach limits starvation by letting the priority of transports increase over time. This causes transport that have been ignored for a long time to become more attractive for AGVs. Experiments have shown that this growing of priorities reduces starvation considerably, although it doesn’t eliminate starvation completely. Due to the lack of infrastructure of AGV-systems to store fields in the environment, fields are realized in our approach by repeatedly broadcasting messages. This results in a high number of transmitted messages.
7.2
Applicability beyond the AGV-simulator
Although our approach focusses on a simulation of an AGV-system, we hope our approach can be applied in a real-world AGV-system without major adaptations. The different abstractions and simplifications of the AGV simulator with respect to a real-world AGV-system are considered below. Besides transport assignment, a real-world AGV-system has other responsibilities, such as collision and deadlock avoidance and the refilling of the batteries of AGVs. We expect that our approach can be combined in a relative seamless and modular way with these and other concepts. This expectancy is based on the fact that no rigorous protocols are used in our approach. An AGV could switch from being guided by fields in the environment to an other behaviour mechanism (eg. deadlock or collision avoidance) without much complications. Also, an existing, simple collision avoidance system was already combined with our approach in a seamless way. Additionally, some other concerns besides transport assignment could be achieved by using fields. For example, the refilling of batteries could be achieved by letting reload stations 59
emit attractive fields, just like pick locations of transports. The degree of influence those fields have on an AGV could be determined by the amount of energy the AGV has left in its battery. In a real-world AGV-system, AGVs drive at variable speeds. A look-ahead buffer, which stores routing decisions made in advance, is used to allow faster and smoother movement. Although in the AGV simulator AGVs drive at constant speeds, we implemented a look-ahead buffer in our approach to stay as close to a real-world AGV-system as possible. In the AGV simulator, segments are straight and AGVs always drive forwards. The segments of a real-world AGV-system can be curved and AGVs have to drive either forwards or backwards on them. However, these aspects create no problem for our approach, in which AGVs only take decisions on a station-by-station basis. The above discussion seems to suggest it is feasible to apply our approach in a real-world AGV-system. However, since the communication-infrastructure of a real-world AGV-systems typically only has a limited bandwidth, it remains to be seen if the large amount of messages that are transmitted in our approach can be supported by a real-world AGV-system.
60
Appendix A Realistic AGV-system layout Below is a layout of a realistic AGV-system, provided by Egemin [11]. The segments are colored green, the stations blue.
61
Appendix B Class diagrams Below are the class diagrams of our implementation. Figure B.1 shows the class diagram associated with the behaviour of transport-agents. Figure B.2 shows the class diagram associated with the behaviour of AGV-agents. The older term ’order’ is used instead of ’transport’.
Figure B.1: The class diagram associated with the behaviour of transport-agents.
62
Figure B.2: The class diagram associated with the behaviour of AGV-agents.
63
Appendix C Interaction diagrams Below are the interaction diagrams of our implementation. Figure C.1 shows the interaction diagram associated with the behaviour of AGV-agents. Figures C.2 to C.7 are continuations of preceding diagrams. Figure C.8 shows the interaction diagram associated with the behaviour of transport-agents. The older term ’order’ is used instead of ’transport’.
Figure C.1: The interaction diagram associated with the behaviour of AGV-agents.
Figure C.2: Continuation of a preceding diagram.
64
Figure C.3: Continuation of a preceding diagram.
65
Figure C.4: Continuation of a preceding diagram.
Figure C.5: Continuation of a preceding diagram.
66
Figure C.6: Continuation of a preceding diagram.
Figure C.7: Continuation of a preceding diagram.
67
Figure C.8: The interaction diagram associated with the behaviour of transport-agents.
68
Bibliography [1] J. Ferber. Multi-Agent Systems. Addison-Wesley, 1999. [2] M. Wooldridge. An Introduction to MultiAgent Systems. John Wiley, 2002. [3] Marco Mamei, Franco Zambonelli, and Letizia Leonardi. Co-fields: A physicall inspired approach to distributed motion coordination. IEEE Pervasive Computing, 2004. [4] N. Boucke. Situated multi-agent approach for distributing control in automatic guided vehicle systems. Master’s thesis, K.U. Leuven, 2004. Only available in Dutch. [5] Automatic guided vehicle simulator. net/taskforces/agentwise/agvsimulator/.
http://www.cs.kuleuven.ac.be/
distri-
[6] Prof. T. Holvoet. Lecture notes on multi-agent systems, 2004. [7] D. Weyns and T. Holvoet. Formal model for situated multi-agent systems. Formal Approaches for Multi-agent Systems, Special Issue of Fundamenta Informaticae, 63(2-3), 2004. [8] T. De Wolf, G. Samaey, T. Holvoet, and D. Roose. Decentralised autonomic computing: Analysing self-organising emergent behaviour using advanced numerical methods. pages 1–12, 2005. [9] D. Weyns, E. Steegmans, and T. Holvoet. Protocol based communication for situated multi-agent systems. pages 118–126, 2004. [10] D. Weyns, E. Steegmans, and T. Holvoet. Towards active perception for situated multiagent systems. Special Issue of Journal on Applied Artificial Intelligence, 2004. [11] Egemin. http://www.egemin.be. [12] L.Breton, S. Maza, and P. Castagna. Simulation multi-agent de syst`emes d’agvs: comparaison avec une approche pr´edictive. 5e Conf´erence Francophone de MOd´elisation et SIMulation ”Mod´elisation et simulation pour l’analyse et l’optimisation des syst`emes industriels etlogistiques”, 2004. [13] Flavio De Paoli and Giuseppe Vizzari. Context dependent management of field diffusion: an experimental framework. 2002. [14] Marco Mamei and Franco Zambonelli. Motion coordination in the quake 3 arena environment: a field-based approach. 2003. [15] Marco Mamei, Franco Zambonelli, and Letizia Leonardi. Programming coordinated motion patterns with the tota middleware. 2003. 69
[16] The sims. http://thesims.ea.com. [17] P. Valckenaers, M. Kollingbaum, H. Van Brussel, O. Bochmann, and C. Zamfirescu. The design of multi-agent coordination and control systems using stigmergy. 2001. [18] A. Farinelli, L. Iocchi, D. Nardi, and V. A. Ziparo. Task assignment with dynamic perception and constrained tasks in a multi-robot system. 2004. [19] T. Knabe, M. Schillo, and K. Fischer. Imporvements to the fipa contract net protocol for performance increase and cascading applications. 2002. [20] T. Knabe, M. Schillo, and K. Fischer. Interorganizational networks as patterns for selforganizing multiagent systems. 2002. [21] N. Boucke, D. Weyns, T. Holvoet, and K. Mertens. Decentralized allocation of tasks with delayed commencement. In G. Chiara, P. Ciorgini, and W. van der Hoek, editors, ´ Proceedings, pages 57–68, 2004. EUMAS04 [22] A. Helleboogh, T Holvoet, D. Weyns, and Y. Berbers. Extending time management support for multi-agent systems. 2005.
70
