International Journal of Research and Reviews in Computer Science (IJRRCS) Vol. 3, No. 2, April 2012, ISSN: 2079-2557 © Science Academy Publisher, United Kingdom www.sciacademypublisher.com/journals/index.php/IJRRCS
1543
Multi Agent-based Rescue Simulation for Disable Persons with the Help from Volunteers in Emergency Situations Kohei Arai1 and Tran Xuan Sang2 1
Graduate School of Science and Engineering, Saga University, Saga city, Japan Faculty of Information Technology, Vinh University, Vinh, Vietnam
2
Email:
[email protected],
[email protected]
Abstract – In this paper, we present a multi agent-based simulation model for helping disabled persons in emergency situation. This simulation model allows the interactions among victims and volunteers during the evacuation process. Disabled persons and volunteers are modeled as agents with different behaviors. When the decisions to help disabled people are made, the volunteers will apply flexible technique by making choices between the shortest route and the least congested route according to route network conditions for every simulation time-step. The shortest path algorithm of Dijkstra and the reinforcement learning method of Q-learning algorithm are used to find the appropriate path to reach to the disabled people. In order to represent the route network in more realistic way, the GIS route network model is used. The simulation results indicate that the simulation using the flexible technique has less rescue time than these using two methods respectively. Keywords – Multi agent-based simulation, Shortest path algorithm, Q-learning algorithm
1.
Introduction
Disabled people are considered as a high risk group when disasters and emergencies occur. The data of the recent disasters i.e. Tsunami, Katrina and earthquake show that the mortality of the disabled people during the disaster was very high (Ashok Hans, 2009). Because handicapped people may face physical barriers or difficulties of communication that they are not able to respond effectively to crisis situations. They were not able to evacuate by themselves. Obviously, the disabled people need assistances to evacuate. In order to simulate the rescue process of volunteers for disabled people, the computer simulation models are used. Most computer based simulation evacuation models are based on flow model, cellular automata model, and multi agent-based model. Flow based model lacks interaction between evacuees and human behavior in crisis. Cellular automata model is arranged on a rigid grid, and interacting with one another by certain rules [1]. A multi agent-based model is composed of individual units, situated in an explicit space, and provided with their own attributes and rules [2]. This model is particularly suitable for modeling human behaviors, as human characteristics can be presented as agent behaviors. Therefore, the multi agent-based model is widely used for evacuation simulation [1-4]. Recently, Geographic Information Systems (GIS) is also integrated with multi agent-based model for emergency simulation. GIS can be used to solve complex planning and decision making problems [5, 6]. In this study, GIS is used to present road network with attributes to indicate the road conditions. We develop a multi agent-based model for rescue disabled people with the help of volunteer to capture the phenomena and complexities during evacuations. The
volunteers have to make decision which victims should be helped in order to give first-aid and transport with the least delay to shelter. The decision making is based on several criteria such as health condition of the victims, location of the victims and location of volunteers. The rest of the paper is organized as follows. Section 2 reviews related works. Section 3 describes the definition of agent behavior and decision making procedure. Section 3 also presents the path finding method by utilizing Dijkstra and Q-learning algorithm. Section 4 provides the experimental results of different evacuation scenarios. Finally, section 5 summarizes the work of this paper and outlines future work.
2.
Related Works
There is a considerable research in emergency simulation by using GIS multi-agent-based models. Ren et al. (2009) presents an agent-based modeling and simulation using Repast software to construct crowd evacuation for emergency response from an area under a fire. Characteristics of the people are modeled and tested by iterative simulation. The simulation results demonstrate the effect of various parameters of agents. Zaharia et al. (2011) proposes agentbased model for the emergency route simulation by taking into account the problem of uncharacteristic action of people under panic condition given by disaster. Drogoul and Quang (2008) discuss the intersection between two research fields: multi-agent system and computer simulation. This paper also presents some of current agent-based platforms such as NetLogo, Mason, Repast and Gama. Bo and Satish (2009) presents an agentbased model for hurricane evacuation by taking into account the interaction among evacuees. For the path finding, the agents can choose the shortest
K. Arai and T.X. Sang / IJRRCS, Vol. 3, No. 2, pp. 1543-1547, April 2012
path and the least congested route respectively. Cole (2005) studied on GIS agent-based technology for emergency simulation. This research discusses about the simulation of crowding, panic and disaster management. Quang et al. (2009) proposes the approach of multi-agentbased simulation based on participatory design and interactive learning with experts’ preferences for rescue simulation. This paper presents the decision making process based on several criteria such as the location of victims, health condition. After the decisions are made, the shortest path is used to reach to the disabled people. Through the view of this background, this study will focus mainly on dynamic route choices under considering the information of victims and route conditions during the rescue simulation. We also use criteria similarly to Quang et al. (2009) to make the decision to rescue the disabled people but we propose a flexible approach for path finding by making choices between shortest route and least congested route for every time-step of the simulation; and the weights of criteria are also evaluated in more reliable way by applying Fuzzy Analytic hierarchy Process (FAHP).
3.
Proposed Rescue System
3.1. Rescue Simulation Model The centralized rescue model is presented which have three types of agent: volunteer, disabled people and route network. The route network is also considered as agent because the condition of traffic in certain route can be changed when disaster occurs. The general rescue model is shown in Figure 1.
Figure 1. Centralized rescue model
After the warning is issued, all the disabled persons send information to the emergency center via special device. This device measures the condition of disabled people such as heart rate, body temperature and this can also trace the location of disabled persons by GPS. Emergency center will collect that information and then broadcast to volunteers’ smart-phones through internet. After checking the condition of victims, volunteers make their own decision to help victims and inform to emergency center. The process of making decision will be described in detail in the following section. Generally, there are four components in our model: environment definition, agent definition, visualization and rescue & route choice decision making. We briefly describe these four components, Environment definition: this module defines the context of environment where each autonomous agent interacts with the environment and other agents to proceeds to its destination subject to certain imposed
1544 criteria. It also presents the real-time information about agents as well as the traffic condition during the simulation. Agent definition: this module sets the attributes of the agents and situates it in the environment based on certain rules. Visualization: this module shows the animation of the simulation and graphically results. Rescue & route decision making: this module carries out the process to assign which volunteers should help which disabled people and to find the appropriate route to reach to the victims.
3.2. Volunteer Agent This study proposes the integration of volunteers’ decision making to help the disabled people into the evacuation modeling so that the different scenarios can be analyzed and evaluated. In our model, each volunteer is an agent. After the warning is issued, a volunteer will made a decision to help disabled people. We assume that each volunteer can help more than one disabled persons depending on the capacity of vehicle and the condition of victims. When the evacuation starts, each volunteer agent makes a decision to help disabled people and starts to move from their location to disabled peoples’ location. At initial state, the shortest route will be selected but it can be changed at each intersection if volunteers get new information about disabled people and/or the congestion level of current road exceeds a pre-defined tolerance limit. 3.3. Disabled People Agent In this research, the disable people will be consider as unmovable agent. We assume that, each disabled people has a device which can measures the condition of disabled people such as heart rate, body temperature, location of disabled. Such kind of information will be sent to emergency center automatically. Volunteer agents will base on this kind of information to help disabled people. 3.4. The Criteria of Volunteer to Choose Disabled People The volunteer’s decision depends on the information of disabled people which receives from emergency center; therefore decisions must follow certain criteria to improve their relief activities. For example, the volunteers must care about condition of disabled people; the more seriously injured people should have the more priority even if they locate further than the others. There are several criteria that volunteers should take in account before starting rescue process [7]. C1: Time from volunteer to disabled people C2: Time from disabled people to nearest other disabled people C3: Health condition of disabled people C4: Time from disabled people to nearest other volunteer Disabled people who have lesser values for criteria of C1, C2, C3 and greater values for criteria of C4 will have higher priority in the volunteer’s decision process as shown in Figure 2.
K. Arai and T.X. Sang / IJRRCS, Vol. 3, No. 2, pp. 1543-1547, April 2012
Figure 2. Priority of the values of Ci
3.5. Determination Important Weight of Criteria The Fuzzy Genetic Algorithm (FGA) is used to calculate the important weight of criteria [9]. The pair-wise comparison among criteria is shown in Table 1. Referring to procedure in the reference [9], a programming with c# had been created with the following inputs: number of criteria (N = 4); size of population (M = 30); crossover probability (pcross = 90%); mutation probability (pmut = 10%); and number of reproduction (L = 100). The solution obtained is w = (0.2882, 0.2219, 0.2738, and 0.2161). These values are also considered as input parameters for rescue simulation. It can be change by adjusting the pair-wise comparison in Table 1. Table 1. Pair-wise comparison among criteria.
Criterion
Linguistic preference
Fuzzy number
Criterion
C1 C1 C1 C2 C2 C3
Good Fair Good Poor Fair Good
(0.667, 0.833, 1) (0.333, 0.5, 0.667) (0.667, 0.833, 1) (0, 0.167, 0.333) (0.333, 0.5, 0.667) (0.667, 0.833, 1)
C2 C3 C4 C3 C4 C4
For each volunteer, the utility function, 4
F (vk ) wi * vik i 1
wi denotes the weight of the
where k i
while v denotes the value of the
i
th
ci
criteria
criteria for the
k th
victim. The sign of value of criterion C 4 will be reversed if utility function is calculated. Certain victim will be selected if corresponding utility function has got minimum value. 3.6. Route Choice Model to Get the Disabled People After the volunteers select the disabled people to help, the volunteer will go to the location of disabled people. The efficient route requires not only to cost less time but also to provide a chance to help more disabled people in case one volunteer can help more than one disabled people.
1545
3.7. Route Choice Procedure The ordinary method of route choice is shortest route method. There are several shortest route algorithms such as Bellman-Ford-Moore, Dijkstra, Threshold Algorithm, Topological Ordering and so on. However, most of these algorithms are mainly static, which means that changes in route network, i.e. changes in route congestion, density, blocked route, will cause the inefficiency of these algorithm.. Therefore, in this research we will use the Dijkstra’s algorithm at the initial state of rescue simulation. In case of the changes in route network, the reinforcement learning methods of Q-learning will be used to find the path. We will not describe the Dijkstra’s algorithm in detail as it is a popular shortest route method. The Q-learning method applied for route finding is described in the following section. Figure 3 presents the procedure of route choice model. Applying path finding method (Dijkstra or Q-learning) is based on traffic level. Traffic level is determined based on traffic density and route condition (blocks or unblock). If total route density at current shortest route exceeds 80% or current road link is block, then the traffic level has got value more than 1. Otherwise the traffic level has got value less than or equal 1. 3.8. Q-learning Method for Route Finding Q-learning is one of the most well known reinforcement learning algorithm. The problem model consists of an agent, states S and a set of actions per state A. In this study, the agents are volunteers; States are locations of volunteers and disabled people; Actions are the movements from current location to others. At each time-step, the agent observes the vector of state xt, then chooses and applies an action ut. As the process moves to state xt+1, the agent receives a reinforcement r(xt, ut). The goal of the training is to find the sequential order of actions which maximizes the sum of the future reinforcements, thus leading to the shortest path from start to finish. The Q-learning transition rule is shown as below: Q(state,action)=R(state,action)+gamma*Max[Q(next state,all actions)] The gamma parameter has a range of 0 to 1 (0 1), and ensures the convergence of the sum. If gamma is closer to zero, the agent will tend to consider only immediate rewards. If gamma is closer to one, the agent will consider future rewards with greater weight, willing to delay the reward. (John, 2011). The Q-Learning algorithm goes as follows: (1). Set the gamma parameter, and environment rewards in matrix R. (2). Initialize matrix Q to zero. (3). For each episode: (i) Select a random initial state. (ii) Do While the goal state hasn't been reached. (iii) Select one among all possible actions for the current state. (iv) Using this possible action, consider going to the next state. (v) Get maximum Q value for this next state based on all possible actions. Compute: Q(state, action) = R(state, action) + gamma * Max[Q(next state, all actions)] (vi) Set the next state as the current state. (vii) End Do, (viii) End For
K. Arai and T.X. Sang / IJRRCS, Vol. 3, No. 2, pp. 1543-1547, April 2012
1546
Find shortest route by Dijkstra method
Make decision for selecting disabled people
Update traffic condition
False
Traffic level>1
True Find shortest route by Qlearning method Figure 3. Route Choice Procedure. Figure 5 GAMA multi-agent platforms
3.9. Simulation Tools and Conditions A group home of Saga city, Japan which is shown in Figure 4 is assumed as an example of rescue simulation. GAMA multi-agent platform is used for the rescue simulation which is shown in Figure 5. In the figure, input parameters can be set using top left boxes. The top right figure shows geographic map for the simulation while the bottom right shows the simulation results of changing the number of victims, the number of evacuated people for the time being. Using GAMA platform, simulations based on Dijkstra algorithm and the proposed rescue system are conducted.
4.
Experimental Results
In this section, we present experimental studies on different scenarios. The goal is to examine the proposed method of route choice model and method of selecting disabled people to rescue. The evacuation time is evaluated from the time at which the first volunteer start moving until the time at which all alive victims arrive at the shelters. The simulation model is created using the GAMA agent-based platform (Amouroux et al. 2009) [8]. Table 2 presents the results of evacuation process with difference number of disabled people and volunteers. The experimental results show that if we use the method of integration of Dijkstra and Q-learning, the dead victims and the evacuation time are less than those using only Dijkstra’s algorithm. Table 2. Evacuation Result
Figure 4. Example of the victims' trajectory on GIS representation with GPS receiver. Clickable map allows display the location/attitude/health conditions on the GIS representation.
Volunteer
Victim
100 200 200 300
100 100 200 200
(a) Volunteer :100
Integration of Dijkstra and Q-learning Dead Evacuation victim time 3 1450 0 980 5 2400 2 2200
Dijkstra’s algorithm Dead victim 10 2 9 4
Victim: 100 Death victim: 3
Evacuation time 1870 1340 3120 2870
Rescue time: 1450
K. Arai and T.X. Sang / IJRRCS, Vol. 3, No. 2, pp. 1543-1547, April 2012
Batty, M., “Agent-Based Technologies and GIS: simulating crowding, panic, and disaster management”, Frontiers of geographic information technology, chapter 4, 81-101, 2005 [7] Chu, T.-quang., Drogoul, A., & Boucher, A., Interactive Learning of Independent Experts ’ Criteria for Rescue Simulations, Journal of Universal Computer Science, vol. 15, no. 13, 2701-2725, 2009. [8] Amouroux, E., Chu, T.-quang., Boucher, A., & Drogoul, A., “GAMA: An Environment for Implementing and Running Spatially Explicit Multi-agent Simulations”, Agent computing and multi-agent systems, 359-371, 2009. [9] Arai K. & Sang T.X., “Fuzzy Genetic Algorithm for Prioritization Determination with Technique for Order Preference by Similarity to Ideal Solution“, International Journal of Computer Science and Network Security, vol.11, no.5, 229-235, May 2011. [10] Arokhlo M.Z, Selamat A & Hashim M., “Multi-agent Reinforcement Learning for Route Guidance System“, International Journal of Advancements in Computing Technology, vol. 3, no.6, July 2011. [6]
(b) Volunteer :200
Victim: 100 Death victim: 0
Rescue time: 980
(c) Volunteer :200
Victim: 200 Dead victim: 5
Rescue time: 2400
(d) Volunteer :300
Victim: 200 Death victim: 2
Rescue time: 2000
Figure 7. Evacuation simulation results for the proposed method.
5.
Conclusion
This paper offers a new paradigm of rescue simulation based on multi-agent model with the integration of Dijkstra and Q-learning for finding the shortest and best path to rescue disabled people. Based on traffic condition, volunteers will find the route defined by Dijkstra method or by Q-learning method. The GIS route network model is also used to simulate the evacuation road. The simulation results indicate that the simulation using the method of integration of Dijkstra and Q-learning has less rescue time than these using two methods respectively. GAMA multi-agent platform is used for the rescue simulation which is shown in Figure 5. In the figure, input parameters can be set using top left boxes. The top right figure shows geographic map for the simulation while the bottom right shows the simulation results of changing the number of victims, the number of evacuated people for the time being.
References [1]
[2]
[3]
[4] [5]
Ren, C., Yang, C., & Jin, S., “Agent-Based Modeling and Simulation on emergency”, Complex 2009, Part II, LNICST 5, 1451 – 1461, 2009. Zaharia, M. H., Leon, F., Pal, C., & Pagu, G., “Agent-Based Simulation of Crowd Evacuation Behavior”, International Conference on Automatic Control, Modeling and Simulation, 529-533, 2011. Chu, T.-quang, Drogoul, A., “Agent-based simulation: definition, applications and perspectives”, Invited Talk for the biannual Conference of the Faculty of Computer Science, Mathematics and Mechanics, 2008. Bo, Z., Satish V., “Agent-based modeling for household level hurricane evacuation”, Winter Simulation Conference, 2009. Cole, J.W., Sabel, C.E., Blumenthal, E., Finnis, K., Dantas, A., Barnard, S., & Johnston D.M., “GIS-based emergency and evacuation planning for volcanic hazards in New Zealand”, Bulletin of the New Zealand society for earthquake engineering, vol. 38, no. 3, 2005.
1547
