neighbors cells. Many extensions and modifications of the basic model can be found in the litterature [Bandini et al. 2001,. Blaise Pascal University, France. 181.
Distributed fire spreading simulation using OpenMOSIX Eric Innocenti Fabrice Bernardi Alexandre Muzy Laurent Capocchi Jean-François Santucci University of Corsica SPE Laboratory, UMR CNRS 6134 Quartier Grossetti 20250 Corte, France +33 4 95450208 {ino, bernardi, amuzy, capocchi, santucci}@univ-corse.fr ABSTRACT Parallel discrete event simulation of complex phenomena like fire spreading is known to be a challenge. We describe in this paper a distributed model and the techniques we used for implementing such a simulation on a cluster of workstations. The need in adapted methodologies in this area is a recurrent problem. Simulation times obtained do not allow developping effective prediction tools. Realistic large-scale simulations are obtained to the detriment of precision and simulation time. This is due to the physical models increasing complexity which is in constant opposition with the available computing resources. We show that an OpenMOSIX cluster could form a basis for a good distributed engine, easily adaptable for spatial model in large-scale simulations. KEYWORDS Distributed computing, parallel algorithms performances, discrete event simulation, fire spread simulation, MOSIX.
1. Introduction These last years, the international community has known an increase of ecological and technological disasters. Many countries were devastated by destructive forest fires or suffered of fatal floods. However, the means of prevention and fight are each day more effective. The use of computer tools is largely accepted in this domain and spreads with the development of new technologies. Their use makes it possible the improvment of the traditional methods and generates development techniques helping to understand and control the time evolution of these phenomena. Among those, computer simulation allows the rise of new concepts adapted to the study of such systems [Ingalls 2002] This work is related to the simulation of complex phenomena integrating space dynamics. We treat the design of a computer model for distributing simulation of fire spread. The need of adapted methodologies in this area is a recurrent problem. Indeed, the simulation times obtained are still too important and do not allow developping effective prediction tools. Obtaining realistic high speed simulations seems to be viable only to the detriment of the precision of the model. This is due to the physical models increasing complexity which is in constant opposition with the available computing resources. We try to find solutions to these problems by combining modeling and simulation concepts, objects and parallel computing techniques. This work relies upon the combination of simulation formalisms to specify discrete event simulators [Zeigler et al. 2000], and upon an innovative modeling method. The developed model allows an optimal use of parallel computing resources available and thus a gain in terms of performances. We discuss the techniques founded on OpenMOSIX for distributing the fire spread simulation. This paper is organized as follows. First, we introduce the essential software paradigms we use in our development (Section 2); second, we describe both the modeling and simulation structures (section 3); third, we explain the fire spread implementation details and analyze the simulation results (section 4); finally, we give some conclusions and some perspectives of work (section 5).
2. Related Works 2.1. Cellular Modeling Many methods can be used to design a model in the computer simulation domain. Among those, methods founded on models inspired by cellular automata are often used [Gardner 1970]. Many examples implementing advanced cellular models can be found in the litterature [Rennard 2000, Hoffmann and Al 2000, Chopard Masselot 1999, Sipper 1994, Maniatty and Al 1994]. History of Cellular Automata (CA) begins in 1940 with Stanislas Ulam (1909-1984), a polish mathematician. He studied the evolution of complex systems generated on the basis of elementary behaviors. He quickly realized he could build complex behavioral structures in a two-dimensional space split in cells, starting from very simple rules. In its first experiment cells could have two states: on or off. Then, evolution of a cell results from the current state and from the rules relative to the neighbors cells. Many extensions and modifications of the basic model can be found in the litterature [Bandini et al. 2001,
Blaise Pascal University, France
181
© OICMS 2005
Worsch 1997, Di Gregorio Serra 1999, Sipper 1999, Talia 2000]. All of them imply variations of the structural and behavioral rules [Langlois Phipps 1997]. For example, modification of the neighborhood definition is allowed. If we consider models based on a two dimension cellular automata, [Rennard 2000] references the following neighborhood illustrated Figure 1.
(a) Von Neumann neighbor
(b) Moore neighbor
(c) Moore extended neighbor
SG
SD
IG
ID
(d) Margolus neighbor
Figure 1. The most used two-dimensional neighborhoods for cellular automata.
2.2. Discrete Event Simulation and DEVS formalism Discrete event simulation implements models whose state variables evolve in time in a discrete way. Contrarily to continuous simulations, this kind of approach is based on state changes ocurring on the time axis. These changes represent the events of the simulation. They are executed in the timestamp order of their occurrences according to the simulated time. 2.2.1 Event Driven Simulation Discrete event simulation implies the development of models where the simulation time progresses by jumps of events occurrences [Erard Déguénon 1999]. In this case, only the significant moments where the model evolves are considered, i.e. during the process of simulation, only events generating a change of the value of the state variables are computed. For that, it is necessary to implement a scheduler to chronologically order the events. Each time progression implies a treatment, and all events are productive. The time progresses more or less quickly according to the computed events. Execution of an event at the head of the scheduler can generate additions or suppressions of other events. Event driven simulation is illustrated figure 2. tsim
Sche. ei
e i : event
e1
e2
e3 e4
e5
Figure 2. Events driven simulation. Two types of events can occur: external and internal events. External event occurrences are not controlled by the model. Internal events on the other hand, come from the events programmed by the model. This distinction on the principal types of events can led to the definition of two state transition functions: the external transition function ‘δext’, and the internal transition function ‘δint’. This kind of approach is specified by [Zeigler 1976, Zeigler et al. 2000], with the DEVS (Discrete eVent System specification) formalism. In the case of event driven simulation events management algorithms are implemented at the simulator level. They constitute the synchronization kernel of the simulation process and are integrated in the simulators. The interested reader can find a detailed description of these algorithms in [Coquillard Hill 1997, Balci 1988, Overeinder et al. 1991]. Algorithm 1 is an example of an event driven simulator.
Blaise Pascal University, France
182
© OICMS 2005
Event approach program Scheduler.initialisation() ; t=0 ; While scheduler not empty and ttfinal Get the first event in the scheduler ; Update clock simulation ; Execute the traitment associated to the event ; End while Get simulation results ; End
Algorithm 1. Discrete event simulator. The different treatments can generate new events inserted in the scheduler, as well as the suppression or the report of events. The execution of the treatments does not modify the clock of the simulator. Only the synchronization kernel is entitled to manage the time. 2.2.2 DEVS formalism Developed in the beginning of the 1970’s, DEVS is an abstract universal formalism used for discrete event modeling [Zeigler 1976]. A basic DEVS atomic model is a structure: DEVS = (XM, YM, S, δext, δint,λ, ta), Where, XM is the set of ports and input values, YM is the set of ports and output values, S is the set of system's states, δext is the external transition function, δint is the internal transition function, λ is the output function, ta is the advance time function. Components of the model are described via the descriptive variables. S represents the set of state variables. X is the set of input variables and Y the set of output variables. The atomic models are influenced by internal and external events. Atomic model activity is described by the internal transition function δint, the output function λ and the external transition function δext. External events are generated on the input ports of the atomic model and imply the model response to the outside. Internal events are programmed if there is not external events occurrence during ta. The reader interested in more details will benefit from the new reference book for DEVS [Zeigler et al. 2000].
2.3. MOSIX/OpenMosix MOSIX is a clustering tool developed by the team of the Professor Amnon Barak at the University of Jerusalem [Barak La'adan 1998]. It is designed as a set of patches applied to the Linux kernel operating system. MOSIX distributes the application tasks homogeneously through the workstation cluster, giving a solution to the distribution problem of load balancing. Although MOSIX is more than twenty years old, its development is mainly associated to its implementation on the GNU/Linux platform. MOSIX relies upon algorithms performing statistics and upon procedures that guarantee correct operations, even in the case of incidents (High Performance Clusters). MOSIX assigns a process to a node, according to the resources available. The objective is to optimize the use of each workstation in the cluster. A MOSIX cluster is fully decentralized, and can be compared to a "Peer to Peer" network. Each workstation manages its own processes comparing them to the other processing nodes and there is not hierarchy between the different workstation of the cluster. We used MOSIX for its simplicity of implementation. Indeed, using MOSIX only requires a patched Linux kernel and configuration files used to define the cluster.
Blaise Pascal University, France
183
© OICMS 2005
3. Distributed Modeling and Simulation Framework 3.1. Model definition With the development of parallel computing, spatial distributed models are well suited for large scale simulations on distributed parallel architectures. They offer a boundless execution support in terms of memory and they guarantee the highest exploitation of the computers power. Simulation models must be parallelized using the principles of the domain decomposition [Leduc 1999]. This technique consists in sharing the computer model which represents the spatial area to be simulated in different sub-elements. For this reason, the propagation domain represented by the simulation model is broken down into different sub-domains; it is quite common to consider as much as sub-domains than desired processes. Sub-domain computations are executed in parallel and at the level of each local process. Distribution of the simulation model raises problems of neighborhood coherence while the simulation runs, due to data localized on the common interfaces of the sub-domains. Indeed, the spatial discretization implies knowing the values around an element situated on a mesh of the domain. Hence, when decomposing the model, each sub-domain must receive one or several layers of cellular elements called “ghost cells” and coming from the neighbors sub-domains. They constitute the boundary overlapping elements of a distributed simulation domain and they allow the simulator to calculate the value of the border elements. These elements are updated at each time step on the values calculated by the process taking care of the subdomain. [Leduc 1999] emphasizes that message passing is the main type of communication in distributed models and represents a restrictive factor for the performances of the simulation. Figure 3 illustrates the domain division implemented, and the mechanisms of messages passing we will employ in order to guarantee the global coherence of the simulation. Spatial discretisation implies the knowledge of the values around to the element situated on a mesh of the domain
Initial model
Sub-domain 1 (MU_Object)
Sub-domain 2 (MU_Object)
1
Sub-domain 3 (MU_Object)
3 2
Exchange of ghost cells between the 1 and 2 subdomains 1 and 2
Messages Border cells (CES_Object)
Figure 3. Division of the model for distributing fire spread simulation. The different sub-domains of the distributed model are implemented in an evolved cellular automaton (MU_Object) and the data exchanged are used to update the attributes of the elementary cellular objects (CES_Object) established as “ghost cells”.
3.2. Distributed architecture of the simulation Distributed discrete event simulation brings a lot of interest because it provides structural framework and services allowing the simulation of large-scale models impossible to exploit on traditional computer architecture. Performances often depend
Blaise Pascal University, France
184
© OICMS 2005
on the method used to process communication during the synchronization of the information needed for the accuracy of the simulation. This communication is founded on message-passing principles. Implementations of these techniques are numerous and varied [Reinefeld et al. 1998]. However, code reuse is compromised since the distribution code get tangle with simulator code providing the basic services for the simulation. We consider distribution as a nonfunctional property of an application, i.e. like a specific device of execution, advisable to compose separately in order to "build" the application. It is necessary to develop a composition mechanism to make this assembly. On the basis of this consideration, we propose to distribute the simulation as an aspect which can be added to the simulator when the simulation is distributed. Subsequently, the distribution of the simulation avoids mixing the basic functional services of the simulator, with those corresponding to the distribution. This considerably improves the evolution of model components. The communication techniques for synchronizing are numerous and diverse, and the installation of a mechanism for choosing a type of communication is also necessary. We design the simulator using the Decorator pattern [Gamma et al. 1994], to control the distribution as a sequence of functions executed in a specific order. The objective is to dynamically associate additional responsibilities to simulator objects. Figure 4 illustrates this scheme. Simulator
1
send() ack()
AcDEVSSimulator send() ack()
Decorator 0..1
send() ack()
DistDecoratorA send() ack() send_ghost_mfs() ack_ghost_mfs()
void Decorator::send(message& m){ if(_simulator!=0){ _simulator->send(m); } } void Decorator::ack(message& m){ if(_simulator!=0){ _simulator->ack(m); } }
DistDecoratorB send() ack() send_ghost_mpi() ack_ghost_mpi()
Decorator::ack(); Decorator::send(); send_ghost_shared() ack_ghost_shared()
Figure 4. The Decorator pattern. In order to simulate the distributed model, we describe it as a set of autonomous logical components executed on several machines without centralized control, and cooperating in order to ensure the simulation of the spreading phenomenon. The need of distributed collective information (ghost cells) implies to exchange common information, and to avoid causality error at each time step and for each node. Communication management between the different nodes implies to use adequate methodology. The Decorator pattern facilitates adaptation according to the execution context. It makes it possible to install a composition mechanism in order to assemble AcDEVSSimulator objects (Simulator) and the various possible methods of communication (Classes DistDecoratorA and DistDecoratorB). The DistDecoratorA and DistDecoratorB objects implement the communication methods for exchanging the ghost cells, respectively send_ghost_mfs(), ack_ghost_mfs() and send_ghost_mpi() and ack_ghost_mpi(). These methods are called after execution of the local treatments implemented in the methods ack() and send() of the concrete component AcDEVSSimulator, being used to ensure the communications between the parents simulators and coordinators on each node.
4. Application to fire spreading 4.1. Modeling and simulating fire spreading Physical models are used to improve decision-aid. The physical model we use (1) has been validated against numerous experiments [Santoni et al. 1998, Dupuy 1995]. Then, the propagation of fire results from the evolution of a front of flame in a domain of 1 m² of pine needles, without slope, nor wind. A preliminary numerical study makes it possible to solve the model using the explicit method of finite differences, leading to a system of differential equations. The domain of propagation is then divided into elementary components constituting the ground and the plants, each one being described by
Blaise Pascal University, France
185
© OICMS 2005
the following algebraic equation: k +1
∂σ Ti k, j+1 = aTi k−1, j + aTi k+1, j + bTi k, j −1 + bTi k, j +1 + cQ v + dTi k, j ∂t i , j
(1)
Where Ti,j represents the temperature of a cell of the cellular automaton. Coefficients a,b,c, and d depend on the time step and on the space step considered. The parameters of the model are given starting from experimental statements of temperature obtained. The numerical results were compared to experimental data for various ignitions and the quality of the predictions is remarkable [Santoni et al. 1998]. However, the precision of these models make them difficult to be simulated under real-time constraints.
4.2. Distributed implementation on a Linux cluster The distribution of the simulation model implies to divide the propagation plan into several sub-domains. The simulation treatments are then distributed onto a cluster of workstations, each node executing its own part of computing. This kind of models allows to simulate more realistic large-scale domains and to overtake the limits imposed by the finite dimension of the computers memory. One of the major problems is the problem of the load balancing (Dynamic Load Balancing) [Lan et al. 2002, Tonguz Yanmaz 2003, Ghanem et al. 2004]. We tackled this problem using automatic migration of processes on nodes. We retained the openMOSIX environment for implementing the distributed model. OpenMOSIX (Multicomputer OS) dynamically controls the load balancing on a cluster of computers. The experiments rely on the Knoppix/MOSIX distribution. The distribution of the computing resources is executed for the different experiments in a transparent way. Figure 5 illustrates the physical architecture developed for distributing the simulation. The fire front is divided in equal areas distributed on the workstations network. Each node computes a part of the fire front and communicates with the others.
Figure 5. Division of the propagation domain for distributing the simulation of fire spreading. Figure 6 sums up the performances obtained for a line ignition on a domain of 115 200 pine needles components (240 X 480), respectively for one, two, three and four nodes.
Blaise Pascal University, France
186
© OICMS 2005
Execution times (second)
3000 2500 2000 1500 1000 500 0 1
2
3
4
Nodes
Figure 6. Performances of the distributed simulation on a cluster of workstations. The simulation presents honorable performances. The object-oriented architecture easily implements distributed models of propagation. This development offers a reference application for developing future large-scale models we plan to specify.
5. Conclusions and prospects This work proposes a methodology for modeling and simulating spatial complex phenomena on distributed architecture. The proposed methodology takes into account many concepts coming from existing works in the domain. We associate techniques and methods of discrete event simulation (DEVS), distributed computing (Ghost cells) and oriented-object programming (Design patterns), in order to simulate large-scale fire spreading. Modeling such spatial systems is facilitated by the use of enhanced cellular automata. The conception of the distributed model raises many design problems. Solutions are based on design patterns and enable to define a modular architecture adjustable to various contexts of execution. The model integrates object technologies and parallelization techniques in a modular and extensible way. Hence, the simulation kernel, more precisely the methods dealing with message exchanges for the distributed simulation, has to be enhanced. This study gives an overview of the performances we can reach using such methods in the domain of parallel and distributed simulation of fire spread, and constitutes a first stage. The validity of the approach and the encouraging results show that openMOSIX cluster could form a basis for a good distributed engine, easily adaptable for spatial model in large-scale simulations. REFERENCES [Balci 1988] Balci, O. (1988). The Implementation of Four conceptual Frameworks for Simulation Modeling in High-Level Languages. Technical Report SRC-88-009. System Research Center, Virginia State University. [Bandini et al. 2001] Bandini, S., Mauri, G., Serra, R. (2001). Cellular automata : From a theoretical parallel computational model to its application to complex systems. Parallel Computing, 27, pp. 539-553. [Barak La'adan 1998] Barak, A., La'adan, O. (1998). The MOSIX Multicomputer Operating System for High Performance Cluster Computing. Journal of Future Generation Computer Systems, Vol. 13, No. 4-5, pp. 361-372. [Chopard Masselot 1999] Chopard, B., Masselot, A. (1999) Cellular automata and lattice Boltzmann methods: a new approach to computational fluid dynamics and particle transport. Future Generation Computer Systems, Vol.16, N°2--3, pp.249-257. [Coquillard Hill 1997] Coquillard, P., Hill, D., R., C. (1997). Modélisation et Simulation d’Ecosystèmes. Des modèles déterministes aux simulations à évènements discrets. Masson. [Di Gregorio Serra 1999] Di Gregorio, S., Serra, R. (1999). An empirical method for modelling and simulating some complex macroscopic phenomena by cellular automata. Future Generation Computer systems 16, pp. 259-271.
Blaise Pascal University, France
187
© OICMS 2005
[Dupuy 1995] Dupuys, J., L. (1995). Slope and fuel load effects on fire behaviour : Laboratory experiments in pine needles fuel beds. International journal of wildland fire. [Erard Déguénon 1999] P.J. Erard, P. Déguénon. (1999). Simulation par évènements discrets. Concepts et réalisations en Simula, Ada et Smalltalk. Presses Polytechniques et Universitaires Romandes. [Gamma et al. 1994] Gamma, E., Helm, R., Johnson, R., et Vlissides, J. (1994). Design patterns. Elements of Reusable Object-Oriented Software. Addison Wesley. ISBN:0-201-63361-2. [Gardner 1970] Gardner, M. (1970). Mathematicals Games, The fantastic combinations of John Conway’s new solitaire game « life ». Scientific American 223, pp.120-123. [Ghanem et al. 2004 ] Ghanem, J., Dhakal, S., Abdallah, C., T., Hayat, M., M., Jerez, H. (2004). Load Balancing in Distributed Systems with Large Time Delays: Theory and Experiments. Dans proceedings IEEE Mediterranean conference on control and automation, Turkey [Hoffmann et al. 2000] R. Hoffmann, K., P., Völkmann, S., Waldschmidt. (2000). Global cellular automata GCA: an universal extension of the CA model. ACRI 2000 work in progress session, Karlsruhe, Germany. [Ingalls 2002] Ingalls, R., G. (2002). Introduction to simulation. Proceedings of the 2002 Winter Simulation Conference. E. Yücesan, C-H. Chen, J.L. Snowdon, J.M. Charnes, eds. pp. 7-16. [Lan et al. 2002] Lan, Z., Taylor, V., Bryan, G. (2002). Dynamic Load Balancing of SAMR applications Systems. Journal of Scientic Programming, Vol 10(4), pp. 319-328.
on
Distributed
[Langlois Phipps 1997] Langlois, A., Phipps, M. (1997). Automates cellulaires, application à la simulation urbaine. Edition Hermes. [Leduc 1999] Leduc, T. (1999). Modélisations par réseau d’automates cellulaires et simulations parallèles du phénomène de subduction-érosion en tectonique des plaques. Thèse de doctorat en systèmes informatiques. Université Paris 6. [Maniatty et al. 1994] B. Maniatty , T. Caraco, « TEMPEST: a fast spatially explicit model of ecological dynamics on parallel machines ». Proceedings of the 1994 simulation multiconference on Grand challenges in computer simulation, p.114-119, April 1994, San Diego, California, United States. [Overeinder et al. 1991] Overeinder, B., J., Hertzberger, L., O., Sloot, P., M., A. (1991). Parallel Discrete Event Simulation. Dans W.J. Withagen, editor, The Third Workshop Computer systems, Faculty of Electrical Engineering, Eindhoven University, pp.19-30. [Reinefeld et al. 1998] Reinefeld, A., Gehring, J., Brune, M. (1998). Communicating Across Parallel Message-Passing Environments. Journal of Systems Architecture, Vol. 44, pp. 261-272. [Rennard 2000] Rennard, J., P. (2000). Introduction aux automates cellulaires. http://www.rennard.org/alife, pp 1-25. [Santoni et al. 1998] Santoni, P., A. (1996). Propagation des feux de forêts : Modélisation dynamique et résolution numérique, validation sur des feux de litière. Thèse de Doctorat, Université de Corse. [Sipper 1994] Sipper, M. (1994). Non-Uniform Cellular Automata: Evolution in Rule Space and Formation of Complex Structures. Dans proceedings of the 4th International Workshop on the Synthesis and Simulation of Living Systems \ Artificial Life IV, R. A. Brooks and P. Maes (eds.), pp. 394-399. [Sipper 1999] Sipper, M. (1999). The emergence of cellular computing. In IEEE Computer, July, pp. 18-26. [Talia 2000] Talia, D. (2000). « Cellular processing tools for high-performance simulation ». Computer. pp. 44-52. [Tonguz Yanmaz 2003] Tonguz, O., K., Yanmaz, E. (2003). On the Theory of Dynamic Load Balancing, IEEE Global Telecom. Conf. (GLOBECOM’03), vol. 7, pp. 3626-3630, 2003. [Worsch 1997] Worsch, T. (1997). Programming environments for cellular automata. Dans Proceedings Cellular Automata for Research and Industry (ACRI96), Springer, Berlin, pp.12. [Zeigler 1976] Zeigler B.P., « Theory of Modelling and Simulation », Wiley, New York. 1976. [Zeigler et al. 2000] B. P. Zeigler, H. Praehofer, T. G. Kim, « Theory of Modeling and Simulation: Integrating Discrete Event and Continuous Complex Dynamic Systems », Academic Press, January, 2000.
Blaise Pascal University, France
188
© OICMS 2005
