New synchronization method for the parallel ... - Semantic Scholar

New synchronization method for the parallel simulations of wireless networks Sławomir Nowak1 , Mateusz Nowak1 , Paweł Foremski1 1

Institute of Theoretical and Applied Informatics, Polish Accademy of Science 44-100 Gliwice, Poland {emanuel, mateusz, pjf}@iitis.pl

Abstract. We present a new synchronization method for parallel discrete event simulation for wireless networks. The method merges paradigms of timestepped and event-driven simulations to achieve reduction of messages exchanged between local processes of simulation. The method is particularly suitable for wireless network simulation, in which objects share the same physical medium and parallel simulation involves a significant overhead for interprocess communication. The article describes the method and compare it to conservative and optimistic synchronization methods, on the basis of results of simulations of selected wireless network model. Keywords: Parallel simulation, wireless network simulation, synchronization.

1 Introduction Discrete event simulations (DES) has appeared as the most convenient approach for the performance evaluation of network protocols and architectures. However with the increasing complexity of simulation scenarios the demands on computational resources also significantly increase. This problem is particularly important for the simulation of wireless networks, due to the complexity of the physical layer, shared medium and possible interferences. Several wireless network simulators have been proposed. Examples are NS2[1], GTNetS [2], and some popular extensions of OMNeT++ [3]: MobilityFramework [4], INET packages [5], MiXiM [6] etc. The popular approach is to simplify the physical layer model to reduce the number of events and computational complexity. Simplifying the complexity of the model, however, leads to less accurate simulation but the scale of simulated model increases. Different simulation tools are different degrees of detail of physical layer implementation and the complexity of the wireless physical layer enforces the use of simplified models, as the tradeoff between the accuracy and the scalability of simulators. The impact of the physical layer modeling accuracy on both the computational cost and the confidence in simulation s was investigated and presented in the literature, among others [7], [8].

1.1 Parallel simulations A single simulation of a network model with thousands of nodes can occupy considerable time to obtain statistically trustworthy results, and many simulatio n studies require several simulation runs. Independent replicated simulation runs method is an alternative, but often one simulation run may depend on the results of earlier runs as input. Using parallel discrete event simulation (PDES) over several processors, cores or hosts, it is possible to achieve a considerable speedup . The simulation’s scenario is divided in a number of logical processes (LPs), each of them has its own clock (LVT, local virtual time) and executes a part of the scenario. LPs share demanded resources among several computers or processors [9]. Even though distributing the model over several computers the simulation execution with given synchronization algorithm may result in slower execution than on a single workstation, but at least it is possible to run the model. The challenging problem with distributed simulation is to synchronize the LPs to ensure each LP processes events in right time stamp order. The case where the incoming event’s timestamp is less than LVT of the given LP is called causality error. There are two kinds of algorithms to handle event synchronization: conservative algorithms and optimistic ones. In the conservative algorithms, a causality error can never happen. It is obtained by avoiding violating the local causality constraint. In the most popular conservative synchronization (known as Chandy/Misra/Bryant algorithm) “null messages”, containing lower timestamp bound of future messages the LP sends, are exchanged between objects for ensuring the given event timestamp is safe to process [10]. In optimistic algorithms the causality error may happen, but they are detected and recovered using a withdrawal mechanism (called rollback ), which moves the state of LP to the moment when processing of delayed event is safe. As rollback withdraws action taken by LP, also events sent in rollbacked period must be cancelled. This leads to rollbacks on another LPs. Consequently, it is necessary to store last states of objects. Rollback operation is time-costly, and necessity of storing past states of LP binds with high memory overhead. The representative optimistic synchronization algorithm is the Time Warp [11]. Parallel simulation of wireless networks , while maintaining accuracy physical layer model, is a particularly difficult issue. It is mostly because of the shared type of the medium and the resulting intensity of communication between objects. In conventional network simulators of higher layers of network protocols each data unit is represented by a single event, with timestamp set to the delivery time. However, to model the full complexity of the wireless physical layer it is required to represent each data frame by two events: the beginning and end of the transmission, separately for each object using a shared medium. When receiving a frame the interference is evaluated in order to determine which signals are interfering with each other. This set of interfering signals can be very large for large scale simulations and require a very frequent synchronization between LPs and are often useless as frequent communication between LPs severely lowers efficiency of simulation.

Although some of the simulation tools supports for parallel simulation (eg, PDNS – distributed version of NS-2 [12] or OMNeT++ [22]) in the case of complex simulation models (e.g. INET for TCP/IP and wireless network simulations) is not possible or is very difficult. The authors attempted to use a package such as INET simulation in parallel, but the results obtained have proved ineffective [13]. There are also available dedicated simulation tools for simulation of wireless and mobile networks in parallel, e.g. MoVeS [14], GloMoSim [15] and some related work was done [16], [17] etc. Available tools and methods however, often limited the accuracy of the simulation at the physical layer, compensating for the increased scalability. In this situation it is reasonable to develop new or modified synchronization methods to improve performance of parallel simulation of wireless communication, while maintaining the accuracy of detailed model of the physical layer. It seems that this is possible by taking into account specific character of wireless communication protocols. In previous work the authors attempted to develop a concept of a new PDES synchronization method of wireless networks with non-zero rate of lost frames, e.g. wireless networks. Thanks to treating straggler messages (all or some of them) as messages informing of damaged frames, better performance of distributed simulation was expected [18], however, failed to achieve the required accuracy of the simulation for sequential simulation. In the next sections the new method of time stepped synchronization for wireless networks in described. The developed simulation tool and the simple model and scenario is presented. Next, obtained simulation results are presented to proof the accuracy of parallel simulation in relation to sequence version and evaluation of communication overhead of synchronization are described. At the end of the paper we summarize the work and present the related future work.

2 New time stepped synchronization method In the proposed synchronization method, as in other synchronization algorithms, simulation scenario is partitioned into a number of LP. Each LP handles events, arranged by increasing timestamps (future events set, FES). The allocation of simulation objects to each LP is defined in the initialization phase. The characteristic feature of the proposed method is to introduce synchronization time steps. During the time step each LP operates as an independent, sequential DES, and barrier synchronization occurs only at the end of each time step. Events created by given LP are directed to internal (local) objects via local future event set (L-FES), and to objects managed by others LP’s via external future event set (E-FES). Each LP has therefore two separate future events sets: L_FES, which stores the events directed to local objects, and E_FES, for events directed to objects outside the LP. During the time step LP processes events from the L_FES, with timestamps of the designated boundaries of the current time step. Events addressed to objects in other LP are stored in E_FES. They are exchanged with other LP’s during the synchronization phase at the end of time step. The obvious advantage of this solution

is the possibility of fully parallel simulation performance by each LP during a given time step. However it’s easy to observe, that delayed events, generated during the time step, but sent after its finish, must cause causality errors . The longer the time step is, the more causality errors occur. Thus the number of causality errors depends on the length of time step. Preliminary studies have shown that for a given scenario, it is possible to calculate time step small enough to obtain no causality errors. This step, however, takes small values in relation to a simulation time limit, which significantly reduces the effectiveness. To increase the efficiency of time-stepped simulation we take advantage from specific character of network transmis sion. Events in network simulation are related to transmission of the frame. In wireless network simulation it is important to take into consideration both beginning and end of frame transmission, as this is the only way to detect frame collisions, interferences and other phenomena typical for this kind of communication. Therefore frame transmission is simulated with two events, related to start of transmission (FRAME_START event) and end of transmission (FRAME_END). Basic consequence of a causality error is start of transmission of the frame, which – in correct simulation – should not be started due to busy channel. As external events are delayed, the LP sending such a erroneous frame discovers the fact the channel is busy at the end of time step, after sending FRAME_START event. Therefore we introduce another type of event, FRAME_CANCEL. If the information about frame cancelation arrived to the destination before it generates the next event, simple cancellation of frame will restore correctness of simulation. Possibility of causality error refers to the optimistic synchronization methods. There are still fundamental differences between them and proposed time -stepped simulation. Optimistic simulation assumes rollback, and operation of restoring state of LP and re-simulation of cancelled time-period. Time stepped simulation does not provide any re-simulations. Cancelling of frame has also only local effects, it does not imply subsequent withdrawal of next events , neither locally nor in other LPs For the method to work properly it is necessary to impose additional boundaries on time, in which causality errors can occur and implement appropriate causality error handling at the level of simulation objects. It is possible to restore the synchronization with the single FRAME_CANCEL event when:

max Tk 

1 min( t FRAME_ END  t FRAME_ START ) , 2

(1)

where Tk is the length of time step k, and tFRAME_START and t FRAME_END means the timestamps of events related to respectively beginning and end of a frame transmission. In other words, the length of a single time step must be less than half the time of minimum frame transmission time in a given scenario. Adoption of the length of time step consistent with (1) ensures that the transmission of the frame, started in time step k will be delivered to the destination objects in step k +2 (or later). Therefore, the causality error may occur in one, specific situation: The object O2 located in LP2 in time step k at time t2 checks the status of the wireless medium, and

states that it can start sending a frame, so it sends a FRAME_START event. On the next time step k+1, the O2 object is receiving a delayed FRAME_START event from simulation object O1 , located in the LP1 . The timestamp of the event t 1 < t 2 . This means a causality error situation (see Fig.1).

LP1 O1

LP2 O2

(C)

(B)

(A)

Frame2 Frame 1

Frame1

Frame2 cancelled Frame1

Frame1 Frame2

Frame2

LP3 O3

Frame1 Frame2 cancelled

Frame1 Frame2

Frame1 START/END

Frame2 START/END Frame2 CANCEL

Frame1 Frame2 cancelled

Causuality error detection

Fig. 1. The causality error in the synchronization method with the time step (A, B, C: subsequent time steps). (A) Object 1 (O 1) detects free channel and send Frame1. O 2 detects free channel, and sent Frame2, (B) Obj2 has detected the causality error, cause channel was not free for Frame2 transmission FRAM E_CANCEL of Frame2 is sent. O 3 has detected both (1 and 2) frames, (C) O 1 and O 3 have received event FRAM E_CANCEL before the FRAM E_END of Frame2 will occur and restore the correct state of objects.

To restore correctness of simulation, object O1 sends FRAME_CANCEL of previously sent event. This event is delivered to all objects which previously received erroneous FRAME_START. If the condition (1) is fulfilled, the FRAME_CANCEL reaches the destination objects: (a) before the FRAME_START event is delivered at time step k+1 or (b) before the FRAME_END is delivered at the time step k+2. In the general case of wireless networks simulations (in which objects receive FRAME_START and then record changes in the wireless medium parameters), this would allow the object to withdraw the state associated with the time errorneus frame without generating further events. This solution involves the need to remember (with an accuracy of one time step) of the state of the object. The objects must be able to withdraw its state before receiving a FRAME_START event (with restoration of counters, timers, collisions indicators etc.). This is clearly an implementation overhead, and is associated with the implementation of object-level simulation and may be supported only slightly by a kernel of simulator. But unlike the optimistic synchronization methods synchronization may be restored by a single FRAME_CANCEL event instead of handling the possible subsequent chain of rollbacks procedures. In this way we achieve a tradeoff between the low efficiency of conservative synchronization and high complexity of implementing the optimistic synchronization.

3 Simulation evaluation For verification of synchronization method presented above, simple pseudo-parallel simulator was implemented. Simple, however realistic model of wireless communication was taken for proofing correctness of simulation.

3.1 Simulation tool The simulation tool was developed to perform the time-stepped DES in time steps using described synchronization. Simulations was done both in sequential and the pseudo-parallel mode. In the pseudo-parallel mode LPs run in a fully independent way, but are called in sequence in the main simulation loop. A simple concept is as follows: do { foreach (LP lp in LPset) { lp.step(stepSize); simTime = simTime + stepSize; } } while (simTime < simulationTimeLimit);

This approach allows us for convenient analysis and debugging in both sequential and parallel way and lets verify the results step by step. The next step was to implement the selected model of the simulation model and determine its parameters to obtain the reference simulation scenario. 3.2 Model of simple wireless network The simplified model of communication in a wireless network was adopted, since the primary objective of the study was to evaluate the methods for synchronization. This model, however, remains consistent with the Distributed Coordination Function (DCF) of the IEEE 802.11-2007 standard [19], i.e. the most popular setup used in existing wireless networks. Model simulates the DCF collision avoidance algorithm whereas each network node must ensure it can access the medium before transmission starts. Once started, transmission cannot be interrupted (eg. by interfering signal) until frame transmission finishes. It is even not possible to tell if any such attempt s was made. This is consistent with radio devices found on the market (eg. Atheros chipset ). A few mechanisms modeled according the standard are used: carrier sensing (CS), NAV timers (NAV), inter-frame spacing (IFS) and exponential back -off (EBO). The IFS facility takes into account if the most recently received frame was successful (see

9.2.3 in [19]). For EBO, a simplified formula (2) for the contention window (CW) is used:

CW (r )  min( 2 3r  1, 1023 ) .

(2)

In the above formula, r is the number of retries for a given frame. Nodes only exchange frames without using acknowledgements (ACK), what reflects a multicast trans mission. In addition, the new IEEE 802.11n standard [20] introduces programmable support for disabling this facility. The RTS/CTS mechanism is not used – it has been poorly adopted by the ISP industry due to its performance and is rarely used nowadays. Topology model includes n hosts, acting as client-server pairs (n/2 pairs). The client requests a file transfer from a dedicated server and the server responds by sending data. At the application layer a protocol similar to the TFTP protocol [21] is used. UDP datagrams are used at the transport layer and IP packets at the network layer. Each TFTP data unit (TFTP-DU) is acknowledged by the customer using a TFTP acknowledged datagram (TFTP-ACK). At the level of network layer we have two types of frames: TFTP-DU frames (sized 1518 bytes = 1452 bytes of useful data + related protocols headers) and TFTPACK frames (sized 68 bytes). Client objects send requests at random time intervals with uniform distribution (0, 1) s. The transmission rate is set to 1Mbps. The dis tance between any two hosts on the network is fixed (constant delay between objects is assumed). 3.3 Synchronization and causality errors handling The experiments adopt a model in which there are two lengths of frame. The duration of the shortest frame (TFTP acknowledged datagram) in accordance with the assumed rate of transmission is 544µs. The maximum length of time step, respecting (1) was set to 200us. If the simulation object detects a causality error, sends a FRAME_CANCEL event and restores its state (counter of sending attempts, backoff timer, the status of last transmission, and relevant statistics). The object receiving FRAME_CANCEL also restores its state (statistics and the status of last transmission). 3.4 Parallel simulation accuracy The results correctness was a necessary condition for the start of testing for further evaluation of the synchronization method. As an example of a graph of average servers rate as a function of number of clientserver pairs is presented (see Fig.2). The parallel version used 10 LPs. Simulation objects are partitioned equally between the LPs, with the principle that a client to work on another LP than dedicated server.

The evaluation of obtained results confirmed consistency of results with those obtained in a sequential manner. Minor deviations results from difficulty in withdrawing multiple randomly generated values. In terms of statistical correctness, however, the results should be regarded as compatible.

Fig.2. Average servers rate as a function of the number of client-server pairs.

3.5 Synchronization’s overhead analysis Quantitative comparison between novel time-stepped algorithm and classical synchronization methods is hard, as the synchronization model in time stepped, NullMessages and Time Warp synchronization algorithms is completely different. Overhead of Jefferson’s Time warp [11] is tightly connected with rollback procedures. While normal operation, with no rollbacks, no administrative messages are sent over the network. Comparing to sequential simulation, Time Warp controlled LP must do network I/O, state savings and Global Virtual Time computations. Overhead of these operations is noticeable, but not high. However, when causality error occurs in given LP, simulation on the LP must be stopped. The rollback operation, based on identifying and copying of proper LP state, is performed. This is memory operation, so its overhead is impossible to compare with network overhead coming from sending of network messages. After doing local state restoration some messages, already sent to other LPs must be invalidated. This is done with so called anti-messages, which can cause further rollbacks on LPs receiving them. The bigger the difference between the simulated time value before and after rollback, the more anti-messages must be sent. Thus the efficiency of Time Warp is very strongly dependent on frequency and depth of rollbacks. Parallel simulation with use of Null Messages is considered less complicated for implementation, but introducing bigger overheads. Th ey come from network communication delays as well as from the algorithm itself (synchronization overhead). Number of messages sent over the network is much bigger than in case of Time Warp. After processing of each message the node computes its Earliest Ou tput Time (EOT), which is lower bound on the timestamp of any message that LP may send. Value of EOT is then sent to every LP the sender has the link with. Thus a lot of

messages is exchanged over the network, increasing communication overhead and their exact number depends strongly on number of links between LPs in the particular simulation scenario. Synchronization overhead is also hard to evaluate, as every LP waits for receiving of all messages needed for computation of time bounds allowing safe processing of events in FES. Fig. 3 presents the number of events exchanged in the model, number of events exchanged between LPs (not all event messages leave the LP) and number of FRAME_CANCEL messages in exemplary scenario, engaging 10 LPs, and with number of s imulated network nodes varying from 15 to 50.

Fig.3. Number of messages exchanged in the parallel simulator. Number of LPs: 10, number of object pairs varying from 15 to 50.

Communication and simulation model used in time stepped simulation described in the paper is even more different. The number of network messages exchanged between LPs does not depend on the number of events. Contrary, network message is broadcasted to every other LP at the end of time step. As the event messages are combined into one, the more events are exchanged, the longer the messages are, but their number remain constant (at the application level – we do not consider packet splitting due to preserving MTU). Performance problems arise, as all LP work synchronously and communicate at the same time. As we show above, efficiency comparison of different synchronization algorithms is not possible, as the number of factors influencing performance of parallel simulation controlled the algorithms is too large. One could however expect, that constant number of messages exchanged while using time-stepped method will cause that the method will be particularly effective for big networks simulations. If LP handle a few hundred or a few thousand network nodes, each communicating with other nodes, the number of single events exchanged in every time step becomes huge. Aggregation of communication may significantly increase performance of parallel simulation.

4 Future works

As the next steps the implementation in a fully parallel architecture will be provided. It will allow the practical evaluation of the efficiency, as well as evaluating scenarios with a greater number of nodes. The further developing of a method, implementing adoptable time step length, instead setting it arbitrarily at the beginning of the simulation is planned. Time step will be selected dynamically, based on the length of the currently transmitted frames. An important element of the future work will be to develop and simulate more complex models of the physical layer, taking into account the modulation (the transmission of symbols instead of bits), radio propagation models , full interference model and MIMO.

5 Summary

The paper proposes a new method of parallel discreet event simulation synchronization. In relatively big simulation scenarios it reduces the communication overhead required for synchronization of parallel processes, leading to a potential increase in efficiency of parallel simulation. The solution has been evaluated on the base of a simple model of wireless network. The correctness of the method in comparison to the sequential simulation has been proved. The disadvantage of the solution is an additional effort of the implementation and the need for resolving the causality errors at the level of the model. This implementation overhead can be significant depending on the complexity of the model. With the assumptions adopted in the synchronization method the casuals errors related to frames transmission can be resolved in the next synchronization step (one frame “cancelling” step).

Acknowledgments The work presented in the paper was supported by Polish Ministry of Science and Higher Education grant no. N N516 407138

References 1. The network simulator, ns-2, http://www.isi.edu/nsnam/ns/ 2. GTNstS homepage http://www.ece.gatech.edu/research/labs/M ANIACS/GTNetS/ 3. OM NeT++ homepage: http://inet.omnetpp.org/

4. 5. 6. 7.

M obility Framework homepage, http://mobility-fw.sourceforge.net/ OM NeT++/INET homepage: http://inet.omnetpp.org/ M iXiM homepage, http://mixim.sourceforge.net Ben Hamida E., Chelius G., Gorce J.M .: Impact of the Physical Layer M odeling on the Accuracy and Scalability of Wireless Network. Simulation 85, 9 574–588 (2009) 8. Kotz, D., Newport C., Gray R.S., Liu J., Yuan Y., Elliott C.: Experimental evaluation of wireless simulation assumption. Proceedings of the 7th ACM Internetional Symposium on M odeling Analysis and Simulation of Wireless and M obile Systems, (2004) 9. Fujimoto, R.: Parallel and Distributed Simulation Sy stems, John Wiley and Sons, Inc, (2000) 10.Chandy, K., M isra, J.: Distributed Simulation: A Case Study in Design and Verification of Distributed Programs. IEEE Transactions on Software Engineering, Vol. 5. No 5., pp.: 440452, (1979) 11.Jefferson D., Sowizral H., Fast concurrent simulation using the Time Warp mechanism, SCS Conf. Distributed Simulation; San Diego, CA; (USA); 24-26 Jan. 1985. pp. 63-69 (1985) 12.Parallel/ Distributed ns homepage, http://www.cc.gatech.edu/computing/compass/pdns 13.Nowak M ., Nowak S.: Parallel simulations with modified INET simulation package, Theoretical and Applied Informatics, Vol. 19, no2, pp.147-156 (2007) 14. Bononi L., Felice M ., D’Angelo G., Bracuto M ., Donatiello L., M oVES: A framework for parallel and distributed simulation of wireless vehicular ad hoc networks, Computer Networks 52, pp.155-179 (2008) 15.Zeng X., Bagrodia R., Gerla M ., 15: A library for Parallel Simulation of Large-scale Wireless Networks, Proceedings of the 12th Workshop on Parallel and Distributed Simulation (PADS'98), pp. 154-161. (1998) 16.Peschlow P., Voss A., M artini P.: Good news for parallel wireless network simulations, Proceedings of the 12th ACM international conference on M odeling, analysis and simulation of wireless and mobile systems doi: 10.1145/1641804.1641828 (2009) 17.Liu W. Chiang Ch., Jha V., Grela M ., Bagrodia R.: Parallel Simulation Environment For M obile Wireless Networks, doi: 10.1.1.47.7806 (1996) 18. Nowak M ., Nowak S.: Synchronisation concept for distributed simulation of networks with packet loss, Studia Informatice, Vol.30, no1, pp 81-89 (2009) 19.Wireless LAN M edium Access Control (M AC) and Physical Layer (PHY) Specifications, IEEE Standard 802, Part 11 (2007) 20.Wireless LAN M edium Access Control (M AC) and Physical Layer (PHY) Specifications, IEEE Standard 802, Part 11, Amendment 5 (2009) 21.Sollins K.: The TFTP protocol (Revision 2), IETF RFC 1350 (1992) 22.Varga A., Sekercioglu A.Y.: Parallel Simulation M ade Easy With OM NeT++, 15Th European Simulation Symposium and Exhibition, Proc. (2003).