converting the widely-used VISSIM simulator for use in ad hoc distributed simulation .... systems that are based solely on historical data, LEAD is designed to be highly responsive to ...... Available from: http://www.nawgits.com/icdn/vii_trb04.html. ... Innamaa, S. and I. Kosonen, Online Traffic Models - A Learning Experience.
Ad Hoc Distributed Simulations Richard Fujimoto, Michael Hunter, Jason Sirichoke, Mahesh Palekar, Hoe Kyoung Kim, Wonho Suh Georgia Institute of Technology Abstract An ad hoc distributed simulation is a collection of autonomous on-line simulations, each modeling some portion of a larger physical system, that are brought together through a mobile, wireless network to predict future states of the overall system. Unlike conventional distributed simulations that are designed in a largely top-down fashion by partitioning the physical system and mapping each element to a logical process, ad hoc distributed simulations are constructed bottom-up, resulting in multiple simulators modeling common, overlapping portions of the physical system. Ad hoc distributed simulations combine elements of conventional distributed simulations and replicated trials, and raise new issues concerning data distribution and synchronization. The ad hoc simulation approach is proposed, as well as an optimistic synchronization protocol designed for use in these systems. An ad hoc distributed simulation prototype is described that couples collections of in-vehicle simulations intended to manage a transportation network. Experimental results are presented demonstrating that this approach can be effective in predicting future system states when compared to a replicated experiment simulating the full transportation network. Finally, the issue of converting existing commercial simulations to the ad hoc distributed simulation approach is discussed, and experiences converting the widely-used VISSIM simulator for use in ad hoc distributed simulation environments are described. 1. Introduction The last decade has seen dramatic advances in the development and deployment of technologies for sensors, mobile computing, and wireless communication networks. At the same time, use of on-line simulation as a means to better manage and control operational systems has been receiving increased attention. Such on-line simulations are fed by real-time data and rapidly predict future system states, e.g., for use in planning or system management. Applications arise in areas such as manufacturing, supply chain optimization, and transportation system operations, to mention a few. As mobile and ubiquitous computing and communications see increased deployment, embedded on-line simulation will be of growing importance in the years ahead, and potentially could become pervasive in everyday life. As a specific example, consider recent trends in the transportation industry. The Vehicle-Infrastructure Integration (VII) initiative by government agencies and private companies will deploy a variety of roadside and mobile (in-vehicle) sensing platforms capable of collecting, processing, and transmitting transportation data [15]. This infrastructure will create a new platform for deploying many on-line simulation applications for purposes such as collision avoidance [6], traffic prediction [7-10], route planning [11], traffic management [1215], and signal timing [16], among others. For example, one can envision in-vehicle simulations predicting areas of congestion based on real-time data from roadside sensor networks combined with historical data that characterize “typical” traveler demand during that time of day. These traffic forecasts could then be used to plan a driver’s route home that evening. These on-line simulations will have the ability to automatically revise their forecasts and recommendations as unexpected events such as crashes occur. Today, it is common for government and industry to sell or freely distribute mobile computing software for purposes such as reporting current traffic conditions. Widespread distribution of predictive in-vehicle simulation software is likely in the not too distant future. If one posits that in-vehicle simulations will see widespread deployment, it raises the interesting possibility of linking these simulations to create large-scale, mobile distributed simulations that are deployed on our nation’s roads and highways. Here, we focus attention on such systems, and explore how they might be constructed and what are the potential benefits of deploying such systems.
Individually, each simulation only models a portion of the traffic network – that which is of immediate interest to the “owner” of the simulator. Collectively, these simulations could be used to create a kind of distributed simulation system with the ability to make forecasts concerning the entire transportation infrastructure as a whole. For example, one could envision a collection of in-vehicle simulations, each publishing projections of future system states, and utilizing projected state information from other simulators, real-time embedded traffic sensor data, and historic traffic behavior patterns. One can envision combining invehicle simulators with simulations running within the roadside infrastructure, e.g., within traffic signal controller cabinets, and simulations running in traffic management centers to create a large-scale model of a city’s transportation system. We term a collection of autonomous, interacting simulations tied together in this fashion an ad hoc distributed simulation. Like a conventional distributed simulation, each simulator within an ad hoc distributed simulation models a portion of the overall system, and simulators exchange time stamped state information to collectively create a model of the overall system. However, in a conventional distributed simulation the system being modeled is designed in a top-down fashion. Specifically, the system is neatly partitioned into nonoverlapping elements, e.g., geographic regions, and a logical process is assigned to model each element. By contrast, an ad hoc distributed simulation is created in a bottom-up fashion with no overarching intelligence governing the partitioning and mapping of the system to logical processes. Rather, the distributed simulation is constructed in an “ad hoc” fashion, in much the same way an arbitrary collection of mobile radios join together to form an ad hoc wireless network. The elements of the physical system modeled by different simulators in an ad hoc distributed simulation may overlap, leading to possibly many duplicate models of portions of the system. Other parts of the system may not be modeled at all. For example, an in-vehicle transportation simulator may be only modeling the portion of the road network along the vehicle’s intended path to reach its destination. Thus, ad hoc distributed simulations differ in important fundamental ways from conventional distributed simulations. Ad hoc distributed simulations are on-line simulation programs, meaning they are able to capture the current state of the system through measurement, and then execute forward as rapidly as possible to project a future state of the system. By assumption, each of the simulators making up the distributed simulation can simulate some portion of the system faster than real time. Ad hoc distributed simulations raise a number of intriguing questions. Can such a distributed simulation make sufficiently accurate predictions of future system states to be useful? Can they incorporate new information and revise projections more rapidly and/or effectively than conventional approaches, e.g., global, centralized simulations? Does networking the individual simulators result in better predictions than simply leaving the individual simulators as independent simulations that do not exchange future state information? How would an ad hoc distributed simulation be organized and operate, and what type of synchronization mechanism is required among them? These are some of the questions we begin to address here. The remainder of this paper is organized as follows. The next section describes more precisely what we mean by the term ad hoc distributed simulation. Related work is described next. Section 4 describes the systems we envision, and presents a proposal for an optimistic synchronization mechanism for ad hoc distributed simulations. The remainder of the paper focuses on a specific domain, ad hoc distributed simulations for transportation system management, focusing on in-vehicle simulators utilizing roadside computing infrastructure. Section 5 describes a cellular automata simulator that was developed to explore the notion of ad hoc distributed simulations. Section 6 describes results from a set of experiments using this ad hoc distributed simulator. Section 7 describes experiences in adapting VISSIM a widely used commercial transportation simulator for use in ad hoc simulation environments, and presents preliminary results from experiments in this regard. Finally, section 8 presents conclusions and outlines directions for future investigation. While much of the work discussed here focuses on transportation system applications, we conjecture that the ad hoc simulation concept can also be applied to other domains. 2. Ad Hoc Distributed Simulations We define an ad hoc distributed simulation informally as a collection of concurrently executing logical processes (LPs) that exchange time stamped state information and have the following properties:
2
Each LP models some portion of a common physical system; the portion modeled by a single LP can vary over time, and in general, the coverage of the set of LPs participating in the simulation cannot be known or predicted in advance. The portion of the physical system modeled by distinct LPs can overlap partially, completely, or not at all; in the case of multiple LPs modeling a common element, some means is required to aggregate and de-conflict state information among the different LPs, as will be described later. The set of LPs making up the simulation is not known prior to the beginning of the execution, and new LPs can join and existing LPs may leave during the execution. Each LP is an on-line simulation in the sense that it can obtain state information from its surrounding environment, e.g., from embedded sensors, and perform its simulation computation faster than real time in order to compute future system states of the portion of the physical system that it is modeling. LPs may use different models of the system, e.g., they may use different representation of the system, or could model the system at different levels of resolution.
An ad hoc distributed simulation is a hybrid that has properties that are similar in some respects to conventional distributed simulations and in other respects are similar to replicated trials of independent simulation runs. The similarities and differences with conventional distributed simulations have been described earlier. An ad hoc distributed simulation is similar to a replicated trials experiment in that there are multiple simulation computations modeling the same portion of the physical system over time, each computing results that, as will be seen later, must be aggregated together in some fashion. Unlike conventional replicated trials, each LP models only a portion of the physical system, and the LPs exchange state information during the execution that can affect the behavior of other replicated simulations, i.e., LPs. 3. Related Work On-line simulation has been receiving increased attention recently, specifically in the context of dynamic data driven application system (DDDAS) [17]. One area of study that has naturally benefited from the DDDAS approach due its reliance on real-time data is environmental forecasting. The LEAD project is a forecasting system that focuses on improving the prediction of regional scale weather phenomena by building an infrastructure which allows researchers to dynamically and adaptively respond to weather patterns [18]. Unlike current forecast systems that are based solely on historical data, LEAD is designed to be highly responsive to important events based on current weather conditions. Similar efforts to construct forecasting systems have been seen with applications to make short-term predictions for wildfire behavior. Using atmosphere-fire models far more complex than the current operational tools, a suite of simulations relying on fire-atmosphere feedback and environment data is able to capture the evolution of the shape of the fire as well as its growth and progression [19]. Reactive Observing Systems (ROS) utilize stationary and mobile sensors embedded in the environment and reconfigure the simulation based on collected observations. This hybrid sensing approach becomes necessary in simulations where relevant sensor locations cannot be determined during initialization and might actually change during the course of observation. AMBROSia (Autonomous Model-Based Reactive Observing System) is a framework that has been developed to optimize and control the set of data samples take at any given time by taking into account the objectives and limitations of the simulation [20]. A large portion of AMBROSia is concerned with developing, automatically validating and adapting models of data while relying on a distributed system for locating relevant data. Although the framework was targeted for monitoring marine ecosystems when it was designed, it is possible that it could be used with other applications. Kennedy and Theodoropoulos have proposed the introduction of artificial intelligence to DDDAS [21]. By applying DDDAS to an artificial system, the simulation system can actually influence the physical system, creating a symbiotic simulation between the two. Such a system consists of autonomous and assistant “DDDAS agents” that ultimately use predictions as a basis of action on the observed physical system or to focus on relevant data sources in the system to absorb. Introducing DDDAS into an agent-based simulation allows predictions to be validated in real-time by continually comparing them to data from the physical system.
3
Research in DDDAS has also attempted to address the challenges presented by the system, namely the ability for the simulation to adapt to runtime conditions that arise based on data and resource availability. Many of these efforts have produced solutions which are not only viable to the specific scenario but a wide range of applications as well. One such project, known as COERCE, is a semi-automated transformation process for building simulations which support dynamic modification and refinement to match runtime conditions [22]. COERCE relies on language constructs at design time to create an easily transformable simulation, synthesis of simulation components to achieve a larger simulation objective and transformation tools which steer a simulation towards desired outputs based on external observations. Symbiotic simulations used to create decision support systems for manufacturing applications are described in [23]. Concepts derived from grid computing and the U.S. Department of Defense High Level Architecture [24] are proposed to create an environment to support interoperable distributed simulations for use in on-line applications. A large emphasis in this effort is in interfacing commercial simulation tools to the proposed architecture. 4. Data Distribution and Synchronization We assume the distributed simulation consists of a collection of logical processes denoted LPi … LPk. Here, we adopt a model not unlike federated distributed simulations such as those defined using the High Level Architecture where each LP (federate) generates state updates to modify objects that are visible to other LPs. Like the HLA, we assume a publication-subscription communication mechanism is available to distributed state updates to LPs that have declared an interest in receiving those updates. We assume LPs only communicate through this state update mechanism. One important departure from the HLA is here, there are typically many LPs that “own” and update a specific state variable, whereas in the HLA, an instance of an object attribute can be owned and updated by at most one federate at any given point in time. 4.1. Local and Global State Let Li(t) denote the internal, local state of LPi at simulation time t. Like the HLA, we define a body of common, shared information that is visible to all LPs; this is called the global state G and consists of a collection of object instances G1, G2, … GN. As mentioned earlier, several LPs will be modeling common elements of the physical system, so many LPs may be generating state updates to the same object of the global state. There are several important aspects of the global state that merit further discussion. First, the global state will often be defined at a higher, more aggregated level of abstraction than the local state. For example, the vehicle simulations described later use vehicle flow rate, averaged over a certain period of time, as the global state, while the local (microscopic) model includes specifications of the location of individual vehicles on the roadway. State updates must be translated between these models whenever a simulator provides updates to the global state. Let vij(t) = fi(Li(t)) denote the function that converts the local state of LPi to the global state value vj corresponding to simulation time t for global state object Gj. In general, the value vij(t’) refers to that defined by the most recent update by LPi to Gj prior to or at simulation time t’. The set {vj(t)} refers to the values vij(t) at simulation time t generated by all LPs updating that object. In the following the time value t is omitted when its meaning is clear from the context, or is not important. Second, because multiple simulators produce updates to the global state, a mechanism is required to combine these values to form a composite value. This composite value is the value that is transmitted to LPs that have subscribed to receive state updates to this global object. For example, one approach is to capture the updates by a distribution, or an average value perhaps weighted according to some criteria such as the reliability of the source. The composite function Vj(t) = Cj({vj(t)}) computes the composite value Vj for Gj corresponding to simulation time t. Once the composite state value has been passed to subscribing LPs, this value must be converted to the local state representation for use within the LP. This is accomplished through a decomposition function Si = f’i(Vj) that performs the inverse operation of the composition function. For example, in the traffic simulator discussed later, the local state defines the position of individual vehicles on the roadway. The global state is the vehicle flow rate on each link in the network, which is easily computed
4
from the local state information. A simple average is used to derive the composite flow rate of each link from the updates provided by the different simulators. The decomposition function generates a set of vehicle at randomly selected locations according to a certain pre-specified distribution. A natural approach to implement this mechanism is a client/server architecture, where global state objects and the associated composition functions are implemented at the server. The server would receive and store state updates, compute composite values, and disseminates this information to other LPs that have subscribed to receive this information. An alternative approach is to disseminate all updates to all subscribers, and compute the average value locally at the receiver. This may be expensive in terms of space and communications overhead, however. The former approach is used here; it is envisioned such servers may reside within the computing infrastructure, e.g., residing in traffic signal controller cabinets or roadside kiosks. In practice, a large-scale implementation would use multiple servers with objects distributed among these servers in some fashion. 4.2. Synchronization A synchronization algorithm is needed to coordinate interactions among LPs. Here, we focus on optimistic synchronization mechanisms, and propose an algorithm motivated by that described in [25]. Briefly, the approach used in that work is based on a construct called space-time memory that maintains multiple, time stamped versions of state variables. Space-time memory is addressed with a time stamp in addition to a memory location. It maintains a log of memory reads and writes in order to support an optimistic, Time Warp-like synchronization protocol. If an LP reads a data value that is later invalidated by a subsequent write to the same variable, the LP that read the erroneous value is rolled back, and an anti-message like mechanism is used to retract incorrect writes performed by rolled back computations. Conventional synchronization protocols must be modified to be used in ad hoc distributed simulations. This is because conventional protocols are predicated on the assumption that there is an exact, “correct” value of the information that is transmitted between LPs, namely, the information that would be transmitted in a sequential simulation where all events are processed in time stamp order. Once the “correct” information arrives, an LP must roll back to use this information rather than information that had previously been available. This paradigm is clearly not appropriate in ad hoc distributed simulations. Rather, each value produced by an LP should be treated only as that LP’s estimate of the correct value, and there is no a priori reason to believe this estimate is necessarily better, or worse, than the estimates provided by other LPs. On the other hand, if many LPs provide estimates that are substantially different from the value that previously had been used, then there is reason to believe this newer information is more accurate, which in turn may indicate a rollback is appropriate. This is the essence of the approach used here. A key objective of this approach is that it allow for some LPs to act as “outliers” that are recorded, but do not necessarily redirect the entire distributed simulation. To capture these concepts, each LP defines a Boolean rollback function R that compares a composite global state value with the locally computed composite value, and determines whether or not a rollback is necessary. Specifically, R(fi(Lj),Vj) returns true if a rollback should occur due to a state update Vj, and false otherwise. For example, a rollback function might wait until the composite value differs from a previously used value by some threshold before rollback operations are initiated. The synchronization mechanism used here is based on space-time memory. Two operations are defined on the space-time memory: READ(t, Gi): returns the composite state value for object Gi corresponding to simulation time t. WRITE(t, Gi, v): writes the value v to object Gi at simulation time t. The READ operation simply determines the composite value of the object at the requested simulation time, and leaves a record in a log for the object indicating that the LP read the object’s value at the specified time. The WRITE operation adds the value v to the state of the object at simulation time t, and recomputes the composite values Vj = Cj({vj}) for simulation times t and greater. The rollback function R(fi(Lj),Vj) is computed for each of the LPs that read an earlier version of the composite value, and for those LPs where a true value is returned, the new value is passed to the LP so that it can be rolled back.
5
A rollback operation is generated for each LP where the rollback function R returned true: ROLLBACK(Gi, Vj, t): Indicates the LP should roll back to simulation time t, and use Vj, as the state of Gi, rather than the previously reported value. When an LP rolls back to simulation time t it must restore its state to that at simulation time t, using the value specified in the rollback operation as the composite value to be used by the LP. This value must then be decomposed to create the local state value used by the LP. In addition, the LP must retract any writes that were performed by rolled back computations. This is accomplished by generating anti-writes, the analog to Time Warp anti-messages: ANTI-WRITE(t, Gi, v): Cancel a previous write of value v by the LP to object Gi at simulation time t. An anti-write notifies the sever that the previous write operation is to be deleted from the state record for the object in question. An anti-write with simulation time stamp t causes the recomputation of composite values for the object for simulation times t and greater. The rollback function is again recomputed, and additional rollbacks are generated, as needed. It is perhaps noteworthy that the LP that performed the write will typically not roll back itself. Some LPs may be “outliers” in that they generate predictions away from the majority of other LPs. The synchronization mechanism does not attempt to coerce outliers to conform to previously reported behavior. If a sufficiently large number of LPs produce similar predictions as the “outlier”, the global state will eventually reflect this behavior. If they do not, outliers will eventually be notified that their predictions are inaccurate when they differ with measurement data arriving form the sensor network. Finally, real-time data from the sensor network must periodically be incorporated into the simulation. This is accomplished by simply treating such data as heavily weighted writes to the system state at simulation time “now”. Simulators that were operating on data that is far from what the sensors are actually reporting will be rolled back, and forced to recompute their projections based on this new information. 5. An Ad Hoc Transportation Simulation We now examine a specific ad hoc distributed simulation that was developed to experiment with these concepts. The global state G consists of a collection of state records for the roads in the network. The global state of an individual road segment Gi stored in space-time memory is its flow rate. The average of all estimates offered by different simulators at time t was used as the composite function. Thus the value of a link’s flow rate Gi at a particular simulation time is computed as the average among the estimates offered by the different simulators that have reported state values. Threshold values are used to trigger rollbacks. The rollback function used for the experiments is if (abs(Gi - Li(t))) > threshold then return true else return false. The threshold values are set according to the level of traffic flow; larger errors in flow rate can be tolerated when traffic is light because the exact value has little impact on transit time, but such errors could have a significant effect on transit times under congested traffic conditions. The output metric of primary importance in these experiments is the end-to-end transit time of vehicles traveling along a certain path through the network. The road topography is described using a link-node model. Each road segment corresponds to a link while intersections correspond to nodes. Each road is further divided into cells. A vehicle can occupy a single cell or multiple cells. The size of cell determines the resolution of the variables used in the simulation such as vehicle speed, with smaller cells yielding higher resolution. Each intersection is represented by cells corresponding to the lanes leading into the intersection and a Boolean flag called the stop-go flag that models a traffic signal. A transportation simulation based on a cellular automata model is used as the individual LPs (clients) for these experiments. The simulation consists of agents modeling vehicles, traffic signal controllers and traffic lights. The simulation operates in a timestep fashion. At every timestep, each agent decides what operation to perform next based on its own characteristics and the state of the system at the end of previous interval. Each vehicle agent includes characteristics such as origin, destination, maximum speed of the vehicle, and driver characteristics such as aggressiveness. Each vehicle has complete knowledge of the road topography around it.
6
At the end of each timestep each vehicle agent makes decisions whether to move, stop, accelerate, or decelerate based on the following four rules [26, 27]: • Acceleration: if the velocity v of a vehicle is lower than max velocity and if the distance to the next car ahead is larger than v + gap, the speed is increased: v = v + acceleration. • Slowing down (due to other cars): If a vehicle at site X sees the next vehicle at site X + j (with j < v), it reduces its speed to j: v = j • Randomization: The velocity a vehicle, if greater than zero, is decreased with probability pdec,: v = v deceleration. • Car motion: each vehicle is advanced v cells. Similarly, the traffic controller may also change its state, specifically the stop-go flag, at the end of each time step. A two-lane, two-way, east-west arterial roadway is simulated for these experiments (one intersection is shown in Figure 5.1). The roadway contains ten equally spaced intersections (1320 ft apart) with each intersection approach containing a shared through / right turn lane and a 600 ft left turn bay All intersections operate under a fixed-time 120 sec cycle, with a 20 sec east/west left turn phase, 50 sec east/west through phase, 15 sec north/south left turn phase, and a 35 sec north/south phase. Turn rates are constant at each intersection, with 95% of the vehicles on each approach traveling through the intersection, 2% turning left and the remaining 3% turning right. All vehicles are assumed to have a constant length of 17 ft and a maximum velocity of 45 ft/sec. Location of Traffic Signal. This is placed one cell (ft) into the intersection.
0
Statistics Collection points
1319
Entry point on Road Exit point on Road
. . . . .
Left Turn Bay
Road 7
1319
. . . Road 6 . .
1298
Road 0
0
Exit point on Intersection
Road 5
2 1298 0 0
……………
0
……………
2
7
on
0
0
…..
…..
0
20 ……………
…………… …..
…..
0
Entry point intersection
Road 1
1298
Intersection
Road 2
1298
0 . . . . .
0
Road 4
1319
. . . Road 3 . .
1319 0 Figure 5.1: Visual representation an intersection in the road network with entry and exit points.
7
6. Experiments and Results 6.1. Test Environment The experiments described in this section are performed over a set of heterogeneous, non-dedicated workstations connected through a local area TCP/IP network. An actual deployment would utilize wireless communications, however, due to limitations in the availability of suitable platforms, a more conventional platform was used for these experiments. Twenty-one machines (20 clients, 1 server) are utilized for each experiment. Each client holds a single LP, and space-time memory is stored in the server, as discussed earlier. Workstations are drawn from the pool of networked machines based on availability, stability and current workstation workload. The minimum hardware configuration for each workstation is a multi-processor configuration using Intel Xeon processors and 1 GB of memory, with per-processor speeds ranging from 2.0 – 3.2 GHz and each machine having either a dual or quad processor configuration. Each machine runs a standard version of the Red Hat Enterprise Linux 4 operating system with the 2.6.9-22.0.1 kernel optimized for multiple processors and a SSH service used for remote monitoring. Clients and server roles are assigned to machines at random for each experiment, with each machine assigned a single client as the desired experiment protocol is to emulate each client executing within a separate vehicle. 6.2. Simulation Configuration For these experiments ten clients simulate intersections 0 through 4 and ten clients simulate intersections 5 through 9 (Figure, 6.1). The links between intersections 4 and 5 are modeled by all ten clients. For each client the arrival rate on all exterior roadway approaches are initialized for the given experiment. Six different experiments are undertaken, five with constant arrival rates (100, veh/hr, 200 veh/hr, 300 veh/hr, 400 veh/hr, 500 veh/r) and one with an initial arrival rate of 100 veh/hr on all external links, with the arrival rate on roadway 0 increasing to 500 veh/hr 1000 seconds into the simulation. This last experiment is intended to model a sudden influx of traffic at the western end of the network. The duration of all simulation experiments is one hour of simulated time.
Figure 6.1 Simulated Network Configuration
The twenty clients each send updated predicted flow rates on all internal simulated links to the server every 60 seconds. The 60 second updates are the aggregate flows over the previous 240 seconds. At the initialization of each client a 240 second fill period is completed before rollbacks are allowed. Clients do not send updates to the server during the fill period. Roll backs may only be triggered by operations on links 5 and 16, the roadway links connecting intersections 4 and 5. For example, clients 1 through 10 each provide the server with a prediction of the flow on roadway 5, while clients 11 through 20 each receive arrival flow data for roadway 5 from the server. Should updated arrival rate predictions by clients 1 through 10 at timestamp ti result in a sufficiently large change in the aggregate
8
arrival rate at timestamp ti that was given to any of clients 11 to 20 than the subject client would be required to roll back to time ti and utilize the updated arrival rate. Table 6.1 contains the thresholds utilized to determine if the change in the arrival rate at a timestamp is sufficiently large to warrant a roll back. The current threshold values attempt to balance the accuracy of simulation with the number of roll backs. Aggregate Input Rate Threshold 0 - 200 cars/sec None 200 - 300 cars/sec 100% 300 - 400 cars/sec 75% 400 - 500 cars/sec 50% > 500 cars/sec 25% Table 6.1: Server lookup table for deciding which threshold to use for calculating rollbacks
6.3. Simulating a Steady State Road Network As noted the first set of experiments simulates the road network under a constant input rate during the simulation time period. To allow for an evaluation of the accuracy of the ad hoc distributed simulation a replicated trial is also performed with all 20 clients simulating the entire ten intersection roadway with the given boundary input rates. The average of the roadway flow rates on the twenty replicated runs represents a “ground truth” and is given for comparison with the distributed clients. The primary accuracy metric considered for these experiments is the predicted traffic flows. For the five different input boundary rates (100, 200, 300, 400 and 500 veh/hr) the primary areas of interest are the two “shared” roads (roadways 5 and 16) in the middle of the network and the eastern most exit link (roadway 10). As described above roadway links 5 and 16 provided the primary data to the server that may result in the updating of downstream of clients. Figure 6.2 shows the average arrival rate over time on roadway 5 for the distributed clients and replicate trials under 100 veh/hr, 300 veh/hr, and 500 veh/hr arrival flow rates. 0.2 0.19 0.18 0.17 0.16 0.15 0.14 0.13 0.12 0.11 0.1 0.09 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01
24
0 33 0 42 0 51 0 60 0 69 0 78 0 87 0 96 0 10 50 11 4 12 0 30 13 20 14 10 15 00 15 90 16 8 17 0 70 18 60 19 5 20 0 40 21 30 22 20 23 10 24 00 24 90 25 80 26 70 27 60 28 50 29 40 30 30 31 2 32 0 10 33 00 33 90 34 80 35 70
0
20 client replicated average - 100 cars/hr Clients 11-20 distributed average - 100 cars/hr Clients 1-10 distributed average - 300 cars/hr 20 client replicated average - 500 cars/hr Clients 11-20 distributed average - 500 cars/hr
Clients 1-10 distributed average - 100 cars/hr 20 client replicated average - 300 cars/hr Clients 11-20 distributed average - 300 cars/hr Clients 1-10 distributed average - 500 cars/hr
Figure 6.2: Average arrival rate on roadway link 5 at 100 veh/hr, 300 veh/hr, and 500 veh/hr constant boundary input rate
9
0.2 0.18 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02
11 40 12 90 14 40 15 90 17 40 18 90 20 40 21 90 23 40 24 90 26 40 27 90 29 40 30 90 32 40 33 90 35 40
99 0
84 0
69 0
54 0
39 0
24 0
0
20 client replicated average - 100 cars/hr Sample Distributed Client - 100 cars/hr 10 client distributed average - 300 cars/hr 20 client replicated average - 500 cars/hr Sample Distirubted Client - 500 cars/hr
10 client distributed average - 100 cars/hr 20 client replicated average - 300 cars/hr Sample Distributed Client - 300 cars/hr 10 client distributed average - 500 cars/hr
Figure 6.3: Average arrival rate on roadway link 10 at 100 veh/hr, 300 veh/hr, and 500 veh/hr constant boundary input rate Accuracy
Under the tested boundary conditions and for the thresholds specified in Table 6.1 there were no rollbacks required on any of the distributed clients. This is to be expected because the network is operating in a steady state behavior; rollbacks are intended to correct clients that are performing inaccurate simulations, e.g., because traffic conditions have unexpectedly changed. Clients 1 through 10 and the replicated runs closely match, with both showing similar variation in the predicted flows on roadway link 5 (Figure 6.1). The absence of rollbacks results in clients 11 through 20 modeling no variation on roadway 5, maintaining the initial flow rate throughout the simulation time period. However, a comparison of clients’ 11-20 average predicted flow rate on roadway link 10, the eastern departure link, shows good agreement with the 20 replicated runs (Figure 6.2). The impact of the lack of upstream variability on the performance of clients 11-20 is relatively minor. Thus, for the constant arrival rate the flows generated by the distributed architecture are similar to that of the single large model, given that the arrivals on the distributed clients are a reasonable reflection of the average of the true arrivals. Initialization effects
Two primary initialization effects are captured. The first is that at the start of the simulation (t=0) vehicles are placed on links to provide initial flows on the internal roadway links. The number of vehicles placed on the links does not necessary coincide with the arrival rate for the particular experiment being simulated. Thus, a short period is required for the simulations to reach a steady state. On Figure 6.2 it is seen that roadway link 5 reaches its equilibrium state approximately 600 seconds into the simulation for clients 1 through 10 and the
10
replicated trials. Thus, approximately 600 seconds are required for the upstream arrivals on the simulation to filter downstream to roadway link 5. The second initialization effect captured is the potentially shorter initialization time required for the ad hoc simulation. From Figure 6.3 (arrivals on the east most roadway link) it is seen that the average of the distributed clients and the replicated trials reach a steady state approximately 600 sec and 1250 sec into the simulation, respectively. The is a direct result of the need for clients 11-20 to reach a steady-state traffic pattern over only 5 intersections while the replicated trials must reach a steady state traffic pattern over 10 intersections. 6.4. Simulating a Change in the Input Rate This experiment explores how well the ad hoc distributed simulation captures the effect of changing arrival rates. In this experiment the initial arrival rate for all roads is set to 100 veh/hr. At simulation time 1000 clients 1 through 10 increase the arrival rate on roadway link 0 to 500 veh/hr, representing an upstream increase in the traffic volume. Unlike the steady state scenario from the previous section which had no rollbacks, the ad hoc successfully simulating this increase in traffic demand will require rollbacks as clients 1 through 10 provide updated network arrival information that must be passed to clients 11 through 20. Figure 6.4 depicts the increase in the arrival rate on roadway link 0 at timestamp 1000 seconds, the average of clients’ 1 through 10 traffic flow on roadway link 5, the average of clients’ 11 through 20 traffic flow on roadway link 5, and the average of clients’ 11 through 20 traffic flow on roadway link 10. As with the previous experiments “ground truth” is represented by the average of 20 replicate simulation runs of the entire 10 intersection corridor, under the same input conditions as the distributed simulations. 0.2 0.19 0.18 0.17 0.16 0.15 0.14 0.13 0.12 0.11 0.1 0.09 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0 240
540
840
1140
1440
1740
2040
2340
2640
2940
3240
Client 1-10 distributed average, Road 0
Client 1-10 distributed average, Road 5
Client 11-20 distributed average, Road 5
Client 11-20 distributed average, Road 10
20 client replicated average, Road 5
20 client replicated average,Road 10
3540
Figure 6.4: Average input rates across various roads in the network in response to an increase in boundary input rate of leftmost road
It may be noted that had the clients simulating the eastern portion of the network not been coupled with the clients simulating the western portion, clients 11-20 would not have rolled back and thus would not have predicted the increased traffic flow at all during this simulation cycle. This is because the congested link is outside the area they are modeling. Allowing clients to exchange predicted flow rate information in an ad hoc
11
distributed simulation allowed clients 11-20 to detect this oncoming congestion earlier than they would have otherwise. Propagation delay based on vehicle travel time
The traffic flow increase is propagated downstream according to the travel speed of the vehicles. In the replicated trials the increased arrival demand reaches roadway 5 approximately 600 seconds after the initial roadway link 0 increase and reaches roadway 10 approximately 1,200 seconds after the initial increase. As expected, clients 1 through 10 demonstrate a similar result. Clients 11 through 20, which simulate intersections 5 through 9, are only aware of the change in the arrival rate through the rollback mechanism. However, the server only determines if a traffic threshold is violated and a rollback required every 60 seconds of simulation time. Also, the traffic threshold violation is based on an aggregation of 4 minutes of traffic flow data, possibly resulting in the several 60 second intervals being required before the aggregate traffic flow value has sufficiently increased to violate a threshold. This process results in a delay between the time that clients 1 through 10 may first detect increased flow on link 5 and the time at which the server implements a rollback on any of clients 11 – 20. In figure 6.4 both roadway link 5 and roadway link 10 show the distributed simulation prediction for increased traffic results lagging behind the replicated run results by up to four minutes. Also, the new sustained flow rate of roadway link 5 and roadway link 10 of the distributed simulation seem consistently slightly less than that of distributed simulation. This is potentially an indication that the rollback thresholds for this traffic volume should be stricter. 7. Adapting a Commercial Traffic Simulator While the simulation model developed as part of this effort (section 5) allowed for the development of an understanding of the behavior of the ad hoc distributed approach it is desirable that future experimentation be conducted with a significantly more robust transportation simulation model. Further, the ability to adapt existing commercial simulation software for use in ad hoc distributed simulations would significantly improve the likelihood that the technology would be used. An initial investigation into implementing the ad hoc strategy using the off-the-shelf transportation simulation model VISSIM was conducted. VISSIM is widely used by private firms and public agencies for transportation system analysis. VISSIM is a discrete, stochastic, time step based microscopic simulation model developed to model urban traffic (freeway and arterial) and public transit operations. The model is capable of simulating a diverse set transportation network features, such as facility types, traffic control mechanism, vehicle and driver types, etc. Individual vehicles are modeled in VISSIM using a psycho-physical driver behavior model developed by Wiedemann. The underlying concept of the model is the assumption that a driver can be in one of four driving modes: free driving, approaching, following, or braking [28]. VISSIM version 4.10 was utilized for this investigation. Investigation into the use of VISSIM has proven very hopeful. Utilizing the VISSIM COM interface [29] it is possible to access many of the VISSIM object, methods, and properties necessary to implement simulation roll backs and automate VISSIM clients disbursed among workstations connected by a local area network. The initial investigation discussed in the section focuses on the ability to implement a roll back mechanism in VISSIM. Through the VISSIM COM interface it is possible at anytime ti during a simulation run to save the current state of the simulation model. At any later time tj, where j > i, it is possible to stop the simulation run, load the simulation state from time ti, update the simulation attributes indicated by the roll back (i.e. arrival rate on some link), and restart the simulation from time ti. An experiment was conducted to implement this roll back mechanism by updating a running VISSIM client to reflect data based on “real world” sensors. This initial experiment is conducted on a single workstation that handles the VISSIM client, server application, and emulates “real world” real-time data flow. As with the previous experiments the VISSIM client provides updated predicted traffic flows to the server at a set interval. The server also receives a “real-time” traffic flow generated by a second VISSIM model, emulating traffic data input similar to what would be received from sensors embedded in the field. When the server identifies that the predicted VISSIM client traffic flow at time ti differs from the real-world data at time ti by a pre-specified threshold the VISSIM client simulation is rolled back to ti-180, updated with the new traffic flow rate, and the client simulation is restarted from time ti-180. This is slightly different roll back implementation than presented in
12
the previous experiments. The rationale for the client implementing a roll back to ti - 180 is based on previous experimental results exposing a delay in the clients implementing the updated arrival rates. By implementing a roll back to an earlier timestamp (i.e. 3 minutes early, the utilized aggregation interval) than the timestamp of the threshold violation (i.e., ti) the client is able to have a fill period that allows for an improved simulation at ti. However, as part of this implement it is necessary that roll backs not be allowed for the period ti-180 to ti, otherwise cascading roll backs to the beginning of the simulation time period may occur. Figure 7.1 represents the network utilized for this experiment. The modeled area is from an actual urban region, representing an arterial roadway network bisected by a major freeway. This area is selected as the link from K to L represents a clear potential critical link in the network that may be studied, where all traffic desiring to travel from east to west side or west to east on the network must cross this link (i.e., the bridge over the freeway). Traffic demand enters the network from points A through J and is set to represent uncongested conditions. For this experiment two hours of time are simulated. The real-world traffic demand is constant during this time at all entry points except point A, where the traffic flow increases from a rate of 100 veh/hr to 600 veh/hr 1800 seconds into the two hour time period. The average entering initial traffic flows at points A through J are given to the VISSIM client at the start of the simulation run. The VISSIM client is dependent upon the roll back mechanism to successfully model the traffic flow increase at point A.
Figure 7.1 Urban Area Network Roll backs are currently only triggered by traffic flow at point A. Every 15 seconds of simulated time the VISSIM client sends its predicted flow at point A to the server. The server also receives the “real-world” traffic flow data every 15 second of real time. Both the real world and VISSIM client 15 second traffic flow data updates are based on a three minute average of the traffic flow. At each 15 seconds of real time the server compares the real time data to that predicted by the VISSIM client, if the pre-specified threshold has been violated a roll back occurs. The VISSIM client simulation clock time runs 20 times faster than the real time clock and thus is guaranteed have a prediction for the current real time. Figures 7.2 and 7.3 show the real-world traffic flow and modeled VISSIM client traffic flow at Point A and link K to L, respectively. The VISSIM client
13
Veh / hr
data is the average of ten replicated trials of the experiment. Through the roll back mechanism the VISSIM client is able to reasonably track the real-world data at point A and the traffic flow through the critical link K to L. Just as important as the simulation results this experiment has also shown that, at least for this initial level of experimentation, it is possible to implement the ad hoc distributed architecture in the off the shelf transportation simulation model VISSIM. 1000 900 800 700 600 500 400 300 200 100 0 0
1000
2000
3000
4000
Real World Three Minute Average
5000
6000
7000
10 Replicated Clients Three Minute Average
8000 Time
Veh / hr
Figure 7.2 Real World and VISSIM Replicated Client Traffic Flow At Point A
1200
1000
800
600
400
200
0 0
1000
2000
3000
4000
Real World Three Minute Avgerage
5000
6000
7000
10 Replicated Clients Three Minute Average
Figure 7.3 Real World and VISSIM Replicated Client Traffic Flow For Link K to L
14
8000 Time
8. Conclusions and Future Research With the widespread availability of distributed mobile and computing software and the strong likelihood of predictive in-vehicle simulation software in the near future, vast opportunities exist for novel approaches to utilize these resources in the planning, monitoring, and operation of our transportation system. In our approach, we envision individual vehicles, roadside base stations, and embedded system sensors contributing to a large scale ad hoc distributed simulation of the transportation network. This collection of autonomous, interacting simulations will enable the creation of an ad hoc distributed simulation modeling the system in a bottom-up fashion, with no central governance assigning regions to the individual simulations. We have conducted initial experiments in the management of 20 client simulations over a 10 intersection corridor. Under steady conditions, the distributed simulation client provided a system representation similar to that of replicated trials. When a spike in the traffic flow was introduced into the network, the distributed clients again successfully modeled the traffic flows; however there was a short delay (up to approximately four minutes) in the upstream client transmitting the increased flows to the downstream clients. The simulation used was a cellular automata model specifically developed for these experiments. Experimentation on more complex transportation systems will require the use of a more robust transportation simulation. Thus, an initial experiment was conducted using the off-the-shelf transportation microscopic simulation VISSIM. In this experiment the roll back algorithm was successfully implemented in VISSIM. Future efforts will continue to explore the nature of the proposed ad hoc simulation architecture. The VISSIM approach will be expanded to a cluster of workstations connected over a local area network. This will allow for the testing of the ad hoc approach over a more complex network, utilizing additional client simulations dispersed over regions of the network. Future experiments will seek to investigate many of the modeling aspects of the ad hoc simulation, such as the calibration of thresholds, the incorporation of real-time data, the impact of dispersed clients, estimating operations on simulation regions not covered by a client, and the entering and exiting of clients during a simulation. Future experiments will also further address the potential merit of the proposed ad hoc approach by gauging the accuracy of the predicted future system states, considering other approaches to organizing the distributed simulations, and comparing the ad hoc performance (accuracy, efficiency, etc.) to that of a single large scale model. Finally, future efforts will be aimed at not only seeking novel methods to harness the technology that is rapidly entering the transportation system, but to also guide the development of that technology and help shape its functionality. 9. References 1. Werner, J. Details of the VII Initiative 'Work in Progress' Provided at Public Meeting. 2005 [cited 2005 April 15]; Available from: http://www.ntoctalks.com/icdn/vii_pubmtg_v1.php. 2. Bechler, M., W.J. Franz, and L. Wolf. Mobile Internet Access in FleetNet. in KiVS 2003. 2003. 3. Werner, J. USDOT Outlines the New VII Initiative at the 2004 TRB Annual Meeting. 2004 [cited 2005 April 12]; Available from: http://www.nawgits.com/icdn/vii_trb04.html. 4. Wilson, C. The Merits of Vehicle-to-Infrasturcture Communication for Intersection Safety. Newsletter of the ITS Cooperative Deployment Network 2004 [cited 2005 April 12]; Available from: http://www.nawgits.com/icdn/wilson_itespring04.html. 5. Werner, J. More Details Emerge about the VII Effort. Newsletter of the ITS Cooperative Deployment Network 2004 [cited 2005 April 12]; Available from: http://www.ntoctalks.com/icdn/vii_details_itsa04.html. 6. Broadhurst, A., S. Baker, and T. Kanade. Monte Carlo Road Safety Reasoning. in IEEE Intelligent Vehicles Symposium. 2005. 7. Innamaa, S. and I. Kosonen, Online Traffic Models - A Learning Experience. Traffic Engineering & Control, 2004. 45(9): p. pp. 338-343. 8. Kosonen, I. and A. Bargiela. Simulation based traffic information system. in 7th World Congress on Intelligent Transport Systems. 2000. Turin, Italy. 9. Kosonen, I., Multi-Agent Fuzzy Signal Control Based On Real-Time Simulation. Transportation Research. Part C: Emerging Technologies, 2003 11(5): p. pp 389-403. 10. Chu, L. and W.W. Recker, Micro-Simulation Modeling Approach To Applications Of On-Line Simulation And Data Fusion. 2004 University of California, Berkeley. Institute of Transportation Studies
15
11. Wang, H.L. and R.L. Cheu, Dynamic Routing Decisions For Commercial Vehicle Operations In Real-Time Traffic Conditions. Transportation Research Record, 2004(1882): p. 201-209. 12. Spelberg, R.L., H. Toetenel, and R. Vermeijs. Freeway Traffic Congestion Management Through Real-Time Simulation. in IASTED International Conference: Modelling and Simulation. 1995 Pittsburgh, Pa. 13. Katwijk, R.T.v., et al. A Test Bed for Multi-Agent Systems and Road Traffic Management. in 84th TRB Annual Meeting. 2005. 14. Mahut, M.F. and N. Tremblay. Space-Time Queues And Dynamic Traffic Assignment: A Model, Algorithm And Applications. in 82nd TRB Annual Meeting. 2003. 15. Hafstein, S., et al., A High-Resolution Cellular Automata Traffic Simulation Model With Application In A Freeway Traffic Information System. Computer-Aided Civil and Infrastructure Engineering, 2004. 19(5): p. pp 338-350. 16. Nishikawa, I., S. Nakazawa, and H. Kita, Area-Wide Control Of Traffic Signals By A Phase Model. Transactions of the Soc of Instrument & Control Engineers, 2003. 39(2): p. 199-208. 17. Darema, F. Dynamic Data Driven Applications Systems: A New Paradigm for Application Simulations and Measurements. in International Conference on Computational Science. 2004. 18. Plale, B., D. Gannon, and D. Reed. Towards Dynamically Adaptive Weather Analysis and Forecasting in LEAD. in International Conference on Computational Science. 2005. 19. Mandel, J., L.S. Bennethum, and M. Chen. Towards a Dynamic Data Driven Application System for Wildfire Simulation. in International Conference on Computational Science. 2005. 20. Golubchik, L., D. Caron, and A. Das. A Generic Multi-scale Modeling Framework for Reactive Observing Systems: An Overview. in International Conference on Computational Science. 2006. 21. Kennedy, C. and G. Theodoropoulos. Intelligent Management of Data Driven Simulations to Support Model Building in the Social Sciences. in International Conference on Computational Science. 2006. 22. Brogan, D., P. Reynolds, Jr., and R. Bartholet. Semi-Automated Simulation Transformation for DDDAS. in International Conference on Computational Science. 2005. 23. Lendermann, P., et al. An Integrated and Adaptive Decision-Support Framework for High-Tech Manufacturing and Service Networks. in Proceedings of the 2005 Winter Simulation Conference. 2005. 24. Kuhl, F., R. Weatherly, and J. Dahmann, Creating Computer Simulation Systems: An Introduction to the High Level Architecture for Simulation. 1999: Prentice Hall. 25. Fujimoto, R.M., The Virtual Time Machine, in International Symposium on Parallel Algorithms and Architectures. 1989. p. 199-208. 26. Nagel, K. and M. Schreckenberg, A Cellular Automata Model for Freeway Traffic. J. Physique I, 1992. 2: p. 2221-2229. 27. Esser, J. and M. Schreckenberg, Microscopic Simulation of Urban Traffic Based on Cellular Automata. International Journal of Modern Physics C, 1997. 8(5): p. 1025-1036. 28. PTV, VISSIM User Manual 4.10. 2005, PTV Planung Transport Verkehr AG: Karlsruhe, Germany. 29. PTV, VISSIM COM, User Manual for the VISSIM COM Interface, VISSIM 4.10-12. 2006, PTV Planung Transport Verkehr AG: Karlsruhe, Germany.
16
