Microscopic Simulation Model Calibration and Validation for Freeway Work Zone Network – A Case Study of VISSIM Byungkyu (Brian) Park and Hongtu (Maggie) Qi Abstract – Microscopic simulation models have been widely accepted for evaluation of various transportation system design and traffic operations and management strategies. In that regard, the calibration and validation of simulation model is crucial for appropriate decision making process. This paper presents an application of previously developed microscopic simulation model calibration and validation procedure for a freeway work zone network. Multiple days of field data were collected to consider variability and these days were divided into calibration and validation datasets. The study results showed that the procedure was effective in the calibration and validation of a freeway work zone network.
in a work zone, are quite different from those of urban arterial networks. In addition, car following logic used in a freeway network is different from that of arterial network. Thus, the objective of this study is to confirm the validity of previously developed microscopic simulation model calibration and validation procedure for a freeway network. MICROSCOPIC SIMULATION MODEL CALIBRATION AND VALIDATION PROCEDURE This section describes previously developed microscopic simulation model calibration and validation procedure. As shown in Figure 1, the procedure consists of 8 steps with several feedback routines.
Index Terms – Microscopic Simulation Model, Calibration and Validation, Genetic Algorithm Experimental Design, Optimization and VISSIM INTRODUCTION In recent years microscopic simulation models have played an important role in the evaluation of various transportation system analysis, traffic operational strategies and ITS alternative prioritizations. Simulation models are especially useful when strategies under consideration require new constructions and/or costly investments. However, the key to successful evaluations relies on the validity of microscopic simulation model. As such, a few studies demonstrated potential benefits of conducting calibration and validation of microscopic simulation models (Kim and Rilett, 2004, Lee et al., 2001). A recent study (Park and Qi, 2005) developed a systematic procedure for microscopic simulation model calibration and validation. The procedure was successfully applied to several case studies including an actuated isolated signalized intersection, an urban arterial network, etc (Park and Qi, 2005; Park et al., 2006). This paper conducts a case study of simulation model calibration and validation for a freeway work zone network using a microscopic simulation model, VISSIM (2001). Traffic flow characteristics of a freeway network, especially Manuscript received on March 20, 2006 and revised on June 26, 2006. This work was in part supported by the Virginia Transportation Research Council and Federal Highway Administration. Byungkyu (Brian) Park is with the University of Virginia, Charlottesville, VA, 22904 USA (phone: 434-924-6347; fax: 434-982-2951; e-mail:
[email protected]). Hongtu (Maggie) Qi is with BMI-SG: a VHB Company, 8330 Boone Boulevard, Suite 400, Vienna, Virginia 22182, USA (e-mail:
[email protected]).
Figure 1. A Procedure for Microscopic Simulation Model Calibration and Validation (source: Park et al., 2006)
Each step is briefly explained. Simulation Model Setup This step includes study site selection, measure of effectiveness (MOE) determination, data collection and network coding. Initial Evaluation Initial evaluation determines whether default calibration parameters provided by the vendor are adequate to represent field conditions. If they are deemed to be adequate, there is no need to conduct calibration and validation. If not, the following steps should be conducted.
Initial Calibration This step implements the first of step of the calibration procedure which consists of (1) identification of calibration parameters and their acceptable ranges, (2) generation of reasonable number of parameter sets using statistical experimental design, and (3) implementation of multiple runs with each parameter set.
CASE STUDY NETWORK AND DATA As mentioned earlier, the case study network is a freeway segment with a work zone located in the City of Covington, VA. Figure 2 shows schematic layout of the study site; subject direction was from right to left (i.e., 65 mph to 45 mph).
Feasibility Test Feasibility test verifies if the distribution (or histogram) of simulation results generated from the initial calibration step includes field data. If the distribution includes the field data, it is deemed as the acceptable ranges. If not, modifications should be made to alter either the ranges or the list of parameters. Parameter Calibration Using Genetic Algorithm (GA) A genetic algorithm (Goldberg, 1999) optimization program finds an optimal calibration parameter set from the feasible parameter ranges accepted at the feasibility test step. Since VISSIM is a microscopic and stochastic simulation model, a small number of runs (e.g., 5 replications) are conducted for each feasible parameter set to reduce variability. An objective function of the GA (i.e., fitness value) is obtained by comparing field data and simulation output using equation 1.
ObjValue =
TTField − TTSim TTField
Equation (1)
Where, ObjValue = objective function value (or fitness value), TTField = average field-measured travel time, and TTSim = average travel time output from multiple simulation runs with same parameter set. Evaluation of the Parameter Set An optimal solution (i.e., calibration parameter set) obtained from the GA optimization step needs to be evaluated with multiple simulation runs. This is because the solution is tested with only 5 replications, which is often not adequate to fully consider variability. Thus, a distribution of calibrated simulation model outputs based on 100 simulation runs is compared with field data, and then a visualization testing is implemented. Validation and Visualization The final step of the proposed procedure is to confirm that the calibrated parameter set works for an untried condition (i.e., validation). It is desirable to collect a new data set under new conditions (e.g., new traffic signal timing plan, new traffic volume, etc.). Data collected on a different day can be used as a new data set. The calibrated parameter set is deemed to be validated if field data falls in the 95th percentile of the model output distribution. In addition, visualization testing is executed for some selected runs (e.g., 25th, 50th, and 75th percentile runs).
Figure 2. Covington Network (Source: GeoStat Center, UVA)
Traffic Counts Table 1 summarizes traffic data collected from the test site over four normal weekdays. Data on June 10, 2003 was reserved for validation, while the other days were used for calibration. In addition to traffic volume and heavy vehicle (HV) data, compliance rate was collected from the field. The compliance rate was calculated as the percentage of the drivers obeying the posted speed limits according to the collected speed data on each day. Table 1. Summary of Field Traffic Data Compliance Traffic Counts HV% Date Rate (%) (vehicles) 06/10/03* 601 17 30 06/24/03 744 15 18 06/25/03 700 14 23 06/26/03 851 13 30
*Data was collected between 5:10 to 6:00 pm. Travel time Table 2 shows the statistics of travel times including mean, standard deviation, and number of records. As seen in Figure 3, the travel time distributions on different days are quite similar. Table 2. Field Travel Time
Mean Travel Time St. Dev. (sec) 06/10/03* 328.21 19.53 06/24/03 330.04 20.91 06/25/03 332.73 19.26 06/26/03 332.19 21.83 *Data was collected between 5:10 to 6:00 pm Date
Size 574 690 700 790
180
300
160
250
120 100
Frequency
No. of Vehicles
140
80 60 40 20 0 240
280
320
360
400
440
480
Travel Time (sec) 6/10/2003
6/24/2003
200 150 100 50 0
6/25/2003
30
6/26/2003
35
Date 6/10/03 6/24/03 6/25/03 6/26/03
Table 3. Field Travel Speed Summary Mean Speed (mph) St. Dev. Size 51.54 6.84 234* 52.9 5.65 405 52.33 5.76 634 51.42 6.83 695
* Data was collected between 5:10 to 6:00. Table 4. Field Free Flow Speed Summary Mean Free Flow Date St. Dev. Size Speed (mph) 6/10/03 51.67 6.71 109 6/24/03 53.30 5.97 240 6/25/03 52.98 5.96 282 6/26/03 52.12 6.60 272 200 180
No. of Vehicles
160 140 120 100 80 60 40 20 0 20
30
40
50
60
70
80
Speed (mph) 6/10/1003
6/24/2003
6/25/2003
6/26/2003
Figure 4. Distribution of Field Speeds
90
45
50
55
60
65
70
Speed (mph)
Figure 3. Average Field Travel Time Distribution
Travel speed Table 3 summarizes the statistics of average travel speeds including mean, standard deviation and number of records. The mean speed also shows small variation over four days. Table 4 lists the statistics of vehicle free flow speed, which were filtered from the raw data of four days only when the headway between two continuous vehicles was greater than 4 seconds. The distributions of speed data on different days are shown in Figure 4, while the histogram of free flow speed is shown in Figure 5. For speed data, it should be noted that the distribution of the collected speeds underestimates field conditions as some high speeds in fast lane were not recorded during data collection.
40
Figure 5. Distribution of Field Free Flow Speeds
APPLICATION OF MICROSCOPIC SIMULATION MODEL CALIBRATION AND VALIDATION PROCEDURE This section applies the microscopic simulation model calibration and validation procedure to the Covington Network using VISSIM simulation model. Simulation Model Setup The freeway work zone network was coded into the VISSIM model, and traffic data were inputted to the model. Initial Evaluation In order to test whether the default parameter set was acceptable, 100 replications were made for the case study network coded in VISSIM. The resulting average travel time from VISSIM was 358 seconds with a very tight distribution; while the field’s average travel time was 331 seconds (see Figure 9). Thus, it was deemed that further steps are needed. Initial Calibration Identification of Calibration Parameters The following is the initial set of parameters identified as relevant to the performance of the simulation model and their acceptable ranges. 1) Speed Index 1 (mph): 1-6 (65-70, 62.5-72.5, 60-75, 67.5-72.5, 65-75, 62.5-77.5) 2) Speed Index 2 (mph): 1-6 (55-60, 52.5-62.5, 50-65, 57.5-62.5, 55-65, 52.5-67.5) 3) Speed Index 3 (mph): 1-6 (45-50, 42.5-52.5, 40-55, 47.5-52.5, 45-55, 42.5-57.5) 4) Simulation Resolution: 1-9 5) Waiting time before diffusion (seconds): 30-90 6) Min. Headway (front/rear, meters): 0.1-0.9 7) Max. Deceleration (m/s2): -5.00 ~ -1.00 8) Reduction Rate (meters per 1m/s2): 20-80 9) Accepted Deceleration (m/s2): -3.0 ~ -0.2 10) Number of observed preceding vehicles: 1 – 5 11) Maximum look ahead distance (meters): 200 – 300
CC0: average standstill distance (meters): 1.0 – 2.0 CC1: headway at a certain speed (seconds): 0.5 – 3.0 CC2: longitudinal oscillation (meters): 0 ~ 15.0 CC3: start of deceleration process (seconds): -30.0 – 0 CC4: minimal closing Δv (m/s): -1.0 ~ 0 CC5: minimal opening Δv (m/s): 0.0 ~ 1.0 CC6: ± dv/dx (10-4 rad/s): 0.0 ~ 20.0 CC7: car following activities ±b (m/s2): 0.0 ~ 1.0 CC8: acceleration behavior when starting (m/s2): 1.0 ~ 8.0 21) CC9: acceleration behavior at v ~ 80 km/h (m/s2): 0.5 ~ 3.0
Parameters 1 to 3 set desired speed distributions along the case study network at three locations where three different work zone posted speed limits were present. They were indexed for the convenience of experimental design. For instance, speed index 1 has six options to define the desired speed distribution on the freeway where the posted speed limit is 65 mph. The values of CC0 to CC9, which are related to the freeway traffic flow model, were obtained from the values presented at the one of the Annual VISSIM Users Group Meetings. Experimental Design for Calibration The Latin Hypercube Sampling (McKay et al., 1979) toolbox in MATLAB (1999) was used to generate 200 scenarios using the initial set of parameters and their ranges. Multiple Runs Five random seeded runs were conducted in VISSIM for each of the 200 cases, for a total of 1000 runs. The average travel time was recorded for each of the 1000 runs. The results from the five multiple runs were then averaged to represent each of the 200 parameter sets. Feasibility Test In order to check whether the selected parameter set was able to produce field condition, a feasibility test was conducted. Figure 6 shows the distribution of simulated travel time for 200 cases. Since the field value fell within the acceptable range of the distribution, it indicates that the ranges for current parameters were sufficient. 35 331.65 sec 30
Frequency
25 20 15 10 5 0 325
330
335
340
345
350
355
360
365
Travel Time (sec)
Figure 6. Feasibility Test Result
370
375
380
To identify the critical parameters, each parameter was plotted against travel time from the simulations. Apparent trend was observed for speed index 3, average standstill distance CC0, and headway at a certain speed CC1. An example of this trend is shown in Figure 7. Desired Speed Distr. (45 mph) 380 370 Travel Time (sec)
12) 13) 14) 15) 16) 17) 18) 19) 20)
360 350 340 330 320 0
1
2
3
4
5
6
7
Speed Index3
Figure 7. Relationship between Speed Index 3 and Travel Time
In addition, ANOVA test was used to identify the key parameters (Milton and Arnold 1995). The key parameters are extremely useful when the initial parameter ranges need to be modified. Table 5. ANOVA Results
Speed Index 1 Speed Index 2 Speed Index 3 Simulation Resolution Waiting Time Before Diffusion (sec) Min. Headway (front/rear) Max. Deceleration -1m/s^2 per Distance Accepted Deceleration Observed Vehicles Look Ahead Distance (max) CC0 CC1 CC2 CC3 CC4 CC5 CC6 CC7 CC8 CC9
Significanc e value (p value)
Significant mean differences (Sig. < 0.5)
0.510 0.035 0.000
2-4 2-3, 2-4*, 2-5*, 2-6 All*
SSR betwee n groups 599.5 1643.1 24326.0
0.809
NA
636.3
0.757
NA
78.0
0.672
NA
217.0
0.621
NA
248.6
0.565
1-3, 2-3
478.4
0.142
1-2, 1-3
544.6
0.757
NA
78.0
0.011
1-3*, 2-3
1230.5
0.815 0.016 0.011 0.037 0.335 0.406 0.321 0.022 0.538 0.582
NA 1-5*, 2-5*, 2-5, 4-5 1-2*, 1-3* 1-2, 1-3*, 2-3 1-2 1-3 3-4 1-3, 2-3* NA NA
57.3 1663.2 1242.4 909.2 305.9 252.1 489.0 1056.8 304.0 401.3
* Significant value is less than 0.05
Table 5 shows ANOVA analysis results. Based on the significance value of the F test and sum of squares (SSR) between groups, speed index 3, headway at a certain speed (CC1), and speed index 2 were the most important parameters to travel times. Speed index 3 determines the desired speed distribution on the longest portion of the network with a posted speed limit of 45 mph and speed index 2 determines the speed distribution on the second longest portion of the network with a posted speed limit of 55 mph. CC1 is an important parameter to determine the safety distance between two continuous drivers on the freeway. This has a strong influence on capacity, especially in the case of high volumes.
conditions. The parameter values for each set as well as the average travel time from simulation are listed in Table 6. Table 6. Comparison of Three Parameter Sets
Convergence of Fitness Value with Generation 0.04 0.04 0.03
Fitness
0.03 0.02 0.02 0.01 0.01 2
4
6
8
10
Number of generations Best
GA-based 5 (65.0-75.0 mph) 2 (52.5-62.5 mph) 4 (47.5-52.5 mph)
Speed Index 1
62.5-67.5 mph
65-70 mph
Speed Index 2
52.5-57.5 mph
55-60 mph
Speed Index 3
42.5-47.5 mph
45-50 mph
5
10
9
60.00
60.00
68.2
0.50
0.50
0.72
-3.00
-3.00
-1.04
50.00
50.00
60
-1.00
-1.00
-0.7
2
4
3
250.00
220.00
257.58
1.50 0.90 4.00 -8.00 -0.35 0.35 11.44 0.25 3.50 1.50
1.80 2.23 4.00 -8.00 -0.35 0.35 11.44 0.25 3.50 1.50
1.74 2.77 4.09 -0.91 -0.97 0.86 10.7 0.67 2.06 2.77
358.2
343.4
331.9
As seen in Figure 9, the calibrated VISSIM model (GA-based parameter set) provided simulation results similar to the field data while the un-calibrated VISSIM models (default parameters and best-guessed parameters) generated higher travel time. Animations of the calibrated VISSIM were viewed and deemed acceptable.
Ave.
Convergence of Fitness Value with Generations 0.06 0.05 0.04
Fitness
Best-guessed
Simulation Resolution Waiting Time Before Diffusion (sec) Min. Headway (front/rear) Max. Deceleration -1m/s^2 per Distance Accepted Deceleration Observed Vehicles Look Ahead Distance (max) CC0 CC1 CC2 CC3 CC4 CC5 CC6 CC7 CC8 CC9 Ave. Travel Time (sec)
Parameter Calibration Using Genetic Algorithm With the parameters and acceptable ranges identified in the previous step, GA was integrated with VISSIM to find the optimal parameter set for the Covington network. GA procedure of 10 generations and 20 populations was repeated twice with different starting random seeds. The convergence of fitness values with generations is shown in Figure 8. It seems that GA can quickly converge to the optimal solution at the beginning generations. The parameter set with the best fitness value was selected in the final evaluation.
0.00 -0.01 0
Default
160
0.03
140
0.02
120
0 0
1
2
3
4
5
6
7
8
9
10
-0.01
Number of generations Best
Ave.
Figure 8. Convergence GA (Two Trials)
Frequency
0.01
Field 2: 332.19 Field 3: 332.73
100 80 60
Field 1:330.04
40 20 0
Evaluation of the Parameter Sets This section presents the comparison of 100 VISSIM simulation results based on default parameters, best-guessed parameters, and GA-based parameters. The values of best-guessed parameters were determined based on engineering judgment and knowledge of local traffic
325
330
335
340
345
350
355
360
365
Travel Time (sec) Default
Best-guess
GA-based
Figure 9. Comparison of Three Parameter Sets with Field Travel Time Data
Validation Traffic data collected on a different day (June 10, 2003) was used for validation of parameter sets obtained from the calibration process. The field travel time was compared to the distributions of one hundred runs using three parameter sets. Field data was a bit outside of the distributions of the calibrated parameters, but much closer to the distribution of GA-based parameter than those of best-guessed and un-calibrated parameters, as shown in Figure 10.
Goldberg, D. E. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley Publishing Co., Inc., Reading, MA, 1989. Kim, K. O. and L. R. Rilett, A Genetic Algorithm Based Approach to Traffic Micro-simulation Calibration Using ITS Data, 83rd Annual Meeting Preprint CD-ROM, Transportation Research Board, Washington, DC, 2004. Lee, D. H., Y. Xu, and P. Chandrasekar, Parameter Calibration for PARAMICS Using Genetic Algorithm, 80th Annual Meeting Preprint CD-ROM, Transportation Research Board, Washington, DC, 2001.
60 50
Frequency
40
MATLAB Users’ Manual. The Mathworks, Inc., Massachusetts, 1999.
30 20
328.21sec
10 0 325
330
335
340
345
350
355
360
365
Travel Time (sec) Default
Best-guess
GA-based
Figure 10. Validation of Three Parameter Sets with Field Travel Time Data
CONCLUSIONS AND FUTURE RESEARCH This paper applied a previously developed microscopic simulation model calibration and validation procedure to a freeway work zone network. The performance of the procedure was tested by comparing distribution of simulation outputs and field travel time data. The results based on the case study of a freeway work zone network indicate that default and best-guessed parameters of VISSIM were not able to replicate field travel time, while the calibrated parameter set can. Thus, the validity of the procedure is proven for a freeway network. The validation of the calibrated network was conducted by comparing field travel time collected from a different day. However, it would be desirable to validate the calibrated network with a data set collected under different control (e.g., new posted speed limit). In addition, since the procedure was tested with VISSIM model, it is recommended that other microscopic simulation models such as CORSIM (1997), PARAMICS (2002), and AIMSUN (2002) be tested as well. REFERENCES AIMSUN User Manual, Version 4.1, TSS-Transportation Simulation System, Barcelona, Spain, 2002 CORSIM User’s Manual. FHWA, U.S. Department of Transportation, Office of Safety and Traffic Operations, McLean, VA, 1997.
McKay, M. D. and R. J. Beckman, A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code. Technometrics, Vol. 21, No. 2, 1979, pp. 239-245. Milton, J. S. and J. C. Arnold, Introduction to Probability and Statistics, 3rd edition, McGraw-Hill, 1995 PARAMICS Modeler Version 3.0 User Guide and Reference Manual, Quadstone Limited, Edinburgh, U.K., 2002 Park, B., J. Won, and I. Yun, Application of Microscopic Simulation Model Calibration and Validation Procedure: A Case Study of Coordinated Actuated Signal System, Accepted for publication in the Journal of the Transportation Research Board, 2006. Park, B. and H. Qi, Development and Evaluation of
Simulation Model Calibration Procedure, Journal of the Transportation Research Board 1934, TRB, National Research Council, Washington, D.C., 2005, pp. 208 – 217. VISSIM Version 3.6 Manual, PTV Planug Transport Verkehr AG. Innovative Transportation Concepts LLC, Karlsruhe, Germany, December 2001.
