Health Care Manag Sci (2013) 16:14–26 DOI 10.1007/s10729-012-9206-y
Determining minimum staffing levels during snowstorms using an integrated simulation, regression, and reliability model Amber Kunkel & Laura A. McLay
Received: 9 May 2012 / Accepted: 29 June 2012 / Published online: 25 July 2012 # Springer Science+Business Media, LLC 2012
Abstract Emergency medical services (EMS) provide lifesaving care and hospital transport to patients with severe trauma or medical conditions. Severe weather events, such as snow events, may lead to adverse patient outcomes by increasing call volumes and service times. Adequate staffing levels during such weather events are critical for ensuring that patients receive timely care. To determine staffing levels that depend on weather, we propose a model that uses a discrete event simulation of a reliability model to identify minimum staffing levels that provide timely patient care, with regression used to provide the input parameters. The system is said to be reliable if there is a high degree of confidence that ambulances can immediately respond to a given proportion of patients (e.g., 99 %). Four weather scenarios capture varying levels of snow falling and snow on the ground. An innovative feature of our approach is that we evaluate the mitigating effects of different extrinsic response policies and intrinsic system adaptation. The models use data from Hanover County, Virginia to quantify how snow reduces EMS system reliability and necessitates increasing staffing levels. The model and its analysis can assist in EMS preparedness by providing a methodology to adjust staffing levels during weather events. A key observation is that when it is snowing, intrinsic system adaptation has similar effects on system reliability as one additional ambulance.
A. Kunkel Computational and Applied Mathematics, Rice University, Houston, TX, USA e-mail:
[email protected] L. A. McLay (*) Virginia Commonwealth University, Statistical Sciences and Operations Research, 1015 Floyd Ave, Box 843083, Richmond, VA 23284, USA e-mail:
[email protected]
Keywords Emergency medical services . Regression . Erlang loss models . Discrete event simulation
1 Introduction Emergency medical services (EMS) provide a vital service during severe weather events such as snowstorms. Properly staffed EMS systems prevent death and disability from severe injuries and illnesses by quickly responding to and treating patients. While substantial research exists on how to set staffing levels to meet EMS demand during normal operating conditions [1], extreme weather events such as blizzards cause an increased load on the system by increasing call volumes and lengthening service times [2]. EMS systems in some regions of the United States have few data or experiences on which to set staffing levels during snowstorms if severe snowstorms rarely occur. As a result, adjustments to severe or inclement weather are rarely evidencebased or quantitative [3]. EMS systems may therefore be unprepared for sharp increases in call volume and service times caused by snow, as demonstrated by the December 2010 blizzard that left some patients in New York City waiting up to 30 h for emergency medical service [4]. EMS and emergency medicine policies typically focus on either “normal” operating conditions or disaster conditions [5]. Disaster planning is generally performed for mass casualty events (such as those on September 11, 2001) or catastrophic weather events (such as Hurricane Katrina) [6, 7]. Less planning and training is performed for severe weather events, which are less serious in scope but which occur more frequently than mass casualty events and catastrophes. Moreover, EMS systems outside of urban areas are often overwhelmed by modest increases in demand that occur during severe events [8]. Therefore, the medical literature suggests a need for the type of analysis performed in this paper.
Determining minimum staffing levels
A growing number of papers highlight the importance of estimating demand for EMS services. McLay et al. [2] examine how average call volumes and service times change as a result of snow and other weather variables. They find that when it is snowing, the number of EMS patients increase, ambulances take longer to respond to and treat patients, and patients are less likely to be transported to a hospital. They compute the offered load on the system during severe weather events; however, their analysis is not used to infer what the appropriate staffing levels should be. Several other papers perform a statistical analysis of EMS data to aid in decision-making. Notably, Channouf et al. [9] introduce a novel time series model to forecast daily and hourly EMS call volumes, and Matteson et al. [10] combine integer-valued time series with a dynamic latent factor structure to forecast hourly call volumes in an EMS system. Setzler et al. [11] use an artificial neural network to forecast the times and locations of EMS calls for service. These latter three papers focus on normal operating conditions. Other researchers observe that service providers adapt to extreme events when service systems experience heavier traffic, leading to quantifiable levels of system adaptation. In this paper, system adaptation describes how an aspect of the system, such as service times, endogenously depends on the level of traffic in the system, such as the number of patients receiving service. Kc and Terwiesch [12] observe that service times in health care systems are not exogenous, as service times in part depend on the system load. This suggests that EMS personnel may adapt to high call volumes during severe weather events by shortening response and service times. Indeed, Alanis et al. [13] also observe this phenomenon in the EMS domain. Green et al. [14] develop a single-period staffing problem based on the newsvender problem that focuses on endogenously determined absentee rates and identify optimal staffing levels (a measure of system adaptation). This paper develops a model for evaluating EMS staffing levels that integrates simulation, reliability, and regression methodologies. The underlying dynamics of the reliability model are that of a multi-server queuing system, where customers are patients that arrive to the system and servers are ambulances that immediately respond to new patients. EMS systems intend to immediately respond to new incoming patients. The system is therefore reliable if there is a high degree of confidence that ambulances can immediately respond to a given proportion of patients (e.g., 99 %). A simulation model calculates the proportion of patients that arrive when no ambulances are available given a specified number of ambulances. The number of ambulances is varied to identify the smallest number of ambulances required to maintain a high degree of confidence that the reliability level is above a pre-specified threshold (99 % in this case). Random variables in the simulation include patient arrival
15
times, patient locations, patient priority, whether a patient requires hospital transport, and patient service times. The parameters of their distributions are estimated using regression. We compare reliability levels and minimal staffing levels across four different weather scenarios: a normal weather scenario, a light snow (flurries) scenario, a heavy snow (blizzard) scenario, and a post-blizzard scenario in which snow remains on the ground. The approach in this paper can be contrasted with other models in literature. Restrepo et al. [15] develop a model for locating ambulances using the Erlang loss formula. While this paper is quite different from ours in scope, it sheds light on how resource allocation policies affect the number of patients that are “lost” (in an Erlang loss model). Patients are lost when the system is in an unreliable state and there are no ambulances available for immediately servicing a patient. Likewise, our analysis compares four different patient queuing disciplines to measure the number of patients that are lost (due to zero-length queues) or wait in queues and to understand the effect of priority queuing during emergency operations. We refer the reader to [16–18] for detailed reviews of call center capacity management and data analysis in call centers. While our approach is similar in spirit to the models in [11, 14, 15], it is unique in that it integrates simulation, regression, and reliability methodologies while accounting for service providers adapting to higher call volumes based on empirical observations. The impact of system-wide adaptation on system reliability is evaluated by including new variables in the regression models that capture the amount of traffic in the system, and the number of ambulances changes the number of queue servers available in the reliability simulation. Hanover County Fire and EMS in Virginia provided a large data set to perform the analysis. Real-time weather conditions (within 30 min of each call) supplements the EMS data set to evaluate the effect of snow on system reliability. The results of our analysis suggest that snow conditions may significantly increase the likelihood that an EMS system is unreliable and thus necessitates an increase in the number of ambulances that should be staffed. More ambulances are needed to immediately respond to at least 99 % of calls (with a high degree of confidence) when snow is falling and there is snow on the ground, as would occur during Nor’easter (blizzard) conditions in Virginia and other Mid-Atlantic regions. Both the number of ambulances used and the patient queuing discipline significantly alter system reliability. Furthermore, when it is snowing, system adaptation decreases the likelihood of a patient being transported to the hospital, which leads to an increase in system reliability and lower minimal staffing levels. A key observation is that when it is snowing, system adaptation has the effect on system reliability as adding one additional ambulance. These findings suggest multiple ways that EMS systems adapt to the higher
16
call volumes and slower travel times caused by snow. In the rest of the paper, we introduce our model and illustrate its results using data from a real-world setting.
2 Modeling framework In this section, we describe the simulation modeling framework that we developed to identify the minimal number of ambulances needed. First, we introduce the simulation model to describe the overall system dynamics. Then, we describe the regression models used to provide the inputs to the simulation model and describe the data. 2.1 Simulation summary The simulation models incoming EMS calls from the arrival of the initial 911 calls to the system to the times that the responding EMS ambulances complete service. This process Fig. 1 Graphical depiction of the simulation. Rectangles represent steps dependent only on summary statistics or weather scenario inputs; rounded rectangles represent steps dependent on response policy; and pentagons represent steps where the system’s selforganization is taken into account
A. Kunkel, L.A. McLay
is summarized in Fig. 1. We consider EMS response to the call in terms of three separate stages. First, a new call arrives and its arrival time, geographical district of origin, and priority are determined. Next, the call awaits a response. Here, we assume that each call involves one patient that requires service. In this stage, the EMS system decides whether to respond to the call and, if appropriate, the patient waits in a queue for the next available ambulance to begin responding. Under normal operating conditions with at least one ambulance available, this step is trivial; we assume that an ambulance responds to each call if one is available. However, these decisions and call wait times may change based on the weather scenario. In the final stage, a single EMS unit responds to the call. In this stage, the dispatched unit either arrives to the call or does not, a decision is made about whether to transport the patient to the hospital, and the emergency medical technicians (EMTs) eventually complete service. The decisions and times associated with each step depend on the results of the earlier
Determining minimum staffing levels
17
steps as well as the overall simulation input parameters; these are described in Section 2.2. This process is repeated for each incoming call until an iteration limit is reached. In this case, the iteration limit is reached when 10,000 calls arrive. We did not include a warm up time since the system has low levels of traffic. The proportion of calls that arrive when there are no units available (NUA) is computed along with the average waiting time for calls added to a queue with a given number of ambulances. We vary the number of ambulances until we find the smallest number of ambulances such that there is a high degree of confidence (95 % in this case) that the system is in an NUA state when fewer than 1 % of the arriving calls arrive. The simulation parameters are assumed to be stationary and exogenous to the model; therefore, the output reports how the system performs in steady state. This assumption is reasonable, since the data suggest that the system parameters are stationary for a large portion of each day (e.g., the call arrival rate is constant from 11 am–7 pm). We acknowledge that the system may not achieve steady state in this timeframe, since it depends on the traffic intensity. A steady state analysis is informative here in that it yields “worst case” estimates, which are consistent with the goal of identifying conservative levels for system reliability. We acknowledge that the EMS systems are not always in steady state, and we point out that the inputs could be defined to be non-stationary to capture the transient performance. Later, some parameters are assumed to be endogenous to the model. The weather scenario influences nearly every stage of the simulation, as would likely occur in a true snow event. Table 1 reports the independent variables for these weather scenarios and their associated values. weather scenario independent variables include the time of day, the day of week, the season, the temperature (in Celcius), snow depth (in inches), and binary variables associated with thunderstorms, public events, holiday weekends, rain, and snow. Within the call arrival stage, weather scenario directly Table 1 Input parameters for the four weather scenarios Time Interval Day of Week Season Temperature (°C) Thunder Public Events Holiday Weekend Rain Snow Snow Depth (in)
affects the regression models for arrival time, district, priority, and total service time. The weather scenario indirectly affects queue times through changes in call volume, the service time, and the number of busy units (ambulances). In this paper, four distinct weather scenarios are considered, differing only in whether snow is falling and the depth of snow on the ground. The weather scenarios include: (1) a normal weather scenario, in which it is not snowing and there is no snow on the ground; (2) a snow flurries scenario, in which it is snowing but there is no snow on the ground; (3) a blizzard scenario, in which it is snowing and there are 6 in. of snow on the ground; and (4) a post-blizzard scenario, in which it is not snowing but there are 6 in. of snow on the ground. Note that although the weather scenarios differ in at most two of the independent variable values, additional independent variables are used in the regression model. This is because data were collected over the course of a year, not just during winter storm events, and these additional weather variables are important for estimating the input parameters. The simulation model evaluates four different response policies for their ability to mitigate decreased reliability caused by the snow scenarios. These response policies reflect different types of responses that an EMS system may adopt for responding to prioritized patients during normal and emergency conditions as part of their “standard operating procedures.” The four policies are: 1. The queue policy: An ambulance is sent to a call if one is available and queue all calls that arrive when there are no units available. Queued patients are served in a firstcome, first-serve manner when ambulances become available. 2. The prioritize policy is identical to the queue policy except that only high priority calls enter a queue (there is a zero length queue for low-priority calls). 3. The drop policy is identical to the queue policy except that there is a zero length queue. It is assumed that other EMS providers respond to these calls.
Normal Weather
Snow Flurries
Blizzard
Post-Blizzard
12 pm-6 pm Thursday Winter −2 No No No No No 0
12 pm-6 pm Thursday Winter −2 No No No No Yes 0
12 pm-6 pm Thursday Winter −2 No No No No Yes 6
12 pm-6 pm Thursday Winter −2 No No No No No 6
18
4. The preemptive policy: High priority calls follow a queue policy whereas ambulances do not respond to low priority calls regardless of the number of units available These four response policies represent a continuum from treating all patients (the queue policy) to policies that only queue high-priority patients (the prioritize patients), to not allowing queues (the drop policy), to dropping all lowpriority patients (the preemptive policy). Other policies could be considered in this continuum, such as treating or queuing all high-priority patients and treating low-priority patients only when there are two or more available ambulances. The above four policies do not allow for a call to be preempted if a more serious call arises to the system to be consistent with operating constraints that are imposed in practice. A single response policy is used over the course of each simulation. In practice, a single EMS system may switch between these response policies based on the offered load to the system. For example, many EMS systems operate using a queue policy most of the time and may switch to a prioritize or a drop policy when a large number of calls overwhelm the system. An EMS system may use a preemptive policy during catastrophic events, such as during mass casualty events. With proper staffing levels, an EMS system should be able to treat all patients during blizzards and other severe weather events, and therefore, the queue policy is the base policy for comparison. Other policies are included for comparison, since they illustrate inadequate staffing levels would reduce system reliability unless some patients are not served. In practice, patients that are turned away through the prioritize, drop, and preemptive policies could be served by other public services (such as fire or police) or by another EMS department through mutual aid, or the patient could travel to another region where EMS service is available. Not all patients require EMS service after they make a request; however, this possibility is included in the model through a parameter that captures the probability that an EMS unit arrives at the scene. In snow events, for example, this often occurs when an ambulance is requested for a minor car accident, and a responding police officer cancels the ambulance at the request of the patient. An EMS system may adapt to the increased system load during snow events or other scenarios. Adaptation is said to occur if heavier traffic (i.e., more busy units) leads to shorter service times or a decreased likelihood that patients require hospital transport. This type of system adaptation can be modeled by including the number of busy units as an independent variable to the regression models for service times and adding both weather scenario and the number of busy units to the regression model for whether a patient goes to the hospital.
A. Kunkel, L.A. McLay
2.2 Regression models Five regression models are used to provide the distribution parameters for the random variable input parameters for the simulation model. We provide a description of the data from Hanover County, Virginia in the next section. First, a negative binomial regression model estimates the average number of calls that arrive during a six-hour period. This model is used to describe the distribution of call arrival times in the simulation. Unlike the later regression models, the negative binomial regression model is based on a 6 hour aggregate data set (described in Section 2.3). The weather scenario is the only independent variable used in the negative binomial regression model. Note that the weather scenario includes temporal variables such as the time of day, day of week, and season. Since the Negative Binomial(r,p) distribution is equivalent to Poisson(Gamma(α0r, β0p/ (1-p))), the simulation uses the outputs from this regression to produce random arrival times according to an exponential (L) distribution, where L~Gamma(α, β). We use negative binomial regression for this analysis instead of Poisson regression, since the likelihood ratio test reports over-dispersion in count data. Once a call arrives, we use a multinomial logistic regression to randomly assign the incoming call to the appropriate geographical district (of seven). The independent variables for this regression model include the weather scenario, and the response variables are districts where the calls originate. Similarly, we use a logistic regression model to determine the probability that an incoming call will be designated low priority (versus high priority). The independent variables include the weather scenario and geographic district, and its response variable is whether the call is low priority. A third logistic regression model estimates whether an ambulance arrives to the call, since some calls are cancelled en route. The only independent variable is the call priority to ensure that ambulances evenly respond to all calls regardless of the weather scenario. If an ambulance arrives, we use a fourth logistic regression to estimate if a patient requires hospital transport. When system adaptation is not considered, the only independent variable is the call priority, a proxy for the severity of a patient’s condition. When system adaptation is taken into account, other independent variables include the weather scenario, geographic district, and the number of busy units. These variables represent factors other than call severity that may influence a patient’s hospital transport decision. There are three models for service times for (1) calls in which no unit arrives, (2) calls in which a unit arrives but does not transport the patient to the hospital, and (3) calls that result in the patient being transported to the hospital. Calls with no arriving unit are assumed to follow an exponential distribution independent of factors such as weather
Determining minimum staffing levels
scenario and district. In the cases when a unit arrives, one of two linear regression models capture the log service times, since service times from the data approximately follow a lognormal distribution. These models capture calls that go and do not go to the hospital. When system adaptation is not taken into account, the independent variables include the weather scenario, priority, and district. When system adaptation is considered, the number of busy units is also an independent variable.
19
& & & & &
2.3 Input data The data set used for this analysis contains information on 14,634 EMS calls that occurred over 19 months (June 2009 – December 2010) in Hanover County, Virginia, a semirural, semi-suburban county in the Richmond metropolitan area. The data set includes real-time weather information obtained from Mathematica 7.0, which is used to retrieve weather conditions within 30 min of each call. Weather Underground historical data (www.wunderground.com) provides data for missing values and introduces snow depth as a weather variable. The data are organized into two data sets for this analysis. The data set used to predict the call volumes includes data aggregated into six-hour time intervals, while the data set used to predict a call’s district of origin, priority, hospital transport, and service time includes information specific to each call. The differences between these two sets and the variables included in each are listed below. Variables present in only the call-specific data set & &
& & & &
District: From which of seven districts in the county the call originated. District 1 is the reference district. Low Priority (binary): Whether the patient was designated by the dispatcher as low priority, indicating a call that is not life-threatening, as opposed to high priority (life-threatening or potentially life-threatening). No Arrival (binary): Whether a unit was dispatched but never arrived to treat the patient. Hospital (binary): Whether the patient was transported to the hospital. Service Time (in minutes): The total service time for the call, beginning when the first unit was dispatched and ending when the final unit completed service. Number of Busy Units: How many EMS calls were in session at the time of the call. Variables Present in Both Data Sets
&
Temperature (in degrees Celsius): Average temperature during the 6 hour time period (aggregate data set) or temperature recorded closest to the call arrival time (call-specific data set).
&
& & &
Time group: Whether the call or calls occurred from 12 am-6 am, 6 am-12 pm, 12 pm-6 pm, or 6 pm12 am. 12 pm-6 pm is the reference time. Day of week: Wednesday is the reference day. Season: Fall is the reference season. Rain (binary): Whether it rained at all during the 6 hour period (aggregate data set) or at the recorded time closest to the call arrival (call-specific data set). Thunderstorm (binary): Whether there was a thunderstorm at all during the 6 hour period (aggregate data set) or at the recorded time closest to the call arrival (callspecific data set). Public Event (binary): Whether there were any public events during the 6 hour period (aggregate data set) or at the recorded time closest to the call arrival (call-specific data set). The events considered include King’s Dominion, the local amusement park, being open; the state fair; events at Richmond International Raceway; the Hanover Tomato Festival; and Field Day of the Past. Holiday Weekend (binary): Whether the day was a federal holiday. Snow (binary): Whether it snowed at all during the 6 hour period (aggregate data set). or at the recorded time closest to the call arrival (call-specific data set) Snow depth (in inches): The daily snow depth reported from Richmond International Airport.
3 Results and discussion This section summarizes the results of the simulations applied to the Hanover County data set. First, we discuss the regression models used to provide the simulation input parameters. These regression models were run in R 2.14.0, and we ran the simulation code in Matlab 7.11.0.584 (R2010b). For each combination of the weather scenario and the response policy, we ran 40 replications of each simulation scenario until 10,000 calls arrived (approximately the number of calls Hanover receives in 1 year). Each replication begins with zero ambulances busy and, unless otherwise specified, a base case of six ambulances. The simulation outputs are used to discuss the effects of snow on EMS system reliability and how system reliability changes based on response policy, system adaptation, and the number of ambulances available. 3.1 Regression summary The simulation uses five different regression models, the results of which are described below. We analyzed all regression models in R using the functions lm, glm, glm.nb, or vglm. Backwards elimination was used to eliminate variables
20
A. Kunkel, L.A. McLay
Table 2 Arrival time and district regression outputs by weather scenario Mean Call Volume Arrival Parameter α Arrival Parameter β District 1 Probability District 2 Probability District 3 Probability District 4 Probability District 5 Probability District 6 Probability District 7 Probability
Normal Weather
Snow Flurries
Blizzard
Post-Blizzard
7.16 22.15 0.34 0.22 0.17 0.36 0.08 0.05 0.04 0.07
9.49 22.15 0.45 0.24 0.24 0.21 0.07 0.07 0.09 0.09
12.00 22.15 0.57 0.24 0.24 0.21 0.07 0.07 0.09 0.09
9.04 22.15 0.43 0.22 0.17 0.36 0.08 0.05 0.04 0.07
insignificant at the 95 % level, and therefore only significant variables were retained in the final models. For simplicity, no interaction terms were considered. To assess the validity of the regression models, 15 % of the data was withheld from the initial models to verify the results. The linear and count model regression models are evaluated by mean squared error (MSE), univariate logistic regression models are evaluated by correct classification rate (CCR), and the multinomial logistic regression model for district is evaluated by log-likelihood. Beginning with the call volume negative binomial regression model, temperature, snow depth, Saturdays, and snow appear to significantly increase call volume, while summers and all time groups except for 12 pm-6 pm lead to decreased Table 3 Regression coefficients for the logistic regressions for priority, ambulance arrival, and hospital transport and the linear regressions for log(service times)
call volumes. This regression model has an MSE of 8.44 for the training data and 7.29 for the test data set (number of callssquared per 6 hours). While these MSE values are relatively high, they reflect the large variance observed in the data. The likelihood ratio test for overdispersion produces a chi-squared test statistic of 74.18, significant for a p-value of
