This article was downloaded by: On: 8 June 2011 Access details: Access Details: Free Access Publisher Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 3741 Mortimer Street, London W1T 3JH, UK
IIE Transactions on Healthcare Systems Engineering
Publication details, including instructions for authors and subscription information: http://www.informaworld.com/smpp/title~content=t912655173
Optimal and approximate algorithms for sequential clinical scheduling with no-shows
Ji Lina; Kumar Muthuramanb; Mark Lawleya a Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN b McCombs School of Business, University of Texas, Austin, TX Online publication date: 16 March 2011
To cite this Article Lin, Ji , Muthuraman, Kumar and Lawley, Mark(2011) 'Optimal and approximate algorithms for
sequential clinical scheduling with no-shows', IIE Transactions on Healthcare Systems Engineering, 1: 1, 20 — 36 To link to this Article: DOI: 10.1080/19488300.2010.549927 URL: http://dx.doi.org/10.1080/19488300.2010.549927
PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
IIE Transactions on Healthcare Systems Engineering (2011) 1, 20–36 C “IIE” Copyright ISSN: 1948-8300 print / 1948-8319 online DOI: 10.1080/19488300.2010.549927
Optimal and approximate algorithms for sequential clinical scheduling with no-shows JI LIN1 , KUMAR MUTHURAMAN2 and MARK LAWLEY3,∗ 1
Purdue University, Weldon School of Biomedical Engineering, 206 S. Martin Jischke Drive, West Lafayette, IN 47907-2032 E-mail:
[email protected] 2 McCombs School of Business, University of Texas, Austin, TX E-mail:
[email protected] 3 Purdue University, Weldon School of Biomedical Engineering, 206 S. Martin Jischke Drive, West Lafayette, IN 47907-2032 E-mail:
[email protected]
Downloaded At: 21:46 8 June 2011
Received October 2010 and accepted December 2010
The accessibility and efficiency of outpatient clinic operations are largely affected by appointment schedules. Clinical scheduling is a process of assigning physician appointment times to sequentially calling patients. A significant problem in clinical operations is patient no-show, that is, scheduled patients not showing for their appointments. Overbooking can compensate revenue loss due to no-show, but naive overbooking can result in longer patient waiting times and uneven physician work loads. In the past few years, new overbooking methods have been developed for sequential scheduling that yield higher expected profit than simple scheduling rules, but these often fail to exploit information about the future call-in process (they are myopic). To fully use this important information, we develop a Markov Decision Processes (MDP) model for sequential clinical scheduling that books patients to optimize the performance of clinic operations. The model is solved by Dynamic Programming (DP) for small problems. Approximate Dynamic Programming (ADP) algorithms based on aggregation and simulation are developed to find schedules for larger problems. Our computational experiments indicate good improvement over myopic methods. Keywords: Sequential scheduling, patient no-shows, outpatient clinics, Markov Decision Process, approximate dynamic programming
1. Introduction Outpatient facilities are driven by appointment scheduling, which significantly influences system performance and cost. Clinics usually schedule according to patient preference or management experience with little analysis of expected schedule performance. This often results in poor use of clinic capacity, excessive patient waiting, and unexpected overtime. Although outpatient scheduling has a long academic research history, few proposed methods have been implemented in practice. One reason is much of the published research neglects essential complicating factors such as sequential patient call-in, no-show, and random service time. Patient no-show is a major factor impacting schedule performance. Guse et al. (2003) and Moore et al. (2001) report patient no-show rates from 22% to more than 50%. Bean and Talaga (1995) observe that no-show is especially prevalent in mental health, pediatrics, and dentistry. Rust et al. (1995) observe no-show rates as high as 80% in some ∗
Corresponding author
C 2011 “IIE” 1948-8300
public pediatric clinics. Clinics use methods such as telephone/mail reminders, patient education, and penalties to encourage patient attendance, however, no-show cannot be totally eliminated (Guse et al., 2003; Johnson et al., 2007). According to Hixon et al. (1999), one third of clinics using reminder systems still report a no-show rate of approximately 20%. No-show penalties fine patients for late cancellation or unexpected non-attendance (Lesaca, 1995). It reduces no-show in some clinics up to 54%, (Bech, 2005), however, it may not be appropriate for disadvantaged, elderly, or uninsured patients (Lowes, 2005). A recent overview of no-show results for psychiatric clinics is provided by Alafaireet et al. (2010). In contrast, overbooking attempts to mitigate the negative effects of no-show patients. This can only be effective if the effects of no-show are well balanced with those of over-show. To achieve this balance, it is necessary to estimate the probability that patients will not attend. This is typically done by regressing patient demographics and conditions, appointment characteristics, and other types of information on patient no-show data. Appointment lead time and past no-show behavior are often the most significant factors in these models. Daggy et al. (in press) provides
Algorithms for scheduling no-shows
21
Fig. 2. The call-in/scheduling process.
Downloaded At: 21:46 8 June 2011
Fig. 1. No-show distribution.
a more detailed literature review and develops a regression model for estimating the no-show probabilities of patients at a large midwestern Veterans Affairs clinic. Fig. 1 presents the no-show probability distribution of this patient panel. This model has a C statistic value of 0.82, which represents the area under the receiver operating characteristic (ROC) curve, indicating that no-show behavior can be effectively modeled at the individual level. Clinics typically maintain scheduling information such as cancellations and no-shows, which provides the basis for no-show probability estimation. Although not extensively studied by other authors, our work indicates that modeling the no-show distribution is very important in developing better schedules. For example, Zeng et al. (2010) show that schedules created with the additional information provided by heterogeneous noshow models dominate schedules developed using a more aggregated homogeneous no-show model. Outpatient clinics typically use some type of sequential scheduling. Customers call for appointment times in a stochastic sequence, and thus the set of patients eventually added to the schedule is unknown during schedule construction. Two processes are relevant for sequential scheduling: the call-in/scheduling process and the service process to execute the schedule. For the first, patients call the clinic or a call center for appointments with a physician. Patients usually see their own physicians to maintain continuity, so the scheduling request is for a specific physician (implying one server per schedule). The calls are handled one at a time by a scheduler. For each call-in, the scheduler assigns a time slot to the patient by using scheduling policies. (Most clinics partition the service session into a set of consultation slots of fixed length.) The selected appointment time is given to the patient before the call terminates. Once an appointment is made for a patient, there is limited opportunity for change. If there is no available appointment on the preferred day, the scheduler transfers the patient’s request to another day. Fig. 2 illustrates this process. The service process (Fig. 3) is the realization of a finalized schedule. Scheduled patients either arrive for their appointments or no-show. Arriving patients are seen by
the physician in the order of scheduled times. Cost occurs when patients have to wait and/or physicians must work overtime. The optimal policy decides the best number of patients to schedule and the optimal arrangement. Sequential scheduling policies can either be myopic or non-myopic. Myopic policies schedule patients to maximize immediate rewards without considering the future call-in process. In contrast, non-myopic policies use information about the future call-in process when making the scheduling decision. The objective of this paper is to develop non-myopic approaches that make optimal use of future call-in information. We believe this work to be the first to provide an optimal model for sequential assignment of heterogeneous no-show patients to time slots with the possibility of slot overbooking and in-clinic patient waiting. Our objective is to optimally balance revenue from seeing patients with the costs of patient waiting time and overtime in each session. The rest of the paper is structured as follows. Section 2 provides a review of what we believe to be the most relevant literature. Section 3 presents our non-myopic model for the sequential call-in process and the slot model for the service process. Section 4 uses dynamic programming to solve for the optimal model, while Section 5 presents two approximate algorithms for overcoming the computational complexity. Section 6 provides computational results, benchmarks, and insights. Section 7 concludes with remarks on the limitations and future directions of the research.
2. Literature review This section provides reviews of the most relevant scheduling literature. The following early studies examine the effects of overbooking by simulation. Rigorous models are not provided. Shonick and Klein (1977) estimate patient no-show probabilities from historical data and use those to overbook until the expected number of arrivals reaches a specified target. They provide good insights but no formal analysis. Kim and Giachetti (2006) also address the question of finding the optimum number of patients to book. Their objective is to maximize clinic profit, which is weighted by revenue, cost of overtime, and a penalty for pa-
Fig. 3. The service process.
Downloaded At: 21:46 8 June 2011
22 tients who leave without being seen. They assume stochastic arrivals and deterministic service times. Their model uses enumeration to compute the optimum number of patients to book for a given session, assuming a homogeneous noshow probability, but they do not determine patient appointment times. LaGanga and Lawrence (2007a,b) use a utility function to evaluate the benefit of serving patients, the cost of waiting time, and overtime. Their objective is to identify the ranges and ratios of cost parameters for which overbooking is beneficial. They assume deterministic service times and a single no-show probability. Using simulation, they determine that their approach works best with high no-show probability and low service time variances. Many researchers study non-sequential scheduling in which an optimal schedule is sought for a given set of patients. (Because decisions are not made in stages, we refer to this as “static.”) They use methods from deterministic operations research such as integer programming, enumeration, and local search (Vanden Bosch et al., 1999; Vanden Bosch and Dietz, 2000, 2001; Kaandorp and Koole, 2007; Hassin and Mendel, 2008; Zeng et al., 2010). Other researchers consider sequential scheduling, in which patients are added to a progressively growing schedule as they call for appointments (Muthuraman and Lawley, 2008; Chakraborty et al., 2010). This type of scheduling can be either myopic or non-myopic, depending on whether information about the expected future call-in process is used. We now review these papers in detail. Vanden Bosch et al. (1999); Vanden Bosch and Dietz (2000; 2001) develop a model to minimize the weighted sum of expected overtime and patient waiting time. They assume a given set of patients, each with a no-show probability and service time distribution (approximated with Coxian random variables), to be scheduled into a given slot structure. Because the scheduling problem is computationally complex, they decompose it into two parts. They first address the problem of finding the best arrival times for the patients, given that the sequence of arrivals is fixed. For this, they develop an algorithm that incrementally shifts scheduled arrivals by one slot length, constantly updating the schedule if the resulting shift reduces cost. The algorithm terminates when no shift produces incremental improvement. Next, they consider the problem of finding good sequences. For this, they develop a pair-wise swapping routine that terminates when no further improvement is observed. Kaandorp and Koole (2007) assume exponential service and a single no-show probability. Their objective is to minimize patient waiting time, physician idle time, and overtime for a given set of patients. To compute the schedule, they develop a local search algorithm, which seeks incremental improvement by shifting patients between adjacent slots. They establish that this approach yields a global optimum when the objective is multimodular. Hassin and Mendel (2008) use a queuing approach to schedule a fixed number of patients with homogeneous no-show. Their objective is
Lin et al. to minimize the cost of expected customer waiting and expected server availability. They assume that regular and overtime server costs are identical. Their model indicates that appointments should be closer together at the beginning and end of a schedule. Further, they show the expected waiting time increases with show up probability and that the length of equal interval spacing has an approximate linear relationship with show up probability. Muthuraman and Lawley (2008) develop a slot model for sequentially building a schedule as patients call in. The model assumes heterogeneous no-show probabilities and exponential service times. They analyze the stochastic overflow and develop a profit function as a weighted sum of reward for service, patient waiting time, and physician overtime. They develop a myopic policy (in that it does not consider future call-in information) for their profit function with a stopping rule that terminates scheduling when the addition of one more patient causes the profit to drop. They prove the profit function to be unimodal, which guarantees the stopping rule to be optimal. Chakraborty et al. (2010) extend the model of Muthuraman and Lawley (2008) to the general service time setting by conditioning on time in service when computing the overflow matrix. This results in a multi-dimensional integration problem that requires numerical methods to solve. A method is developed that reduces the integration to a single dimension when the service time can be modeled as a gamma distribution. This is particularly advantageous since the gamma’s parameters can be adjusted to model many distributional shapes. Zeng et al. (2010) prove that the profit function of Muthuraman and Lawley (2008) is not multi-modular with a given set heterogeneous no-show patients in a static setting. Thus, a local search cannot guarantee a global optimum for the static slot scheduling problem (as it does with the homogeneous patients of Kaandorp and Koole (2007)). They also show that heterogeneous no-show groups (more refined no-show modeling) provide better schedules than the homogeneous case. Some studies do not minimize direct waiting (Green et al., 2006; Patrick et al., 2008) and some assign patients to days but not time slots (Gupta and Wang, 2008; Liu et al., 2010) so that they do not analyze the performance of the schedule for a specific day. Detailed reviews are as follows. Green et al. (2006) model a diagnostic medical facility with scheduled outpatient and random inpatient and emergency services. The service session is divided into time slots of identical length. All scheduled outpatients have the same no-show probability, and emergency patient requests must be served immediately. Their objective is to maximize a profit function consisting of rewards for patients served, costs for lead time waiting, and penalties for patients not served. Finite horizon dynamic programming is used to develop switching curves for deciding the next patient type to serve. Patrick et al. (2008) develop a rolling horizon model for diagnostic facilities with multi-priority including inpatient and different levels of outpatient service. They
Downloaded At: 21:46 8 June 2011
Algorithms for scheduling no-shows
23
assume a fixed capacity for each day without considering no-show for outpatients. The waiting cost is due to exceeding the priority-specific waiting time target. They transform an MDP model to its equivalent linear programming model and solve it by approximate linear programming. Gupta and Wang (2008) use an MDP model to balance long term scheduling and same day visits for slot-based scheduling. They assume that all scheduled patients keep their appointments and that service times are bounded by the slot length. They divide the call-in period into small intervals (representing MDP stages) such that at most one scheduling decision can be made during an interval. The scheduler attempts to place the caller with his or her primary physician. If an appointment time is not available, other physicians can be considered at greater cost. Some patients may be rejected to either preserve slots for patients in the physician’s panel or for same-day patient scheduling. Cost is incurred when patients are rejected or slots go unused. Liu et al. (2010) develop a model that assigns patients to days using no-show and cancellation probabilities, which depend on the appointment delay. They do not provide patient appointment times during the day, and thus do not need to consider service time distributions. They develop heuristic policies based on several simple scheduling rules. These include open access and a two day policy that assigns patients into the next two days within certain probabilities. They use simulation to compare the performance of the heuristic policies. The models discussed above differ from ours in significant ways. First, many assume deterministic or exponential service times, whereas our work makes no assumption and thus supports general service time distributions. Second, many either do not include no-show or consider only homogeneous no-shows. Those that do consider hetero-
geneous no-shows either do not provide optimal models for sequential slot-based scheduling or consider only the static scheduling problem (they do not consider sequential scheduling). Moreover, some authors assign patients to days but not to exact time slots and thus have no need to model patient overflow between consecutive service periods, which provides a significant simplification. Finally, in most outpatient clinics, particularly primary care, patients seek non-emergency services, and thus it is not necessary to reserve unscheduled capacity for emergent patients. Since patients are non-emergent, appointment lead times are less important but direct waiting in the clinic is costly since it affects patient satisfaction. Table 1 compares the characteristics of the above cited literature with our proposed model.
3. The clinic model This section introduces the Markov Decision Process (MDP) model for optimal scheduling and the slot model for schedule evaluation. Table 2 provides selected notation. 3.1. The MDP model and optimal policy In our MDP model, we assume that a calling patient can be classified into a no-show group immediately after the call is answered and the patient is identified. We also assume that calls are answered first-come-first-served and that the call-in rate of each no-show group can be estimated from historical data. We partition the call-in period into N time intervals. Each interval is sufficiently small such that the scheduler can handle no more than one call during any given interval. According to this discretization, the time
Table 1. Characteristics of existing studies and the proposed model Authors
No-show
Service Time
Methods
Shonick and Klein (1977) Kim and Giachetti (2006) Laganga and Lawrence (2007) Vanden Bosch et al. (1999–2001) Hassin and Mendel (2008) Kaandorp and Koole (2007) Zeng et al. (2010)
Non-Sequential Scheduling Homogeneous Deterministic Naive overbooking Homogeneous Deterministic Enumeration Homogeneous Deterministic Simulation Heterogeneous Erlang/Coxian Local search Homogeneous Exponential Nonlinear programming Homogeneous Exponential Local search Heterogeneous Exponential Local search
Muthuraman & Lawley (2008) Chakraborty et al. (2010) Green et al. (2006) Patrick et al. (2008) Gupta and Wang (2008) Liu et al. (2010) The proposed model
Heterogeneous Heterogeneous Homogeneous N/A N/A Heterogeneous Heterogeneous
Sequential Scheduling Exponential General Deterministic Deterministic Deterministic No consideration General
Myopic policy Myopic policy DP ADP Booking-limit policy Heuristic policy DP/ADP
Simple rules and simulation Static scheduling
Non-optimal No direct waiting Does not assign patients to slots Optimal/Approximate
24
Lin et al.
Table 2. Notation list Notation ci cI F(S) i j I J Li n N p j , j = 0 q j , j = 0
Downloaded At: 21:46 8 June 2011
q0 Q r R S i j S+ tnow V(S, n) Xi Yi t
ij
Definition Cost of a single patient overflow from slot i to i + 1 Overtime cost of a patient Profit function of a finalized schedule S A time slot Patient type of no-show Number of time slots Number of no-show types Number of service completion in slot i given infinite number of patients A call-in time interval/stage Terminal call-in time interval/stage Probability that a type j patient attend for the appointment Probability that a request of type j patient is received in stage n Probability that no appointment request received in stage n Arrival matrix Reward for serving a patient Overflow matrix State of a schedule Matrix which indicate an assignment of a type j patient in slot i Adding a type j patient in slot i of schedule S Current clock time Value function of a schedule S at stage n Number of patients arrival at start of slot i Number of patients overflow from slot i to slot i + 1 Set of all possible schedules after t stages
interval n is a stage in Markov Decision Process. We let q j represent the probability that the scheduler serves a patient of no-show type j in a given stage, and we assume that calls arrive independently. We further assume that the scheduling decision can be made and the state changes within the stage. The scheduler stops answering calls after the final stage N. Note that under these assumptions Jj=0 q j = 1, where J q0 = 1 − j =1 q j is the probability that the scheduler is idle for a stage. The service session is divided into slots, i ∈ {1, 2, . . . , I}. The schedule being constructed is represented by a I × J matrix, S, whose (i j )th element Si j denotes the number of type j patients scheduled for slot i . If no patient calls in n, the schedule remains the same in n + 1. Otherwise, the possible actions are either providing the patient with a time slot or transferring the patient request to another day. Let i j ∈ I×J represent the assignment matrix. It will have one nonzero, unit element, say (i, j ), when a patient of type j is to be assigned to slot i . Fig. 4 shows the relationship between schedule states at stages n and n + 1. All the values of possible consequent states are used to provide an expected value for the current state. The value function of a state S at n < N is given by the Bellman
Fig. 4. State transition where n < N.
equation, V(S, n) =
J j =1
max V(S + i j , n + 1),
q j max
i ∈{1,...,I}
V(S, n + 1) + q0 V(S, n + 1)
(1)
The schedule is finalized at the end of the final stage, and the final value function is a function of the final schedule S. Therefore, V(S, N) = F (S)
(2)
The function F is the expected performance of a final schedule. Any function which can provide the value of a finalized schedule can be used. 3.2. The slot model and profit function The slot model presented here is originally developed in Muthuraman and Lawley (2008). We describe it to make the paper self contained. For a given schedule, S, each slot has a fixed length δti (not necessarily equal). Physician examination is the major part of the service process, thus other activities such as registration and nurse triage are not included in the model. An arriving patient is punctual and is seen immediately if the physician is idle; otherwise, the patient enters a queue and waits to be served. If patients are not served in their scheduled slots, they overflow to the next slot with a cost. If the physician cannot finish by the end of the day, overtime is needed until all patients are served. This slot model can be categorized as variable block/fixed interval under the classification of Cayirli and Veral (2003). As shown in Fig. 5, Xi is the number of patients arriving for slot i , and Yi is the number of patients overflowing from
Algorithms for scheduling no-shows
25
Fig. 5. The slot model. {1, . . . , I} is the set of time slots. Xi is the number of patients arriving for slot i . Yi is the number of patients overflowing from slot i to slot i + 1. Li is the number of service completions in slot i given an infinitely long queue.
The computation of the overflow matrix, R, depends on the service time distribution. For a general service time, the service completion in slot i , Li , can be derived by conditioning on the service time spent for the patient overflowed from slot i − 1 (see Chakraborty et al. (2010) for derivations). Further, the overflow matrix can be used to compute measures such as waiting time,time in system, and average queue length.
Downloaded At: 21:46 8 June 2011
4. Scheduling policies and characterization slot i to i + 1, including the patient being served. Thus, Xi + Yi −1 is the number of patients in the system at the beginning of slot i . Let Li be the number of service completions in a slot given an infinite number of patients in the queue. As the number of patients waiting at the beginning of slot i is Xi + Yi −1 , the actual number of services completed in slot i is min(Li , Xi + Yi −1 ). Therefore, the overflow from slot i to slot i + 1 is Yi = max(Xi + Yi −1 − Li , 0). Let r be the reward of serving a patient, and ci be the cost of a single patient overflowing from slot i to i + 1. The performance of a finished schedule is represented by a profit function F which rewards patients served and penalizes waiting and overtime. F=
I
r E [Xi ] −
i =1
I
ci E [Yi ]
(3)
i =1
For a schedule with maximal patients of M, we introduce the arrival matrix, Q, and the overflow matrix, R, both of the size M × M. The element Qi m represents the probability that m patients arrive for slot i . Ri k represents the probability that k patients overflow from slot i to i + 1. We M M mQi m , and E [Yi ] = k=0 kRi k. now have E [Xi ] = m=0 Let π ∈ be the set of non-negative integer J-vectors such that Jj=1 π j = m. Then, Qi m = Pr {Xi = m} = =
Si j π∈
j
πj
Pr {(π1 , . . . , π J )}
π∈ π p j j (1
− p j ) Si j −π j
(4)
Pr{Xi + Yi −1 − Li = k} k > 0 Ri k = Pr {Yi = k} = (5) Pr{Xi + Yi −1 − Li ≤ 0} k = 0 Let fLi (m) = Pr {Li = m} and FLi (m) = Pr {Li < m} which can be obtained from the distribution of service time. Then, ⎧ ⎪ k=0 (1 − FLi (m + h)) Qi m Ri −1,h ⎪ ⎨ m h Ri k = ⎪ (1 − fLi (m + h − k)) Qi m Ri −1,h k > 0 ⎪ ⎩ m
h
(6)
MDP models are usually difficult to solve due to dimensionality. Because the clinic model has a finite time horizon and factorially increasing states, traditional methods to reduce computational complexity on infinite horizon and small state set are not suitable. Moreover, the profit function has many local optima (Zeng et al., 2010), so there are currently no known algorithms to solve it both optimally and fast. Thus, in the following subsection, we develop backward induction to find the exact solution for small cases. This is necessary for comparing optimal solutions with myopic solutions for small cases to provide indication of whether information about the future call-in sequence is really useful. It also serves as the basis for approximate algorithms. 4.1. Backward induction and optimal policy The solution to the MDP model is divided into two parts: the offline part to generate the optimal policy and the online part to schedule patients according to the policy. For the offline part, decision tables for each stage are created by backward induction to solve the Bellman equation from the final stage to the first. At stage N, equation (2) is used to generate the value function for all the possible schedules. For stage n < N, equation (1) is used to generate value functions at n using value function information at stage n + 1. The optimal policy is given by the optimal actions to take. The optimal action A∗ is given by A∗ (S, n, j ) ⎧ ⎨Assign to i ∗ , = ⎩ Transfer another day,
∗
if V(S + i j , n + 1) ≥ V(S, n + 1) (7) Otherwise
Where the optimal slot, i ∗ , is determined by state, time, and call-in patient type. So, i ∗ (S, n, j ) = arg max V(S + i j , n + 1) i ∈{1,...,I}
(8)
Computation of the optimal policy must be finished before the start of the call-in/scheduling process. Thus, when a patient calls for an appointment, the online scheduling method checks the decision table to retrieve the optimal slot. The management of the state space is also a challenge.
26
Lin et al.
To save time searching states for the DP algorithm, fast approaches such as hashing are recommended, a perfect hash can reduce the complexity of the look up step to O(1). 4.2. Characterization of the model Proposition 1 establishes the relationship between value function and profit function. Further, it establishes that if the profit function and value function of a given schedule are equal, then the schedule is optimal. Proposition 2 shows once the optimal policy stops to assign all patient types, no patient can be assigned any more. Proposition 1. For any stage n, if V(S, n) = F(S), then (S, n) is optimal.
Proof. We need to prove: if V(S, n) = F(S), then A∗ (S, n , j ) = Transfer, ∀ j and n ≥ n. According to (1), V(S, n) ≥ Jj=1 q j V(S, n + 1) + q0 V(S, n + 1), for n < N. Thus,
Downloaded At: 21:46 8 June 2011
V(S, n) ≥ V(S, n + 1),
n n; equality holds when A∗ (S, n , j ) = Transfer, ∀ j . Proposition 2. For any state S at a stage n, if the optimal policy is to transfer all patient types, then for any n ≥ n, the optimal policy is also to transfer all patient types.
Proof. According to (1), the Bellman equation at stage n, the optimal policy is to transfer all patients, iff V(S, n + 1) ≥ maxi =0 V(S + i j , n + 1), ∀ j . For stage n + 1, the Bellman equation is V(S, n + 1)
J ij q j max max V(S + , n + 2), V(S, n + 2) = j =1
i ∈{1,...,I}
+ q0 V(S, n + 2) by (9), V(S + i j , n + 1) ≥ V(S + i j , n + 2), Therefore,
∀i, j .
V(S, n + 1) ≥ max V(S + i j , n + 2), ∀ j
(10)
i =0
According to the Bellman equation at stage n + 1, (10) holds, iff V(S, n + 2) ≥ maxi =0 V(S + i j , n + 2), ∀ j . Thus, for n + 1, the optimal policy is to transfer all patient types. By induction, we can show that for any n ≤ n ≤ N the optimal policy is to transfer all patient types. 4.3. Semi-optimal policy Under the optimal policy, some patients will be transferred to other service periods. In Section 6, our results indi-
cate that, for heterogeneous no-show, higher probability no-show patients are typically transferred during the early part of schedule construction. That is, patients with good attendance potential are preferred. Many clinic providers resist transferring patients when slots are available. Thus, we propose a revised model, referred to semi-optimal, that accepts all patient requests before overbooking and allows transferring thereafter. For an I slot system, overbooking starts when |S| = I, where |S| represents the number of patients in the schedule. For |S| > I, the Bellman equation is the same as equation (1). For |S| ≤ I, the Bellman equation is modified to remove the transferring action as an option. 4.4. Computational complexity of optimal solution The computational complexity of the optimal solution depends on the number of schedule realizations at each stage. For a system with I slots and J patient types, it is easy to see that, for the first stage, the number of ways to as). Alsign patients, including transferring, is |1 | = ( 1+I·J I·J though we omit the proof, it is not difficult to show that ). For a system with N stages, the total num|n | = ( n+I·J I·J N N n+I·J ber of schedule realizations is n=1 |n | = n=1 ( I·J ). For an eight slot schedule with two patient types and thirty stages, the total number of schedule realizations is approximately 2.74 × 1012 . Thus, in the following, we develop approximation methods to handle this complexity.
5. Approximation algorithms In the following subsections, we present two Approximate Dynamic Programming (ADP) methods for solving the optimal policy and semi-optimal policy. The first uses a slot aggregation method in which computation is reduced by making the slot assignment in two steps. The first step uses the optimal policy to assign the patient to an aggregated slot, and the second step uses the myopic policy to assign the patient to one of the original slots within the aggregated slot. The second method uses forward simulation to estimate the value function for successive schedules as more patients are added. In this approach, a number of sample paths are traversed for a given schedule, and the schedule’s value function is estimated using the average value of the set of final schedules observed. Section 6 provides insights into the performance of these approaches. 5.1. State aggregation method To reduce the complexity of the optimal policy, we need to reduce both the time and space used in computation. We develop an aggregation method that merges selected neighboring slots into one. For example, an eight slot system might be converted to four slots by merging adjacent slots. That is, slots [τ0 , τ1 )[τ1 , τ2 ), . . . , [τ7 , τ8 ) could be aggregated into [τ0 , τ2 )[τ2 , τ4 )[τ4 , τ6 )[τ6 , τ8 ). The Bellman equation is
Algorithms for scheduling no-shows then solved using backward induction to build the decision table for the aggregate system. Thus, computation can be brought within manageable limits by choosing an appropriate aggregation level (AL). In the aggregate scheduling process (Algorithm 1), a patient assigned to an aggregate slot must be further assigned to one of the original slots. For example, if a patient is optimally assigned to slot [τ0 , τ2 ), it is necessary to further specify whether the patient should go into [τ0 , τ1 ) or [τ1 , τ2 ). To do this, we use the myopic policy as defined in Muthuraman and Lawley (2008). SI/K = null, SI = null, tnow = 0; while tnow ≤ tN do Wait for patient call; j = patient no-show type; n = current stage; hK hj ij SI/K = SI ;
Downloaded At: 21:46 8 June 2011
i =(h−1)K+1
Look up A∗ (SI/K , n, j ) ; if A∗ (SI/K , n, j ) = Assign h ∗ then i∗ = arg max F(S + i j ); i ={(h ∗ −1)K+1,...,h ∗ K} ∗
SI = SI + i j ; else
27 method, we generate a sample path of future call-ins based on the call-in rate. Starting from the examined schedule, we assign patients along the sample path using some selected policy. Upon reaching the end of the path, we compute the profit function of the final schedule. Repeating this many times, we use the average of the ending profit functions to estimate the value function of the examined schedule. To describe this more precisely, we need the following definitions: Definition 1. Let Wθ (S, j, n) be the value of state S at stage n, with a patient type j call-in, and using scheduling policy θ. Definition 2. Let Vθ (S, n) be the value of state S at stage n using scheduling policy θ. Definition 3. Let i θ be the action (slot assignment) given by scheduling policy θ. These values are related as follows: Wθ (S, j, n) = max Vθ (S + i j , n + 1) i ∈{0,...,I}
Vθ (S, n) =
J
q j Wθ (S, j, n)
(11)
(12)
j =0
S = S;
end end Algorithm 1: Aggregate scheduling calling patients (online, Aggr ) Note that in Algorithm 1, we let K = AL, that is, K is the number of adjacent slots aggregated into one (we assume that K is an integer divisor of I). Further, we let SI be the original schedule and SI/K be the aggregated sched K hj ij SI , for j = 1, . . . , J. Table ule, where SI/K = ih=(h−1)K+1 look up for the optimal aggregate slot occurs before the if statement, and the myopic computation for the specific slot occurs in the if statement. Although the aggregation algorithm reduces required computation, it still requires generating a full decision table for the aggregated system. If too much aggregation is required to achieve the required computation limit, the method could provide poor approximations. Thus, in the next subsection, we explore the second approximate algorithm. 5.2. Simulation based method Backward induction requires the evaluation of all possible states including many that are never visited, a waste of computational space and time. To avoid this, we develop a simulation method to step forward through time to get an estimate of the value function for a given schedule. In this
We equate transferring with assigning to slot i = 0, letting 0 j = i 0 = 0 I×J . Now suppose the state is S at time t and a patient of type j calls for an appointment (call this patient pt j ). In our model, we can take any one of I + 1 actions; that is, we can assign pt j to one of the I slots or transfer schedule pt j . Each action results in exactly one new state. We rank the actions by estimating the value of each state-action pair, Vθ (S + i j , n + 1). Therefore, Vθ (S + i j , n + 1) needs to be estimated for ∀i ∈ {0, 1, . . . , I}. To estimate these, we randomly generate K sample paths, ω1 to ω K , of future call-ins based on the historical call-in rate. Note that ωk is a sequence vector of patient types of length N − n. Each element of ωk is ωlk ∈ {0, 1, . . . , J}, where the 0 element represents no call during a stage. Then indexing across i (the slot model), we assign pt j to slot i ∈ {0, 1, . . . , I}, and, starting with ω1 , we assign each patient type along the sample path, one at a time and in order, to a schedule slot, i ∈ {0, 1, . . . , I}, using some selected policy, θ. Then we compute the profit function of the realized schedule Sk. We repeat this for all the K sample paths, then average the profit functions (see Algorithm 2). This average approximates the value of the state/action pair, at stage n. ˆ θ (S + i j , n + 1). Thus, We denote this as V K ˆ θ (S + i j , n + 1) = 1 V F(Sk) and K k=1 θ ˆ θ (S + i j , n + 1) (13) i = arg max V i ∈{0,1,...,I}
28
Lin et al.
Let n = time interval/stage, S = schedule, j = patient type; for i = 0 to I do Vsum = 0; for k = 1 to K do Sk = S + i j ; Generate ωk; for l = 1 to N-n do i θ = θ(Sk, n + l, ωlk); k θ Sk = Sk + i ,ωl ; end Vsum = Vsum + F(Sk); end ˆ θ (S + i j , n + 1) = Vsum /K V end ˆ θ (S + i j , n + 1); i ∗ = arg max V i ∈{0,...,I} ∗
Downloaded At: 21:46 8 June 2011
S = S + i j ; Algorithm 2: Assigning a calling patient using simulation based scheduling, (Si mθ )
Proposition 3. s
WMyo (S, j, n) ≤ VMyo (S + i j , n + 1)
(14)
Proof. When the value function approximation in (13) is precise, according to the algorithm, VMyo (S + s i j , n + 1) = maxi ∈{0,1,...,I} VMyo (S + i j , n + 1). Then, m s VMyo (S + i j , n + 1) ≤ VMyo (S + i j , n + 1). Hence, m s WMyo (S, j, n) = VMyo (S + i j , n + 1) ≤ VMyo (S + i j , n + 1) Theorem 1.
VMyo (S, n) ≤ VSi m(Myo) (S, n)
Proof. For this proof, we will let jn ∈ {0, 1, . . . , J} represent the patient type calling at stage n. Also, we will let i nθ ∈ {0, 1, . . . , I} represent the slot assigned at stage n using policy θ. Clearly, VMyo (S, n) = jn ∈{0,...,J} q jn WMyo (S, jn , n). Thus, according to (14), we have VMyo (S, n) = q jn WMyo (S, jn , n) jn ∈{0,...,J}
≤
s
q jn VMyo (S + i n jn , n + 1)
jn ∈{0,...,J}
In the remainder of this section, we will let θ be the myopic policy, that is, θ = Myo, and Si mθ = Si m(Myo) . Thus, VMyo (S, n) is the value of a given state S at stage n by using myopic policy for n ≥ n, no matter how S was constructed, and VSi m(Myo) is that value under the simulation method. ˆ Myo (S + i j , n + 1) converges to It is easy to show that V ij VMyo (S + , n + 1) as K → ∞. This means that (given θ = Myo for n + 1 to N) the estimated value of S + i j at stage n + 1 will converge to the true value as K increases. Thus, for a given call-in of type j at S and n, Si m(Myo) ranks slots {0, 1, . . . , I} by estimating the true value of S + i j at n + 1, which is VMyo (S + i j , n + 1). Si m(Myo) does not rank slots based on F(S + i j ), as would the myopic policy. Thus, when Si m(Myo) selects a slot for assignment, it is because we think (based on our estimate) that the resulting state has the highest expected future reward, not because it has the highest expected myopic profit. (As K increases, we have greater confidence that this is true, although we can currently give only empirical results about what K should be. See Section 6.) Let i Si m(Myo) = i s and i Myo = i m , when t is s m small, F(S + i j ) will typically be less than F(S + i j ); that is, the myopic choice will not typically be made (as is illustrated in Section 6). We need to establish that Si m(Myo) typically leads to more rewarding final states than does Myo. Proposition 3 establishes that the value, under myopic assignment, of being in state S at stage n, and having a callin of patient type j , is less than or equal to the value of the state at n + 1 resulting from having assigned that patient using the simulation based method. This result is necessary to prove Theorem 1, which establishes that simulation based scheduling can do no worse than myopic scheduling.
For each jn ∈ {0, . . . , J}, a state at time n + 1 has a value under myopic policy s
VMyo (S + i n jn , n + 1) s = q j(n+1) WMyo (S + i n jn , j(n+1) , n + 1) j(n+1) ∈{0,...,J}
≤
s
s
q j(n+1) VMyo (S + i n jn + i (n+1) j(n+1) , n + 2)
j(n+1) ∈{0,...,J}
Therefore,
VMyo (S, n) ≤
q jn
jn ∈{0,...,J}
q j(n+1) VMyo
j(n+1) ∈{0,...,J} s
s
(S + i n jn + i (n+1) j(n+1) , n + 2) Pr ( jn , j(n+1) )VMyo
=
jn , j(n+1) ∈{0,...,J} s
s
(S + i n jn + i (n+1) j(n+1) , n + 2) Continuing this derivation to stage N, we get VMyo (S, n) ≤ Pr ( jn , . . . , j N )VMyo jn ,..., j N ∈{0,...,J} s
s
s
(S + i n jn + i (n+1) j(n+1) + . . . + i (N−1) j(N−1) , N) (15) The right side of (15) is just VSi m(Myo) (S, n). Thus, VMyo (S, n) ≤ VSi m(Myo) (S, n). Theorem 1 implies that the simulation based method will do no worse than the myopic policy if we choose K to be sufficiently large. This will be numerically explored in the next section.
Algorithms for scheduling no-shows
29
The approximate algorithms are also effective for the semi-optimal policy discussed in Section 4.3. The mathematical analysis of Theorem 1 still holds. The proof can be derived in a similar way without the transferring action.
Slot assigned by optimal policy 0
0
0
1
3
0
0
0
0
0
2
4
1
0
2
Profit of myopic policy Profit of optimal policy Value function of optimal policy
250
250
200
200
150
150
100
100
50
50
This section provides computational results for the policies developed in the previous sections. Offline computations are programmed in C, and online computations are programmed in MATLAB R2008b. Section 6.1 applies the optimal and semi-optimal policies to small systems solved by backward induction. Section 6.2 provides results for the aggregation method, and 6.3 for the simulation based approach. Table 3 provides a summary of the results.
Expected Profit
6. Computational results and insights
0
Downloaded At: 21:46 8 June 2011
6.1. Performance of the optimal policy for small systems In this section, we apply the optimal and semi optimal policy using backward induction to a small system with two patient types and four slots; that is, I = 4 and J = 2. The reward to serve a patient, r , is set to be greater than the cost of a single patient overflow before the end of the clinic session, ci , but less than the overtime cost for a patient, c I ; that is, ci < r < c I . Unless otherwise specified, we set r = 100, ci = 40 for i < I, and c I = 240. And we set patient types 1 and 2 with attendance probabilities of 0.4 and 0.8, respectively, and the call-in probability of each group is 0.4. That is, p = (0.4, 0.8) and q = (0.4, 0.4) for two no-show types. The service time is set to be exponential.
0
1
1
2
2
0 0 1 1 Sequence of patient types
0
1
1
2
1
2
0
Fig. 6. Example of the opti mal policy, I = 4, J = 2, N = 15, λ = 1.5.
We first illustrate and compare the schedule evolution of the optimal policy with the myopic policy as the callin period progresses. Fig. 6 shows this process. The solid line shows the value function of the schedule, the dashed line shows the profit function under the myopic policy, and the dash-dotted line shows the profit function under the optimal policy. The myopic policy assigns slots based on maximizing the profit function, while the optimal policy assigns slots
Table 3. Improvement of policies over the myopic policy with computational consumptions Improve.(%) Policy/algorithm Opt/Backward DP Opt/Aggr (AL = 2) Opt/Sim(Myo) Opt/Aggr (AL = 2) Opt/Aggr (AL = 4) Opt/Sim(Myo) Opt/Sim(Myo) Semi − Opt/Backward DP Semi − Opt/Aggr (AL = 2) Semi − Opt/Sim(Myo) Semi − Opt/Aggr (AL = 2) Semi − Opt/Aggr (AL = 4) Semi − Opt/Sim(Myo) Semi − Opt/Sim(Myo) Enumeation Local search MyopicPolicy MyopicPolicy MyopicPolicy
Computational time(sec)
I
Avg
Std
Per Assignment
Offline
Storage size
4 4 4 8 8 8 16 4 4 4 8 8 8 16 4 16 4 8 16
6.05 5.92 5.83 3.85 3.40 5.68 5.93 2.80 1.48 2.47 2.11 0.93 2.94 2.66 6.73 5.95 0 0 0
4.23 3.97 4.13 2.44 2.49 2.77 1.64 2.17 1.81 2.35 1.60 1.67 1.62 1.09 3.47 2.66 0 0 0
0.03 0.03 0.42 0.11 0.11 1.8 120 0.03 0.03 0.47 0.03 0.03 5.8 125 NA NA 0.01 0.05 0.38
7 × 103 2 0 1 × 104 60 0 0 7 × 103 2 0 1 × 104 60 0 0 3 × 105 1.9 × 105 0 0 0
56.1 MB 500 KB 0 1.60 GB 3.73 MB 0 0 56.1 MB 500 KB 0 1.60 GB 3.73 MB 0 0 0 0 0 0 0
30
Lin et al. Slot assigned by semi−optimal policy
30 0
25
1
1
0
0
3
0
0
0
0
2
0
3 250
200
200
150
150
100
100
50
50
20
Expected Profit
Frequency of improvement
4
Profit of myopic policy Profit of semi−optimal policy Value function of semi−optimal policy
250
15
10
5
−5
0
5 10 Percentage of improvement
15
20
Fig. 7. Improvement of opti mal policy over myopi c policy. Parameters same as Fig. 6.
by maximizing the value function. Thus, as noted earlier, the profit of the myopic policy exceeds the profit of the optimal policy during much of the schedule building process. It is often only toward the latter part of the call-in period that the optimally constructed schedule begins to exhibit greater profit than the myopic policy. Also, although both methods transfer patients after the schedule capacity is reached, the optimal policy also selectively transfers patients before schedule capacity is attained. For example, Fig. 6 illustrates that at the beginning of the call-in period, the optimal policy will transfer type 1 patients. It will assign type 1 patients later, when it is less certain of getting type 2 patients to fill available capacity. In Fig. 6, the finalized schedule under the optimal policy is S = 02 11 01 10 with an expected profit $224.33, and the finalized schedule under the myopic policy is S = 21 20 01 10 with an expected profit $206.99. (Note that columns 1-4 correspond to slots 1-4, respectively. The top row gives the number of type 1 patients assigned to each slot, and the bottom row gives the number of type 2 patients assigned to each slot.) The optimal policy improves the expected profit by 8.38%. To verify the overall performance of the optimal policy, we use 100 call-in sequences with the same parameter settings as given above. Fig. 7 shows an average improvement of 6.05%. In Fig. 8, the schedule under the semi-optimal finalized policy is S = 02 11 11 10 with an expected profit $216.52. (Note that this is the same call-in sequence as in Figure 6). The semi-optimal policy improves the expected profit by 4.61% over myopic. As before, we use 100 call-in sequences with the same parameter settings as given above and provide the results in Fig. 9. The average improvement is 2.80%, 3.25% less than optimal.
0
0
1
1
2
2
0 0 1 1 Sequence of patient types
0
1
1
2
1
2
0
Fig. 8. Example of semi-optimal policy. I = 4, J = 2, N = 15 λ = 1.5.
If patient no-show is homogeneous, the call-in sequence has less impact on schedule performance. To see this, note that with heterogeneous no-shows, schedules differ in both the number and types of patients in each slot. Thus, exchanging the slots of two patients of different types changes the schedule and affects the expected profit. However, with homogeneous no-show, this is not true, since all patients have the same type. Thus, with homogeneous no-show, the optimal policy has less potential for improvement over the myopic policy. To test this, we experimented with single no-show patients. The parameter settings are the same as the examples in the previous subsection except the no-show 35
30
25 Frequency of improvement
0 −10
Downloaded At: 21:46 8 June 2011
2
20
15
10
5
0 −6
−4
−2
0 2 Percentage of improvement
4
6
8
Fig. 9. Improvement of semi-optimal policy over myopi c policy. Parameters same as Fig. 8.
Algorithms for scheduling no-shows
31
Slot assigned by optimal policy 1
2
1
0
3
3
0
1
0
0
0
0
0
0
0
Slot assigned by optimal policy using Aggr algorithm
0
0
0
0
0
0
0
0
0
0
Profit of myopic policy Profit of optimal policy Value function of optimal policy
Expected Profit
Expected Profit
150
2
0
0
0
0
0
3
1
3
0
1 250
200
200
150
150
100
100
50
100
100
50
50
50
0
0
1
1
1
0
1
1
0
1
1 0 1 1 1 0 1 Sequence of patient types
0
1
1
1
1
1
1
1
1
0
1
1
2
2
0 0 1 1 Sequence of patient types
0
1
1
2
1
2
0
Fig. 10. Example of the optimal policy in homogeneous no-show, p = 0.6.
Fig. 12. Example of opti mal policy using Aggr algorithm I = 4, J = 2, N = 15, AL = 2, λ = 1.5.
rate is a scalar, which is the average of the heterogeneous example. In Fig. 10 and Fig. 11, the optimal policy gives a final schedule SOpt = (3 1 2 0) and the myopic policy gives SMyo = (2 2 1 1). The optimal policy assigns only 1 patient in slot 2, while the myopic policy assigns 2 patients in slot 2. This prevents the myopic policy from assigning more patients to slot 1 and 3, and thus the myopic policy cannot reach the optimal. But the difference is small, approximately 0.19%. We repeated this for 100 randomly generated call-in sequences and found very similar results. Thus, the optimal is most effective for heterogeneous noshow patients.
These experiments with small systems lead us to believe that there could be great value in non-myopic scheduling for larger systems, and thus we proceed to evaluate our heuristics.
Slot assigned by optimal policy 210
1
1
1
0
1
1
0
1
1
0
1
1
1
0
1
0
1
1
1
1
1
1
1
1
210
Profit of myopic policy Profit of optimal policy Value function of optimal policy
205
205
200
200
Expected Profit
Downloaded At: 21:46 8 June 2011
1
200
150
0
0
Profit of myopic policy Profit of optimal policy using Aggr algorithm Approximate value function of optimal policy using Aggr algorithm
250
200
0
195
1
1
1
0
1
1
0
1
1 0 1 1 1 0 1 Sequence of patient types
0
1
1
1
1
1
1
1
Fig. 11. Same example as Figure 10 with an amplified Y axis.
1
195
6.2. Performance of the aggregation algorithm For the computation in this subsection, we uniformly aggregate an equal number of slots of the original schedule based on a specified aggregation level (AL). Thus, for an eight-slot schedule, if AL = 2, every two consecutive slots are aggregated giving the aggregated schedule four slots; if AL = 4, every four slots are aggregated giving the aggregated schedule two slots. Fig. 12 provides the results of optimal policy using aggregation (Aggr) algorithm for the four slot system with the same settings as in Section 6.1. The aggregation algorithm reduced the improvement from 8.38% to 5.61%. The final schedule is S = 12 01 11 00 , which has the same number of patients in each type as the optimal policy, but different assignments. Fig. 13 provides the histogram of improvement, with an average improvement of 5.92%. The results for the semi-optimal policy using aggregation is provided in Fig. 14 with a finalized schedule of S = 12 01 20 01 . Fig. 15 provides the results for 100 call-in sequences with the same parameter settings as given above. The average improvement is 1.48%, 1.32% less than optimal policy using backward induction. Now consider an eight-slot system, which is intractable for the optimal policy with the backward induction method. When AL = 2, the average improvement over myopic policy is 3.85%. When AL = 4 the average improvement is 3.40%. With AL = 2 the offline computation time
32
Lin et al.
30
35
30
25
Frequency of improvement
Frequency of improvement
25
20
15
10
20
15
10
5
5
0 −10
−5
0 5 Percentage of improvement
10
Fig. 13. Improvement of opti mal policy using Aggr algorithm over myopi c policy. Parameters same as Fig. 12.
−2
0
2 4 Percentage of improvement
6
8
Fig. 15. Improvement of semi-optimal policy using Aggr algorithm over myopi c policy. Parameters same as Fig. 14.
6.3. Performance of Si m(Myo) Algorithm for the decision table is about 2 hours and the table size is 1.6GB. With AL = 4, the decision table is generated in about 10 seconds and its size is 3.74MB. Thus, computational requirements are significantly reduced by increasing aggregation level. Similar results are also provided by the semi optimal policy (see Table 3). Although aggregation appears to handle eight slot problems well, it still requires extensive computation which may not tractable for larger systems. The next subsection applies the simulation based approach to these larger systems.
This subsection provides computational results for the simulation method. We first examine the number of replications, K, in the algorithm. Fig. 16 shows that the value function estimate appears to converge after about 200 replications for a four-slot example. However, K need not to be so large in the Si m(Myo) algorithm. This is because the Si m(Myo) algorithm uses value function estimates to rank slots for assignment. Thus, we need only sufficient replications such that the slot ranking converges. Fig. 17 illustrates the convergence of the slot ranking for the four-slot example. In this example, the ranking appears to converge
Slot assigned by semi−optimal policy using Aggr algorithm 250
1
3
2
1
0
0
3
0
0
0
0
1
0
231
4 250
Profit of myopic policy Profit of semi−optimal policy using Aggr algorithm Approximate value function of semi−optimal policy using Aggr algorithm
230
200
200
150
150
100
100
50
50
229 Approximate value function
0
Expected Profit
Downloaded At: 21:46 8 June 2011
0 −4
15
228
227
226
225
224
0
0
1
1
2
2
0 0 1 1 Sequence of patient types
0
1
1
2
1
2
0
Fig. 14. Example of semi-optimal policy using Aggr algorithm. I = 4, J = 2, N = 15 λ = 1.5.
223
0
50
100
150
200 250 300 350 Number of Monte Carlo simulation
400
450
500
Fig. 16. Convergence of approximate value function. I = 4, J = 2, n = 10, N = 24, λ = 1.5.
Algorithms for scheduling no-shows
33 25
Transfer Assign to slot 1 Assign to slot 2 Assign to slot 3 Assign to slot 4
230
20
Frequency of improvement
Approximate value function
228
226
224
222
15
10
5
220 5
10
15
20 25 30 35 Number of Monte Carlo simulation
40
45
0
50
2
3
4
5
6 7 8 Percentage of improvement
9
10
11
Fig. 17. Approximate value functions of (S + i j , n + 1) ∀i . I = 4, J = 2, n = 10, N = 24, λ = 1.5.
Fig. 19. Improvement of opti mal policy using Si m(Myo) algorithm over myopi c policy. Parameters same as Fig. 18.
after only about 10 to 15 simulation runs. This method, although experimental, can be used to estimate the required number of replications. In the following we will set K = 10. The scheduling evolution of Si m(Myo) algorithm is very close to the optimal policy for a four-slot system. The average improvement over myopic policy is 5.83%, 0.22% less than the optimal policy does. The results of four- and eightslot systems using the Si m(Myo) algorithm are provided in Table 3. We now illustrate a sixteen-slot system which can present a real schedule in a half-day session for four hours with 15 minutes slots. Fig. 18 provides an example of the sixteenslot model with optimal policy. The final state given by
Si m(Myo) algorithm is S = 02 01 11 01 01 01 11 01 01 11 01 01 02 01 01 10 with an expected of $996.16. The myopic policy generates S = 3 1 2 1 1 0 profit 2 0 2 0 0 2 0 2 1 0 with an expected profit of $927.85. 0101110101201010 The average improvement for 100 sequences is 5.93% as shown in Fig. 19. Fig. 20 illustrate the result of the same case with the semi-optimal policy. The final stategiven by Si m(Myo) algorithm is S = 02 01 11 11 01 20 01 01 11 20 11 01 11 20 01 with an expected profit of $964.01. The first 16 patient requests are guaranteed for assignments, only a few later requests in the overbooking procedure are transferred to another service session. As a consequence, in Fig. 21, the average improvement of 100 sequences drops to 2.66%. Slot assigned by semi−optimal policy using Sim. algorithm
Slot assigned by optimal policy using Sim. algorithm
6 110161 5 0100 3 0 4 2113130 6 91414150 0 0120 0 0 4 9138 7 0 0110 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 4 0 0136 0 0 0 0 0 0150 8 0 0 0 0 0 0 3 0 0 0 1 0 0 0 11214109 0 0 5 0 0 0 0 0 0 0 0 7 0 0 0 0 0 7 3 01016211130
Profit of myopic policy Profit of semi−optimal policy using Sim. algorithm Approximate value function of semi−optimal policy using Sim. algorithm
1000
1000
800
800
600
600
400
400
200
200
0
121122010101212101111210121112222210211111111211111110112222 Sequence of patient types
0
Fig. 18. Example of opti mal policy using Si m(Myo) algorithm. I = 16, J = 2, N = 60, λ = 1.5.
Expected Profit
Profit of myopic policy Profit of optimal policy using Sim. algorithm Approximate value function of optimal policy using Sim. algorithm
Expected Profit
Downloaded At: 21:46 8 June 2011
0
1000
1000
800
800
600
600
400
400
200
200
0
121122010101212101111210121112222210211111111211111110112222 Sequence of patient types
0
Fig. 20. Example of semi-optimal policy with Si m(Myo) algorithm. I = 16, J = 2, N = 60 λ = 1.5.
34
Lin et al. We tested 100 call-in sequences for a four-slot system and a sixteen-slot system, parameter settings are the same as the system in the previous subsections. The result is as shown in Table 3. For the four-slot system, the improvement of sequential optimal policy is 6.05% over the myopic policy, 0.67% less than non-sequential enumeration. For the sixteen-slot system, the improvement of sequential optimal policy using simulation algorithm is 5.93%, only 0.02% less than the LS algorithm. Our proposed sequential optimal policy falls just short of the non-sequential optimum.
25
Frequency of improvement
20
15
10
5
0
7. Concluding remarks
0
1
2
3 4 Percentage of improvement
5
6
Downloaded At: 21:46 8 June 2011
Fig. 21. Improvement semi-optimal policy with Si m(Myo) algorithm over myopi c policy. Parameters same as Fig. 20.
6.4. Benchmarking with non-sequential methods To further investigate the performance of our methods, we use non-sequential scheduling methods as a benchmark. Non-sequential methods schedule a given set of patients with known no-show probabilities. There is no call-in period and no random sequence effect to limit the quality of the generated schedules, as with sequential scheduling. As a result, an optimal sequential schedule for a given set of patients can be no better than an optimal nonsequential schedule for the same set of patients, that is, the non-sequential schedule provides a bound for the sequential schedule. Exhaustive enumeration and Local Search (LS) are two commonly used methods for non-sequential scheduling. Exhaustive enumeration generates all possible schedules, and thus guarantees the global optimum, but is computationally tractable for only small systems. Local search explores the neighborhood of an evolving schedule, switching patients from slot to slot when incremental improvements can be made. Local search can achieve the global optimum for single no-show (Kaandorp and Koole, 2007) and heuristically good schedules for multiple no-shows (Zeng et al., 2010). In the following, we use enumeration as the benchmark for small systems and local search, specifically a variant of the method presented by Zeng et al. (2010), as the benchmark for larger systems. According to Zeng et al. (2010), the surface of the profit function is not multi-modular, and thus multiple local optima exist. The LS algorithm (Zeng et al., 2010) converges quickly to a local optimum but needs an initial schedule to start. Our algorithm chooses multiple initial schedules (each with the same set of given patients) to start the local search algorithm and chooses the maximal peak to be the benchmark. Some of the initial states are random and some are computed heuristically.
The optimal policy and semi-optimal policy both improve the schedule over the myopic policy. Aggregation and the simulation based approximate algorithms are effective in handling the curse of dimensionality for the cases considered in this paper. Aggregation reduces the dimension of the state space, and simulation reduces the number of states evaluated. They can be used together to contribute more on saving computation for a practical system with larger size. Examples show that the semi-optimal policy is better than the myopic policy. The improvement percentage is 2.5–3%. This improvement is not as small as it looks. The estimated total daily revenue for a clinic in South Carolina is $9,924.65 (Moore et al., 2001). Assuming 250 working days per year, a 2.8% improvement from the myopic policy provides as approximate increase in clinic profit of $70K per year. Considering that sophisticated overbooking methods such as those provided in Muthuraman and Lawley (2008) do not seem to be widely used, the benefit of our method could be even greater in practice. Further, this improvement of clinic revenue implies that more patients actually get access to care. The sequential optimal policy performs very close to static enumeration when schedules created by each are compared. Schedules create with our approximate algorithms perform almost as well as static schedules created using heuristic methods. Thus, schedules created by our methods approach what can be realistically achieved if the set of patients to be scheduled were known and the best methods for finding the optimal static schedule were applied. The study has the following limitations. Firstly, due to computational complexity, only a small number of no-show classes was considered in the computational study. It is not currently clear how many no-show classes, time stages, and schedule slots can be simultaneously handled. Secondly, the slot model considers only the provider consultation stage of the patient visit, whereas clinic practice has multiple stages such as check-in, vitals, and lab tests. Thirdly, the call arrival process is assumed to be stationary, which in practice might not be true, and finally, the model optimizes the schedule
Downloaded At: 21:46 8 June 2011
Algorithms for scheduling no-shows in time slots for a single day, but not across days. We note that this work is not unique in these limitations, and in fact most published work exhibits many of the same limitations. While there are many directions for future work, we will mention two that we think are important and interesting. As noted earlier, implementation and validation are the ultimate goals of our work. We are currently pursuing this direction with several clinical partners and their software vendors. Such work is challenging because most clinics use the limited appointment scheduling applications provided in their vendor’s practice management package. Thus, real implementation requires integration with those packages, which, as the reader might infer, implies the successful resolution of a variety of difficult issues. We also want to integrate call reminder systems and other interventions intended to reduce no-show behavior with our scheduling methods. We note that reminder services are costly and thus need to be used efficiently. Relevant questions here are as follows: For a given a schedule, which patients in the schedule should we remind and when should we decide? If we decide at the time of scheduling, then we might assume a lower no-show probability for the patient (but this increases model complexity and computation). If the decision is delayed and the schedule is overbooked, excessive reminding could result in higher than desired patient waiting and overtime costs. Thus, optimally integrating scheduling and reminder systems is both intellectually challenging and highly relevant from a practical point of view.
Acknowledgment This work was supported by a grant from National Science Foundation. We thank the Regenstrief Center at Purdue University and the physicians and staff of the Indiana University Medical Group and Wishard Primary Care Clinics of Indianapolis, Indiana for their participation and support.
References Alafaireet, P., Houghton, H., Petroski, G., Gong, Y., and Savage, G. T. (2010). Toward determining the structure of psychiatric visit nonadherence. Journal of Ambulatory Care Management, 33(2), 108–116. Bean, A. G. and Talaga, J. (1995). Predicting appointment breaking. Journal of Health Care Marketing, 15(1), 29–34. Bech, M. (2005) The economics of non-attendance and the expected effect of charging a fine on non-attendees. Health Policy, 74(2), 181– 191. Cayirli, T. and Veral, E. (2003). Outpatient scheduling in health care: a review of literature. Production and Operations Management, 12(4), 519–549. Chakraborty, S., Muthuraman, K., and Lawley, M. (2010). Sequential clinical scheduling with patient no-shows and general service time distributions. IIE Transactions, 42(5), 354–366. Daggy, J., Lawley, M., Willis, D., Thayer, D., Suelzer, C., DeLaurentis, P.C., Chakraborty, S., and Sands, L. (In Press). Using no-show modeling to improve clinic performance. Health Informatics.
35 Green, L. V., Savin, S., and Wang, B. (2006). Managing patient service in a diagnostic medical facility. Operations Research, 54(1), 11–25. Gupta, D. and Wang, L. (2008). Revenue management for a primary-care clinic in the presence of patient choice. Operations Research, 56(3), 576–592. Guse, C. E., Richardson, L., Carle, M., and Schmidt, K. (2003). The effect of exit-interview patient education on no-show rates at a family practice residency clinic. The Journal of the American Board of Family Practice, 16, 399–404. Hassin, R. and Mendel, S. (2008). Scheduling arrivals to queues: A singleserver model with no-shows. Management Science, 54(3), 565–572. Hixon, A. L., Chapman, R. W., and Nuovo, J. (1999). Failure to keep clinic appointments: Implications for residency education and productivity. Family Medicine, 31(9), 627–630. Johnson, B. J., Mold, J. W., and Pontious, J. M. (2007). Reduction and management of no-shows by family medicine residency practice exemplars. Annals of Family Medicine, 5, 534–539. Kaandorp, G. and Koole, G. (2007). Optimal outpatient appointment scheduling. Health Care Management Science, 10(3), 217–229. Kim, S. and Giachetti, R. E. (2006). A stochastic mathematical appointment overbooking model for healthcare providers to improve profits. IEEE Transactions on Systems, Man and Cybernetics, Part A, 36(6), 1211–1219. LaGanga, L. R. and Lawrence, S. R. (2007a). An appointment overbooking model to improve client access and provider productivity. Technical report, University of Colorado at Boulder. LaGanga, L. R. and Lawrence, S. R. (2007b). Clinic overbooking to improve patient access and increase provider productivity. Decision Sciences, 38, 251–276. Lesaca, T. (1995). Assessing the influence of a no-show fee on patient compliance at a cmhc. Administration and Policy in Mental Health and Mental Health Services Research, 22(6), 629–631. Liu, N., Ziya, S., and Kulkarni, V. G. (2010). Dynamic scheduling of outpatient appointments under patient no-shows and cancellations. Manufacturing and Service Operations Management, 12(2), 347–264. Lowes, R. (2005). Practice pointers: How to handle no-shows. Medical Economics. Moore, C. G., Wilson-Witherspoon, P., and Probst, J. C. (2001). Time and money: Effects of no-shows at a family practice residency clinic. Family Medicine, 33, 522–527. Muthuraman, K. and Lawley, M. A. (2008). A stochastic overbooking model for outpatient clinical scheduling with no-shows. IIE Transactions, 40(9), 820–837. Patrick, J., Puterman, M. L., and Queyranne, M. (2008). Dynamic multipriority patient scheduling for a diagnostic resource. Operations Research, 56(6), 1507–1525. ISSN 0030-364X. Rust, C. T., Gallups, N. H., Clark, W. S., Jones, D. S., and Wilcox, W. D. (1995). Patient appointment failures in pediatric resident continuity clinics. Archives of Pediatrics and Adolescent Medicine, 149(6), 693– 695. Shonick, W. and Klein, B. W. (1977). An approach to reducing the adverse effects of broken appointments in primary care systems: development of a decision rule based on estimated conditional probabilities. Medical Care, 15(5), 419–429. Vanden Bosch, P. M. and Dietz, D. C. (2000). Minimizing expected waiting in a medical appointment system. IIE Transactions, 32(9), 841– 848. Vanden Bosch, P. M. and Dietz, D. C. (2001). Scheduling and sequencing arrivals to an appointment system. Journal of Service Research, 4(1), 15–25. Vanden Bosch, P. M., Dietz, D. C., and Simeoni, J. R. (1999). Scheduling customer arrivals to a stochastic service system. Naval Research Logistics, 46(5), 549–559. Zeng, B., Turkcan, A., Lin, J., and Lawley, M. (2010). Clinic scheduling models with overbooking for patients with heterogeneous no-show probabilities. Annals of Operations Research, 178(1), 122–144.
36 Biographies Ji Lin is a PhD Candidate in the Weldon School of Biomedical Engineering at Purdue University. His research interests include stochastic optimization, simulation, statistics and decision making in healthcare and other systems. He has a Bachelor’s degree from the School of Electronic, Information and Electrical Engineering at Shanghai Jiao Tong University in 2000, and a Master’s degree in Industrial Engineering from Texas Tech University (2004). Before attending graduate school, he worked for TOSE Co., Ltd. as a software engineer. He is a member of IIE and INFORMS.
Downloaded At: 21:46 8 June 2011
Kumar Muthuraman is currently an Associate Professor at the McCombs School of Business at the University of Texas in Austin. He received his Ph.D in 2003 from Stanford University and has previously been an Assistant Professor at the School of Industrial Engineering at Purdue University. His primary interests are in computational methods for stochastic
Lin et al. control problems especially in quantitative finance. His secondary interests include stochastic optimization problems in inventory management and healthcare. Mark Lawley is Professor of Biomedical Engineering in the Weldon School of Biomedical Engineering at Purdue University. Before joining Biomedical Engineering in 2007, he served nine years as Assistant and Associate Professor of Industrial Engineering, also at Purdue, two years as Assistant Professor of Industrial Engineering at the University of Alabama, and he has held engineering positions with Westinghouse Electric Corporation, Emerson Electric Company, and the Bevill Center for Advanced Manufacturing Technology. As a researcher in academics, he has authored over 100 technical papers including book chapters, conference papers, and refereed journal articles, and has won four best paper awards for his work in systems optimization and control. He received the PhD in Mechanical Engineering from the University of Illinois at Urbana Champaign in 1995 and is a registered Professional Engineer in the State of Alabama.
