DYNAMIC SCHEDULING TO MINIMIZE LOST SALES ... - CiteSeerX

DYNAMIC SCHEDULING TO MINIMIZE LOST SALES SUBJECT TO SET-UP COSTS Eungab Kim and Mark P. Van Oyen

Joseph L. Rotman School of Management University of Toronto Toronto, Ontario, Canada M5S 3E6 Department of Industrial Engineering and Management Sciences Northwestern University Evanston, IL 60208-3119

Corresponding Author: Mark P. Van Oyen 847-491-7008 FAX: 847-491-8005 E-mail: [email protected] URL: http://primal.iems.nwu.edu/ vanoyen/ To appear: Queueing Systems (QUESTA) 1998

Keywords: make-to-stock scheduling, CONWIP, set-up costs, closed queueing network, heuristic policy, Markov decision process December 14, 1996 Revised July 21, 1998

Dynamic Scheduling to Minimize Lost Sales Subject to Set-up Costs 1 Eungab Kim Joseph L. Rotman School of Management, University of Toronto, Toronto, Ontario, CANADA M5S 3E6 Mark P. Van Oyen Department of Industrial Engineering and Management Sciences Northwestern University, Evanston, IL 60208-3119

Abstract

We consider scheduling a shared server in a two-class, make-to-stock, closed queueing network. We include server switching costs and lost sales costs (equivalently, server starvation penalties) for lost jobs. If the switching costs are zero, the optimal policy has a monotonic threshold type of switching curve provided that the service times are identical. For completely symmetric systems without set-ups, it is optimal to serve the longer queue. Using simple analytical models as approximations, we derive a heuristic scheduling policy. Numerical results demonstrate the eectiveness of our heuristic, which is typically within 10% of optimal. We also develop and test a heuristic policy for a model in which the shared resource is part of a series network under a CONWIP release policy.

1. Introduction We begin with a production control perspective to the system treated, then describe it from a queueing-theoretic perspective. We consider the problem of scheduling a shared processor in a twoclass M=M=1 make-to-stock production system with lost sales. We focus on systems with tandem production lines for each job type. Demands for nished goods of type n 2 f1
2g arrive according to independent Poisson(n ) arrival processes. Demands of type n are met from the output buer of line n which contains the nished good inventory (FGI). If the FGI buer for type n jobs is empty, the customer is lost and the system incurs a lost sales cost, Sn . As we will argue later, it is frequently advisable to employ a release (admission control) policy that enforces a constant total work-in-process (WIP) Mn for each job type n. With an innite supply of raw materials, this reduces to a system with dynamics such that a new job is admitted to the input of line n at each instant a demand is satised. A set-up switching cost is incurred upon switching from one job type to the other. We wish to prescribe a control/scheduling policy to dynamically select the type of job to be served next so as to minimize the long run average cost due to lost sales and switching of the server. We will argue that systems with multiple stages in the serial production process can be approximated based on a two-stage model. It is instructive to take a closer look at our model as an innovation in the control of queueing systems. Our model captures the issue of scheduling a shared processor in a two-class tandem, 1 This material is based upon work supported by the National Science Foundation under Grant No. DMI-9522795 to Northwestern University and was performed as part of the rst author's dissertation.

1

Markov queueing network. The network is closed with Mn jobs of type n. We formulated this model as a closed queueing network to capture a common and eective mechanism for admission control. The rst station of the network has a single processor that sequentially serves jobs of both types with exponential processing times of rate n when serving type n. Separate queues are maintained at station 1 for job types 1 and 2. The second stage of the system contains two stations, one for jobs of type 1 and the other for type 2. Processing times are exponential(n ), and it is the queues of these stations which we previously interpreted as the output/FGI buers of the shared processor. The service periods at the station for type n at stage 2 replace the interarrival periods of the previous production interpretation of the model. Each job class can be modeled as a closed Markov tandem queueing system however, the shared server at station 1 couples the two sets of tandem queues. Even if a rst-come-rst-served discipline is used, it is dicult to capture the interaction of the systems with precision. Moreover, the scheduling policy employed is a crucial determinant of system performance. As we will see, the structure of an optimal policy is complex. The goal is to identify a scheduling policy which minimizes the innite horizon expected cost per unit time. We are particularly interested in costs that arise from two sources. First, we include a setup cost Kn for queue n to discourage frequent switches. Second, we incur a \lost sales" cost Sn at each instant the processor of type n at the second stage completes a service period and the result is a ctitious transition due to processor starvation (that is, idling without jobs in its queue). Of course, ctitious services at the output stage are preempted by the arrival of real jobs. Equivalently, we can penalize the starvation of the output processor of type n at a rate of Sn n per unit of time that server is starved. This choice for the cost structure follows from the fact that we take the demand process and the station processing times to be uncontrollable. Thus, a policy will provide good performance provided the server switches infrequently while maintaining a nonzero queue at both the output stage of type 1 and type 2. The problem presented here contains a dicult, fundamental tradeo that can be interpreted from the system's prospective as a line balancing problem and also from the server's point of view as a processor sharing problem. From the viewpoint of the system, the server should schedule service of the job types to achieve a balance between the levels of the output buers based on the demand rate and the switching penalties. To elaborate, suppose the server serves line n = 1. Because of the lost sales cost, while serving type 1 and increasing the output buer level of line 1 (or decreasing the number of cards in card queue 1), the server should prevent type 2 job from starvation of the demand process (i.e. having an empty output buer). Since a switching cost is incurred upon switching from line 1 to 2 however, the server has an incentive to minimize the rate of switching per unit time. We can also consider the server's perspective. On the one hand, the workstation must \work ahead" to allow time to switch to the other line and serve it while avoiding stockouts during pro2

duction of the other product type. On the other hand, our desire to avoid large inventories and long lead times is consistent with our formulation of the problem as a closed queueing network model. The processor sharing service discipline model is typically used in computer science applications to describe situations in which the processor equally divides its service among the competing demands. Physical constraints preclude such processor sharing in many systems, but the concept remains relevant. Based on the costs and system parameters, an optimal policy denes switching regions in the state space that optimize the long run fraction of time spent in each queue, and the statistics of the dynamic \run lengths" (batch sizes) in each queue. The model we formulate is a basic one that can be applied to a variety of systems. Computer and information systems frequently have a server or gateway which must process every job regardless of class before routing the incoming \job" to a type-specic processing network. Our modeling eort was directly motivated by the following class of manufacturing and production control problems. To achieve a balance between high job throughput on the one hand, and short cycle times and work in process (WIP) levels on the other, Hopp and Spearman 8] and Spearman et al. 17] have proposed constant work in process (CONWIP) job release (admission control) policies for production lines or routings. Reduced WIP levels and lead times provide many strategic advantages (see Spearman and Zazanis 18]) including reduced buer/storage capacity requirements, reduced losses due to defects, improved ability to meet changes in demand, and increased revenues in cases where short due dates allow the rm to charge higher prices. Both practice and theory indicate that CONWIP is an eective admission control or release policy for production systems, and mean-value analysis has been helpful in easily analyzing the performance (throughput, cycle (mean sojourn) time, and average queue lengths at any station) of serial CONWIP production lines (see Hopp and Spearman 8]). A CONWIP system possesses a xed number of cards. Jobs are released with an associated card when one becomes available from the card queue. At the end of the line, each completed job surrenders its card, which is then made available to release a new job into the system. The total number of cards is set to achieve nearly 100% utilization of the bottleneck workstation on the line (routing). Thus, the CONWIP achieves job ow control in a manner essentially similar to the \window ow control" mechanism (with xed window size) employed in some data communication protocols such as TCP/IP. Our work focuses on scheduling an (expensive) workstation that is shared by two multi-stage serial production lines that operate simultaneously under a CONWIP release policy for that job type. The shared workstation can produce parts for only one line at a time moreover, a lump sum switching cost Kn is required to switch to line n. The workstation receives perfect information concerning the state of the system from the production control system (e.g., the CONWIP controller). Finished goods are accumulated at the end of each line to fulll job demands, which arrive 3

randomly over time. As a starting point, we focus our analysis on a model of a single station with two job classes, a single server, WIP input buers, and FGI output buers. Demands to an empty FGI buer incur Sn per unmet demand. The CONWIP release policy for this station becomes a KANBAN cell with a xed card count for the total WIP and FGI of each job type. When thinking about a stochastic model for two serial production lines with an interaction in the form of a shared processor, it is natural to think of a Jackson network model however, a number of complications arise. First, we observe that there are two distinct classes of jobs with distinct routing in the network. Thus, models with a single job class and independent and identically distributed (i.i.d.) routing at the output of the shared processor do not capture this important feature. Similarly, one might think of the problem of Klimov 11], originally solved in 1974. Klimov's problem is the subject of continuing work, including that of Bertsimas and Ni~noMora 2], who used generalized conservation laws to characterize the polyhedral space of achievable performance and establish indexability. Klimov was the rst to show the existence of an optimal priority (index) ordering among the classes, provided we have a single server attending an open Jackson network, innite buers, linear job holding costs, and costless switching of the server. The problem we address violates each of these assumptions. Turning to the resource allocation issue at the heart of our problem, the multi-armed bandit (dynamic resource allocation index) approach pioneered by Gittins represents a signicant and relevant approach. Although we employ insight from reward rate indices in computing our heuristic, our problem requires additional isights, because two key features of the standard bandit problem are decisively violated: First, there is the assumption that at any instant, only the bandit process being served evolves, while the others remain frozen with respect to state. In contrast to this, our model can be classied as one of the so-called \restless bandits" (see Whittle 23]) in which all bandits evolve simultaneously. Second, the standard theory assumes that changes in resource allocation are costless. Instead, the switching costs of our model preclude viewing jobs in isolation. Rather, we must take into account the fact that the number of jobs consecutively served at a queue (bandit process) directly impacts the expected reward per unit time earned. Because the proposed research addresses scheduling a single server in a multi-class queue subject to set-up times, it is also related to polling system models and single-machine scheduling problems. Polling systems are resource allocation models that have been applied to a wide variety of settings in which a common resource or medium is shared by many users or classes of jobs (see Levi and Sidi 12] and Takagi 19]). In these models, typically a common resource (server) sequentially provides service (polling) to a number of distinct classes of jobs from innite queues, according to some scheduling policy. Performance analysis is the common approach. When optimization is considered, the performance objectives of these models tend to focus on throughput and minimization of total weighted average queue lengths subject to set-up times. 4

Analyses of the dynamic scheduling problem for multi-class make-to-stock systems without setup penalties are found in Zheng and Zipkin 24], Wein 22], Ha 7], Veatch and Wein 20], and Kim and Van Oyen 9], while Qiu and Loulou 14], Markowitz and Wein 13], Federgruen and Katalan 4], 5], and Elhafsi and Bai 3] include set-up times and/or costs. Except for Kim and Van Oyen, all of these papers allow backorders, and Veatch and Wein also allow lost sales. Zheng and Zipkin 24] studied a two-class M/M/1 homogeneous system without setups in which the production authorization is triggered whenever the inventory level of either class drops below a given base stock level. They showed that with respect to minimizing the costs, it is optimal to serve the class with the larger number of jobs to be processed (i.e., lower inventory level). Wein 22] studied a multi-class system with general arrival process and service time distributions. Under heavy trac, Wein addressed the scheduling problem using a Brownian motion approximation. He provided a \parametric" scheduling policy where the production authorization is triggered when the weighted sum of inventory levels drops below a threshold value or some classes are in danger of backorder. The production priority over classes is given as the 00b c00 rule. That is, if some classes are in danger of backorder, the top priority is assigned to a queue, among these classes, with the largest bn n index where bn is the backorder cost and n is the average service rate for class n. If no classes are in danger of backorder, the top priority is given to a queue with the smallest cn n index where cn is the nished goods inventory holding cost for class n. Ha 7] studied a two-class M/M/1 heterogeneous system which is similar to the model we consider except that he allowed unmet demands to be backordered and included inventory holding costs, but he did not include any set-up penalties. He proved that the b rule is optimal when both classes experience backorder and, provided that both classes have the same service rates, there exist two monotonic threshold switching curves one for production authorization and one for production priority. He also proved that if the initial inventory levels are not greater than the base stock levels, a base stock model utilizing a switching curve for production priority is optimal. Veatch and Wein 20] studied a M/M/1 multi-class system models with backorders and with lost sales. They developed several index policies for production priority and idling policies for production authorization and numerically investigated the relationship between these policies. Kim and Van Oyen 9] addressed systems with nite buers and lost jobs, a class of problems for which few results have appeared. For problems with holding costs and without set-ups under the criterion of total expected discounted cost, we proved that the optimal policy has a monotonic threshold switching curve, provided that the cost of queueing delay is less than that of job rejection. Qiu and Loulou 14] studied the issue of the truncation eect that occurs when nding the optimal policy using dynamic programming algorithms in case that the unmet demands are backordered. They provided a bound on the error that results from truncation of the state space and conjectured the existence of the monotonic threshold switching curve which is optimal for the 5

two-class system. Markowitz and Wein 13] work toward a framework for the generation of heuristics to solve single-station systems with setup costs or times and due-dates in mixed make-to-stock and maketo-order systems. They restrict themselves to the class of cyclic policies and employ heavy-trac averaging principles. Other work on periodic base-stock policies for the \stochastic economic lot scheduling problem" includes the work of Federgruen and Katalan 4], 5]. Elhafsi and Bai 3] consider a make-to-stock problem with a deterministic, constant demand rate for nished goods (the inventory state is continuous). Control of the production rate is continuous. They model a single machine that is subject to exponentially distributed times to failure and to repair (independent of the production rate). There are two job types and exponential setup times. Assuming linear holding cost rates for holding positive inventory and linear penalties for backlogged jobs, they numerically analyze the structure of optimal and heuristic control policies. The contributions of this paper include the formulation of an appropriate model for scheduling in CONWIP and KANBAN systems with lost sales, the investigation of the structure of an optimal scheduling policy, and the invention of an eective heuristic policy. For cases with zero switching costs, we show that there exists a monotonic threshold type of switching curve provided that the service times are identical. For completely symmetric systems, we show that it is optimal to serve the longer queue. To the best of our knowledge, this is the rst eort to characterize an optimal policy for our model of multi-class make-to-stock systems with lost sales. We then proceed to develop heuristic policies for the shared resource as a single station and in a network context. The success of our heuristic in a CONWIP network provides some evidence supporting our conviction that our model of a single station shared resource is useful in large networks, in which it is practical to perform scheduling subject only to local network information (input buer states at the shared server and the buer states immediately downstream). The rest of this paper is organized as follows: Section 2 provides a dynamic programming formulation of the single station model. We analyze the processor sharing problem with a threefaceted approach. First, in Sections 3 and 4 we characterize the optimal policy that minimizes the operating costs for systems without set-ups. Second, in Section 5 we develop simple, precomputable heuristics based upon simple approximate models of the dominant issues that govern an optimal policy. This results in a heuristic scheduling policy which possesses a monotonic threshold curve. Moreover, it is optimal for symmetric systems without set-up costs. Third, Section 6 provides a numerical comparison of the performance of our heuristic with respect to an optimal policy. In Section 7 we extend the single workstation (shared resource) model to place the shared resource in a networked system with tandem queues for each job type. The shared resource is located at the second stage and the rst stage has two independent workstations, each of which feeds the shared resource. We develop three heuristic policies. Section 8 compares the performance of these 6

heuristics using simulation.

2. Problem Formulation A single server is to be allocated to jobs in a system of two job types, each of which has an input (WIP) buer and an output (FGI) buer. Service of job type n, n 2 f1
2g requires an exponential ;1 amount of time with mean ;1 n (0 < n < 1) and transfers a job from WIP buer n to output buer n. Successive services in node n are i.i.d. and independent of all else. Exogenous demands for job type n arrive to output buer n according to a Poisson process with strictly positive rate n (independent of all other processes). We assume that if a job of type n is available, the input buer of type n is increased by one otherwise, the demand is not met and both buers remain unchanged. Thus for n 2 f1
2g, the lengths of the input buer, xn (t), plus the output buer, yn (t),
remains constant: xn (t) + yn (t) = Mn for all t. Since this nite-buer system is inherently stable, P we assume that  = 2i=1 n 2 0
1) where n = n =n . A switching cost of Kn (0  Kn < 1) is incurred at each instant (including time 0) the server switches to queue n from queue j 6= n to process a job. We assume that the switching action is achieved instantaneously (zero switchover times). A lost sales cost, Sn , is incurred at each instant a demand of type n nds that output buer n is empty and is then rejected. A policy species, at each decision epoch, that the server either remain working at the present queue, idle in the present queue, or switch to another queue for service. Without loss of optimality, the class of admissible strategies, U , is taken to be the set of non-anticipative, stationary, nonrandomized, Markov policies that are based on perfect observations of the queue length processes (see Ross 15]). In addition, U is restricted to the class of greedy policies, which never idle in a nonempty queue. For simplicity in identifying the structural properties of an optimal policy, our analytical model allows service preemption and thus the set of decision epochs is assumed to be the set of all arrival epochs and service completion epochs. In developing and testing our heuristics, however, we make a non-preemptive service assumption. This is accomplished in the numerical dynamic programming (DP) by including an additional state component that prohibits alternative actions until the current service is completed. With IR+ (ZZ+ ) denoting the nonnegative reals (integers), let xn (t)
t 2 IR+ be the queue length (number of cards) of line n (either in the WIP queue or in service) at time t. Denote the state space Q by S = 2n=1 f0
1
: : :
Mn g  f1
2g where Mn is the number of cards that is used in line n. The state of the system at time t under policy  is described by the vector x(t) = (x1 (t)
x2(t)
n(t)) 2 S
where n(t) denotes that the server is located at queue n(t) at time t. Suppose at a decision epoch t, the state is (x1 (t)
x2(t)
n(t)), and let the action space be A = f1
2g. Then, action a(t) = n 2 A, where n 6= n(t), causes the server to switch to queue n and serve it if xn (t) > 0 otherwise, idle in queue n. The action a(t) = n(t) results in the service of a job in n(t) if xn(t) (t) > 0 otherwise, idle in the current empty queue. No other actions are possible. 7

Let fn (t) : t 2 IR+ g be the right-continuous process describing the location of the server at time t under policy  . Dene ;n (T ) = ft 2 0
T ) : n (t; ) 6= n
n (t) = ng to be the set of random instances of switching to type n under  . Let An (t) be the indicator of the arrival process in queue n at time t, that is, if a job of type n arrives at queue n, then An (t) = 1 otherwise, An (t) = 0. Dene !n (T ) = ft 2 0
T ) : xn (t; ) = Mn
An (t) = 1g to be the set of random instances that an arrival of type n nds the output buer n empty. The -discounted cost under policy  , given the initial state x(0), can be expressed as

82 19 < X ;
t 0 X = X @ A J" (x) = Tlim E e S + K n n 
!1 : n=1 t2
n (T ) t2;n (T )

(2.1)

The optimal scheduling problem considered here can be formulated as a discrete-time stochastic dynamic programming problem by using uniformization (see Bertsekas 1]). This uniformized version has a transition rate = 1 + 2 +  for all states. Without the loss of generality, we scale the time unit so that 1 + 2 +  = 1. Let  be the discount factor of this discrete-time Markov decision process (MDP). Denote by x = (x1
x2) the queue length vector of queues 1 and 2 and by n the server location. Let rt(x
n) (st (x
n)) denote the expected discounted cost to go from state (x
n) given that the remain (switch) action is taken at t. With the initial condition VT (x
n) = 0, the DP equation of the discrete time model is given as follows:

Vt(x
n) = minfrt(x
n)
st(x
n)g
t = 0
: : :
T ; 1


(2.2)

st(x
n) = Kn1 + rt(x
n  1)


(2.3)

where

rt (x
n) =

2 X

j =1

j Sj 11(xj = Mj ) +

2 X

j =1

n1= An x =

(

and

Dn x =

j Vt+1 (Aj x
n) + Vt+1(Dnx
n)


(

2 if n=1 1 if n=2,

((x1 + 1 ^ M1 )
x2) if n=1 (x1
(x2 + 1 ^ M2 )) if n=2,

(

((x1 ; 1 _ 0)
x2) if n=1 (x1
(x2 ; 1 _ 0)) if n=2.

8

(2.4)

3. Optimality of a threshold policy without set-ups In this section, we assume that there are no switching penalties (K1 = K2 = 0) and service
times are identical for both job types (1 = 2 = ). With respect to minimizing the total expected discounted costs over a horizon, T , we show that there exists a monotonic threshold switching curve. We begin with T 2 IN, then extend the result in a straightforward manner to the case of T = +1 and to the average cost per unit time criterion. We dene
!n Vt (x
m) = Vt(Anx
m) ; Vt(x
m).

In other words, !n Vt(x
m) is the marginal cost of holding one more job of type n when the server is serving queue m. We denote by  (x
m) the action in state (x
m). For concreteness, we assume that if rt(x
m) = st (x
m), the action remain is chosen. Lemma 1 states properties held by the t-stage optimal cost-to-go function Vt. Property (i) is a sucient condition to guarantee that a threshold policy is optimal and it is monotonic. We say Vt is supermodular and convex if it satises property (ii) and (iv), respectively. In particular, we say Vt is diagonally dominant if it meets property (iii) (see Ha 7] for terminology). Property (v) provides a upper bound on the marginal cost incurred when one more job is held in the system. Property (vi) is a technical property. This lemma can be proven as a special case (with zero holding costs) of Lemma 1 and Theorem 2, respectively, of Kim and Van Oyen 9], so we omit the proof. Lemma 1: With m = f1
2g, and n = f1
2g, we have (i) !n rt(x
n)  !n rt(x
n  1)
0  xn  Mn ; 1
0  xn1  Mn1
(3.1) (ii)

!n Vt(x
m)  !n Vt (An1 x
m)
0  xn  Mn ; 1
0  xn1  Mn1 ; 1


(3.2)

!n Vt(An1 x
m)  !n Vt (An x
m)
0  xn  Mn ; 2
0  xn1  Mn1 ; 1


(3.3)

!n Vt (x
m)  !n Vt(An x
m)
0  xn  Mn ; 2
0  xn1  Mn1


(3.4)

(iii) (iv) (v) !n Vt (x
m)  1 ;1f; ((n1 ++)g) n Sn
0  xn  Mn ; 1
0  xn1  Mn1
n1 T ;t

(vi)

n !nVt(x
m) < n Sn
0  xn  Mn ; 1
0  xn1  Mn1 :

(3.5) (3.6)

Property (i) of Lemma 1 states that the marginal cost of an additional customer in queue n given that the server remains at time t is greater when the server is in queue n  1 than for queue 9

n thus providing greater incentive to switch to queue n as xn increases. It is optimal to switch to n  1 if and only if st (x
n) < rt(x
n), which by (2.3) is equivalent to rt(x
n  1) ; rt (x
n) < ;Kn1 :

(3.7)

If this condition is satised, the version of (i) of Lemma 1 with n replaced by n  1 gives

rt (An1x
n  1) ; rt(An1 x
n)  rt (x
n  1) ; rt(x
n) < ;Kn1 :

(3.8)

If it is optimal to switch from n to n  1 with xn1 in queue n  1, then also switch with xn1 + 1, which guarantees that a threshold function exists and is optimal. Similarly, if (3.7) is not satised, then (i) of Lemma 1 implies

rt (Anx
n  1) ; rt(An x
n) rt (x
n  1) ; rt(x
n) ;Kn1 :

(3.9)

If xn is large enough that it is optimal to remain in queue n, then also remain with xn + 1, which establishes that the threshold function monotonically increases in xn . Thus, Lemma 1 is sucient to justify the following result. Theorem 1: There exists a T -stage optimal policy with a nonstationary monotonic threshold type of switching curve. That is, for t = 0
1
: : :
T and n = 1
2, there exists a threshold function #nt () : f1
2
: : :
Mn g ! f1
2
: : :
Mn1
1g, #nt (xn ) := inf fxn1 2 f1
2
: : :
Mn1 g : st (x
n) < rt(x
n)g

(3.10)

such that (a) it is optimal to switch from n to n  1 at t if xn1 #nt (xn ). (b) The threshold function #nt (xn ) is increasing in xn : #nt (xn )  #nt (xn + 1)
0  xn  Mn ; 1:

(3.11)

Now we extend the above results to the case of T = +1. Since the one stage costs for our scheduling problem are bounded and nonnegative, we have a monotone increasing sequence of value functions in the horizon T , V (x) = limt!1 Vt(x) where V is the optimal value function over an innite horizon (see Proposition 1, Ch 5 of Bertsekas 1]). This implies that Lemma 1 holds for V (x). Therefore, Theorem 1 also holds for V (x). Furthermore, by proposition A7.3 of Bertsekas 1], a Blackwell optimal policy exists for our  {discounted cost problem and by proposition A7.2, it is optimal for the average cost per unit time problem. Therefore, the optimal policy for the average cost per unit time criterion also has a monotonic threshold switching curve.

10

4. Characterization of an optimal policy for symmetric systems In this section, we consider a special case of our model in which both queues are symmetric (including M1 = M2 = M ) and have zero switching costs. With complete symmetry, x1 = x2 implies that an optimal policy is indierent to service in queue 1 or queue 2, because the state and dynamics are the same for either action. The following theorem says in (ii) that the optimal policy always serves the longer queue. Statement (i) indicates that the value function increases as the
number of jobs in the system is distributed in a less balanced way. Dene Axn = min(xn + 1
M )
and Dxn = max(xn ; 1
0). Theorem 2: For 1  xn  M
1  xn1  M ; 1
and xn > xn1 , (i) Vt(Dxn
xn1
m)  Vt(xn
Dxn1
m)
m = 1
2
(4.1) (ii)

Vt(xn
xn1
n  1) = st(xn
xn1
n  1):

(4.2)

Proof: (i) Suppose m = n  1 to be concrete, but because switches are costless, this part of the state is not important and Vt (x
n) = Vt (x
n  1)
8x 2 S . We distinguish combinations of actions in states (Dxn
xn1
n  1) and (xn
Dxn1
n  1). Suppose (i) and (ii) hold at time t. We show them for t ; 1.

1.  (Dxn
xn1
n  1) =  (xn
Dxn1
n  1) = remain:

rt;1(Dxn
xn1
n  1) = Vt(xn
xn1
n  1) +Vt(Dxn
Axn1
n  1) +Vt (Dxn
Dxn1
n  1)
rt;1(xn
Dxn1
n  1) = n Sn 11(xn = M ) + Vt(Axn
Dxn1
n  1) +Vt(xn
xn1
n  1) +Vt (xn
D2xn1
n  1) :

(4.3) (4.4) (4.5) (4.6) (4.7) (4.8)

If xn < M , by induction, (4.3)  (4.6) and (4.5)  (4.8). To show (4.4)  (4.7), we consider the following two cases: (a) If Dxn = xn1 , then (4.4) { (4.7) = 0 because Vt(xn1
Axn1
n1) = Vt (Axn1
xn1
n 1). (b) If Dxn > xn1 , then Dxn xn1 + 1 and by induction, (4.4)  (4.7). 11

If xn = M , by (vi) of Lemma 1, (4.3)  (4.6). The same justication is applied to the remaining terms. 2.  (Dxn
xn1
n  1) = remain and  (xn
Dxn1
n  1) = switch:

st;1(xn
Dxn1
n  1) = nSn 11(xn = M ) + Vt(Axn
Dxn1
n) +Vt (xn
xn1
n) +Vt (Dxn
Dxn1
n) :

(4.9) (4.10) (4.11)

Using the fact Vt(x
n) = Vt(x
n  1)
8x 2 S , (4.5) (4.11). The justication for the remaining terms is the same as case 1. 3.  (Dxn
xn1
n  1) = switch and  (xn
Dxn1
n  1) = remain: By monotonicity, (i) of Lemma 1, this case is impossible. 4.  (Dxn
xn1
n  1) = switch and  (xn
Dxn1
n  1) = switch:

st;1 (Dxn
xn1
n  1) = Vt(xn
xn1
n) +Vt(Dxn
Axn1
n) +Vt (Dn2 xn
xn1
n)


(4.12) (4.13) (4.14)

The only dierence from case 1 is to justify (4.14)  (4.11). First, if Dxn = xn1 , (4.14) { (4.11) = 0 because Vt(Dn2 xn
Dxn
n) = Vt (Dxn
Dn2 xn
n). Second, if Dxn > xn1 , then, Dn2 xn xn1 and by induction, (4.14)  (4.11). (ii)

st(xn
xn1
n  1) = n Sn11(xn = M ) + Vt+1(Axn
xn1
n) +Vt+1(xn
Axn1
n) +Vt+1 (Dxn
xn1
n)


(4.15) (4.16) (4.17)

rt(xn
xn1
n  1) = n Sn 11(xn = M ) + Vt+1(Axn
xn1
n  1) +Vt+1 (xn
Axn1
n  1) +Vt+1 (xn
Dxn1
n  1)


(4.18) (4.19) (4.20)

Since Vt+1 (x
n) = Vt+1(x
n  1)
8x 2 S , (4.16) = (4.19), and (4.15) = (4.18). Therefore, Vt(xn
xn1
n  1) = st(xn
xn1
n). 2 The results in Theorem 2 also hold for T = +1 and the average cost problem using the same argument as in the previous section. 12

5. A heuristic policy In this section, under the assumption of positive set-up costs (K1
K2 > 0), we develop a heuristic that is simple to implement and performs well for our make-to-stock model. When developing our heuristic, we only consider the long-run average cost per unit time criterion, non-preemptive service, and  < 1 because we believe these assumptions are most commonly encountered in applications. Consistent with Theorem 1, we restrict our attention to a policy that has a monotonic threshold type of switching curve. Although we were unable to complete a rigorous proof of Theorem 1 for problems with set-up costs, extensive numerical investigations of hundreds of examples have not revealed any counterexamples. Hence, the development of the heuristic focuses on identifying the threshold values which cause the server to switch queues. We have investigated several distinct approaches that can be used to decide switching and idling rules, however, we report here only the most eective one. First, we discuss when to switch from a non-empty queue, then consider when to idle.

Switching Condition

To derive a switching condition which dictates when to switch from a non-empty queue, say n, we compare the expected switching costs paid by the system if the server switches to queue n  1 with the expected lost sales cost that will be incurred in queue n  1 if the server continues to serve one more job in queue n. This is an appropriate tradeo to balance, because the objective function is additive in shortage costs and set-up costs. For this comparison, we assume that if the server switches to queue n  1, it should return to queue n because the server leaves it with unnished jobs. We also assume that the switching penalty paid by the system during the above action sequence is measured by Kn + Kn1 divided by the total number of jobs served in queue n  1 prior to returning queue n. Therefore, it follows that as queue n  1 has more jobs, less expected switching cost per job served will be incurred if the server switches to queue n  1. On the other hand, if the server serves one more job in queue n, more expected lost sales cost will be incurred in queue n  1, because it will continue to grow. In the following, we formalize these ideas. By choosing to serve a job of type n, it may happen that queue n  1 reaches Mn1 , starting from xn  1, before the service completion in queue n. The probability that this event will occur n 1 ;xn 1 is pM where n1

pn1 = n1=(n1 + n). The expected number of lost sales in queue n  1 prior to the service completion in queue n is then 1 X

i=1

n 1 ;xn ipM n1

iq Mn 1 ;xn n1 = pn1

1+

1 +1

=(1 ; pn1 )

(5.1)

where qn1 = 1 ; pn1 , because the distribution on the number of lost sales, given that queue n  1 has Mn1 jobs, is a modied geometric distribution with mean pn1 =(1 ; pn1 ). Therefore, the 13

short-term expected lost sales cost which results from serving one more job in queue n rather than switching to queue n  1 in state (x1
x2
n) is approximately given by Cn1 (xn1 ), where Cn1 (xn1 ) =
Pn11;(pxn1 ) Sn1 : (5.2) n1


Mn 1 ;xn 1 +1 where Pn1 (xn1 ) = pn1 . Recall that the lost sales cost Sn1 is incurred for each arrival of type n  1 who nds that the output buer n  1 is empty. From (5.2), it is obvious that the expected lost sales cost is quickly increasing in xn1 as xn1 approaches Mn1 . This approximation becomes more accurate as xn1 goes to Mn1 , which is the essential region. Now suppose that the server switches to queue n  1, and assume that it exhausts queue n  1 and switches back to queue n. In general, this assumption does not hold for make-to-stock models, unlike make-to-order models in which a top-priority queue is served exhaustively. The server may have to switch back to queue n before exhausting queue n  1 because queue n can be in danger of lost sales while processing queue n  1. We will relax this assumption later. Under this assumption, we approximate the expected number of jobs that the server processes upon switching to queue n  1 by xn1=(1 ; n1), which is exact if the buer size of queue n  1 is unlimited. Here, we assume that the server switches to queue n  1 only when xn1 > 0. Therefore, if the server switches to queue n  1 when it has xn1 jobs, the expected switching cost per served job is given by Kn + Kn1 : (5.3) xn 1

Dene n1 as

1;n 1

n1 =
inf fxn1 2 f0
: : :
Mn1 g : (Kn + Kn1 )(1 ; n1 )=xn1 < Cn1 (xn1 )g :

(5.4)

We call the region with xn1 n1 the danger zone of queue n  1 2 f1
2g. Note that n1 ( n ) is independent of xn (xn1 ). We divide the state space into four regions assuming that the current queue is n: 1. 1  xn  Mn and xn1 < n1 2. 1  xn < n and xn1 n1 3. xn n and xn1 n1 4. xn = 0 and 0  xn1  Mn1 In other words, Region 1 represents the states in which queue n  1 is not in danger of lost sales. In Region 2, queue n  1 is in danger of lost sales but queue n is not. Region 3 has the states in which both queues are in danger of lost sales. Region 4 corresponds to the states where the current queue n is empty. The server will typically remain in queue n for Region 1, but switches to queue n  1 for Region 2. We now discuss how to determine the actions in Region 3. Suppose that the server remains in queue n and serves one more job. Then, the expected lost sales cost incurred in queue n  1 is given by Cn1 (xn1 ). On the other hand, suppose that the 14

server switches to queue n  1. We assume here that upon switching to queue n  1, the server processes queue n  1 only until the net FGI output buer n  1 is increased by one job and returns to queue n because queue n is also in danger of lost sales. The expected costs corresponding to this action sequence will be Cn (xn ) + Kn + Kn1 . Now we dene the threshold of queue n  1 when xn n as

^n1 (xn ) =
inf fxn1 2 f0
1
: : :
Mn1 g : Cn1 (xn1) > Cn (xn ) + Kn + Kn1g:

(5.5)

We state the following switching rule when xn n (Region 3): If xn1 ^n1 (xn ), switch to queue n  1 otherwise, remain in queue n. Note that when xn and xn1 approach the boundaries Mn and Mn1 , the lost sales cost terms Cn and Cn1 increase sharply and the consideration of switching cost is reduced. For states in which both queues are away from the boundaries, we have chosen to include a fairly severe switching penalty term. If switching costs are employed as a surrogate penalty in systems with signicant set-up times, the switching cost term will produce the desired eect of limiting switching as the FGI buers get low. The following lemma states that the switching curve corresponding to Regions 2 and 3 is monotonic. Lemma 2:

(i) ^n1 (xn ) is increasing in xn , n  xn  Mn , (ii) ^n1 ( n ) n1 . Proof: (i) We prove this by showing ^n1 (xn + 1) ^n1 (xn ), for xn n . From the denition of ^n1 (xn + 1), we get

Cn1 ( ^n1 (xn + 1)) > Cn (xn + 1) + Kn + Kn1 > Cn (xn ) + Kn + Kn1 :

(5.6) (5.7)

Therefore, by denition of ^n1 (xn ), we have ^n1 (xn + 1) ^n1 (xn ). (ii) Since

Kn + Kn1 > Kn
+nK1n 1 , ;n

1

we have

1

Cn1 ( ^n1 ( n)) > Kn
+nK1n 1
;n

1

1

because Cn1 ( ^n1 ( n )) > Cn ( n )+Kn +Kn1 . Therefore, by denition of n1 , we have ^n1 ( n )

n1 . 2 Our heuristic uses (5.4) to establish that the server never switches in Region 1. Based on the exhaustive service assumption, the switching curve in Region 2 is given by a constant threshold 15

value. If this assumption does not hold, the switching curve begins at 2 , but may monotonically increase as x1 approaches 1 , the danger zone for queue 1. We proceed to rescale the danger zone level, n1 , in Region 2 and dene Region 4. Region 4 has its own pair of threshold values. We will later show how under heavy trac, these thresholds also need to be modied, which we also include in our term \rescaling" of the thresholds, as presented in Tables 2 and 4. As pointed out earlier, the switching rule, (5.4), is based on the exhaustive service assumption in queue n  1 upon switching to it. Either under heavy trac or when queue n is close to the danger zone, n upon switching to queue n  1, the exhaustive service assumption becomes restrictive. The region which invalidates the exhaustive service assumption can be approximately seen by the set of states in which xn1 =(n1 ; n1) > ( n ; xn )=n
xn < n (5.8) which quanties the condition under which the mean busy period in queue n  1 with xn1 jobs exceeds the mean time required for queue n to reach its danger zone. We can easily see that (5.8) is true when xn is close to n or when queue n  1 has heavy trac. We now relax the exhaustive service assumption. If xn1 n1 , it is appropriate to consider a server who switches to queue n  1 but switches back to queue n before exhausting queue n  1. Then, from the switching rule of (5.4), this action sequence causes n1 , the danger zone level of queue n  1, to increase because the switching cost shared by jobs served in queue n  1 will be increased. We formalize this observation. First, identify Xn , the set of lengths of queue n which make the server return to queue n before exhausting queue n  1 upon switching to it:

n

o

Xn =
xn 2 f1
: : :
n ; 1g : xn + n n1 =(1 ; n1)];1 n1 > n


(5.9)

where n and n1 are solutions of (5.4) and n  n1 =(1 ; n1)];1 n1 is the mean number of arrivals to queue n in the time required to exhaust queue n  1, starting with n1 jobs. Xn denes the region of our interest: f(xn
xn1
n) : xn 2 Xn
n1  xn1  Mn1 g. Second, for xn 2 Xn , nd jn1 (xn) dened as jn1 (xn) =
maxfj 2 IN : xn + n j;1 (5.10) n1  n g:

jn1 (xn) approximately represents a deterministic number of jobs processed by the server in queue n  1 before returning to queue n. Thus, the switching cost shared per served job becomes (Kn + Kn1)=jn1(xn ). From (5.9) and (5.10), it follows that jn1 (xn ) < 1;nn1 1
xn 2 Xn . Third, recompute "n1 (xn ), the danger zone of queue n  1 for each xn 2 Xn , which we dene as

(

)

"n1 (xn ) = inf xn1 2 f1
: : :
Mn1g : Kjn + (Kxn1 < Cn1 (xn1)
xn 2 Xn : n1 n )


16

(5.11)

Idling Condition

To specify the rule for Region 4, suppose that the server is in queue n. If the server switches to queue n  1 when xn1 > 0 is small, this leads to excessive switching costs. On the other hand, if the server switches to queue n  1 when xn1 is close to Mn1 , then it can incur excessive lost sales costs in queue n  1 while the server is serving it. Suppose that the server determines to await one more arrival in queue n  1 and then switches to it rather than to switch right now. The expected switching cost per served job is decreased because the number of jobs that share the switching cost increases by one unit however, the expected lost sales cost is increased because queue n  1 grows by one unit. In the proceeding, we compare these marginal costs to determine idling threshold values. First, we determine the rule for idling under light trac, which we take to be  < 0:9. Under light trac, it suces to consider an action sequence in which the server switches to queue n  1 and exhausts it. The expected switching cost per served job corresponding to this action sequence is approximately given by Kn1 : (5.12) xn 1 1;n 1

Now we compute the expected lost sales cost which can be incurred in queue n  1 even though the server is processing it. To compute this cost, consider a simple random walk, fSk : k 1g where the kth stage increment is given by:

Wk =

P

(

1

with p

;1 with q = 1 ; p.

Let Sk = ki=1 Xi
k 1 and suppose E (W ) 6= 0. For given integers A, B > 0, the probability that Sk reaches a value at least A before it reaches a value less than or equal to ;B is given by (see Ross 16]) (q=p)B ; 1 : P =
(q=p (5.13) )A+B ; 1 Substituting xn1 , the current queue length of queue n  1, for B and Mn1 ; xn1 for A, we can approximate the probability that queue n  1, starting from xn1 , reaches Mn1 before it is exhausted as follows: xn 1 x 1 ; ;n1
(n1 =n1 ) n 1 ; 1 I (5.14) Pn1 (xn1 ) = ( = )Mn 1 ; 1 = Mn 1 : n1 n1 1 ; ;n1 From (5.14), the expected lost sales cost in queue n  1 before queue n  1 becomes exhausted is then approximated by

X Sn1PnI1 (xn1) i p"in1 qn1 i=1 1

= Sn1 PnI1 (xn1 )=(1 ; p"n1 )


17

(5.15)

where p"n1 = n1 =(n1 + n1 ). Based on our computational experience, we approximate the cost as in (5.15) instead of Sn1 PnI1 (xn1 )"pn1 =(1 ; p"n1 ), which puts much more emphasis on the lost sales penalty than the switching cost. Dene I^n1 , the idling threshold by

( (

)

)

I I^n1 = min inf xn1 2 f1
: : :
n1g : Kxnn11 < P1n1;(p"xn1 ) Sn1
n
n1 1;n 1


(5.16)

so that the expected lost sales cost in queue n  1 slightly outweighs the additional cost of switching. Shortage costs for queue n are neglected because of the light trac assumption and xn = 0. Now consider a system with heavy trac, which we take to be  0:9. Unlike the case of light trac, the server may not exhaust queue n  1 upon switching to it rather, the server may return to queue n because jobs are quickly accumulated in queue n and queue n can be in danger of lost sales. On the other hand, if the server returns to queue n, leaving unnished jobs in queue n  1 prior to exhausting queue n  1, the chance that the server will also leave queue n before exhausting it also increases. These action sequences result in overly frequent switches. The simplest way to resolve this problem is to rescale I^n and I^n1 in an appropriate manner. Observe that the idling threshold values are increasing in switching costs but decreasing in lost sales costs and the trac ratio, n1 . Based on this observation, we suggest the following ad hoc rule for rescaling the idling threshold value, I^n1 , so as to decrease it:



Kn1 ^ (5.17) n1 =(n1 + n1 )Sn1 e
In1
where dxe is the smallest integer that is greater than or equal to x. In the right-hand side of (5.17), the term n1 =(n1 + n1 ) (equally, n1 =(n1 + 1)) measures the relative arrival rate in queue n  1. Note that it is increasing in n1. With only the lump-sum lost sales cost, Sn1, we cannot accurately measure the expected lost sales cost which will be occurred in queue n  1, because the number of lost sales is also aected by the arrival rate, n1 . For this reason, the lump-sum lost sales cost, Sn1 , is scaled using the relative arrival rate n1 =(n1 + n1 ). Therefore, the term Kn1 (5.18) n1=(n1 + n1 )Sn1 In1 = min dI^n1

reects the relative importance of lost sales over switching. We incorporate the idling rule into the threshold policy as follows: The server switches according to the threshold curve #n1 (xn ) dened as 8 " (x )
x 2 X , > n1 n n n > ^ > ( x )
 x < n1 n n n  Mn,
#n1 (xn ) = > I^n1
xn = 0  < 0:9, > I x n = 0  0:9, > : n1
otherwise. n1 That is,

18

1. If xn1 #n1 (xn ), switch to queue n  1. 2. Otherwise, process one more job in queue n or idle until the next arrival to the system if the current queue n is empty.

The following lemma states that the switching curve, #n1 (xn ), generated by our heuristic policy has the monotonicity with respect to xn . Lemma 3: The switching curve generated by our heuristic policy has the following properties: (i) "n1 (xn ) is increasing in xn , xn 2 Xn .
(ii) n1  "n1 (i ), where i = minfxn : xn 2 Xn g. (iii) #n1 ( n ; 1)  #n1 ( n ) (iv) #n1 (xn ) is increasing in xn . Proof: (i) We prove this by showing "n1 (xn )  "n1 (xn + 1)
xn + 1 2 Xn . From (5.10), it is easily seen that jn1 (xn ) jn1 (xn + 1). Using this and the denition of "n1 (xn + 1), we have

Cn1 ( "n1 (xn + 1)) > Kjnn+1K(xnn)1 . Therefore, by the denition of "n1 (xn ), it follows that "n1 (xn )  "n1 (xn + 1). (ii) By the denition of "n1 (i ) and the fact that jn1 (i ) < n1 =(1 ; n1 ), we have n1 Cn1 ( "n1 (i )) > Kjn + K (5.19) ( i n1 ) > Kn +n K1 n1 : (5.20) 1;n 1

Therefore, the result follows from the denition of n1 . (iii) If n ; 1 2= Xn , #n1 ( n ; 1) = n . Therefore, the result follows from (ii) of Lemma 2. Suppose now that n ; 1 2 Xn . From (5.5), we have Cn1 ( ^n1 ( n )) > Cn ( n )+ Kn + Kn1 . Thus,

Cn1 ( ^n1 ( n )) > jKn n1+(Knn;1)1 . Therefore, by denition of "n1 ( n ; 1), we have ^n1 ( n ) "n1 ( n ; 1). (iv) From (i) and (ii) of Lemma 3, #n1 (xn ) is increasing in xn
xn < n , from (i) of Lemma 2, #n1 (xn ) is increasing in xn
xn n , from (iii) of Lemma 3, #n1 ( n ; 1)  #n1 ( n ), and from the denition of I^n1 , #n1 (0)  #n1 (xn )
xn 1. Therefore, #n1 (xn ) is increasing in xn , xn 2 f0
1
: : :
Mng. 2 Figures 1{2 show the threshold values and policy generated by our heuristic before rescaling and after rescaling, respectively, for the following example: M1 = M2 = 10, K1 = K2 = 1, S1 = S2 = 10, 1 = 2 = 0:45, and 1 = 2 = 1. In Figure 1, we have #n1 (xn ) = n1 if xn < n  ^n1 (xn) if xn n  I^n1 if xn = 0. Figure 2 shows the results after rescaling, in which "n1 (xn ) 19

is used for xn 2 Xn and In is used for idling. Figure 3 exhibits the optimal policy for the above example. With termination criterion 10;7 , the optimal and heuristic performance were computed using the VIPE algorithm and value iteration, respectively (see Kim et al. 10] for a denition of the VIPE algorithm). These gures illustrate our ndings that rescaling of threshold values more precisely predicts the optimal policy. Our heuristic is optimal in symmetric queues with zero switching costs. If K1 = K2 = 0, from (5.4), n1 = n = 1. Hence, the switching decision when the current queue is not empty is governed by Region 3. In Region 3, the switching action from queue n to queue n  1 is allowed only when Cn1 (xn1 ) > Cn (xn ), which is equivalent to xn1 > xn for a symmetric system. When the current queue is empty, the idling threshold values, In and In1 equal 1 from (5.16). Therefore, for all states, our heuristic species switching to queue n  1 if and only if xn1 > xn , which is optimal by Theorem 2, and this justies the following result. Lemma 4: Our heuristic policy is optimal for symmetric systems without set-up penalties.

6. Numerical Results

To test our heuristic, we compare its performance with respect to the optimal performance on a variety of problems. The cases that we test include symmetric as well as asymmetric queues, varying buer sizes, and cases without switching costs. The data for our test examples are summarized in Tables 1, 3, 5, 7, 9, and the numerical results are exhibited in Tables 2, 4, 6, 8, 10. Table 1 has 18 symmetric examples that are comprised of three similar example sets, which investigate the impact of switching cost, lost sales cost, and server utilization (trac ratio) on the performance of our heuristic. Each example set has 6 dierent server utilizations  = 0.1, 0.5, 0.8, 0.9, 0.95, 0.99. The three example sets dier by the ratio of lost sales cost over switching cost, S=K . For these example sets, this ratio is set to 2, 10, and 50 so that lost sales cost becomes increasingly signicant with respect to switching cost. In Tables 2, 4, 6, 8, 10, we tabulate the optimal average costs per unit time, the average costs per unit time under our heuristic, and the sub-optimality of our heuristic, which is dened as the change in percent with respect to the optimal cost. In particular, in Tables 2{4, we report the numerical results when the threshold values are not scaled. The optimal and heuristic performance were computed using the VIPE algorithm and value iteration, respectively. The termination criterion,

, was set to 10;7 . The buer size of each queue is set to M1 = M2 = 10 in Tables 1, 3, and 9, but it is varied in Tables 5 and 7. Table 2 shows that for symmetric examples, the sub-optimality of our heuristic is within 8%, and under 1.2% in more than half of these cases. In addition, when   0.5, our heuristic provides nearly the optimal values, consistent with the fact that the threshold values of our heuristic are originally computed based on a light trac assumption. When S=K = 50, the sub-optimality of our heuristic is nearly within 1% except for one case, in which our heuristic is sensitive to the trac 20

ratio. The numerical results also show that the scaling of the threshold values greatly improves the performance of our heuristic under heavy trac. Based on these test examples, we suggest that our heuristic performs very well for symmetric systems. Table 3 exhibits 30 asymmetric examples. Examples 19-32 have trac ratio less than 0.9 while Examples 33-48 have heavy trac, i.e.  0:9. Examples 27-32 are designed to examine the eect of the exhaustive service assumption used when determining the idling threshold values, in that the heuristic rule (5.17) is not applied even though they have trac ratios fairly close to 0.9. In Examples 33-43, (5.18) < 1 and in Examples 44-48, (5.18) > 1. Therefore, the heuristic rule (5.17) is not applied to Examples 44-48 even though they have heavy trac. Table 4 shows the numerical results for examples in Table 3. As we expected, our heuristic works very well under light trac (i.e.,   0:6) with an average percent of suboptimality equal to 5.4% when  < 0:9 and rising to 7.65% for  0:9 As observed in symmetric cases, the suboptimality is increasing as  ! 0:9, the boundary of our heavy trac region. When (5.18) < 1, the sub-optimality of our heuristic is within 10%. However, the results of Examples 44-48 indicate that if (5.18) > 1, our heuristic's suboptimality increases to 19% and 25% for two of ve cases. For most examples, the scaling of the threshold values improves the performance of our heuristic when  0:7. In particular, recalling that the heuristic rule (5.17) is not applied to Examples 4448, the results of those examples indicate that, under heavy trac, rescaling the threshold values corresponding to the idling rule in Region 4 is also crucial for improving the performance of the heuristic. We also investigated the eects of buer sizes on the performance of our heuristic. The test examples for both symmetric and asymmetric queues are summarized in Tables 5 and 7, respectively. In Table 5, we varied M1 and M2 for Examples 8, 9, 11 in Table 1. Table 6 shows the numerical results for examples in Table 5. Compared with the results of M1 = M2 = 10 (Examples 8, 9, and 11 in Table 2), numerical results of examples 8, 49, 52, 55 indicate that the suboptimality of our heuristic is almost insensitive to the buer sizes under light trac. In examples 11, 51, 54, 57 we observe that all the buer sizes tested have 2.5 times better performance than M1 = M2 = 10. Many numerical tests beyond these nine examples indicate that there do not exist any noticeable eects of the buer sizes in symmetric systems. Nine examples in Table 7 are composed of three example sets. In each example set, we xed M2 at 10 and varied M1 with 10, 15 , and 20. For all examples, we xed K1 = K2 = 0:1,  = 0:9, and 1 = 22. The results in Table 8 indicate that for the asymmetric examples, having the same buer sizes for both queues aects the performance of our heuristic negatively. The rst example set indicates that if both queues are almost symmetric, the performance of our heuristic is insensitive to the buer sizes. These results suggest that Tables 2 and 4 represent dicult test cases and we can expect that unequal capacities or capacities other than 10 will not usually yield worse performance. 21

Table 9 has 14 examples without switching cost penalties. The rst three examples are symmetric and are the special cases of Examples 15-17 in Table 1, while the other 11 examples are asymmetric and correspond to Examples 31-36 and 44-48 in Table 3. Cases 67{69 of Table 10 reect Lemma 4, that our heuristic gives the optimal average cost for symmetric systems without switching costs. In the remaining cases, the performance of our heuristic is improved when compared with Examples 31-36 except in case 74 for which the sub-optimality increases almost by three times, compared with example 35. We observe a similar phenomenon in Examples 76-80, which compare to Examples 44{48. When switching costs are not considered, our heuristic is promising in that it gives a solution very close to the optimal. Because it provides poor performance for some cases, however, we suggest that a more delicate approach would help when determining threshold values under heavy trac.

7. Extension of the model

For a single server make-to-stock system in isolation, we have characterized the optimal switching curve under the assumption of equal service rates and developed an ecient heuristic policy. In this section, we extend our model to place the shared resource in a networked system with three workstations as is described in Figure 4 and develop ecient heuristics. The extended model has tandem queues for each job class, with the shared resource model of Section 5 as the second stage. The rst stage of the system contains two independent workstations, each of which feeds the shared workstation. Jobs of type n served by the shared workstation are stored in output buer n until they are removed by the external demand process. When a demand of type n cannot be met by available FGI in output buer n, it is lost and incurs a penalty Sn . Otherwise, the FGI level in output buer n is decreased by one job, and the card for it is removed and sent the input buer n at the rst stage. Thus, our model incorporates a CONWIP release policy for the system. We assume that raw material is always available at the rst stage. Scheduling the extended model diers from that of the single workstation problem in important ways. For the single workstation problem, xn + yn = Mn for n = 1
2. For the extended model, however, the number of jobs in input buer n of the shared workstation, xn , and FGI output buer n level, yn , are not deterministically related. This makes the development of heuristics harder. In fact, the single workstation problem can be viewed as a special case of the extended model in which the two workstations in the rst stage have innite service rates. In this section, we develop and test three heuristics. The rst two heuristics are based on the heuristic developed in Section 5, and the third one is designed to minimize lost sales costs. The rst heuristic, called Heuristic-A, directly uses the results of the single workstation heuristic. This heuristic emphasizes the information about the status of both FGI output buers. Given the output buer n level, yn , Heuristic-A assumes that the size of input buer n of the shared resource, xn , is equal to Mn ; yn and it determines threshold values, #1 (x2 ), #2 (x1), I1, and I2 following the 22

same procedure as introduced in Section 5. Recall that the threshold values in Section 5 are given in terms of xn . The rule for Heuristic-A becomes 1. If the output buer of current queue n is not full, (i.e. yn < Mn ) (a) If yn1  Mn1 ; #n1 (xn ) and xn1 1, switch to queue n  1, (b) Otherwise, process one more job at queue n. 2. If the output buer of current queue n is full: (a) If yn1  Mn1 ; In1 and xn1 1, switch to queue n  1, (b) Otherwise, idle in current queue n until the next arrival to the system. This rule is the same as the heuristic in Section 5, except that even if the output buer n  1 is below its threshold value, Mn1 ; #n1 (xn ), switching is allowed only when the input buer n  1 has at least one job. Heuristic-A uses a condition xn1 1 to avoid the undesirable situation where the server switches to an empty queue from current non-empty queue. The weakness of Heuristic-A is that the assumption xn = Mn ; yn does not hold in general for the extended model. If the shared resource is a bottleneck or the service rates of the rst stage are much larger than the demand rates, however, we expect Heuristic-A to work well because xn  Mn ; yn . In contrast, if the rst stage is a bottleneck, we expect Heuristic-A not to work well. The second heuristic, called Heuristic-B, is based on the lengths of both the input buers and output buers of the shared resource. Like Heuristic-A, Heuristic-B has predetermined-determined threshold values using switching and idling conditions that are similar to those presented in Section 5. We rst discuss switching condition and then the idling condition.

Switching Condition

Suppose the server is in queue n and the current input and output buer sizes are xn and yn (< Mn ), respectively. Analogous to (5.4), the rst step is to identify the output buer level of queue n  1, given xn1 , which satises

(

)

yn 1 +1 n1 S n1 (xn1) = sup yn1 : (Kn + Kn1 )(1 ; n1)=xn1 < 1p; pn1 n1
xn1 1


(7.1)

As before, n1 (xn1 ) ( n (xn )) species the danger zone of queue n  1 (n) given xn1 (xn ), and is taken as ;1 if (7.1) is not satised. After identifying the danger zones, similar to Region 3 in Section 5, we have a region in which both output buers are in danger of lost sales: f(x1
x2
y1
y2) : yn1  n1 (xn1 )
n = 1
2g. For states in this region, we determine the threshold value in the following way:

^n1 (yn
xn) =
supfyn1 : C^n1 (yn1) > C^n (yn ) + Kn + Kn1 g: 23

(7.2)

where

^ynn11 +1
p ^ Cn1 (yn1 ) = 1 ; p^ Sn1
n1

p^n1 =

(

n 1 n 1 +n
n 1 n 1 +n


(7.3)

xn 6= 0 xn = 0


and ^n1 (yn
xn) := ;1 if (7.2) is not satised. The term C^n1 (yn1 ) of (7.2) represents the expected lost sales cost incurred in queue n  1 when the server serves one more job if xn > 0 or idles until the next arrival to queue n if xn = 0. The right-hand side is the expected cost incurred when the server switches to queue n  1, serves one job, and returns to queue n. Here we assume that the server switches back to queue n after serving only one job upon switching to queue n  1 because the output buer n is also in danger of lost sales. Since switching rule (7.1) assumes exhaustive service, we relax this assumption as in Section 5. First, we identify Yn (xn
xn1 ), the set of lengths of FGI output buer n which make the server return to queue n before exhausting queue n  1 upon switching to it, given WIP levels xn
xn1:

Yn (xn
xn1) =
fyn > n (xn ) : yn ; n xn1=(1 ; n1 )];1 n1  n (xn )g:

(7.4)

Second, for yn 2 Yn (xn
xn1), nd jn1 (yn
xn ) dened as

jn1(yn
xn) =
maxfsupfj : yn ; n j;1 n1 > n (xn )g
1g


(7.5)

which approximately represents a deterministic number of jobs processed by the server in queue n  1 before returning to queue n. Thus, the switching cost shared per served job becomes (Kn + Kn1)=jn1(yn
xn ). Third, recompute "n1 (yn
xn
xn1), the danger zone of queue n  1 for each yn 2 Yn (xn
xn1 ), which we dene as

(

)

(

)

yn 1 +1 " n1 (yn
xn
xn1 ) = min sup yn1 : Kn + Kn1 < pn1 Sn1
n1 (xn1 ) : jn1 (yn
xn) 1 ; pn1


(7.6)

Idling Condition

Suppose that the server is in queue n and xn = 0. We now determine the idling threshold given xn1 as follows:

(

)

I
yn1 ) S In1(xn1 ) = sup yn1 : Kxnn11 < Pn11(x;n1 n1
xn1 1
p"


n1

1;n 1

(7.7)

where p"n1 = n1 =(n1 + n1 ) and

PnI1 (xn1
yn1) =

xn 1 ; ;n1

xn 1 ; ;( n1

24

1

yn

1+

1)

:

(7.8)

The left-hand side of the inequality in (7.7) represents the expected switching cost incurred when the server switches to queue n  1 and exhausts it, while the right-hand side is the expected lost sales cost which is incurred in queue n  1 before it is exhausted. The computation of idling threshold In1 (xn1 ) is based on random walk as in the single workstation problem. For the single workstation problem, (7.8) represents the probability that the FGI output buer n  1 level is decreased by yn1 units prior to the service completion of the xn1 jobs in WIP. For the extended model, this is an approximation because an arrival of type n does not mean that xn is increased by one unit because of the existence of upstream workstations. Nevertheless, we use this random walk approximation to avoid using any knowledge of the other stations in the network, thereby maintaining general applicability. We now state the rule for Heuristic-B, assuming that the server is queue n. The server switches to queue n  1 according to the threshold curve #n1 (yn
xn
xn1 ) dened as 8" (yn
xn
xn1 )
yn 2 Yn (xn
xn1) > > < ^nn1 0  yn  n (xn ) #n1 (yn
xn
xn1 ) = I 1((xyn
xn) )
yn = M n n 1 n1 > : n1 (xn1)
otherwise. That is, 1. If xn1 1 and yn1  #n1 (xn1
yn ), then switch to queue n  1. 2. Otherwise, process one more job at queue n or idle in queue n until the next arrival to the system.

Like Heuristic-A, Heuristic-B also distinguishes the region (c.f. Region 3 in Section 5) in which both output buers are in danger of lost sales and rescales the threshold values to relax the exhaustive service assumption. On the other hand, Heuristic-B uses information of both input and output buers for the shared resource, which is the main potential advantage over Heuristic-A. The storage requirement for the threshold value #n1 (yn
xn
xn1) is a main disadvantages of Heuristic-B, compared with Heuristic-A. Denote maxfM1
M2g + 1 by M . In Heuristic-A, the storage for #n1 (xn ) requires up to 2M array sizes. On the other hand, #n1 (yn
xn
xn1 ) of Heuristic-B requires up to 2M 3 array sizes in the worst case (although typical problems with small danger zones will require much less). The third heuristic, called Heuristic-C, is the simplest among the three heuristics and focuses on minimizing only the lost sales costs. The idea of Heuristic-C comes from Theorem 2, which says that the optimal policy always serves the class with the shorter FGI output buer for the two-queue symmetric system with zero switching costs. The decision is made based on the status of both output buers as follows. Suppose the server is in queue n. 25

1. If yn = yn1 = 0, switch to queue n  1 if n Sn < n1 Sn1  otherwise, remain in queue n. 2. If yn = Mn and yn1 = Mn1 , idle in queue n until the next arrival to the system. 3. If yn = Mn and yn1 < Mn1 , switch to queue n  1 if xn1 1 otherwise, idle in queue n until the next arrival to the system. 4. If yn < Mn and yn > yn1 , switch to queue n  1 if xn1 1 otherwise, remain in queue n. Although Heuristic-C does not consider the impact of switching costs at all, we expect that Heuristic-C will be eective for the systems that are symmetric and have small switching costs.

8. Simulation Study of Heuristics for the Extended Model We tested our three heuristics by simulation, because the magnitude of the state space makes dynamic programming prohibitive. For each example, we ran the simulation until the shared resource completed 25,000 jobs for each class n, n = 1
2 and repeated it 10 times. We report the mean and 95% condence interval on the performance of the three heuristics. The data for test examples and the simulation results are summarized in Tables 11 and 12, respectively. In all cases, the number of cards is xed at M1 = M2 = 20. Table 11 has 25 symmetric cases (Examples 1{18 and Example 25) and 6 asymmetric cases (Examples 19{24). In this simulation study, we focused on the symmetric examples because they are helpful to identify the eects of lost sales costs, switching costs, trac conditions, and service rates on the performance of the three heuristics. We rst describe the symmetric examples and then asymmetric examples. Examples 1{18 are categorized into three example sets, which dier by the cost ratio S=K set to 2, 10,and 50, respectively. Example 25 is the same as Example 14 except that switching costs are not considered. For all cases, the service rates of the shared workstation are xed at 1 = 2 = 1. In each example set, the service rates of the rst stage are varied as 1, 0.25, and 5. For each service rate, we consider two types of arrival rates of the external demands, 1 = 2 = 0:25 and 0.45. The simulation results support our intuitive conjectures about the performance of the three heuristics. As a general conclusion, the data suggests that heuristic-B is almost always the best heuristic of the three. Heuristic-C does not show the best performance in any case. The best performance of Heuristic-C compared Heuristic-B is -1.23% (see Example 16) and the worst is -1607% (see Example 5). We observe that Heuristic-C performs better when the rst stage is a bottleneck (see Examples 4{5, 9{10, and 15{16) than when it is not. We can conjecture the reason as follows. If the rst stage is a bottleneck, the policy should focus on avoiding lost sales in output FGI buers because most of jobs are held in the rst stage. On the other hand, if the shared resource is a bottleneck, the policy should put an emphasis on minimizing switching costs as well as lost sales costs because most of jobs are held in input buers of the shared resource. Hence, it is 26

not surprising that Heuristic-C, which does not consider the switching cost eects in the rule, does not work well when the shared workstation is a bottleneck. In each example set, the worst performance of Heuristic-A compared with Heuristic-B comes when the rst stage is a bottleneck under low arrival rates, 1 = 2 = 0:25 (see Examples 3, 9, and 15). In these examples, the relative performance of Heuristic-A, compared with that of Heuristic-B, is about -265%, -410%, and -11%, respectively. This phenomenon can be explained by the fact that the approximation xn  Mn ; yn , which is assumed in developing Heuristic-A, does not work well because the rst stage is a bottleneck. On the other hand, under high arrival rates, 1 = 2 = 0:45, (see Examples 4, 10, 16), the performance dierences between Heuristic-A and B are within 1% and Heuristic-C is within 4% except for Example 4, even though the rst stage is a bottleneck. When the rst stage has much faster service rates than the shared resource, Heuristic-A works relatively better. In particular, when S=K = 50 and the rst stage has ve times faster service rates than the shared resource (see Examples 17{18), both heuristic-A and Heuristic-B have almost the same performance. This result supports our conjecture that xn will be well approximated by Mn ; yn if the shared workstation is a bottleneck. In addition to this, we also observe that the relative advantage of Heuristic-B with respect to Heuristic-A increases as S=K decreases. Example 19 is the same as Example 1 except that 1 = 2 and 1 = 0:5. In Example 20, we increased K1 and S1 of Example 19 by three times. Examples 21 and 22 are the same as Examples 19 and 20, respectively, except that the arrival rate of type 1 demands is replaced by 1 = 0:9. Examples 23{24 are randomly generated. In asymmetric examples, we observe behavior similar to the symmetric. In Examples 21{22, the second workstation of the rst stage is a bottleneck. As is expected, Heuristic-A works relatively better in Examples 19{20 than Examples 21{22. In contrast, Heuristic-C works better in Examples 21{22. When the switching costs are zero (see Example 25), it is not surprising to see that Heuristic-C, which is designed to minimize only lost sales costs, works well.

9. Conclusion In this paper, we developed a two-class make-to-stock queueing model (and an associated MDP model) with a shared resource. When there are no set-ups, we partially characterized the structure of an optimal policy as a monotonic threshold switching curve. In the special case when both classes are identical and the system does not have switching penalties, we characterized the optimal policy as always serving the class with the shorter FGI output buer level. Based on these results and numerical investigations, we proposed an eective heuristic scheduling policy for the class of problems with set-up costs. We used basic principles of stochastic processes (results for M/G/1 queues, random walks, and probability theory) to develop closed-form, analytical rules to precompute a monotonic threshold switching curve. Numerical dynamic programming (DP) algorithms 27

Example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

K1 K2 S1 S2 1 2 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

0.2 0.2 0.2 0.2 0.2 0.2 1 1 1 1 1 1 5 5 5 5 5 5

0.2 0.2 0.2 0.2 0.2 0.2 1 1 1 1 1 1 5 5 5 5 5 5

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1

.05 0.25 0.4 0.45 0.475 0.495 .05 0.25 0.4 0.45 0.475 0.495 .05 0.25 0.4 0.45 0.475 0.495

2

.05 0.25 0.4 0.45 0.475 0.495 .05 0.25 0.4 0.45 0.475 0.495 .05 0.25 0.4 0.45 0.475 0.495



0.1 0.5 0.8 0.9 0.95 0.99 0.1 0.5 0.8 0.9 0.95 0.99 0.1 0.5 0.8 0.9 0.95 0.99

Table 1: Input data for Examples 1-18. were used to test the performance of our heuristic with respect to the optimal policy. These tests indicated that our heuristic is typically well under a 10% loss of optimality. Based on our approach for the isolated shared resource, we extended the heuristic to account for the essential dierences found in an arbitrary network context. A two-stage system provided sucient generality to make the resulting heuristic applicable to any series network. As a reference, we used simulation to compare the performance of our heuristic with a very simple heuristic that eectively minimizes shortage costs when switches are costless. The results suggest that our heuristic is highly eective, sometimes providing performance improvements of over 1,600%.

28

10 9 8

Second Queue Length

7 6 5 4 3 2 1 0 0

1

2

3

4 5 6 First Queue Length

7

8

9

10

Figure 1: Heuristic policy without rescaling of threshold values. 10 9 8

Second Queue Length

7 6 5 4 3 2 1 0 0

1

2

3

4 5 6 First Queue Length

7

8

9

10

Figure 2: Heuristic policy with rescaling of threshold values.

 : Action of staying at the current queue.

o : Action of switching from queue 1 to queue 2. * : Action of switching from queue 2 to queue 1.

29

10 9 8

Second Queue Length

7 6 5 4 3 2 1 0 0

1

2

3

4 5 6 First Queue Length

7

8

9

10

Figure 3: Optimal policy.

 : Action of staying at the current queue.

o : Action of switching from queue 1 to queue 2. * : Action of switching from queue 2 to queue 1.

Type 1 Card (M1)

µ3

WIP buffer

FGI output buffer

λ1

µ1

y1 Outside Demand

x1

Input Card Queue

µ2 µ4

λ2 Shared Resource

x2

y2

Type 2 Card (M2)

Figure 4: A two-stage tandem two-class make-to-stock model. 30

Example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Optimal Heuristic % Suboptimal No scaling % Suboptimal 0.000555 0.000555 0.00 0.000555 0.00 0.003317 0.003318 0.04 0.003318 0.04 0.008260 0.008311 0.62 0.008818 6.76 0.012982 0.013522 4.16 0.014710 13.31 0.016578 0.017142 3.40 0.019732 19.03 0.020251 0.021065 4.02 0.024894 22.93 0.000571 0.000571 0.00 0.000571 0.00 0.003863 0.003863 0.00 0.003864 0.02 0.014056 0.014386 2.35 0.014615 3.98 0.029951 0.031179 4.10 0.033943 13.33 0.045448 0.047585 4.70 0.051141 12.53 0.062522 0.066722 6.72 0.070726 13.12 0.000629 0.000629 0.00 0.000629 0.00 0.004624 0.004624 0.00 0.004627 0.06 0.030477 0.032716 7.35 0.032981 8.22 0.094587 0.095275 0.73 0.102667 8.54 0.165291 0.166852 0.94 0.179838 8.80 0.250203 0.252981 1.11 0.268463 7.30 Table 2: Performance Results for Examples 1-18.

31

Example 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

K1

0.2 0.2 0.1 2/15 0.1 0.1 2/15 0.1 0.5 0.2 0.1 2/15 0.1 2/15 0.1 0.1 0.8 0.8 0.3 0.3 0.8 0.4 0.8 0.1 2/15 0.4 0.5 0.2 1.2 1

K2

0.1 0.1 0.1 1/15 0.1 0.1 1/15 1 0.1 0.2 0.1 1/15 0.1 1/15 0.1 0.1 0.2 0.2 0.3 0.3 0.1 0.6 0.5 0.1 1/15 0.4 0.25 0.5 0.3 0.4

S1

2 2 1 4/3 1 1 4/3 1 4 1 1 4/3 1 4/3 4 4 4 3 2.2 2.2 3.5 5 4 1 4/3 1 1 1 3 3

S2

0.5 0.5 1 2/3 5 1 2/3 5 1 2 1 2/3 1 2/3 1 1 1 3 2 2 6 3 4 1 2/3 1.2 1.2 1.2 1 1

1

1 1 1.5 1.5 1 1.5 1.5 1 2 2 1.5 1.5 1.5 1.5 2 2 2 1 1.2 1.2 1.5 1.2 1.6 1.5 1.5 2 2 2 1.5 1.2

2

1 1 0.75 0.75 1.5 0.75 0.75 1.5 1 1 0.75 0.75 0.75 0.75 1 1 1 1 1 1 1 1.6 1.2 0.75 0.75 1 1 1 1.5 1.8

1

0.3 0.1 1/3 1/3 0.3 .5 .5 0.3 0.8 0.6 0.6 0.6 0.66 0.66 .9 1.2 0.4 0.7 0.5 0.8 0.7 0.55 0.7 0.72 0.72 1 1.4 0.4 0.9 0.5

Table 3: Input data for Examples 19-48.

32

2

0.1 0.3 1/6 1/6 0.4 .25 .25 0.7 0.4 0.5 0.3 0.3 0.33 0.33 .45 0.3 0.5 0.2 0.5 0.3 0.45 0.75 0.6 0.36 0.36 0.45 0.2 0.7 0.5 1



0.4 0.4 0.44 0.44 0.57 0.67 0.67 0.77 0.8 0.8 0.8 0.8 0.88 0.88 0.9 0.9 0.9 0.9 0.92 0.95 0.92 0.93 0.94 0.96 0.96 0.95 0.9 0.9 0.93 0.97

Example 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

Optimal 0.002913 0.002965 0.003360 0.003306 0.006507 0.007612 0.007542 0.059096 0.049222 0.036611 0.014784 0.014003 0.025079 0.022370 0.046246 0.058887 0.075755 0.101698 0.088157 0.124348 0.191466 0.217394 0.259127 0.045168 0.037248 0.106181 0.073673 0.069053 0.166124 0.192890

Heuristic % Suboptimal No scaling % Suboptimal 0.002977 2.20 0.002978 2.23 0.003001 1.21 0.003001 1.21 0.003424 1.90 0.003424 1.90 0.003427 3.66 0.003426 3.63 0.006522 0.23 0.006536 0.45 0.007886 3.60 0.007803 2.51 0.008059 6.85 0.008156 8.14 0.059941 1.43 0.063963 8.24 0.051008 3.63 0.052517 6.69 0.037615 2.74 0.038269 4.53 0.016443 11.22 0.016449 11.26 0.015070 7.62 0.015216 8.66 0.028829 14.95 0.029237 16.59 0.025606 14.47 0.026783 19.73 0.049828 7.75 0.056726 22.66 0.062103 5.46 0.081851 39.00 0.082309 8.65 0.090784 19.84 0.103197 1.47 0.119780 17.78 0.090225 2.35 0.097613 10.73 0.131087 5.42 0.152687 22.79 0.201366 5.17 0.209380 9.36 0.225552 3.75 0.246547 13.41 0.263750 1.78 0.293511 13.27 0.048734 7.88 0.051644 14.34 0.039116 5.02 0.043290 16.22 0.117164 10.34 0.138875 30.79 0.087493 18.76 0.096784 31.27 0.071825 4.01 0.077783 12.64 0.181001 8.96 0.207978 25.19 0.241677 25.29 0.263614 36.67

Table 4: Performance Results for Examples 19-48.

33

Example 49 50 51 52 53 54 55 56 57

M1 M2 K1 K2 S1 S2 1 2 5 5 5 15 15 15 25 25 25

5 5 5 15 15 15 25 25 25

0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1

1

2

1 0.25 0.25 1 0.4 0.4 1 0.475 0.475 1 0.25 0.25 1 0.4 0.4 1 0.475 0.475 1 0.25 0.25 1 0.4 0.4 1 0.475 0.475

Table 5: Input Data for Examples 49-57

Example Optimal Heuristic % Suboptimal 49 0.011630 0.011655 0.21 50 0.050302 0.051860 3.10 51 0.104426 0.106110 1.61 52 0.002228 0.002242 0.63 53 0.006184 0.006502 5.14 54 0.025510 0.026005 1.94 55 0.001211 0.001211 0.00 56 0.002447 0.002473 1.06 57 0.010793 0.011007 1.98 Table 6: Performance Results for Examples 49-57.

Example 58 59 60 61 62 63 64 65 66

M1 M2 K1 K2 S1 S2 1 2 1 2 10 15 20 10 15 20 10 15 20

10 10 10 10 10 10 10 10 10

0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

1 1 1 3 3 3 3 3 3

1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 2 2 2

1 1 1 1 1 1 1 1 1

Table 7: Input Data for Examples 58-66.

34

0.6 0.6 0.6 0.6 0.6 0.6 1.2 1.2 1.2

0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3

Example 58 59 60 61 62 63 64 65 66

Optimal 0.029216 0.019357 0.013955 0.041719 0.025404 0.017537 0.053007 0.029917 0.021592

Heuristic % Suboptimal 0.030082 2.96 0.019710 1.82 0.014495 3.87 0.046674 11.88 0.027619 8.72 0.019024 8.48 0.057133 7.78 0.032217 7.69 0.022955 6.31

Table 8: Performance Results for Examples 58-66.

Example K1 K2 S1 67 0 0 5 68 0 0 5 69 0 0 5 70 0 0 1 71 0 0 4/3 72 0 0 4 73 0 0 4 74 0 0 4 75 0 0 3 76 0 0 1 77 0 0 1 78 0 0 1 79 0 0 3 80 0 0 3

S2

1

5 5 5 1 2/3 1 1 1 3 1.2 1.2 1.2 1 1

1 1 1 1.5 1.5 2 2 2 1 2 2 2 1.5 1.2

2

1 1 1 0.75 0.75 1 1 1 1 1 1 1 1.5 1.8

1

0.4 0.45 0.475 0.66 0.66 .9 1.2 0.4 0.7 1 1.4 0.4 0.9 0.5

Table 9: Input data for Examples 67-80.

35

2

0.4 0.45 0.475 0.33 0.33 .45 0.3 0.5 0.2 0.45 0.2 0.7 0.5 1



0.8 0.9 0.95 0.88 0.88 0.9 0.9 0.9 0.9 0.95 0.9 0.9 0.93 0.97

Example 67 68 69 70 71 72 73 74 75 76 77 78 79 80

Optimal 0.012422 0.070163 0.140290 0.008853 0.006568 0.019288 0.025579 0.000908 0.045786 0.036147 0.019707 0.025292 0.051095 0.076711

Heuristic % Suboptimal 0.012422 0.00 0.070163 0.00 0.140290 0.00 0.009115 2.96 0.006707 2.12 0.019703 2.15 0.026929 5.28 0.001152 26.87 0.045874 0.19 0.036850 0.19 0.019736 0.15 0.026855 6.18 0.063180 23.65 0.090691 18.22

Table 10: Performance Results for Examples 67-80.

36

Example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

K1 K2 S1 S2 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.3 0.1 0.3 0.3 0.3 0

0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 1 0

0.2 0.2 0.2 0.2 0.2 0.2 1 1 1 1 1 1 5 5 5 5 5 5 0.2 0.6 0.2 0.6 0.6 0.6 5

0.2 0.2 0.2 0.2 0.2 0.2 1 1 1 1 1 1 5 5 5 5 5 5 0.2 0.2 0.2 0.2 1 10 5

3

1 1 0.25 0.25 5 5 1 1 0.25 0.25 5 5 1 1 0.25 0.25 5 5 1 1 1 1 0.25 2 1

4

1 1 0.25 0.25 5 5 1 1 0.25 0.25 5 5 1 1 0.25 0.25 5 5 1 1 1 1 1 0.25 1

1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Table 11: Input data for Examples 1-25.

37

1

0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.5 0.5 0.9 0.9 0.8 0.8 0.45

2

0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.45 0.25 0.25 0.25 0.25 0.55 0.55 0.45

Ex. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Heuristic-A (%) Heuristic-B (%) Heuristic-C (%) % Di1 0.00162 (0.00005) 0.00151 (0.00002) 0.02502 (0.00017) 1555 0.00785 (0.00040) 0.00542 (0.00029) 0.04562 (0.00027) 742 0.03122 (0.00161) 0.00855 (0.00054) 0.03035 (0.00044) 255 0.13004 (0.00046) 0.09109 (0.00079) 0.11205 (0.00102) 23 0.00152 (0.00004) 0.00147 (0.00002) 0.02509 (0.00041) 1608 0.00640 (0.00027) 0.00515 (0.00039) 0.04527 (0.00039) 779 0.00161 (0.00003) 0.00159 (0.00003) 0.02491 (0.00012) 1464 0.01252 (0.00150) 0.01105 (0.00199) 0.04653 (0.00076) 321 0.14890 (0.01570) 0.02924 (0.00238) 0.05574 (0.00183) 91 0.41773 (0.00289) 0.42024 (0.00419) 0.43707 (0.00380) 4.0 0.00159 (0.00003) 0.00160 (0.00006) 0.02498 (0.00012) 1459 0.01032 (0.00160) 0.00967 (0.00121) 0.04655 (0.00077) 381 0.00177 (0.00010) 0.00171 (0.00010) 0.02499 (0.00022) 1362 0.03194 (0.00563) 0.03182 (0.00684) 0.05798 (0.00509) 82 0.14104 (0.01083) 0.12713 (0.01278) 0.17133 (0.01811) 35 2.01251 (0.01663) 2.02200 (0.02286) 2.04680 (0.01847) 1.2 0.00170 (0.00008) 0.00171 (0.00008) 0.02498 (0.00018) 1360 0.02678 (0.00561) 0.02707 (0.00719) 0.05144 (0.00439) 90 0.00275 (0.00014) 0.00213 (0.00004) 0.03493 (0.00037) 1538 0.00484 (0.00018) 0.00421 (0.00008) 0.06960 (0.00087) 1553 0.05192 (0.00260) 0.01982 (0.00108) 0.07206 (0.01124) 264 0.06323 (0.00215) 0.03812 (0.00337) 0.14752 (0.01272) 287 0.88182 (0.00347) 0.51584 (0.00282) 0.63010 (0.00310) 22 3.21541 (0.05218) 3.17671 (0.07224) 3.52356 (0.04419) 11 0.01020 (0.00445) 0.00927 (0.00250) 0.01013 (0.00449) 9.3 Table 12: Performance Results for Examples 1-25. 1 : Heuristic;C ;Heuristic;B  100. Heuristic;B

38

References 1] D.P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Clis, 1987. 2] D. Bertsimas and J. Ni~no-Mora, Conservation laws, extended polymatroids and mult-armed bandit problems a unied approach to indexable systems, Math. of Operations Research 21 (1996) 257{306. 3] M. Elhafsi and S. Bai, Optimal and near optimal control of a two-part-type stochastic manufacturing system with dynamic setups, Preprint (1996). 4] A. Federgruen, and Z. Katalan, Determining production schedules under base-stock policies in single facility multi-item production systems. preprint (1995). 5] A. Federgruen, and Z. Katalan, The stochastic economic lot scheduling problem: Cyclical base-stock policies with idle times, Management Science 42:6 (1996) 783{796. 6] J.C. Gittins, Multi-armed Bandit Allocation Indices, Wiley, New York, 1989. 7] A.Y. Ha, Optimal Dynamic scheduling policy for a make-to-stock production system, Operations Research, 45 (1997) 42{53. 8] W.J. Hopp and M.L. Spearman, Factory Physics: Foundations of Manufacturing Management IRWIN, Chicago, 1996. 9] E. Kim and M.P. Van Oyen, Beyond the c rule: Dynamic scheduling of a two-class loss queue. Forthcoming in Mathematical Methods of Operations Research, 48:1 (1998). 10] E. Kim, M.P. Van Oyen, and M. Rieders, General dynamic programming algorithms applied to polling systems, Working paper, Northwestern Univ., Evanston, IL (1996). 11] G.P. Klimov, Time sharing service systems I, Theory Prob. Appl. 19 (1974) 532{551. 12] H. Levy and M. Sidi, Polling systems: applications, modelling, and optimization, IEEE Trans. Commun. 38 (1990) 1750{1760. 13] D.M. Markowitz and L. Wein (1996) Heavy trac analysis of dynamic cyclic policies: A unied treatment of the single machine scheduling problem, Working Paper 3925-96-MSA MIT Sloan School of Management, Cambridge. 14] J. Qiu and R. Loulou, Multiproduct Production/inventory control under random demands, IEEE Transactions on Automatic Control 40:2 (1995) 350-356. 15] S.M. Ross, Introduction to Stochastic Dynamic Programming Academic Press, New York, 1983. 16] S.M. Ross, Stochastic Processes Wiley, New York, 1983. 17] M.L. Spearman, D.L. Woodru, and W.J. Hopp, CONWIP: A pull alternatives to KANBAN, International Journal of Production Research 28:5 (1989) 879{894. 18] M.L. Spearman and M.A. Zazanis, Push and pull production systems: Issues and comparisons, Operations Research 40:3 (1992) 521{532. 19] H. Takagi, Queueing Analysis of Polling Models: Progress in 1990{1993, in J.H. Dshalalow, Ed. Frontiers in Queueing: Models, Methods and Problems. CRC Press, 1994. 20] M.H. Veatch and L.M. Wein, Scheduling a make-to-stock queue: index policies and hedging points, Operations Research 44:4 (1996) 634{647. 21] M.H. Veatch and L.M. Wein, Optimal control of a two-station tandem production/inventory system, Operations Research 42:2 (1994) 337-350. 22] L.M. Wein, Dynamic scheduling of a multiclass make-to-stock queue, Operations Research 40:4 (1992) 724-735.

39

23] P. Whittle, Restless bandits: activity allocation in a changing world. In A Celebration of Applied Probability, ed. J. Gani, J. Appl. Prob. 25A (1988) 287{298. 24] Y. Zheng and P. Zipkin, A queueing model to analyze the value of centralized inventory information, Operations Research 38:2 (1990) 296-307.

40