deterministic retrial times are optimal in queues with

DETERMINISTIC RETRIAL TIMES ARE OPTIMAL IN QUEUES WITH FORBIDDEN STATES* VIDYADHAR. G. KULKARNI Department of Operations Research, University of North Carolina, Chapel Hill, NC 27514, USA.

SURESH. P. SETHI Faculty of Management, University of Toronto, Toronto, Ontario, Canada M5S 1V4. ABSTRACT Consider a customer X, who wishes to obtain service at a queueing system. When X arrives at this system, he has three possible courses of action to choose from, i) join the system; ii) not to join the system now, but return later after a random amount of time; or iii) to leave without obtaining service. Suppose that the system has a set of forbidden states and when X, upon arrival, finds the system in one of these forbidden states, he has no choice but to return later. Otherwise, he joins the system. The problem addressed in this paper is the following: Given that every unsuccessful retrial costs a certain amount and every unit of time spent waiting outside the system costs a certain amount, then after how long a period, possibly random, should X try his luck again? We show that under the assumption that all forbidden states are regenerative, there are deterministic {i.e. non-random) optimal retrial times that minimize the expected cost of getting served. Several applications are discussed. Keywords : Queueing, Optimal Retrial Times, Regenerative States

RESUME Considerons un consommateur X qui veut obtenir un service dans un systeme de files d'attente. Lorsque X arrive aupres de ce systeme, il peut choisir parmi trois cours d'action possibles, i) joindre le systeme; ii) ne pas joindre le systeme maintenant, mais y retoumer plus tard apres un temps aleatoire; ou iii) partir sans obtenir un service. Supposons que le systeme a un ensemble d'etats interdits et lorsque, en anivant, X trouve le systeme dans un de ces etats interdits, il n'a pas d'autre choix que de retoumer plus tard. Sinon, il joint le systeme. Le probleme traite dans cet article est le suivant: Etant donne que chaque re-essai a un certain cout et que chaque unite de temps passe a attendre en dehors du systeme a egalement un certain cout, alors apres quelle periode, si possible aleatoire, X devrait - il essayer a nouveau? Nous montrons que sous l'hypothese que tous les etats interdits sont regeneratifs, il y a des temps de re-essais optimaux deterministiques (c.a.d. non aleatoire) qui minimisent le cout espere d'etre servi. Plusieurs applications sont discutees. Mots des : Files d'attente. Temps optimaux de re-essai, Etats regeneratifs.

1. INTRODUCTION Consider a customer X, who wishes to obtain service at a queueing system. When X arrives at this system, he has three possible courses of action to choose from, depending upon the observed state of the system: (i) He may decide to join the system, if possible, and wait in the system until he gets served; (ii) He may decide not to join the system, because it is too congested, and return later after a random amount of time; or * Accepted January 1988.

INFORvol. 27, no. 3, Aug. 1989 374

QUEUES WITH FORBIDDEN STATES

375

(iii) He may decide to leave altogether without obtaining service, if he deems the expected cost incurred in obtaining the service is more than the benefits derived from the service. It should be noted that as far as X is concerned he is facing a given queueing system. The queueing system may have any form whatsoever as long as its behavior is in no vray affected by the presence of X. In effect, X is the only smart customer who is capable of making decisions; the rest of the customers in the system behave in a pre-specified way. Mandelbaum and Yechiali (1983) study the problem when a single smart customer X faces an M|G|1 queue and tries to optimally decide whether or not he should join the queue. They obtain optimal policies for X. Kulkami (1982) extends their model to include more general queueing systems. In this paper we study a different but related problem that X faces in the retrial situation described above. Here we assume that the system has a set of forbidden states and when X, upon arrival, finds the system in one of these forbidden states, he has no choice but to return later. Otherwise, he joins the system. The optimization problem here is that of finding the retrial times. More specifically the problem that we address is the following: Given that every unsuccessful retrial costs a certain amount and every unit of time spent waiting outside the system costs a certain amount, then after how long a period, possibly random, should X try his luck again? The specific answer that we obtain is that under the assumption that all forbidden states are regenerative, there are deterministic {i.e. non-random) optimal retrial times that minimize the expected cost of getting served. Of course, the actual values of the retrial times will depend on the nature of the system and the values of various cost parameters involved. This problem arises in a variety of situations. Consider a queueing system where X would enter if and only if service is immediately available. Here those states of the system where all servers are busy are forbidden states. For example, iiX calls an airline to make a flight reservation and hears a tape recording asking him to hold {i.e., the system is in a forbidden state), he has to decide when he should try again (assumiii^ >e does not want to hold). This problem has regenerative forbidden states if the arrival process is Poisson and service times are independent exponentials. As a second example, consider the following problem in preventive maintenance. A system consists of n components in parallel. Lifetimes of components are independent exponentials. At time zero all components are functioning. A repairman checks the system at times T\, T1 + T2, etc. Each time he observes which components are down, but if at least one component is up {i.e. the system is functioning) he does not fix anything. If upon a retrial, he finds that the system has failed he repairs it. Repair time is genenilly distributed. When repair is complete all components are up and the repairman starts his periodic checking again. Even though this example does not precisely fit our model, one can use our results to show that iJie optimal time of next checking is deterministic. Here all the up states are forbidden and regenerative, while down state is not forbidden and not regenerative. As a third example, consider an ./l/|C?|l queue with server breakdown. Suppose the server stays operational for a random amount of time and then fails. It stays in the failed mode for an exponential amount of time independent of ever^/thing else and then beccimes operational. The customer whose service gets preempted by failure repeats his service with resampled duration. Now suppose X (our smart customer) wants to join this queue only if the server is up and come back after a while if the server is down. Let state j denote the state when the server is down and there are ; customers in the

376

V. G. KULKARNI AND S. P. SETHI

queue. Then (jJ > 0) are forbidden states, and are regenerative. The states in which the server is up are non-regenerative and non-forbidden. The problem is formally stated and an expression is derived for the total discounted cost in section 2. In section 3, we prove that there are deterministic retrial times (whose values depend on the observed state of the system) which minimize the expected total discounted cost. In section 4, we give several examples. The results of the paper are summarized in section 5, which concludes the paper. 2. COST ANALYSIS Let Q{t) denote the number of customers in the queueing system at time t. Customer X wants to use this given system. Let 5" c {0,1,2,...} be the set of forbidden states for X. Without any loss of generality assume that X arrives at this queueing system for the first time at time 0. Then if Q{Qi) E S, X cannot enter the system and must return at some random time TI whose distribution may depend on G(0). We shall show later that under certain cost criteria, the optimal distribution of tx is degenerate, i.e. tx is a constant. If Q{tx) ES, X comes back at time tx + T2. The distribution of T2 may depend on Qitx). This process repeats itself until for some N>0, Q ( 2 f T,) ^ S. {T/, ? = 1,2,... A''}, where iV is a random index, are called the retrial times and are conditionally independent given _2i T/), n > 0}. The «th retrial time Tn depends only on the state last observed, i.e., li~^ ti). The assumptions about the distributions of the retrial times are precisely reflected in the following equation:

F/(x) = P{tn+x < x\Q{J2 ^i) eS;O0, t>0).

(2.2)

Note that Pij(t) is independent of s due to the regenerative nature of the forbidden states. If X comes to the system and finds it in state / e S, it costs him bi > 0 dollars and an additional C/ > 0 dollars per unit time spent outside the system until his next retrial. The future costs are continuously discounted at rate a > 0. Let (/>/(F) be the expected total discounted cost to X if Q(0) = /. (F = [F^], keS). Then it can be easily seen that ij(j)jiF)

(2.3)

where

fii = bi + {Ci/a){l - r Jo

e-^'dFiit)),

(2.4) (2.5)

QUEUES WITH FORBIDDEN STATES

377

Let ^{F) = [^i{F)]', A = [Aij] and f£ == [fii], (^ and /£ are column vectors). The equation (2.3) can be written as: ~ ~ [I-A]MF)=ii

(2.6)

Now, a > 0 and JF,(O) = 0, V/ e S imply that A is substochastic and therefore [/ - A] is invertible. Hence [/-^]~V

(2.7)

Now let Pi = P{Q{0) ^ i} {i e 5") be the initial distribution and ^ -• [^,] be a row vector; note that J2ies Pi — 1- Then the expected total discounted cost of" waiting and retrials is given by A\-^li.

(2.8)

Notice that the expected total discounted cost depends upon the particular retrial distributions Fi that are used and the initial distribution ^ . The problem is to minimize (j)p{F) over all the distributions Fi with support on [0, oo). It is shown in the next section that there exist degenerate distributions Fi which minimize fi{F), viewed as a function of i^ is minimized by degenerate distributions D = [A]/e5- Writing [/ - A]-"^ --= Adj[/ - ^]/Det[/ - A]

(3.1)

reveals that the expected total cost 4>fi{F) in eq. (2.8) is a ratio of two multilinear functional of F If all Fj {j e S\{i}), denoted as ,F, are fixed, then iiP'i, iF), viewed as a function of F, only, is minimized by a degenerate distribution Di placing a unit mass at r,. Let I ^ {F = {Fk; k e S) : F,, is a distribution on [0, oo)} and D = {D ^ {Dk;k eS) : D^ is a degenerate distribution on [0, oo)}. The main result of this paper is given in the following theorem:

378

V. G. KULKARNI AND S. P. SETHI

Theorem 1: There exists an F* ef such that 4,p(F*) = rnm(t>ii(F)

(3.3)

if and only if there exists a D* G P such that (l)li(D*) = mm(j>p(D).

(3.4)

Moreover, (pp(D*) = ^p(F*). Proof: Suppose there exists an F* = (F^) satisfying eq. (3.3). If every F* e D, we are done. If not, define a sequence 4)'^, / = 1,2,..., by using Lemma 1, as follows:

where DJ is the distribution that minimizes (j>p(., F2,F^,...).

where D^ e D minimizes (I)I(D;,;F^,...);

and so on. Now

MF*) > 0J^ > 0 | > • • • > 0. Hence we have lim cf>'p=MJ^*)^M^*) (-•oo

i-

—

—

where D* = [D*]. But we cannot have (j)i(D*) < p(F*) since F* satisfies eq. (3.3). Hence we must have

and obviously this D* satisfies equation (3.4). Now suppose there is a Z>* = [£>*] € D which satisfies (3.4). Now, if there exists an F* eT satisfying eq. (3.3), we are done by the above argument. If there does not exist an F* satisfying eq. (3.3), there exists an F such that

This is so because D* is feasible for the minimization problem in eq. (3.3). Using Lemma 1 in a similar way, we obtain the sequence. /> = (S>p

and so on. Clearly, (/>^ > 0 and (/i^ is a decreasing sequence. Hence lim 'B^L = 1—•oo

—

which is a contradiction, since D* is chosen to satisfy eq. (3.4). Hence, there must exist an F* satisfying eq. (3.3) such that M^*) = M^*^ *

QUEUES WITH FORBIDDEN STATES

379

We now restrict attention exclusively to deterministic retrial policies (DRP). This is justified in view of the above theorem. Note that the necessary and sufficient conditions for the existence of a D* satisfying eq. (3.4) (i.e. the DRP problem) are the same as the necessary and sufficient conditions for the existence of an F* satisfying eq. (3.3) These conditions are given in Theorem 2. A DRP can be described by giving the deterministic retrial time x, € [0, oo] for / 6 5. By a DRP denoted as x(= [Xi]), we mean a policy under which retrial time Xi is used in state /. Let 0,(x) be the expected total discounted cost of following policy x if the system is initially in state /. It is easy to see that (f)i(x) [i e S) satisfy the following equations:

jes

where m(i,Xi) - bi + (Ci/a)(l - e-"^')

(3.6)

is the discounted cost of making one retrial and waiting for x, units of time. Now, for i e S, let (/)*=niin0,(x)

(3.7)

where the minimum is taken over all DRPs x. It is straightforward to see that

cj>* = mm{m(i,Xi) + J2^~'"''Pvi''i)'f>*j}""'-^

(3.8)

jes

Sufficient conditions for the uniqueness of solution for the above equation are stated in the following theorem. Theorem 2: Equation (3.8) has a unique solution if

(i) a > 0, (ii) Pij(t) (t > 0) are continuous Junctions oft (i,j e S), and (iii) c = sup,g5 Ci < 00,

b = swpi^s ^i < °°

^' — ^^ies bi > 0,

X-- siap^g^' A,- < a).

where the sojourn time in state i (i e S) is an exponentially distributed random variable with parameter A,. Proof: Let Xoo be a policy under which x, = oo for all / € 5". Then, from equation (3.5),

-

(ieS).

(3.9)

Hence, 0 < ^* < 4>i{Xoc) < b + -

(iG S).

(3.10)

Thus 0} process under the DRP x. Let i be the current state. Then x, is the retrial time. Let T be the time up to first transition. Then T is an exp(A,) random variable. We have

380

V. G. KULKARNI AND S. P. SETHI

E Hi{x) = E[Hi{x)\T: > X,]P{T > x,} + E[Hi{x)\t < Xi] = [

Jxi

E{Hi{x)\r = s]Aie-^''ds + f' E[Hi{x)\x = s]Xie Jo

Now for s > Xi E[Hi{x)\T = s] = bi+ f ' ae-^'dt + £•//,-(x)e-°^' and for s < Xj E{Hi{x)\x = s) = bi + r ae-'^'dt. Jo Substituting yields /•OO

rXi

EHiix) = \ {hi + / ae-'^'dt + EHi{x)e-''''>}Xie-^>'ds Jx, Jo

+ f '{bi+ f Cie-'^'

Jo which can be simplified to get '

*

Jo

J

I

Hence I

'^ '

1 -• exp(-(A,- + a)Xi)

Then Mx) > EHiix) >

(3.11)

bi 1 - exp(—(A + a)Xi)

The first inequality follows because the right hand side is just the total expected discounted cost before the process leaves state / for the first time. The second inequality is a consequence of 1 > A,. Now let a be a constant such that (3.12) It is easy to see from equations (3.11) and (3.12) that Xi b + -.

(3.13)

Then, equation (3.10) implies that we may restrict our attention to only those DRPs for which Xi > a for all i € S. Thus equation (3.8) can be equivalently written as -'^'PijiXi)rj}.

(3.14)

QUEUES WITH FORBIDDEN STATES

381

Then from continuity of pij(Xi) and an argument similar to that in Ross (1970), it can be seen that the right hand side of equation (3.14) is a contraction operator (applied to 0*) with contraction e'"" < 1. The uniqueness of solution to equation (3.14) (and hence (3.8)) then follows from the usual fixed point theorem for contraction operators. » Remark 1. It follows from standard Markov Decision theory that any X; € [a, oo] which minimizes the right hand side of eqviation (3.14) is an optimal retrial time in state /. Remark 2 : The following interative procedure can be used to compute (j)*. For i € S, define Pij(Xi)(i>j(n)}.

(3.16)

Again from Ross (1970), it follows that ^i(«) —> (j)* uniformly in i. Remark 3 : If 5 is a finite set, condition (iii) of Theorem 2 is trivially true. Remark 4: If YJJ&S PU (0 < 1 for ^U ' € -^ ^^'^ t>0, then it can be showti that Theorem 2 holds for a = 0. In almost all practical cases, a can be 0, i.e. we can use the above iterative procedure to compute minimum expected total undiscounted cost. In the next section we give three examples of this situation. 4. EXAMPLES 4.1. M\M\1\1 Queue Suppose that X faces an M|Af |1|1 system. The state space is thus {0,1} and the forbidden state is {1}, i.e., X can enter if and only if the server is idle. All other customers who find the server busy go away. The forbidden state in this case is regenerative. It can be easily shown that

where X is the arrival rate and n is the service rate. From equation (3.5) we get 0, fli = ^{1 + \/d + d) and a2= fi{l-VS + d). Letfci= 62 = $1 per =02 = $c per 1/fi hrs. Equations (3.16) become (with a = 0) 1 (« -t-1) = min{l -h {n -I-1) =

n) +Pi2{xi)hin)} (4.9)

{

which are solved numerically in a recursive manner starting with (pi (0) = ^2 (0) = 0. The convergence slows as l = $15.08. If the queue is in steady state to begin with, Po = P{QiO) = 0} = 2/7, pi = P{QiO) - 1} = 2/7 and P2 = P{Q{0) = 2} = 3/7. Hence 0). The Pij it) functions are available in the literature, but are too complicated to perform numerical calculations. One intuitively expects that x^. < jc^^^ < ... and that /axf ii > k) should be decreasing functions of 3 and c, but this remains to be established. 4.3 Optimal Requesting A student submits ki> 1) high priority jobs to a computer at a beginning of a day. The computer processes them sequentially one at a time. The job lengths (in computer time needed) are exponentially distributed, each with parameter ^. The jobs are independent of each other. The student logs on intermittently to see how many jobs are still to be executed. As soon as all the jobs are done he can proceed with further work. If every logon costs $1 and the time spent waiting for the job outputs costs $c per 1/fi units of time, how often should the student logon in order to minimize the total cost? Suppose the jobs are submitted at time 0. Let Qit) be the number of unfinished jobs at time t. The state space is {0,l,2,...k} and the set of forbidden states is 5" = {1,2,..., k}. All forbidden states are regenerative and the transition probabilities are S.

(4.10)

Equation (3.8) becomes (with a = 0) i-X

P* = min{l '—

(4.11) ;=o

which can be rewritten as ieS

b* = nxbil —

(4.12)

These equations can be solved recursively without taking recourse to iterative methods. The solutions are plotted in Figure 3. It shows /ix* (/ = 1,2,3,..., 10) as functions of the waiting cost c. As one would expect intuitively, x* -^ oo as c ^ 0. Also, for a fixed c, X* is an increasing function of i and for a fixed / it is a decreasing function of c. As a numerical example, let fc = 5, fi = 3/hr. and waiting cost = $3/hr. This makes c = $1. From Figure 3, we get the results in Table 1. Table 1 2

xf (min)

*i^)

1 23 3.15

2 41 4.45

•

3 58 5.65

4 75 6.80

5 91.6 7.91

Thus the first logon should be tried 91.6 minutes after the five jobs are submitted. If it is found that there are still two more jobs to be done (including the one under execution) the next trial should be made after 41 minutes and so on. The expected total cost of this optimal policy is $7.91. 5. SUMMARY In this paper we have considered the problem of finding the optimal retrial times for an observer who has to Keep conducting retrials until he finds the system in an acceptable

QUEUES WITH FORBIDDEN STATES

385

Figure 3. state. We have shown that if the forbidden states are regenerative and the cost stracture is linear, deterministic retrial times are optimal. We have derived the Bellman equation (eq. (3.8)) satisfied by the optimal retrial times. We have also derived sufficient conditions (Theorem 2) to ensure uniqueness of the solution to this equation. Finally three examples are given in which we have solved this equation numerically to obtain optimal retrial times. ACKSfOWLEDGEMENT This research is supported in part by AFOSR Grant 84-0140 and NSERC Grant A4619. ItEFERENCES 1. Barlow, R.E. and Proschan, F. (1965). Mathematical Theory of Reliability, John Wiley and Sons, New York. 2. Kulkami, V.G. (1982). Optimal retrial policies for constrained Markov chains, UNC/ORSA/ TR-82/2, Curriculum in Operations Research and Systems Analysis, University of North Carolina, Chapel Hill. To appear in Stochastic Models. 3. Kulkami, V.G. (1983). A game theoretic model for two types of customers competing for service. Operations Research Letters, 12, 3, 119-122. 4. Mandelbaum, A. and Yechiali, U. (1983). Optimal entering mles for a customer with wait option at an Af |G|1 Queue, Management Science, 29, 2, 174-187. 5. Ross, S.M. (1970). Applied Probability Models with Optimization Applications, Holden-Day. C. Vidyadhar Kulkarmi received a Bachelor of Technology degree in mechanical engineering from the Indian Institute of Technology, Bombay, India, in 1976, and M.S. and Ph.D. degrees in operations research from Cornell University, Ithaca, NY in 1978 and 1980, respectively. He is currently an Associate Professor in the Department of Operations Research at the University of North Carolina at Chapel Hill. His research interests are in stochastic models of queues, reliability systems, computer systems and stochastic mod-s^-is^S^ els.

386

V. G. KULKARNI AND S. P. SETHI

Suresh P. Sethi is General Motors Research Professor of Operations Management at the University of Toronto. He has an MBA from Washington State University and an M.S. and a Ph.D. in Industrial Administration from Carnegie-Mellon University. He was a Connaught Senior Research Fellow in 1984-85. He is currently a Principal Investigator for the Manufacturing Research Corporation of Ontario, a Center of Excellence funded by the Provincial Government. His research interests are in production planning problems under uncertainty, problems of scheduling in manufacturing, and stochastic dynamic optimization problems. His articles on these and other topics have appeared in a variety of journals including Management Science, Operations Research, Mathematics of Operations Research, SIAM J. of Control and Optimization, SIAM Review, Advances in Applied Probability, J. ofEcon. Theory, Naval Research Logistics Quarterly, Journal of Optimization Theory and Applications, IEEE Transactions on Automatic Control, European Journal of Operational Research, International Journal of Production Research, and The EMS Magazine. He has also co- authored a book and co-edited a special issue for INFOR on the subject of optimal control theory and applications.