a comparison of individual shorter{queue selection with ... - CiteSeerX

A COMPARISON OF INDIVIDUAL SHORTER{QUEUE SELECTION WITH GLOBALLY MORE OPTIMAL SYSTEMS: AN APPLICATION OF LAQT Lester Lipsky, Tao Zhang, Seong-Lim Kang and Brian Mackay Department of Computer Science and Engineering Room 204, Bronwell Building, U-155 and The Taylor L. Booth Center for Computer Applications and Research 233 Glenbrook Road, U-31 University of Connecticut Storrs, Connecticut 06269-3155 . BRC/CSE-TR-93-12 October 19, 1995

ABSTRACT We examine the system time of a customer who infrequently needs service, but when he does, he selects the shorter of two steady-state M/G/1 queues (all other customers choose a queue at random). We compare his system time with that of the other customers, and with the system time for customers accessing an equivalent M/G/2 queue. We also compare with a single double-fast M/G/1 queue. For the case where the servers are exponential, we also compare with the queueing discipline where every customer picks the shorter queue. Our results show that the \cheater" does not do nearly as well as one might expect. This example, in the framework of LAQT, also provides three dierent computational methods for handling systems which are usually described in a Kronecker-Product space but are intrinsically independent. A discussion of their relative computational complexities is also included.

1

1 Introduction Consider the following problem. A single shared resource (e.g., a XEROX copier) is clearly overused, and must be either replaced by one which is twice as fast (call it option 1), or be augmented by a second, identical one. If the latter option is chosen, then two sub-options are available: either: (option 2) The second copier is placed in the same room as the rst (An M/G/2 queue), or (option 3) The second copier is placed in another convenient location (two M/G/1 queues). The three options are shown in Figure 1. It is well known that even if the load is split exactly in two (both servers have the same arrival rate in option 3), option 2 will always be better than option 3, for any arrival rate, and any service time distribution. (The mean system time for a customer in a steady-state M/G/2 queue is always less than the system time for an equivalent M/G/1 queue with half the arrival rate.) It is also well known that option 1 is always twice as good as option 3, and for M/M/1 queues it is also better than option 2. For a large class of service time distributions however, it has been found that option 2 can yield smaller mean system times than option 1 for utilization factors close to 1, and large coecients of variation (C 2 := 2 = x2 >> 1).

Option1

Option2

Option3

Figure 1: Three hardware options for doubling of maximal service rates. Option 1 is a single, double-fast server; Option 2 has two identical servers at a single location; Option 3 has two identical servers at dierent locations. Now suppose that an infrequent user of option 3 (he doesn't use the system often enough to aect the steady-state solution) nds out what the present length of each queue is, and goes to the shorter one (he \cheats"). How long does he have to wait on average, as compared to the options described in the previous paragraph? We will call this option c. In the next section we examine the case where the service time distribution is exponential, and show that if the arrival rate of the other customers to each of the servers is the same, the system time for the cheater (from his arrival to his departure) is exactly the same as it would be with option 2, although the distribution of system times is dierent. We also compare with the situation where every customer chooses the shorter queue [CHOW77]. In succeeding sections, using a linear algebraic approach to queueing theory (LAQT), we examine systems where the service time is non-exponential. In this paper we will examine the ve types of queues mentioned above. We distinguish them by using the appropriate subscripts as given by their option number. Thus, the steady-state probability that an arriving customer will nd n customers in the queue already (counting the one in service), for a system using option i, (i = option 1, 2, 3, c or s), is denoted by ai (n). 2

2 Cheater Problem For Exponential Service Times For exponential service times, the performance characteristics for options 1 to 3 are well known, and can be found in any book on queueing theory. It is also well known that for any system with Poisson arrivals (as is the case here) an arriving customer will see the same thing as a random observer, i.e., ai (n) = ri (n) for i = 1; 2; 3, where ri (n) is the steady-state probability that there will be n customers in the queue. These probabilities are given after the following prerequisite de nitions.

1 := mean arrival rate to server 1; as described in option 3; 2 := mean arrival rate to server 2; as described in option 3;  := 1 + 2 = total arrival rate; all options: x := mean service time for the original server; j := j x j = 1; 2;  := j x=2 = (1 + 2 )=2:

(1) (2)

The steady-state queue-length arrival probabilities are:

a1 (n) = (1

) n

(1 ) n for n > 0; and a (0) = 1  a2 (n) = 2 (1 2 + ) 1+ n a3j (n) = (1 j ) j ; j = 1; 2: Note that the double-fast server of option 1 has a mean service time of x=2.

(3) (4) (5)

The mean system time for all three options can be found by calculating the mean queue lengths and then using Little's Theorem, Ti = qi =. By de nition,

qi := thus the mean queue lengths are:

and Then the mean system times are:

and

1 X

n=1

n ri (n);

q1 = 1   ; q2 = 1 2 2 ; q3 = 1 1  + 1 2 
1 2

T1 = 12
1 x  ; T2 = 1 x 2 = 1 +1 
1 x  ; 



T3 = 1 1 1  + 1 2 
1 2 3

(6) (7) (8)

If the arrival rates to each of the servers in option 3 are equal, then 1 = 2 = =2 and

1 = 2 = . In this case T3 takes on its minimum value and becomes T3 = 1 x 


(9)

Clearly, then (for exponential servers with Poisson arrivals) T1 < T2 < T3 = 2 T1 for all  < 1. But what can the cheater (using option 3) expect? Let Pr(k; m) be the joint probability that the cheater will nd k customers at server 1 and m customers at server 2. Since the customers' behavior is uncorrelated as to which server they use, we have

Pr(k; m ) = a31 (k) a32 (m) = (1 1 ) (1 2 ) k1 m2 : The probability that the cheater will nd n customers in the shorter queue is given by

ac (n) = Pr(k = n; m > n ) + Pr(k > n; m = n ) + Pr(k = n; m = n ) Clearly, as long as 1 and 2 are less than 1 Pr(k = n; m > n ) =

1 X

m=n+1

Pr(n ; m ) = (1 1 ) (1 2 ) n1

1 X

m=n+1

n n+1

Similarly,

= (1 1 ) (1 2 ) 11 2
2

n n+1 Pr(k > n; m = n ) = (1 1 ) (1 2 ) 12 1
1 Upon putting this all together and simplifying, ac (n) turns out to be

m2 (10) (11)

ac(n) = (1 1 2 ) (1 2 )n : The mean length of the shorter queue as seen by the arriving cheater is simply

qc =

1 X

n ac(n) = 1 12
1 2 n=1

Now, the cheater's behavior does not satisfy Little's theorem, but since the service distribution is exponential, the residual time remaining for the customer in service when the cheater arrives is the same as the mean service time, namely x. Thus the system time for the cheater is simply the mean time for everyone in front of him to nish, plus the time it takes to serve him, i.e.,    1 2  Tc = x (1 + qc ) = x 1 + 1   = 1 x 
(12) 1 2 1 2

4

Figure 2: Comparison of Response Times for Four Options with Poisson Arrivals to

Exponential queues. The functions, (1 )Ti ; i = 1; 2; 3; c, taken from Equations (6), (7), (8), (9) are plotted versus . Here, T2 = Tc. Also included is the option where each customer joins the shorter queue (JSQ) at arrival time. This expression has a maximum when 1 = 2 . Therefore, as might be expected, the cheater gains when the system is not in balance. But when the system is balanced (when 1 = 2 ) he experiences the same mean system time as he would if both servers were in the same room and he was treated fairly. That is, Tc(1 = 2 ) = 1 x 2 = T2 : (13) However, since ac (n) is not equal to a2 (n), the distributions of the two system times must be dierent, even though they have the same mean. Although the formulas for the four systems as given in the above equations are rather well known, we compare them in Figure 2. In this way, the reader will be more prepared to examine the unfamiliar curves for non-exponential servers given in Section 4.3. Instead of plotting T versus  we plot (; )T, the reason being that we are interested in comparing the dierent options, not showing that they are unbounded at  = 1. For further comparison we have included a curve, (Ts ), for option 3 where every customer (not just the cheater) chooses the shorter queue. [CHOW77] has found an algorithm for computing Ts , the results of which are reproduced here. We know of no solutions for this case if the service times are non-exponential. We see that when  is small, option 2 (as well as the cheater) has response times similar to option 3. But as  approaches 1, Option 2 approaches Option 1. The fact that the two curves are equal at  = 1 indicates that T2 =T1 = 1, but the dierence of the two slopes indicates that T2 T1 is not 0. In fact, T2 T1 = 2 (1x+ ) ; which for  = 1 is x=4. As is always the case, T3 is twice as large as T1 , for all . Similar results

5

will be shown for several non-exponential distributions.

Figure 3: Comparison of density functions of system times for options 1, 2 and c, as given by equations (14), (15) and (16). In all three curves,  = 1:0, and  = 0:9. The service times are exponentially distributed, therefore bs1 (t) and bsc (t) are also, but bs2 (t) is not. Since the ai 's are geometric in nature, it can (more or less) easily be shown that the density functions for the system times of options 1, 2, and c are given by the following formulas:  )e 2(1 )t

bs1 (t) = 2(1 8 >


h  11 +  e

:

.

t

 (2 + 3

 e

2(1 4 2 (1 2 )



)t i

(14)

 6= 1=2 :  = 1=2

: t) e t

(15)

2 )e (1  )t ; (16) where  = 1=x. These are plotted in Figure 3 for values  = 0:9 and  = 1:0. We see that bsc(t) = (1

2

the system times for the cheater and the double-fast server are exponentially distributed, but option 2 is not. The coecient of variation (2 = T2) for options 1 and c is 1, but for option 2 it is (1 2 + 3 ), which is less than 1 for all values of  except 0 and 1. It has a minimum at  = 2=3. Equation (15) looks rather messy but it has a simple representation (sometimes called hypoexponential), as given in Figure 4. The corresponding vector-matrix pair is

p = 1 +1  [22 ; (1





)(1 + 2)]; B =  2(1 0 ) 01


6

Figure 4: A \Black-Box" representation of b (t), for all  < 1. See Section 2 for 2

explanations. A complete explanation of this representation will be given in the next section. Remember now, what we have discussed so far is only true for Poisson arrivals to exponential servers. If the service time distribution is non-exponential, the situation becomes much more dicult. We discuss this in the next section.

7

3 LAQT Description of General Service Times Before discussing the cheater problem, we give a short review of how one represents the behavior of servers using matrices. This procedure was rst used by Marcel Neuts [NEUT75] in discussing Phase Distributions, and has been used extensively by him and his coworkers and others, as well as the present authors and coworkers who call this general topic Linear Algebraic Queueing Theory (LAQT). For further details, see [LIPS92].

Figure 5: A \Black-Box" representation of a general server with a matrix exponential distribution. Any general distribution can be represented arbitrarily closely by the following construction. Imagine that a server can be represented by a \black box" (although we think we know what's inside, we're not allowed to see what's going on) containing m phases, as shown in Figure 5. A phase is a mathematical abstraction which has the properties of an exponential server (but possibly with complex service rate). A customer, upon entering the box goes to phase i with probability pi . He stays at phase i for a random time taken from the exponential Probability Distribution Function (PDF), 1 exp( i t), and then either goes to phase j with probability Pij , or leaves the box with probability qi . De ne the following matrix objects using the conventions: bold-faced, upper-case letters [P] stand for matrices (two-subscripted arrays [Pij ]); bold-faced lower-cased letters [p] stand for row-vectors (p = [p1 ; p2 ;


; pm]) and bold-faced lower-cased primed letters [q0 ] are column vectors. Thus the entrance probabilities are collectively written as the entrance vector, p. The exit probabilities are collected in q0 , and the transition probabilities are the elements of the (sub-stochastic) matrix P. We can write (P)ij = Pij , (p)i = pi , and (q0 )i = qi . From their de nitions, since the customer must go somewhere upon leaving any phase, m X j =1

Pij + qi = 1 for all i

and since he must go to some phase when entering the box, m X

De ne the universal column vector:

i=1

pi = 1: 2

0 :=

6 6 6 6 4

8

1 1





1

3 7 7 7 7 5

(17)

[(0 )i = i = 1], then the above sets of equations can be concisely written as matrix equations: P0 + q0 = 0 or q0 = (I P)0 ; and p0 = 1: Furthermore, de ne the diagonal matrix, M, whose non-zero elements are the phase service rates: (M)ii = Mii = i : Also, let B := M(I P) and V := B 1: Then the PDF for the time the customer is in the box can be shown to be generated by < p ; B > in the following way: Let X be the random variable for the time the customer is in the black box, then B (x) := Pr(X  x) = 1 p
exp( xB)
0 = 1 [exp( xB)]; where, for any matrix, X, we use the notation [X] := p X 0 to emphasize that the quantity on the right is a scalar. Also, the reliability function is given as R(x) := 1 B (x) = [exp( xB)]: Finally, the probability density function (pdf) for the process is: b(x) := dB (x) = [B exp( xB)]: (18)

dt

For obvious reasons, these functions are called Matrix Exponential (ME) functions. We say that < p ; B > generates, or is a representation of b(x). Matrix representations are not unique. In fact, for each vector-matrix pair, < p ; B >, there are in nitely many other pairs which generate the same b(x). It follows that the moments of the distribution depend upon V in the following way: Z 1 xn b(x) dx = n! [Vn ]: (19) 0

The Laplace transform for the distribution turns out to be

B  (s) :=

Z

0

1

e sx b(x) dx = [( I + sV ) 1 ]:

This scalar function of s can be shown to be a ratio of polynomials if m is nite [i.e., B  (s) = q1 (s)=q2 (s), where q1 (s) and q2 (s) are polynomials]. Thus nite ME functions have Rational Laplace Transforms (RLT). They are also known as Kendall Distributions and are equivalent to Coxian Servers. Note that phase distributions [PH] form a proper subset of these classes of functions. Also, Neuts [NEUT82], and Carroll, et.al. [CARR82] have independently found the steady-state solutions of M/ME/1 queues. We present their results here, using the notation of [LIPS92]. First de ne the following matrices: Q = 0 p (note that Q is an m  m matrix of rank 1, satisfying the relation Q2 = Q), A := I + 1 B Q and U := A 1;



where  is the mean arrival rate of customers to the server. 9

i (n) :=steady-state probability that there are n customers in the queue,

and the one being served is at phase i. n includes the customer being served, and (n) is a row vector whose m components are the i (n)'s. The scalar probabilities, equivalent to those de ned in Section 1 are related to the 's by the relation: m X r(n) = (n) 0 = i (n): i=1

It was shown [NEUT82], [CARR82] that the vector probabilities are given by so the scalar probabilities are

(n) = (1

) p U n ;

r(n) = (1

 ) [Un ]:

As with the M/M/1 queues, the arrival probabilities are the same as the steady state probabilities, so we can write a(n) = (n) and a(n) := a(n)0 = r(n). The vector probability, (n), contains all the information that can be known about the status of the customer in service. It is not merely a mathematical inconvenience for describing nonexponential behavior. It contains within it the information needed to nd the residual time remaining for the customer in service. For instance, suppose an arriving customer would like to know how much time remains until the customer in service nishes. Observing that there are n customers ahead of him in the queue, he knows that the system is in state: (20) pI := a(1n) a(n) = [U1 n] pUn:

That is, (pI )i is the probability that the customer in service will be at phase i at the arrival moment. (The subscript, I, stands for the Initial, or startup vector, related to any process. If a customer had just entered the box, then the initial vector would have been p, the entrance vector. Also note that pI 0 = 1.) Then, according to LAQT, the mean time remaining until the one in service nishes (given that there are n in the queue when the observing customer arrived), is given by 1 pUn
V0 = [Un V]
tr (n) := pI V 0 = [U (21) n] [Un ] In fact the pdf for the process is generated by < pI ; B >. That is, the distribution of the time remaining until the customer in service nishes, is given by: 1 pUn B exp( xB) 0 = [Un B exp( xB)] ; (22) br (x) := pI B exp( xB) 0 = [U n] [Un ]

with a mean time given by tr (n) [compare (20), (21) and (22) with Equations (18) and (19)]. All these expressions are far easier to compute than it might seem at rst examination, particularly on computer systems which automatically compute matrix functions.

10

4 Cheater Description for General Service Times We are now ready to examine the three options discussed in Section 1. The rst option (the double-fast server) is a steady-state M/G/1 queue, whose mean system time is given by the Pollaczek-Khinchin (PK) formula:   T1 = x2 1 1  + 1   (23) where, as before, x is the mean service time of the original server, and 2

:= C 2 1 ; C 2 being the coecient of variation de ned in the Introduction. Since C 2 can take on any value from 0 to 1, lies in the range, 1=2  < 1. For exponential servers, = 0. The second option (two servers in the same room) is an M/G/2 queue. There are no known general analytic solutions for such systems. We therefore use the matrix analytic solutions given in [TEHR83] and [LIPS92] to get numerical results. These equations are somewhat cumbersome to reproduce here, but they are easy enough to compute for any ME distributions with moderate values for m. The third option (two servers in dierent locations) is made up of two M/G/1 queues with dierent arrival rates. To get the mean system time we apply Little's Theorem to the PK formula for mean queue lengths:      1  1    1 2 1 2 1  T3 =  ( q1 + q2 ) = x  1  + 1  +  1  + 1 
1 1 1 2 As with the M/M/1 queue, when 1 = 2 = =2, this equation reduces to exactly 2 T1.

4.1 The Cheater's Mean System Time

The mean time the cheater must spend waiting to be served is more complicated than for the M/M/1 queue, because now not only must we calculate the time he must spend waiting for the customers in front of him to nish, but also the residual time for the one in service to nish as well. We know that the steady state vector arrival probability for each of the two servers is

a (n) = (1 i )pUn ; i

where

i

U := A = I + 1 B Q i = 1; 2: i

1

i

i

(24)

Next de ne the double vector probability that the cheater will nd k customers at server 1 and l customers at server 2. [a(k ; l )]ij := Steady-state probability that an arriving customer will nd k customers at server 1 and l customers at server 2, where the customer in service at 1 is in phase i, and the customer in service at 2 is in phase j. Clearly, this is simply: [a(k ; l )]ij = [a1 (k)]i [a2 (l)]j : These quantities are vectors in a direct-product space. We will treat them as being free associative. That is, a1 (n) a2 (n) is the same thing as a2 (n) a1 (n), as long as we are only doing algebraic 11

manipulations. The general rules are as follows: Objects from dierent spaces (corresponding to the dierent servers), automatically commute with each other. But the order of objects in any given space must strictly adhere to their commutation rules. Only when we must perform actual numerical computations must a speci c ordering of the binary subcripts be speci ed. The conventional orderings are Kronecker Products. Our application here is rather straightforward, and this naive approach will not get us into trouble. See Chapter VII of [LIPS92] for more details. The mean system time for the cheater is made up of three parts. First he must wait for the customer in service to nish. Next he must wait for all the other customers in the queue ahead of him to nish, and then he must be served. Now, 1 X

l=n+1

a(n; l) = a (n) 1

1 X

l=n+1

a (l)

(25)

2

is the probability that server 1 has the shorter queue, which in fact has n customers, and the components of this double vector represent the vector probabilities of where the customers in service are at the moment the cheater decides. Therefore, the time spent by the cheater waiting for the one in service to nish is [see (20), (21) and (22)]:

A :=

1 X 1 X n=1 l=n+1

+

a(n; l)V 0 0 + 1

1 X n=1

1

2

1 X

1 X

n=1 k=n+1

a(k; n)V 0 0 2

1

2

a(n; n)[n V + (1 n)V ] 0 0 : 1

2

1

2

This average time includes the probability that the shorter queue may actually be empty, so the cheater will not have to wait at all. The last summation on the right corresponds to both servers having the same number of customers. Then the cheater must make a decision of which one to go to, thus the unspeci ed probability, n . We point out here, that if 1 6= 2 , then even though the two servers are identical, they will yield dierent residual times. Since, in our examples we will assume the equality of the arrival rates, we will not discuss this issue further. The time the cheater waits for the other customers in front of him to be served is simple. Since they each start from the beginning, they each take on average x units of time. Therefore:

B := x

1 X 1 X

n=1 l=n+1

(n 1)a(n; l)01 02 + x

1 X

1 X

n=1 k=n+1

(n 1)a(k; n)01 02 + x

1 X

n=1

(n 1)a(n; n)01 02 :

The time the cheater takes is, of course, x, but we express it in the following way:

C := x

1 X 1 X

n=1 l=n+1

+ x

1 X l=1

a(n; l)0 0 + x 1 2

a(0; l)0 0 + x 1 2

1 X

1 X

n=1 k=n+1

1 X k=1

a(k; n)0 0 + x 1 2

1 X

n=1

a(k; 0)0 0 + x a(0; 0)0 0 : 1 2

1 2

a(n; n)0 0

1 2

(26)

Note that the rst three terms on the right-hand-side of C exactly cancel the negative terms of B . Also, the last three terms of C correspond exactly to the expressions in A for n = 0 (the terms not included in the sums). This is due to the fact that ai (0) := (1 i )pi , leading to, ai (0)Vi0i = (1 i )piVi0i = (1 i ) [V] = (1 i) x = ai (0)0i x; 12

where we point out that V1 is the same as V2 except that they operate on dierent vector spaces. So the match is made. We put this all together to get the mean system time for the cheater: Tc = A + B + C = Trc + x qc ; (27) where 1 X 1 1 X 1 X X Trc = a(n; l)V1 01 02 + a(k; n)V2 01 02 n=oo l=n+1

1 X

+ and

qc =

1 X 1 X n=1 l=n+1

n=0

n=0 k=n+1

a(n; n)[n V + (1 n)V ] 0 0 1

n a(n; l)01 02 +

1 X

2

1 X

n=1 k=n+1

1

(28)

2

n a(k; n)01 02 +

1 X n=1

n a(n; n)01 02 :

(29)

Trc corresponds to the mean time until the next departure from the chosen queue (either the

one in service at the moment the cheater arrives, or the cheater himself if the queue is empty). qc is the mean number of customers in the chosen queue when the cheater arrives. We now present three dierent ways for computing Tc, and compare their computational complexities. Each of the sums in the above equation is of one of two types. We examine the leading term rst, using (24) and (25).

A1 :=

1 X 1 X

n=0 l=n+1

a(n; l)V 0 0 = 1 1 2

= (1 1 )(1 2 )

1 X n=0

1 X 1 X

n=0 l=n+1

p U n V 0 1

1 1

1

1 X

l=n+1

a (n) V a (l)0 0 1

1

2

1 2

p U l 0 : 2

2

(30)

2

There are similar expressions for the next two sums as well, which we call A2 and A3 respectively. P Now, matrix sums of the form Ul satisfy the same formulas as their scalar counterparts. That is: 1 X Ul1 = [I U1 ] 1 U1n+1 = U1 K1U1 n; where Ki := [I

l=n+1

U ] , and commutes with U . We then have: i

1

i

A1 = (1

1 ) (1

2 )

1 X

n=0

[Un1 V1 ]
[K2 U2 U2 n ]:

(31)

This expression can be evaluated as it stands, or it can be manipulated to the form:

A1 = (1 = (1

1 )(1 1 )(1

2 )

1 X

n=0

p K U U n 0 p U n V 0 2

2 )p2 K2 U2

Let Q21 := 02 p1 , and de ne:

F := 21

1 X

n=0

2

"

2

1 X n=0

2

1

2

1

1

1

#

U n0 p U n V 0 : 2

2

1

1

1

1

U n Q Un = Q + U Q U + U Q U +


2

21

1

21

13

2

21

1

2

2

21

1

2

(32)





(33) = Q21 + U2 Q21 + U2 Q21 U1 + U2 2 Q21 U1 2 +


U1 : Finally, we see that F21 = Q21 + U2 F21 U1: (34) This last expression looks very simple, but it is not easy to solve as it stands. F21 can be evaluated recursively using (34), but that turns out to be computationally inecient. We will use (34) later when we evaluate the terms involved in qc . But for now, look at the following

Algorithm for Calculating F

21

BEGIN PROCEDURE . *Initialization . F21 (0) := Q21 . U1 (1) := U1 . U2 (1) := U2 . *Loop Until Converged . FOR k = 1 TO max, DO . F21(k) := F21 (k 1) + U2(k) F21(k 1) U1 (k): . U1(k+1) := U1(k)
U1(k) . U2(k+1) := U2(k)
U2(k) . END FOR . F21 = F21 (max) END PROCEDURE The reader can convince yourself by successive substitutions, that U(i k+1) = UNi , and

F

21

k =

( )

NX1 n=0

U n Q U n; where N = 2k : 2

21

1

That is, each successive iteration of the algorithm generates twice as many terms as existed after the previous iteration. Thus one can sum over one million terms in only twenty iterations. We now return to (28). Let n be the same for all n. Then (28) in Form 1 becomes

Trc = A1 + A2 + A3 = (1 1 )(1 2 ) (1)

+ (1 1 )(1 2 ) +(1 1 )(1 2 ) Form 2 yields:

1 X n=0

1 X n=0

[U1 n V1 ] [U2 n ] + (1

1 X n=0

[U1 n V1 ] [K2 U2 U2 n ]

[K1 U1 U1 n ]

)(1 1 )(1 2 )

(35) 1 X n=0

[U1 n ] [U2 n V2 ]:

Trc(2) = (1 1 )(1 2 ) (p2 [K2 U2 + I2 ] F21 V1 01 + p1 [K1 U1 + (1 )I1 ]F12 V2 01 ) : 14

We return to (32) for the third computational form. Moving both pi 's to the beginning and 0i 's to the end, A1 can be rewritten yet again as:

A1 = (1

"

2 ) p1 p2 K2 U2

1 ) (1

1 X

n=0

U2 n U1 n

#

V 0 0 : 1 1 2

(36)

Note that we were able to move p1 to the left because all the objects on its left are in 2-space. However, V1 must always stay to the right of U1 , since they do not commute. Since U1 and U2 formally commute, we can write

U n U n = (U U )n 2

1

2

1

Therefore, (36) becomes:

A1 = (1

2 ) p1 p2 K2 U2

1 ) (1

"

1 X n=0

[U2 U1 ]n

#

V 0 0

1 1 2

(37) = (1 1 )(1 2 )p1 p2 K2 U2 [I U2 U1 ] 1 V1 01 02: This looks simplest of all, but U2 U1 must be expressed as a Kronecker product before any calculations [in particular nding the numerical inverse of (I U2 U1 )] can be performed. The Kronecker product of two m  m matrices is of dimension m2  m2 . For those readers who wish to actually carry out such computations we suggest Chapter VII of [LIPS92]. We only give some concise mathematical relations here. The Kronecker product of two m vectors is an m2 vector. For instance:

p := p p 1

2

is a row vector with m2 components.

0 := 01 02 is an m2 column vector of all 1's. Any matrix operating on a vector in 1 space can be emdedded in the product space by the following D^1 := D1 I2: Also,

D^ := I D : 2

It follows that:

1

2

D^
D^ = D D = D^
D^ =6 D D : 1

2

1

2

2

1

2

1

(The ordering in is critical, since that represents a speci c way of ordering the elements of the resultant matrix. But the abstract algebraic properties remain the same, namely that all 1 subscripted hatted objects commute with all 2 subscripted hatted objects.) With these conventions understood the third form of Trc can be written as: i h Trc(3) = (1 1 ) (1 2 )p K^1 U^1 H V^2 + K^2 U^2 H V^1 + H [V^1 + (1 )V^2 ] 0 ;

where

H := [I U U ] : 1

15

2

1

If 1 = 2 then the formulas simplify, and the distinction between 1 and 2 is minimal. That is, U1 and U2 are identical in form. so we drop the subscripts. The three equivalent expressions become: [(2KU + I)Un ] [Un V]; (38) (2) 2 Trc = (1 ) [(2KU + I)FV]; (39) ^ K ^ U^ + ^I ) H V^ ]: Trc(3) = (1 )2 [(2 (40) In comparing Trc(2) and Trc(3), we see that F21 = p1 H 02 .

4.2 Computational Complexity

A cursory examination of the three Equations (38), (39), and (40) does not tell the story as to which one is best to use. They each require drastically dierent amounts of time to evaluate. We analyze them here.

Form 1: There are two computationally intense parts here. First of all, the sum of terms must

be large enough to guarantee convergence. If  is very close to 1, the number of terms can be extremely large. One might carry out the sum until the contributions are negligible (dangerous technique). Alternatively one might generate terms until the ratio [Un+1 V]= [Un V] is constant to many signi cant digits, and then assume that the series is geometric beyond that, thereby yielding a closed expression for the sum of the remaining terms. Let N be the number of terms computed. For each term, the computation of expressions of the form Uv0 and uU are needed, so as to build up terms with increasing Un . Each of these requires multiplication of a an m vector with an m  m matrix. this takes on the order of m2 numerical operations. Therefore, the order of complexity of computing Trc(1) is

C1 = O( N
m2 ):

Form 2: The major computational eort here is computing F. From the algorithm given, one must multiply matrices with each other. Most matrix multiplications can be done as matrices times vectors, but there is no getting around the squaring of U(k) . For each iteration, this requires O(m3 ) operations. The number of iterations required should compare with the number of terms needed in Form 1. This should be approximately log2 (N ). Therefore, C2 = O( log(N )
m3 ):

Form 3: This expression requires no iterations, BUT it does require the inversion of H, which is an m2  m2 matrix. The number of the order of the cube of the dimension of the matrix. Therefore, C3 = O( m6 ): Thus we see that the three formulas require vastly dierent amounts of eort for evaluation, depending on the parameters of the problem. We are not prepared to say which is best in general, but clearly, if m is even moderate, say > 9, then Form 3 can be very time consuming indeed. We have used Trc(2) in all our calculations in this paper.

16

4.3 Expressions For qc

We now return to (29) for the evaluation of qc . The rst form for qc is similar to Trc(1). The Vi 's are replaced by n. Thus:

qc(1) = (1 1 )(1 2 ) + (1 1 )(1 2 )

1 X

n=0

1 X

n=0

+ (1 1 )(1 2 )

n [U1 n ] [K2 U2 U2 n ]

n [K1 U1 U1 n ] [U2 n ] 1 X

n=0

n [U1 n ] [U2 n ]:

For qc(2) , just as with (32), we have an in nite sum to evaluate, namely

G := 21

1 X

n=1

n U2 n Q21 U1 n :

Then, since p2 G21 01 = p1 G12 02 ,  drops out and:

qc(2) = (1

1 )(1

2 ) (p2 K2 U2 G21 01 + p1 K1 U1 G12 02 + p2 G21 01 ) :

We can evaluate G21 with the following algorithm:

Algorithm for Calculating G

21

BEGIN PROCEDURE . *Initialization . G21 (0) := U2 F21 U1 . U1 (1) := U1 . U2 (1) := U2 . *Loop Until Converged . FOR k = 1 TO max, DO . G21(k) := G21 (k 1) + U2(k) G21(k . U1(k+1) := U1(k)
U1(k)

1)

U k: 1

( )

:= U2 (k)
U2 (k) . END FOR . G21 = G21 (max) END PROCEDURE As with the algorithm for F21 , the reader can convince yourself that

U

2

k

( +1)

G

21

k =

( )

NX1 n=1

n U2 n Q21 U1 n + N

1 X n=N

17

U n Q U n; where N = 2k : 2

21

1

(41)

It is unfortunate that the computation of G21 cannot be done in parallel with that for F21 , since G21 requires F21 as initial input. Thus all the U(i k) 's must either be stored or recalculated. The Kronecker product form for qc(3) is also similar to its mate, Trc(3). Now, the sum to be considered is: 1 X W := n (U1 U2 )n: n=1

As mentioned previously, expressions like this satisfy the same formulas as their scalar analogs. So, since 1 X n xn = (1 x x)2 ; we can write

n=1

W = ( U U ) [I U U ]
1

The expression for qc becomes

2

1

h

2

2

i

qc(3) = (1 1 )(1 2 )p K^1 U^1 + K^2 U^2 + I W 0 : If 1 = 2 , then the formulas for qc simplify to:

qc(1) = (1 )2

1 X

n=0

n [(2KU + I) Un ] [Un ];

(42)

(43) qc(2) = (1 )2 [(2KU + I) G]; ^ K ^ U^ + ^I) W]: qc(3) = (1 )2 [(2 (44) As with F and H, G and W are related by G21 = p1 W 02 . The computational complexity discussion for Trc(i) is equally valid for qc(i) . Then the mean

system time for the cheater is given by (27), using one of the three versions of (38), (39) and (40), plus (42).

18

5 Numerical Results For Various Service Times We have already seen that if the service distribution is exponentially distributed, the cheater will see the same mean system time as a customer to an M/M/2 queue. Thus even the cheater should not be in favor of separating two identical servers, since he cannot gain, and everyone else would lose. Here we wish to study what happens if the service times are not exponentially distributed. Of course, it is impossible to search through all classes of pdf's. On the other hand, the usual procedure of using Erlangians [Em (x)] to represent functions with C 2 < 1, and hyperexponential functions [H2 (x)] to represent functions with C 2 > 1 is inadequate. We have therefore included two other classes of functions to see how dierent functions with the same C 2 behave.

5.1 Representations of Distribution Functions

Figure 6: A Graphic representation of an Erlangian m distribution, Em(x). As is well known, Erlangian functions are of the form )m 1 e x Em (x) :=  (x (m 1)! with a mean of x = m=, and C 2 = 1=m. Their representation, shown in Figure 6, is m exponential servers in tandem. This gure clearly tells us what the various matrix objects described in Section 2 are. The entrance vector is p = [ 1 0 : : 0 ]. The transition matrix is 2 3 0 1 0 : 0 6 0 0 1 : 0 7 6 7 6 : : : : : 7 6 P = 6 : : : : : 77 ; 6 7 4 0 0 : 0 1 5 0 0 : 0 0 and the transition rate matrix is M = I. This leads to 2 3 2 3 1 1 : : 1 1 1 0 : 0 6 0 1 : : 1 7 6 0 1 1 : 0 77 6 7 6 6 : : : : : 7 6 : 7 1 : : : : 1 6 6 7 B = M (I P ) =  6 : : : : : 7 and V = B =  6 : : : : : 77 : 6 7 6 7 4 0 0 : 1 1 5 4 0 0 : 1 15 0 0 : 0 1 0 0 : 0 1 19

The utilization factor is  = m=. From these formulas, A and U, and all the other formulas, can be computed.

Figure 7: A pictorial representation of a hyperexponential 2 distribution, H (x). 2

The hyperexponential functions [H2 (x)], are made up of two exponential servers as alternate paths, as in Figure 7. Here, p = [ p 1 p ], P = O (O is the matrix of all zeros), and

M=



1 0 0 2



Thus B = M, x = p=1 + (1 p)=2 , and V is a diagonal matrix with Vii = 1=i. In the calculations that follow, we have assumed that p = 0:1, that 1 < 2 , and have selected the i 's so that x = 1 and C 2 is as given on the various graphs.

bHE (x)

Figure 8: A graphic representation of two Erlangian 2's in parallel, what we call a

hyper-Erlangian distribution. Since the PK formula for mean system time, (23), depends only on the rst and second moments of the service distribution, we have used two other classes of functions to see if other factors besides x and C 2 aect the total times of the cheater and other customers in the various systems. One type, which we call Hyper-Erlangian, [bHE (x)] is described in Figure 8. These functions dier from the Erlangians in that they can have C 2 > 1, and they dier from the

20

H2 (x)'s in that bHE (0) = 0, whereas H2 (0) = p1 + (1 p)2 > 0. We have p = [ p 0 1 p 0 ], and have set p = :1. In this case, 2 3 1 0 0 0 6 7 B = 64 01 01 02 00 75
0 0 2 2 The last class of functions we considered are called Truncated Power-Tail distributions [bT (x)] [LOWR93]. They are de ned as follows:

m mX1    j 1  1 (

 ) bT (x) := 1 m
1  e x= j ;

j =0

(45)

RT (x) =) xc

(46)

where > 1, and  < 1. These are hyperexponential functions with m phases whose service rates are given by = j , for j = 0 to m 1. The probability of going to phase j is j (1  )=(1 m ). These functions have the unusual property that for a large range of x, the range depending on the number of terms we use. In our calculations here we set m = 8. Then, with  = :5 and = 1:86267344, we have C 2 = 10:0000000, and  turns out to be about 1.2 over the range 1:2 < x < 30. [Note that for m = 1, ; and  are related by the formula,   = 1, which for these values of  and would give  = 1:114361622 and C 2 = 1!. Also, (46) is true for all x > 1:2.]

5.2 Comparison of Response Times

It is well known that all M/G/1 queues with the same rst and second moments have the same system time for any given arrival rate. Their system time is given by the PK formula, Equation (23). The same cannot be said about M/G/2 queues, or the cheater. As expected, all these queues have unbounded system times as  approaches 1. Therefore, as with Figure 6, we have plotted fi () := (1 ) Ti as a function of , since fi ( = 1) is nite for all i. We see that, C 2 + 1 for i = 1; 2; and c; lim f (  ) = (47) i !1 4 while it is twice as large for option 3. On such plots, the response times for M/G/1 queues (options 1 and 3) are represented by straight lines. It is interesting to observe that for the HE distribution, options 2 and c cross each other several times.

21

Figure 9: Comparison of Response Times for Four Options with Poisson Arrivals

to Erlangian n queues. The functions, fi ( ); for systems i = 1; 2; 3; c, and various Erlangians, are plotted versus  for the two service distributions, E2 (t) and E4 (t). From Equation (47), the curves for the dierent distributions can be identi ed by their values at  = 1. En functions have C 2 = 1=n. Thus for E2 , fi (1) = 3=8, and for E4 , fi (1) = 5=16. As can be seen in Figure 9, for Erlangian distributions (and perhaps for all distributions with C 2 < 1?), option 1 always outperforms option 2 which just barely outperforms the cheater. However, as shown in Figure 10, if C 2 is large enough (here, C 2 = 10 for all distributions), then the cheater and option 2 outperform option 1 for  large enough (in this case, for  > :25). From the gure we see that for the H2 function, option 2 signi cantly outperforms the cheater, while for the hyper-Erlangian, option 2 only marginally outperforms the cheater. On the other hand, the cheater gets much better service than option 2 for the distribution described in (45) and (46) where  = 0:5 and = 1:86267344. Figure 10 is also interesting because it shows clearly that the rst and second moments of a distribution are not the only important parameters for determining queueing system performance.

Figure 10: Comparison of Response Times for Four Options with Poisson Arrivals to three dierent non-exponential servers with C 2 = 10. The functions, fi ( ); for systems i = 1; 2; 3; c, are plotted versus . All three distributions have the same curve for options 1 and 3. For options 2 and c, the solid lines correspond to the H2 function, \

" corresponds to the distribution given by Equations (45) and (46), and \ " is for the hyper-Erlangian.

22

6 Unbalanced Systems All the numerical results we have discussed so far refer to balanced systems. Since it is unlikely that two independent queues would have exactly the same arrival rate, it is only fair to ask what the results would be if 1 and 2 diered somewhat. We take a brief look at that possibility here. Although we have established the formulas for unbalanced systems, we will only discuss M/M/1 queues. Unbalanced systems are not relevant to options 1 and 2, since they are single queue systems. For system 3, it is clear that if the arrival rates to two identical servers are dierent, then the one with the smaller arrival rate will yield a smaller system time than a balanced system, while the one with the greater arrival rate will have a larger system time. Their weighted average must be larger than the balanced system, since we know that balanced systems are optimal. This implies that small imbalances must have only a second order aect on response time. This is shown below. The cheater sees the reverse of what the other customers see. Clearly, if the system is unbalanced, he has a greater chance of nding a signi cant dierence in the length of the two queues. Thus for him, the balanced system is the worst possibility, and once again small imbalances should yield only second order changes in response times. In examining imbalance, let us assume that x = 1. Then from (1), i = i , and from (2),  = =2. Next, de ne  so that

1 =  +  (1 ); and 2 =   (1 ): Then

(48)

1 2 = 2 2 (1 )2 ;

and

1

2 = 2(1 ) :

(49) We are assuming that no system will become so imbalanced that one of the servers becomes oversaturated, in which case there would be no steady state. The paramerization we have chosen guarantees that this cannot happen as long as jj < 1. We insert these formulas for the i 's into (12) to get (the subscript 'u' stands for unbalanced) Tcu := 1 1  = 1 2 + 12 (1 )2
1 2 Thus for the cheater, the dierence between the unbalanced, and balanced systems (13) [as well as the M/G/2 system, (7)] is given by 2 Tcu = (1 + )2 + 2 (1 2 )
As expected, the dierence is positive for all  and all , and is quadratic in . Note that the dierence is nite even when  = 1, in which case it is equal to (=2)2 . This nite result is due to the way we de ned . Observe from (49) that as  approaches 1, the dierence between the two 's goes to 0 irrespective of the value of .

T2

Tcu = Tc

For completeness, we now look at option 3 to see how imbalance aects system time. From (8), inserting (48), we get 



 (1 ) =  + 2 (1 )
T3u = 21 1  +  (1(1 )) + 1   + (1 ) (1 )(1 2 ) 23

As promised, we see that the change due to imbalance depends on 2 . Notice that the system time blows up for  = 1 for any , since in that case 2 = 1. The dierence between these balanced and unbalanced systems works out to be

T3u

2 T3 = (1 )(1 2 )


That is, the dierence is of order 2 , and becomes unboundedly large as  approaches 1. The ratio of the two, however, is bounded for  < 1, and is equal to 1 T3u =  + 2 (1 ) =)  ! 1 2  (1  ) 1 2 T3

7 Conclusions A customer who can choose the shorter of two queues, each of which accesses a single server, will obviously have a much shorter waiting time than all other customers who must choose a queue and server randomly. However it is not clear how his waiting time compares with those customers who access two servers through a single queue, i.e., an M/G/2 queue. Using LAQT, we have shown for a wide class of service time distributions, that he does no better than the M/G/2 queue. We have described three dierent algorithms which can be used to to do these calculations, and examined the computational complexity of each. Depending on the dimension of the matrix representation, and other properties of the distribution of interest, each of the three algorithms could be best. The methods described here could be used to study other problems involving the behavior of two separate queues.

24

8 REFERENCES [CARR82] Carroll, J. L., Appie van de Liefvoort, and Lester Lipsky, \Solutions of M/G/1//Ntype Loops with Extensions to M/G/1 and GI/M/1 Queues," Operations Research 30, pp. 490{514, 1982. [CHOW77] Chow, Y. C., and W. H. Kohler, \Dynamic Load Balancing in Homogeneous Two{ Processor Distributed Systems," Computer Performance, North Holland, pp. 39{52, 1977. [LIPS92] Lipsky, Lester, QUEUEING THEORY, A Linear Algebraic Approach, MacMillan, New York, 1992. [LOWR93] Lowrie, Walter, and Lester Lipsky, \A Model For The Probability Distribution of Medical Expenses," Proceedings of CONFERENCE OF ACTUARIES IN PUBLIC PRACTICE, (1993). [NEUT75] Neuts, M. F., \Probability Distributions of Phase Type," in Liber Amicorum Prof. Emeritus H. Florin, Department of Mathematics, University of Louvain, Belgium, 173{206, 1975. [NEUT82] Neuts, M. F., \Explicit Steady{state Solutions to Some Elementary Queueing Models," Operations Research 30, pp. 480 489, 1982. [TEHR83] Tehranipour, Aby, Explicit Solutions of Generalized M/G/C//N Systems Including an Analysis of Their Transient Behavior, Ph.D. Thesis, University of Nebraska, Lincoln, December 1983.

25