Response time distributions in networks of queues Michael Grottke, Varsha Apte, Kishor S. Trivedi, and Steve Woolet
1 Introduction The problem of computing the response (sojourn) time distribution in queuing networks has been researched extensively during the past few decades. (For a somewhat dated survey see [6].) In case of open queuing networks, a considerable amount of work has been done in computing the response time distribution in the domain of Jackson networks. Closed form solutions have been derived for the (Laplace-Stieltjes transform of) response time distributions through a particular path in product-form queuing networks [38]. Furthermore, many results exist for response time distributions in networks with a specific topological structure such as tandem, central server, single queue with feedback and so on. The communications literature also shows a focus on end-toend packet delay in tandem-type queuing networks, with characteristics specific to communication systems. However, it is very difficult to derive exact closed form solutions for networks with even slightly non-restrictive topology as well as service and arrival characteristics. In the face of such difficulties, the two approaches taken are: (1) numerical solution and (2) approximate solution. In case of closed Markovian queuing networks, the tagged customer approach [30] may be used to numerically compute the response time distribution of a network with a general topology. However, this technique consists of generating the state Michael Grottke Department of Statistics & Econometrics, University of Erlangen-Nuremberg, Germany, e-mail:
[email protected] Varsha Apte Department of Computer Science, IIT Bombay, India, e-mail:
[email protected] Kishor S. Trivedi Department of Electrical & Computer Engineering, Duke University, Durham, USA, e-mail: kst@ ee.duke.edu Steve Woolet IBM Systems & Technology Group, IBM Corporation, Durham, USA, e-mail: steve_woolet@ us.ibm.com
1
2
Michael Grottke, Varsha Apte, Kishor S. Trivedi, and Steve Woolet
space of the queuing network and may result in a very large state space. Section 2 will describe this numerical approach. Methods for efficiently analyzing response time densities in very large Markov and semi-Markov models have recently been shown in [7] and [12]. In case of open queuing networks with unlimited capacity queues, the tagged customer approach is not feasible at all. Thus, a lot of research has been devoted to finding approximations to response time distribution which are space and time efficient. Abate et al. [1] consider approximations for G/GI/1 queue sojourn time tail probabilities. Au-Yeng et al. [2] present a technique using generalized lambda distributions for approximating response time densities in Markov and semi-Markov models. Van Houdt and Blondia [44] approximate waiting time distributions by steady-state analysis of reset Markov chains. Van Velthoven et al. [45] present methods for calculating the response time distribution of impatient customers in discretetime queues. One class of approximations which address the response time distribution problem have been termed “independent flow time approximation” (IFTA) by Boxma and Daduna [6]. This approximation states that “arrival state distributions seen by a test customer on the arrival at successive network stations are independent of each other and equal the stationary arrival state distributions at these stations as seen by an arbitrary customer” [6]. It has been applied by Harrison [14], as well as Shanthikumar and Buzacott [40]. In the approach suggested by Harrison [14], decomposition of queues is used to compute the response time distribution. The basic idea is to find arrival rates to each queue and then analyze each queue in isolation. If we know the response time distribution of a job at each of the queues in the network, then the response time of a job through a particular path in the queuing network can be computed as the convolution of the response time distributions at each queue in the path. A response time computation technique that builds on Harrison’s method was presented in its nascent form by Woolet [48]. The key idea there was as follows: Assume that the response time distribution at each queue in the queuing network is phase-type, i.e., it corresponds to the absorption time distribution of a continuoustime Markov chain (CTMC). Then if we construct the CTMCs corresponding to all queues, “glue” them together according to the topology of the network, and add an absorbing state representing departure from the network, we have another CTMC. The absorption time distribution of this CTMC approximates the response time distribution in the same way as Harrison’s method does. The advantage of this method is that it (1) provides a clear representation, in the form of a CTMC, for the approximate computation technique, (2) maps the problem to a well-explored problem of transient solution of CTMCs and (3) allows us to extend the original technique to more general topologies and service time distributions, by taking advantage of this CTMC representation. This method also maintains the advantage of space and time efficiency that Harrison’s method has. It must be noted that though we do use CTMCs in this method, they do not represent the state space of the queuing network. In fact, the size of the CTMC is linear in the number of queues in the network. In [48], this method was presented with some examples for a network of
Response time distributions in networks of queues
3
M/M/c/b ≤ ∞ queues with no loss of customers. Section 3 will describe this application to open queuing networks. We extend this method in Sect. 4 to cover queues in which the service time follows a phase-type distribution. Two approaches that can be adopted if the response time distribution at each queue is not phase-type are described in Sect. 5. First, we can fit a phase-type distribution to the response time distribution and again map the problem to a CTMC transient solution problem. Most of the results in the literature provide the response time distribution at a queue only in the form of its Laplace-Stieltjes transform (LST). This poses the additional problem of matching two distributions, given only their LSTs. We explore this problem and present our observations and suggestions. Another approach is to leave the CTMC domain altogether and map the queuing network to a semi-Markov process as was done by Mainkar [28]. Since the response time distribution at each queue and the routing probabilities are available, the same idea of “gluing” together states which represent response time distributions will result in a semi-Markov process whose kernel, holding times in states, and embedded discrete-time Markov chain (DTMC) are known. Again, we approximate the response time distribution of the queuing network by the absorption time distribution of the semi-Markov process. The number of states of this semi-Markov process is linear in the number of queues in the queuing network. In this chapter, these basic ideas are used to develop approximations of response time distributions for a variety of queuing networks.
2 Closed Markovian networks As stated previously, closed form solutions for the response time distributions of queuing networks are obtainable only for simple queuing system models. Methods for computing the Laplace transforms of the response time distribution in queuing networks satisfying the non-overtaking condition are proposed by Lemoine [24] as well as Walrand and Varaiya [46]. Melamed and Yadin [31] have shown that numerical computation of the response time distribution is possible, although this requires the construction and solution of large Markov models.
2.1 Tagged customer approach The tagged customer approach is one method of numerical computation of the response time distribution for queuing networks with general interconnectivity. Melamed and Yadin [31] present this approach for evaluating the response time distribution in a discrete state Markovian queuing network. An arbitrary customer is picked as the tagged customer and its passage is tracked through the network. The response time distribution, conditioned on the state of the queuing network at the time of arrival of the tagged customer, is computed. Deriving this conditional
4
Michael Grottke, Varsha Apte, Kishor S. Trivedi, and Steve Woolet
response time distribution of the tagged customer is transformed into a problem of solving for the distribution of the time to absorption of a finite-state CTMC. For closed product form queuing networks, an arriving customer will see the network in equilibrium with one less customer and we can establish the distribution of the other customers in the network at the instant of arrival of the tagged customer (see the arrival theorem of Sevcik and Mitrani [39] or Lavenberg and Reiser [23]). All the states in which the tagged customer may find the queuing network upon arrival (i.e., how the remaining jobs are distributed among the queues in the network) must be determined to make possible the computation of the unconditional response time distribution. Computing the response time distribution using the tagged customer approach is therefore a two-step process.
2.2 Example: Central server model Figure 1 shows a closed queuing network model of a computer system, a central server model (CSM), that will be used to describe the tagged customer approach. Customers first join the CPU queue. The CPU, Disk 1 and Disk 2 are assumed to have exponentially distributed service times with parameters µC , µD1 and µD2 , respectively. The service discipline at all the queuing centers is assumed to be first come, first served (FCFS). This closed system contains n customers. A customer will request access to Disk 1 with probability p1 and Disk 2 with probability p2 after receiving a burst of service at the CPU; the customer rejoins the CPU queue for another burst of service after completing access to the disks. With probability p0 = 1 − (p1 + p2 ) the customer may complete execution, after which it is replaced by a statistically identical customer. p0 new program path
11 00 00 11
p1
Disk1
11 00 00 11
CPU p2
Disk2
Fig. 1 Central server model of a computer system
As mentioned earlier, finding the response time distribution of a queuing network is transformed into solving for the absorption time distribution of a CTMC. Figure 2 shows the CTMC whose absorption time distribution needs to be computed for the case of the CSM with only n = 2 customers.
Response time distributions in networks of queues
5
010 00 p0 µC
µD1
110 11
020 21
p1 µC
p2 µC
µD2
p1 µC 020 22
011 31
µD1
p1 µC
p2 µC
µD1
µD1
101 31 p1 µC
002 32
µD2
µD2 p2 µC
p2 µC
µD2 p0 µC
002 31
p1 µC
200 11
µD1
p2 µC 110 21
p0 µC p2 µC
200 12
µD1 011 21
µD2 µD2
p1 µC
101 11 p0 µC 001 00
100 00 State label : (i j k l m) No. of jobs in CPU No. of jobs in Disk1 No. of jobs in Disk2 Queue in which tagged customer is present (1=CPU, 2=Disk1, 3=Disk2) m: Position of tagged customer in queue l = 0, m = 0: Job has exited from the system i: j: k: l:
Fig. 2 CTMC model of the CSM for computing the response time distribution
It is interesting to note that a rather complex CTMC is obtained even for a system with only two customers. In Fig. 2, the number of customers at the CPU, Disk1, and Disk2, respectively, are indicated by the first three components of the state label. The position of the tagged customer is indicated by the next two components; the index of the queue in which the tagged customer resides corresponds to the first component, the position of the tagged customer in the queue corresponds to the second. The queues are numbered as follows: 1 (CPU), 2 (Disk1), and 3 (Disk2). The state 00 is used to indicate that the tagged customer has departed from the system, i.e., has reached the absorbing state of the CTMC. States (10000), (01000),
6
Michael Grottke, Varsha Apte, Kishor S. Trivedi, and Steve Woolet
and (00100) are the three absorbing states in the Markov chain, and are explicitly identified in the figure by the squares enclosed within the circles. The three states explicitly identified in this figure by double circles are those states in which the tagged customer may arrive into the queuing system. These are states (20012), (11011), and (10111), which correspond to the other job being at the CPU, Disk1, and Disk2, respectively. We can compute the absorption time distribution of the CTMC assuming that we start with any of these arrival states as the initial state. Each absorption time distribution represents the conditional response time distribution of the tagged customer, conditioned on a specific position of the other customer at the instant of arrival of the tagged customer into the queuing system. Let I be the set of all states in the CTMC whose solution yields the response time distribution. The set of absorbing states in the CTMC is represented by A (⊆ I), and the set of states in which the tagged customer will find the network at the instant of arrival is represented by S (⊆ I). S is the set of all possible states of the network with one less customer for a closed queuing network. The random variable representing the response time for an arbitrary customer arriving when the queuing network is in state i, where i ∈ S, is Ri . Define pi j (t) as the transient probability of being in state j at time t given that the initial state of the CTMC in Fig. 2 is i. Then P(Ri ≤ t) =
∑ pi j (t).
j∈A
To compute the unconditional response time distribution, we require the (steadystate) probabilities πi (n − 1) that the tagged customer will see the network with the other n − 1 customers in state i (i ∈ S) at the instant of arrival. These probabilities are computed based on a further CTMC. In Fig. 3, this CTMC is shown for the case of n = 2 customers. It has three states corresponding to the non-tagged customer being present at the CPU, Disk1, and Disk2, respectively. µ D2
µ C p1 010
100
001 µC p 2
µ D1
State label : (i j k) i: No. of jobs in CPU j: No. of jobs in Disk1 k: No. of jobs in Disk2 Fig. 3 CTMC model for computing the steady-state probabilities of the non-tagged customer
Let the random variable representing the unconditional response time be R. Then a general expression for the unconditional response time distribution is
Response time distributions in networks of queues
P(R ≤ t) = =
7
∑ πi (n − 1)P(Ri ≤ t) = ∑ πi (n − 1) ∑ pi j (t)
i∈S
i∈S
∑ ∑ πi(n − 1)pi j (t) =
∑
j∈A i∈S
j∈A
p j (t),
j∈A
where p j (t) represents the unconditional transient probability of being in state j. For n = 2 customers, these probabilities are obtained by solving the CTMC in Fig. 2 for its transient probability vector at time t, given the initial probability of the state i (∀i ∈ S) of the CTMC is πi (n − 1) and the initial probabilities of all the other states (i ∈ I − S) are zero. The unconditional response time distribution is thus directly computed by assigning the initial probabilities for the CTMC and carrying out the transient analysis only once. For this example, we set µC = 50.0, µD1 = 30.0, µD2 = 20.0, p1 = 0.45, and p2 = 0.3. Figure 4 shows the response time distributions of the CSM for different numbers of customers n (5, 10 and 15). Note that when there are fewer customers competing for resources (i.e., the number of customers is smaller), a customer has a higher probability of completing by a given time t.
1
3
0.9
+ 3
0.8
P(R ≤ t)
0.7
0.4 3 3 0.3 3 3 3+ 3+ 0.2 3 + + 3+ 2 2 0.1 +2 +2 2 + 2 2 0 0
+
3
0.6 0.5
2
3
+ 3
+
2 +
2
2 5 custs. 3 10 custs. + 15 custs. 2
2
0.5
1
1.5 t
2
2.5
3
Fig. 4 Response time distribution of the CSM for different number of customers
For a general method of constructing the two CTMCs for computing the response time distribution see [33], and for automated construction and solution of such models using stochastic Petri nets see [32]. See [7] and [12] for methods for efficiently analyzing response time densities in very large Markov and semi-Markov models.
8
Michael Grottke, Varsha Apte, Kishor S. Trivedi, and Steve Woolet
3 Open Markovian networks of M/M/c/b ≤ ∞ queues As we have seen in the previous section, even for a rather simple closed system in which only few jobs are present, the total number of states that the system can take may be high. For open Markovian networks, in which the number of jobs in the system is not fixed, computing the response time distribution via the tagged customer approach basically involves an infinite number of states. However, there are methods of approximating the distribution that can give good results with much less effort. One such method uses the original network and knowledge of the response time distribution of its nodes to construct a CTMC for which the distribution of the time to reach the absorbing state can be solved to find the approximate response time distribution of the queuing network. In its simplest form, this method requires that the response times at the individual queues can be represented by the time until absorption in a CTMC. Fortunately, this is the case for many types of queues. For five fairly simple types of queues, we present the CTMCs corresponding to their response time distributions in Sect. 3.1. These “response time blocks” can then be used to build the CTMC for the entire queuing network via the procedure laid out in Sect. 3.2. The approach is illustrated with three examples in Sect. 3.3.
3.1 Response time blocks Throughout this section, we assume that all nodes use FCFS scheduling. Let λ and µ denote the arrival rate to the node and the service rate of each server of the node, respectively. To maintain stability, we assume that λ < cµ , where c is the number of servers in the node. This is equivalent to the condition that the traffic intensity at the node, ρ = cλµ , is smaller than one.
3.1.1 M/M/1 For an M/M/1, FCFS server with arrival rate λ and service rate µ , the response time distribution is given by Gross et al. [13] as F(t) = 1 − exp(−(µ − λ )t), assuming that λ < µ . The response time block is shown in Fig. 5. State “In” indicates the starting state of the piece of the CTMC model representing the M/M/1 queue in the corresponding network model. The “Out” state will be either an “In” state for another network model node or the absorbing state representing the job leaving the network model. If the output of the queuing node can proceed on more than one path, then the “Out” state shown would actually be multiple states and the incoming arcs to these states would be weighted with the appropriate probabilities.
Response time distributions in networks of queues
9
µ−λ In
Out
Fig. 5 The M/M/1 response time block
3.1.2 M/M/∞ The M/M/∞ server is the simplest node for which to find the response time distribution. Since there are always enough servers for any customer, the response time distribution is the same as the service time distribution, F(t) = 1 − exp(−µ t). Figure 6 shows the response time block for this distribution. It is very similar to Fig. 5, except that for the M/M/∞ case the rate of leaving the “In” node is simply µ. µ In
Out
Fig. 6 The M/M/∞ response time block
3.1.3 M/M/c If there are assumed to be some finite number, c, of servers each having the same service rate, µ , and an infinite queue, we have an M/M/c, FCFS queue. The M/M/c, FCFS response time distribution given by Gross et al. [13] as (1) F(t) = Wc (1 − exp(−µ t)) cµ − λ µ [1 − exp(−µ t)] − [1 − exp(−(cµ − λ )t)] , + (1 − Wc) (c − 1)µ − λ (c − 1)µ − λ where
#−1 "c−1 (cρ )c (cρ )k (cρ )c 1 1 Wc = 1 − · + · · ∑ c! 1−ρ c! 1−ρ k=0 k!
is the steady-state probability that a newly arriving job finds less than c jobs present in the node and therefore does not have to queue for service.
10
Michael Grottke, Varsha Apte, Kishor S. Trivedi, and Steve Woolet
The distribution in Eq. (1) is a mixture of Wc fraction following an exponential distribution with parameter µ and (1 − Wc ) fraction following a two-stage hypoexponential distribution with parameters µ and cµ − λ . It can be described by the building block shown in Fig. 7. The upper path represents the exponentially distributed portion and the lower path is the hypoexponentially distributed portion. State T is strictly a transient state that is required to obtain the hypoexponential distribution. µWc In
Out
µ (1 − Wc )
cµ − λ T
Fig. 7 The M/M/c, FCFS response time block
3.1.4 M/M/c/b If we limit the previous case to a finite queue length (or buffer size) of b (where the queue includes the customer being serviced), we have the M/M/c/b, FCFS queue. The distribution given here is for an open network, assuming the job is accepted for service. In [43], the distribution for this case has been given as c−1
F(t) =
∑ qn(1 − exp(−µ t))
n=0
b−1
+ ∑ qn n=c
(
n−c+1 c (1 − exp(−µ t)) c−1 #) n−c−i+1 " i n−c (µ ct) j c 1 1− ∑ exp(−µ ct) , −∑ c j! j=0 i=0 c − 1
where #−1 " c−1 c b−1 n k (c c ρ ) ρ ) (c if n = 0, 1, ..., c − 1, · ∑ + ∑ ρk n! c! k=c k=0 k! qn = #−1 " c−1 cρ n k c b−1 c ρ ) (c c k if n = c, c + 1, ..., b − 1 c! · ∑ k! + c! ∑ ρ k=c k=0
(2)
Response time distributions in networks of queues
11
represents the conditional steady-state probability that a newly arriving job which is not blocked by the node due to a full buffer finds n other jobs present at the node. This distribution is a mixture of an exponential distribution with parameter µ and b − c hypoexponential distributions. Each hypoexponential distribution has a different number of phases. More specifically, the ith hypoexponential distribution (i = 1, . . . , b − c) consists of i + 1 phases; one phase has parameter µ , and the remaining i phases have parameter cµ . Figure 8 shows the response time block representation of the distribution. Similar to the M/M/c block, the states T2 , · · · , Tb−c+1 are strictly transient states required to obtain the hypoexponential distributions. The probabilities V j that appear in the transition rates are given by
Vj =
c−1 ∑ qn
In
µ(1−V1 )
µ V1
if j = 1,
n=0
qc+ j−2 b−1 ∑n=c+ j−2 qn
T2
cµ (1−V2 )
if j = 2, ..., b − c + 1.
cµ(1−Vb−c−1 )
c µV2
Tb−c
cµ (1−Vb−c )
cµVb−c
Tb−c+1 cµ
Out
Fig. 8 The M/M/c/b response time block
3.1.5 M/M/1/b By limiting the M/M/c/b case to a single server, we have an M/M/1/b, FCFS queue. The distribution given in this section has the same assumptions and qualifications as that in the M/M/c/b section. The response time distribution for the M/M/1/b, FCFS case is given by Gross et al. [13] as ! n 1 − ρ b−1 n (µ t)k exp(−µ t) . F(t) = ∑ ρ 1− ∑ 1 − ρ b n=0 k! k=0 Note that this distribution is a mixture of Erlang distributions where each has parameter µ and the number of phases varies from 1 to b. Setting c = 1 and
12
Michael Grottke, Varsha Apte, Kishor S. Trivedi, and Steve Woolet
Vj =
1−ρ if 1 ≤ j ≤ b, 1 − ρ b− j+1
we can again use Fig. 8 to represent the M/M/1/b response time block (cf. [48]).
3.2 Building the Markov chain from a queuing network With the building blocks for five types of queues described in Sect. 3.1, we can now outline a procedure for automatically mapping a queuing network to a response time CTMC. Consider a network consisting of m queues 1, 2, . . . , m, where each queue is of one of the types described in Sect. 3.1. Let λ0i be the external arrival rate to node i in the queuing network. Suppose ri0 is the probability that a customer exits the network after receiving service at node i. Let rii′ be the probability that a customer proceeds to node i′ after receiving service at node i. If queue i′ is a finite capacity queue and it is full to its capacity, then we consider the following possibilities [36]: (1) The customer waits at queue i and is retried. (We call this policy WAIT.) (2) The customer is lost. (This policy is termed LOSS.) Suppose the probability of loss at queue i′ is pbi′ . (Thus pbi′ = 0 for infinite capacity queues.) Then the procedure for building the overall CTMC is described as follows: Step 1: Calculate effective arrival rates to each node of the queuing network. The effective arrival rate λi′ to node i′ is given by [15] m
λi′ = λ0i′ + ∑ λi (1 − pbi)rii′
for i′ = 1, 2, ..., m.
(3)
i=1
If all queues in the network are either infinite capacity queues or finite capacity queues with WAIT policy, i.e., if pbi = 0 ∀i, then (3) is a simple linear system of equations. However, things are more complicated if customers can get lost in the system. Let queue i of the network be an M/M/c/b queue with LOSS policy and the same service rate µi for each of the c servers. Assuming that the arrivals to this queue are Poisson, the expression for pbi is cc ρib pbi = · c!
"
(cρi )k cc b ∑ k! + c! ∑ ρik k=c k=0 c−1
#−1
.
Since ρi = cλµii , substituting this expression into Eq. (3) results in a system of equations that may need to be solved using fixed-point iteration to yield the effective arrival rates. The buffer-full probabilities can then be calculated based on these values of the arrival rates. The effective arrival rates as well as the bufferfull probabilities may be approximate because we assume that the arrivals to all queues are Poisson, which is not necessarily the case.
Response time distributions in networks of queues
13
Step 2: Create a CTMC with a state S f . S f is the state which denotes exit out of the queuing network. If the full buffer policy at any queue is LOSS then a state Sl is also created which denotes loss due to encountering full buffers. For each queue i, create a set of states Si = {Si1 , Si2 , . . . , Sini }, consisting of all the states but the “Out” state contained in the response time block representation of this queue (see Sect. 3.1). Therefore, 1 if queue i is M/M/∞ or M/M/1, 2 if queue i is M/M/c, ni = b if queue i is M/M/1/b, b − c + 1 if queue i is M/M/c/b. Sm
Then the state space of the CTMC is given by S = ( max{ni | i = 1, 2, . . . , m}. Since
i=1 Si )
S
{S f , Sl }. Let d =
m < |S| ≤ dm + 2, the total number of states |S| is linear in m. Step 3: The response time distribution of the types of queues considered above are phase-type [42] and hence, in general, can be represented as follows: Let the sojourn time in the response time block state j of queue i follow an exponential distribution with rate µi j and let the probability of exiting the queue from that state be Vi j . Assume we define probability pbi for all queues, i.e., it is zero for infinite capacity queues. Then the generator matrix Q of the CTMC defined in Step 2 is constructed based on the following principles. The progress within the response time block of queue i, from state j to j + 1, is preserved as in the block: QSi j ,Si( j+1) = µi j (1 − Vi j ). To represent routing from node i to i′ , QSi j ,Si′ 1 = µi jVi j rii′ (1 − pbi′ ). For denoting exit from the network, QSi j ,S f = µi jVi j ri0 . If there is any queue with LOSS policy in the queuing network, then we also have, in addition to the above, QSi j ,Sl = µi jVi j ∑ rii′ pbi′ . i′ 6=i
Note that in case of WAIT, we assume that the time until retry is the same as the time to exit queue i starting from the state from which exit was attempted. We also assume that on retry the queue to which the job is routed is sampled again.
14
Michael Grottke, Varsha Apte, Kishor S. Trivedi, and Steve Woolet
The rest of the entries of Q are zeros except for the diagonal entries, which are QSi ,Si = −
∑
QSi ,Si′ .
Si′ ∈S,Si′ 6=Si
Step 4: If we define λ = ∑m i=1 λ0i , then λ0i /λ is the probability that an external customer arrives at queue i. Then the initial state probability is set to λ0i /λ for state Si1 and to 0 for all other states. The cumulative distribution function of the response time R of the network can now be found by computing PS f (t), the probability of being in state S f at time t. Since S f is an absorbing state, this probability is equal to the probability of exiting the network before time t, given that the customer entered it at time 0. Note that if we have LOSS, this distribution will be defective, i.e., limt→∞ P(R ≤ t) = limt→∞ PS f (t) < 1. This is because there is a non-zero probability that the customer will never reach the state S f . Let us assume • that successive service times are independent (Kleinrock’s independence assumption), • that the independent flow time approximation as described in Sect. 1 holds, • that external arrivals are Poisson, • that all queues are single server with exponentially distributed service time and FCFS queuing discipline, • that all queues are infinite capacity queues, • that the network is feed-forward, and • that all the paths in the network are overtake-free [6]. If the above conditions are met, arrivals at all queues are Poisson, and further, successive sojourn times in a path are independent. In that case, PS f (t) as computed by the above algorithm gives the exact distribution of sojourn time. When any of the above conditions are violated, the absorption time distribution is an approximation. The successive sojourn times in queues in a path with overtaking or feedback are correlated [6], thus violating the implicit independence assumption in our method.
3.3 Examples We will now illustrate the approach developed above using a computer system example and a distributed system example. We also show how this methodology can be used to find the distribution of the sample average of response times.
Response time distributions in networks of queues
15
3.3.1 Computer system Consider the simple model of a computer system (Fig. 9), which is comprised of two M/M/1 queues.
µ1 λ
1
CPU
01
µ2
p
I/O
p
0
Fig. 9 Model I, an open network with feedback
Defining the effective arrival rates to the CPU queue and the I/O queue to be λ1 and λ2 , respectively, we follow Step 1 and formulate the equation system
λ1 = λ01 + λ2 , λ2 = λ1 p1 . This system of equations can easily be solved for the effective arrival rates,
λ01 λ01 = 1 − p1 p0 p1 λ01 . λ2 = p0 λ1 =
and
We now build the CTMC from the queuing network according to Steps 2 and 3. The state space S consists of the “In” nodes of the two M/M/1 response time blocks, denoted by S11 and S21 , respectively, as well as the exit node S f . Since both queues have infinite capacity, pb1 = pb2 = 0. The rate of leaving the “In” state of an M/M/1 response time block is calculated as the service rate minus the arrival rate; consequently, we have µi1 = (µi − λi ) for i = 1, 2. Moreover, the probabilities of leaving the current node is equal to one for the states S11 and S21 , i.e., V11 = V21 = 1. According to the model, a job leaving the CPU can either exit the network with probability r10 = p0 or progress to the I/O queue with probability r12 = p1 ; all jobs finished at the I/O queue return to the CPU queue; i.e., r21 = 1 and r20 = 0. Therefore, the only off-diagonal entries of Q that are not equal to zero are QS11 ,S21 = (µ1 − λ1 )p1 , QS11 ,S f = (µ1 − λ1 )p0 , QS21 ,S11 = µ2 − λ2 . The corresponding CTMC is depicted in Fig. 10.
16
Michael Grottke, Varsha Apte, Kishor S. Trivedi, and Steve Woolet
( µ 1 − λ 1) p1 S21
S11 µ2− λ 2 ( µ 1 − λ 1 ) p0
Sf
Fig. 10 CTMC corresponding to the response time distribution of model I
It can be shown that the expected response time in this queuing model is given by [42, p. 561] 1 1 . + E(R) = p0 µ1 − λ01 p0pµ2 − λ01 1
Obviously, the same expected response time is obtained in the simple network without feedback shown in Fig. 11. However, as we will see, the cumulative distribution function of the response time in this “equivalent” model II is different from the one in the original model I.
λ 01
p µ 0 1
p0 µ2 p
"CPU"
"I/O"
1
Fig. 11 Model II, an “equivalent” network without feedback
The response time CTMC for queuing network model II is built from two M/M/1 response time blocks with µ11 = p0 µ1 − λ01 and µ21 = p0pµ1 2 − λ01 . As before, pbi = 0 and Vi1 = 1 for i = 1, 2. Since all jobs proceed from queue 1 to queue 2 and then leave the network, r12 = r20 = 1. The non-zero off-diagonal elements of Q are thus QS11 ,S21 = p0 µ1 − λ01, p 0 µ2 QS21 ,S f = − λ01. p1 The CTMC is shown in Fig. 12. For a specific example, we assume that the arrival rate from outside the system is λ01 = 1 job per second and that a job leaves the system with a probability of p0 = 0.2 after being processed at the CPU. We further assume that the CPU can process jobs at a rate of µ1 = 10 jobs per second and the I/O can process jobs at a rate of µ2 = 5 jobs per second.
Response time distributions in networks of queues
17
p 0 µ2 p1 − λ 01
p0 µ1 − λ 01 S11
S21
Sf
Fig. 12 CTMC corresponding to the response time distribution of model II
0.6 0.4
Model I simulation Model I approximation Model II simulation and approximation
0.0
0.2
P (R
