queueing networks, that have product-form stationary distributions. ... rem 8 says that the network has a product form and is biased locally balanced if and.
Queueing Systems 28 (1998) 377–401
377
Markov network processes with product form stationary distributions X. Chao a , M. Miyazawa b , R.F. Serfozo c and H. Takada b a
Department of Industrial and Manufacturing Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA b Department of Information Sciences, Science University of Tokyo, Noda, Chiba 278-0022, Japan c School of Industrial and System Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
Received 3 March 1997; revised 15 February 1998
This study concerns the equilibrium behavior of a general class of Markov network processes that includes a variety of queueing networks and networks with interacting components or populations. The focus is on determining when these processes have product form stationary distributions. The approach is to relate the marginal distributions of the process to the stationary distributions of “node transition functions” that represent the nodes in isolation operating under certain fictitious environments. The main result gives necessary and sufficient conditions on the node transition functions for the network process to have a product form stationary distribution. This result yields a procedure for checking for a product form distribution and obtaining such a distribution when it exits. An important subclass of networks are those in which the node transition rates have Poisson arrival components. In this setting, we show that the network process has a product form distribution and is “biased locally balanced” if and only if the network is “quasi-reversible” and certain traffic equations are satisfied. Another subclass of networks are those with reversible routing. We weaken the known sufficient condition for such networks to be product form. We also discuss modeling issues related to queueing networks including time reversals and reversals of the roles of arrivals and departures. The study ends by describing how the results extend to networks with multi-class transitions. Keywords: Markov network process, product form, stationary distribution, queueing network, negative customer, migration process, quasi-reversibility, biased local balance, reversible routing, multi-class transitions
1.
Introduction
This study characterizes a large class of Markov network processes, including queueing networks, that have product-form stationary distributions. Such a distribution is a product of its marginal distributions (this is different from the use of “product form” to refer to weak coupling such as in [3,20]). Product forms are usually associated with the notion of quasi-reversibility. Loosely speaking, a queueing system is quasireversible if Poisson arrivals imply Poisson departures when the system is stationary; J.C. Baltzer AG, Science Publishers
378
X. Chao et al. / Markov network processes
see Kelly [11], Muntz [15], Walrand [21] and Whittle [22]. A queueing network is quasi-reversible if each of its nodes in isolation operates like a quasi-reversible queue. For a conventional queueing network, it is well known that quasi-reversibility is a sufficient condition for it to have a product form distribution; see, for instance, [10,12,22]. A similar result was recently proved in [4,5] for networks with special signals such as negative customers that delete customers. In some cases, quasi-reversibility is also a necessary condition for a network to have a product form distribution. This is true for a conventional queueing network with Poisson arrivals when all the nodes are internally balanced (their arrival and departure rates with arbitrary routing probabilities are identical) and each node has another restrictive property when it is in an empty state; see Malinkovsky [14]. In addition, Harrison and Williams [8] proved that a diffusion process corresponding to a queueing network with feed-forward routing has the product form if and only if each node is quasi-reversible, appropriately defined for diffusions. Such results led to the conjecture that quasi-reversibility is a necessary condition for a product form in a queueing type of network (e.g., see [7, p. 181, lines 16–22]). This is generally not true, however, as we show in example 18. We also prove that if a network has a product form distribution and satisfies a “biased local balance” condition introduced in [5], then it is quasi-reversible. This result is a consequence of our general characterization of product form distributions, which we now describe. The focus of our study is a Markov network process that represents a network with a finite number of interacting components or a queueing network. Our results also apply to vector-valued Markov processes, but we use network terminology to describe our results. The state of the network process is a vector of the states of the individual nodes that may be very general but countable. Each node has three types of state changes, called arrival, departure and internal transitions. These transitions are distinguished only by the rates or probabilities at which they occur; they may occur separately or concurrently. A transition of the network involves changes at only one or two nodes. An “internal transition” of the network consists of an internal change at one node. A “departure-arrival transition” consists of a departure transition at one node that triggers an arrival transition at another node selected by a routing probability or rate. Such a transition can also be viewed as a mutual interaction between a pair of nodes triggered by a message, activity, device, etc. This type of Markov network process covers a variety of situations including: (a) Queueing networks with phase type services or Markov modulated arrivals or services [16,22]. (b) Migration processes with interactive populations that model population genetics, clustering processes and polymerization processes (see [11, chapters 6–8] and [22]). (c) Networks with special signals such as negative customers that reduce customers at their arriving nodes (e.g., see [6]). The results herein also apply to networks with multiple types of customers. For simplicity, we present the results for homogeneous transitions and then comment on how
X. Chao et al. / Markov network processes
379
they extend to multi-class transitions, which are typically associated with multi-class customers. The general approach in our analysis is to relate the product form stationary distribution of the network to a product of stationary distributions of node transition rates that describe the nodes in isolation operating under certain fictitious environments. Our first result, theorem 2, describes key parameters in the node transition rates if the network is a product form (it has a product form stationary distribution). Using this, we obtain necessary and sufficient conditions on the node transition rates for the network to be a product form, see theorem 3. This result yields a general procedure to check whether the network is a product form and to obtain the product form distribution when there is one. Next, we consider the network, when the nodes in isolation have Poisson arrivals under the fictitious environments, which is the case for quasi-reversible nodes. Theorem 8 says that the network has a product form and is biased locally balanced if and only if the network is quasi-reversible and certain traffic equations are satisfied. We also present corollaries of the main results when the network selects the arrival node in a departure-arrival transition according to a reversible Markov transition rate or probability. A special case is the result that if the node transition rate functions are reversible, then the Markov process is also reversible and its stationary distribution is a product of the node distributions. This result first appeared in Kingman [13]; also see [11,17]. Our results do not assume the node transition rates are reversible even though the routing is reversible. Also, the network process satisfies a certain local balance condition, but it may not be reversible. The rest of this paper discusses modeling issues. We first describe a queueing network as a Markov network in which the nodes in isolation operate as reactors to given arrivals (see (iii) of section 6). This is equivalent to the node transition rates having Poisson arrival components. The characterization of product form leads to several conclusions that are useful in queueing network applications. In particular, we discuss sufficient conditions for product forms and other modeling conditions under which product form leads to quasi-reversibility. This includes the conventional approach for quasi-reversible queueing networks as well as the characterization of [14]. We next consider two dualities, one for time reversal and the other for arrival and departure transitions. Arrival and departure transitions may be reversed in our formulation resulting in a new class of network models. This paper consists of eight sections. In section 2, we define the Markov network process and the node transition rates for isolated nodes under fictitious environments. In section 3, we present the main product form characterization for the network. Section 4 discusses quasi-reversibility and biased local balance. Section 5 focuses on networks with reversibility routing. Section 6 describes how the results apply to queueing networks. Section 7 discusses networks in reverse time and the reversal of the roles of arrivals and departures. Finally, extensions to multi-class transitions are given in section 8.
380
2.
X. Chao et al. / Markov network processes
Notation
The object of our study is an open stochastic network with a finite set of nodes M , which contains the outside node 0. Each node j ∈ M is described by a state xj in a countable set Ej . The network is represented as a Markov jump process on the product space E = {x = (xj : j ∈ M ): xj ∈ Ej }. Its transition rates are X q(x, y) = qjk (x, y), x, y ∈ E, (1) j,k
where
( qjk (x, y) =
qjd (xj , yj )rjk qka (xk , yk )1(y` = x` , ` 6= j, k) qji (xj , yj )1(y` = x` , ` 6= j)
if j 6= k, if j = k.
Here 1(statement) is the indicator function that is 1 or 0 according as the “statement” is true or false. Think of qja , qjd and qji as state-dependent rate components associated with “arrival”, “departure” and “internal” transitions, respectively, at node j. We call them rate components since they are only parts of a compound transition rate. The rjk is the rate component or tendency for a departure from P node j to trigger an arrival at node k; it is often assumed to be a probability, with 1 − k6=j rjk being the probability of an attempted internal change at node j. The qji (xj , yj ) can be augmented by multiplying it by a factor rjj , but we will assume that such a coefficient is already included in qji . For simplicity, we assume the network process is irreducible and positive recurrent. The usual convention for a Markov process is to disregard bogus transitions from a state back to itself. For our analysis, however, it is convenient to include bogus jumps, and so we assume q(x, x) are well-defined rates (possibly 0). We adopt the same convention for all the other transition functions in this study. The main results give conditions under Q which the stationary distribution of this network process is a product form π(x) = j∈M πj (xj ), where πj is the jth marginal distribution. The results also apply to closed queueing networks and other networks, where E is a subset of a product space and π and πj are invariant measures instead of normalized distributions (see remark 6). Our approach is to relate the marginal distributions of the network process to stationary distributions of one-dimensional Markov processes defined as follows. For each j ∈ M , consider a Markov jump process on Ej with transition rates qj (xj , yj ) = βja qja (xj , yj ) + βjd qjd (xj , yj ) + qji (xj , yj ),
xj , yj ∈ Ej .
(2)
Think of this process as representing the state of node j as if it were operating in isolation. The three terms in the summation are transition rates associated respectively with an arrival into node j, a departure from node j and an internal change. For a fixed xj and yj , any combination of these three terms may be positive. Note that, in general, this arrival transition does not represent the arrival process at the node in the network, but specifies a fictitious arrival environment for the isolated node. The coefficients βja , βjd at this point are dummy variables. Our results determine the form
X. Chao et al. / Markov network processes
381
of these coefficients in order for the stationary distribution of qj to be the jth marginal stationary distribution of the network process. In other words, qj is identified so as to determine the marginal distribution. There is no coefficient on qji , which is consistent with it having no coefficient in (1). For simplicity, we assume the transition function qj is irreducible and positive recurrent. For our analysis, we will assume that each πj is an arbitrary positive probability measure on Ej . The role of πj will be specified in the theorem statements. Associated with each transition rate component qjs (xj , yj ), for s = a, d, i, we define αsj (xj ) =
X
qjs (xj , yj ),
yj
α ˜ sj (xj ) = πj (xj )−1 α ¯ sj =
XX xj
X
πj (yj )qjs (yj , xj ),
(3) (4)
yj
πj (xj )qjs (xj , yj ).
(5)
yj
Assume each α ¯ sj is finite. Keep in mind that α ˜ sj (xj ) and α ¯ sj are functions of πj . Also, note that X πj (xj )˜ αsj (xj ) = α ¯ sj . (6) xj
The following is an example of the type of network we consider in section 4. Example 1. Suppose the network process has the following structure. The set of nodes is given by M = {0, 1, 2, . . . , m}. Customers move among the nodes where they are processed, and the state of each node j ∈ M is a pair xj = (nj , zj ), where nj is the number of customers at the node and zj is the “environment” of the node. Whenever the network is in state x = (xj : j ∈ M ), there are two types of transitions that may occur. First, the environment at some node j may change from zj to zj0 . The time until such a transition is exponentially distributed with rate ηj (zj , zj0 ). Second, a single customer may move from some node j to some node k and the environments at nodes j, k change from zj , zk to zj0 , zk0 , respectively. The time until such a transition is exponentially distributed with rate µj (nj , zj , zj0 )rjk λk (zk0 ). Then the network process has transition rates of the form (1), where qja (xj , x0j ) = λj (zj0 )1(n0j = nj + 1), qjd (xj , x0j ) = µj (nj , zj , zj0 )1(n0j = nj − 1 > 0), qji (xj , x0j ) = ηj (zj , zj0 )1(n0j = nj ).
382
X. Chao et al. / Markov network processes
P Note that αaj (xj ) = z 0 λj (zj0 ) is independent of xj . For the coefficients βja , βjd of qj , j we will see that it is natural to define X βjd = rjk α ¯ ak , k6=j
and to put no restriction on the other coefficient βja . The latter is treated as a dummy variable. Under these assumptions, suppose each qj has a stationary distribution πj . Think of πj as a function of βja . Note that X πj (nj + 1, zj0 )µj (nj + 1, zj0 , zj ). α ˜ dj (nj , zj ) = πj (nj , zj )−1 zj0
Assume πj is such that α ˜ dj (nj , zj ) is independent of (nj , zj ). Then it follows that α ˜ dj (nj , zj ) = α ¯ dj . Denote this quantity by α ¯ dj (βja ), since it as well as πj is a function a a of βj . In addition, assume there exist βj ’s that satisfy the traffic equations X βja = α ¯ dk βka rkj , j ∈ M. k6=j
Let πj be the distributions associated with these βja ’s. In this example, αaj (xj ) = α ¯ aj and αdj (xj ) = α ¯ dj , which are the defining properties of a quasi-reversible node as we will soon see. Q Then it follows by theorem 8 below that the stationary distribution of q is π(x) = j∈M πj (xj ). 3.
Characterization of product form distributions
The results in this section give conditions on the one-dimensional processes defined by qj under which the network process has a product form stationary distribution. We begin by showing that if the network process has a product form stationary distribution, then the coefficients βja , βjd of qj must be of the form (7) below. Q Theorem 2. If π(x) = j∈M πj (xj ) is the stationary distribution of q, then each πj is the stationary distribution for qj with coefficients X X α ¯dk rkj , βjd = rjk α ¯ ak . (7) βja = k6=j
k6=j
Proof. The balance equations for q that π satisfies are X X q(x, y) = π(y)q(y, x), x ∈ E. π(x) y
y
(8)
X. Chao et al. / Markov network processes
Since
383
π(y) = π(x)πj (yj )πk (yk )/ πj (xj )πk (xk ) ,
for y such that x` = y` for all ` 6= j, k, it follows by the definition of q that (8) is X X i d a αj (xj ) + αj (xj ) π(x) rjk αk (xk ) j
k6=j
X X i a d ˜ j (xj ) rkj α ˜ k (xk ) , α ˜ j (xj ) + α = π(x) j
x ∈ E.
(9)
k6=j
For a fixed j ∈ M , we will consider the sum of these equations over all x` ∈ E` , for ` ∈ M \{j}. First, note that X [left side of (9)] x` :`6=j
=
X
x` :`6=j
X X π(x) αij (xj ) + αdj 0 (xj 0 )rj 0 j αaj (xj ) + αdj (xj ) rjk αak (xk ) j 0 6=j
X X i d a αj 0 (xj 0 ) + αj 0 (xj 0 ) + rj 0 k αk (xk ) j 0 6=j
k6=j
k6=j,j 0
= πj (xj ) αij (xj ) + βja αaj (xj ) + βjd αdj (xj ) + Aj , where
X X i d a Aj = α ¯j0 + α ¯j0 rj 0 k α ¯k . j 0 6=j
k6=j,j 0
A similar computation using (6) yields X i [right side of (9)] = πj (xj ) α ˜ j (xj ) + βja α ˜ aj (xj ) + βjd α ˜ dj (xj ) + Aj . x` :`6=j
Since (9) is an equality, the preceding sums are equal, and equating them yields ˜ ij (xj ) + βja α ˜ aj (xj ) + βjd α ˜ dj (xj ), αij (xj ) + βja αaj (xj ) + βjd αdj (xj ) = α
xj ∈ E.
(10)
These are the balance equations divided by πj (xj ) for qj . Hence πj is the stationary distribution for qj . The following result characterizes a product form distribution for the network process. Here we use the function Djk (xj , xk ) = α ¯ dj − αdj (xj ) rjk α ¯ dk − α ˜dk (xk ) rkj α ˜ aj (xj ) . ¯ ak − αak (xk ) − α ¯ aj − α
384
X. Chao et al. / Markov network processes
Q Theorem 3. The stationary distribution of q is π(x) = j∈M πj (xj ), x ∈ E, if and only if each πj is the stationary distribution of qj for some coefficients βja , βjd , and the πj ’s are such that (7) holds and Djk (xj , xk ) + Dkj (xk , xj ) = 0,
j 6= k ∈ M , xj ∈ Ej , xk ∈ Ek .
(11)
The proof of this result uses the following lemma. Lemma 4. Suppose each Q πj is the stationary distribution of qj for some coefficients βja , βjd . Then π(x) = j∈M πj (xj ) is a stationary distribution for q if and only if X ˜ aj (xj ) − βjd α ˜ dj (xj ) βja αaj (xj ) + βjd αdj (xj ) − βja α j
X X X d a a d αj (xj ) = rjk αk (xk ) − α ˜ j (xj ) rkj α ˜ k (xk ) , j
k6=j
If βja , βjd are of the form (7), then (12) is equivalent to XX Djk (xj , xk ) = 0, x ∈ E. j
x ∈ E.
(12)
k6=j
(13)
k6=j
Proof. Recall that the balance equations for q and π of product form are (9). Then to prove the first assertion, it suffices to show that (9) is equivalent to (12). To this end, recall that the balance equations for qj are (10) multiplied by πj (xj ). Summing (10) on j and subtracting (9) divided by π(x) from the sum shows that (9) is equivalent to (12). This proves the first assertion. The second assertion follows since substituting (7) into (12) and rearranging terms yields (13). Proof of theorem 3. First, assume the stationary distribution of q is π(x) = Q j∈M πj (xj ). Then by theorem 2, the πj is the stationary distribution for qj with coefficients that satisfy (7). To prove (11), note that (13) holds by lemma 4. Also, for any k 6= `, (7) and (6) imply X π` (x` ) D`k (x` , xk ) + Dk` (xk , x` ) = 0. (14) x`
Q Then multiplying (13) by `6=j,k π` (x` ), and summing it on x` for ` 6= j, k and using (14) yields (11). For the converse, assume each πj is the stationary distribution of qj and that (7) and (11) are satisfied. Since (11) implies (13), it follows by lemma 4 that π(x) = Q π (x j j ) is the stationary distribution of q. j∈M Theorem 3 yields the following procedure for establishing the existence of a product form stationary distribution for the network process and obtaining the distribution when it exists. An illustration of this procedure is in section 6.
X. Chao et al. / Markov network processes
385
Procedure for obtaining a product form distribution Step 1. For each node j, obtain the stationary distribution πj of qj as a function of the coefficients βj = (βja , βjd ) viewed as a dummy vector. Since πj is a function of βj , so is α ¯ sj , and we write it as α ¯sj (βj ), for s = a, d. Step 2. Find βj ’s that satisfy the traffic equations X X βja = α ¯ dk (βk )rkj , βjd = rjk α ¯ak (βk ), j ∈ M. (15) k6=j
k6=j
Step 3. Let πj be the distribution associated with the βj ’s obtained in step 2. Verify (11) for these distributions and coefficients. Q If these steps are successful, then π(x) = j πj (xj ) is the stationary distribution of the network process. Equations (15) are often called traffic equations, since for queueing networks, βja and βjd are the average number of arrivals and departures, respectively, for node j. Finding βj ’s that satisfy (15) is a fixed point problem whose solution is usually established by Brouwer’s fixed point theorem. For a particular application, one may be able to construct an algorithm to compute a fixed point. There may be more than one solution, but any solution will work. Such a fixed point exists if the network has a product form stationary distribution. This is due to the following observation. Restatement of theorem 3. The network process has a product form stationary distribution if and only if there exist βj ’s that satisfy steps 1–3 above. The next result is a variation of theorem 3. It follows immediately from theorem 2 and lemma 4. Q Theorem 5. The stationary distribution of q is π(x) = j∈M πj (xj ), x ∈ E, if and only if each πj is the stationary distribution of qj for some coefficients βja , βjd , and the πj ’s are such that (12) holds. Remark 6 (Results for closed networks). We have assumed the state space E is a product space and π and πj ’s are probability distributions. However, from their proofs, it is clear that the sufficient conditions in theorems 3 and 5 for a product form distribution apply even when E is a subset of the product space of the Ej ’s and the π and πj ’s are invariant measures instead of normalized distributions. In particular, the results with these modifications apply to closed networks. On the other hand, the necessary conditions in these theorems are generally Q not valid in these situations because, in the proof of theorem 2, the summation of `6=j π` (x` ) over x ∈ E for each fixed xj ∈ Ej may depend on xj .
386
4.
X. Chao et al. / Markov network processes
Quasi-reversibility and biased local balance
In this section, we characterize product form distributions for the network process in terms of the following quasi-reversibility condition for the nodes. Definition 7. The transition rate qj is quasi-reversible with respect to πj if πj is the ˜ dj (xj ) are independent of xj ; that is stationary distribution of qj and αaj (xj ) and α ¯ aj , αaj (xj ) = α
α ˜ dj (xj ) = α ¯ dj ,
and
for each xj ∈ Ej .
(16)
To see the meaning of this definition, consider a qj in which only one of the rate components qja (xj , yj ), qjd (xj , yj ) and qji (xj , yj ) is positive for each xj , yj . Then αaj (xj ) = α ¯ aj implies that the times of a-transitions for qj form a Poisson process with ˜ dj (xj ) = α ¯ dj implies that the times of d-transitions in equilibrium for rate α ¯ aj . Also, α ¯ dj (e.g., see [18]). qj form a Poisson process with rate α Note that in theorem 3, the key condition (11) for the network process to have a product form stationary distribution is satisfied if each qj is quasi-reversible. In addition to the usual balance equations for a process, we will use the following notion, introduced in [5]. The Markov transition rate q is biased locally balanced with respect to P a positive probability measure π on E and real numbers γ = {γj : j ∈ M } satisfying j γj = 0 if XX XX π(x) qjk (x, y) + γj = π(y)qkj (y, x), x ∈ E, j ∈ M. (17) k
y
k
y
The π is necessarily the stationary distribution for q since the global balance equations are the sum of these local balance equations over j. Also, we say q is locally balanced with respect to π when the γj ’s are 0. For the next result, we consider the network process under the added assumption that each αaj (xj ) is independent of xj , or, equivalently, ¯ aj , αaj (xj ) = α
for each xj and j ∈ M .
(18)
This is the first part of the quasi-reversibility condition. Because of theorem 2, we make the natural assumption that the coefficient βjd of qj is given by X rjk α ¯ ak , j ∈ M. (19) βjd = k6=j
No restriction is placed on the other coefficient βja . Theorem 8. Under the assumptions (18) and (19), the following statements are equivalent: Q (i) The q is biased locally balanced with respect to π(x) = j∈M πj (xj ) and γ.
X. Chao et al. / Markov network processes
387
(ii) Each qj is quasi-reversible with respect to πj for some βja , and the πj ’s are such that X α ¯ dk rkj , j ∈ M. (20) βja = k6=j
If these statements hold, then ¯ aj βja − α ¯ dj βjd . γj = α
(21)
Proof. Suppose (i) holds. By the definitions of q, π and assumptions (18), (19), it follows, similarly to (9), that the biased local balance equation (17) divided by π(x) is X ˜ ij (xj ) + α ˜ aj (xj ) α ˜ dk (xk )rkj . (22) αij (xj ) + βjd αdj (xj ) + γj = α k6=j
Define βja by (20). Fix j ∈ M . Multiplying (22) by πj (xj ), then summing over xj and using (6), we have X ¯ aj α ˜ dk (xk )rkj . α ¯ dj βjd + γj = α k6=j
Fix ` 6= j. Multiplying this equation by k 6= j, ` and using (6) and (20), we obtain
Q
k6=j,` πk (xk ),
then summing over xk , for
¯ aj βja + α ˜ d` (x` ) − α ¯ d` r`j 1(` 6= j). α ¯ dj βjd + γj = α
(23)
Summing this on j and using (19) and (20) yields X α ˜ d` (x` ) − α ¯ d` r`j = 0. j6=`
This proves α ˜ d` (x` ) = α ¯ d` . Thus, (16) holds. Next, note that (23) implies (21). From (16), applying (21) to (22) yields (10), which is the balance equation divided by πj (xj ) for qj and πj . Hence πj is the stationary distribution for qj . This proves that (i) implies (ii). ¯ aj and α ˜ dj (xj ) = α ¯ dj Now, assume (ii) holds. Then (10) holds, and using αaj (xj ) = α in (10), we have αij (xj ) + βja α ¯ aj + βjd αdj (xj ) = α ˜ ij (xj ) + βja α ˜ aj (xj ) + βjd α ¯ dj ,
xj ∈ E.
(24)
¯ dj = α ˜ dj (xj ) to this equation Define γj by (21). Applying (21) and then (20) and α yields (22), and substituting (19) and (18) into (22) gives (17). This completes the proof that (ii) implies (i). What is the difference between theorems 3 and 8? In the former, both of the coefficients βja , βjd of qj are unspecified dummy variables, while in the latter, βjd is given by (19) and only βja is a dummy variable. Consequently, in theorem 3 the
388
X. Chao et al. / Markov network processes
conditions (7), (11) required for a product form distribution for the network are more involved than the conditions (16), (20) required in theorem 8. The following is a procedure for applying theorem 8; compare this with the procedure in the preceding section. Quasi-reversible procedure for a product form distribution Step 1. For each node j, obtain the stationary distribution πj of qj as a function of the coefficient βja viewed as a dummy variable. Since πj is a function of βja , so is ¯ dj (βja ). α ¯ dj , and we write it as α Step 2. Find βja ’s that satisfy the traffic equations X βja = α ¯ dk βka rkj , j ∈ M. k6=j
Step 3. Let πj be the distribution associated with the βj ’s obtained in step 2. For these distributions, verify the quasi-reversibility condition α ˜ dj (xj ) = α ¯ dj βja , for each xj ∈ Ej and j ∈ M . (25) If these steps are successful, then π(x) = of the network process.
Q j
πj (xj ) is the stationary distribution
Remark 9. There may be solutions to the traffic equations in step 2 even though (25) is not satisfied. In this case, one might be able to obtain a product form stationary distribution by verifying condition (11). 5.
Networks with reversible routing
The following are corollaries of theorem 3.2 when the routing rates of the network are reversible. Here, we assume, for simplicity, that rjk is irreducible and rjj = 0. Let wj , j ∈ M , denote its stationary distribution. rjk is reversible if wj rjk = wk rkj ,
j, k ∈ M.
Corollary 10. If rjk is reversible, then theorem 3 holds with Djk (xj , xk ) replaced by a a ∗ (xj , xk ) = wk α ¯ k − αak (xk ) − wj α ˜ dk (xk ) α ¯j − α ˜ aj (xj ) . Djk ¯ dj − αdj (xj ) α ¯ dk − α Proof. The assertion follows from theorem 3 by substituting rjk = wk rkj /wj in Djk (xj , xk ). Corollary 11. Suppose rjk is reversible. Assume each πj is the stationary distribution of qj for some coefficients βja , βjd , and the πj ’s are such that (7) is satisfied and α ˜aj (xj ) = wj−1 αdj (xj ),
α ˜ dj (xj ) = wj αaj (xj ),
xj ∈ Ej ,
(26)
X. Chao et al. / Markov network processes
Then π(x) =
Q j∈M
389
πj (xj ) is the stationary distribution of q. In addition, ¯ dj /¯ αaj = βja /βjd , wj = α
j ∈ M.
(27)
¯ aj . This and (26) imply Proof. First note that the last equality in (26) implies α ¯ dj = wj α ∗ (x , x ) = 0. Then corollary 10 yields the first assertion. Furthermore, that each Djk j k d a ¯ j and the reversibility of rjk , we have from α ¯ j = wj α X X X βja = α ¯ dk rkj = α ¯ ak wk rkj = wj rjk α ¯ ak = wj βjd . k6=j
k6=j
k6=j
Thus (27) holds.
Corollary 11 generalizes results in Kingman [13], and Pollett [17]. In these studies, the network process is reversible, but the network process in corollary 11 need not be. Condition (26) is a local balance condition which is implied by detailed balance (reversibility). Recall that theorem 8 is for a network with quasi-reversible nodes, while corollary 11 is for a network whose nodes need not be quasi-reversible, but the routing is restricted to be reversible. The next result is for networks with both types of nodes. Corollary 12. Suppose there is a subset J ⊂ M such that the assumptions of corollary 11 hold for the nodes in J and the assumptions (18), (19) and theorem 8(ii) hold for the nodes in K = {k: k ∈ / J, or k ∈ J and rk` + r`k 6= 0, for some ` ∈ / J}. Q Then π(x) = j∈M πj (xj ) is the stationary distribution of q. In particular, if node 0 has a single state, i.e. it is a Poisson source, J can be reduced to J \ {0}. Proof. The set K contains all the nodes in M \ J and those nodes in J that are directly connected to set M \ J. Hence, the nodes that only satisfy the conditions of corollary 11 are not directly connected to the nodes that only satisfy conditions of theorem 8. This implies that each Djk (xj , xk ) is 0 for all j, k, so the main assertion follows by theorem 3. If node 0 has single state 0, Dk0 (xk , 0) = D0k (0, xk ) = 0
for k ∈ M \ {0}.
This proves the last part.
The following example illustrates the use of corollary 11. Example 13. Suppose the network process represents customers moving in a network in which the state xj , denoted here by nj , represents the number of customers at node
390
X. Chao et al. / Markov network processes
j ∈ {1, 2, . . . , m}. The network has an outside source, denoted by node 0, which has a single state 0. Assume the transition rates for the network are given by (1), where q0a (0, 0) = 1, q0d (0, 0) = µ0 , n0 = n − ` > 1, ` > 0, j (`), µj (n)bP 0 qjd (n, n0 ) = µj (n) ∞ `=n bj (`), n = 0, n > 1, 0, otherwise, a 0 0 qj (nj , n ) = λj (n)1(n = n + 1), for j = 1, 2, . . . , m. In what follows, we consider node j 6= 0. Assume the network does not have internal transitions. According to these rates, node j in isolation operates as a batch service system. Whenever it contains n customers, arrivals enter at the rate λj (n); also, batches exit at the rate µj and the size of a batch is min{n, `}, where ` is selected by the batch-size probability distribution bj (`). Assume bj (0) > 0 and that the mean of bj exceeds 1. Note that in the network process, a batch departure at a node triggers only a single customer arrival at some node. This is because each arrival transition rate qja only allows single-unit increments. Another feature is that a node may have bogus departures when it is not empty, Namely, whenever nodes j and k contain nj > 1 and nk customers, respectively, there is a null departure at node j and an arrival at node k at the rate µj (nj )bj (0)rjk λk (nk ). We will derive the stationary distribution of the network by appealing to corollary 11. Assume the routing probabilities rjk are reversible with stationary distribution wj . Furthermore, for the node j transition rate qj given by (2), we select its beta coefficients such that βja /βjd = wj , which is consistent with (27). We conjecture that qj has a stationary distribution πj of the form πj (n) = cj ρnj /λj (n),
n > 0,
(28)
for some ρj , where cj is the normalizing constant, while π0 (0) = 1. Before verifying this conjecture, let us see what else is needed to satisfy the assumptions of corollary 11. First consider condition (26). Clearly, α ˜ aj (n) = 0 = wj−1 αdj (0), and, for n > 1, we have αdj (n) = µj (n) and α ˜ aj (n) =
1 πj (n − 1)λj (n − 1) = λj (n)/ρj . πj (n)
Consequently, the first part of (26) holds, namely α ˜ aj (n) = wj−1 αdj (n), if and only if λj (n)/µj (n) = ρj wj−1 ,
n > 1.
(29)
Hereafter, we assume this is true. Under this assumption, a similar calculation as above shows that the second part of condition (26) is satisfied. Furthermore, another
X. Chao et al. / Markov network processes
391
easy check shows that the πj ’s satisfy (7). Thus, the assumptions of corollary 11 are satisfied. It remains to show that the stationary distribution πj is given by (28). The balance equations for qj are πj (0)βja λj (0) = βjd
∞ X
πj (` )µj (` )
`=1
=
bj (m),
m=`
πj (n) βja λj (n) + βjd µj (n) βja λj (n
∞ X
− 1)πj (n − 1) +
βjd
∞ X
πj (n + ` )µj (n + ` )bj (` ),
n > 1.
`=0
Substituting (28) into the first balance equation and using a little algebra and (29), we obtain ∞ X ρ`j bj (` ) = ρj . (30) `=0
The same equation is obtained by substituting (28) into the balance equation for n > 1 and dividing both sides by ρnj . Equation (30) has a unique solution ρj ∈ (0, 1) because the left hand side is a strictly increasing convex function in ρj that begins at bj (0) > 0 and ends at 1 with a tangent equal to the mean batch size, which we assumed exceeds 1. Then πj given by (28) will be a valid distribution provided c−1 j
=
∞ X
ρnj /λj (n) < ∞,
n=0
which we assume is true. Thus by corollary 11, the stationary distribution of the network is the product of πj ’s given by (28). It is worth noting that each qj is neither reversible nor quasireversible. 6.
Queueing networks
One class of stochastic networks that deserves special attention is queueing networks. A queueing network is a system in which items or information/commands move from one node to another and trigger the states of the nodes to change. A state change, called a departure event, is initiated at one node, and this event is then “routed” as an arrival event to another node that triggers a state change at the arriving node. In contrast, in the stochastic networks discussed earlier, a state change might be due to mutual interactions at nodes. We now consider such queueing networks, introduced in [4] and further studied in [5]. Suppose the network process has transition rates q given by (1) with the
392
X. Chao et al. / Markov network processes
following additional assumptions, and call it a queueing network process. The rate component qja (xj , x0j ) will be a probability and we denote it by paj (xj , x0j ), where P a 0 x0 pj (xj , xj ) = 1. Thus, j
αaj (xj ) = α ¯ aj = 1, xj ∈ Ej , j ∈ M. (31) P Next, we normalize rjk such that k∈M rjk = 1. Since feedback loops are natural in queueing networks, we allow rjj > 0. This class of networks is a subclass of that considered in section 4. Our aim here is to describe the modeling of queues. With these assumptions, the network process is characterized by the following system dynamics: (i) When the state of node j is xj , the departure transition rate that changes the state from xj to yj is qjd (xj , yj ). (ii) A departure from node j is transferred to node k as an arrival with probability rjk (recall that node 0 represents the outside). (iii) An arrival at node k changes its state from xk to yk with probability pak (xk , yk ). (iv) The internal transition rate at node j (the transition that does not trigger a state change at another node) is qji (xj , yj ) when its state is xj . This transition includes the case of (ii) for j = k, i.e., X qji (xj , yj ) = qjn (xj , yj ) + qjd (xj , x0j )rjj paj (x0j , yj ), x0j
where qjn is a pure internal transition at node j. These dynamics are typical for conventional queueing networks. However, one should note that this framework is more general than the conventional one, because the state changes in (i) and (iii) are arbitrary and arrivals, departures and internal changes may occur at the same time. For instance, if xj represents the number of customers at node j and paj (xj , xj − 1) = 1(xj > 0), then an arrival at node j reduces the number of customer by 1. Similarly, by appropriately defining qjd , one may have a situation where a departure at a node increases the number of customers. For the queueing network we have defined, each node j has βjd = 1 − rjj and its transition rates in isolation are qj (xj , yj ) = βja paj (xj , yj ) + (1 − rjj )qjd (xj , yj ) + qji (xj , yj ),
xj , yj ∈ Ej .
(32)
The qjd (x, yj ) is the departure transition rate, qji (xj , yj ) is the internal transition rate, and βja is the average arrival rate. If πj (xj ) is the stationary distribution of (32) with ¯ dj is the average departure rate from node j, which is a dummy parameter βja , then α a function of βj . For the queueing network process, theorem 3 is as follows.
X. Chao et al. / Markov network processes
393
Theorem 14. The stationary distribution of the queueing network process is π(x) = Q j∈M πj (xj ) if and only if each πj is the stationary distribution of qj in (32) for some coefficient βja , and the πj ’s are such that X βja = α ¯ dk rkj , j ∈ M , (33) k6=j
and
α ˜ dj (xj ) − α ¯ dj rjk α ˜ ak (xk ) − 1 + α ˜ dk (xk ) − α ¯ dk rkj α ˜aj (xj ) − 1 = 0,
(34)
for j 6= k ∈ M , xj ∈ Ej , xk ∈ Ek . Here only the one equation (33) from (15) is needed since βjd = 1−rjj . Equation (33) states that βja is the total arrival rate at node j from all other nodes. This is why (33) and (15) are called traffic equations. Keep in mind that (33) are nonlinear ¯ dj is a function of βja . Note that either one of the following equations in the βja ’s since α conditions is sufficient for (34). (a) Both nodes j and k are quasi-reversible. (b) Both nodes j and k are noneffective for arrivals. Node j is said to be noneffective for arrivals if α ˜ aj (xj ) = 1 for all xj ∈ Ej . (c) Either node j or node k is quasi-reversible and noneffective for arrivals. These sufficient conditions are further weakened if rjk = 0 or rkj = 0. Usually, the outside source is noneffective for arrivals. If, in addition, it is quasi-reversible, the outside is a Poisson source, which is the case (c). So, we do not need to check (34) for nodes connected only to the Poisson source. Remark 15. If node j is quasi-reversible, the marginal distribution πj can be determined by another transition rate function qj0 given by qj0 (xj , yj ) = βja+ paj (xj , yj ) + qjd (xj , yj ) + qjn (xj , yj ), where βja+ =
X
α ¯ dk rkj ,
xj , yj ∈ Ej ,
i.e., βja+ = βja + rjj α ¯ dj .
k∈M
In fact, balance equations for qj and qj0 are identical under quasi-reversibility. Here, βja+ is the total arrival rate including the feedback rate, and qj0 does not include feedback transition as internal transition. This form of the transition rate is standard in the quasi-reversibility literature, but it is not convenient to use for non quasi-reversible case. Theorem 8 concerning quasi-reversible nodes also applies to the queueing network process. The only simplification is that equation (21) reduces to γj = βja −
394
X. Chao et al. / Markov network processes
(1 − rjj )¯ αdj . Although quasi-reversibility is part of a sufficient condition for a product form distribution of the network, this condition is not necessary. Example 18 below presents a network with a product form distribution, but all the nodes are not quasireversible. More examples are in [19]. In certain situations, however, it turns out that quasi-reversibility is not far from being necessary. Corollary Q 16. Suppose the queueing network process has the stationary distribution π(x) = j∈M πj (xj ). Assume node j satisfies rjk∗ 6= 0 and
rk∗ j = 0,
for some k∗ 6= j,
(35)
and α ˜ ak∗ (xk∗ ) 6= 1
for some xk∗ ∈ Ek∗ .
(36)
Then node j is quasi-reversible with respect to πj . Proof. Theorem 14 ensures that πj is the stationary distribution for qj and that (34) holds. Under the hypotheses, (34) reduces to ¯ dj rjk α α ˜ dj (xj ) − α ˜ ak (xk ) − 1 = 0. Since this is true for k = k∗ that satisfy the hypotheses and each xj , it follows that ¯ dj , for each xj . Thus, node j is quasi-reversible with respect to πj . α ˜ dj (xj ) = α Remark 17. Under the first assumption in corollary 16, condition (36) is satisfied if, for some k 6= j, the Markov transition probabilities pak (xk , yk ) on Ek are transient. To see this, first note that α ˜ ak (xk ) = 1, for each xk , if and only if πk is the positive stationary measure for the transition probabilities pak (xk , yk ). Thus, if these probabilities are transient, then (36) is satisfied. In a conventional queueing network, a customer entering a node always “increases” the number of customers at the node. In such a network, pak is clearly transient. Hence if the network has a product form distribution and (35) holds, then node j is quasi-reversible. This is related to a result of Harrison and Williams [9] for a conventional network with feed-forward routing that says the stationary distribution of the corresponding diffusion process is a product form if and only if each node is quasi-reversible, which is appropriately defined for the diffusion process. If the network has more structure, we can get stronger results from theorem 14. For instance, suppose node j has an empty state, denoted by 0, such that there are no departures or internal transitions from it and neither can it be reached by arrivals or internal transitions. That is, α ˜ aj (0) = αdj (0) = 0,
qji (xj , x0j ) = 0 if either xj = 0 or x0j = 0.
This is typical for nodes in a conventional network with customers. Let the outside node 0 be Poisson, so the assumption above does not hold for node 0. For this network,
X. Chao et al. / Markov network processes
395
substituting xj = 0 into (10), we have βja = (1 − rjj )˜ αdj (0). So, (34) with xj = 0 leads to rjk ¯ dj ˜ dk (xk ) − α ¯ dk rkj = 0, j, k = 6 0, α ˜ ak (xk ) − 1 − α βja+ − α 1 − rjj ¯ dj . Thus, if the network has the product form stationary where βja+ = βja + rjj α ¯ dj , node k with rkj > 0 has distribution and node j is internally balanced, i.e., βja+ = α to be quasi-reversible. Since node 0 is a Poisson source, it is both quasi-reversible and noneffective for arrivals. Thus if all other nodes are internally balanced, the product form holds if and only if all nodes such that 1 − rjj − rj0 > 0, called nonterminal nodes, are quasi-reversible. This is the result of [14]. The following is a counterexample to that quasi-reversibility is necessary for a product form queueing network. Example 18. Consider the queueing network process with node set M = {0, 1, 2}, and E1 = E2 = {0, 1} and E0 = {0}. Assume the nonzero routing probabilities are rjk = 1/2, j 6= k in {= 0, 1, 2}, and that qj is defined with pa0 (0, 0) = 1, paj (0, 0) = 0,
q0d (0, 0) = 2, paj (0, 1) = 1,
paj (1, 0) = 1,
paj (1, 1) = 0, j = 1, 2,
q1d (0, 0) = 3/2,
q1d (0, 1) = 3/2,
q1d (1, 0) = 0,
q1d (1, 1) = 0,
q2d (0, 0) = 1/6,
q2d (0, 1) = 1/2,
q2d (1, 0) = 3/2,
q2d (1, 1) = 0,
qji (xi , x0j ) = 0,
j = 1, 2, and xj , x0j = 0, 1.
We will apply the procedure in section 3 to compute the stationary distribution of the network. Since we are considering a queueing network, each βjd equals 1 and the βja ’s are the only dummy variables. Also, only the first traffic equation in (15) is ¯ aj equals 1. relevant, since each βjd and α Step 1. For dummy variables (β1a , β2a ), the node transition rates are q1 (0, 0) = 3/2, q2 (0, 0) = 1/6,
q1 (0, 1) = β1a + 3/2, q2 (0, 1) = β2a + 1/2,
q1 (1, 0) = β1a , q2 (1, 0) = β2a + 3/2,
q1 (1, 1) = 0, q2 (1, 1) = 0.
The stationary distributions for these rates are easily calculated and are β1a , 2β1a + 3/2 β a + 3/2 π2 (0) = 2 a , 2β2 + 2
π1 (0) =
β1a + 3/2 , 2β1a + 3/2 β a + 1/2 π2 (1) = 1 a . 2β2 + 2
π1 (1) =
¯ dj ’s are And based on πj , the α α ¯ d0 = 2,
α ¯ d1 =
6β1a , 4β1a + 3
α ¯ d2 =
13/6β2a + 7/4 . 2β2a + 2
396
X. Chao et al. / Markov network processes
Step 2. The traffic equations for this network are 1 d 1 d ¯ + α ¯ , β0a = α 2 1 2 2
1 d ¯ , β1a = 1 + α 2 2
1 d ¯ . β2a = 1 + α 2 1
The solution of these equations is β0a = 1, β1a = β2a = 3/2, α ¯ d1 = α ¯ d2 = 1. Thus π1 (0) = 1/3,
π1 (2) = 2/3,
π2 (0) = 3/5,
π2 (1) = 2/5.
Step 3. To check condition (34), we first note that α ˜ d1 (0) = 3/2, α ˜ a1 (0) = 2,
α ˜ d1 (1) = 3/4, α ˜ a1 (1) = 1/2,
α ˜ d2 (0) = 7/6, α ˜ a2 (0) = 2/3,
α ˜ d2 (1) = 3/4, α ˜ a2 (1) = 3/2.
Then a routine check verifies (34). For instance, when j = 1, k = 2 and xj = xk = 0, ¯ d1 r12 α ˜ d2 (0) − α ¯ d2 r21 α α ˜ d1 (0) − α ˜ a2 (0) − 1 + α ˜ a1 (0) − 1 3 1 2 7 1 = −1 −1 + − 1 (2 − 1) = 0. 2 2 3 6 2 Other cases of (34) are similarly checked to be zero. Hence by corollary 16, the stationary distribution of the network is the product form π(x1 , x2 ) = π1 (x1 )π2 (x2 ) for x1 , x2 = 0, 1. Note that neither node 1 or 2 is quasi-reversible since ¯ d1 = 1, α ˜ d1 (0) = 3/2 6= α
α ˜ d2 (0) = 7/6 6= α ¯ d2 = 1.
We conclude this section by showing how quasi-reversibility can be used to obtain a product form distribution for a nonconventional network. Example 19. Consider a queueing network with exogenous Poisson arrivals, Markovian routing probabilities rjk and constant departure rates µj at the nodes. Assume the network operates like a Jackson network with the following exception. Whenever a customer is assigned by the probabilities rjk to enter node j, it either enters with probability aj (thereby adding one unit to node j), or it does not enter but it deletes one customer there with probability bj = 1 − aj , provided a customer is there. Then the transition rates for the network are given by (1), where qjd (xj , yj ) = µj 1(yj = xj − 1 > 0), paj (xj , yj ) = aj 1(yj = xj + 1) + bj 1 yj = max(0, xj − 1) .
q0d (0, 0) = λ,
pa0 (0, 0) = 1,
Clearly, qj defined by (32) is an M/M/1 queue with arrival rate βja aj and service rate µj + βja bj . So its stationary distribution πj is x
πj (xj ) = (1 − ρj )ρj j ,
j 6= 0,
(37)
X. Chao et al. / Markov network processes
397
provided ρj ≡ βja aj /(µj + βja bj ) < 1, which we assume. Each node j is quasireversible since α ˜ dj (xj ) = πj (xj )−1 πj (xj + 1)qjd (xj + 1, xj ) = ρj µj . Now, suppose βja is a solution of the traffic equation (33), which in this case is βja = λr0,j +
X k6=j,0
µk βka ak rkj . µk + βka bk
Then by corollary 14 we conclude that the stationary distribution of the network is the product of the πj ’s. 7.
Time-reversals and departure-arrival reversals
In this section, we show that a product form network process in “reverse time” has the same type of transition rate function as the original process. We also point out that by reversing the roles of arrival and departure transitions in the network one obtains a dual network process whose structure is typically different from the original process. We first consider the network process in reverse time, see, for instance, [11]. Suppose the network transition rate q is ergodic and its stationary distribution is π. The time-reversal of q is the transition rate qˆ(x, y) = π(x)−1 π(y)q(y, x),
x, y ∈ E.
This qˆ has the same stationary distribution as q. Now, assume π is the product of stationary distributions πj of the node transition rates qj given by (2). The time reversal of qj is qˆj (xj , yj ) = πj (xj )−1 πj (y)qj (yj , xj ) = βja qˆja (xj , yj ) + βjd qˆjd (xj , yj ) + qˆji (xj , yj ),
xj , yj ∈ Ej ,
where qˆjs (xj , yj ) = πj (xj )−1 πj (yj )qjs (yj , xj ),
s = a, d, i.
Now, an easy check shows that qˆ has the same form (1) as q with rjk and qjs replaced respectively by rjk and qˆjs (s = a, d, i). This is consistent with π being the product of the πj ’s, which are also the stationary distributions of the qˆj ’s. Next, we discuss the idea of reversing the roles of arrivals and departures in the network. The key part of the transition rate q in (1) is the product qjd (xj , yj )rjk qka (xk , yk ). From this and the other network assumptions, it is clear that all the results above apply to the process with the roles of a and d reversed. One interpretation of this reversal is
398
X. Chao et al. / Markov network processes
that the process is the same, but in the results, a and d are simply interchanged. For instance, in the new theorem 8 the assumption (18) would apply to αdj and (19)–(21) would apply with a and d interchanged. Another interpretation is that the new theorems would apply to any network with routing and transition components rˆjk and qˆjs (xj , yj ), for s = a, d, i, that satisfy qˆjd (xj , yj )ˆ rjk qˆka (xk , yk ) = qkd (xk , yk )rkj qja (xj , yj ).
(38)
Such a network, which has different system dynamics than the original one, could be viewed as a dual of the original network. Examples are in [2].
8.
Networks with multi-class transitions
This section extends the results of the previous sections to the case of multi-class transitions. These extensions are rather straightforward, and so they are presented without proofs. Let Tj be the class of arrival and departure transitions of node j. One may use different classes for arrivals and departures, but a single class can cover such a case s (x , x0 ) be the by introducing null transitions if necessary. For each u ∈ Tj , let qju j j transition rate on Ej of class u for s = a, d. We assume that internal transitions are independent of the class and use the same notation qji as in section 2. The routing component rjk is now extended to rju,kv . The transition rates for the Markov network process are defined as X qjk (x, y), x, y ∈ E, (39) q(x, y) = j,k
where qjj (x, y) = qji (xj , yj )1(y` = x` , ` 6= j) X X d a qju (xj , yj )rju,kv qkv (xk , yk )1(y` = x` , ` 6= j, k), qjk (x, y) = u∈Tj v∈Tk
As in section 2, for a distribution πj on Ej , we define, for s = a, d, αsju (xj ) =
X
s qju (xj , yj ),
yj
α ˜ sju (xj ) = πj (xj )−1 α ¯ sju =
XX xj
yj
X
s πj (yj )qju (yj , xj ),
yj s πj (xj )qju (xj , yj ).
for j 6= k.
X. Chao et al. / Markov network processes
399
The corresponding notation for s = i is defined as in section 2. It is assumed that α ¯ sju < ∞ for s = a, d, i. The transition function qj of the local process at node j is now changed to X a a d d qju (xj , yj ) + βju qju (xj , yj ) + qji (xj , yj ), xj , yj ∈ Ej , qj (xj , yj ) = βju u∈Tj s (s = a, d) are determined by where coefficients βju XX a = α ¯ dkv rkv,ju , βju
(40)
k6=j v∈Tk
d βju =
XX
rju,kv α ¯ akv ,
(41)
k6=j v∈Tk
which are called traffic equations. Finally, we redefine Djk as Dju,kv (xj , xk ) = α ¯ akv − αakv (xk ) ¯ dju − αdju (xj ) rju,kv α ¯ aju − α ˜ dkv (xk ) rkv,ju α ˜ aju (xj ) . − α ¯ dkv − α Theorem 3 for the multi-class network is as follows. Theorem 20. The following statements are equivalent: Q (i) The stationary distribution of q is π(x) = j∈M πj (xj ). (ii) Each πj is the stationary distribution for qj with coefficients (40) and (41), and X X Dju,kv (xj , xk ) + Dkv,ju (xk , xj ) = 0, u∈Tj v∈Tk
j 6= k ∈ M , xj ∈ Ej , xk ∈ Ek .
(42)
For the queueing network framework of section 6, we replace (31) and the probability condition, respectively by αaju (xj ) = 1, j ∈ M , u ∈ Tj , xj ∈ Ej , XX rju,kv = 1,
(43) (44)
k v∈Tk
which then implies d βju =1−
X
rju,jv .
(45)
v∈Tj
The biased local balance condition for this multi-class network has the same form (17) with qjk now defined as in (39). Chao and Miyazawa [5] showed that this condition is satisfied if quasi-reversibility holds for all pairs ju. Here, qj is said to be quasi-reversible with respect to πj if πj is the stationary distribution of qj and αaju (xj )
400
X. Chao et al. / Markov network processes
and α ˜ dju (xj ) are independent of xj ∈ Ej for each j ∈ M , u ∈ Tj . However, the biased local balance plus product form stationary distribution do not imply quasi-reversibility for the multi-class case. For example, it is easy to see that, if X α ˜ dju (xj ) − α ¯ dju rju,kv = 0, (46) u∈Tj
then (42) is satisfied, thus the network is a product form. On the other hand, one can show using the same arguments as in the proof of theorem 4.2 that (46) implies biased local balance. However, (46) is clearly weaker than quasi-reversibility. Thus, quasi-reversibility is sufficient but may not be necessary for a product form and biased local balance when there are multiple class of transitions. Our final result is the multi-class analogue of corollary 11. Corollary 21. Suppose rju,kv is reversible on M 0 ≡ {ju: j ∈ M , u ∈ Tj } with stationary distribution wju . Assume each qj has coefficients (40) and (41). If πj is the stationary distribution of qj and −1 d αju (xj ), α ˜ dju (xj ) = wju αaju (xj ), xj ∈ Ej . (47) α ˜ aju (xj ) = wju Q then π(x) = j∈M πj (xj ) is the stationary distribution of q. If this is the case, a /β d . wju = α ¯ dju /¯ αaju = βju ju
Acknowledgements This work originally started with Takada and Miyazawa [19], and combines ideas and results due to Xiuli Chao and Richard Serfozo. References [1] F. Baskett, K.M. Chandy, R.R. Muntz and F.G. Palacios, Open, closed and mixed networks of queues with different classes of customers, J. ACM 22 (1975) 248–260. [2] R.J. Boucherie, X. Chao and M. Miyazawa, Arrival first networks of queues with product form solution, Preprint (1997). [3] K.M. Chandy, J.H. Howard, Jr. and D.F. Towsley, Product form and local balance in queueing networks, J. ACM 24 (1977) 250–263. [4] X. Chao and M. Miyazawa, A probabilistic decomposition approach to quasi-reversibility and its applications in coupling of queues, Preprint (1996). [5] X. Chao and M. Miyazawa, On quasi-reversibility and local balance: An alternative derivation of the product-form results, to appear in Oper. Res. (1996). [6] E. Gelenbe, Product-form queueing networks with negative and positive customers. J. Appl. Probab. 28 (1991) 656–663. [7] P.W. Glynn, Diffusion approximations, chapter 4 in: Handbooks on OR & MS, Vol. 2, eds. D.P. Heyman and M.J. Sobel (1990) pp. 145–198. [8] J.M. Harrison and R.J. Williams, Brownian models of open queueing networks with homogeneous customer populations, Stochastica 22 (1987) 77–115.
X. Chao et al. / Markov network processes
401
[9] J.M. Harrison and R.J. Williams, Brownian models of feedforward queueing networks: Quasireversibility and product form solutions, Ann. Appl. Probab. 2 (1992) 263–293. [10] W. Henderson, C.E.M. Pearce, P.K. Pollett and P.G. Taylor, Connecting internally balanced quasireversible Markov processes, Adv. Appl. Probab. 24 (1992) 934–959. [11] F.P. Kelly, Reversibility and Stochastic Networks (Wiley, New York, 1979). [12] F.P. Kelly, Networks of quasi-reversible nodes, in: Applied Probability–Computer Science: The Interface, Vol. I, eds. R.L. Disney and T.J. Ott (1982) pp. 3–26. [13] J.F.C. Kingman, Markov population processes, J. Appl. Probab. 6 (1969) 1–18. [14] Y.V. Malinkovsky, A criterion for pointwise independence of states of units in an open stationary Markov queueing network with one class of customers, Theory Probab. Appl. 35(4) (1990) 797–802. [15] R.R. Muntz, Poisson departure processes and queueing networks, IBM Research Report RC4145, A shorter version appeared in: Proc. of the 7th Annual Conf. on Information Science and Systems, Princeton (1972) pp. 435–440. [16] M.F. Neuts, Structured Stochastic Matrices of M/G/1 Type and Their Applications (Marcel Dekker, New York, 1981). [17] P.K. Pollett, Connecting reversible Markov processes, Adv. in Appl. Probab. 18 (1986) 880–900. [18] R.F. Serfozo, Poisson functionals of Markov processes and queueing networks, Adv. in Appl. Probab. 21 (1989) 595–611. [19] H. Takada and M. Miyazawa, Necessary and sufficient conditions for product-form queueing networks, Science University of Tokyo (1997). [20] N.M. van Dijk, Queueing Networks and Product Forms: A Systems Approach (Wiley, New York, 1993). [21] J. Walrand, An Introduction to Queueing Networks (Prentice-Hall, Englewood Cliffs, NJ, 1988). [22] P. Whittle, Systems in Stochastic Equilibrium (Wiley, New York, 1986).
