c 2006 Society for Industrial and Applied Mathematics
SIAM J. MATRIX ANAL. APPL. Vol. 29, No. 1, pp. 117–142
MATRIX MEASURES AND RANDOM WALKS WITH A BLOCK TRIDIAGONAL TRANSITION MATRIX∗ HOLGER DETTE† , BETTINA REUTHER† , W. J. STUDDEN‡ , AND M. ZYGMUNT§ Abstract. In this paper we study the connection between matrix measures and random walks with a block tridiagonal transition matrix. We derive sufficient conditions such that the blocks of the n-step block tridiagonal transition matrix of the Markov chain can be represented as integrals with respect to a matrix valued spectral measure. Several stochastic properties of the processes are characterized by means of this matrix measure. In many cases this measure is supported in the interval [−1, 1]. The results are illustrated by several examples including random walks on a grid and the embedded chain of a queuing system. Key words. Markov chain, block tridiagonal transition matrix, spectral measure, matrix measure, quasi-birth-and-death process, canonical moments, Chebyshev matrix polynomials AMS subject classifications. 60J10, 42C05 DOI. 10.1137/050638230
1. Introduction. Consider a homogeneous Markov chain with state space (1.1)
Cd = {(i, j) ∈ N0 × N | 1 ≤ j ≤ d}
and block tridiagonal transition matrix ⎛
(1.2)
B0 ⎜ C1T ⎜ P =⎜ ⎝ 0
A0 B1 C2T
⎞
0 A1 B2 .. .
A2 .. .
..
⎟ ⎟ ⎟, ⎠ .
where d ∈ N is finite, and A0 , A1 , . . . , B0 , B1 , . . . , C1 , C2 , . . . are d × d matrices containing the probabilities of one-step transitions (here and throughout this paper C T denotes the transpose of the matrix C). If the one-step block tridiagonal transition matrix is represented by (1.3)
P = (Pii )i,i =0,1,...
with d × d block matrices Pii , the probability of going in one step from state (i, j) to (i , j ) is given by the element in the position (j, j ) of the matrix Pii . In the state (i, j), i is usually referred to as the level of the state and j is referred to as the phase ∗ Received
by the editors August 16, 2005; accepted for publication (in revised form) by M. Benzi June 22, 2006; published electronically December 21, 2006. http://www.siam.org/journals/simax/29-1/63823.html † Ruhr-Universit¨ at Bochum, Fakult¨ at f¨ ur Mathematik, 44780 Bochum, Germany (holger.dette@ rub.de,
[email protected]). The work of the first author was supported by the Deutsche Forschungsgemeinschaft (De 502/22-1, SFB 475, Komplexit¨ atsreduktion in multivariaten Datenstrukturen). ‡ Department of Statistics, Purdue University, West Lafayette, IN 47907-1399 (studden@ stat.purdue.edu). § Department of Applied Mathematics, AGH University of Science and Technology, al. Mickiewicza 30, 30-059 Krakow, Poland (
[email protected]). 117
118
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
of the state. Some illustrative examples will be given below. Block tridiagonal transition matrices of the form (1.2) naturally appear in the analysis of the embedded Markov chains of continuous-time Markov processes with state space (1.1) and block tridiagonal infinitesimal generator (see, e.g., the monographs of Neuts (1981) and Neuts (1989) or the recent work of Marek (2003) and Dayar and Quessette (2002) among many others) and these models have significant applications in the performance evalutation of communication systems (see, e.g., Ost (2001)). Markov chains with transition matrix (1.2) are known in the literature as level dependent quasibirth-and-death processes and several authors have contributed to the analysis of such processes (see Hajek (1982), Gaver, Jacobs, and Latouche (1984), Ramaswami and Taylor (1996), Bright and Taylor (1995), Latouche, Pearce, and Taylor (1998), Bean, Pollett, and Taylor (2000), and Li and Cao (2004) among many others). Bright and Taylor (1995) considered the problem of calculating the equilibrium distribution of a quasi-birth-and-death process for finite dimensional block matrices, while Ramaswami and Taylor (1996) investigated level dependent processes with infinite dimensional blocks. Quasistationary distributions of these processes were considered by Bean, Pollett, and Taylor (2000). Latouche, Pearce, and Taylor (1998) discussed the existence and the form of invariant measures for quasi-birth-and-death processes. In the present paper we propose an alternative methodology for analyzing some level dependent quasi-birth-and-death processes which is based on some spectral analysis of the transition matrix. For this we note that matrices of the form (1.2) are also closely related to a sequence of matrix polynomials recursively defined by (1.4)
xQn (x) = An Qn+1 (x) + Bn Qn (x) + CnT Qn−1 (x), n ∈ N0 ,
where Q−1 (x) = 0 and Q0 (x) = Id denotes the d × d identity matrix. If An = Cn+1 and Bn is symmetric it follows that there exists a matrix measure Σ = {σij }i,j=1,...,d on the real line (here σij are signed measures such that for any Borel set A ⊂ R the matrix Σ(A) is nonnegative definite) such that the polynomials Qj (x) are orthonormal with respect to a left inner product, i.e., Qi , Qj = (1.5) Qi (x)dΣ(x)QTj (x) = δij Id R
(see, e.g., Sinap and Van Assche (1996), or Duran (1995)). In recent years several authors have studied properties of matrix orthonormal polynomials (see, e.g., Rodman (1990), Duran and Van Assche (1995), Duran (1996, 1999), and Dette and Studden (2001) among many others). In the present paper we are interested in the relation between Markov chains with state space Cd defined in (1.1) and block tridiagonal transition matrix (1.2) and the polynomials Qj (x) defined by the recursive relation (1.4). In the case d = 1 this problem has been studied extensively in the literature (see Karlin and McGregor (1959), Whitehurst (1982), Woess (1985), Van Doorn and Schrijner (1993, 1995), and Dette (1996) among many others), but the case d > 1 is more difficult, because in this case a system of matrix polynomials {Qj (x)}j≥0 satisfying a recurrence relation of the form (1.4) is not necessarily orthogonal with respect to an inner product induced by a matrix measure. In section 2 we characterize the transition matrices of the form (1.2) such that there exists an integral representation for the corresponding n-step transition probabilities in terms of the matrix measure and corresponding orthogonal
MATRIX MEASURES AND RANDOM WALKS
119
matrix polynomials, i.e.,
Qj (x)dΣ(x)QTj (x) = xn Qi (x)dΣ(x)QTj (x) , Pijn where Pijn denotes the d × d block of the n-step block tridiagonal transition matrix P n in the position (i, j). In other words, the element in the position (k, l) of Pijn is the probability of going in n steps from state (i, k) to (j, l) and admitting an integral representation. We also derive a sufficient condition such that the spectral (matrix) measure Σ (if it exists) is supported on the interval [−1, 1]. In section 3 we discuss several illustrative examples where this condition is satisfied including some examples from queuing theory. Section 4 continues our more theoretical discussion and some consequences of the integral representation are derived. We present a characterization of recurrence by properties of the blocks of the transition matrix, which generalizes the classical characterization of recurrence of a birth-and-death chain (see Karlin and Taylor (1975)). Finally, in section 5 we present some applications of our results, which demonstrate the potential of our approach. In particular we derive a very simple necessary condition for positive recurrence of a quasi-birth-and-death process and a new representation of the equilibrium distribution in terms of the random walk measure Σ and the orthogonal polynomials Qj (x). 2. Random walk matrix polynomials. A matrix measure Σ is a d × d matrix Σ = {σij }i,j=1,...,d of finite signed measures σij on the Borel field of the real line R or of an appropriate subset. It will be assumed here that for each Borel set A ⊂ R the matrix Σ(A) = {σij (A)}i,j=1,...,d is symmetric and nonnegative definite, i.e., Σ(A) ≥ 0. The moments of the matrix measure Σ are given by the d × d matrices Sk = tk dΣ(t), k = 0, 1, . . . , (2.1) and only measures for which all relevant moments exist will be considered throughout this paper. Let Gi (i = 0, . . . , n) denote d × d matrices; then a matrix polynomial is n defined by P (t) = i=0 Gi ti . The inner product of two matrix polynomials, say, P and Q, is defined by P, Q = P (t)Σ(dt)QT (t), (2.2) where QT (t) denotes the transpose of the matrix Q(t). Sinap and Van Assche (1996) call this the “left” inner product. Orthogonal polynomials are defined by orthogonalizing the sequence Ip , tIp , t2 Ip , . . . with respect to the above inner product. If S0 , S1 , . . . is a given sequence of matrices such that the block Hankel matrices ⎞ ⎛ S0 · · · S m ⎜ .. ⎟ H 2m = ⎝ ... (2.3) . ⎠ Sm
...
S2m
are positive definite, it is well known (see, e.g., Marcell´ an and Sansigre (1993)) that a matrix measure Σ with moments Sj (j ∈ N0 ) and a corresponding infinite sequence of orthogonal matrix polynomials with respect to dΣ(x) exist. Moreover, these matrix polynomials satisfy a three term recurrence relation.
120
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
Let {Qj (x)}j≥0 denote a sequence of matrix polynomials defined by the recurrence relationship (1.4), where the matrices Cj (j ∈ N) and Aj (j ∈ N0 ) in (1.2) are assumed to be nonsingular. The following results characterize the existence of a matrix measure Σ such that the polynomials Qj (x) are orthogonal with respect to dΣ(x) in the sense of (2.2). Theorem 2.1. Assume that the matrices An (n ∈ N0 ) and Cn (n ∈ N) in the onestep block tridiagonal transition matrix (1.2) are nonsingular. There exists a matrix measure Σ on the real line with positive definite Hankel matrices H 2m (m ∈ N0 ) such that the polynomials {Qn (x)}n∈N0 defined by (1.4) are orthogonal with respect to the measure dΣ(x) if and only if there exists a sequence of nonsingular matrices {Rn }n∈N0 such that the following relations are satisfied: Rn Bn Rn−1 is symmetric ∀ n ∈ N0 , (2.4) RnT Rn = Cn−1 · · · C1−1 (R0T R0 )A0 · · · An−1 ∀ n ∈ N. Proof. Assume that the polynomials {Qn (x)}n∈N0 are orthogonal with respect to the measure dΣ(x), that is, (2.5) R
Qi (x)dΣ(x)QTj (x) = 0,
whenever i = j and (2.6) R
Qi (x)dΣ(x)QTi (x) = Fi > 0
(i ∈ N0 ),
where we use the notation Fi > 0 for a positive definite matrix Fi ∈ Rd×d (the fact that the matrix Fi is positive definite follows from a straightforward calculation using −1/2 the assumption that H 2m is positive definite for all m ∈ N0 ). Define Rn = Fn ˜ n (x) = Rn Qn (x); then it is easy to see that the polynomials {Q ˜ n (x)}n∈N are and Q 0 orthonormal with respect to the measure dΣ(x). Therefore it follows from Sinap and Van Assche (1996) that there exist d×d nonsingular matrices {Dn }n∈N and symmetric matrices {En }n∈N0 such that the recurrence relation (2.7)
˜ n (x) = Dn+1 Q ˜ n+1 (x) + En Q ˜ n (x) + DnT Q ˜ n−1 (x) xQ
˜ −1 (x) = 0, Q ˜ 0 (x) = R0 ). On the other hand, we obtain is satisfied for all n ∈ N0 , (Q ˜ n (x) = Rn Qn (x) the recurrence relation from (1.4) and the representation Q (2.8) −1 ˜ T −1 ˜ ˜ n (x) = Rn An R−1 Q ˜ xQ n+1 n+1 (x) + Rn Bn Rn Qn (x) + Rn Cn Rn−1 Qn−1 (x),
and a comparison of (2.7) and (2.8) yields (2.9)
−1 −1 Dn+1 = Rn An Rn+1 , En = Rn Bn Rn−1 , DnT = Rn CnT Rn−1 ,
MATRIX MEASURES AND RANDOM WALKS
121
where the matrix En is symmetric. Now a straightforward calculation gives −1 T T = (Rn+1 Cn+1 Rn−1 )T = (RnT )−1 Cn+1 Rn+1 , Rn An Rn+1
or equivalently −1 T Rn+1 Rn+1 = Cn+1 (RnT Rn )An .
This yields by an induction argument RnT Rn = Cn−1 · · · C1−1 R0T R0 A0 · · · An−1 , n ∈ N, and proves the first part of Theorem 2.1. For the converse assume that the relations in (2.4) are satisfied and consider the ˜ n (x) = Rn Qn (x). These polynomials satisfy the recurrence relation polynomials Q (2.8) and from (2.4) it follows that the matrices En = Rn Bn Rn−1 are symmetric (n ∈ N0 ), while −1 T = (Rn+1 Cn+1 Rn−1 )T Dn+1 = Rn An Rn+1
by the second assumption in (2.4). Therefore the recurrence relation for the polynomi˜ n (x) is of the form (2.7) and by the discussion following Theorem 3.1 in Sinap and als Q van Assche (1996) these polynomials are orthonormal with respect to a matrix mea˜ n (x) sure dΣ(x). This also implies the orthogonality of the polynomials Qn (x) = Rn−1 Q with respect to the measure dΣ(x). ˜ n (t) have leading coefficient Id Because the polynomials Qn (t) = R0−1 D1 · · · Dn Q we obtain that the matrix (2.10) Qn , Qn = Qn (t)dΣ(t)QTn (t) = R0−1 D1 · · · Dn DnT · · · D1T (R0T )−1 is nonsingular. On the other hand it follows from Dette and Studden (2001) that the − , of S2n in left-hand side of (2.10) is equal to the Schur complement, say, S2n − S2n H 2n . Because the matrix H 2n is positive definite if and only if H 2n−2 and the Schur complement of S2n in H 2n are positive definite it follows by an induction argument that all Hankel matrices obtained from the moments of the matrix measure Σ are positive definite. Remark 2.2. Throughout this paper a matrix measure Σ with corresponding orthogonal matrix polynomials Qi (x) is called a random walk matrix measure or spectral measure and the polynomials Qi (x) will be called random walk matrix polynomials if the assumptions of Theorem 2.1 are satisfied. Because the polynomials ˜ i (x) = Ri Qi (x) defined in the proof of Theorem 2.1 are orthonormal with respect Q to the measure dΣ(x) it follows that ˜0, Q ˜0 = Q ˜ 0 (x)dΣ(x)Q ˜ T = R0 S0 R T , (2.11) Id = Q 0 0 or equivalently (2.12)
R0−1 ((R0T )−1 ) = (R0T R0 )−1 = S0 ,
122
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
where S0 is the 0th moment of the matrix measure Σ (see (2.1)). We finally note that the matrices Rn in Theorem 2.1 are not unique. If {Rn }n∈N0 is a sequence of ˜ n }n∈N = matrices satisfying (2.4), these relations are also fulfilled for the sequence {R 0 {Un Rn }n∈N0 , where Un (n ∈ N0 ) are arbitrary orthogonal matrices. Before we present some examples, where the conditions of Theorem 2.1 are satisfied we derive some consequences of the existence of a random walk measure. For this let Q(x) = (QT0 (x), QT1 (x), . . . )T denote the vector of matrix polynomials defined by the recursive relation (1.4); then it is easy to see that the recurrence relation (1.4) is equivalent to (2.13)
xQ(x) = P Q(x),
which gives (by iteration) xn Q(x) = P n Q(x).
(2.14) Therefore
n
x
(2.15)
Q(x)dΣ(x)QTj (x)
=P
n
Q(x)dΣ(x)QTj (x),
and from the orthogonality of the random walk polynomials we obtain the representation (2.16)
Pijn
=
n
x
Qi (x)dΣ(x)QTj (x)
−1 Qj (x)dΣ(x)QTj (x)
for the block in the position (i, j) of the n-step block tridiagonal transition matrix P n. Theorem 2.3. If the assumptions of Theorem 2.1 are satisfied, the block Pijn in the position (i, j) of the n-step block tridiagonal transition matrix P n of the random walk can be represented in the form (2.16), where Σ denotes a random walk measure corresponding to the one-step transition matrix P . Remark 2.4. Note that the random walk measure is not necessarily uniquely determined by the random walk on the grid Cd . However, using the case i = j = 0 in (2.16) it follows for the moments of the random walk measure (2.17)
n P00 = Sn S0−1
(n ∈ N0 ),
n is the first block in the n-step transition matrix of the random walk. Therewhere P00 fore the moments of a random walk measure are essentially uniquely determined. In the following we will derive a sufficient condition such that the random walk measure (if it exists) is supported on the interval [−1, 1]. In this case the measure is determined by its moments. Theorem 2.5. Assume that the conditions of Theorem 2.1 are satisfied and define the block diagonal matrix R = diag (R0 , R1 , R2 , . . . ). If the matrix R is symmetric and the matrix
(2.18)
P˜ = RT P R−1
MATRIX MEASURES AND RANDOM WALKS
123
has nonnegative entries, then the random walk matrix measure Σ = {σij }i,j=1,...,d corresponding to the polynomials in (1.4) is supported on the interval [−1, 1], that is, supp(σij ) ⊂ [−1, 1] ∀ i, j = 1, . . . , d. Proof. Note that the matrix in (2.18) is symmetric (because the assumptions of Theorem 2.1 are satisfied) and that the entries of P˜ are nonnegative, by the assumptions of the theorem. According to Schur’s test (see Halmos and Sunder (1978), Theorem 5.2) it follows that
P˜ 2 ≤ 1
(2.19)
if we can find two vectors, say, v, w, with positive components such that P˜ v ≤ w
and P˜ w ≤ v
(where the symbol ≤ means here inequality in each component). If v = w = R1 (here 1 denotes the infinite dimensional vector with all elements equal to one), then the representation (2.18) implies that P˜ v = P˜ R1 = RT P 1 ≤ RT 1, which shows that (2.19) is indeed satisfied. Now let (2.20)
Πj = Cj−1 . . . C1−1 R0T R0 A0 . . . Aj−1 = RjT Rj ,
and consider the inner product (2.21)
x, yΠ =
∞
xTj Πj yj
j=0
(with x = (xT0 , xT1 , . . . ); y = (y0T , y1T , . . . ); xj ∈ Rd , yj ∈ Rd ) and its corresponding norm, say, · Π . Define (2.22)
2 (Rd ) = {x = (xT0 , xT1 , . . . ) | xj ∈ Rd (j ∈ N0 ); x 2Π < ∞}.
T From the definition of P and Πj it is easy to see that Πi Pij = Pji Πj (for all i, j ∈ N0 ), which implies that P is a selfadjoint operator with respect to the inner product ·, ·Π . Moreover, we have for any x
P x Π = xT P T ΠP x = xT RT P˜ T P˜ Rx = P˜ Rx 2 ≤ P˜ 2 Rx 2 ≤ xT RT Rx = xT Πx = x Π , where we used the representation Π = RT R and (2.19). Consequently, P Π ≤ 1, which proves the theorem. We note that there are many examples where the assumptions of Theorem 2.5 are satisfied and we conjecture, in fact, that a random walk measure is always supported in the interval [−1, 1]. In the case d = 1 this property holds because in this case the assumptions of Theorems 2.1 and 2.5 are obviously satisfied. This was shown before by Karlin and McGregor (1959), and an alternative proof can be found in Dette and Studden (1997), Chapter 8.
124
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
Our next result gives a relation between the Stieltjes transforms of two random ˜ where only the matrices B0 and B ˜0 differ in the correwalk measures, say, Σ and Σ, sponding one-step block tridiagonal transition matrices P and P˜ . Theorem 2.6. Consider the one-step block tridiagonal transition matrix P in (1.2) and the matrix ⎛
˜0 B ⎜ C1T ⎜ P˜ = ⎜ ⎝ 0
(2.23)
A0 B1 C2T
⎞
0 A1 B2 .. .
A2 .. .
..
⎟ ⎟ ⎟, ⎠ .
and assume that there exists a random walk measure Σ corresponding to the one˜0 R−1 is symmetric, where R0 is step transition matrix P such that the matrix R0 B 0 a matrix such that (2.4) is satisfied. Then there exists also a random walk measure ˜ corresponding to the matrix P˜ . If Σ and Σ ˜ are determinate, then the Stieltjes Σ transforms of both matrix measures are related by (2.24)
⎧ ⎫−1 ˜ −1 ⎬ dΣ(t) ⎨ dΣ(t) ˜0 ) = . − S0−1 (B0 − B ⎩ ⎭ z−t z−t
˜0 R−1 is symmetric and the matrices P and P˜ Proof. Because the matrix R0 B 0 differ only by the element in the first block, the sequence of matrices R0 , R1 , . . . can be used to symmetrize the matrices P and P˜ simultaneously (see the proof of Theorem 2.1). Consequently, there exists a random walk measure corresponding to the random walk with one-step block tridiagonal transition matrix P˜ . Let {Qn (x)}n∈N0 denote the system of matrix orthogonal polynomials defined by the recursive relation (1.4) and ˜ n (x)}n∈N by the same recursion, where the matrix B0 has been replaced define {Q 0 ˜0 . A straightforward calculation shows that the difference polynomials by B ˜ j (x) − Qj (x) Rj (x) = Q also satisfy the recursion (1.4) with initial conditions R0 (x) = 0, R1 (x) = A−1 0 (B0 − ˜0 ). In particular, these polynomials are “proportional” to the first associated orB thogonal matrix polynomials (2.25)
Q(1) n (x) =
Qn (x) − Qn (t) dΣ(t) x−t
(n ∈ N0 ),
that is, (2.26)
T ˜ Rn (x) = Q(1) n (x)R0 R0 (B0 − B0 ).
Recall from the proof of Theorem 2.1 that the systems {Rn Qn (x)R0−1 }n∈N0 and ˜ n (x)R−1 }n∈N are orthonormal with respect to the random walk measures {Rn Q 0 0 T ˜ dμ(x) = R0 dΣ(x)R0T and d˜ μ = R0 dΣ(x)R ˜ are 0 , respectively, and that μ and μ determinate. Therefore we obtain from Markov’s theorem for matrix orthogonal
125
MATRIX MEASURES AND RANDOM WALKS
polynomials (see Duran (1996)) that ˜ dΣ(t) d˜ μ(t) T −1 = R0−1 (R ) (2.27) z−t z−t 0 T T −1 ˜ n (z)R−1 )−1 (Rn Q ˜ (1) = lim R0−1 (Rn Q n (z)R0 )(R0 ) 0 n→∞
˜ n (z))−1 Q ˜ (1) = lim (Q n (z) n→∞
T ˜ −1 Q(1) (z) = lim {Qn (z) + Q(1) n (z)R0 R0 (B0 − B0 )} n n→∞
−1 ˜0 )}−1 = lim {{(Qn (z))−1 Q(1) + R0T R0 (B0 − B n (z)} n→∞
T −1 = lim {R0T {(Rn Qn (z)R0−1 )−1 Rn Q(1) R0 n (z)R0 } n→∞
˜0 )}−1 +R0T R0 (B − B =
R0T
= =
dμ(t) z−t
dΣ(t) z−t dΣ(t) z−t
−1
−1 R0 +
˜0 ) −B
R0T R0 (B0
−1
−1 +
R0T R0 (B0
+
S0−1 (B0
˜0 ) −B −1
−1
˜0 ) −B
,
(1)
˜ n (x) denotes the first associated orthogonal matrix polynomial obtained by where Q (1) ˜ n (x) and we have used the fact that Q ˜ (1) the analogue of (2.25) from Q n (x) = Qn (x) for the third equality (note that this identity is obvious from the definition of P and P˜ in (1.2) and (2.23), respectively). Remark 2.7. Note that Theorem 2.1 and some of its consequences are derived under the assumption of nonsingular matrices An and Cn . As pointed out by a referee there are several applications in queuing theory where these matrices do not have full rank (see Latouche and Ramaswami (1999)). In this remark we indicate how the nonsingularity assumptions regarding the matrices Cn can be relaxed (note that this covers most of the commonly used queuing models). For this purpose we rewrite the conditions in Theorem 2.1 as (2.28)
T Cn+1 Rn+1 Rn+1 = RnT Rn An ∀ n ∈ N0
and (2.29)
Rn Bn = En Rn ∀ n ∈ N0 ,
for some sequence of symmetric matrices (En )n∈N0 . Note that the conditions (2.28) and (2.29) were derived in the proof of Theorem 2.1 (under the assumptions of Theorem 2.1 they are in fact equivalent to (2.4)). We will now demonstrate that these
126
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
conditions are in fact sufficient for the proof of the existence of a random walk measure using some spectral theory of selfadjoint operators (see, e.g., Berezanskii (1968)). In the following we indicate how such a measure can be derived; further details can be found in Berezanskii (1968), pages 501–607. For this purpose define Πj = RjT Rj
(j ∈ N0 )
and consider the space L(Rd , L(Rd , Rd )) which can be identified with T | Xj ∈ Rd×d , X, X < ∞ , 2 (Rd×d ) := X = X0T , X1T , . . . T where the matrix valued pseudo inner product is defined by (Y = Y0T , Y1T , . . . ) X, Y :=
∞
XjT Πj Yj .
j=1
Note that the space 2 (Rd×d ) equipped with the inner product X, Y =
1 traceX, Y d
is a Hilbert space and isometric isomorph to the space 2 (Rd ) defined in (2.22). Moreover, the matrix P in (1.2) defines an operator acting on 2 (Rd×d ) and 2 (Rd ), denoted by P and J, respectively; that is, (P X)n = An Xn+1 + Bn Xn + CnT Xn−1 (n ∈ N0 , X−1 = 0), (Jx)n = An xn+1 + Bn xn + CnT xn−1 (n ∈ N0 , x−1 = 0). Note that (2.28) and (2.29) imply the symmetry conditions PijT Πi = Πj Pji
(2.30)
(this follows by an elementary calculation), and recalling the definition of the inner product ·, ·Π in (2.21) we obtain Jx, yΠ =
∞ ∞ i=0 j=0
(Pij xj )T Πi yi =
∞ ∞
xTj Πj Pji yi = x, JyΠ .
i=0 j=0
In other words, J is a selfadjoint operator acting on 2 (Rd ). Let (Eλ )λ denote the corresponding resolution of the identity (i.e., J = λEλ ); then (Eλ )λ induces a resolution of the identity, say, (Eλ )λ , corresponding to the operator P on 2 (Rd×d ) in the following way: (Eλ U )x := Eλ (U x), U ∈ 2 (Rd×d ), x ∈ Rd . Now define E (j) = (0d , . . . , 0d , Id , 0d , . . . )T ∈ 2 (Rd×d ) as the jth “unit” vector (here 0d is the d × d matrix with all entries equal to 0) and the spectral measure by Σ(λ) = E (0) , Eλ E (0) ∈ Rd×d ;
MATRIX MEASURES AND RANDOM WALKS
127
then it follows by similiar arguments as in Berezanskii (1968), pages 562–565, that Qi (x)dΣ(x)QTj (x) = E (i) , E (j) = δij Πj , where δij denotes Kronecker’s symbol. The same arguments as used in the derivation of Theorem 2.3 now imply n Pij Πj = xn Qi (x)dΣ(x)QTj (x), which is the statement of Theorem 2.3. Other results of this paper can be generalized in a similiar way. For example, Theorem 2.5 remains valid if there exists a matrix P˜ with nonnegative entries such that P˜ R = RT P. The details are omitted for the sake of brevity. 3. Examples. In this section we present several examples where the conditions of Theorem 2.1 are satisfied. 3.1. Random walks on the integers. Consider the classical random walk on Z (see, e.g., Feller (1950)) with one-step up, down, and holding transition probabilities pi , qi , and ri (respectively), where pi + qi + ri ≤ 1, i ∈ Z, where the strict inequality pi + qi + ri < 1 is interpreted as a permanent absorbing state i∗ , which can be reached from the state i with probability 1 − pi − qi − ri . By the one-to-one mapping ⎧ ⎪ ⎨Z → C2 , ψ: (i, 1) if i ∈ N0 , ⎪ ⎩i → (−i − 1, 2) else, this process can be interpreted as a process on the grid C2 , where transitions from the first to the second row are only possible if the process is in state (0, 1). The transition matrix of this process is given by (1.2) with 2 × 2 blocks
q0 0 r0 rn B0 = (3.1) ; Bn = ; 0 r−n−1 p−1 r−1
0 0 pn q An = (3.2) ; CnT = n . 0 q−n−1 0 p−n−1 It is easy to see that the conditions of Theorem 2.1 are satisfied with the matrices ⎞ ⎛ p0 ...pn−1 0 1 0 q1 ...qn ⎠, (3.3) , Rn = ⎝ R0 = q0 q0 q−1 ...q−n 0 0 p−1 p−1 p−2 ...p−n−1
and consequently, there exists a random walk matrix measure corresponding to this process, say, Σ, which is supported in the interval [−1, 1] (see Theorem 2.5). For the calculation of the Stieltjes transform of this measure we use Theorem 2.6 and obtain −1 dΣ(t) ˜ −1 ˜0 ) = Φ (z) − R0T R0 (B0 − B (3.4) . Φ(z) = z−t ˜ is the Stieltjes transform of a random walk measure Σ ˜ with transition matrix Here Φ (1.2), where the matrix B0 in (3.1) has been replaced by
0 r0 ˜ B0 = , 0 r−1
128
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
˜0 is given by and the matrix B0 − B ˜0 = B0 − B
q0 . 0
0 p−1
˜ − denote the corresponding ˜ is diagonal and if Φ ˜ + and Φ Note that the matrix Φ diagonal elements, we obtain from (3.4) the representation
−1 ˜ + (z) −q0 1/Φ ˜ − (z) −q0 1/Φ
˜ + (z) ˜ − (z)Φ ˜ + (z) 1 q0 Φ Φ . = ˜ + (z) ˜ − (z) ˜ − (z) q0 Φ− (z)Φ ˜ + (z)Φ Φ 1 − q2 Φ
Φ(z) =
dΣ(t) = z−t
0
In particular, for the classical random walk (pi = p, qi = q, ri = 0 for all i ∈ Z) we have z 2 − 4pq z − + ˜ − (z) = p Φ+ (z), ˜ (z) = − , Φ Φ 2pq q and a straightforward calculation gives the result ⎛ !⎞ z 1 √ −1 √ 1 − 2q ⎜ z 2 −4pq z 2 −4pq ⎟ ! Φ(z) = ⎝ ⎠, p 1 z −1 √ 2 √ 2 2q 1 − q z −4pq
z −4pq
which was also obtained by Karlin and McGregor (1959) by a probabilistic argument. 3.2. An example from queuing theory. In a recent paper Dayar and Quessette (2002) considered a system of two independent queues, where queue 1 is an M/M/1 and queue 2 is an M/M/1/d − 1. Both queues have a Poisson arrival process with rate λi (i = 1, 2) and exponential service distributions with rates μi (i = 1, 2). It is easy to see that the embedded random walk corresponding to the quasi-birth-anddeath process representing the length of queue 1 (which is unbounded) and the length of queue 2 (which varies between 0, 1, . . . , d − 1) has a one-step transition matrix of the form (1.2), where the blocks Bi , Ai , and Ci are given by ⎞ ⎛ λ2 0 λ1 +λ2 λ2 ⎟ ⎜ μ2 0 ⎟ ⎜ γ−μ1 γ−μ1 ⎟ ⎜ .. .. .. ⎟, (3.5) B0 = ⎜ . . . ⎟ ⎜ ⎟ ⎜ μ2 λ2 0 ⎝ γ−μ1 γ−μ1 ⎠ μ2 0 λ1 +μ2 ⎛
(3.6)
⎜ ⎜ ⎜ Bi = ⎜ ⎜ ⎜ ⎝
0 μ2 γ
λ2 γ−μ2
0 ..
.
⎞ λ2 γ
..
.
μ2 γ
..
.
0 μ2 γ−λ2
λ2 γ
0
⎟ ⎟ ⎟ ⎟ , i ≥ 1, ⎟ ⎟ ⎠
MATRIX MEASURES AND RANDOM WALKS
⎛
(3.7)
⎜ ⎜ ⎜ A0 = ⎜ ⎜ ⎜ ⎝ ⎛
(3.8)
and
(3.9)
⎜ ⎜ ⎜ Ai = ⎜ ⎜ ⎜ ⎝
⎛ ⎜ ⎜ ⎜ Ci = ⎜ ⎜ ⎜ ⎝
λ1 λ1 +λ2
⎞
. λ1 γ−μ1
λ1 λ1 +μ2
⎞ ⎟ ⎟ ⎟ ⎟ , i ≥ 1, ⎟ ⎟ ⎠
λ1 γ
..
. λ1 γ
μ1 γ−μ2
⎟ ⎟ ⎟ ⎟, ⎟ ⎟ ⎠
λ1 γ−μ1
..
λ1 γ−μ2
129
λ1 γ−λ2
⎞ μ1 γ
..
. μ1 γ
μ1 γ−λ2
⎟ ⎟ ⎟ ⎟ , i ≥ 1, ⎟ ⎟ ⎠
respectively, and γ = λ1 + λ2 + μ1 + μ2 , λ1 < μ1 . A straightforward calculation shows that the assumptions of Theorem 2.1 are satisfied, where the matrices Rn are diagonal and given by ⎞ ⎛ √ √ d−3 d−2 (λ + μ )λ 1 2 2 (λ1 + λ2 )μ2 λ2 λ2 λ2 ⎠, R0 = diag ⎝ , 1, √ , , . . . , √ , μ2 μ2 μ2 (γ − μ1 )λ2 (γ − μ1 )μ2d−2 ⎞ ⎛ √ d−3 d−2 γλ λ λ (γ − λ )λ 1 1 2 λ1 (γ − μ2 )μ2 γλ1 2 2 ⎠, R1 = diag ⎝ , ,..., , d−3 λ2 (γ − μ1 )μ1 (γ − μ1 )μ1 (γ − μ1 )μ1 μ2 (γ − μ1 )μ1 μd−2 2 " i−1 λ1 Ri = R1 , i ≥ 2. μ1 It also follows from Theorem 2.5 that the corresponding random walk matrix measure is supported in the interval [−1, 1]. 3.3. The simple random walk on the grid. Consider the random walk on the grid Cd , where the probabilities of going from state (i, j) to (i, j + 1), (i, j − 1), (i − 1, j), (i + 1, j) are given by u, v, , r, respectively, where u + v + + r = 1. In this case it follows that Ai = rId (i ≥ 0), Ci = Id (i ≥ 1), ⎛ ⎞ 0 u ⎜ v 0 u ⎟ ⎜ ⎟ ⎜ ⎟ v 0 u ⎜ ⎟ Bi = ⎜ (3.10) ⎟ , i ≥ 0, . . . .. .. .. ⎜ ⎟ ⎜ ⎟ ⎝ v 0 u ⎠ v 0
130
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
and it is easy to see that the conditions of Theorem 2.1 are satisfied with # # # # i u u2 ud−1 r , R0 = diag 1, , Ri = ,..., R0 , i ≥ 1. 2 d−1 v v v It now follows from Theorem 2.5 that the corresponding random walk matrix measure is supported in the interval [−1, 1]. For the identification of the Stieltjes transform of the spectral measure we note that the orthonormal polynomials defined by (2.7) have √ constant coefficients given by D = Dn = rId , ⎞ ⎛ √ vu √ √0 ⎟ ⎜ vu vu √ ⎟ ⎜ √0 ⎟ ⎜ vu 0 vu ⎟ ⎜ E = En = ⎜ (3.11) ⎟. .. .. .. ⎟ ⎜ . . . ⎟ ⎜ √ √ ⎝ vu √0 vu ⎠ vu 0 Therefore it follows from the work of Duran (1999) that the Stieltjes transform of the random walk measure is given by 1/2 dΣ(t) 1 zId − E − (zId − E)2 − 4rId . = z−t 2r From the same reference we obtain that the support of the random walk measure is given by the set √ √ supp(Σ) = {x ∈ R | xId − E has an eigenvalue in [−2 r, 2 r]}. (3.12) It is well known (see Basilevsky (1983)) that the eigenvalues of the matrix E in (3.11) are given by
√ jπ , j = 1, . . . , d, 2 uv cos d+1 with corresponding normalized eigenvectors #
d πj 2 xj = sin . d+1 d+1 =1 Therefore it follows from (3.12) that $
% √ √ √ √ πd π supp(Σ) = −2 r + 2 uv cos , 2 r + 2 uv cos d+1 d+1 (note that supp(Σ) ⊂ [−1, 1]). For the calculation of the random walk measure we determine the spectral decomposition of the matrix −H(x) = 4Id − D−1/2 (xId − E)D−1 (xId − E)D−1/2 1 4rId − (xId − E)2 . = r The eigenvalues of this matrix are given by
2 √ 1 πj 4r − x − 2 vu cos , λj (x) = r d+1
131
MATRIX MEASURES AND RANDOM WALKS
and by the results in Duran (1999) the weight of the matrix measure is given by dΣ(x) =
1 √
2π r
U Λ(x)U T dx,
where the matrix Λ(x) is defined by 1/2 , Λ(x) = diag(max(λ1 (x), 0), . . . , max(λd (x), 0)) and the elements of the matrix U = {uj }j,=1,...,d are given by #
jπ 2 sin . uj = d+1 d+1 3.4. Finite state spaces. The assertions of section 2 remain correct for random walks on a finite grid, where the corresponding random walk measure has a finite support. As an example consider a random walk on the finite grid C = Cd,N = {(i, j) ∈ N0 × N| 0 ≤ i ≤ N − 1, 1 ≤ j ≤ d}, where the probabilities of going from state (i, j) to (i, j +1), (i, j −1), (i−1, j), (i+1, j) are given by u, v, , and r, respectively, where u + v + + r = 1. Then the transition matrix P is given by the finite dimensional block tridiagonal matrix ⎞ ⎛ 0 B0 A0 ⎟ ⎜ C1T B1 A1 ⎟ ⎜ ⎟ ⎜ .. .. .. P =⎜ ⎟ . . . ⎟ ⎜ T ⎝ CN −2 BN −2 AN −2 ⎠ T 0 CN BN −1 −1 with Ai = rId , 0 ≤ i ≤ N − 2, Ci = Id , 1 ≤ i ≤ N − 1, and matrices Bi defined by (3.10). A straightforward calculation shows that the corresponding random walk matrix polynomials are given by # n # 1 r Qn (x) = Un A , r 2 n = 0, . . . , N − 1, where Un (z) denotes the Chebyshev polynomial of the second kind and the matrix A is given by ⎛ ⎞ x −u ⎜ −v x −u ⎟ ⎜ ⎟ ⎜ ⎟ −v x −u 1⎜ ⎟ A= ⎜ ⎟. . . . .. .. .. ⎟ r⎜ ⎜ ⎟ ⎝ −v x −u ⎠ −v x Moreover, observing the representations UN
N z! & jπ = , z − 2 cos 2 N +1 j=1
√ det A =
uv r
d
Ud
1 √ x , 2 uv
132
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
we obtain that the zeros of the polynomials QN (x) are given by
√ √ iπ jπ + r cos , i = 1, . . . , d; j = 1, . . . , N. λij = 2 uv cos d+1 N +1 In particular, it follows for the rate of convergence of the probability of no absorption that
n
√ √ π π P (Xn ∈ Cd,N | X0 = x) = O 2n + r cos . uv cos d+1 N +1 3.5. A random walk on a tree. Consider a graph with d rays which are connected at one point, the origin. On each ray the probability of moving away from the origin is p and moving in one step toward the origin is q, where p + q = 1. From the origin the probability of going to the ith ray is di > 0 (i = 1, . . . , d) (see Figure 1, where the case d = 4 is illustrated). It is easy to see that this process corresponds to a random walk on the grid Cd with block tridiagonal transition matrix P in (1.2), where Bi = 0 if i ≥ 1, Ci = qId for all i ≥ 1, A0 = diag (d1 , p, . . . , p) , Ai = pId for all i ≥ 1, and ⎛ ⎞ 0 d2 · · · · · · dd ⎜ q 0 ··· ··· 0 ⎟ ⎜ ⎟ B0 = ⎜ . . .. ⎟ , ⎝ .. .. . ⎠ q
0
···
···
0
d where i=1 di = 1. Moreover, this matrix clearly satisfies the assumptions of Theorem 2.1 with " " " " " d2 dd d1 d2 p dd p ,..., , R1 = diag , , ,..., R0 = diag 1, q q q q2 q2 and Ri =
# i−1 p R1 , i ≥ 2. q
4
1
@ p @ I @ @q q @ @ R @ @
q
p
3
q
@ d4 I @ d1 @s d3 @ 2 d@ R @
q
p q
@ I @ q@ q @p @ R @ @ @
2
Fig. 1. A random walk on a tree.
133
MATRIX MEASURES AND RANDOM WALKS
By an application of Theorem 2.6 and the inversion formula for Stieltjes transforms we obtain for the corresponding random walk measure ⎡
a(x) b2 (x) b3 (x) .. .
b2 (x) f2 (x) e2,3 (x) .. .
b3 (x) e2,3 (x) f3 (x) .. .
... ... ...
⎢ ⎢ ⎢ ⎢ dΣ(x) = ⎢ ⎢ ⎢ ⎣ bd−1 (x) e2,d−1 (x) e3,d−1 (x) . . . e2,d (x) e3,d (x) ... bd (x)
bd (x) e2,d (x) e3,d (x) .. .
⎤
⎥ ⎥ ⎥ ⎥ ⎥ dx, ⎥ ⎥ ed−1,d (x) ⎦ fd (x)
where the functions a, bi , ek, , and fk are given by d ( i=2 d2i d1 + d21 q − (d1 − p)x2 ) 4pq − x2 a(x) = , d d 2pπ(( i=2 d2i + d1 q)2 − ( i=2 d2i + (d1 − p)q)x2 ) dk x 4pq − x2 , k = 2, . . . , d, bk (x) = − d d 2π(( i=2 d2i + d1 q)2 − ( i=2 d2i + (d1 − p)q)x2 ) d 4pq − x2 (px2 − i=2 d2j d1 − d21 q) ek, (x) = , d d 2π(d21 q − (d1 − p)x2 )(( i=2 d2i + d1 q)2 − ( i=2 d2i + (d1 − p)q)x2 ) dk d
k = 2, . . . , d − 1, = 3, . . . , d, d d d1 ( i=2 d2i + d1 q)( i=2,i =k d2i + d1 q) 4pq − x2 fk (x) = d d 2π(d21 q − (d1 − p)x2 )(( i=2 d2i + d1 q)2 − ( i=2 d2i + (d1 − p)q)x2 ) d d −(( i=2 d2i (d1 − p) + i=2,i =k d2i p + d1 (d1 − p)q)x2 ) 4pq − x2 + , d d 2π(d21 q − (d1 − p)x2 )(( i=2 d2i + d1 q)2 − ( i=2 d2i + (d1 − p)q)x2 ) k = 2, . . . , d. √ √ Note that the random walk measure is supported in the interval [−2 pq, 2 pq]. 4. Further discussion. In the present section we derive further consequences of the existence of a random walk measure corresponding to the block tridiagonal transition matrix (1.2). Throughout this section we assume that the conditions of Theorem 2.1 are satisfied and that the corresponding random walk measure is supported in the interval [−1, 1]. 4.1. Recurrence. We denote by (4.1) Hij (z) =
∞ n=0
(Pijn )z n =
Qi (x)dΣ(x)QTj (x) 1 − xz
−1 Qj (x)dΣ(x)QTj (x)
the (matrix) generating function of the block (i, j), where the last identity follows from Theorem 2.3 and Lebesgue’s theorem. Therefore we obtain that a state (i, ) ∈ Cd is
134
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
recurrent if and only if (4.2) ∞
eT Piin e = lim eT Hii (z)e z→1
n=0
=
eT
Qi (x)dΣ(x)QTi (x) 1−x
−1 Qi (x)dΣ(x)QTi (x)
e = ∞,
where eT = (0, . . . , 0, 1, 0, . . . , 0)T denotes the th unit vector in Rd . We summarize this observation in the following corollary. Corollary 4.1. Assume that the conditions of Theorem 2.1 are satisfied for the block tridiagonal transition matrix P in (1.2) corresponding to a random walk on Cd and that the corresponding spectral measure is supported in the interval [−1, 1]. A state (i, ) ∈ Cd is recurrent if and only if condition (4.2) is satisfied. Moreover, if the random walk is irreducible it is recurrent if and only if the condition eTj
(4.3)
1
dΣ(x) −1 S ej = ∞ 1−x 0
−1
is satisfied for some j ∈ {1, . . . , d} (in this case it is satisfied for any j ∈ {1, . . . , d}). Corollary 4.2. Assume that the conditions of Theorem 2.1 are satisfied for the matrix P in (1.2) corresponding to an irreducible random walk on Cd and that the corresponding spectral measure is supported in the interval [−1, 1]. The random walk is positive recurrent if and only if one of the measures dτ (x) = eT dΣ(x)S0−1 e ( = 1, . . . , d) has a jump at the point 1. In this case all measures dτ (x) ( = 1, . . . , d) have a jump at the point 1. Proof. Let dτ (x) = eT dΣ(x)S0−1 e ; then the probability of returning from state (0, ) to (0, ) in k steps is given by αk =
k eT (P00 )e
=
eT
1 k
x −1
dΣ(x)S0−1 e
1
= −1
xk dτ (x).
The random walk is positive recurrent if and only if α = limk→∞ αk exists and is positive. Considering the sequence α2n it follows by the dominated convergence theorem that this is the case if and only if τ has a jump at x = −1 or x = 1. If τ has no jump at x = 1 we obtain 1 1 2n+1 2n+1 x dτ (x) + x dτ (x) τ (−1) = lim − n→∞
=
−1 2n+1 − lim P00 n→∞
−1−
≤ 0,
and consequently τ has no jump at x = −1. Therefore the random walk is positive recurrent if and only if τ has a jump at x = 1. Remark 4.3. For an irreducible random walk with a random walk measure Σ satisfying S0 = Id the properties of recurrence and positive recurrence are characterized by the diagonal elements of the corresponding random walk measure Σ. 4.2. Canonical moments and random walk measures. In this section we will represent the Stieltjes transform of a random walk matrix measure Σ which is
MATRIX MEASURES AND RANDOM WALKS
135
supported in the interval [−1, 1] in terms of its canonical moments, which were recently introduced by Dette and Studden (2001) in the context of matrix measures. We will use this representation to derive a characterization of recurrence of the process in terms of blocks of the matrix P. Theorem 4.4. The Stieltjes transform of a random walk measure Σ, which is supported in the interval [−1, 1] has the following continued fraction expansions: dΣ(x) 1/2 = lim S0 zId + Id − 2ζ1T − zId + Id − 2ζ2T − 2ζ3T − zId + Id − 2ζ4T n→∞ z−x −1 −1 T T T T −2ζ5T − · · · − zId + Id − 2ζ2n − 2ζ2n+1 4ζ2n ζ2n−1 · −1 −1 1/2 · · · · 4ζ4T ζ3T 4ζ2T ζ1T S0 (z + 1)Id − Id − (z + 1)Id − −1 −1 −1 −1 1/2 T T · · · − (z + 1)Id − 2ζ2n+1 2ζ2n . . . 2ζ2T 2ζ1T S0 ,
1/2
= lim S0 n→∞
where the quantities ζj ∈ Rd×d are defined by ζ0 = 0, ζ1 = U1 , ζj = Vj−1 Uj if j ≥ 2 and the sequences {Uj } and {Vj } are the canonical moments of the random walk measure Σ. The convergence is uniform on compact subsets of C with positive distance from the interval [−1, 1]. In particular, the following representation holds: . / ∞ dΣ(x) 1 1/2 1/2 T −1 T −1 T T = S0 Id + (4.4) (V1 ) . . . (Vl ) Ul . . . U1 S0 . 1−x 2 l=1
Proof. Let P n (t) denote the nth monic orthogonal polynomial with respect to the matrix measure dΣ(t); then it follows from Dette and Studden (2001) that P n (t) can be calculated recursively as T T T T P n+1 (t) = (t + 1)Id − 2ζ2n+1 P n (t) − 4ζ2n (4.5) − 2ζ2n ζ2n−1 P n−1 (t), where P 0 (t) = Id , P −1 (t) = 0, the quantities ζj ∈ Rd×d are defined by ζ0 = 0, ζ1 = U1 , ζj = Vj−1 Uj if j ≥ 2, and the sequences {Uj } and {Vj } are the canonical moments of the random walk measure Σ. Note that Dette and Studden (2001) define the canonical moments for matrix measures on the interval [0, 1], but the canonical moments are invariant with respect to transformations of the measure. More precisely, it can be shown that measures related by an affine transformation t → a + (b − a)t (a, b ∈ R, a < b) have the same canonical moments. The results for the corresponding orthogonal polynomials can also easily be extended to matrix measures on the interval [−1, 1]. The quantities (4.6)
Δ2n := P n , P n = 22n (S0 ζ1 . . . ζ2n )T
are positive definite (see Dette and Studden (2001)) and consequently the polynomials −1/2
Pn (z) = Δ2n P n (z) are orthonormal with respect to the measure dΣ(x). Now a straightforward calculation shows that these polynomials satisfy the recurrence relation (4.7)
tPk (t) = Ak+1 Pk+1 (t) + Bk Pk (t) + ATk Pk−1 (t),
k = 0, 1, . . . ,
136
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
with initial conditions −1/2
P−1 (t) = 0, P0 (t) = S0
(4.8) and coefficients
−1/2
1/2
An+1 = Δ2n Δ2n+2 ,
(4.9)
−1/2
1/2
(4.10)
T T Bn = −Δ2n (Id − 2ζ2n − 2ζ2n+1 )Δ2n ,
(4.11)
T T ζ2n−1 Δ2n−2 ATn = 4Δ2n ζ2n
−1/2
1/2
(note that the matrix Δ2n = 4Δ2n−2 ζ2n−1 ζ2n is symmetric and therefore the two (1) representations in (4.9) and (4.11) for the matrix An are in fact identical). If Pn (z) denotes the first associated orthogonal polynomial corresponding to Pn (z) we obtain from Zygmunt (2002) the representation Fn (z) = (Pn+1 (z))−1 Pn+1 (z) (1)
= S0 {zId − B0 − A1 {zId − B1 − A2 {zId − B2 − · · · − An {zId − Bn }−1 ATn }−1 . . . AT2 }−1 AT1 }−1 .
(4.12)
Now a straightforward application of (4.9)–(4.11) yields 1/2
Fn (z) = S0 {zId + Id − 2ζ1T − {zId + Id − 2ζ2T − 2ζ3T − {zId + Id − 2ζ4T T T T T −2ζ5T − · · · − {zId + Id − 2ζ2n (4.13) − 2ζ2n+1 }−1 4ζ2n ζ2n−1 }−1 · · · · · 4ζ4T ζ3T }−1 4ζ2T ζ1T }−1 S0 , 1/2
and an iterative application of the matrix identity Id + A−1 B = (Id − (B + A)−1 B)−1 and Markov’s theorem (see Duran (1996)) gives dΣ(x) 1/2 = lim S0 (z + 1)Id − Id − (z + 1)Id − n→∞ z−x −1 −1 −1 −1 1/2 T T · · · − (z + 1)Id − 2ζ2n+1 2ζ2n . . . 2ζ2T 2ζ1T S0 (note that this transformation is essentially a contraction). This proves the first part of the theorem. For the second part we put z = 1 and use formula (1.3) in Fair (1971) to obtain dΣ(x) 1 1/2 = lim S0 Id − Id − Id − (4.14) n→∞ 2 1−x −1 −1 −1 −1 1/2 T T · · · − Id − ζ2n+1 ζ2n ... ζ1T S0 n+1 1 1/2 −1 T 1/2 −1 T S0 Xj+1 ζj Xj−1 Xj−1 ζj−1 Xj−2 Xj−1 . . . X1 X2−1 ζ1T S0 , n→∞ 2 j=0
= lim
where X0 = Id , X1 = Id , Xn+1 = Xn − ζnT Xn−1
(n ≥ 1).
137
MATRIX MEASURES AND RANDOM WALKS
Now a straightforward induction argument shows that Xn+1 = VnT . . . V1T and (4.14) reduces to (4.4), which proves the remaining assertion of the theorem. Our next result generalizes the famous characterization of recurrence in an irreducible birth-and-death chain to the matrix case. Theorem 4.5. Assume that the conditions of Theorem 2.1 are satisfied for the block tridiagonal transition matrix of a random walk and that the corresponding spectral measure is supported in the interval [−1, 1]. The state (0, ) is recurrent if and only if 1/2
eT S0
∞
−1 −1 T −1 T Ti+1 Ai Ci Ti−1 Ti−1 A−1 i−1 Ci−1 Ti−2 Ti−1 ·
i=0 −1/2
−1 −1 T · · · · T1 T2−1 A−1 1 C1 T0 T1 A0 T0 S0
e = ∞,
where Ti = Qi (1) (i ∈ N0 ), T−2 = T−1 = Id , and Qi (x) denotes the ith random walk polynomial defined by (1.4). In particular, an irreducible random walk on the grid Cd is recurrent if and only if one of the diagonal elements of the matrix 1/2
S0
∞
−1/2
−1 −1 T −1 −1 −1 T −1 −1 T Ti+1 Ai Ci Ti−1 Ti−1 A−1 i−1 Ci−1 Ti−2 Ti−1 . . . T1 T2 A1 C1 T0 T1 A0 T0 S0
i=0
is infinite (in this case all diagonal elements of this matrix have this property). Proof. A combination of Corollary 4.1 and Theorem 4.4 shows that the state (0, ) is recurrent if and only if ⎡ ⎤ ∞ 1 1/2 −1/2 (4.15) t = eT S0 ⎣Id + (VjT )−1 . . . (VlT )−1 UjT . . . U1T ⎦ S0 e = ∞, 2 j=1 where U1 , U2 , . . . are the canonical moments of the random walk measure Σ and Vj = Id − Uj (j ≥ 1). In the following we express the right-hand side in terms of the blocks of the one-step block tridiagonal transition matrix P corresponding to the random walk. For this consider the recurrence relation (1.4) and define Tn = Qn (1). Note that the polynomials Qn (t) = A0 . . . An−1 Qn (t) are monic and satisfy the recurrence relation −1 Qn+1 (t) = tQn (t) − A0 . . . An−1 Bn A−1 n−1 . . . A0 Qn (t) −1 −A0 . . . An−1 CnT A−1 n−2 . . . A0 Qn−1 (t).
Therefore a comparison with (4.5) yields (4.16)
−1 T T A0 . . . An−1 Bn A−1 n−1 . . . A0 = −Id + 2ζ2n + 2ζ2n+1 , −1 T T A0 . . . An−1 CnT A−1 n−2 . . . A0 = 4ζ2n ζ2n−1 .
Using these representations and the fact that Uk Vk = Vk Uk (see Dette and Studden (2001), Theorem 2.7) it is easy to see that −1 T T T Tn = Qn (1) = 2n A−1 n−1 . . . A0 V2n−1 V2n−2 . . . V1 ,
and it follows from the same reference that these matrices are nonsingular for all n ∈ N0 . Therefore we can define (4.17)
ˆ n (x) = Tn−1 Qn (x), Q
138
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
and it is easy to see that these polynomials satisfy the recurrence relation ˆ n (x) = Aˆn Q ˆn Q ˆ n+1 (x) + B ˆ n (x) + CˆnT Q ˆ n−1 (x), xQ
(4.18) where (4.19)
ˆn = T −1 Bn Tn , Cˆ T = T −1 C T Tn−1 Aˆn = Tn−1 An Tn+1 , B n n n n
ˆn + Cˆ T = Id ). Combining (4.16) with (4.19) we obtain (note that Aˆn + B n T T ˆn Aˆ−1 . . . Aˆ−1 = −Id + 2ζ2n + 2ζ2n+1 , Aˆ0 . . . Aˆn−1 B n−1 0 −1 T ˆ−1 T T ˆ ˆ ˆ ˆ A0 . . . An−1 Cn An−2 . . . A0 = 4ζ2n ζ2n−1 ,
ˆn + Cˆ T = Id ) it follows that and by an induction argument (noting that Aˆn + B n T T ˆ−1 2U2n U2n−1 = Aˆ0 . . . Aˆn−1 CˆnT Aˆ−1 n−1 . . . A0 , T 2V2n+1 V T = Aˆ0 . . . Aˆn−1 Aˆn Aˆ−1 . . . Aˆ−1 . 2n
n−1
0
Finally, we obtain for the left-hand side of (4.15) t=
∞ 1 T 1/2 T −1 T −1 T e S0 (V1 ) . . . (V2j ) U2j . . . U1T 2 j=0
−1/2 T T +(V1T )−1 . . . (V2j+1 )−1 U2j+1 . . . U1T S0 e
1/2
= eT S0
∞
ˆ−1 ˆ T ˆ−1 −1/2 e ˆ T ˆ−1 Aˆ−1 j Cj Aj−1 . . . A1 C1 A0 S0
j=0 ∞ 1/2
= eT S0
−1 −1 T −1 T Tj+1 Aj Cj Tj−1 Tj−1 A−1 j−1 Cj−1 Tj−2 Tj−1 ·
j=0 −1/2
−1 −1 T · · · · T1 T2−1 A−1 1 C1 T0 T1 A0 T0 S0
e
with Ti = Qi (1) (i ∈ N0 ), which proves the assertion of the theorem. Remark 4.6. It is interesting to note that the condition in Theorem 4.5 simplifies substantially if all the matrices Ti , Ai , Ci are commuting. In this case an irreducible random walk is recurrent if and only if 1/2
eT S0
∞
−1/2
−1 −1 Ti+1 Ti (C1 . . . Ci )T (A0 . . . Ai )−1 S0
e = ∞
i=0
for some ∈ {1, . . . , d}. Example 4.7. Consider the random walk on the tree introduced in section 3.5. By Corollary 4.1 the state (0, 1) (which corresponds to the origin) is recurrent if and only if
−1 2√pq dΣ(x) a(x) T dΣ(x) dx, ∞ = e1 e1 = √ 1−x 2 pq 1 − x where the function a is defined in section 3.5 and we have used the fact that dΣ(x) = S0 = (R0T R0 )−1 (see Remark 2.2). √ Because the support of the spectral measure is √ given by the interval [− 4pq, − 4pq] it follows that the condition p = q = 12 is necessary for the recurrence of the random walk. Now a straightforward calculation shows that the state (0, 1) (i.e., the center of the graph) is recurrent if and only if the d d condition 2 i=2 d2i = i=2 di is satisfied (in all other cases the integral is finite).
MATRIX MEASURES AND RANDOM WALKS
139
5. Applications. In this section we briefly discuss some applications of our approach. 5.1. Representations of the invariant measure. Note that an irreducible quasi-birth-and-death process always has an invariant measure with a matrix product form (see Latouche, Pearce, and Taylor (1998)). In particular, if the process is positive recurrent, the invariant measure coincides with the stationary distribution x = (xT0 , xT1 , . . . ), which can be represented as xTk = xT0
(5.1)
k−1 &
˜, R
=0
˜ }∞ is the minimal nonnegative solution of the equations where the set {R =0 T Rk = Ak + Rk Bk+1 + Rk Rk+1 Ck+2
(5.2)
(k ≥ 0)
and x0 satisfies ˜ 0 C T ) = xT , xT0 (B0 + R 1 0
(5.3)
normalized so that xT e = 1, where e denotes a vector with all entries equal to one. We will now investigate these properties from the viewpoint derived in this paper and suppose that the assumptions of Theorem 2.5 are satisfied for an irreducible aperiodic Markov chain on the grid Cd . Note that the limits Li = lim Piin n→∞
exist and do not depend on i. By the Theorem of dominated convergence and (2.16) it follows that 0 1 Li = lim Qi (1)Σ(1)QTi (1) + (−1)n Qi (−1)Σ(−1)QTi (−1) Zi−1 (5.4) , n→∞
where Zi = Qi (x)dΣ(x)QTi (x) and Σ(1) and Σ(−1) denote the mass of the random walk matrix measure at the points 1 and −1, respectively. Considering the subsequence of odd positive integers it follows that Σ has no mass at −1 and (5.4) reduces to (5.5)
Li = lim Piin = Qi (1)Σ(1)QTi (1)Zi−1 . n→∞
Note that the left-hand side of this equation does not depend on i and therefore (5.5) provides several representations for the same quantity (by using different values of i). For example, if we put i = 0 and note that the rank of the matrices Li is 1 we obtain from the identity L0 = Σ(1)Z0−1 that the rank of the weight Σ(1) is 1. Moreover, if the random walk is positive recurrent, the stationary distribution is given by x = (xT0 , xT1 , . . . ) = eT0 (L0 , L1 , . . . ), and it follows that xTk = eT0 Lk = eT0 Σ(1)QTk (1)Zk−1
(k ≥ 0),
140
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
which is an alternative representation for the stationary distribution. In particular, we have for the vector x0 in (5.1) xT0 = eT0 Σ(1)S0−1 . Moreover, if the matrices Qk (1) are nonsingular, we obtain by straightforward calculation that the representation (5.1) holds with (5.6)
˜ j = Zj (QT (1))−1 QT (1)Z −1 . R j j+1 j+1
Using the relations (2.4) it follows by straightforward algebra that the sequence ˜ j }j∈N is in fact a solution of the system (5.2) and (5.3), which yield to the station{R 0 ary distribution. We finally note that the proof of Theorem 2.1 shows that the matrix Zj−1 = RjT Rj can be expressed in terms of the blocks Aj , Cj and the matrix S0 . 5.2. A necessary condition for positive recurrence. As a second application we use the identity (5.5) for two values i, k and obtain Qi (1)Σ(1)QTi (1) = Qk (1)Σ(1)QTi (1) = Li which reduces for i = 0 to (5.7)
(Qi (1) − Qk (1))Σ(1) = 0
(i, k ≥ 0).
Recall that by Corollary 4.2 the irreducible random walk is positive recurrent if and only if all measures eT dΣ(x)S0−1 e have a jump at the point 1. In this case it follows from (5.7) that all matrices Qi (1) − Qk (1) are singular (otherwise Σ(1) would be the null matrix). Consequently we obtain the following result. Theorem 5.1. Assume that the block tridiagonal transition matrix of an irreducible random walk satisfies the assumptions of Theorem 2.5. If the process is positive recurrent, then the matrices Qi (1) − Qk (1) are singular for all i, k ∈ N0 . Example 5.2. Consider the random walk on the tree presented in section 3.5. In Example 4.7 it is demonstrated that the random walk is recurrent if and only if p = q = 12 and d i=2
di = 2
d
d2i ,
i=2
which will be assumed in the following discussion. A straightforward calculation shows that Q0 (1) = Id , ⎛ ⎞ 2 −2d2 −2d3 . . . −2dd ⎜ −1 2 0 ... 0 ⎟ ⎜ ⎟ ⎜ −1 0 2 . . . 0 ⎟ Q1 (1) = ⎜ ⎟, ⎜ .. .. .. .. ⎟ . . ⎝ . . . . . ⎠ −1 0 0 ... 2 and consequently we obtain |Q1 (1) − Q0 (1)| = |Id − 2B0 | = 1 − 2d2 − · · · − 2dd = 2d1 − 1.
MATRIX MEASURES AND RANDOM WALKS
141
Therefore, if the random walk would be positive recurrent it follows that d1 = 12 . Because the tree corresponding to the random walk is symmetric we conclude that the role of d1 and d2 can be interchanged. Consequently the random walk can only be positive recurrent if two of the probabilities dj are equal to 1/2 and the others vanish. However, this corresponds to the symmetric random walk on Z, which is not positive recurrent. In other the random walk considered in section 3.5 is recurrent if d words: d p = q = 12 and i=2 di = 2 i=2 d2i but never positive recurrent. Acknowledgments. This work was done while H. Dette was visiting the Department of Statistics, Purdue University, and this author would like to thank the Department for its hospitality. The authors would like to thank two unknown referees for their constructive comments on an earlier version of this paper and Isolde Gottschlich, who typed this paper with considerable technical expertise. REFERENCES A. Basilevsky (1983), Applied Matrix Algebra in the Statistical Sciences, North–Holland, Amsterdam. N. G. Bean, P. K. Pollett, and P. G. Taylor (2000), Quasi-stationary distributions for leveldependent quasi-birth-and-death processes, Comm. Statist. Stochastic Models, 16, pp. 511–541. Ju. M. Berezanskii (1968), Expansions in Eigenfunctions of Selfadjoint Operators, Trans. Math. Monogr. 17, AMS, Providence, RI. L. Bright and P. G. Taylor (1995), Calculating the equilibrium distribution in level dependent quasi-birth-and-death processes, Comm. Statist. Stochastic Models, 11, pp. 497–525. T. Dayar and F. Quessette (2002), Quasi-birth-and-death processes with level-geometric distribution, SIAM J. Matrix Anal. Appl., 24, pp. 281–291. H. Dette (1996), On the generating functions of a random walk on the nonnegative integers, J. Appl. Probab., 33, pp. 1033–1052. H. Dette and W. J. Studden (1997), The Theory of Canonical Moments with Applications in Statistics, Probability, and Analysis, Wiley, New York. H. Dette and W. J. Studden (2001), Matrix measures, moment spaces, and Favard’s theorem for the interval [0, 1] and [0, ∞), Linear Algebra Appl., 345, pp. 163–193. A. J. Duran (1995), On orthogonal polynomials with respect to a positive definite matrix of measures, Canad. J. Math., 47, pp. 88–112. A. J. Duran (1996), Markov’s theorem for orthogonal matrix polynomials, Canad. J. Math., 48, pp. 1180–1195. A. J. Duran (1999), Ratio asymptotics for orthogonal matrix polynomials, J. Approx. Theory, 100, pp. 304–344. A. J. Duran and W. Van Assche (1995), Orthogonal matrix polynomials and higher-order recurrence relations, Linear Algebra Appl., 219, pp. 261–280. W. Fair (1971), Noncommutative continued fractions, SIAM J. Math. Anal., 2, pp. 226–232. W. Feller (1950), An Introduction to Probability Theory and Its Applications, Vol. I, John Wiley & Sons, New York. D. P. Gaver, P. A. Jacobs, and G. Latouche (1984), Finite birth-and-death models in randomly changing environments, Adv. in Appl. Probab., 16, pp. 715–731. B. Hajek (1982), Birth-and-death processes on the integers with phases and general boundaries, J. Appl. Probab., 19, pp. 488–499. P. R. Halmos and V. S. Sunder (1978), Bounded Integral Operators on L2 -Spaces, Springer-Verlag, New York. S. Karlin and J. McGregor (1959), Random walks, Illionis J. Math., 3, pp. 66–81. S. Karlin and H. M. Taylor (1975), A First Course in Stochastic Processes, Academic Press, New York. G. Latouche, C. E. M. Pearce, and P. G. Taylor (1998), Invariant measures for quasi-birthand-death processes, Comm. Statist. Stochastic Models, 14, pp. 443–460. G. Latouche and V. Ramaswami (1999), Introduction to Matrix Analytic Methods in Stochastic Modeling, ASA-SIAM Ser. Stat. Appl. Probab. 5, SIAM, Philadelphia, Chapter 12. Q. L. Li and J. Cao (2004), Two types of RG-factorizations of quasi-birth-and-death processes and their applications to stochastic integral functionals, Stoch. Models, 20, pp. 299–340.
142
H. DETTE, B. REUTHER, W. STUDDEN, AND M. ZYGMUNT
´n and G. Sansigre (1993), On a class of matrix orthogonal polynomials on the real F. Marcella line, Linear Algebra Appl., 181, pp. 97–109. I. Marek (2003), Quasi-birth-and-death processes, level geometric distributions. An aggregation/disaggregation approach, J. Comput. Appl. Math., 152, pp. 277–288. M. F. Neuts (1981), Matrix Geometric Solutions in Stochastic Models. An Algorithmic Approach. The John Hopkins University Press, Baltimore. M. F. Neuts (1989), Structured Stochastic Matrices of M/G/1 Type and Their Applications, Marcel Dekker, New York. A. Ost (2001), Performance of Communication Systems: A Model-Based Approach with MatrixGeometric Methods, Springer-Verlag, Berlin. V. Ramaswami and P. G. Taylor (1996), Some properties of the rate operators in level dependent quasi-birth-and-death processes with a countable number of phases, Comm. Statist. Stochastic Models, 12, pp. 143–164. L. Rodman (1990), Orthogonal matrix polynomials, in Orthogonal Polynomials, NATO Adv. Sci. Inst. Ser. C Math. Phys. Sci. 294, P. Nevai, ed., Kluwer, Dordrecht, The Netherlands, pp. 345– 362. A. Sinap and W. Van Assche (1996), Orthogonal matrix polynomials and applications, J. Comput. Appl. Math., 66, pp. 27–52. E. A. van Doorn and P. Schrijner (1993), Random walk polynomials and random walk measures, J. Comput. Appl. Math., 49, pp. 289–296. E. A. van Doorn and P. Schrijner (1995), Geometric ergodicity and quasi-stationarity in discretetime birth-death processes, J. Austral. Math. Soc. Ser. B, 37, pp. 121–144. T. A. Whitehurst (1982), An application of orthogonal polynomials to random walks, Pacific J. Math., 99, pp. 205–213. W. Woess (1985), Random walks and periodic continued fractions, Adv. in Appl. Probab., 17, pp. 67–84. M. J. Zygmunt (2002), Matrix Chebyshev polynomials and continued fractions, Linear Algebra Appl., 340, pp. 155–168.
