General discrete random walk with variable absorbing probabilities Theo van Uem Amsterdam School of Technology, Weesperzijde 190, 1097 DZ Amsterdam, The Netherlands. Email:
[email protected] Abstract We obtain expected number of arrivals, probability of arrival, absorption probabilities and expected time before absorption for a general discrete random walk with variable absorbing probabilities on a finite interval using Fibonacci numbers. Keywords: Random walk, difference equations, Fibonacci numbers 2000 Mathematics Subject Classification: Primary 60G50, Secondary 60J05
1. Introduction A general discrete random walk with variable absorbing probabilities on the integers is described in every state i by the one-step forward probability pi , the one-step backward probability qi , the probability to stay for a moment in the same position ri and si is the probability of immediate absorption in state i where p i + q i + ri + s i = 1 . For this type of random walk we use the notation [ p i qi ri si ] . Feller (1968) analyzed [ p i q i ri ] random walk and gives necessary and sufficient conditions for the existence of an invariant probability distribution (p. 396) and for a persistent random walk (p. 402). El-Shehawey (1994) studies [ p i qi ri ] random walk on the non-negative integers and relates his model to an example of carcinogenesis. Dette (1996) uses Stieltjes transforms to get explicit representations for the generating functions of the n-step transition and the first return probabilities in a [ p i q i ri ] model for the non-negative integers, including a permanent absorbing state i * which can only be reached from state i with probability 1 − ( pi + qi + ri ) . Rudolph (1999) analyzed [ p i qi ri ] general discrete random walk on the integers with two absorbing boundaries. He determines absorption probabilities and absorption time using a fundamental matrix. In this paper we have the additional freedom of absorption in any point at any time with variable probabilities. In section 2 we solve a set of difference equations which is related to the expected number of arrivals and expected time before absorption. Fibonacci numbers play an important role in this setting. In section 3 we study [ pi qi ri si ] random walk on [0,N].
2. Solution of related difference equations 2.1 Solution of xi +1 = λi xi + µ i −1 xi −1 First we will study a special set of difference equations, which we need to investigate our random walk. We are interested in the solution of : x i +1 = λi xi + µ i −1 xi −1 (i = 1,2,.....N ); x1 = λ 0 ; x 0 = 1
We use the Fibonacci sequence:
f 0 = f1 = 1, f n +1 = f n + f n −1 (n = 1,2.....)
1
(2.1)
2
Theo van Uem
and relate the solution of (2.1) to matrices: λ0 1 λ0 1 , F3 = λ1 µ 0 1 , ….. µ0 λ 2 λ 2 µ1 where Fn +1 (n=2,3,…) with elements τ ij (i=1,2,…,n+1; j=1,2, .. f n +1 ) is defined by:
λ0 F0 = (1) , F1 = (λ0 ) , F2 = λ1
τ ij
1
1 . . . n-1 n n+1
…
fn
f n +1
…
Fn
Fn −1
1
λn
f n +1
λn
…
µ n −1
… …
1
µ n −1
For F6 we get:
τ ij i\j 1 2 3 4 5 6
f1 1
λ0 λ1 λ2 λ3 λ4 λ5
f2
f3
f4
2 1
3
4
λ0
µ0 λ2 λ3 λ4 λ5
1
λ0 λ1
µ1 λ3 λ4 λ5
f5
5 1
6
λ0 λ1 λ2
µ0
1
1
µ2 λ4 λ5
µ2 λ4 λ5
1
µ3 λ5
f6
7 1
8
9
λ0
µ0 λ2
1
1
1
λ0 λ1 λ2 λ3
µ3 λ5
µ1 µ3 λ5
10 1
11
12
13 1
λ0
µ0 λ2 λ3
1
λ0 λ1
µ1 λ3
1
µ2
µ2
1
1
1
1
1
µ4
µ4
µ4
µ4
µ4
µ0 1
We recognize F1 , F2 , F3 , F4 in the same matrix:
τ ij i\j 1 2 3 4 5 6
f1 1
λ0 λ1 λ2 λ3 λ4 λ5
f2
f3
2 1
3
4
λ0
µ0 λ2 λ3 λ4 λ5
1
λ0 λ1
µ1 λ3 λ4 λ5
f5
f4 5 1
6
λ0 λ1 λ2
µ0
1
1
µ2 λ4 λ5
µ2 λ4 λ5
1
µ3 λ5
f6
7 1
8
9
λ0
µ0 λ2
1
1
1
λ0 λ1 λ2 λ3
µ3 λ5
µ1 µ3 λ5
10 1
11
12
13 1
λ0
µ0 λ2 λ3
1
λ0 λ1
µ1 λ3
1
µ2
µ2
1
1
1
1
1
µ4
µ4
µ4
µ4
µ4
µ0 1
fn n
We define:
Fn* = ∑∏ τ kj j =1 k =1
From the definition of Fn +1 we get directly: so Fn* is a solution of (2.1). The definition of Fn +1 also gives: τ i , f n + j = τ i , j
Fn*+1 = λ n Fn* + µ n −1 Fn*−1 (1 ≤ j ≤ f n −1 )
(2.2)
General discrete random walk with variable absorbing probabilities
3
Theorem 1 The solution of the linear system (2.1) is: fi
i
xi = ∑∏τ kj
(i = 1,2....., N + 1)
(2.3)
j =1 k =1
where: If j ≤ f i +1 : λi −1 ( j = 1,2........ f i −1 ) τ ij = µ i −2 ( j = f i −1 + 1,......... f i ) ( j = f i + 1,.......... f i +1 ) 1
(2.4)
If j > f i +1 : ∃ n∈Ν : 1 + f n ≤ j ≤ f n +1
Let j m = j − ( f n + f n − 2 + ......... f n − 2 m )
n − 1 m = 0,1,....., , where [ ] is the entier function. 2
and k = min{m ∈ Ν | j m ≤ f i +1 }
τ ij = τ ijk
then:
λi−1 ( j k = 1,2........ f i −1 ) = µ i −2 ( j k = f i −1 + 1,......... f i ) ( j k = f i + 1,.......... f i +1 ) 1
Proof:
Using (2.3) with i=1 and x1 = λ0 gives: τ 11 = λ0 , as desired. When substituting (2.3) in (2.1) we find for j ≤ f i +1 : f i +1 i +1
∑∏τ kj j =1 k =1
fi
fi −1 i −1
i
= λi ∑∏τ kj + µ i −1 ∑∏τ kj , so: j =1 k =1
j =1 k =1
f i +1
fi
∑τ 1 jτ 2 j ........τ i +1, j = ∑ τ 1 jτ 2 j ........τ i , j λi +
fi −1
∑τ 1 jτ 2 j ........τ i −1, j .1.µ i −1
j =1
j =1
j =1
We now use (2.2) : τ k , j + fi = τ k , j ( j ≤ f i −1 ) : f i +1
fi
fi +1
j =1
j =1
j = f i +1
∑τ 1 jτ 2 j ........τ i +1, j = ∑ τ 1 jτ 2 j ........τ i, j λi + ∑τ 1 jτ 2 j ........τ i −1, j .1.µ i −1 We see that: τ i +1, j = λi
τ i, j = 1
; τ i +1, j = µ i −1
( j ≤ f i ) and: ( f i + 1 ≤ j ≤ f i +1 )
which gives the desired result (2.4). If j > f i +1 then: ∃ n∈Ν : 1 + f n ≤ j ≤ f n +1 1 ≤ j 0 = j − f n ≤ f n −1 j1 = j 0 − f n − 2 ≤ f n −3 ………. j k = j k −1 − f n − 2 k ≤ f n − 2 k −1 and (use (2.2)): τ i, j = τ i, j0 = τ i, j1 = ... = τ i, jk . This completes the proof of Theorem 1.
Remark 1: Because of j k −1 ≤ f n − 2 k +1 and j k −1 > f i +1 we have : k
0; p −1 > 0 We have: x n = p n −1 x n −1 + q n +1 x n +1 + rn x n + δ (n, i0 ) (0 ≤ n ≤ N )
(3.1) (3.2) (3.3)
which generates the following forward and backward equations: 1 − ri p x i − i −1 xi −1 = λi x i + µ i −1 xi −1 (i = i0 + 1, i0 + 2,....., N ) ) xi +1 = qi +1 q i +1 1 − ri q xi − i +1 xi +1 = ρ −i xi + θ − (i +1) x i +1 (i = i0 − 1, i0 − 2,.....,0) xi −1 = pi −1 p i −1
(3.4) (3.5)
The solution of (3.4) and (3.5) is (we use the notation of section 2.2): (i +1)
x i0 + k +1 = xi0 +1 Ak 0
(1−i0 )
x i0 −( k +1) = xi0 −1 Ak
(i + 2)
(λ , µ ) + µ i0 xi0 Ak −01 (λ , µ ) ( 2 −i )
( ρ , θ ) + θ −i0 xi0 Ak −1 0 ( ρ , θ )
Using (3.1),(3.2),(3.6) and (3.7) we get:
(k = 1,2.....N − i0 )
(3.6)
(k = 1,2.....i0 )
(3.7)
6
Theo van Uem
(i +1)
(i + 2 ) (λ , µ ) = 0 0 −1
x N +1 = xi0 +1 AN0−i (λ , µ ) + µ i0 xi0 AN0−i 0
(1−i0 )
x −1 = xi0 −1 Ai
0
( 2−i0 ) (ρ ,θ 0 −1
( ρ , θ ) + θ −i0 xi0 Ai
(3.8)
) =0
(3.9)
We also have: (1 − ri0 ) xi0 = 1 + p i0 −1 xi0 −1 + qi0 +1 xi0 +1
(3.10)
Using (3.8.),(3.9) and (3.10), we get the expected number of arrivals in the starting point i0 : A ( 2−i0 ) ( ρ , θ ) A (i0 + 2) (λ , µ ) i0 −1 N −i −1 + q i0 +1 µ i0 (i +01) xi0 = 1 − ri0 + pi0 −1θ −i0 (1−i ) Ai 0 ( ρ , θ ) AN0−i (λ , µ ) 0 0
−1
(3.11)
For k = 0,1,..N − i0 we find the expected number of arrivals in i0 + k + 1 (use (3.4)):
{
(i + 2)
xi0 + k +1 = µ i0 xi0 Ak −01
(i +1)
}{A
(i + 2 ) (i +1) (λ , µ ) Ak 0 (λ , µ ) 0 −1
(λ , µ ) AN0−i (λ , µ ) − AN0−i 0
}
(i0 +1) (λ , µ ) −1 N −i0
and for k = 0,1,...i0 we find the expected number of arrivals in i0 − ( k + 1) (use (3.5)):
{
( 2 −i )
(1−i0 )
xi0 −( k +1) = θ −i0 xi0 Ak −1 0 ( ρ , θ ) Ai
0
}{A
( 2−i0 ) (1−i ) ( ρ , θ ) Ak 0 ( ρ , θ ) 0 −1
( ρ , θ ) − Ai
}
(1−i0 ) ( ρ , θ ) −1 i0
It is now easy to obtain the probability to visit j when starting in i: If ij:
[
f ij = θ −i xi x −j 1 Ai(−2−j −i )2 ( ρ , θ ) Ai(1−i ) ( ρ , θ ) − Ai(−21−i ) ( ρ , θ ) Ai(−1−ji−)1 ( ρ , θ ) Ai(1−i ) ( ρ , θ )
]
−1
−1
If i=j: A ( 2 −i ) ( ρ , θ ) (i + 2 ) − q µ AN −i −1 (λ , µ ) f ii = 1 − xii−1 = ri − p i −1θ −i i(−11−i ) i +1 i ( i + 1 ) A A i (ρ ,θ ) N −i (λ , µ ) Next we handle the special case where the starting point is the left border of our finite interval: i0 = 0 ( i0 = N proceeds along the same lines). Instead of two artificial states we now use one artificial state (N+1) with q N +1 > 0; x N +1 = 0; We now have: 1 − ri p xi − i −1 xi −1 = λi xi + µ i −1 xi −1 (i = 1,2,....., N ) x i +1 = q i +1 q i +1
1 − r0 i = 0 : x1 = q1
1 1 x 0 − = λ0 x0 − q1 q1
with solution: xi = x0 Ai(0) (λ , µ ) −
Using the artificial state we get:
1 (1) A (λ , µ ) q1 i −1
(i = 1,2.....N + 1)
General discrete random walk with variable absorbing probabilities
x N +1 = x 0 AN( 0+)1 (λ , µ ) −
7
1 (1) A (λ , µ ) = 0 q1 N
So: x0 = xi =
AN(1) (λ , µ ) q1 AN( 0+)1 (λ , µ )
AN(1) (λ , µ )
(3.12)
q1 AN(0+)1 (λ , µ )
Ai( 0) (λ , µ ) −
1 (1) A (λ , µ ) q1 i −1
(i = 0,1,......., N )
Remark 2: (3.12) can also be derived from (3.11): use Theorem 2 with i0 = 0, p −1 = 0 . 3.2 Expected time before absorption. We define: mk = expected time before absorption when starting in k (k=0,1,2,...N). We have: mi = p i ( mi +1 + 1) + qi (mi −1 + 1) + ri (mi + 1) + s i .0 (0 ≤ i ≤ N ) So: (1 − ri )mi = pi mi +1 + qi mi −1 + (1 − s i ) (0 ≤ i ≤ N )
(3.13)
First we handle with 0 < i0 < N , where i0 is the starting point of our walk. The forward and backward equations are: (1 − ri ) q 1 − si mi − i mi −1 − = ω i mi + ϕ i −1mi −1 + α i (i = i0 + 1, i0 + 2,....., N ) pi pi pi (1 − ri ) p 1 − si = mi − i mi +1 − = η −i xi + ζ −(i +1) xi +1 + β −i (i = i0 − 1, i0 − 2,.....,0) qi qi qi
mi +1 = mi −1
Using the notation of section 2.2 we have: (i +1)
mi0 + k +1 = mi0 +1 Ak 0
(1−i0 )
mi0 −( k +1) = mi0 −1 Ak
k
(ω , ϕ ) + ϕ i0 mi0 Ak −01 (ω , ϕ ) + ∑ Ak −0 n (i + 2)
( i + n +1)
n =1
k
( 2 −i )
(η , ζ ) + ζ −i0 mi0 Ak −1 0 (η , ζ ) +
(ω , ϕ )α i0 + n (k = 1,...N − i0 ) (3.14)
∑ Ak − n
( n +1−i0 )
n =1
(η , ζ )β n −i0 ( k = 1,2.....i0 ) (3.15)
Taking i=0 and i=N in (3.13) we see that m−1 and m N +1 are part of (3.13) and therefore: (i +1)
(i + 2 ) (ω , ϕ ) + 0 −1
m N +1 = mi0 +1 AN0−i (ω , ϕ ) + ϕ i0 mi0 AN0−i 0
(1−i0 )
m −1 = mi0 −1 Ai
0
( 2 −i0 ) (η , ζ 0 −1
(η , ζ ) + ζ −i0 mi0 Ai
)+
N −i0
∑
n =1 i0
(i + n +1) (ω , ϕ )α i0 + n 0 −n
AN0−i
∑ Ai −n
n =1
( n +1−i0 ) 0
=0
(η , ζ )β n −i0 = 0
(3.16) (3.17)
We also have: (1 − ri0 )mi0 = pi0 mi0 +1 + qi0 mi0 −1 + (1 − si0 )
(3.18)
Using (3.16), (3.17) and (3.18) we obtain the expected time before absorption in the starting point:
8
Theo van Uem
( i +1)
(1−i0 )
(1 − si0 ) AN0−i Ai 0
mi0 =
0
(1−i0 )
N −i0
− pi0 Ai
0
(i +1)
(1−i0 )
(1 − ri0 ) AN0−i Ai 0
0
∑
( i + n +1) α i0 + n 0 −n
AN0−i
n =1
(1−i0 )
+ pi0 ϕ i0 Ai
0
(i + 2) 0 −1
AN0−i
(i +1)
− q i0 AN0−i
+
0
i0
∑ Ai0 −n
( n +1−i0 )
n =1 ( i +1) ( 2 −i ) qi0 ζ −i0 AN0−i Ai −1 0 0 0
β n−i0 (3.19)
mi0 +1 and mi0 −1 can easily be derived from (3.16) and (3.17). The general solution is now given by (3.14) and (3.15). Next we handle with i0 = 0 ( i0 = N proceeds along the same lines). mi +1 =
(1 − ri ) q 1 − si mi − i mi −1 − = ω i mi + ϕ i −1 mi −1 + α i pi pi pi m1 = ω 0 m0 + α 0
Solution: i
mi = m0 Ai( 0) + ∑ Ai(−kk) α k −1
(i = 0,1,....., N )
k =1
m N +1 is part of equation (3.13), so we have: m N +1 = m0 AN( 0+)1 +
N +1
∑ AN( k+)1−k α k −1 = 0
k =1
Resulting in: − Ai( 0)
N +1
∑ AN( k+)1−k α k −1
k =1
mi =
AN(0+)1
i
+ ∑ Ai(−kk) α k −1 (i = 0,1,....N )
(3.20)
k =1
Example 1: [ pi qi ri si ] random walk on [0,3] and start in 0. We are interested in probability of absorption and expected time before absorption in the starting point 0. From (3.12) and using Theorem 1 (or, easier, the matrices F3 and F4 in section 2.1) we get: x0 =
A3(1) (λ , µ ) q1 A4( 0) (λ , µ )
=
λ1λ 2 λ3 + µ1λ3 + λ1 µ 2 = q1 [λ 0 λ1λ 2 λ3 + µ 0 λ 2 λ3 + λ 0 µ1λ3 + λ 0 λ1 µ 2 + µ 0 µ 2 ]
{(1 − r1 )(1 − r2 )(1 − r3 ) − p1 q 2 (1 − r3 ) − (1 − r1 ) p 2 q3 } * {(1 − r0 )(1 − r1 )(1 − r2 )(1 − r3 ) − p 0 q1 (1 − r2 )(1 − r3 ) − (1 − r0 ) p1q 2 (1 − r3 ) − (1 − r0 )(1 − r1 ) p 2 q 3 + p 0 q1 p 2 q3 }−1
P(absorption in 0)= s 0 x 0 (3.20) gives: 4
− ∑ A4( k−)k α k −1 m0 =
k =1
A4( 0)
1 − s0 1 − s3 1 − s1 1− s2 )(ω1ω 2 ω 3 + ϕ1ω 3 + ω1ϕ 2 ) + ( )(ω 2 ω 3 + ϕ 2 ) + ( )ω 3 + ( ) p0 p1 p2 p3 = = ω 0 ω1ω 2 ω 3 + ϕ 0 ω 2 ω 3 + ω 0ϕ1ω 3 + ω 0 ω1ϕ 2 + ϕ 0ϕ 2 (
{(1 − s0 )[(1 − r1 )(1 − r2 )(1 − r3 ) − p1q2 (1 − r3 ) − (1 − r1 ) p2 q3 ] + (1 − s1 )[ p0 (1 − r2 )(1 − r3 ) − p0 p2 q3 ] + p0 p1 (1 − s2 )(1 − r3 ) + p0 p1 p2 (1 − s3 )} * {(1 − r0 )(1 − r1 )(1 − r2 )(1 − r3 ) − p 0 q1 (1 − r2 )(1 − r3 ) − (1 − r0 ) p1q 2 (1 − r3 ) − (1 − r0 )(1 − r1 ) p 2 q 3 + p 0 q1 p 2 q3 }−1
General discrete random walk with variable absorbing probabilities
9
Example 2: [pqs] random walk on [0,N]. We consider a special [ p i qi ri s i ] random walk on [0,N]: p i = p, q i = q, ri = 0, si = s (i = 1,2,..., N − 1); p + q + s = 1 p 0 = p, s 0 = 1 − p, q N = q, s N = 1 − q. We start in 0. Using (3.12) and Theorem 2, we find after some calculations:
N + 1 ( − pq ) k k x 0 = [( N k+=2)0/ 2] N + 2 ∑ k (− pq) k k =0 [( N +1) / 2]
∑
(3.21)
Example 3: Simple random walk on [0,N] with two partial absorbing barriers. We consider a special [ p i qi ri s i ] random walk on [0,N]; we start in 0 and we choose: p i = p, q i = q, ri = 0, s i = 0 (i = 1,2,..., N − 1); p + q = 1 p 0 = p, s 0 = 1 − p, q N = q, s N = 1 − q. The probability of absorption in the barrier 0 is now given by qx 0 , where x0 is given by (3.21) , now with p + q = 1 .
References DETTE H. (1996) On the generating functions of a random walk on the non-negative integers, J. Appl. Prob. 33, 1033-1052 EL-SHEHAWEY M.A. (1994) On the frequency count for a random walk with absorbing boundaries: a carcinogenesis example, J. Phys. A: Math. Gen., 27, 7035-7046. FELLER W. (1968) An Introduction to probability theory and its applications, (third edition) Vol. 1 John Wiley, New York . RUDOLPH G. (1999) The fundamental matrix of the general random walk with absorbing boundaries. Technical Report of the Collaborative Research Center “Computational Intelligence” CI75, University of Dortmund.
