CHUN-HUA GUO. Key words. matrix equations, minimal nonnegative solution, Markov chains, cyclic reduction, iterative methods, convergence rate

Transcription

1 CONVERGENCE ANALYSIS OF THE LATOUCHE-RAMASWAMI ALGORITHM FOR NULL RECURRENT QUASI-BIRTH-DEATH PROCESSES CHUN-HUA GUO Abstract The minimal nonnegative solution G of the matrix equation G = A 0 + A 1 G + A 2 G 2, where the matrices A i i = 0, 1, 2 are nonnegative and A 0 +A 1 +A 2 is stochastic, plays an important role in the study of quasi-birth-death processes QBDs The Latouche-Ramaswami algorithm is a highly efficient algorithm for finding the matrix G The convergence of the algorithm has been shown to be quadratic for positive recurrent QBDs and for transient QBDs In this paper, we show that the convergence of the algorithm is linear with rate 1/2 for null recurrent QBDs under mild assumptions This new result explains the experimental observation that the convergence of the algorithm is still quite fast for nearly null recurrent QBDs Key words matrix equations, minimal nonnegative solution, Markov chains, cyclic reduction, iterative methods, convergence rate AMS subject classifications 15A24, 15A51, 60J10, 60K25, 65U05 1 Introduction A discrete-time quasi-birth-death process QBD is a Markov chain with state space {i, j i 0, 1 j m}, which has a transition probability matrix of the form P = B 0 B A 0 A 1 A A 0 A 1 A A 0 A 1 where B 0, B 1, A 0, A 1, and A 2 are m m nonnegative matrices such that P is stochastic In particular, A 0 + A 1 + A 2 e = e, where e is the column vector with all components equal to one The matrix P is also assumed to be irreducible Thus, A 0 0 and A 2 0 The matrix equation, 11 G = A 0 + A 1 G + A 2 G 2 plays an important role in the study of the QBD see [12] and [16] It is known that 11 has at least one solution in the set {G 0 Ge e} ie, the set of substochastic matrices The desired solution G is the minimal nonnegative solution We assume that A = A 0 + A 1 + A 2 is irreducible Then, by the Perron-Frobenius Theorem see [17], there exists a unique vector α > 0 with α T e = 1 and α T A = α T The vector α is called the stationary probability vector of A By Theorem 723 in [12], the QBD is null recurrent if α T A 0 e = α T A 2 e; positive recurrent if α T A 0 e > α T A 2 e; and transient if α T A 0 e < α T A 2 e For our purpose, we may use this criterion as an alternative definition for the three classes of QBDs This work was supported in part by a grant from the Natural Sciences and Engineering Research Council of Canada Department of Mathematics and Statistics, University of Regina, Regina, SK S4S 0A2, Canada chguo@mathureginaca 1

2 2 CHUN-HUA GUO The minimal nonnegative solution of 11 can be found by any of the following three fixed-point iterations see [3], [5], [9], [10], [14], [15], [18]: G n+1 = A 0 + A 1 G n + A 2 G 2 n, G 0 = 0, G n+1 = I A 1 1 A 0 + A 2 G 2 n, G 0 = 0, G n+1 = I A 1 A 2 G n 1 A 0, G 0 = 0 Among the three iterations, iteration 14 has the fastest rate of convergence An inversion free version of 14 has also been proposed in [1] and analysed in [1] and [5] These four iterations are adequate for most situations However, the convergence of all four iterations is sublinear when the QBD is null recurrent see [5] The convergence of these methods is also extremely slow if the QBD is nearly null recurrent The algorithm proposed by Latouche and Ramaswami [11] is a little more complicated However, it works very well even for nearly null recurrent QBDs The algorithm is as follows: Algorithm 11 Set For k = 0, 1,, compute H 0 = I A 1 1 A 2 ; L 0 = I A 1 1 A 0 ; G 0 = L 0 ; T 0 = H 0 U k = H k L k + L k H k ; H k+1 = I U k 1 H 2 k; L k+1 = I U k 1 L 2 k; G k+1 = G k + T k L k+1 ; T k+1 = T k H k+1 It is shown in [11] that the matrices H k and L k are well defined and nonnegative and that the sequence {G k } converges quadratically to the matrix G for positive recurrent QBDs and for transient QBDs The algorithm is called a logarithmic reduction algorithm in [11] We will call it the LR algorithm for Logarithmic Reduction or for Latouche-Ramaswami A similar method is proposed in [2] for positive recurrent QBDs Since the LR algorithm has the greatest advantage over the fixed-point iterations when the QBD is nearly null recurrent, it is important to know the convergence rate of the LR algorithm when the QBD is null recurrent Before we can determine the convergence rate, we will take a closer look into the LR algorithm and present some preliminary results 2 Preliminaries It was mentioned in [11] that G W Stewart pointed out that the LR algorithm is related to the cyclic reduction technique We will make this point more transparent and derive two equations relating H k and L k Let G and F be the minimal nonnegative solution of 11 and 21 F = A 2 + A 1 F + A 0 F 2, respectively We have the following fundamental result see [12], for example

3 CONVERGENCE OF THE LATOUCHE-RAMASWAMI ALGORITHM 3 Theorem 21 If the QBD is positive recurrent, then G is stochastic and F is substochastic with spectral radius ρf < 1 If the QBD is transient, then F is stochastic and G is substochastic with ρg < 1 If the QBD is null recurrent, then G and F are both stochastic It is clear that the matrix G is also the minimal nonnegative solution of G = L 0 + H 0 G 2 Thus, we have the infinite system 22 W 0 H 0 0 L 0 I H 0 L 0 I 0 I G G 2 = K for appropriate K 0 and W 0 As in [2], we apply the cyclic reduction algorithm to 22 and get a reduced system Multiplying both sides of the reduced system by a proper block diagonal matrix, we get an infinite system with the same structure as 22, but with G replaced by G 2 After repeated application of the cyclic reduction algorithm and the block diagonal scaling, we obtain for each n 0, 23 W n H n 0 L n I H n L n I 0 I G 2n G 2 2n = K n 0 0, where H n and L n are as in the LR algorithm From equation 23, we have 24 L n + G 2n H n G 2 2n = 0 for each n 0 Therefore, G = L 0 + H 0 G 2 = L 0 + H 0 L 1 + H 1 G 4 = L 0 + H 0 L 1 + H 0 H 1 L 2 + H 2 G 8 = In general, 25 G = G k + 0 i k H i G 2 2k, where G k is as in the LR algorithm It is clear that the matrix F is also the minimal nonnegative solution of F = H 0 + L 0 F 2 By repeating the whole process leading to the equation 24, we get for each n 0, 26 H n + F 2n L n F 2 2n = 0 From 26, we can see that H n F 2n for each n 0 Thus, we have by G G k F 2 2k 1 G 2 2k

4 4 CHUN-HUA GUO Therefore, if the QBD is positive recurrent or transient, the quadratic convergence of {G k } is an immediate consequence of Theorem 21 In this situation, it is also very easy to determine the limits of the sequences {H k } and {L k } The following result is necessary Theorem 22 Let Q be a stochastic matrix If Q r has a positive column for some integer r 1, then there is a unique vector q 0 such that q T Q = q T and q T e = 1 the vector q is called the stationary probability vector of Q Moreover, there are constants K > 0 and β 0, 1 such that Q n eq T Kβ n for all n 0 In particular, lim n Q n = eq T For a proof of this result, see [6] See also [7] for the special case when Q r is positive for some integer r 1 Obviously, the condition that Q r has a positive column for some r 1 is necessary for lim n Q n = eq T If the QBD is positive recurrent, then G is stochastic and ρf < 1 Assuming that G p has a positive column for some integer p 1, we get from 24 and 26 that lim n H n = 0 and lim n L n = eg T, where g is the stationary probability vector of G If the QBD is transient, then ρg < 1 and F is stochastic Assuming that F p has a positive column for some integer p 1, we have lim n L n = 0 and lim n H n = ef T, where f is the stationary probability vector of F The limits of {H n } and {L n } were determined in [11] in a different way If the QBD is null recurrent, then ρg = 1 and ρf = 1 In this case, 27 tells us nothing about the convergence rate of the LR algorithm It is also much more difficult to determine the limits of the sequences {H n } and {L n } These issues will be resolved in the next section 3 Convergence rate of the LR algorithm for the null recurrent case We start with an algebraic proof of a basic result about the LR algorithm An probabilistic proof was given in [11] Lemma 31 For each k 0, H k + L k e = e Proof First, H 0 +L 0 e = I A 1 1 A 0 +A 2 e = e Assuming that H k +L k e = e k 0, we have H k +L k 2 e = e So, I H k L k L k H k e = Hk 2 +L2 k e Therefore, H k+1 + L k+1 e = I H k L k L k H k 1 Hk 2 + L2 k e = e We have thus proved the result by induction In the above proof, we have used the fact that the sequences {H k } and {L k } are well defined ie, the matrices I H k L k L k H k are nonsingular It is noted in [11] that, when the QBD is null recurrent, it is not true in general that one of the two sequences {H k } and {L k } converges to 0 Our next result shows that neither of the two sequences can converge to 0 for null recurrent QBDs Lemma 32 For the null recurrent QBD, there is a sequence {α k } such that for all k 0, α k 0, αk T e = 1, αt k H k + L k = αk T, and αt k H ke = αk T L ke = 1 2 Proof Recall that α is the stationary probability vector of A 0 + A 1 + A 2 So, α T I A 1 = α T A 0 + A 2 = α T I A 1 H 0 + L 0 Let ˆα T = α T I A 1 Since α > 0 and A 0 0, ˆα T = α T A 0 + A 2 0 and c 0 = ˆα T e > 0 Since the QBD is null recurrent, we have α T A 2 e = α T A 0 e and thus ˆα T H 0 e = ˆα T L 0 e Let α 0 = ˆα/c 0 It is clear that α 0 has all the properties in the lemma, noting that α T 0 H 0 e + α T 0 L 0 e = α T 0 e = 1 Assuming that an α i i 0, with all the properties in the lemma, has been found, we are going to find an α i+1 satisfying these properties

5 CONVERGENCE OF THE LATOUCHE-RAMASWAMI ALGORITHM 5 Since we have α T i = α T i H i + L i = α T i H i + L i 2 = α T i H 2 i + L 2 i + H i L i + L i H i, α T i I H i L i L i H i = α T i H 2 i + L 2 i = α T i I H i L i L i H i H i+1 + L i+1 Since α i 0 and I H i L i L i H i is nonsingular, αi T I H il i L i H i = αi T H2 i +L2 i 0 Thus, αi T H2 i + L2 i e > 0 and we can define α T i+1 = αt i H2 i + L2 i α T i H2 i + L2 i e = αt i I H il i L i H i α T i H2 i + L2 i e It remains to prove α T i+1 H i+1e = α T i+1 L i+1e, which is equivalent to α T i H2 i e = αt i L2 i e Note that α T i H 2 i e α T i L 2 i e = α T i H i e L i e α T i L i e H i e = α T i H i L i e + α T i L i H i e = α T i I L i L i e + α T i I H i H i e = α T i L 2 i e α T i H 2 i e Thus, αi T H2 i e = αt i L2 i e Remark 31 The result in the above lemma has also been obtained independently by Ye [19] In [19] it is assumed that αi T H2 i + L2 i e 0 for each i In the first version of this paper, the author used the assumption that Hi 2 + L2 i e > 0 for each i Without this assumption, the short argument in the proof of the lemma showing αi T H2 i + L2 i e > 0 for each i was pointed out by two referees Our further analysis will rely on Theorem 22 In order to apply Theorem 22, we make the following assumption: 31 deta 0 + za 1 + z 2 A 2 zi has no zeros on the unit circle other than z = 1 This assumption may be verified easily when the matrices A 0, A 1, A 2 have special structures see [13], for example From [4] we know that, in the null recurrent case, assumption 31 is equivalent to the assumption that λ = 1 is a simple eigenvalue of G and F and there are no other eigenvalues of G or F on the unit circle It is easy to show that the latter assumption is in turn equivalent to the next assumption 32 G p and F q have each a positive column for some p 1 and some q 1 Note that assumption 32 for G is satisfied if G k in the LR algorithm has a positive column for some k 0, since G G k In particular, assumption 32 for G is satisfied if L 0 has a positive column Similar comments can be made on assumption 32 for F Since assumptions 31 and 32 are equivalent, Theorem 22 can be applied to G and F under assumption 31 We let f and g be the unique stationary probability vector of F and G, respectively Since H k + L k e = e for all k 0, the sequences {H k } and {L k } are bounded and hence have convergent subsequences Let {H nk } and {L nk } be convergent with lim H n k = H, lim L n k = L

6 6 CHUN-HUA GUO Then, by equations 24 and 26 and Theorem 22, L + eg T Heg T = 0, H + ef T Lef T = 0 Therefore, H = af T with a = e Le, and L = bg T with b = e He Note that a + b = 2e H + Le = e We have thus proved the following result Lemma 33 For the null recurrent QBD with assumption 31, if H, L is a limit point of {H k, L k }, then H = af T and L = bg T with a 0, b 0, and a + b = e To prove that the convergence of the LR algorithm is linear with rate 1/2, we will need to show that lim H k = 1 2 ef T, lim L k = 1 2 egt Lemma 33 is only one small step towards this goal Many other auxiliary results will be needed Although we are unable to show the convergence of the sequences {H k } and {L k } at the moment, it is fairly easy to show that the sequence {α k } in Lemma 32 converges Lemma 34 For the null recurrent QBD with assumption 31, lim α k = 1 f + g 2 Proof Let α be any limit point of {α k } and lim α nk = α We will prove that α = 1 2 f + g We may assume without loss of generality that lim H n k = af T, lim L n k = bg T for some a, b 0 with a + b = e By taking limits in we get α T n k = α T n k H nk + L nk, α T n k H nk e = α T n k L nk e, α T = α T af T + bg T, α T a = α T b Thus, α T a = α T e a = 1 α T a So, α T a = α T b = 1/2 and α T = 1 2 f T + g T, or α = 1 2 f + g As we have already seen, in the null recurrent case, the two equations 24 and 26 are not sufficient to determine the convergence of the sequences {H n } and {L n } We have to seek additional information from the recursions for the sequences {H n } and {L n } The next result is one such finding Lemma 35 For the null recurrent QBD with assumption 31, if and g T a 1, then lim H n k = af T, lim L n k = bg T, lim H n k +1 = âf T, lim L n k +1 = ˆbg T,

7 CONVERGENCE OF THE LATOUCHE-RAMASWAMI ALGORITHM 7 with â = 1 + gt a 1 + 2g T a a + gt a 1 + 2g T a b, ˆb = g T a 1 + 2g T a a gt a 1 + 2g T a b Proof Let ãf T, bg T be any limit point of {H nk +1, L nk +1} and let lim H n sk +1 = ãf T, lim L n sk +1 = bg T Since we get by letting k, I H nsk L nsk L nsk H nsk H nsk +1 = H nsk 2, I af T bg T bg T af T ãf T = af T af T Post-multiplying the above equality by e gives By Lemma 32, ã = f T a + f T bg T ãa + g T af T ãb λa + µb α T n k H nk e = α T n k L nk e = 1 2, α n sk +1 T H nsk +1e = α nsk +1 T L nsk +1e = 1 2 By taking limits in the above identities and using Lemma 34, we have f T + g T a = f T + g T b = f T + g T ã = f T + g T b = 1 So, f T a = 1 g T a = g T e g T a = g T b Similarly, f T b = g T a, f T ã = g T b, f T b = g T ã Thus, λ + µ = f T a + f T bg T ã + g T af T ã = f T a + f T bg T ã + f T ã = f T a + f T b = f T e = 1 Now, Thus, µ = g T af T ã = g T af T λa + µb = 1 µg T af T a + µg T af T b = 1 µg T a1 g T a + µg T a g T a1 g T aµ = g T a1 g T a Since g T a 1, we have µ = g T a/1 + 2g T a and λ = 1 µ = 1 + g T a/1 + 2g T a So, and ã = 1 + gt a 1 + 2g T a a + gt a 1 + 2g T a b, b = e ã = a + b ã = g T a 1 + 2g T a a gt a 1 + 2g T a b

8 8 CHUN-HUA GUO The proof is completed since the limit point is uniquely determined by a and b We can now move a little closer to our goal Lemma 36 For the null recurrent QBD with assumptions 31 and 33 Each limit point af T of the sequence {H n } is such that 0 < g T a < 1, the sequence {H n, L n } has a limit point 1 2 ef T, 1 2 egt Proof Take any subsequence {H nk, L nk } such that lim H n k, L nk = a 0 f T, b 0 g T By the previous lemma, for each integer r 1, where 34 lim H n k +r, L nk +r = a r f T, b r g T, a k+1 = 1 + gt a k 1 + 2g T a k a k + gt a k 1 + 2g T a k b k, and b k+1 = e a k+1 for each integer k 0 Let p k = g T a k We have by 34 p k+1 = 1 + p kp k 1 + 2p k + p k1 p k 1 + 2p k = 2p k 1 + 2p k Since p 0 = g T a 0 > 0 by assumption, it is easy to show that lim p k = 1 2 By 34 we have which can be rewritten as a k+1 = Since lim g T a k = 1 2, we have for k large enough Thus gt a k g T e + a k 1 + 2g T a k, a k a k e = g T a k ak 1 2 e a k e 2 3 a k 1 2 e lim a r = lim b r = 1 r r 2 e Therefore, we can find a subsequence {H mk, L mk } such that lim H m k, L mk = 1 2 ef T, 1 2 egt This completes the proof The next result is quite straightforward Lemma 37 Let the relation between H k+1, L k+1 and H k, L k in the LR algorithm be denoted by H k+1, L k+1 = T H k, L k

9 CONVERGENCE OF THE LATOUCHE-RAMASWAMI ALGORITHM 9 Then 1 2 ef T, 1 2 egt is a fixed point of T Proof It is easy to verify that The result follows since 1 I 2 ef T 1 2 egt egt 2 ef T 1 2 ef T = 1 2 ef T 2, 1 I 2 ef T 1 2 egt egt 2 ef T 1 2 egt = 1 2 egt 2 M I 1 2 ef T 1 2 egt 1 2 egt 1 2 ef T = I 1 4 ef T + g T is a nonsingular M-matrix note that Me = e/2 Thus, we have shown that the sequence {H n, L n } defined by H k+1, L k+1 = T H k, L k k 0 has a limit point 1 2 ef T, 1 2 egt that is a fixed point of T By a theorem on general fixed-point iterations see [8, p 21], for example, we can conclude that the whole sequence {H n, L n } converges to this fixed point if the spectral radius of the Fréchet derivative of the operator T at the fixed point is less than 1 But, unfortunately, the spectral radius is not less than 1 in our case the spectral radius is equal to 4 when the matrices A 0, A 1, A 2 are 1 1 However, the sequence {H n, L n } can still converge since H n, L n may approach 1 2 ef T, 1 2 egt in a special way A delicate analysis for the error H n 1 2 ef T, L n 1 2 egt is necessary For notational convenience, let H = 1 2 ef T and L = 1 2 egt It is easy to see that H 2 = LH = 1 2 H, L2 = HL = 1 2 L We start with expressing H k+1 H in terms of H k H and L k L: H k+1 H = I H k L k L k H k 1 H 2 k I HL LH 1 H 2 = I H k L k L k H k 1 H 2 k H 2 + I H k L k L k H k 1 I HL LH 1 H 2 = I H k L k L k H k 1 H 2 k H 2 + I H k L k L k H k 1 I HL LH I Hk L k L k H k I HL LH 1 H 2 = I H k L k L k H k 1 H 2 k H 2 + H k L k HL + L k H k LHH = I H k L k L k H k 1 H k H k H + H k HH +H k L k L + H k HL + L k H k H + L k LHH To simplify the expression, observe that Thus, and H k H + L k LH = 1 2 Hk + L k e H + Le f T = 0 L k LH = H k HH Hk HL + L k LH H = 1 2 H k H + L k LH = 0

10 10 CHUN-HUA GUO Therefore, H k+1 H = I H k L k L k H k 1 H k H k H + H k HH Similarly, we can get H k H k HH + L k H k HH L k+1 L = I H k L k L k H k 1 L k L k L + L k LL L k L k LL + H k L k LL Now, for any ɛ 0, 1 4, we can find δ > 0 such that whenever H k H δ and L k L δ, 35 H k+1 H = I HL LH 1 HH k H + H k HH with W k ɛ H k H, and HH k HH + LH k HH + W k L k+1 L = I HL LH 1 LL k L + L k LL LL k LL + HL k LL + Z k with Z k ɛ L k L To get rid of the inverse in 35, we use 36 Since I HL LH 1 = I 1 2 H + L 1 = I H + L H + L2 + H + L HH k H + H k HH HH k HH + LH k HH = HH k H + H + LH k HH HH k HH + LH k HH = HH k H + 2LH k HH, and H + L i HH k H + 2LH k HH = HH k H + 2LH k HH for all i 1, we get by 35 and 36 that H k+1 H = HH k H + H k HH HH k HH + LH k HH Similarly, +HH k H + 2LH k HH + W k = H k HH + 2HH k H HH k HH + 3LH k HH + W k L k+1 L = L k LL + 2LL k L LL k LL + 3HL k LL + Z k For the scalar case, H = L = 1 2 So, the estimate for H k+1 H becomes H k+1 H 2H k H If we could replace 3LH k HH by 3HH k HH in the estimate, we would have H k+1 H 1 2 H k H instead Thus, we should try to show that 3H + LH k HH is small in the general case Let α = f + g/2 We have 3H + LH k HH = 3 2 eα T H k Hef T = 3 2 eα α k T H k Hef T,

11 CONVERGENCE OF THE LATOUCHE-RAMASWAMI ALGORITHM 11 since αk T H k He = 1 2 αt k 1 2e = 0 by Lemma 32 Similarly, 3H + LL k LL = 3 2 eα α k T L k Leg T Since lim α k = α by Lemma 34, we can find integer k 1 such that for all k k 1, 3H + LH k HH = P k, 3H + LL k LL = Q k with P k ɛ H k H and Q k ɛ L k L Now we have H k+1 H = H k HH + 2HH k H 4HH k HH + P k + W k 37 = 1 2 H k H 1 2 I 4HH k HI 2H + P k + W k, and L k+1 L = L k LL + 2LL k L 4LL k LL + Q k + Z k 38 = 1 2 L k L 1 2 I 4LL k LI 2L + Q k + Z k Next we will estimate the term H k HI 2H in 37 and the term L k LI 2L in 38 By the equations 24 and 26, we have H k I G 2 2k F 2 2k = F 2k G 2k F 2 2k, L k I F 2 2k G 2 2k = G 2k F 2k G 2 2k Now, Similarly, H k HI 2H = H k I ef T = F 2k G 2k F 2 2k + H k G 2 2k F 2 2k ef T = F 2k ef T G 2k eg T F 2 2k eg T F 2 2k ef T +H k G 2 2 k eg T F 2 2k + eg T F 2 2k ef T L k LI 2L = G 2k eg T F 2k ef T G 2 2k ef T G 2 2k eg T +L k F 2 2 k ef T G 2 2k + ef T G 2 2k eg T By Theorem 22, there are constants C 1 > 0 and β 0, 1 such that H k HI 2H C 1 β 2k, L k LI 2L C 1 β 2k for all k 0 Now, by 37 and 38, we have H k+1 H ɛ H k H + C 2 β 2k, L k+1 L ɛ L k L + C 2 β 2k for any k k 1 with H k H < δ and L k L < δ Let r = ɛ < 1 Since H, L is a limit point of {H k, L k } by Lemma 36, we can find l k 1 such that H l H < δ, L l L < δ, rδ + C 2 β 2l < δ, and β 2l r Now, it is clear that

12 12 CHUN-HUA GUO 39 and 310 are valid for all k l and that β 2l+j 1 r j for all j 0 Thus, we can obtain for any m 1 that H l+m H r m H l H + C 2 r m 1 β 2l + r m 2 β 2l β 2l+m 1 and that r m H l H + C 2 mr m, L l+m L r m L l L + C 2 mr m Therefore, lim H k = H and lim L k = L Moreover, since ɛ > 0 can be arbitrarily small, we also have lim sup k Hk H 1 2, k lim sup Lk L 1 2 In summary, we have proved the following result Theorem 38 For the null recurrent QBD with assumptions 31 and 33, we have lim H k = 1 2 ef T, lim L k = 1 2 egt It is clear that assumption 33 is necessary for the conclusion of the above theorem Since the assumption cannot be verified directly, we will give a sufficient condition that is easier to verify Proposition 39 Let the components of f and g be f i and g i i = 1, 2,, m, respectively, and let S f = {i 1 i m, f i = 0}, S g = {i 1 i m, g i = 0} If assumption 31 and the assumption that 311 S f S g or S g S f are satisfied, then assumption 33 is also satisfied Proof Let lim H n k = af T, lim L n k = bg T It is shown in the proof of Lemma 35 that f T + g T a = f T + g T b = 1, f T a = g T b, f T b = g T a If g T a = 1, then g T b = f T a = 0 By assumption 311, we would have f T +g T a = 0 or f T + g T b = 0, which is a contradiction Similarly, we get a contradiction if g T a = 0 Remark 32 Assumption 311 is certainly satisfied if one of F and G is irreducible in particular, if one of H 0 and L 0 is irreducible since one of S f and S g is an empty set in this case We are now ready to determine the convergence rate of the LR algorithm for the null recurrent case Recall that, for the sequence {G k } generated by the LR algorithm, 312 G G k = H 0 H 1 H k G 2k+1

13 CONVERGENCE OF THE LATOUCHE-RAMASWAMI ALGORITHM 13 Proposition 310 For each k 0, H 0 H 1 H k 0 Proof Equation 847 in [12] states that H 0 H 1 H k ij is the probability of first passage to the state 2 k+1, j before any of the states 0, starting from 1, i If all of these entries were equal to 0, it would be impossible to reach the states 2 k+1, and the transition probability matrix P would not be irreducible Remark 33 The above proof is provided by a referee In the first version of the paper, the author gave the statement in Proposition 310 as an assumption The next theorem is our main result It shows that the sequence {G k } converges to the minimal nonnegative solution of 11 at precisely the rate of 1/2 Theorem 311 For the null recurrent QBD with assumptions 31 and 33, k lim Gk G = 1 2 Proof Since lim H k = 1 2 ef T, we have lim H k e = 1 2e Therefore, for any ɛ 0, 1 2, we can find an integer k 0 such that 1 2 ɛe H ke 1 2 +ɛe for all k > k 0 Note that, by 312, for k > k 0 Thus, G G k = G G k e = H 0 H k0 H k0 +1 H k e 1 2 ɛk k 0 H 0 H k0 e G G k ɛk k 0 H 0 H k0 e In view of Proposition 310, it follows readily that lim sup k Gk G ɛ, k lim inf Gk G 1 2 ɛ Since ɛ can be arbitrarily small, we have lim k G k G = Improvement of the approximate solution in the null recurrent case By 312 and Theorem 38, it is easy to get a much better approximation to the matrix G from the sequence {G k } generated by the LR algorithm In fact, we have by 312 G G k 2G G k+1 = H 0 H k G 2k+1 G 2k+2 +H 0 H k 1 H k 2H k H k+1 G 2k+2 The first term converges to zero quadratically by Theorem 22 since G 2k+1 G 2k+2 = G 2k+1 eg T G 2k+2 eg T The second term is also much smaller than G G k+1 since lim H k 2H k H k+1 = 0 and lim H k H k+1 = 1 4 ef T by Theorem 38 Therefore, Gk+1 = 2G k+1 G k = G k+1 + G k+1 G k can be a much better approximation to G in the null recurrent case Of course, improvements may also be achieved for nearly null recurrent QBDs by using the above strategy 5 Examples In this section, we will present a few examples to illustrate the theoretical results in Section 3 and the simple strategy described in Section 4 for the improvement of the approximate solution For all examples, assumption 31 is checked through the equivalent assumption 32 Example 51 Consider the equation 11 with A 0 = , A 1 = , A 2 =

14 14 CHUN-HUA GUO It is easy to verify that the corresponding QBD is null recurrent We also find that G 1 = L 0 + H 0 L 1 is irreducible and has a positive column and that F 1 = H 0 + L 0 H 1 has a positive column Since G G 1 and F F 1, assumptions 31 and 311 are satisfied By Proposition 39, assumption 33 is also satisfied For this example, the exact minimal nonnegative solutions of 11 and 21 can be found to be G = Accordingly, we have , F = g T = 1431, 874, 120/2425, f T = 1, 0, 4/5 For the matrices H 18 and L 18, found by the LR algorithm using double precision, we have H ef T = , L egt = Note that H 18 and L 18 are already very close to 1 2 ef T and 1 2 egt, respectively We also find that, for the matrices G k computed by the LR algorithm, G G 17 = and G G 18 = Note that G G G G 17 For G 18 = 2G 18 G 17, we have G G 18 = So, G18 is a much better approximation for G For the next example, assumption 311 is satisfied although neither of S f and S g is empty Example 52 Consider the equation 11 with A 0 = , A 1 = , A 2 =

15 CONVERGENCE OF THE LATOUCHE-RAMASWAMI ALGORITHM 15 The corresponding QBD is clearly null recurrent Since 0 05 H 0 = L 0 = 0 05 for this example, assumptions 31 and 311 are satisfied It is easy to find that 0 1 G = F = 0 1 So, we actually have S f = S g = {1} For this example, we have H k = 1 2 ef T and L k = 1 2 egt for all k 0 We also have for each k /2 k+1 G k = 0 1 1/2 k+1 So, {G k } converges to G linearly with rate 1/2 and G k = 2G k G k 1 = G for all k 1 We can also find examples for which 311 is not satisfied Example 53 Consider the equation 11 with A 0 = , A 1 = The corresponding QBD is clearly null recurrent Since H 0 =, L = , A 2 = for this example, assumption 31 is satisfied It is easy to find that G =, F = So, we have S f = {1} and S g = {2} Therefore, assumption 311 is not satisfied However, the conclusions in our main results in Section 3 still hold In fact, we have H k = 1 2 ef T and L k = 1 2 egt for all k 0 We also have for each k 0 G k = 1 1/2 k /2 k+1 0 So, {G k } converges to G linearly with rate 1/2 and G k = 2G k G k 1 = G for all k 1 There are also examples for which assumption 31 is not satisfied The next example is provided by a referee Example 54 Consider the equation 11 with A 0 = , A 1 = 0, A 2 = The corresponding QBD is clearly null recurrent In this case, 0 1 G = F = 1 0

16 16 CHUN-HUA GUO So, 311 is true, but 31 is not satisfied It is easy to find that H k = L k = 1 2 I for each k 1 and that 0 1 1/2 G k = k+1 1 1/2 k+1 0 for each k 0 Thus, {G k } converges to G linearly with rate 1/2 and G k = 2G k G k 1 = G for all k 1 We do not have any examples of null recurrent QBDs for which the convergence of the LR algorithm is not linear with rate 1/2 Acknowledgments The author is grateful to the three referees for their very helpful comments REFERENCES [1] Z-Z Bai, A class of iteration methods based on the Moser formula for nonlinear equations in Markov chains, Linear Algebra Appl, , pp [2] D Bini and B Meini, On the solution of a nonlinear matrix equation arising in queueing problems, SIAM J Matrix Anal Appl, , pp [3] P Favati and B Meini, On functional iteration methods for solving nonlinear matrix equations arising in queueing problems, IMA J Numer Anal, , pp [4] H R Gail, S L Hantler, and B A Taylor, Spectral analysis of M/G/1 and G/M/1 type Markov chains, Adv Appl Probab, , pp [5] C-H Guo, On the numerical solution of a nonlinear matrix equation in Markov chains, Linear Algebra Appl, , pp [6] D J Hartfiel, Markov Set-Chains, Lecture Notes in Math 1695, Springer, Berlin, 1998 [7] R A Horn and C R Johnson, Matrix Analysis, Cambridge University Press, Cambridge, 1985 [8] M A Krasnoselskii, G M Vainikko, P P Zabreiko, Ya B Rutitskii, and V Ya Stetsenko, Approximate Solution of Operator Equations, Wolters-Noordhoff Publishing, Groningen, 1972 [9] G Latouche, Newton s iteration for non-linear equations in Markov chains, IMA J Numer Anal, , pp [10] G Latouche, Algorithms for evaluating the matrix G in Markov chains of P H/G/1 type, Cahiers Centre Études Rech Opér, , pp [11] G Latouche and V Ramaswami, A logarithmic reduction algorithm for quasi-birth-death processes, J Appl Probab, , pp [12] G Latouche and V Ramaswami, Introduction to Matrix Analytic Methods in Stochastic Modeling, SIAM, Philadelphia, PA, 1999 [13] G Latouche and P G Taylor, Level-phase independence for GI/M/1-type Markov chains, J Appl Probab, , pp [14] B Meini, New convergence results on functional iteration techniques for the numerical solution of M/G/1 type Markov chains, Numer Math, , pp [15] M F Neuts, Moment formulas for the Markov renewal branching process, Adv in Appl Probab, , pp [16] M F Neuts, Matrix-Geometric Solutions in Stochastic Models: An Algorithmic Approach, Johns Hopkins University Press, Baltimore, MD, 1981 [17] R S Varga, Matrix Iterative Analysis, Prentice-Hall, Englewood Cliffs, NJ, 1962 [18] Q Ye, High accuracy algorithms for solving nonlinear matrix equations in queueing models, in Advances in Algorithmic Methods for Stochastic Models Proceedings of the 3rd International Conference on Matrix Analytic Methods, G Latouche and P G Taylor, eds, Notable Publications Inc, NJ, 2000, pp [19] Q Ye, On Latouche-Ramaswami s logarithmic reduction algorithm for quasi-birth-and-death processes, Research Report , Department of Mathematics, University of Kentucky, 2001