The Role of Generalized Matrix Inverses in Markov Chains

Transcription

1 The Role of Generalized Matrix Inverses in Markov Chains Jeffrey J Hunter Professor Eeritus of Statistics, Massey University, Professor of Matheatical Sciences, AUT University, New Zealand ANZAPW 2013, July 8 11, 2013, University of Queensland, Brisbane

2 Outline 1. Introduction 2. Generalized atrix inverses 3. Solving systes of linear equations 4. G-inverses of Markovian kernels, I P 5. Stationary distributions of Markov chains 6. Moents of first passage ties in Markov chains 7. G-inverses in ters of stationary distributions and eans of first passage ties 8. G-inverses in ters of stationary distributions, first and second oents of first passage ties 9. Keeny s constant

3 1. Introduction Let P = [p ij ] be the transition atrix of a finite irreducible, discrete tie Markov chain (MC) {X n } (n 0) with finite state space S = {1, 2,, }. i.e. p ij = P{X n = j X n 1 = i} for all i, j S. Such MCs have a unique stationary distribution {π j }, (1 j ). Let T ij be the first passage tie RV fro state i to state j, i.e. T ij = in{ n 1 such that X n = j given that X 0 = i}. T ii is the first return to state i. Let ij = E[T ij X 0 = i], be the ean first passage tie fro state i to state j.

4 Generalized atrix inverses (g-inverses) of I P are typically used to solve systes of linear equations to deduce expressions for {π j } and the { ij }, either in atrix for or in ters of the eleents of the g-inverse. Further, the eleents of every g-inverse of I P can be expressed in ters of the {π j } and the { ij } of the associated MC.

5 2. Generalized Matrix Inverses A generalized inverse of a atrix A is any atrix A such that AA A = A. A is a one condition g-inverse, A (1) A is an equation solving g-inverse If A is non-singular, A = A 1, the inverse of A, and is unique. In general A is not unique. Multi-condition g-inverses: Consider real conforable atrices X (which we assue to be square) Condition 1: AXA = A Condition 2: XAX = X Condition 3: (AX) T = AX Condition 4: (XA) T = XA Condition 5: AX = XA

6 Let A (i,j,..) be any (i,j,..) condition g-inverse of A then A (1,2) is a pseudo-inverse (Rao, 1955) is a reciprocal inverse (Bjerhaar, 1951) is a reflexive inverse (Rhode, 1964) A (1,3) is a least squares g-inverse A (1,4) is a iniu nor g-inverse A (1,2,4) is a weak generalized inverse (Goldan & Zelen, 1964) A (1,2,3,4) is the Moore-Penrose g-inverse (Moore, 1920; Penrose, 1955) A (1,2,5) is the group Inverse (exists and is unique if r(a) = r(a 2 ). ) (Erdeyli,1967)

7 3. Solving systes of linear equations A necessary and sufficient condition for to have a solution is AXB = C AA CB B = C. If this consistency condition is satisfied, the general solution is given by X = A CB + W A AWBB where W is an arbitrary atrix. (Penrose 1955, Rao,1955)

8 Special cases (i) The general solution of XB = C is X = CB + W(I BB ) provided CB B = C, where W is arbitrary. (ii) The general solution of AX = C is X = A C + (I A A)W provided AA C= C, where W is arbitrary. (iii) The general solution of AXA = A is X = A AA + W A AWAA, where W is arbitrary, (since AA AA A = AA A = A.)

9 Note that (iii) provides a characterization of A{1}, the set of all g-inverses of A given any one g-inverse, A : A{1} = {A + H A AHAA, H arbitrary} or A{1} = {A + (I A A)F + G(I AA ), F, G, arbitrary}.

10 4. G-inverses of Markovian kernels, I P Let P be the transition atrix of a finite irreducible Markov chain with stationary probability vector π T. Let e T = (1, 1,, 1) and t and u be any vectors. I P + tu T is non-singular π T t 0 and u T e 0. If π T t 0 and u T e 0 then [I P + tu T ] 1 is a g-inverse of I P. (Hunter,1982)

11 All one condition g-inverses of I P can be expressed as A (1) = [I P + tu T ] 1 + ef T + gπ T where f, g, t, and u are arbitrary with u T e 0 and π T t 0. (Hunter,1982) Given any g-inverse G of I P and t, u, with π T t 0, u T e 0, we can copute [I P + tu T ] -1 as [I P + tu T ] 1 = I eut u T e G I tπ T π T t + eu T (π T t)(u T e). (Hunter, 1988)

12 Paraetric characterisation of g-inverses of I - P (Hunter, 1990) If G is any g-inverse of I P there exist unique paraeters α, β, and γ such that G = G(α, β, γ ) = [I P + αβ T ] 1 + γ eπ T, with the property that α, β and γ involve 2 1 independent paraeters with the properties that π T α = 1, β T e = 1 and γ + 1 = β T Gα.

13 Construction of the unique characterisation Given G, any g-inverse of I P, Let A I (I P)G and B I G(I P). Then A = απ T, B = eβ, and G = [I P + αβ T ] 1 + γ eπ T, where α = Ae, β T = π T B ( = e T i B for all i), and γ +1 = π T Gα = β T Ge = β T Gα. α, β, γ uniquely characterise the g-inverse as G(α, β, γ ).

14 Application: Let G = G(α,β,γ ) be a (α,β,γ) g-inverse of I P. G A{1, 2} γ = 1. G A{1, 3} α = π / π T π. G A{1, 4} β = e T / e T e. G A{1, 5} α = e, β T = π T. We subdivide the A{1,5} category: G A{1, 5a} α = e, G A{1, 5b} β T = π T. so that G A{1, 5} G A{1, 5a} and G A{1, 5b}.

15 Theore Let G = G(α, β, γ ) be a (α, β, γ) g-inverse of I P, where P is the transition atrix of a finite irreducible MC with stationary probability vector π. (a) G A{1, 5a} Ge = ge for soe g. If Ge = ge for soe g then g = 1 + γ. (b) G A{1, 5b} π T G = hπ T for soe h If π T G = hπ T for soe h then h = 1 + γ Corollary: Let G = G(α, β, γ ) be a (α, β, γ) g-inverse of I P. If Ge = ge for soe g and π T G = hπ T for soe h then g = h = 1 + γ and G A{1, 5}. i.e. G = G(e, π, γ ).

16 Special cases of g-inverses of I P (a) Z = [I P + Π ] 1 where Π = eπ T (Keeny & Snell's fundaental atrix, Z, 1960) Shown to be a 1-condition g-inverse, Hunter (1969). Z is a (1,5) g-inverse with for G = G(e, π, 0). (b) A # = [I P + eπ T ] 1 eπ T = Z Π (Group inverse) (Meyer, 1975) A # is the unique (1, 2, 5) g-inverse with for G = G(e, π, -1).

17 Special cases of g-inverses of I P (c) G = G(π /π T π, e/e T e, 1) is the Moore-Penrose g-inverse G = [I P + απe T ] 1 απ where α = (π T π ) 1/2 (Styan, Paige, Wachter, 1975) Alternative for: G = [I P + πe T ] 1 eπ T π T π. Equivalence coes fro the fact that A δ = [I P + δtu T ] 1 eπ T δ (π T t)(u T e) does not depend on δ. (Hunter, 1988)

18 5. Stationary distributions Finite irreducible MC s {X n } have a unique stationary distribution {π j },(1 j ) which, for aperiodic MC s is the liiting distribution, i.e.li P{X n n = j X 0 = i} = li P{X n n = j} = π j,(1 j ). Let π T = (π 1, π 2,..., π ) be the stationary prob. vector for the irreducible MC with transition atrix P = [p ij ]. We need to solve π j = π i p ij with π i = 1, i.e. π T (I P) = 0 T with π T e = 1. This is an equation of the type XB = C, i =1 with X = π T,B = I P, C = 0 T. i =1

19 Procedures using A = I (I P)G If G is any g-inverse of I P and A = I (I P)G then π T = v T A v T Ae, where v T is such that v T Ae 0. Note: Ae 0 so that we can always find such a v T.

20 Procedure using A = I (I P)G Let G be any g-inverse of I P, and A = I (I P)G = [a ij ]. Let r be the sallest integer i (1 i ) such that then π j = a rj ark k=1, j = 1,...,. a k=1 ik 0, If G is a (1,3) or (1,5) g-inverse of I P then r = 1.

21 Procedures using G If G = [I P + tu T ] 1 where u, t such that u T e 0, π T t 0, π T = ut G u T Ge. (Paige, Styan, Wachter,1975), (Keeny,1981), (Hunter,1982). If G is a (1,4) g-inverse of I P then π T = et G e T Ge.

22 Special g - inverses of for G = [I P + tu T ] 1 Identifier g-inverse Paraeters [I P + tu T ] 1 α β T γ G ee [I P + ee T ] 1 e e T / (1/) 1 [I P + e p (r)t b ] -1 e (r)t p b 0 (r) G eb G eb [I P + e e b T ] -1 e (c) G ae (c,r) G ab (c) G ab e b T 0 [I P + p a (c) e T ] -1 p a (c) / π a e T / (1/π a ) 1 [I P + p a (c) p b (r)t ] -1 p a (c) / π a p b (r)t (1/π a ) 1 [I P + p a (c) e b T ] -1 p a (c) / π a e b T (1/π a ) 1 G ae [I P + e a e T ] -1 e a /π a e T / (1/π a ) 1 (r) G ab [I P + e a p b (r)t ] -1 e a /π a p b (r)t (1/π a ) 1 G ab [I P + e a e b T ] -1 e a /π a e b T (1/π a ) 1 G tb (c) [I P + t b e b T ] -1 (t b e e b + p b (c) ) t b e b T 0

23 Special case: G = [I P + eu T ] 1 (with u T e 0). π T = u T G, Hence if u T = (u 1,u 2,,u ) and G = [g ij ] then π j = u k g kj, j = 1, 2,...,. k=1 Thus G ee = [I P + ee T ] 1 π j = g kj = g i j G eb k=1 k=1 (r ) = [I P + ep b (r )T ] 1 π j = p bk g kj G eb = [I P + ee b T ] 1 π j = g bj

24 Special case: G = [I P + te T ] 1 (with π T e 0). π T = et G e T Ge. Hence if G = [g ij ] then π j = g i j g ii, j = 1, 2,...,. Holds for G (c) ae = [I P + p (c) a e T ] 1, G ee = [I P + ee T ] 1 ( π j = g i j ), G ae = [I P + e a e T ] 1.

25 Special case: G = [I P + te b T ] 1 (with π T e 0). π T = e T G b e T b Ge, Hence if G = [g ij ] then π j = g bj g bi, j = 1, 2,...,. Holds for G (c) ab = [I P + p (c) a e T b ] 1, G ab = [I P + e a e b T ] 1, G eb = [I P + ee b T ] 1 ( π j = g bj ), G (c) tb = [I P + t b e T b ] 1 ( π j = g bj ).

26 6. Moents of first passage tie distributions Let T ij be the first passage tie RV fro state i to state j, {X n } irreducible T ij are proper r.v. s. For all i, j S, and k 1 let ij (k ) = E[T ij k X 0 = i]. The ij (k ) are well defined and finite. Let (1) ij = ij, the ean first passge tie fro state i to state j. Let M = [ ij ] be the atrix of ean first passage ties Let M (2) = [ (2) ij ] be the atrix of second oents.

27 Well known that ij = 1+ kj. k j p ik M satisfies the atrix equation (I P)M = E PM d, where E = [1] = ee T, and M d = [δ ij ij ] =(Π d ) 1 D (with Π = eπ T ). i.e. equation of the type AX = C, where X = M, A = I P and C = E PD.

28 If G is any g-inverse of I P, then M = [GΠ E(GΠ) d + I G + EG d ]D. (Hunter, 1982) Thus if G = [g ij ], and g ii = g, then j=1 ij ij = [g jj g ij + δ ij ] π j + (g ii - g ji ), for all i, j.

29 Joint coputation for π j and ij 1. Copute G = [g ij ], ANY g-inverse of I P. 2. Copute sequentially rows 1, 2, r ( ) of A = I (I P)G [a ij ] until is the first non-zero su. a k =1 rk, (1 r ) 3. Copute π j = a rj a rk, j = 1,...,. k =1 4. Copute jj = a k =1 rk a rj, j = 1,...,, and for, i j. { } ij = (g jj g ij ) a k =1 rk a + { (g } rj g k =1 ik jk ).

30 Let H = G(I Π) then M = [I H + EH d ]D. H leads to sipler eleental fors for M: If H = [h ij ], ij = [h jj h ij + δ ij ] π j, for all i, j, i.e. ij = 1 π j i = j, (h jj h ij ) π j, i j. We can siplify this further under special conditions:

31 Theore : G A{1, 5a} M = [ I G + EG d ]D. Thus under any of the following equivalent conditions: (i) G A{1, 5a} (ii) Ge = ge, g a constant, (iii) GE E(GΠ ) d D = 0, (iv) GΠ E(GΠ) d = 0, we have that M = [ I G + EG d ]D and ij = [g jj g ij + δ ij ] π j, for all i, j,

32 Significance of H = G(I Π) Let G = G(α, β, γ ) be any 1 - condition g-inverse of I P, Then H = G(I Π ) is a g-inverse of I P with H = G( e,β, 1) A{1, 2, 5a}. Further K = (I Π )H = (I Π )G(I Π ) is a g-inverse of I P with K = G( e,π, 1) A{1, 2, 5}, the group inverse. Special cases with the siple eleental for for the ij : (a) G = [I P + Π ] 1 = Z where Π = eπ T Keeny & Snell's fundaental atrix (γ = 0) (Keeny and Snell, 1960) (b) G = [I P + Π ] 1 Π = Z Π = A # Meyer's Group inverse (γ = 1) (Meyer, 1975)

33 Second oents of the first passage ties M (2) satisfies the atrix equation (I P)M (2) = E + 2P(M M d ) PM (2) d. where M (2) d = 2D(ΠM) d D, with D = M d = (Π d ) 1. G is any g-inverse of I P, M d (2) = D + 2D{(I Π)G(I Π)} d D. G A{1,5a} M (2) d = D + 2DG d D 2D(ΠG) d D, G A{1,5b} M (2) d = D + 2DG d D 2D(GΠ) d D, G A{1,5} M (2) d = 2DG d D (1+ 2γ )D, In particular, M (2) d = D + 2DA # D = 2DZ D D. d d

34 Second oents of first passage ties If G is any g-inverse of I P, M (2) = 2[GM E(GM) d ] + [I G + EG d ][M (2) d + D] M, = 2[GM E(GM) d ] + 2[I G + EG d ]D(ΠM) d M. If G A(1, 5a) then M (2) = 2[GM E(GM) d ] + MD -1 M d (2). In particular, M (2) = 2[ZM E(ZM) d ] + M(2Z d D I), = 2[A # M E(A # M) d ] + M(2A # D + I). d (Hunter, 2007b)

35 (2) Eleental expressions for ij If G = [g ij ] then ij k=1 (2) = 2 (g ik g jk ) kj ij + (δ ij g ij + g jj )( (2) jj + jj ). If Ge = ge G A(1,5a), then ij k=1 (2) = 2 (g ik g jk ) kj + ij (2) jj. jj jj i=1 (2) + jj = 2 jj π i ij. (Hunter, 2007b)

36 Coputational considerations - 1 Two relevant papers: [1] Heyan and O Leary (1995) ( Coputations with Markov chains (2nd International Workshop on M.C. s) [2] Heyan and Reeves (1989) (ORSA J Coputing) [1]: deriving eans and variances of first passage ties fro either the fundaental atrix Z or the group generalized inverse A # leads to a significant inaccuracy on the ore difficult probles. it does not ake sense to copute either the fundaental atrix or the group generalized inverse unless the individual eleents of those atrices are of interest.

37 Coputational considerations - 2 [2]: Coputation of M using Z or A # - 3 sources of error 1. An algorith for coputing π. 2. Copute the inverse of I - P + Π. (Matrix ay have negative eleents - can cause round off-errors in coputing inverse.) 3. Matrix evaluation of M - the atrix ultiplying D ay have negative eleents. Additional work to copute M (2) - three atrix ultiplications are required - two of which involve a diagonal atrix - in each of these ultiplications there is a atrix with possibly negative eleents. There has to be a better way!

38 Sipler coputation technique Let G eb = [g ij ] = [I P + ee b T ] 1 ( a special A(1, 5a) g- inverse) π j = g bj, j =1,2,...,; ij = δ + g ij jj g ij 1 g bj, i = j, = g bj (g jj g ij ) g bj, i j. Thus following one atrix inversion (actually only the b-th row for the stationary distribution), one can find the stationary probabilities and the ean first passage ties. (2) ij = var[t ij ] = jj [1+ 2 jj (g jj - g (2) bj )], i = j, 2 jj [g (2) jj - g (2) ij + ij (g jj - g (2) bj )] - ij, i j. jj [1 jj + 2 jj (g jj - g (2) bj )], i = j, 2 jj [g (2) jj g (2) ij + ij (g jj - g (2) bj )] - ij (1- ij ), i j.

39 7. G-inverses in ters of stationary probabilities and ean first passage ties Let G = G(α,β,γ ) be any 1-condition g-inverse of I P then, since Gα = (γ + 1)e and β T G = (γ + 1)π T, ( ). (i) g jj = 1 1+ γ α k j α k j g jk ( ). i j (ii) g jj = 1 β j (1+ γ )π j β i g ij If we have expressions for the g ij when i j we can deduce expressions for all the g jj.

40 We do not, in general have any inforation fro the previous results about g i = g j=1 ij for specific i. If α k = α, a constant ( = 1, since π T α = απ T e = α) g i = 1+ γ, a constant fo all i. Thus when α = e, i.e when G A{1, 5a} g ij = g jj + π j ij, for all i j. We explore the special case when α = e later. We need a procedure that yields expressions for g i

41 Let G = G( α, β, γ ) be any 1-condition g-inverse of I P Let H = G(I Π ) [h ij ] h ij can be expressed in ters of the {β k } paraeters of the g-inverse. If δ j k j β k kj, then h jj = π j δ j (j =1,2,..) h ij = π j (δ j ij ), i j, (i, j =1,2,..)

42 We can now find an expression for the row sus of G, in ters of the {α k } paraeters of the g-inverse, the π j and the ij. If δ j β k kj (j = 1,, ) then k j g i = 1+ γ + π k α k ik π k α k δ k. k i k=1 By expressing g ij in ters of h ij we can find an expression for the eleents of any g-inverse of I P. In particular, fro H = G(I Π), g ij = h ij + g i π j. This leads to the following KEY result

43 KEY THEOREM Let G = [g ij ] = G(α, β, γ) be any g-inverse of I P. Then the g ij can be expressed in ters of the paraeters {α j }, {β j }, γ, the stationary probabilities {π j }, and the ean first passage ties { ij }, of the Markov chain as g ij = 1+ γ +δ j ij + π k α k ik π k α k δ k i k=1 k 1+ γ +δ j + π k α k k j jk π k α k δ k=1 k π j, i j, π j, i = j. where δ j β k kj (j = 1,, ). k j

44 New interconnections For all G = G(α,β,γ ) one condition g-inverses of I P, with δ j = β k kj, k j π j (δ j + g i ij ), i j, g ij = π j (δ j + g j ), i = j. leading to the alternative expression δ j = g jj π j g j.

45 Special cases 1: δ j β k kj (j = 1,, ). k j G = G(e,β,γ) A{1, 5a} π j (δ j +1+ γ ij ), i j, g ij = π j (δ j +1+ γ), i = j. G = G(e,β, 1) A{1, 2, 5a} π j (δ j ij ), i j, g ij = π j δ j, i = j.

46 Special cases 2: η j ( kj ), (j = 1,, ). k j G = G(e, e,γ) A{1, 4, 5a} π j (1+ γ +η j ij ), i j, g ij = π j (1+ γ +η j ), i = j. G = G(e, e, 1) A{1, 2, 4, 5a} (unique g-inverse) π j (η j ij ), i j, g ij = π j η j, i = j.

47 Special cases 3: τ j π k kj = π k kj +1. k=1 k j G = G(e,π,γ ) A{1, 5} π j (τ j + γ ij ), i j, g ij = π j (τ j + γ ), i = j. Z = G(e,π,0) A{1, 5} with γ = 0 (unique, fundaental atrix) π j (τ j ij ), i j, g ij = π j τ j, i = j. A # = G(e,π, 1) A{1, 2, 5} (unique, group inverse) π j (τ j 1 ij ), i j, g ij = π j (τ j 1), i = j. (Ben-Ari, Neuann (2012))

48 Note re τ j Since jj (2) + jj = 2 jj π i ij, i =1 τ j = π i ij = (2) + jj jj i =1 2 jj = (2) jj 2 jj = 1+ π (2) j jj 2. [In Hunter (2008) expressions for τ j in ters of eleents of g-inverses of I P are given.]

49 Special case 4: G = G(π π T π,e, 1) A{1, 2, 3, 4} (Moore-Penrose) g ij = η j ij + (1 2 π k ) π 2 2 k ik (1 π k k i ) π 2 k η k π j, i j, η j + (1 2 π k ) π 2 2 k ik (1 π k ) π 2 k η k π j, i = j. k j k=1 k=1 where η j ( kj ), (j = 1,, ). k j

50 8. G-inverses in ters of stationary probs, first and second oents of passage ties. If G = [g ij ] = G(e, π, γ ) A{1, 5} π j γ + π (2) +1 j jj 2 ij, i j, g ij = π j γ + π (2) +1 j jj, i = j. 2 Special cases: Z = G(e, π, 0) and A # = G(e, π, 1). Result for A # given by Ben-Ari and Neuann (2012) - using analytic continuation, Laurent expansions and Taylor series expansions of generating functions.

51 9. Keeny s constant. K = π j ij = π j ij +1. j=1 j i The interesting observation is that this su is in fact a constant, independent of i. This is in contrast to τ j = which varies with j π i ij = π i ij +1. If G = [g ij ] is any generalised inverse of I P, (g j=1 jj i=1 K = 1+ g j π j ). i j If G = G(e, β, γ) A{1, 5a}, K = tr(g) γ. In particular, K = tr(z) = tr(a # )+1.

52 Siplification properties For all G = G(α,β,γ ), one condition g-inveres of I P, let δ j = β k kj. k j Then π k δ k = K 1, k=1 where K = π j=1 j ij is Keeny's constant (constant for all j = 1, 2,...,.)

53 References 1 Ben-Ari, I. & Neuann M. (2012). Probabilistic approach to Perron root, the group inverse, and applications, Linear and Multilinear Algebra, 60 (1), Erdelyi, I. (1967). On the atrix equation Ax = Bx. J. Math. Anal. Appl. 17, Hunter, J.J. (1969). On the oents of Markov renewal processes, Adv. Appl. Probab., 1, Hunter, J.J. (1982). Generalized inverses and their application to applied probability probles, Linear Algebra Appl., 45, Hunter, J.J. (1983). Matheatical Techniques of Applied Probability, Volue 2, Discrete Tie Models: Techniques and Applications. Acadeic, New York. Hunter J.J. (1986). 'Stationary distributions of perturbed Markov chains' Linear Algebra Appl. 82, Hunter, J.J. (1988). Characterisations of generalized inverses associated with Markovian kernels, Linear Algebra Appl., 102, Hunter, J.J. (1990). Paraetric fors for generalized inverses of Markovian kernels and their applications, Linear Algebra Appl., 127, Hunter, J.J. (1992) "Stationary Distributions and Mean First Passage Ties in Markov Chains using Generalized Inverses" Asia-Pacific Journal of Operational Research, 9, , (1992) Hunter, J. J. (2006). Mixing ties with applications to perturbed Markov chains, Linear Algebra Appl., 417,

54 References 2 Hunter, J.J. (2007). Siple procedures for finding ean first passage ties in Markov chains Asia-Pacific Journal of Operational Research, 34, Hunter, J.J. (2008). Variances of First Passage Ties in a Markov chain with applications to Mixing Ties, Linear Algebra Appl., 429, Hunter, J.J. (2009) Coupling and Mixing in a Markov chain Linear Algebra and it s Applications, 430, Keeny, J.G. and Snell, J.L. (1960). Finite Markov Chains. Van Nostrand, New York. Meyer, C.D. Jr. (1975). The role of the group generalized inverse in the theory of finite Markov chains, SIAM Rev., 17, Moore, E. H. (1920). On the Reciprocal of the General Algebraic Matrix. (Abstract), Bulletin of the Aerican Matheatical Society 26, Penrose, R. A. (1955). A Generalized Inverse for Matrices, Proceedings of the Cabridge Philosophical Society 51, Paige, C.C., Styan, G.P.H., and Wachter, P.G. (1975). Coputation of the stationary distribution of a Markov chain, J. Statist. Coput. Siulation, 4,