}, (n 0) be a finite irreducible, discrete time MC. Let S = {1, 2,, m} be its state space. Let P = [p ij. ] be the transition matrix of the MC.

Transcription

1

2 Abstract Questions are posed regarding the influence that the colun sus of the transition probabilities of a stochastic atrix (with row sus all one) have on the stationary distribution, the ean first passage ties and the Keeny constant of the associated irreducible discrete tie Markov chain. Soe new relationships, including soe inequalities, and partial answers to the questions, are given using a special generalized atrix inverse that has not previously been considered in the literature on Markov chains.

3 Outline 1. Introduction 2. Properties of the generalized inverse, H 3. Stationary distributions 4 Relationship between H and Z 5. Mean first passage ties 6. Keeny s constant 7. Exaples

4 Introduction Let {X n }, (n 0) be a finite irreducible, discrete tie MC. Let S = {1, 2,, } be its state space. Let P = [p ij ] be the transition atrix of the MC. {X n } has a unique stationary distribution { j }, j S. and finite ean first passage ties { ij }, (i, j) S S. P stochastic the row sus are all one. p j=1 ij = 1, i S. Let {c j } be the colun sus of the transition atrix. p i=1 ij c j =, j S.

5 Questions What influence does the sequence {c j } have on { j }? What influence does the sequence {c j } have on { ij }? Are there relationships connecting the {c j },{ j },{ ij }? Can we deduce bounds on the { j } and the { ij } involving the {c j }? What effect does the {c j } have on Keeny s constant K = j=1 j ij?

6 Technique We use the generalized atrix inverse H [ I P + ec T ] 1 where e is the colun vector of ones and c T = (c 1,c 2,..., c ) is the row vector of colun sus of P. Let = e T where T = ( 1, 2,..., ). H can also be expressed in ters of Keeny and Snell s fundaental atrix Z = [I P + ] 1, or in ters of Meyer s group g-inverse (I P) # = Z.

7 Previous results 1. Kirkland considers the subdoinant eigenvalue 2 associated with the set S(c) of stochastic atrices with colun su vector c T. The quantity (c) = ax{ 2 (A) A S(c)} is considered. The vectors c T such that (c) < 1 are identified. In those cases, nontrivial upper bounds on (c) and weak ergodocity results for forward products are provided.

8 Previous results 2. Kirkland considers an irreducible stochastic atrix P and studies the extent to which the colun su vector for P provides inforation on a certain condition nuber (P), which easures the sensitivity of the stationary distribution vector to perturbations in P.

9 Properties of H T t 0 and u T e 0 I P + tu T is non-singular. [I P + tu T ] 1 is a generalised inverse of I P. T e = 1 0 and c T e = 0 I P + ec T is non-singular. H = [I P + ec T ] 1 is a generalized inverse of I P.

10 Key properties of H If H = [I P + ec T ] 1 = [h ij ] then c T H = T. Thus j = c i=1 i h ij for all j S. Further He = e so that h ii h j=1 ij = 1 for all i S. Note also that c T He = 1.

11 Proof of key properties of H (I P + ec T )H = I H PH + ec T H = I. Preultiply by T. Since T P = T, c T H = T j = c i=1 i h ij for all j S. H(I P + ec T ) = I H HP + Hec T = I. Postultiply by e. Since Pe = e and c T e =, He = e h ii h j=1 ij = 1, for all i S.

12 Properties of the eleents of H Let e i T (e j ) be the i-th (j-th) eleentary row (col) vector. Let h j (c) He j denote the j-th colun of H. Let h i j = e T h j (c) be the su of the eleents of the j-th col. Let h i (r )T e i T H denote the i-th row of H. Let h ii = h i (r )T e be the su of the eleents of the i-th row. Let h rowsu = He = T Let h colsu h ij = e i T He j. (c) h j=1 j = [h 1i,h 2i,...,h i ] T, = e T H = (r )T h j = [h i1,h i2,...,h i ], j=1

13 Properties of the eleents of H (a) (Row properties) (r h )T (r i p )T i H = e T i T, (r h )T (r i h )T i P = e T i c T, and h ii = 1/. (b) (Colun properties) h (c) j Ph (c) j = e j j e, h (c) j Hp (c) j = e j (c j )e, and h i j = 1 ( 1) j.

14 Properties of the eleents of H (c) (Eleent properties) h ij = h ij = p ik h kj + ij j, k=1 h ik p kj + ij c j. k=1 (d) (Row and Colun su properties) h rowsu = e, T h colsu = e T ( 1) T. Explicit row and colun sus of the eleents of H. Explicit expressions for individual h ij not readily available.

15 Stationary distributions MC irreducible exists a unique stationary distribution { j }, j S. The stationary probabities found as the solution of the stationary equations: j = i p ij ( j S) with i = 1. i=1 We have shown j = i=1 c i h ij with c i =. i=1 i=1

16 Doubly stochastic atrices Let the stationary probability vector be T = ( 1, 2,..., ) so that T = T P with T e = 1. For doubly stochastic P, e T = e T P so that c T = e T, c = e = e. c i = 1 for all i S i = 1 for all i S.

17 Generalized inverses Keeny and Snell s fundaental atrix of ergodic MCs Z = [I P + ] 1. Meyer s group inverse of I P, (I P) # = Z. Z and (I P) # are generalized inverses of I P. If G is any generalized inverse of I P, (I P)G(I P) is invariant and = (I P) #.

18 Relationship between H and Z If H = [I P + ec T ] 1 and Z [I P + e T ] 1 then (a) Z = H + H, (b) H = Z ect Z, (c) (1+ ) = H + ec T Z, (d) (1+ ) T = T H + c T Z.

19 Eleental relationships If H = [h ij ] = [I P + ec T ] 1 and Z = [z ij ] = [I P + e T ] 1, then (a) z ij = h ij + j k h kj, k=1 (b) h ij = z ij + 1 j 1 z kj, k=1 c k (c) (1+ ) j = k h kj + c k z kj. k=1 k=1

20 Properties of Z and H For ergodic Markov chains the diagonal eleents of Z, z ij, are positive. Matlab exaples show that a siilar relationship holds for the diagonal eleents of H, h ij. (Forally established later.) z ij h ij = j k=1 k h kj = ( c k z kj j ), i.e. z ij h ij is independent of i, and thus = z jj h jj. Consequently, z jj z ij = h jj h ij for all i, j. k=1

21 Mean first passage ties For an irreducible finite MC with transition atrix P, let M = ij be the atrix of expected first passage ties fro state i to state j. M satisfies the atrix equation (I P)M = E PM d, where E = ee T = [1], M d = [ ij ij ] = ( d ) 1 D. If G is any g-inverse of I P, then M = [G E(G) d + I G + EG d ]D.

22 Mean first passage ties Under any of the following three equivalent conditions: (i) Ge = ge, g a constant, (ii) GE E(G) d D = 0, (iii) G E(G) d = 0, M = [I G + EG d ]D. H satisfies (i) ( He = (1/ )e and g = 1/ ) M = [I H + EH d ]D. Z satisfies (i) (since Ze = e and g = 1) M = [I Z + EZ d ]D.

23 Mean first passage ties Thus 1 1 =, i = j, j c i=1 i h ij ij = h jj h ij = h h jj ij, i j. j c i=1 i h ij Thus a knowledge of the {c i } and the {h ij } leads directly to expressions for the { ij }.

24 New relationships For all j {1,2,,}, c i ij =. i=1 ij a new connection between the {c j } and { ij }. i j i=1 c i ij = c j j 1+ h jj j, i=1 ij = i j = 1+ h jj j.

25 Expressions for { j } For all j {1,2,,}, c j j = = ij + c i ij i=1 i=1 1. ij + c i ij i j i j j = j = c + h j jj = 1+ c i ij i=1 h jj 1+ ij i=1 h jj, 1+ c i j i ij = h jj 1 1+ ij i j.

26 Positivity of the z jj and h jj T M = e T Z d D = ( z 11 1,...,z jj j,...,z ) z jj > 0 for all j. jj h jj = 1+ ij > 0 h jj > 0 for all j. i j c i Since j ij = h jj h ij = z jj z ij > 0 h jj > h ij. No surety regarding the sign of any of the {h ij } for i j.

27 Doubly stochastic atrices If c i = 1 for all i, then i j = 1+ 2 h jj = 2 z jj. Also i j ii z jj z ii h jj h ii.

28 Keeny s constant The expression K i = ij j=1 j is in fact independent of i {1,2,...,} K i = K, "Keeny's constant". K has any iportant interpretations in ters of properties of the Markov chain. K is used in the properties of ixing of Markov chains (expected tie to stationarity) K is used in bounding overall differences in the stationary probs of a MC subjected to perturbations.

29 Expressions for K If G = [g ij ] is any g-inverse of I P, then K = 1+ tr(g) tr(g) = 1+ (g j=1 jj K = 1 (1 ) + tr(h) = 1 (1 ) + K = tr(z) = z j=1 jj. g j i j ), h j=1 jj, For any irreducible -state MC, K + 1 2, tr(h) = h j=1 jj

30 Doubly stochastic MCs If c i = 1 for all i, then ii = K = 1+ tr(h) = tr(z). Further, for all i, a new result. ii ( + 1) 2,

31 Exaple Two state MCs Let P = =, p11 p 12 1 a a p 21 p 22 b 1 b (0 a 1, 0 b 1). Let d = 1 a b. MC irreducible 1 d < 1. MC has a unique stationary probability vector T = ( 1, 2 ) = (b (a + b),a (a + b)) = (b (1 d),a (1 d)). 1< d < 1 MC is regular and the stationary distribution is the liiting distribution of the MC. d = 1 MC is irreducible periodic, period 2.

32 Exaple Two state MCs c T = (c 1,c 2 ) = (1 a + b,1+ a b). a and b specify all the transition probabilities. c 1 and c 2 do not uniquely specify the transition probabilities since c 1 + c 2 = 2. We cannot solve for a and b in ters of c 1 and c 2 Note c 2 c 1 = 2(a b).

33 Exaple Two state MCs H = [I P + ec T ] 1 = Z = [I P + e T ] 1 = M = (a + b) 1 a + b b + b 1+ a (1 b) (1 a) 1+ b a a + b a a a + b b a + b a + b a + b = (a + b) b 1 a 1 b (a + b) a...

34 Exaple Two state MCs K = , a + b K = 1.5 a = b = 1. c 1 c 2 b a and h 11 h 22 a b = i1 i2 =

35 Exaple Three state MCs 1 p 2 p 3 p 2 p 3 P = p ij = q 1 1 q 1 q 3 q 3 r 1 r 2 1 r 1 r 2 Six constrained paraeters with 0 < p 2 + p 3 1, 0 < q 1 + q 3 1 and 0 < r 1 + r Let 1 q 3 r 1 + q 1 r 2 + q 1 r 1, 2 r 1 p 2 + r 2 p 3 + r 2 p 2, 3 p 2 q 3 + p 3 q 1 + p 3 q 3,

36 Exaple Three state MCs MC is irreducible (and hence a stationary distribution exists) 1 > 0, 2 > 0, 3 > 0. Stationary distribution given by ( 1, 2, 3 ) = 1 ( 1, 2, 3 ).

37 Exaple Three state MCs Let 12 = p 3 + r 1 + r 2, 13 = p 2 + q 1 + q 3, 21 = q 3 + r 1 + r 2, 23 = q 1 + p 2 + p 3, 31 = r 2 + q 1 + q 3, 32 = r 1 + p 2 + p 3, Let = p 2 + p 3 + q 1 + q 3 + r 1 + r 2 = = = M =

38 Exaple Three state MCs H = c c 3 31 c c 3 ( ) c 2 ( ) c 3 13 c c 3 ( ) c c 3 32 c 1 ( ) c 3 23 c c 2 ( ) c 1 ( ) c 2 32 c c 2 23

39 Exaple Three state MCs Keeny s constant: K = 1 +. For all three-state irreducible MCs, K 2. K = 2 achieved in the inial period 3 case when p 2 = q 3 = r 1.

40 Exaple Three state MCs Under the iposition of colun totals with c 1 + c 2 + c 3 = 3, we can reduce the free paraeters to p 2,p 3,q 1,q 3,c 1 and c 2 by taking r 1 = c 1 1+ p 2 + p 3 q 1, r 2 = c 2 1 p 2 + q 1 + q 3. Let 1 q 1 + q 3 p 2, 2 p 2 + p 3 q 1,then r 1 1 r 2 2, c 1 c 2 r r No universal inequalities connecting c 1 c 2 with 1 2.

41 Exaple Three state MCs The following table gives paraeter regions where the stated inequalities occur, in the case where r 1 > 0. c 1 c 2 c 2 c in r 2 2,r r 2 r r 2 r r r 1 r 1 1 r 2 2 r r 1 r 2 r ax r 2 2,r 1 r 2 r r 2 2 r 1 1 r r r r 1 + 1

42 Exaple Five state MC Five state irreducible MC fro Keeny and Snell [10] (p199). Rearrange the states so that the colun sus are ordered with c 1 c 2 c 3 c 4 c 5 with transition atrix P = c T = (c 1,c 2,c 3,c 4,c 5 ) = (1.229, 1.051, 0.965, 0.910, 0.854) T = ( 1, 2, 3, 4, 5 ) = (0.3216, , , , ) !. Not what we expected!

43 Exaple Five state MC H = All the diagonal eleents of H,h jj, are positive (as expected) All the off-diagonal ters are negative. Each row su is The colun sus are given as T h colsu = ( , , , , ) also ordered according to the order in c T.

44 Exaple Five state MC M = The vector of row sus of M is ( i1, i2, i3, i4, i5 ) = ( , , , , ), leads to no ordered relationships. The vector of colun sus is ( 1i, 2i, 3i, 4i, 5i ) = ( , , , , ), with c i c j ii j i for i j. No general results of such a nature for general finite MCs.

45 Exaple Eight state MC P = (Funderlic and Meyer, exaple involving the analysis of radiophosphorous kinetics in an aquariu syste. ) States reordered so that P has colun sus with c i > c j for i < j. c T = (2.326, 1.140, 1.069, 0.934, 0.890, 0.799, 0.795, 0.047)

46 Exaple Eight state MC c T = (2.326, 1.140, 1.069, 0.934, 0.890, 0.799, 0.795, 0.047) T = (0.2378, , , , , , , ). Note, for exaple 1 < 2 even though c 1 > c H = Diagonal eleents of H are positive. No obvious pattern for off-diagonal elents. Rows sus of H = 0.125, ( =1/8). Colun sus of H do not exhibit any pattern

47 Exaple Eight state MC M = No ordered relationship within the row sus of M No ordered relationship within the colun sus of M Keeny s constant =

48 Conjectures: c i c j for all i, j i j for all i, j h ij < 0 for all i j Valid in the two-state case and the special 5-state case. Not true in general. Exaple: 1/2 1/2 0 If P = 1/2 0 1/2 1/2 1/2 0 then H = 2 9 c T = (3/2,1,1/2), T = (1/2, 1/3, 1/6), h 23 > /2 1 3/2 5/2 1/2 3/2 1/2 7/2 Are there any general inter-relationships? If h ij < 0 for all i j, does i j for all i, j? does c i c j for all i, j?. (Kirkland)

49 References - 1 [1] R. Funderlic, C.D. Meyer, Sensitivity of the stationary distribution vector for an ergodic Markov chain, Linear Algebra Appl. 76 (1986) [2] J. J. Hunter, Generalized inverses and their application to applied probability probles, Linear Algebra Appl. 45 (1982) [3] J. J. Hunter, Matheatical Techniques of Applied Probability, Volue 1, Discrete Tie Models: Basic Theory, Acadeic, New York, [4] J. J. Hunter, Matheatical Techniques of Applied Probability, Volue 2, Discrete Tie Models: Techniques and Applications, Acadeic, New York,1983. [5] J. J. Hunter, Paraetric Fors for Generalized inverses of Markovian Kernels and their Applications, Linear Algebra Appl. 127 (1990) [6] J. J. Hunter, Mixing ties with applications to perturbed Markov chains, Linear Algebra Appl. 417 (2006) [7] J. J. Hunter, Siple procedures for finding ean first passage ties in Markov chains, Asia-Pacific J of Operational Research, 24 (6), (2007),

50 References - 2 [8] J. J. Hunter, Variances of First Passage Ties in a Markov chain with applications to Mixing Ties, Linear Algebra Appl. 429 (2008) [9] J. J. Hunter, Soe stochastic properties of sei-agic and agic Markov chains, Linear Algebra Appl. 433 (2010), [10] J. G. Keeny, J.L. Snell, Finite Markov Chains, Van Nostrand, New York, [11] S. Kirkland, Subdoinant eigenvalues for stochastic atrices with given colun sus, Electronic Journal of Linear Algebra, 18 (2009), , ISSN [12] S. Kirkland, Colun sus and the conditioning of stationary distribution for a stochastic atrix, Operators and Matrices, 4 (2010), , ISSN [13] C. D. Meyer Jr., The role of the group generalized inverse in the theory of finite Markov chains, SIAM Rev.17 (1975)