Google PageRank wth Stochastc Matrx Md. Sharq, Puranjt Sanyal, Samk Mtra (M.Sc. Applcatons of Mathematcs) Dscrete Tme Markov Chan Let S be a countable set (usually S s a subset of Z or Z d or R or R d ). Let {X 0, X 1, X 2,...} be a sequence of random varables on a probablty space takng values n S. Then {X n : n 0, 1, 2,...} s called a Markov Chan wth state space S f for any n Z 0, any j 0, j 1,..., j n 1 S, any, j S one has Pr(X n+1 X 0 j 0, X 1 j 1,..., X n j) Pr(X n+1 X n j). In addton, f Pr(X n+1 X n j) Pr(X 1 X 0 j), j S and n Z 0 then we say {X n : n Z 0 } s a tme homogeneous Markov Chan. Notaton: We denote tme homogeneous Markov Chan by MC. Note: The set S s called state space and ts elements are called states. Column-Stochastc Matrx A column-stochastc matrx ( or column-transton probablty matrx) s a square matrx P ((p j )),j S (where S may be a fnte or countably nfnte set) satsfyng: () p j 0 for any, j S () S p j 1 for any j S Smlarly, row-stochastc matrx can be defned consderng j S p j 1 for any S. Consder the MC, {X n : n S} on the state space S. Let p j Pr(X 1 X 0 j), j S. 1
Then P ((p j )),j S s the column-stochastc matrx. We call P as the stochastc matrx of MC, {X n : n S}. Lemma: If A s a n n matrx whose rows(or columns) are lnearly dependent, then det(a) 0. Let r 1, r 2,..., r n be the rows of A. Gven, r 1, r 2,..., r n are dependent, hence n α 1, α 2,..., α n α 0 and α r 0 α 1 r 1 Consder a matrx A α 2 r 2 wth rows as.. α n r n Now, det(a ) det(a) n α. α 1 r 1 α det(a α 2 r 2 r 0 ) α. 2 r 2 α 2 r 2 α n r n.. α n r α n r n n det(a ) 0 and hence det(a) 0 ( n α 0). Theorem: A stochastc matrx P always has 1 as one of ts egenvalues. Let S {1, 2,..., n} and P ((p j )) 1,j n. Consder the dentty matrx I n, I n ((δ j )) 1,j n where δ j s Kronecker delta. p j 1 and δ j 1 (p j I j ) 0 1 j n 2
Consequently, the rows of P I n are not lnearly ndependent and hence det(p I n ) 0 (by the above lemma). P has 1 as ts egenvalue. Defnton: P (n) ((p (n) j )),j S where p(n) j Pr(X n X 0 j),, j S. A lttle work and we can see that P (n) P n n Z 1. Also P (n) s a column-stochastc matrx as S Pr(X n X 0 j) 1 Classfcaton of states of a Markov Chan Defnton 1: j (read as s accessble from j or the process can go from j to ) f p j (n) > 0 for some n Z 1. Note: j n Z 1 and j j 0, j 1, j 2,..., j n 1 S such that p jj1 > 0, p j1 j 2 > 0, p j2 j 3 > 0,..., p jn 2 j n 1 > 0, p jn 1 > 0. Defnton 2: j (read as and j communcate) f j and j. Essental and Inessental States s an essental state f j S j, then j (e., f any state j s accessble from, then s accessble from j). States that are not essental are called nessental states. Let ξ be set of all essental states. For ξ, let ξ() {j : j} where ξ() s the essental class of. Then ξ( 0 ) ξ(j 0 ) ff j 0 ξ( 0 ) (e., ξ() ξ(k) φ ff k / ξ()). Defnton: A stochastc matrx P havng one essental class and no nessental states (e., S ξ ξ() S) s called rreducble, and the correspondng MC s called rreducble. Let A be a n n matrx. The spectral radus of a n n matrx, ρ(a) s defned as ρ(a) max 1 n { λ : λ s an egenvalue of A} norm of a vector x s defned as x max 1 n x 3
norm of a matrx A s defned as A max ( n a j ). 1 n j1 Also A 2 ρ(a A) ρ(aa ) A 2. If V s a fnte dmensonal vector space, then all norms on V are equvalent. A A 2 A 2 A Lemma: If P s a n n column-stochastc matrx, then P 1. If P s column-stochastc, then P s row-stochastc (e., We know that P s stochastc P max ( p j ) 1 j n P max ( p j ) 1 j n P max 1 j n 1 P 1 P 1 p j 1). Also we know that f V s any fnte dmensonal vector space, then all norms on V are equvalent. P 1 Theorem: If P s a stochastc matrx, then ρ(p ) 1. Let λ be an egenvalue of P 1 n. Then t s also an egenvalue for P. Let x be an egenvector correspondng to the egenvalue λ of P. P x λ x λ x λ x P x P x λ x x 4
λ 1 Also we have proved that 1 s always an egenvalue of P, hence ρ(p ) 1. Defnton: Let ξ. Let A {n 1 : p (n) > 0}. A φ and the greatest common dvsor(gcd) of A s called the perod of state. If j, then and j have same perod. In partcular, perod s constant on each equvalence class of essental states. If a MC s rreducble, then we can defne perod for the correspondng stochastc matrx snce all the sates are essental. Defnton: Let d be the perod of the rreducble Markov chan. The Markov chan s called aperodc f d 1. If q (q 1, q 2,..., q n ) s a probablty dstrbuton for the states of the Markov chan at a gven terate wth q 0 and P q ( P 1j q j, j1 P 2j q j,..., j1 n q 1, then P nj q j ) s agan a probablty dstrbuton for the states at the next terate. A probablty dstrbuton w s sad to be a steady-state dstrbuton f t s nvarant under the transton,.e. P w w. Such a dstrbuton must be an egenvector of P correspondng to the egenvalue 1. The exstence as well as the unqueness of the steady-state dstrbuton s guaranteed for a class of Markov chans by the followng theorem due to Perron and Frobenus. Theorem:(Perron, 1907; Frobenus, 1912) If P s a stochastc matrx and P be rreducble, n the sense that p j > 0, j S, then 1 s a smple egenvalue of P. Moreover, the unque egenvector can be chosen to be the probablty vector w whch satsfes lm P (t) [w, w,..., w]. Furthermore, for any probablty vector q we have lm P (t) q w. Clam: lm j w 5 j1
P ((p j )),j S p j > 0, j S we have, δ mn,j S p j > 0 (P (t+1) ) j (P (t) P ) j p (t+1) j k p kj Let m (t) Now, mn j S p(t) j and M (t) max j S p(t) j m (t+1) mn j S 0 < m (t) M (t) < 1 k p kj m (t) the sequence (m (t) ) s non-decreasng. Also, M (t+1) the sequence (M (t) Hence, lm m (t) max j S k p kj M (t) ) s non-ncreasng. m M lm M (t) exst. We now try to prove that m M. Consder M (t+1) m (t+1) max max max j S [ max (t) [M l S p kj m (t) k p kj mn k p kl k (p kj p kl ) p kj M (t) k (p kj p kl ) + + k (p kj p kl ) ] (p kj p kl ) + + m (t) (p kj p kl ) ] where (p kj p kl ) + means the summaton of only the postve terms (p kj p kl > 0) and smlarly (p kj p kl ) means the summaton of only the negatve terms (p kj p kl < 0). 6
Let + (p kj p kl ) (p kj p kl ) + and Consder (p kj p kl ) (p kj p kl ) (p kj p kl ) If max M (t+1) m (t+1) 1 (M (t) + (p kj p kl ) p kj p kl p kj (1 + (p kl p kj ) + (p kj p kl ) + m (t) ) max (p kj p kl ) + 0, then M (t) m (t). p kl ) (p kj p kl ) +. If max (p kj p kl ) + 0, for the par j, l that gves the maxmum, let r be the number of terms n k S for whch p kj p kl > 0, and s be the number of terms for whch p kj p kl < 0. Then, r 1 and ñ r + s 1 as well as ñ n. e., + + (p kj p kl ) + p kj 1 Hence the estmate of M (t+1) M (t+1) m (t+1) p kj + p kl 1 sδ rδ 1 ñδ 1 δ < 1. m (t+1) (1 δ)(m (t) s p kl m (t) ) (1 δ) t (M (1) m (1) ) 0 7
as t. Let w M m. But, M m m (t) j M (t) lm j w j S lm P (t) [w, w,..., w] lm P (t) [w, w,..., w] P lm P (t 1) P [w, w,..., w] [P w, P w,..., P w] Hence, w s the egenvector correspondng to the egenvalue λ 1. Let x( 0) be an egenvector correspondng to the egenvalue λ 1. P x x P (t) x x lm P (t) x [w, w,..., w]x (w 1 ( x ), w 2 ( x ),..., w n ( x )) ( x )w. S S S S But, lm P (t) x x x ( S x )w ( S x 0 x 0) Hence, egenvector correspondng to egenvalue 1 s unque upto a constant multple. Fnally, for any probablty vector q, the above result shows that lm P (t) q (w 1 ( q ), w 2 ( q ),..., w n ( q )) w. S S S Let q be a probablty dstrbuton vector. Defne q (+1) P q () Z 0 where q (0) q From the above theorem q (t) P (t) q (0) P (t) q t Z 1 lm P (t) q w lm q (t) w 8