Saionary Disribuion Design and Analysis of Algorihms Andrei Bulaov
Algorihms Markov Chains 34-2 Classificaion of Saes k By P we denoe he (i,j)-enry of i, j Sae is accessible from sae if 0 for some k 0 If is accessible from and is accessible from, hen and communicae, Thus, wo saes communicae iff hey are in he same srongly conneced componen of he digraph The relaion of communicaing is an equivalence relaion (check!) A M.C. is said o be irreducible if all saes communicae o each oher Fac: k P i, j > A M.C. is irreducible iff he digraph is srongly conneced
Algorihms Markov Chains 34-3 Classificaion of Saes (cnd) Le r i, j denoe he probabiliy ha saring from sae he firs occurrence of sae happens a ime. r i Pr( X a j and X s a j for s < X 0 Sae is recurren if I is ransien if, j ai r i, i < r i, i ) A M.C. is recurren if every sae is recurren The expeced ime o ge from o Recurren ime h, h i, j r i, j
Algorihms Markov Chains 34-4 Classificaion of Saes (cnd) Sae is periodic if here exiss such ha muliple of A M.C. is periodic if any of is saes is periodic A sae or chain ha is no periodic is called aperiodic unless s is a In a finie (!!) M.C. a sae is ergodic if i is aperiodic and recurren A M.C. is ergodic if all is saes are ergodic Fac. s P i, i 0 A finie M.C. is ergodic if i is irreducible and aperiodic Gambler s Ruin All saes are ransien excep /2 /2 /2 /2 /2 -L- -L - 0 /2 W W+ for -L- and W+ /2 /2 /2 /2 These wo are absorbing
Algorihms Markov Chains 34-5 Fae of he Gambler Gambler s Ruin: Find he probabiliy ha he Gambler goes bankrup in he end /2 /2 /2 /2 /2 -L- -L - 0 W /2 /2 /2 /2 /2 Le P i P0, i. Since all saes -L j W are ransien, lim Pj 0 Le lim P. Then lim q. q W + P L Le be he random variable equal he Gambler s gain afer rounds Since he game is fair, 0. Therefore E[ G W ] + jp j L W+ j 0
Algorihms Markov Chains 34-6 Fae of he Gambler Since he game is fair, 0. Therefore 0 ] [ + W L j j jp G E 0 ) )( ( ) ( ] [ lim + + q L q W G E 2 + + + L W L q
Algorihms Markov Chains 34-7 Saionary Disribuion A saionary disribuion (equilibrium disribuion) of a M.C. is a probabiliy disribuion π over is saes such ha π π P Thus π is an eigenvecor of P wih eigenvalue / 2 / 4 Land of Oz P / 2 0 / 4 / 4 / 2 ( x, y, z) ( x, y, z) / 2 / 4 ( x, y, z) ( 2 5 / 4 / 2 / 2 / 4, 0 / 4 5, 2 5 ) / 4 / 2 / 2
Algorihms Markov Chains 34-8 Saionary Disribuion (cnd) Theorem (Saionary Disribuion) Any finie, irreducible, and ergodic Markov chain has he following properies:. he chain has a unique saionary disribuion,, ; 2. for all i and j he limi lim P i, j exiss and is independen of i; 3. π j j, j lim P h j, j Gambler s Ruin: Two saionary disribuions I is no irreducible and no recurren
Algorihms Markov Chains 34-9 Mixing Time For a M.C. wih ransiion marix P and saionary disribuion π, he mixing ime is he minimal such ha for any se A of saes Pr( A) π ( A) X Bayer, Diakonis: The number of riffle shuffles needed o mix an ordinary 52 card deck is 7 4 The specral gap is he difference beween eigenvalue and he second larges eigenvalue In general, he mixing ime is shor (rapid mixing) if he specral gap is large
Algorihms Markov Chains 34-0 Page Rank Le G (V,E) be a digraph. The (simplified) PageRank of G is a funcion PR: V [0,] such ha for all u V PR ( u) v N )( in PR( v) oudeg v ( u) ( ) are he ransiion probabiliies of he M.C. whose graph is G and for each sae he ransiion probabiliies are equal Page Rank is hen is saionary disribuion
Algorihms Markov Chains 34- Page Rank (cnd) Transform he graph:. Ge rid of sinks: Add an arc from each sink o every node 2. Inroduce he damping facor (d 0.85) he probabiliy ha he random surfer makes he nex click. Technically, add an exra `damp node wih he probabiliy of going here d. Then rea i as all oher sinks
PR )( Algorihms Markov Chains 34-2 Page Rank (cnd) Therefore PR is he soluion o he sysem of equaions ( u) v N in ( u) PR( v) ( + ) oudeg d v n d In oher words, PageRank is he saionary disribuion of M.C. wih ransiion marix P' d P( G) + n d E n where P(G) is he adjacency marix of G (wih sinks processed), and is he n n marix wih all-ones enries
Algorihms Markov Chains 34-3 Compuing Page Rank To compue PageRank use he saionary disribuion heorem (or, equivalenly, he Perron-Frobenius heorem) Sar wih disribuion p(0) (,,) Then compue p() ( ) If is a leas mixing ime, we have a good approximaion of PageRank Due o is consrucion (damping facor) P ends o have large specral gap Therefore a reasonable approximaion of PageRank requires few ieraions
Algorihms Markov Chains 34-4 Simple Queue A queue is a line where cusomers wai for service. Wha is he `normal lengh of he queue - If he queue has fewer han n cusomers, hen a new cusomer joins wih probabiliy λ - If he queue is no empy, hen wih probabiliy µ he head of he line is served and leaves he queue - Oherwise he queue is unchanged Therefore he lengh of he queue is a M.C. wih ransiion probabiliies P λ if µ i, i+ i < n; Pi, i if i > P i, i λ, λ µ, µ, if if if i i 0, i n, n, 0
Algorihms Markov Chains 34-5 Simple Queue (cnd) This Markov chain is irreducible, finie, and aperiodic. Therefore i has a unique saionary disribuion, which gives us he `normal lengh of he queue. We have : As is easily seen, π is a soluion of his sysem Also, π ( λ π + µπ π 0 ) 0 i λπ i + ( λ µ ) πi + µπi+ n λπ n + ( µ π n π ) n π i i 0 πi 0 ( ) i µ λ
Algorihms Markov Chains 34-6 Simple Queue (cnd) Therefore n i 0 π 0 π For all 0 i n i π i n i 0 π n j 0 0 ( ) λ i µ ( λ ) j µ ( λ ) i µ ( λ ) n j j 0 µ
Algorihms Markov Chains 34-7 Random Walks A random walk on a (di)graph G is a M.C. defined by he sequence of moves of a paricle beween verices of G along he edges of G In his process, he place of he paricle a a given ime sep is he sae of he sysem. If he paricle is a verex i and if i has oudeg(i) ougoing edges, hen he probabiliy ha he paricle follows he edge (i,j) and moves o a neighbour j is ( ) Thus he ransiion marix is given by, ( ) if j is a neighbour of i, 0 oherwise
Algorihms Markov Chains 34-8 Ergodic Random Walks Lemma A random walk on an undireced graph G is aperiodic if and only if G is no biparie Proof. Graph G is no biparie if and only if i conains a cycle of odd lengh. Since every edge is a cycle of lengh 2, any non-biparie graph is aperiodic On he oher hand, every biparie graphs is periodic wih period 2. QED
Algorihms Markov Chains 34-9 Saionary Disribuion of Random Walks Lemma A random walk on an undireced graph G converges o a saionary disribuion π, where Proof. Since deg ( ) 2, i follows ha deg ( ), 2 and π is a proper disribuion over V. Le P be he ransiion marix and N(v) denoe he neighbours of v. The relaion is equivalen o ( ) ( ) ( ). ( ) QED
Algorihms Markov Chains 34-20 Recurren Time of Random Walks Corollary For any verex u in G h, ( ). Lemma If (u,v) is an edge in G, hen h, <2. Proof. Compue h, in wo differen ways: 2 deg ( ) h, deg ( ) (+h, ) ( ). Therefore 2 ( ) (+h, ). And we conclude h, <2. QED
Algorihms Markov Chains 34-2 Cover Time of Random Walks The cover ime of a graph G (V,E) is he maximum over v V of he expeced ime o visi all of he nodes in he graph by a r. w. saring from v Lemma The cover ime of G (V,E) is bounded from above by 4 V E. Proof. Choose a spanning ree of G. There is a cyclic our on his ree ha raverses every edge of he ree exacly once in each direcion. Le,,, be he sequence of verices in he our. The expeced ime o go hrough he verices in he our is an upper bound on he cover ime. Hence he cover ime is bounded by h, < 2 2 2 <4.
Algorihms Markov Chains 34-22 s--conneciviy Algorihm An algorihm o check if wo nodes are conneced in a graph ha uses almos no memory given graph G (V,E), verices s, V. sar a random walk from s if he walk reaches wihin 4 `here is a pah oherwise oupu `here is no pah seps, oupu
Algorihms Markov Chains 34-23 s--conneciviy Algorihm: Analysis Theorem The s--conneciviy algorihm reurns he correc answer wih probabiliy /2, and i is only errs by reurning ha here is no pah from s o when here is such a pah. Proof. If here is no pah, he algorihm reurns he correc answer. If here is a pah, he algorihm errs only if i does no find a pah wihin 4 seps of he walk. The expeced ime o reach is bounded by he cover ime of he componen ha conains s and. Tha is, by 4 < 2. By Markov s inequaliy, he probabiliy ha he walk needs more han 4 seps o reach from s is a mos /2. QED