Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes

Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes Elena Virnik, TU BERLIN Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.1/19

Overview Two Real World Problems: 1. Model of the blood circulation in a human body 2. Google TM Matrix - determining PageRank TM Introduction to Markov chains Problem setting and solution approaches Introduction to algebraic multigrid (AMG) Some numerical tests Summary and outlook Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.2/19

Real World Problem: Biotech Blood Circulation in a Human Body Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.3/19

Real World Problem: Biotech Blood Circulation in a Human Body Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.3/19

Real World Problem: Biotech Blood Circulation in a Human Body (I T T )x = 0 Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.3/19

Example of a blood flow matrix Entry t ij represents the probability that a blood particle passes from organ i to organ j within one time step BloodFlow 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.4/19

Google TM Matrix Example: S1 S2 S3 The link structure of the Internet is transformed into a column stochastic matrix as follows: S4 We search the web from a starting page following the rules: 1. With probability 0.85 follow a link on the page 2. With probability 0.15 choose a random site Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.5/19

Google TM Matrix Example: S1 S2 S3 The link structure of the Internet is transformed into a column stochastic matrix as follows: T T = S4 0.15 4 0.85 + 0.15 2 4 0.15 4 0.85 + 0.15 3 4 0.15 4 0.85 + 0.15 3 4 0.15 4 0.15 4 0.15 4 0.85 + 0.15 2 4 0.85 + 0.15 2 4 0.15 4 0.85 + 0.15 2 4 0.85 + 0.15 3 4 0.85 + 0.15 1 4 0.15 4 Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.5/19

Google TM Matrix Example: S1 S2 S3 The link structure of the Internet is transformed into a column stochastic matrix as follows: S4 The eigenvector x to the eigenvalue 1 is the PageRank TM. T T x = x Here, the solution is x = (0.26, 0.29, 0.12, 0.33) T. S4 is the site with the highest PageRank. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.5/19

Introduction to Markov Chains [Markov Chain] (Named after Andrei Markov) A process is called finite homogeneous Markov chain (MC) if s 1,..,s n are the states of the process P[s i s j ] =: t ij is time independent Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.6/19

Introduction to Markov Chains [Markov Chain] (Named after Andrei Markov) A process is called finite homogeneous Markov chain (MC) if s 1,..,s n are the states of the process P[s i s j ] =: t ij is time independent The matrix T = (t ij ) i,j=1,..,n satisfies t ij 0, n t ij = 1, i,j = 1,..n j=1 and is called (state) transition matrix. In many applications T is singular. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.6/19

Probability Distribution The probability distribution vector x k = (x k i ) is defined by x k i = probability that the system is in state s i after k steps Properties: x k i 0 and n i=1 xk i = 1 for each k. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.7/19

Probability Distribution The probability distribution vector x k = (x k i ) is defined by x k i = probability that the system is in state s i after k steps Properties: x k i 0 and n i=1 xk i = 1 for each k. Q.: What is the probability distribution at time k? Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.7/19

Probability Distribution The probability distribution vector x k = (x k i ) is defined by x k i = probability that the system is in state s i after k steps Properties: x k i 0 and n i=1 xk i = 1 for each k. Q.: What is the probability distribution at time k? A: Let M = (T,x 0 ). Then for each k > 0 (x k ) T = (x k 1 ) T T = (x 0 ) T T k. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.7/19

Is there a stable state? A distribution vector x is said to be stationary if x T T = x T. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.8/19

Is there a stable state? A distribution vector x is said to be stationary if How to compute it? x T T = x T. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.8/19

Is there a stable state? A distribution vector x is said to be stationary if How to compute it? x T T = x T. Iterate (x k ) T = (x k 1 ) T T = (x 0 ) T T k If x := lim k (x 0 ) T T k exists, then it is stationary. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.8/19

Is there a stable state? A distribution vector x is said to be stationary if How to compute it? x T T = x T. Iterate (x k ) T = (x k 1 ) T T = (x 0 ) T T k If x := lim k (x 0 ) T T k exists, then it is stationary. Problem: Does lim k (x 0 ) T T k always exist? Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.8/19

Is there a stable state? A distribution vector x is said to be stationary if How to compute it? x T T = x T. Iterate (x k ) T = (x k 1 ) T T = (x 0 ) T T k If x := lim k (x 0 ) T T k exists, then it is stationary. Problem: Does lim k (x 0 ) T T k always exist? Answer:NO! Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.8/19

Is there a stable state? A distribution vector x is said to be stationary if How to compute it? x T T = x T. Iterate (x k ) T = (x k 1 ) T T = (x 0 ) T T k If x := lim (x 0 ) T T k exists, then it is stationary. k Problem: Does lim ] [ ] 1 0 Example: T = (x 0 ) T T k always exist? Answer:NO! [ k [ ] 0 1 0 1, T 1 0 2 =, T 0 1 3 =. But 1 0 ] [ ] 1 0 1 2 = [ ] 1 1 1 0 2 2 there is a stable distribution [ 1 2 Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.8/19

Is there a stable state? A distribution vector x is said to be stationary if How to compute it? x T T = x T. Iterate (x k ) T = (x k 1 ) T T = (x 0 ) T T k If x := lim k (x 0 ) T T k exists, then it is stationary. Problem: Does lim k (x 0 ) T T k always exist? Answer:NO! Transform fixed point equation into a linear system: Ax := (I T T )x = 0 Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.8/19

How do we solve (I T T )x = 0? Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.9/19

How do we solve (I T T )x = 0? 1. Solvability of a singular system: Ax = b is solvable if b im(a). The system is then called consistent. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.9/19

How do we solve (I T T )x = 0? 1. Solvability of a singular system: Ax = b is solvable if b im(a). The system is then called consistent. our system is consistent! Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.9/19

How do we solve (I T T )x = 0? 1. Solvability of a singular system: Ax = b is solvable if b im(a). The system is then called consistent. our system is consistent! 2. Possible approaches to solve Ax = (I T T )x = 0: Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.9/19

How do we solve (I T T )x = 0? 1. Solvability of a singular system: Ax = b is solvable if b im(a). The system is then called consistent. our system is consistent! 2. Possible approaches to solve Ax = (I T T )x = 0: Direct methods (e.g. Gauss, LU) Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.9/19

How do we solve (I T T )x = 0? 1. Solvability of a singular system: Ax = b is solvable if b im(a). The system is then called consistent. our system is consistent! 2. Possible approaches to solve Ax = (I T T )x = 0: Direct methods (e.g. Gauss, LU) Iterative methods Splitting methods (e.g. Jacobi, SOR) Krylov subspace methods (e.g. CG, GMRES) AMG Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.9/19

Direct solvers Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.10/19

Direct solvers LU works despite singularity if A is an M-Matrix Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.10/19

Direct solvers LU works despite singularity if A is an M-Matrix A of the form A = si B, s > 0, B 0 where s ρ(b) is called M-Matrix. If s > ρ(b) then A is nonsingular M-Matrix s = ρ(b) then A is singular M-Matrix Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.10/19

Direct solvers LU works despite singularity if A is an M-Matrix A of the form A = si B, s > 0, B 0 where s ρ(b) is called M-Matrix. If s > ρ(b) then A is nonsingular M-Matrix s = ρ(b) then A is singular M-Matrix A = (I T T ) is a singular M-Matrix as ρ(t) = 1 Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.10/19

Direct solvers LU works despite singularity if A is an M-Matrix A of the form A = si B, s > 0, B 0 where s ρ(b) is called M-Matrix. If s > ρ(b) then A is nonsingular M-Matrix s = ρ(b) then A is singular M-Matrix A = (I T T ) is a singular M-Matrix as ρ(t) = 1 BUT: high computational effort - only useful for small n Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.10/19

Krylov subspace methods e.g GMRES Let A R n n, r R n. Then K m := {r,ar,a 2 r,..,a m 1 r} is the m-dimensional Krylov space. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.11/19

Krylov subspace methods e.g GMRES Let A R n n, r R n. Then K m := {r,ar,a 2 r,..,a m 1 r} is the m-dimensional Krylov space. Basic Idea of GMRES for solving Ax = b: Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.11/19

Krylov subspace methods e.g GMRES Let A R n n, r R n. Then K m := {r,ar,a 2 r,..,a m 1 r} is the m-dimensional Krylov space. Basic Idea of GMRES for solving Ax = b: Let r = b Ax 0 = Ax 0 be the initial residual Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.11/19

Krylov subspace methods e.g GMRES Let A R n n, r R n. Then K m := {r,ar,a 2 r,..,a m 1 r} is the m-dimensional Krylov space. Basic Idea of GMRES for solving Ax = b: Let r = b Ax 0 = Ax 0 be the initial residual Choose x such that x = x 0 + V y where columns of V orthogonally span K m so that b Ax 0 2 = r 2 is minimized. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.11/19

AMG as a preconditioner in GMRES The concept of a preconditioner M an easily invertible matrix such that M 1 A 1 solve à x = b, where à = M 1 A, and b = M 1 b instead of the original system. Goal: make M 1 A I. Idea: take as M 1 the solution from AMG iteration. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.12/19

Introduction to AMG Define coarse grids: a sequence of smaller and smaller systems of equations A m u m A 1 = b m, where = A The interpolation operator P and the restriction operator R are defined as follows: Let A 1 = A R n n. Then R : R n R N P : R N R n. It is A 2 = RA 1 P and A 2 R N N. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.13/19

AMG structure The AMG consists of two main parts: Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.14/19

AMG structure The AMG consists of two main parts: setup phase solution phase Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.14/19

AMG structure The AMG consists of two main parts: setup phase choose coarser grid F,C define grid transfer P,R define coarse grid operators A H = RA h P solution phase Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.14/19

AMG structure The AMG consists of two main parts: setup phase choose coarser grid F,C Undirected graph of A define grid transfer P,R Coarse grid nodes Fine grid nodes define coarse grid operators A H = RA h P solution phase Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.14/19

AMG structure The AMG consists of two main parts: setup phase choose coarser grid F,C define grid transfer P,R define coarse grid operators A H = RA h P solution phase Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.14/19

AMG structure The AMG consists of two main parts: setup phase choose coarser grid F,C define grid transfer P,R Restriction: Interpolation: define coarse grid operators A H = RA h P solution phase Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.14/19

AMG structure The AMG consists of two main parts: setup phase choose coarser grid F,C define grid transfer P,R define coarse grid operators A H = RA h P solution phase Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.14/19

AMG structure The AMG consists of two main parts: setup phase choose coarser grid F,C define grid transfer P,R define coarse grid operators A H = RA h P solution phase : V-cycle smoothing R smoothing R P coarse grid correction smoothing P smoothing Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.14/19

Recursive application Essentiality for setting up a multi-level cycle: Properties of the finest grid operator should carry over to the coarsest grid. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.15/19

Recursive application Essentiality for setting up a multi-level cycle: Properties of the finest grid operator should carry over to the coarsest grid. J.W.Ruge and K.Stueben ([4]) show this for symmetric, weakly diagonally dominant M-Matrices, Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.15/19

Recursive application Essentiality for setting up a multi-level cycle: Properties of the finest grid operator should carry over to the coarsest grid. J.W.Ruge and K.Stueben ([4]) show this for symmetric, weakly diagonally dominant M-Matrices, In our case the main properties are the row sum zero property and the singular M-matrix property. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.15/19

Numerical tests 1.Blood circulation: GMRES AMG+GMRES AMG n nnz TIME ITS grid TIME ITS TIME 108 232 0,06 31 1,18 0,11 2 1,29 221 446 0,21 46 1,44 0,06 2 1,5 324 676 0,35 68 1,9 0,17 2 2,07 777 1592 1,08 123 3,02 0,17 2 3,19 1227 2498 5,16 245 4,26 0,58 2 4,84 Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.16/19

Numerical tests 2.Blood circulation - nearly decoupled: ǫ ǫ ǫ... ǫ...... ǫ ǫ ǫ ǫ is taken to be 10 6 in the following computation. ǫ Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.16/19

Numerical tests 2.Blood circulation - nearly decoupled: GMRES AMG+GMRES AMG n TIME ITS grid TIME ITS TIME 432 1,46 153 2,67 0,29 7 2,96 884 4,74 241 3,73 0,46 7 4,19 1296 14,15 361 5,78 1,03 9 6,79 3108 91,70 626 13,40 6,99 9 20,39 4908 >317 >1000 29,87 64,48 7 94,35 Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.16/19

Numerical tests 3.Google TM Matrix: A 500 nodes example starting at http://www.tu-berlin.de 0 50 100 150 200 250 300 350 400 This matrix represents the link structure. An entry t ij represents a link from page j to page i. The right eigenvector corresponding to the eigenvalue 1 is the PageRank. 450 500 0 50 100 150 200 250 300 350 400 450 500 nz = 5783 Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.16/19

Numerical tests 3.Google TM Matrix: A 500 nodes example starting at http://www.tu-berlin.de PageRank In Out URL 0.0554 243 13 http://www.tu-berlin.de 0.0260 204 30 http://www.ub.tu-berlin.de/index.html 0.0195 187 30 http://www.ub.tu-berlin.de 0.0175 184 40 http://www.ub.tu-berlin.de/index_kontakt.html 0.0160 183 121 http://www.ub.tu-berlin.de/index_index.html 0.0159 183 72 http://www.ub.tu-berlin.de/service/standorte.html 0.0159 183 52 http://www.ub.tu-berlin.de/benutzerfuehrer/allgemeines/oeffnungs 0.0157 183 37 http://www.ub.tu-berlin.de/index_neuigkeiten.html 0.0149 167 50 http://www.ub.tu-berlin.de/service/index.html Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.16/19

Summary and Outlook AMG+GMRES was tested on some Bloodflow matrices Nearly decoupled Markov chains (Bloodflow) Google TM Matrix Generally, AMG preconditioner accelerates the convergence of GMRES for larger matrices For smaller matrices and for random Markov chains, the preconditioner does not accelerate convergence To-do s: Convergence estimate for singular M-matrices Application to other areas Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.17/19

Bibliography [1] S. Bächle. Numerische Berechnung der stationären Wahrscheinlichkeiten von Markov Ketten mit dem GMRES Verfahren. Master s thesis, Universität Karlsruhe (TH), 2002. [2] A. Berman and R.J. Plemmons. Nonnegative Matrices in the Mathematical Sciences. Academic Press, New York, 1979. [3] I-wen Kuo. A note on factorizations of singular m-matrices. Linear Algebra and its Applications, 197/198:511 530, 1977. [4] J.W. Ruge and K. Stueben. Algebraic multigrid. Multigrid Methods, pages 73 130, 1987. [5] Richard S. Varga and Da-Yong Cai. On the LU factorization of M-matrices. Numerische Mathematik, 38:179 192, 1981. Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.18/19

List of Slides 1. Overview 2. Real World Problem: Biotech 3. Example of a blood flow matrix 4. Google TM Matrix 5. Introduction to Markov Chains 6. Probability Distribution 7. Is there a stable state? 8. How do we solve (I T T )x = 0? 9. Direct solvers 10. Krylov subspace methods e.g GMRES 11. AMG as a preconditioner in GMRES 12. Introduction to AMG 13. AMG structure 14. Recursive application 15. Numerical tests 16. Summary and Outlook 17. Bibliography 18. List of Slides Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes p.19/19