Algebraic Multigrid as Solvers and as Preconditioner

Ò Algebraic Multigrid as Solvers and as Preconditioner Domenico Lahaye domenico.lahaye@cs.kuleuven.ac.be http://www.cs.kuleuven.ac.be/ domenico/ Department of Computer Science Katholieke Universiteit Leuven Celestijnenlaan 2 A B-31 Heverlee Belgium

Structure of the Presentation Motivation for Iterative solvers Stationary Iterative Solvers Multigrid Methods Algebraic Multigrid Methods Krylov Subspace Methods Algebraic Multigrid Methods as Preconditioner of the Presentation Domenico Lahaye Å Structure

Motivation for Iterative solvers 2D Eliptic partial differential equation x ( ν u x ) y ( ν u y ) = f Discretization by e.g. finite difference or finite elements Resulting system of linear equation A x = b A discrete differential operator x and b unknown function u and f in discrete points respectively dim(a) large (acurate discretizations) A x = b has to be solved repeatedly (parameter studies) Fast and memory efficient solution procedures required for Iterative solvers Domenico Lahaye Å Motivation

Motivation for Iterative solvers (2) Example: Numerical models for electrical motors A = (,, u) B = A H = ν B computation of forces and torques A x = b dim(a) 1 5 1 6 solved in time stepping or optimization loop for Iterative solvers (2) Domenico Lahaye Å Motivation

Motivation for Iterative solvers (3) Iterative methods Given x, compute {x m } m 1 such that e m = x x m if m stationary methods e k = M k e multigrid methods domain decomposition methods non-stationary methods Krylov subspace methods Direct methods Rely on a factorization of A Matrix-vector multiply as computational kernel sparsity exploited A v in O(N) flops, N = dim(a) for Iterative solvers (3) Domenico Lahaye Å Motivation

Motivation for Iterative solvers (4) Representative sparsity structure of A 2 4 6 8 1 12 14 16 18 2 4 6 8 1 12 14 16 18 123 for Iterative solvers (4) Domenico Lahaye Å Motivation

Basic Iterative Schemes for A x = b Matrix splitting A = M N with M 1 v easy to compute Iterative scheme x m+1 = M 1 N x m + M 1 b (iterand) e m+1 = (I M 1 A) e m (error) A = D L U Jacobi method M JAC = D (diagonal) Gauss-Seidel method M GS = D L (triangular) Successive relaxation M SOR = 1 ω D L ω > Asymptotic rate of convergence given by ρ(i M 1 A) or I M 1 A Iterative Schemes for A x = b Domenico Lahaye Å Basic

Basic Iterative Schemes for A x = b (2) Model Problem Analysis Continuous problem: u = f on Ω = [,1] [,1] + boundary conditions Discretization on uniform mesh, mesh width h, using central finite differences Linear system A x = b with [A] = 1 h 2 1 1 4 1 1 Ω h = Iterative Schemes for A x = b (2) Domenico Lahaye Å Basic

Basic Iterative Schemes for A x = b (3) Convergence history of lexicografic Gauss-Seidel (fixed mesh) 1 4 for a fixed mesh (fixed h ) initial stage: fast convergence later stage: stalling convergence Residual 1 3 1 2 1 1 1 2 3 4 5 6 Number of iterations Iterative Schemes for A x = b (3) Domenico Lahaye Å Basic

Basic Iterative Schemes for A x = b (4) Convergence history of lexicografic Gauss-Seidel (varrying mesh size) 1 5 1 4 for finer meshes ( h ) I M 1 A = 1 O(h 2 ) increasing number of iterations Residual 1 3 1 2 1 1 1 h = 1/32 h = 1/16 1 1 h = 1/8 1 2 1 2 3 4 5 6 Number of iterations Iterative Schemes for A x = b (4) Domenico Lahaye Å Basic

Basic Iterative Schemes for A x = b (5) Slow convergence is caused by smooth error components Initial error After 5 iterations After 1 iterations.5.35.25.4.3.2.1.3.25.2.15.1.2.15.1.1.5.5.5.2 1.8.6.4.2.2.4.6.8 1.1 1.8.6.4.2.2.4.6.8 1.5 1.8.6.4.2.2.4.6.8 1 Smooth error components can be represented on coarser grids Iterative Schemes for A x = b (5) Domenico Lahaye Å Basic

Multigrid Methods exploit PDE background of the linear problem multigrid methods = smoother (basic iterative scheme) + coarse grid correction two-grid scheme apply add apply xk k+1 h pre smoothing correction post x h smoothing project on coarser grid interpolate to fine grid compute coarse grid correction multigrid scheme by recursive application to solve coarse grid problem Methods Domenico Lahaye Å Multigrid

Multigrid Methods (2) Geometric multigrid methods A h x h = b h on Ω h = A H x H = b H on Ω H = construct restriction I H h : Ωh Ω H interpolation I h H : ΩH Ω h coarse grid correction K h,h = I h I h H (AH ) 1 I H h Ah smoother S h = (I (Q h ) 1 A h ) Q h splitting of A h multigrid iteration matrix M h,h (ν 1, ν 2 ) = (S h 2 )ν 2 K h,h (S h 1 )ν 1 Methods (2) Domenico Lahaye Å Multigrid

Multigrid Methods (3) Geometric multigrid methods (2) Different orders in visiting coarser grids different cycles (V-, W-, F-cycle) M h,h multigrid iteration matrix I (M h,h ) 1 A = C C small indepent of h mesh-width independent (h-independent) convergence Methods (3) Domenico Lahaye Å Multigrid

Multigrid Methods (4) multigrid methods require a hierarchy of grids this hierarchy can be cumbersome to construct example left: Ω H =??? Methods (4) Domenico Lahaye Å Multigrid

Algebraic Multigrid Methods Automatic construction of Ω H and A H x H = b H Using information contained in A h only and no information on differential equation geometry of Ω h notion of strength of coupling between grid points exploited Strength of coupling influences the smoother Illustration for anisotropic diffusion equation diffusion equation with discontinuous diffusion coefficient Multigrid Methods Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (2) Anisotropic Diffusion Equation Equation ǫ 2 u x 2 2 u = f with ǫ 1 on [,1] [,1] y2 Finite difference discretization on Ω h A h x h = b h with [A] = 1 h 2 1 ǫ 4 ǫ 1 weak coupling in x-direction, strong coupling in y-direction after point Gauss-Seidel smoothing, error is smooth in y-direction oscilatory in x-direction coarsening feasable in y-direction only! Multigrid Methods (2) Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (3) Initial error After 5 iterations After 1 iterations.5.4.3.4.3.2.1.1.3.2.1.1.2.3.2.1.1.2.2 1.4 1.3 1.8.6.4.2.2.4.6.8 1.8.6.4.2.2.4.6.8 1.8.6.4.2.2.4.6.8 1 Ω h = Ω H = Multigrid Methods (3) Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (4) Problem with discontinuous diffusion coefficient 1. x ( ν u x ) y ( ν u y ) = f with ν = ν = 1 ν = 1 6.625.375. after point Gauss-Seidel smoothing, error oscilatory accros interface of discontinuity coarsening not feasable close to the interface Multigrid Methods (4) Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (5) Initial error After 5 iterations After 1 iterations.5.35.35.4.3.3.3.25.25.2.2.2.1.15.15.1.1.1.5.5.2 1 1 1.8.6.4.2.2.4.6.8 1.8.6.4.2.2.4.6.8 1.8.6.4.2.2.4.6.8 1 Ω h = Ω H = Multigrid Methods (5) Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (6) geometric multigrid construct hierarchy of coarser grids smoother adapted to constructed hierarchy work invested in the smoother e.g. using line smoothers algebraic multigrid simple, fixed smoother: point Gauss-Seidel coarse grid correction adapted to the local properties of the smoother work invested in the coarse grid correction Multigrid Methods (6) Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (7) Setup phase construction of splitting Ω h = C h F h and set Ω H = C h construction of interpolation I h H strength of coupling between nodes coded in A h exploited matrix dependent interpolation: (I H h ) ij A h ij /Ah ii Galerkin coarsening: A H = I H h Ah I h H apply recursively using A H as input Solve phase multigrid cycling Multigrid Methods (7) Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (8) Permanent Magnet Machine Test Case Multigrid Methods (8) Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (9) Adaptive construction of triangulation in 13 refinement steps Nodes in triangulation 1 5 1 4 2 4 6 8 1 12 Adaptive refinement step Multigrid Methods (9) Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (1) Initial and intermediate triangulation Multigrid Methods (1) Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (11) Number of iterations for AMG Number of iterations 16 14 12 1 8 6 4 2 V(1,1) V(2,2) W(1,1) multigrid behavior 2 4 6 8 1 12 Adaptive refinement step Multigrid Methods (11) Domenico Lahaye Å Algebraic

Algebraic Multigrid Methods (12) CPU-time measurements for AMG 72 6 One level solver AMG CPU time (in secs.) 48 36 24 RAMG speedup by factor 7.5 12 5 1 15 2 Number of grid points Multigrid Methods (12) Domenico Lahaye Å Algebraic

Krylov Subspace Methods Non-stationary iterative methods for solving A x = b Given x and r = b A x find x m x + K m (A, r ) = span { r, A r,..., A m 1 r } Given V m = [v 1,..., v m ] a basis for K m (A, r ) y m a vector of m coefficients x m = x + V m y m basis V m Arnoldi method bi-lanczos method Lanczos method vector y m residual projection methods residual norm minimization No Krylov method optimal for large class of problems Subspace Methods Domenico Lahaye Å Krylov

Krylov Subspace Methods (2) The Arnoldi Method for constructing V m Initial vector v 1 = r / r Given V m = [v 1,..., v m ] a basis for K m (A, r ) compute A v m and orthogonalize with respect to V m normalize and set resulting vector equal to new basis vector v m+1 recurrence relation A V m = V m+1 H m+1,m H m+1,m = long term recurrences: expensive in memory and CPU-time ( H m,m...h m+1,m H m,m upper Hessenberg ) Subspace Methods (2) Domenico Lahaye Å Krylov

Krylov Subspace Methods (3) The Bi-Orthogonal Lanczos Method for constructing V m Avoid long term recurrences using bi-orthogonality V T m W m = D m = diag[d 1,..., d m ] W m basis for K m (A T, r ) recurrence relation A V m = V m+1 T m+1 T m+1 tridiagonal short term recurrences (near) breakdown if v T i w i = ( v T i w i ) Subspace Methods (3) Domenico Lahaye Å Krylov

Krylov Subspace Methods (4) Determing the vector y m x m x + K m (A, r ) r m = b A x m = r A V m y m residual projection methods V T m r m = H m,m y m = β 1 e 1 m m linear system: solve by LU-factorization residual minimization methods minimize r m H m+1,m y m = β 1 e 1 (m + 1) m linear system: solve in least-squares sense Subspace Methods (4) Domenico Lahaye Å Krylov

Krylov Subspace Methods (5) A x = b with A symmetric, positive definite A = A T H m,m = (H m,m ) T H m,m tridiagonal A pos. def. H m,m = V T m A V m pos. def. LU-fact. without pivoting update LU-fact. from step m to m + 1 coupled 2-term recurrences {r m 1, p m 1, r m, p m } { r m+1, p m+1 } p T i A p j = for i j conjugate gradient method (CG) Subspace Methods (5) Domenico Lahaye Å Krylov

Krylov Subspace Methods (6) Convergence of Krylov Subspace Methods Krylov subspace methods converge superlinearly Bound for CG x x m A 2 [ cond2 (A) 1 cond2 (A)+1] m x x A Preconditioning: A x = b M 1 A x = M 1 b M 1 A I M z = r easy to solve Preconditioned CG: m replace r m by z m = M 1 r m Possible preconditioners for CG: M JAC, M SSOR,... Subspace Methods (6) Domenico Lahaye Å Krylov

Krylov Subspace Methods (7) The Convergence of CG: A closer look CG: m T m y m = β 1 e 1 T m = V T m A V m eigenvalues of T m are called Ritz-values Ritz-values converge to spec(a m ) for increasing m fast converges towards outliers in spec(a m ), if any cfr. Krylov subspace methods for eigenvalue computations convergence of Ritz values governs convergence of CG Subspace Methods (7) Domenico Lahaye Å Krylov

AMG as Preconditioner Problem with anti-periodic boundary conditions A z Γ1 = A z Γ2 Number of iterations Adaptive classical classical refinement step AMG AMG/CG 148 15 1 div 16 2 div 17 3 div 17 4 div 18 5 div 17 6 div 17 as Preconditioner Domenico Lahaye Å AMG

AMG as Preconditioner (2) convergence of the Ritz values towards spec(i M 1 A ) during CG-iteration Ritz values of (I inv(m)*a) 1.9.8.7.6.5.4.3.2.1.1 1 2 3 4 5 6 7 8 9 1 11 12 13 14 15 AMG/CG Iteration step spec(i M 1 A ) has a few outliers outliers are captured fast by CG-iteration as Preconditioner (2) Domenico Lahaye Å AMG

AMG as Preconditioner (3) Permanent Magnet Machine Test Case as Preconditioner (3) Domenico Lahaye Å AMG

AMG as Preconditioner (4) Number of iterations for AMG Number of iterations 16 14 12 1 8 6 4 2 V(1,1) V(2,2) W(1,1) V(1,1)/CG CG accelerates and stabilizes convergence 2 4 6 8 1 12 Adaptive refinement step as Preconditioner (4) Domenico Lahaye Å AMG

AMG as Preconditioner (5) CPU-time measurements for AMG 72 6 SSOR/CG RAMG RAMG/CG CPU time (in secs.) 48 36 24 12 AMG speedup by factor 7.5 AMG/CG speedup by factor 15 5 1 15 2 Number of grid points as Preconditioner (5) Domenico Lahaye Å AMG

Conclusions We presented multigrid and algebraic multigrid for two-dimensional diffusion problems. Algebraic multigrid methods allow to solve problems on complicated geometries. Algebraic multigrid methods are more efficient than single-level iterative solvers. Using algebraic multigrid as a precondioner improves its stability and speed. Domenico Lahaye Å Conclusions