Algebraic multigrid and multilevel methods A general introduction Yvan Notay ynotay@ulbacbe Université Libre de Bruxelles Service de Métrologie Nucléaire May 2, 25, Leuven Supported by the Fonds National de la Recherche Scientifique, Maître de recherches http://homepagesulbacbe/ ynotay ntroduction Large sparse discrete PDE systems A u = b terative methods accelerated by preconditioning: easily invertible B such that B A Multigrid & multilevel methods: often very efficient Basic principle two-grid: obtain fast the convergence by solving a smaller problem, on a coarser grid Recursive use: the coarse grid problem is solved using the same two-grid preconditioner This seminar: emphasis on algebraic methods that work using only the information in A Algebraic multigrid and multilevel methods p/66 Algebraic methods: field of application Robust for scalar elliptic PDEs with standard discretization Emphasis on theory for symmetric problems self-adjoint PDEs, but work in unsymmetric cases as well eg convection diffusion problems Ongoing research for systems of PDEs efficient preconditioning of each diagonal block Does not work well for indefinite problems some eigenvalues with negative real part; eg: Helmholtz Remark: AMG is the generic name of a family of methods, but also the specific name of Ruge & Stüben method Algebraic multigrid and multilevel methods p3/66 Outline An introductory example 2 Needed ingredients: algebraic coarsening and algebraic interpolation Algebraic multigrid and multilevel methods p2/66 3 The different schemes and their algebraic properties 4 Algebraic interpolation 5 Algebraic coarsening: standard from AMG and aggregation 6 hecking & correcting the coarsening 7 From two- to multi-level: cycling strategies 8 Some numerical illustrations Algebraic multigrid and multilevel methods p4/66
An example PDE: u = 2 e x 52 +y 5 2 in Ω =,, u = on Ω Uniform grid with mesh size h, five-point finite difference An idea Fine grid system to solve: A u = b oarse grid auxiliary system: 9 8 7 6 5 4 3 2 8 6 4 2 2 4 6 8 Solution with h = 5 Solution with h = 25 How it works Error on the fine grid after interpolation 9 8 7 6 5 4 3 2 8 6 4 2 2 4 6 8 Algebraic multigrid and multilevel methods p5/66 A u = b u may be computed and prolongated by interpolation on the fine grid: u = p u u may serve as initial approximation, ie, one solves A u + x = b or A x = b A p A b Let us repeat Algebraic multigrid and multilevel methods p6/66 A u + x = b or A x = b A p A b = r x 3 3 25 2 Restrict on the coarse grid: 5 5 r = r r 5 5 2 Solve on the coarse grid: 8 6 4 2 u u u = 9 2 4 6 8 Algebraic multigrid and multilevel methods p7/66 3 Prolongate: x 2 = A r x 2 = p x 2, u 2 = u + x 2 Algebraic multigrid and multilevel methods p8/66
Still working? Error controlled through residual Error on the fine grid after interpolation Repeating the process nitial residual rhs After coarse grid correction x 3 3 25 2 5 5 5 x 3 35 3 25 2 5 5 5 2 5 5 25 2 5 5 5 5 2 5 8 6 4 2 u u u = 9 2 4 6 8 8 6 4 2 u u 2 u = 8 2 4 6 8 8 6 4 2 2 4 6 8 8 6 4 2 b A p A b b = 742 2 4 6 8 Explanation Assume for simplicity that b = r b One has etc Similarly u u u u 2 r p A r has rank n ρ A p A = u p A r b = p A r A u, = p A r A 2 u, = b A p A r b = A p A r r Algebraic multigrid and multilevel methods p9/66 r = ρ p A r A Algebraic multigrid and multilevel methods p/66 Smoother enters the scene u u and r very oscillatory improve u with a simple iterative method, efficient in smoothing the error & residual Example: symmetric Gauss-Seidel SGS Algebraic multigrid and multilevel methods p/66 L u +/2 = b A L u, L = lowa U u 2 = b A U u +/2 U = uppa Same as u 2 = u + M r, M = L D U D = diaga Thus: u u 2 = M A u u r 2 = A M r One may repeat: r m+ = A M m r Algebraic multigrid and multilevel methods p2/66
Smoothing effect Smoothing + coarse grid correction Residual after G correction Adding SGS step Adding now a G correction and again SGS step 25 2 x 4 2 5 5 3 6 5 5 5 2 8 6 4 2 2 4 6 8 = b Adding 3 SGS steps 5 4 2 742 5 8 6 2 4 6 8 Adding 8 SGS steps b = 39 2 2 3 8 6 4 2 2 4 6 8 5 4 3 2 2 8 6 4 2 2 4 6 8 45 4 35 3 25 2 25 2 5 r previous = 746 r previous = 55 5 5 5 8 6 4 2 What we learned 2 4 6 8 = b For each coarse grid correction: u u m+ 5 4 2 8 8 = p A r A u u m 6 2 4 6 8 b = 2 Algebraic multigrid and multilevel methods p3/66 annot work alone because ρ p A r A For each smoothing step u u m+ = M A u u m Not efficient alone because ρ M A However ρ M A p A r A M A Rmk: if A = A T, we assume M = M T Algebraic multigrid and multilevel methods p5/66 How it works 2 5 5 3 25 2 5 5 5 8 x 3 8 6 6 4 4 2 2 nitial residual 2 4 6 8 2 8 6 4 2 2 4 6 8 6 4 2 Algebraic multigrid and multilevel methods p4/66 multigrid step 2 multigrid steps 4 multigrid steps 2 4 6 8 = b 77 5 4 2 8 6 4 2 2 4 x 6 8 6 2 2 4 4 6 6 8 8 b = 39 3 b = 7 7 Algebraic multigrid and multilevel methods p6/66
Some remarks Geometric multigrid Simple in its principles omplicate to analyze Not robust: simple ideas not always lead to efficient schemes There is a lot of research works on multigrid applications Algebraic multigrid More user friendly black box More robust sacrificing somewhat on efficiency Two-grid AMG as a preconditioner AMG as preconditioner v = B AMG r computed as t = M r ; w = r A t 2 y = w + J T F w F 3 Solve  z = y 4 z F = J F z 5 v = t + z + M w A z Algebraic multigrid and multilevel methods p7/66 B AMG A = M A p  pt A M A Algebraic multigrid and multilevel methods p9/66 Algebraic multigrid: ingredients oarsening: F/ partitioning of the unknowns J F nterpolation J F and prolongation p = satisfying p e = e For the restriction, one often takes r = β J TF = β p T with β such that r e = e For A one may rely on the Galerkin approximation: A = r A p or  = p T A p with coarse grid correction given by p  pt p  pt A = p  pt 2 A projector Multilevel is not multigrid! oarse grid correction: p  pt with p = Let s try an additive complement q Q qt with q = Q q T A q = A orresponding preconditioner: B HBBD = J F Q  J F Algebraic multigrid and multilevel methods p8/66 J T F Algebraic multigrid and multilevel methods p2/66
Hierarchical finite element bases Additive two-level Matrix in hb_tl:  = J T A J Two-grid with additive complement: Q B HBBD = where J F B HBBD =  Q  J T F Finite element matrices are better conditioned whenever expressed in the hierarchical basis Algebraic multigrid and multilevel methods p2/66 = J A  B HBBD J T, which is the block diagonal part of  Further, B HBBD A = J B HBBD J T J T  J = J B HBBD J, Algebraic multigrid and multilevel methods p23/66 Generalized hierarchical basis n finite element applications with regular refinement, J F J = performs the basis transformation hb_tl nb hb_tl: coarse nodal basis 2 h + compl functions h Matrix in this basis:  = J T A J = = J T F A A A F A F +J T F A A F A J F A F +A J F  The strengthened BS constant We assume A symmetric and positive definite Definition γ = v= v F max, w= w Algebraic multigrid and multilevel methods p22/66 v T  w v T  v /2 w  w /2 Property f  = l Âl and if, l, γ l is such that, for all v =, w = then: v F w /2 /2 v T  l w γ l v T  l v w w Âl, γ max l γ l γ may often be bounded away from Algebraic multigrid and multilevel methods p24/66
The strengthened BS constant cont Let  = One has A  F κ D   F Â, D = A  = λ max D  λ min D  = + γ γ Let S A = A A F A A F =  ÂF A One has  λ min S A = γ 2, λ max  S A Two-level block factorization, ÂF Algebraic multigrid and multilevel methods p25/66 Two-grid HBBD preconditioning Preconditioning by HBBD v = B HBBD y F = Q r F r computed as 2 y = r + J T F r F 3 Solve  v = y 4 z F = J F v 5 v F = z F + y F B HBBD = J F = q Q qt Q  κ + γ γ J T F + p  pt Algebraic multigrid and multilevel methods p26/66 Two-level block factorization cont  = = A  F  F   F A  F Q = B HBBF B HBBF A SA Q  = J B HBBF J T A ÂF Q ÂF Algebraic multigrid and multilevel methods p27/66 Preconditioning by HBBF v = B HBBF y F = Q r F r computed as 2 y = r A F y F + J T F r F A y F 3 Solve  v = y 4 z F = J F v 5 v F = z F + Q r F A F v A z F B HBBF = J Q ÂF Q  κ γ 2 Q ÂF J T Algebraic multigrid and multilevel methods p28/66
Two-level hierarchical basis multigrid Preconditioning by HBMG v = B HBMG y F = Q r F r computed as 2 y = r A F y F + J T F r F A y F 3 Solve  v = y 4 z F = J F v + y F 5 v F = z F + Q r F A F v A z F B HBMG = J Q ÂF 2 Q Q A Q Block factorization without hb κ γ 2  Algebraic multigrid and multilevel methods p29/66 Hierarchical basis & multigrid Elementary algebra yields with B HBMG A = R A p  pt A R A Reminder: R = Q B AMG A = M A p  pt A M A Algebraic multigrid and multilevel methods p3/66 Block factorization without hb cont A = = A A F A F A A F A A F PF F = B MBF A SA P  A A F P A F Preconditioning by MBF v = B MBF r computed as y F = PF F r F 2 y = r A F y F 3 Solve  v = y 4 v F = P r F A F v B MBF = PF F A F PF F  κ γ 2 Possibly unstable! A F P Algebraic multigrid and multilevel methods p3/66 Algebraic multigrid and multilevel methods p32/66
Block factorization without hb cont B HBBD = q Q qt + p  pt B HBBF = q Q q T + p  pt, B HBMG B MBF = q Q + Q T Q A Q T Q p = A F + Q A J F = q P qt + p  pt, PF F p = A F q T + p  pt, p has to define a correct interpolation Algebraic multigrid and multilevel methods p33/66 orrect interpolation Essential requirement: PF F p = A F good for low energy modes, ie vectors v such that Av Scalar elliptic PDEs: one such vector: T e = f one satisfies the row-sum criterion then P e F = A e F, A e F + A F e P A F e e F Algebraic multigrid and multilevel methods p35/66 Block factorization without hb cont PF F A B MBF A h λ min λ max κ λ min λ max κ MLU preconditioning of A 6 2 2 5 25 245 32 2 2 5 27 254 64 2 2 5 29 258 28 2 2 5 29 258 LU preconditioning of A 6 88 9 24 5 42 278 32 88 9 24 38 228 6 64 87 9 25 76 497 283 28 87 9 25 58 5 258 Algebraic multigrid and multilevel methods p34/66 orrect interpolation cont f A is a symmetric M-matrix with nonnegative row-sum SPD with nonpositive offdiagonal entries, several results available For instance: A = J T A J with satisfies J = γ PF F A F κp A Algebraic multigrid and multilevel methods p36/66
Algebraic analysis of AMG HBBF, HBMG: κ γ ; MBF: κ 2 γ! 2 AMG Assumption: 2 M A SPD or, equivalently, ρ M A < One has where µ = max z κ B AMG A µ z F J F z T X z F J F z z T A z with X being the top left block of X = M 2 M A M, Algebraic analysis of AMG cont Further, γ µ 2 λ min XF F A γ 2 λ min MF F A 2 λ max M A Example: SGS smoothing: λ max M A = γ µ 2 λ min MF F A γ 2 γ 2 Algebraic analysis of AMG cont Quality of the interpolation measured with τ = max z D = diaga There holds τ τ λ min DF F A max Algebraic multigrid and multilevel methods p37/66 z F J F z T D z F J F z z T A z γ 2 λ max D A γ, 2 λ min DF F A What we learned All methods work or fail together Algebraic multigrid and multilevel methods p38/66 They are relatively equivalent with respect to algebraic analysis except additive HBBD However they mimic geometric methods that behave differently in a multigrid or multilevel context The F/ partitioning has to be such that A is well conditioned The interpolation J F has to be such that γ is away from MBF needs special care; it does not require explicitly J F, but  needs to be provided Algebraic multigrid and multilevel methods p39/66 Algebraic multigrid and multilevel methods p4/66
Algebraic interpolation onsider A F A A A F A F A A A F A SA = Algebraic interpolation cont Direct interpolation in AMG for M-matrices with nonnegative row-sum: j i A ij A F ij A ii A F ij if a ij strong j J F ij = a ij strong if a ij weak Block diagonal γ = A A F is the ideal algebraic interpolation However: Â = J TF A J F has to remain sparse J F Algebraic interpolation cont Algebraic multigrid and multilevel methods p4/66 Possible improvement: take also into account indirect couplings J F less sparse Essentially positive-type matrices with nonnegative row-sum: split A = A M + A P where offdiaga P = maxo, offdiaga and A P e = ; apply previous scheme to A M ; the bound on τ depends now on κa M A General case: no obvious solution so far if A is not weakly diagonally dominant Property: τ max i F Reminder: τ γ 2 Algebraic coarsening j i A ij A F ij j a ij strong Algebraic multigrid and multilevel methods p42/66 Standard coarsening in AMG First classify the negative couplings in strong and weak, according some given threshold Next, repeat, till all nodes are marked either coarse or fine : select an unmarked node as next coarse grid node, according to some priority rule designed so as to favor a regular covering of the matrix graph; 2 select as fine grid nodes all nodes strongly negative coupled to this new coarse grid node Algebraic multigrid and multilevel methods p43/66 Algebraic multigrid and multilevel methods p44/66
AMG coarsening: how it works Aggregation Five-point stencil Nine-point stencil Algebraic multigrid and multilevel methods p45/66 Group nodes into aggregates G i partitioning of [, n] Possible prolongation { p : if i G j p ij = otherwise oarse grid matrix:  = p T A p given by  = a kl ij k G i l G j Optionally select a node in each aggregates; other nodes are then F nodes Associated interpolation: { if i G j i F, j : J F ij = otherwise Algebraic multigrid and multilevel methods p47/66 AMG coarsening: pros and cons Each F node is strongly negative coupled to at least node standard interpolation works Slow coarsening in case of low connectivity, anisotropy or strong asymmetry Too fast coarsening in case of high connectivity May be cured with aggressive coarsening Requires specialized interpolation The number of nonzero entries per row tends to grow from level to level May be sensitive to the Strong/Weak coupling threshold All in all, works reasonably in many cases Algebraic multigrid and multilevel methods p46/66 Example: pairwise aggregation Definition: S i = { j i a ij < β max aik < a ik } nitialization: F = ; = ; U = [, n] ; For all i : m i = { j U i S j } Algorithm: While U do select i U with minimal m i 2 select j U such that a ij = min k U a ik 3 if j S i : 3a = {j}, F = F {i}, G j = {i, j}, U = U\{i, j} 3b update: m k = m k for k S i and k S j otherwise: 3a = {i}, G i = {i}, U = U\{i} 3b update: m k = m k for k S i Algebraic multigrid and multilevel methods p48/66
Double pairwise aggregation Algorithm: Apply simple pairwise aggregation to A Output: F,, and G i, i 2 ompute the auxiliary matrix A =, i, j with a ij = k G i l G j a ij a kl 3 Apply simple pairwise aggregation to A Output: F 2, 2, and G 2 i, i 2 4 = 2, F = F F 2, G i = 2 j G G j, i i Example d = Algebraic multigrid and multilevel methods p49/66 Double pairwise aggregation First coarse grid Second coarse grid Example 2D problem with anisotropy & discontinuity Five-point finite difference approx uniform mesh of 2 u a x x a 2 u 2 y y 2 { = f in Ω =,, u = on y =, x = elsewhere on Ω u n a x = d, a y =, f = in 65, 95 5, 65 a x =, a y = d, f = in 25, 45 25, 45 a x = d, a y = d, f = in 5, 25 65, 95 a x =, a y =, f = elsewhere, where d is a parameter Some remarks Algebraic multigrid and multilevel methods p5/66 Geometric multigrid does not benefit from semi-coarsening A may be badly conditioned has to be compensated by specialized smoothers n c = 3794, n n c = 383, nz n c = 546 n c = 25, n n c = 42, nz n c = 62 Algebraic multigrid and multilevel methods p5/66 Geometric schemes fix the coarsening and the interpolation; the smoother the approximation to A is adapted to the problem Algebraic schemes fix the smoother the approximation to A ; the coarsening is adapted to the problem With algebraic schemes, the adaptation is automatic Algebraic multigrid and multilevel methods p52/66
Aggregation: pros and cons ontrol of the coarsening speed nsensitive to the Strong/Weak coupling threshold Maintain the sparsity in coarse grid matrices, that are nevertheless reasonable, up to some scaling factor The interpolation that is naturally associated with aggregation is bad not an issue for MBF-based methods Smoothed aggregation: optionally sparsify A into Ã, in such a way that Ae = Ãe ; then: p sm agg = ω D Ã p agg where D = diagã hecking the F/ partitioning A has to be well conditioned This may be a posteriori checked ompatible relaxation AMG Algebraic multigrid and multilevel methods p53/66 Perform smoothing on a random rhs while frozing the values at variables f the error at F variables does not decay quickly, adapt the partioning by moving to some of the slowly convergent F variables Remark Amounts to check the conditioning of MF F A Remember that κ AMG λ min MF F A 2 λ max M A Algebraic multigrid and multilevel methods p55/66 llustration 4 35 3 25 2 5 5 Performance of AMG and MBF with aggregation Problem 3: d = Agg AMG, h = 5 Agg AMG, h = Agg MBF, h = 5 Agg MBF, h = 5 2 25 3 35 4 4 35 3 25 2 5 5 Problem 3: d = same legend as for the left figure 5 2 25 3 35 4 Relative solution cost vs scaling of the coarse grids Algebraic multigrid and multilevel methods p54/66 hecking the F/ partitioning cont Dynamic MLU The size of the pivots in a modified LU P e F = A e F factorization is a good indication of the conditioning For instance, in some cases, letting P = L Q U with diagl = diagu = Q, if Q ξdiaga for some ξ > 2, then κp A 2 ξ Algebraic multigrid and multilevel methods p56/66
Dynamic MLU: algorithm Repeat=False 2 reinitialize: Q = diaga, L = lowera, U = uppera 3 for k =,, n, k F : if q kk γ a kk : eliminate row & column k in A according to the MLU algorithm otherwise: F = F \{k}, = {k} ; Repeat=True 4 f Repeat, GoTo, possibly decreasing the value of γ From two- to multi-level Exploit recursively the same ideas Algebraic multigrid and multilevel methods p57/66 Succession of grids levels, each with its own F/ partitioning and interpolation J F, and also with its ideal preconditioner in which the matrix at the coarser level is inverted exactly At some point the coarse grid matrix in indeed small enough to be factorized exactly At every other level, the ideal preconditioner is adapted, exchanging the exact solution to Âv = y for an approximate solution Approximate Âv = y with application of the preconditioner: V cycle inner iterations: W cycle Algebraic multigrid and multilevel methods p59/66 Dynamic MLU: example Double pairwise aggregation, second coarse grid Without dynamic MLU With dyanmic MLU n c = 25, n n c = 42, nz n c = 62 n c = 24, n n c = 29, nz n c = 594 From two- to multi-level cont Algebraic multigrid and multilevel methods p58/66 W cycles may be based on fixed point iterations, but Krylov G, GMRES is more robust Then: Except at the coarsest level, the so defined preconditioner is slightly variable from step to step Flexible Krylov subspace methods FG, FGMRES nner iterations are exited when the relative residual error is less than 35, or when the number of iterations reaches int[nza/nzâ] Algebraic multigrid and multilevel methods p6/66
From two- to multi-level cont AMG: often efficient with V cycle simplicity, consistency with slow coarsening The use of V cycle is based on experiment and mimicry of geometric schemes it may be not robust to rely on V cycle Block factorization methods: require W cycle geometric schemes do require it too need coarsening fast enough A non self-adjoint 3D problem Algebraic multigrid and multilevel methods p6/66 Seven-point FD approx upwind scheme of ν u + v u = in Ω =,,, { u = on z =, x, y u = elsewhere on Ω vx, y, z = 2 x x2 y z 2 x y y 2 x 2 y z z ; ν = corresponds to the Laplace equation Uniform mesh with constant mesh size h Stretched mesh: refined in such a way that the ratio of maximum mesh size to minimum mesh size is equal to 2, the ratio of subsequent mesh sizes being constant Algebraic multigrid and multilevel methods p63/66 Numerical results previous problem MBF with aggregation & dynamic MLU sol = d ost of resolution ost of unprec G iter 28 for geom multigrid on model problems h = 6 h = 2 n n n c inner iter sol n c inner iter sol 399 76 2 67 4 76 2 688 2 397 2 2 698 399 2 23 829 4 396 24 24 84 398 2 25 93 395 24 24 842 398 24 26 94 2 395 24 24 825 398 2 26 96 4 395 96 26 88 398 24 27 956 6 395 25 26 922 398 2 3 79 Algebraic multigrid and multilevel methods p62/66 3D problem: numerical results grid 2 2 2 grid n n ν n c inner iter sol n c inner iter sol Uniform mesh 4 2 5 796 4 2 5 82 376 2 7 97 38 2 7 83 2 375 2 8 97 384 94 8 7 4 393 2 2 369 393 2 2 368 6 393 2 26 672 396 2 3 23 Stretched mesh 39 94 6 793 394 88 7 858 39 94 6 8 394 88 7 858 2 392 65 2 955 395 94 7 853 4 346 8 2 7 364 87 23 258 6 364 2 27 7 328 2 27 863 Algebraic multigrid and multilevel methods p64/66
Some references Many textbooks on multigrid, but few address algebraic schemes U Trottenberg, W Oosterlee, and A Schüller Multigrid Academic Press, London, 2 is recommended for a general introduction to multigrid; it contains in appendix the best available review on AMG: K Stüben An ntroduction to Algebraic Multigrid n Trottenberg et al, 2 Appendix A Other results in research papers Let mention mine! Algebraic multigrid and algebraic multilevel methods: a theoretical comparison Aggregation-based algebraic multilevel preconditioning see homepage for details and download Algebraic multigrid and multilevel methods p65/66 PhD Fellowship Area: numerical nuclear reactor simulation ollaboration between ULB and Framatome ANP Location: Framatome ANP GmbH in Erlangen, Germany main European research center of the group with periodical stays in Brussels Task: adaptation of advanced preconditioned iterative techniques to nuclear reactor simulation Please contact me for further information ynotay@ulbacbe Algebraic multigrid and multilevel methods p66/66