Multigrid Methods and their application in CFD Michael Wurst TU München 16.06.2009 1
Multigrid Methods Definition Multigrid (MG) methods in numerical analysis are a group of algorithms for solving differential equations They are among the fastest solution techniques known today 16.06.2009 2
Outline 1. Typical design of CFD solvers 2. Methods for Solving Linear Systems of Equations 3. Geometric Multigrid 4. Algebraic Multigrid 5. Examples 16.06.2009 3
Different CFD solvers Typical design of CFD solver CFD solver Coupled solver Segregated solver 16.06.2009 4
SIMPLE Algorithm Typical design of CFD solver Preprocessing segregated, sequential solution of decoupled transport equations Momentum Equations pressure correction equation: a tight tolerance for guaranteeing mass conservation time loop Non-linear loop (outer iterations) Pressure Correction Equation Turbulence Equations other equations... Convergence Multigrid methods Last timestep Postprocessing 16.06.2009 5
Coupled Solution Algorithm Typical design of CFD solver Preprocessing momentum equations and pressure correction equation are such discretized that one gets a big coupled block equation system this equation system becomes very large fast solver necessary time loop Non-linear loop (outer iterations) Momentum Equations Pressure Correction Equation Turbulence Equations other equations... Convergence Multigrid methods Last timestep Postprocessing 16.06.2009 6
Coupled Solution Algorithm Typical design of CFD solver Big coefficient matrix consisting of the momentum matrixes, the pressure correction matrix and coupling matrixes The solution vector contains velocity componentes and pressure r r A 0 0 A uu up u qu 0 A 0 A r v r q vv vp v r = r 0 0 Aww A w q wp w A A A A r r p q Pu Pv Pw PP p 16.06.2009 7
Basic Definitions Methods for Solving Linear Systems of Equations Linear System of Equation: A: sparse matrix of size n n, symmetric, pos. diagonal elements, non-positive off diagonal elements (M-Matrix) u: exact solution Au = f aiju j = fi v: approximation to the exact solution Two measures of v as an approximation to u: (Absolute) error: e = u - v Residual: r = f Av Measured by norms: L norm: e max e L 2 - norm: = j 1 j n e 2 = n 2 ej j = 1 1 2 16.06.2009 8
Direct vs. Iterative Methods Methods for Solving Linear Systems of Equations Direct methods i.g. Gauss elimination / LU decomposition solve the problem to the computational accuracy high computational power Iterative methods / Relaxation methods Gauss-Seidel / Jacobi relaxation Solve the problem only by an approximation could be sufficient and so be less time consuming 16.06.2009 9
Iterative methods Methods for Solving Linear Systems of Equations aiju j = fi Jacobi relaxation: 1 u = f a u ( n+ 1) ( n) i i ij j aii j i Gauss-Seidel relaxation: 1 u = f - a u - a u aii j<i j>i (n+1) (n+1) (n) i i ij j ij j 16.06.2009 10
Properties of Iterative methods Methods for Solving Linear Systems of Equations Example: Poisson equation -u'' = 0 u(0) = u(n)= 0 Discretisation: -u j-1 + 2u j - u j+1 = 0 2 h -u + 2u - u = 0 1 j n+1 j -1 j j+1 u 0 = u n = 0 Exact solution: u = 0 error e = u v = -v 16.06.2009 11
Properties of Iterative methods Methods for Solving Linear Systems of Equations Different starting values: Sinus waves: v j jkπ = sin n Fourier modes 16.06.2009 12
Properties of Iterative methods Methods for Solving Linear Systems of Equations Error vs. Number of iteration 16.06.2009 13
Properties of Iterative methods Methods for Solving Linear Systems of Equations Realistic starting value: 1 jπ 6jΠ 32jΠ v j = sin + sin + sin 3 n n n k = 1: low frequency wave k = 6: medium frequency wave k = 32: high frequency wave 16.06.2009 14
Properties of Iterative methods Methods for Solving Linear Systems of Equations Error: written in eigenvectors of A: (0) e = n 1 ckw k k= 1 Eigenvectors correspond to the modes of the problem Our problem: 1 k n 2 w k,j jkπ = sin n 1 k n 1 n 2 k n 1 Low frequency modes Do not dissappear Smoother High frequency modes Disappear 16.06.2009 15
Improvements of iterative solvers Geometric Multigrid Idea: Have a good initial guess How? Do some preliminary iterations on a coarse grid (grid with less points) Good, because iterations need less computational time How does an error look like on a coarse grid? It looks more oscillatory! 16.06.2009 16
Improvements of iterative solvers Geometric Multigrid How does an error look like on a coarse grid? If error is smooth on fine grid, maybe good to move to coarse grid. 16.06.2009 17
Possible schemes for improvement Geometric Multigrid Nested iteration: Relax on Au = f on a very coarse grid to obtain an initial guess for the next finer grid... Ω h Ω 2 h Ω Ω h Relax on Au = f on 4 to obtain an initial guess for Relax on Au = f on to obtain an initial guess for Relax on Au = f on Ω h to obtain a final approximation to the solution. Ω 2h Problems: Relax on Au = f on? Last iteration: Error still smooth? 16.06.2009 18
2nd possibility: Use of the residual equation Au = f Au Av = f Av Ae = r 16.06.2009 19
Possible schemes for improvement Geometric Multigrid Correction scheme: Ω h Relax on Au = f on to obtain an approximation v h Compute the residual r = f - A v h Relax on the residual equation Ae = r on to obtain an approximation to the error e 2h Correct the approximation obtained on with the error estimate obtained on 2 : v v + e Ω h Ω h Ω 2h h h 2 h 2 Problems: Relax on Ae = r on? Ω h Ω 2h Ω h Transfer from to? 16.06.2009 20
Transfer operators Geometric Multigrid Transfer from coarse to fine grids: Interpolation / Prolongation Ω Ω 2h h Transfer from fine to coarse grids: Restriction h Ω Ω 2h 16.06.2009 21
Transfer operators Interpolation / Prolongation Geometric Multigrid Interpolation / Prolongation: from coarse to fine grid Points on fine and on coarse grid: Points only on the fine grid: h v = v 2 2 j h j h 1 2h 2h v 2 j + 1 = ( v j + v j + 1 ) 2 16.06.2009 22
Transfer operators Restriction Geometric Multigrid Restriction: from fine to coarse grid 2h h h h Full weigthening: v j = ( v 2 j 1 + 2v 2 j + v 2 j + 1) 1 4 16.06.2009 23
Properties of transfer operators Geometric Multigrid Interpolation / Prolongation h I 2h 1 2 1 1 2 1 1 1 = 2 2 1... 1 2 1 h I 2 h Restriction 1 2 1 1 2 1 1 = 1 2 1 4... 1 2 1 Variational property: h ( h ) h I = c I 2 2h T 16.06.2009 24
Properties of transfer operators Geometric Multigrid Transfer of vectors: Transfer of matrix A: h A A 2 h Geometric answer: A 2h is discretisation of the problem on the coarse grid 2h 2h h h Algebraic answer: A = I A I (Galerkin condition) h 2h 16.06.2009 25
Recapitulation Geometric Multigrid Iterative methods can effectively reduce high-oscillating errors until only a smooth error remains Smooth errors look less smooth on coarse grids Transfer of vectors and matrices from coarse to fine grids possible with two conditions: Galerkin condition 2 2 A = I A I h h h h h 2h Variational property h ( h ) h I = c I 2 2h T How can we put this in a good solution algorithm? 16.06.2009 26
V-Cycle Geometric Multigrid h h h Relax on A u = f ν times with initial guess v h 1 2h 2h h Compute f = I h r h h h Relax on A 2 u 2 = f 2 ν times with initial guess v 2h 1 Compute f 4h = I 4h 2h 2h r h h h Relax on A 4 u 4 = f 4 ν times with initial guess v 4h 1 Compute f 8h = I 8h r 4h Correct Solve A Lh u Lh = f Lh 4 Correct h 4 h 4 + h 8 v v I h 8hv Relax A 4h u 4h = f 4h times with initial guess v 4h Relax A 2h u 2h = f 2h times with initial guess v 2h Correct h h h Relax A u = f ν times with initial guess v h 2 4h v v + I v ν 2 2 h 2 h 2 h 4 h 4h ν 2 v v + I v h h h 2 h 2h 16.06.2009 27
V-Cycle Geometric Multigrid Relax h Restricition 2h 4h 8h Prolongation 16h 16.06.2009 28
Other cycles W Cycle Geometric Multigrid h 2h 4h 8h 16.06.2009 29
Other cycles Full Multigrid Cycle (FMG) Geometric Multigrid ν 0 -times ν 0 -times h 2h ν 0 -times 4h 8h 16.06.2009 30
Geometric vs. Algebraic multigrid Algebraic Multigrid Geometric Multigrid: structured meshes Problem: unstructured meshes, no mesh at all Algebraic Multigrid (AMG) Questions: 1) What is meant by grid now? 2) How to define coarse grids? 3) Can we use the same smoothers (Jacobi, Gauss-Seidel) 4) When is an error on a grid smooth? 5) How can we transfer data from fine grids to coarse grids or vice versa? 16.06.2009 31
Grid Algebraic Multigrid GMG: known locations of grid points well-defined subset of the grid points define coarse grid AMG: subset of solution variables form coarse grid Au = f u u1 u = 2 M un 16.06.2009 32
Smooth error Algebraic Multigrid Defined as an error which is not effectively reduced by an iterative method Jacobi method: e i+1 = ( I D 1 A) e i Measurement of the error with the A-inner product: e A = ( Ae,e) 1 2 Smooth error: ( I D A)e A e e D Ae A e A A 1 D A e A e A 16.06.2009 33
Smooth error Algebraic Multigrid Smooth error: 1 D A e A e A ( D 1 A e, A e) ( e, A e) ( D 1 r, r ) ( e, r ) n 2 r a n i i = 1 ii i = 1 r e i i r i a ii e i Ae 0 16.06.2009 34
Implications of smooth error Algebraic Multigrid ii i ij j j i Ae 0 a e + a e 0 a e a e ii i ij j j i 16.06.2009 35
Selecting the coarse grid - requirements Algebraic Multigrid Smooth error can be approximated accurately. Good interpolation to the fine grid. Should have substantially fewer points, so the problem on coarse grid can be solved with little expense. 16.06.2009 36
Selecting the coarse grid Influence and Dependence Algebraic Multigrid Definition 1: Given a threshold value 0 < θ 1, the variable (point) ui strongly depends on the variable (point) if: u j a θ max ij k i { a } ik Definition 2: u i u j u i If the variable strongly depends on the variable, then the variable strongly influences the variable. u j 16.06.2009 37
Selecting the coarse grid definitions Algebraic Multigrid Two important sets: S i : set of points that strongly influence i, that is the points on which the point i strongly depends. { : θ max{ }} S = j a a i ij ik k i T S i : set of points that strongly depend on the point i. { : j} T S = j i S i 16.06.2009 38
Selecting the coarse grid - Example Algebraic Multigrid Poisson equation: u = 0 -u i -1 + 2ui - u -u i+1 j-1 + 2ui - u j+1 2 + 2 = 0 h h 1 2 ( -u i -1 - u i+1 + 4u i - u j-1 - u j+1 ) = 0 h 4 9 14 19 24 3 8 13 18 23 i 2 7 12 17 22 1 6 11 16 21 0 5 10 15 20 16.06.2009 39 j
Selecting the coarse grid - Example Algebraic Multigrid Discretisation on 5x5 grid: For example, Point 12: 1 2 ( -u 7 - u 11 + 4u 12 - u 13 - u 17 ) = 0 h 4 9 14 19 24 3 8 13 18 23 2 7 12 17 22 1 6 11 16 21 a a a a a 12,7 12,11 12,13 12,17 12,12 = 1 = 1 = 1 = 1 = 4 0 5 10 15 20 { : θ max { }} S 12 = { 7, 11, 13, 17} S = j a a i ij ik k i { j} T S = j i S i T : S 12 = { 7, 11, 13, 17} 16.06.2009 40
Selecting the coarse grid - Example Algebraic Multigrid 1) Define a measure to each point of its potential quality as a coarse (C) T point: amount of members of λ i S i 4 9 14 19 24 2 3 3 3 2 3 8 13 18 23 3 4 4 4 3 2 7 12 17 22 3 4 4 4 3 1 6 11 16 21 3 4 4 4 3 0 5 10 15 20 2 3 3 3 2 16.06.2009 41
Selecting the coarse grid - Example Algebraic Multigrid 2) Assign point with maximum to C-point S i T 3) All points in become fine (F) points λ i 4) For each new F point j: increase the measeure for all each unassigned point k that strongly influence j: k S λ k j 2 3 3 3 2 2 3 4 F 3 3 4 4 4 3 3 5 F C F 3 4 4 4 3 3 4 5 F 4 3 4 4 4 3 3 4 4 5 3 2 3 3 3 2 2 3 3 3 2 5) Do 2)-4) until all points are assigned 16.06.2009 42
Selecting the coarse grid - Example Algebraic Multigrid 2 3 4 F 3 2 3 4 F 3 C F C F C 3 5 F C F 3 5 F C F F C F C F 3 4 5 F 4 3 4 6 F 5 C F C F C 3 4 4 5 3 3 5 F C F F C F C F 2 3 3 3 2 2 3 4 F 3 C F C F C 16.06.2009 43
Definition of transfer operators Algebraic Multigrid Interpolation: from coarse to fine grids e if i C = e if i F i h ( I2he) ω i ij j j C i Each fine grid point i can have three different types of neighboring points: The neighboring coarse grid points that strongly influence i The neighboring fine grid points that strongly influence i Points that do not strongly influence i, can be fine and coarse grid points This information is contained in ω ij 16.06.2009 44
Example Algebraic Multigrid Example: -au -cu + bu = xx yy xy Discretised with a two-dimensional mesh, divided into 4 parts; a=1000 c=1 b=0 a=1 c=1 b=2 a=1 c=1 b=0 a=1000 c=1 b=0 16.06.2009 45
Example Algebraic Multigrid Grid 2h 1 0.8 0.6 'Y' 0.4 0.2 0 0 0.2 0.4 0.6 0.8 1 'X' 16.06.2009 46
Example Algebraic Multigrid Grid 4h 1 0.8 0.6 'Y' 0.4 0.2 0 0 0.2 0.4 0.6 0.8 1 'X' 16.06.2009 47
Example Algebraic Multigrid Grid 8h 1 0.8 0.6 'Y' 0.4 0.2 0 0 0.2 0.4 0.6 0.8 1 'X' 16.06.2009 48
Advantages & Disadvantages of AMG Algebraic Multigrid 10 0 10-2 Normalized Residual 10-4 10-6 10-8 10-10 SIP (Iterations: 2410) V-Cycle (3 Grid-Levels) W-Cycle (3 Grid-Level) FMG-Cycle (3 Grid-Levels) 10-12 0 100 200 300 400 500 Number of Iterations 16.06.2009 49
Advantages & Disadvantages of AMG Algebraic Multigrid Advantages Fast and robust Good for segregated solvers (SIMPLE) Disadvantages The Galerkin Operation is a very expensive step Diffucult to parallelize High setup-phase High storage requirements Not for coupled solvers A cure are the aggregation based AMGs 16.06.2009 50
Aggregation based AMG Algebraic Multigrid In the simplest case strongly connected coefficient are simply summed up Example: II 18 19 20 21 22 23 I 13 14 15 16 17 12 7 8 9 10 11 12 1 2 3 4 5 6 cell 7 influences strongly cell 1 cell 2 influences strongly cell 1 build a new cell I from cell 1,2,7 do the same to get the new cell II 16.06.2009 51
Aggregation based AMG Algebraic Multigrid II 18 19 20 21 22 23 13 14 15 16 17 12 I 7 8 9 10 11 12 1 2 3 4 5 6 To get the coefficients of the new coarse linear equation system sum up A I,II =A 7,13 +A 7,8 +A 2,8 A I,II =A 13,7 +A 8,7 +A 8,2 A I,I =A 1,1 +A 2,2 +A 7,7 A II,II =A 8,8 +A 9,9 +A 13,13 +A 14,14 +A 15,15 +A 18,18 +A 19,19 +A 20,20 16.06.2009 52
Aggregation based AMG Algebraic Multigrid Advantages The Galerkin operation becomes a simple summation of coefficients The setup-phase becomes very fast The procedure is easy to parallelize Through giving maximum and minimum size of cells on coarser grids, one can pre-estimate memory effort in a finite volume method, the coefficients are representing flux sizes from one cell to another, through summation on keeps the conservativness of the discretized system over all coarser levels Disadvantages The convergence rate becomes small compared to original AMG, but in the case of solution of the non-linear Navier-Stokes equation the reduction of the residual within one outer iteration has not to be very tight, reducing of about one to two orders of magnitude suffices The Agglomeration AMG is ideally applicable to the coupled solution of Navier-Stokes Equation System 16.06.2009 53
Thank you! Discussion 16.06.2009 54