OUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative methods ffl Krylov subspace methods ffl Preconditioning techniques: Iterative methods ILU

Transcription

1 Preconditioning Techniques for Solving Large Sparse Linear Systems Arnold Reusken Institut für Geometrie und Praktische Mathematik RWTH-Aachen

2 OUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative methods ffl Krylov subspace methods ffl Preconditioning techniques: Iterative methods ILU Sparse Approximate Inverse (SPAI) Domain Decomposition ffl Problem dependent preconditioning for (Navier-)Stokes equations

3 Model problems Poisson equation: ( u = f in Ω ρ R N ; u = 0 ; Convection-diffusion equation ( ν u + b(x) r u = f in Ω ρ R N ; u = 0 ; System of pde's: (Navier)-Stokes equation.

4 Discretization of elliptic BVP FE, FV or FD method! Ax = b ; A 2 R n n Properties: ffl n large: n ο h 2 or ο h 3 ( h : mesh size) ffl A sparse : nnz(a) ο n ffl Poisson equation: A symmetric positive definite Convection-diffusion equation: A is an M-matrix ffl FD method: A has a systematic block structure ffl cond(a) " 1 for h # 0 ffl A = LU : unacceptable fill-in Iterative methods Basic iterative methods: Jacobi, Gauss-Seidel, SOR. Krylov subspace methods: CG, GMRES, BiCGSTAB,.... Multigrid methods.

5 Matrix splitting methods A = M N ; where Λ M is nonsingular; Λ for arbitrary y we can solve Mx = y with relatively low costs: Iterative method: Mx k+1 = Nx k + b ; k 0 Examples: For A = D L U : M = D : Jacobi M = D L : Gauss-Seidel M = 1 D L,! 2 (0; 2) : SOR! For the error e k = x k x Λ we have Iteration matrix C = I M 1 A = M 1 N e k+1 = (I M 1 A)e k : Note: kek k 1 k ke 0 k» kc k k 1 k (sharp) and lim k!1 kck k 1=k = ρ(c) :

6 For efficiency of the method one should consider: ffl arithmetic costs per iteration ffl rate of convergence, ρ(c) ffl properties w.r.t. parallelization Jacobi method: - costs per iteration: ++ - parallelization: ++ - rate of convergence: Lemma. Let A be a symmetric weakly diagonally dominant matrix with a i;j» 0 for all i 6= j. Then: 1 2 cond(d 1 A)» ρ(i D 1 A)» 1 1 cond(d 1 A) : Example: Poisson equation. Ω = (0; 1) 2. Standard FD. h : mesh size # : number of iterations for kek k ke 0 k ß h 1/40 1/80 1/160 1/320 #

7 Krylov subspace methods The Conjugate Gradient method. Assumption: A is SPD. Krylov subspace: K k (A; r) := spanfr; Ar; A 2 r;:::;a k 1 rg : r 0 = b Ax 0. In CG method: x k 2 x 0 + K k (A; r 0 ); < x k x Λ ; z > A = 0 for all z 2 K k (A; r 0 ), Ax k b? K k (A; r 0 ) Φ ΦΦ 3 x Λ x 0 Φ ΦΦ K k Φ ΦΦ (A; r 0 ) Φ ΦΦ - Φ ΦΦ x k x 0 Φ ΦΦ the right angle is w.r.t. the inner product < ; > A! projection method".

8 CG Algorithm 8 x 0 a given starting vector; r 0 = b Ax 0 >< >: for k 0 (if r k 6= 0) : p k = r k <rk ;Ap k 1 > <p k 1 ;Ap k 1 > pk 1 ( if k = 0 then p 0 := r 0 ) x k+1 = x k + ff opt (x k ; p k )p k with ff opt (x k ; p k ) = <pk ;r k > <p k ;Ap k > r k+1 = r k ff opt (x k ; p k )Ap k Arithmetic costs per iteration: ++ Parallelization: + Rate of convergence: kx k x Λ k A» 2 ψp cond(a) 1 p cond(a)+1! k kx 0 x Λ k A Example: Poisson equation.ω = (0; 1) 2. Standard FD. h : mesh size # : number of iterations for kek k ke 0 k ß h 1/40 1/80 1/160 1/320 #

9 CG is a nonlinear method with superlinear convergence phenomena: ^ρ k := kxk x Λ k A kx k 1 x Λ k A BiCG type of methods for non-spd problems Determine x k 2 x 0 + K k (A; r 0 ) such that Ax k b? K k (A T ; r 0 ) : Similar algorithm. Breakdowns / instabilities may occur. No convergence theory.

10 Preconditioning Idea: consider W 1 Ax = W 1 b, W 1 2AW 1 2 ~x = W 1 2b Determine W ß A with: Wx = y can be solved with "low" computational costs Wx = y can be solved efficiently on a parallel machine cond(w 1 A) < cond(a) Preconditioning based on a linear iterative method Linear iterative method: Preconditioner: W := M i.e., x k+1 = x k M 1 (Ax k b): Wx = y, x = M 1 y one iteration with starting vector 0 applied to Az = y. Lemma Let A and M be SPD matrices with ρ(c) < 1; C := I M 1 A : Then cond(m 1 A)» 1+ρ(C) 1 ρ(c) Note: x! 1+x 1 x increases monotonically on [0; 1) : Examples: Jacobi, (S)SOR, Multigrid.

11 Incomplete LU factorization methods A = LU Graph of the matrix A: Sparsity pattern S: row wise LU factorization. For i = 2;:::;n For k = 1;:::;i 1 := a ik =a kk ; a ik := ; For j = k + 1;:::;n a ij := a ij a kj : G(A) := f (i; j) j 1» i; j» n and a ij 6= 0 g: f (i; i) j 1» i» n g ρ S ; G(A) ρ S : Incomplete row wise LU factorization. For i = 2;:::;n For k = 1;:::;i 1 If (i; k) 2 S then := a ik =a kk ; a ik := ; For j = k +1;:::;n If (i; j) 2 S then a ij := a ij a kj :

12 Results in incomplete factorization A = ~ L ~ U + R: Preconditioner: W = ~ L ~ U. Remarks: ffl existence / stability not guaranteed ffl ~ L and ~ U are sparse: ~L i;j = 0 for (i; j) =2 S ffl Wx = y can be solved with acceptable costs ffl sparsity pattern can chosen adaptively (thresholding) ffl w.r.t. parallelization: ffl very popular preconditioning technique Example: Poisson equation. Ω = (0; 1) 2. Standard finite differences. h : mesh size # : number of iterations for kek k ke 0 k ß h 1/40 1/80 1/160 1/320 ICCG, # MICCG, #

13 SPAI preconditioner Frobenius norm : kak 2 F := P n i;j=1 a2 i;j. S := f A 2 R n n j a i;j = 0 for (i; j) =2 S g Determine N 2 S such that ki NAk F = min ki NAk F N2S Let N k be the k-th row of N. Due to ki NAk 2 F = nx k=1 ke T k N k Ak 2 2 ; The minimization problem decouples into n independent low-dimensional least-squares problems. Preconditioner: W = N 1. Remarks: ffl N always exists ffl Wx = y, x = Ny (Matrix-vector multiplication) ffl W.r.t. parallelization: ++ ffl In many cases cond(w 1 SPAIA) > cond(w 1 ILUA)

14 Domain decomposition: additive Schwarz preconditioner Domain ο R n Subdomains ο R n i ; i = 1;:::;K Restrictions r i : R n! R n i Prolongations p i = r T i : R n i! R n Subdomain matrices A i = r i Ap i : R n i! R n i Additive Schwarz method: x k+1 = x k KX i=1 p i A 1 i r i (Ax k b) Linear iterative method! preconditioner Note: ffl Block-Jacobi type of method ffl Favourable properties w.r.t. parallelization

15 Problem dependent preconditioning Navier-Stokes equations: ν u + (u r)u + rp = f in Ω ; div u = 0 in Ω ; + Boundary conditions. Linearization: ν u + (a r)u + rp = g in Ω; div u = 0 Structure of discrete problem: L B B T ; uh p h b1 = ;, Ax = b L ο ν h + a r h ο decoupled convection-diffusion equations B ο r h

16 Schur complement technique Ax = b, Sp h = B T L 1 b 1 S = B T L 1 B Lu h = b 1 Bp h For Stokes-equations: ffl A is symmetric indefinite ffl S is SPD ffl cond(s) = O(1)! ffl Inner-outer iteration: Λ CG for Sp h = B T L 1 b 1 Λ efficient solver for Lx = y (Laplace equation)

17 For Navier-Stokes-equations: ffl A is nonsymmetric ffl S is nonsymmetric ffl Inner-outer iteration: Λ BiCG type of method for Sp = B T L 1 b 1 Λ robust solver for Lx = y (convection-diffusion equation) ffl Preconditioner for S needed: open problem!