Inexactness and flexibility in linear Krylov solvers

Transcription

1 Inexactness and flexibility in linear Krylov solvers Luc Giraud ENSEEIHT (N7) - IRIT, Toulouse Matrix Analysis and Applications CIRM Luminy - October 15-19, 2007 in honor of Gérard Meurant for his 60 th birthday

2 The original title 1 Inexactness joint work with J. Langou (Unv. Colorado Denver) and S. Gratton (CNES, Toulouse) on inexact/relaxed GMRES (SISC-07) AV k + [ A 1 v 1,..., A k v k ] = V k+1 Hk 2 and flexibility in linear Krylov solvers joint work with S. Gratton and X. Pinel (CERFACS) on flexible GMRES with deflated restarting (ongoing) [AM 1 v 1,..., AM k v k ] = V k+1 Hk Converged title Convergence in backward error of inexact GMRES.

3 Outline 1 Introduction 2 3 4

4 Outline Introduction 1 Introduction L. Giraud 4/30 Inexactness and flexibility in linear Krylov solvers

5 The Arnoldi algorithm A an n n non singular matrix Krylov subspace K k (A, v 1 ) = span{v 1, Av 1,.., A k 1 v 1 } Arnoldi algorithm on A, starting with v 1 generates an orthonormal set of vectors v j such that AV k = V k+1 Hk, with V k = [v 1,..., v k ], H k upper-hessenberg. Breakdown of the algorithm when K k (A, v 1 ) is an A-invariant subspace. L. Giraud 5/30 Inexactness and flexibility in linear Krylov solvers

6 Inexact GMRES method Take the basic GMRES method, and perturb the matrix-vector products v k+1 = (A + A k )v k Easy way to control the inner accuracy. Why? The matrix is not known with full accuracy (Parameter estimation, Schur complement in non-overlapping DDM,...) Computing Ax with a poor accuracy is cheap (FMM) L. Giraud 6/30 Inexactness and flexibility in linear Krylov solvers

7 Inexact GMRES algorithm - MGS variant 1. x 0 initial guess, r 0 = b (A + A 0 )x 0, β = r 0 and v 1 = r 0 /β 2. For k=1,2,... Do 3. Compute w k = (A + A k )v k 4. For i=1,...,j, Do 5. h i,k = wk T v i 6. w k = w k h i,k v i 7. EndDo 8. h j+1,j = w k 9. If h j+1,j = 0 Goto v k+1 = w k /h j+1,j 11. enddo 12. Set-up the (m + 1) m matrix H m = (h i,j ) 1 i m+1,1 j m 13. Compute, y m argmin of βe 1 H my 14. Compute, x m = x 0 + V my m L. Giraud 7/30 Inexactness and flexibility in linear Krylov solvers

8 Outline Introduction 1 Introduction L. Giraud 8/30 Inexactness and flexibility in linear Krylov solvers

9 Historical developments Relaxed GMRES method - Consider the normwise backward error Ax b η A,b (x) =, and 0 < ε < 1. A x + b Numerous numerical illustrations in Bouras, Frayssé (SIMAX-05) that if a relaxed GMRES is run on a computer, using perturbations controlled ( so that min 1, max A k A ( ε, ε Ax k 1 b )), the GMRES iterate x k reaches for some k n a backward error η A,b (x k ) less than ε. L. Giraud 9/30 Inexactness and flexibility in linear Krylov solvers

10 Some properties of the BF criterion BF criterion A k A ( ( )) ε min 1, max ε, Ax k 1 b, Often a pure relaxation criterion in practice ( Ax k b is decreasing along the iterations). Never perform perturbations A k A smaller than the target backward error ε, greater than 1, criterion weakness : knowledge of the exact Ax k 1 b required, scaling issues... L. Giraud 10/30 Inexactness and flexibility in linear Krylov solvers

11 Exact relations in the inexact algorithm Exact arithmetic assumed From the Gram-Schmidt process follows the inexact Arnoldi relation AV k + [ A 1 v 1,..., A k v k ] = V k+1 Hk Least squares y k = argmin H k y r 0 e 1 True residual r k = b Ax k Computed residual r k = V k+1 ( r0 e 1 H k y k ). The norm rk is readily available from the incremental solution of the least squares min H k y r 0 e 1 L. Giraud 11/30 Inexactness and flexibility in linear Krylov solvers

12 Exact relations in the inexact algorithm Exact arithmetic assumed From the Gram-Schmidt process follows the inexact Arnoldi relation AV k + [ A 1 v 1,..., A k v k ] = V k+1 Hk ( A 1,..., A k ) Least squares y k = argmin H k y r 0 e 1 True residual r k = b Ax k Computed residual r k = V k+1 ( r0 e 1 H k y k ). The norm rk is readily available from the incremental solution of the least squares min H k y r 0 e 1 L. Giraud 12/30 Inexactness and flexibility in linear Krylov solvers

13 Inexact GMRES algorithm as an exact GMRES on a perturbed matrix Simoncini, Szyld (SISC-03) and Van den Eshof, Sleijpen (SIMAX-04) define G k = [ A 1 v 1,..., A k v k ]. The inexact Arnoldi reads (A + G k V T k )V k = V k+1 Hk. The computed residuals norm r k are non increasing, if then r k r k ɛ A k σ min( H m)ɛ r k 1 Information on the exact residual obtained from r m r m + r m r m L. Giraud 13/30 Inexactness and flexibility in linear Krylov solvers

14 Inexact GMRES algorithm as an exact GMRES on a perturbed matrix Simoncini, Szyld (SISC-03) and Van den Eshof, Sleijpen (SIMAX-04) define G k = [ A 1 v 1,..., A k v k ]. The inexact Arnoldi reads (A + G k V T k )V k = V k+1 Hk. The computed residuals norm r k are non increasing, if there exists a family of matrices A k such that A k σ min( H m( A 1,... A m))ɛ r k 1, k m, then r k r k ɛ Information on the exact residual obtained from r m r m + r m r m L. Giraud 14/30 Inexactness and flexibility in linear Krylov solvers

15 Next steps Introduction What we would like to get 1 Remove the dependency among the A i, 2 Control the possible singularity of H m, 3 Design an implementable algorithm that can reach a prescribed backward error accuracy. η A,b (x k ) = min {τ > 0 : A τ A, b τ b A, b and (A + A)x k = b + b} = Ax k b A x k + b, η b (x k ) = min {τ > 0 : b τ b and Ax k = b + b} = Ax k b. b b L. Giraud 15/30 Inexactness and flexibility in linear Krylov solvers

16 Control the possible singularity of H m If A k c σ min(a) n for 0 < c < 1 V m+1 Hm = (A + G m V T m)v m σ min (A + G m V T m) σ min ( H m ) with G m V T m = m i=1 A iv i v T i m i=1 A iv i v T i cσ min (A) then 0 < (1 c)σ min (A) σ min ( H m ) because σ min (A) G m V T m σ min (A + G m V T m). 1 With such perturbations, the r k will be monotonically decreasing until happy breakdown where it will be zero. 2 Replacing the constraint on the perturbation size based on σ min ( H m ) by the more stringent bound (1 c)σ min (A) still ensures the former result on the residual gap. L. Giraud 16/30 Inexactness and flexibility in linear Krylov solvers

17 Convergence of relaxed GMRES for η b Theorem η b (x k ) r k r k b } {{ } ε g + r k b }{{} ε c Let us denote by m the step where the breakdown occurs in the inexact GMRES algorithm. Let c be such that 0 < c < 1 and let ε c and ε g be any positive real numbers. Assume for all k m, A k 1 ( n σ min(a) min c, ) (1 c) b ε g. (1) r k 1 Then there exists l, 0 < l m, such that the following stopping criterion is satisfied r l ε c b (2) and η b (x l ) ε c + ε g.

18 Convergence of relaxed GMRES for η A,b Theorem η A,b (x k ) r k r k A x k + b + r k A x k + b Let us denote by m the step where the breakdown occurs in the inexact GMRES algorithm. Let c, x 0 and x be such that 2c x 0 x and 0 < c < 1. Let ε c and ε g be any positive real numbers. Suppose that for all k m A k 1 ( n σ γ ) min(a) min c, (1 c) r k 1 ε g, (3) where γ = 1 4+2ε cκ(a) A x + b. There exists l, l m, such that the following stopping criterion is satisfied r l ε c A x l (4) and η A,b (x l ) ε c + ε g.

19 Outline Introduction 1 Introduction L. Giraud 19/30 Inexactness and flexibility in linear Krylov solvers

20 The matrices are generated using the rand Matlab command Strategy S A k = σ min(a) 4n ( 3γ ) min 1, 2 r k 1 ε Strategy S b A k = σ min(a) 4n ( 3γ b ) min 1, 2 r k 1 ε Simpler to implement but more stringent in term of perturbation size L. Giraud 20/30 Inexactness and flexibility in linear Krylov solvers

21 Implemented algorithm Relaxed GMRES with strategy S 1: Choose a convergence threshold ε = ε c + ε g 2: Choose an initial guess x 0 3: r 0 = b Ax 0 ; β = r 0 4: v 1 = r 0 / r 0 ; 5: for k = 1, 2,... do 6: z = (A + A k )v k, A k being such that strategy S holds 7: for i = 1 to k do 8: h i,k = v T i z 9: z = z h i,k v i 10: end for 11: h k+1,k = z 12: v k+1 = z/h k+1,k 13: Solve the least-squares problem min βe 1 H k y for y k 14: if r k = βe 1 H k y k ε c A x k then 15: Set x k = x 0 + V k y k 16: Exit 17: end if 18: end for

22 10 0 Full GMRES with ILU(0.1) ε = 1e b.e. for S* perturbation size for S* b.e. for S b perturbation size for S b b.e. for exact GMRES PDE225 n = 225 K 2 (A Prec ) = A Prec 2 = Figure: Relaxed GMRES with strategy S and S b - PDE225 - ε = L. Giraud 22/30 Inexactness and flexibility in linear Krylov solvers

23 10 0 Full GMRES with ILU(0.001) ε = 1e b.e. for S* perturbation size for S* b.e. for S b perturbation size for S b b.e. for exact GMRES UTM300 n = 300 K 2 (A Prec ) = e+07 A Prec 2 = Figure: Relaxed GMRES with strategy S and S b - UTM300 - ε = L. Giraud 23/30 Inexactness and flexibility in linear Krylov solvers

24 Backward stability of GMRES Related papers J. Drkošová, M. Rozložník, Z. Strakoš and A. Greenbaum, Numerical stability of the GMRES method, BIT, vol. 35, p , C.C. Paige, M. Rozložník and Z. Strakoš, Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES, SIAM J. Matrix Anal. Appl., vol. 28 (1), p , Design of some relaxations heuristics Heuristic S(ε) A k = ε A Exact GMRES run in a floating point arithmetic with machine precision ε Heuristic S (ε) A k = max ε A, σ ««min(a) 3γ min 1, 4n r k 1 εg Heuristic S b (ε) A k = max ε A, σ ««min(a) 3γ b min 1, 4n r k 1 εg

25 Heuristics matrix n t ε N ex N ε Nε Nε b e05r e05r GRE GRE GRE CAVITY PDE SAYLR UTM WEST BFW398A Table: # iterations of GMRES with various strategies. L. Giraud 25/30 Inexactness and flexibility in linear Krylov solvers

26 Relaxed FMM for 3D Maxwell Solution PhD Dissertation J. Langou (EADS-CERFACS), Parallel out-of-core FMM code - EADS-IW 10 0 Cetaf without precond (0 o,90 o ) precfmm 3 precfmm 2 precfmm L. Giraud 26/30 Inexactness and flexibility in linear Krylov solvers

27 Outline Introduction 1 Introduction L. Giraud 27/30 Inexactness and flexibility in linear Krylov solvers

28 Similar results can be derived for GMRES with relaxed right-preconditioning, inexact initial residual. Relaxation/Inexactness for GMRES understood in exact arithmetic. Backward stability of relaxed/inexact GMRES for Householder in finite precision proved... MGS to be done. Implementation possible in many scientific computing simulations: Electromagnetism (FMM), domain decomposition (inexact local solvers), block preconditioners (inexact block solvers). L. Giraud 28/30 Inexactness and flexibility in linear Krylov solvers

29 Merci pour votre attention L. Giraud 29/30 Inexactness and flexibility in linear Krylov solvers

30 Merci pour votre attention Happy anniversaire Gérard L. Giraud 30/30 Inexactness and flexibility in linear Krylov solvers

31 Bibliography A. Bouras and V. Frayssé. Inexact matrix-vector products in Krylov methods for solving linear systems: a relaxation strategy. SIAM Journal on Matrix Analysis and Applications, 26(23): , A. Bouras, V. Frayssé, and L. Giraud. A relaxation strategy for inner-outer linear solvers in domain decomposition methods. Technical Report TR/PA/00/17, CERFACS, Toulouse, France, L. Giraud, S. Gratton, and J. Langou. Convergence in backward error of relaxed GMRES. SIAM J. Scientific Computing, 29(2): , V. Simoncini and D. B. Szyld. Theory of inexact Krylov subspace methods and applications to scientific computing. SIAM J. Scientific Computing, 25(2): , J. van den Eshof and G. L. G. Sleijpen. Inexact Krylov subspace methods for linear systems. SIAM Journal on Matrix Analysis and Applications, 26(1): , 2004.