Recent advances in approximation using Krylov subspaces. V. Simoncini. Dipartimento di Matematica, Università di Bologna.

Transcription

1 Recent advances in approximation using Krylov subspaces V. Simoncini Dipartimento di Matematica, Università di Bologna and CIRSA, Ravenna, Italy 1

2 The framework It is given an operator v A ɛ (v). Efficiently solve the given problem in the approximation space K m = span{v, A ɛ1 (v), A ɛ2 (A ɛ1 (v)),...}, v C n with dim(k m ) = m, where A ɛ A for ɛ 0 (ɛ may be tuned) for A = A, ɛ = 0 K m = span{v, Av, A 2 v,..., A m 1 v} 2

3 Examples of A: Solution of (preconditioned) large linear systems, Ax = b n n A = A Shift-and-Invert eigensolvers Ax = λmx, x = 1, A = (σm A) 1 Spectral Transformation for the exponential x = exp(a)v, A = (γi A) 1... Goal: Achieve approximation x m to x within a fixed tolerance, by using A ɛ (and not A), with variable ɛ 3

4 Many applications in Scientific Computing A(v) function (linear in v): Shift-and-Invert procedures for interior eigenvalues Schur complement: A = B T S 1 B S expensive to invert Preconditioned system: AP 1 x = b, where etc. P 1 v i P 1 i v i K m = span{v, A(v), A(A(v)),...}, v C n 4

5 The exact approach To focus our attention: A = A. Krylov subspace: K m = span{v, Av, A 2 v,..., A m 1 v} V m = [v 1,..., v m ] orthogonal basis 1. v m+1 = Av m 2. v m+1 orthogonalize v m+1 w.r.to {v i } Key relation in Krylov subspace methods: AV m = V m H m + v m+1 h m+1,m e T m v = V m e 1 v 5

6 The exact approach. cont d K m Krylov subspace V m = [v 1,..., v m ] orthogonal basis AV m = V m H m + v m+1 h m+1,m e T m = V m H m with v = V m e 1 v System: x m K m x m = V m y m (x 0 = 0) Residual r m = v Ax m = V m+1 (e 1 v H m y m ) Eigenpb: (θ, y) eigenpair of H m (θ, V m y) Ritz pair for (λ, x) Residual: r m = θv m y AV m y = v m+1 h m+1,m e T my 6

7 AV m = V m+1 H m + The inexact key relation A = A A ɛ (v) = Av + f F m }{{} [f 1,f 2,...,f m ] How large is F m allowed to be? system: F m error matrix, f j = O(ɛ j ) r m = b AV m y m = b V m+1 H m y m F m y m eigenproblem: (θ, V m y) = V m+1 (e 1 β H m y m ) F m y m }{{} computed residual =: r m r m = θv m y AV m y = v m+1 h m+1,m e T my F m y 7

8 A dynamic setting F m y = [f 1, f 2,..., f m ] η 1 η 2. = m f i η i i=1 η m The terms f i η i need to be small: f i η i < 1 m ɛ i F my < ɛ If η i small f i is allowed to be large 8

9 Linear systems: The solution pattern y m = [η 1 ; η 2 ;... ; η m ] depends on the chosen method, e.g. Petrov-Galerkin (e.g. GMRES): y m = argmin y e 1 β H m y, η i 1 σ min (H m ) r i 1 r i 1 : GMRES computed residual at iteration i 1. Simoncini & Szyld, SISC 2003 (see also Sleijpen & van den Eshof, SIMAX 2004) Analogous result for Galerkin methods (e.g. FOM) 9

10 Ritz approximation: Eigenproblem: The structure of the Ritz pair (θ, y) eigenpair of H m y = [η 1 ; η 2 ;... ; η m ], η i 2 δ m,i r i 1 δ m,i quantity related to the spectral gap of θ with H m r i 1 : Computed eigenresidual at iteration i 1 Analogous results for Harmonic Ritz values and Lanczos approx. Simoncini, SINUM

11 Relaxing the inexactness in A A v i not performed exactly (A + E i ) v i True (unobservable) vs. computed residuals: r m = b AV m y m = V m+1 (e 1 β H m y m ) F m y m - GMRES: If (Similar result for FOM) E i σ min(h m ) m 1 r i 1 ε i = 1,..., m then F m y m ε r m V m+1 (e 1 β H m y m ) ε r i 1 : GMRES computed residual at iteration i 1 11

12 An example: Schur complement } B T S {{ 1 B } x = b A y i B T S 1 Bv i At each Krylov subspace iteration: 8 < Solve Sw i = Bv i : Compute y i = B T w i Inexact 8 < Approx solve Sw i = Bv i : Compute by i = B T bw i bw i w i = ŵ i + ɛ i ɛ i error in inner solution so that Av i B T ŵ i = B T w }{{} i B T ɛ i = (A + E }{{} i )v i Av i E i v i 12

13 Numerical experiment } B T S {{ 1 B } x = b at each it. i solve Sw i = Bv i A r m 10 2 ε inner Inexact FOM δ m = r m (b V m+1 H m y m ) magnitude δ m 10 7 ~ r m number of iterations 13

14 Eigenproblem Inverted Arnoldi: Ax = λx Find min λ y A(v) = A 1 v Matrix sherman inner residual norm imaginary part 0 magnitude ε exact res. inexact residual real part number of outer iterations 14

15 Problems to be faced Make the inexactness criterion practical E i σ min(h m ) m 1 r i 1 ε E i l m 1 r i 1 ε (CERFACS tr s of Bouras, Frayssè, Giraud, 2000, Bouras, Frayssè SIMAX05) What is the convergence behavior? What if original A was symmetric? 15

16 Selecting l m : system AP 1 x = b ε inner ε inner magnitude 10 6 r m magnitude 10 6 r m δ m ~ r m δ m ~ r m number of iterations number of iterations Left: l m = 1 Right: estimated l m Final Requested Tolerance:

17 Convergence behavior Does the inexact procedure behave as if E i = 0? The Sleijpen & van den Eshof s example: Exact vs. Inexact GMRES b = e 1 E i random entries E k A = residual norm inexact number of iterations exact 17

18 Inexactness and convergence (linear systems) Av i (A + E i )v i For general A and b convergence is the same as exact A Problems for: Sensitive A (highly nonnormal) Special starting vector / right-hand side Superlinear convergence as for A (Simoncini & Szyld, SIREV 2005) 18

19 Flexible preconditioning AP 1 x = b x = P 1 x Flexible: P 1 v i P 1 i v i, bx m span{v 1, AP 1 1 v 1, AP 1 2 v 2,..., AP 1 m 1 v m 1} Directly recover x m (Saad, 1993): [P 1 1 v 1, P 1 2 v 2,..., P 1 m v m ] = Z m x m = Z m y m Inexact framework but exact residual 19

20 A practical example A B B T 0 x y = f g P = I 0 0 B T B Application of P 1 corresponds to solves with B T B P Use CG to solve systems with B T B Variable inner tolerance: At each outer iteration m, r inner k l m r outer m 1 ε 20

21 Outer tolerance: 10 8 Electromagnetic 2D problem r inner k l m r outer m 1 ε 0 ε Elapsed Time Pb. Size Fixed Inner Tol Var. Inner Tol. Var. Inner Tol. ε = ε = / r ε = / r (54) 11.4 (54) 14.7 (54) (58) 62.8 (58) 70.7 (58) (54) (54) (54) 21

22 Structural Dynamics (A + σb)x = b Solve for many σ s simultaneously (AB 1 + σi) x = b (Perotti & Simoncini 2002) Inexact solutions with B at each iteration: Prec. Fill-in 5 Prec. Fill-in 10 e-time [s] # outer its e-time [s] # outer its Tol Dynamic Tol % enhancement with tiny change in the code (see also van den Eshof, Sleijpen and van Gijzen, JCAM 2005) 22

23 Inexactness when A symmetric A symmetric A + E i nonsymmetric Assume V T m V m = I H m upper Hessenberg Wise implementation of short-term recurr. /truncated methods (V m non-orth. W m, H m tridiag./banded T m ) - Inexact short-term recurrence system solvers (Golub-Overton 88, Golub-Ye 99, Notay 00, Sleijpen-van den Eshof 04,... ) - Inexact symmetric eigensolvers (Lai-Lin-Lin 1997, Golub-Ye 2000, Golub-Zhang-Zha 2000, Notay 2002,... ) - Truncated methods (Simoncini - Szyld, Numer.Math. 2005) 23

24 Ax = b A sym. (2D Laplacian) Preconditioner: P nonsymmetric perturbation (10 5 ) of Incomplete Cholesky ^ T m+1,m T m+1,m 10 4 PCG 10 4 norm of residual 10 6 magnitude * I W W m+1 m+1 m k = k = 2, number of iterations number of iterations 24

25 One more application: Approximation of the exponential A symmetric negative semidefinite (large dimension), v s.t. v = 1, exp(a)v x m = V m y m, y m = exp(h m )e 1 Problem: Find acceleration process for A to speed up convergence Hochbruck & van den Eshof (SISC 2006): Determine x m exp(a)v as x m = V m y m K m ((γi A) 1, v) for some scalar γ y m = exp(h m )e 1 has a structured decreasing pattern (Lopez & Simoncini, SINUM 2006) 25

26 Conclusions A may be replaced by A ɛi with increasing ɛ i and still converge Stable procedure for not too sensitive (e.g. non-normal) problems Property inherent of Krylov approximation Many more applications for this general setting References at: simoncin 26