Recent advances in approximation using Krylov subspaces V. Simoncini Dipartimento di Matematica, Università di Bologna and CIRSA, Ravenna, Italy valeria@dm.unibo.it 1
The framework It is given an operator v A ɛ (v). Efficiently solve the given problem in the approximation space K m = span{v, A ɛ1 (v), A ɛ2 (A ɛ1 (v)),...}, v C n with dim(k m ) = m, where A ɛ A for ɛ 0 (ɛ may be tuned) for A = A, ɛ = 0 K m = span{v, Av, A 2 v,..., A m 1 v} 2
Examples of A: Solution of (preconditioned) large linear systems, Ax = b n n A = A Shift-and-Invert eigensolvers Ax = λmx, x = 1, A = (σm A) 1 Spectral Transformation for the exponential x = exp(a)v, A = (γi A) 1... Goal: Achieve approximation x m to x within a fixed tolerance, by using A ɛ (and not A), with variable ɛ 3
Many applications in Scientific Computing A(v) function (linear in v): Shift-and-Invert procedures for interior eigenvalues Schur complement: A = B T S 1 B S expensive to invert Preconditioned system: AP 1 x = b, where etc. P 1 v i P 1 i v i K m = span{v, A(v), A(A(v)),...}, v C n 4
The exact approach To focus our attention: A = A. Krylov subspace: K m = span{v, Av, A 2 v,..., A m 1 v} V m = [v 1,..., v m ] orthogonal basis 1. v m+1 = Av m 2. v m+1 orthogonalize v m+1 w.r.to {v i } Key relation in Krylov subspace methods: AV m = V m H m + v m+1 h m+1,m e T m v = V m e 1 v 5
The exact approach. cont d K m Krylov subspace V m = [v 1,..., v m ] orthogonal basis AV m = V m H m + v m+1 h m+1,m e T m = V m H m with v = V m e 1 v System: x m K m x m = V m y m (x 0 = 0) Residual r m = v Ax m = V m+1 (e 1 v H m y m ) Eigenpb: (θ, y) eigenpair of H m (θ, V m y) Ritz pair for (λ, x) Residual: r m = θv m y AV m y = v m+1 h m+1,m e T my 6
AV m = V m+1 H m + The inexact key relation A = A A ɛ (v) = Av + f F m }{{} [f 1,f 2,...,f m ] How large is F m allowed to be? system: F m error matrix, f j = O(ɛ j ) r m = b AV m y m = b V m+1 H m y m F m y m eigenproblem: (θ, V m y) = V m+1 (e 1 β H m y m ) F m y m }{{} computed residual =: r m r m = θv m y AV m y = v m+1 h m+1,m e T my F m y 7
A dynamic setting F m y = [f 1, f 2,..., f m ] η 1 η 2. = m f i η i i=1 η m The terms f i η i need to be small: f i η i < 1 m ɛ i F my < ɛ If η i small f i is allowed to be large 8
Linear systems: The solution pattern y m = [η 1 ; η 2 ;... ; η m ] depends on the chosen method, e.g. Petrov-Galerkin (e.g. GMRES): y m = argmin y e 1 β H m y, η i 1 σ min (H m ) r i 1 r i 1 : GMRES computed residual at iteration i 1. Simoncini & Szyld, SISC 2003 (see also Sleijpen & van den Eshof, SIMAX 2004) Analogous result for Galerkin methods (e.g. FOM) 9
Ritz approximation: Eigenproblem: The structure of the Ritz pair (θ, y) eigenpair of H m y = [η 1 ; η 2 ;... ; η m ], η i 2 δ m,i r i 1 δ m,i quantity related to the spectral gap of θ with H m r i 1 : Computed eigenresidual at iteration i 1 Analogous results for Harmonic Ritz values and Lanczos approx. Simoncini, SINUM 2005 10
Relaxing the inexactness in A A v i not performed exactly (A + E i ) v i True (unobservable) vs. computed residuals: r m = b AV m y m = V m+1 (e 1 β H m y m ) F m y m - GMRES: If (Similar result for FOM) E i σ min(h m ) m 1 r i 1 ε i = 1,..., m then F m y m ε r m V m+1 (e 1 β H m y m ) ε r i 1 : GMRES computed residual at iteration i 1 11
An example: Schur complement } B T S {{ 1 B } x = b A y i B T S 1 Bv i At each Krylov subspace iteration: 8 < Solve Sw i = Bv i : Compute y i = B T w i Inexact 8 < Approx solve Sw i = Bv i : Compute by i = B T bw i bw i w i = ŵ i + ɛ i ɛ i error in inner solution so that Av i B T ŵ i = B T w }{{} i B T ɛ i = (A + E }{{} i )v i Av i E i v i 12
Numerical experiment } B T S {{ 1 B } x = b at each it. i solve Sw i = Bv i A 10 1 10 0 10 1 r m 10 2 ε inner Inexact FOM δ m = r m (b V m+1 H m y m ) magnitude 10 3 10 4 10 5 10 6 δ m 10 7 ~ r m 10 8 0 20 40 60 80 100 120 number of iterations 13
Eigenproblem Inverted Arnoldi: Ax = λx Find min λ y A(v) = A 1 v Matrix sherman5 0.04 10 2 0.03 10 0 inner residual norm 0.02 10 2 0.01 10 4 imaginary part 0 magnitude 10 6 0.01 10 8 0.02 10 10 ε 0.03 10 12 exact res. inexact residual 0.04 200 100 0 100 200 300 400 500 600 real part 10 14 0 2 4 6 8 10 12 number of outer iterations 14
Problems to be faced Make the inexactness criterion practical E i σ min(h m ) m 1 r i 1 ε E i l m 1 r i 1 ε (CERFACS tr s of Bouras, Frayssè, Giraud, 2000, Bouras, Frayssè SIMAX05) What is the convergence behavior? What if original A was symmetric? 15
Selecting l m : system AP 1 x = b 10 2 10 2 10 0 ε inner 10 0 10 2 10 2 ε inner 10 4 10 4 magnitude 10 6 r m magnitude 10 6 r m 10 8 10 8 10 10 10 10 10 12 δ m ~ r m 10 12 δ m ~ r m 10 14 0 10 20 30 40 50 60 70 80 number of iterations 10 14 0 10 20 30 40 50 60 70 80 number of iterations Left: l m = 1 Right: estimated l m Final Requested Tolerance: 10 10 16
Convergence behavior Does the inexact procedure behave as if E i = 0? The Sleijpen & van den Eshof s example: Exact vs. Inexact GMRES b = e 1 E i random entries 10 0 10 2 10 4 E k A = 2 6 4 1 0 0 0 1 2 0 0 0 1 3... 0............. 0 0 1 100 3 7 5 residual norm 10 6 10 8 10 10 10 12 inexact 10 14 0 5 10 15 20 25 number of iterations exact 17
Inexactness and convergence (linear systems) Av i (A + E i )v i For general A and b convergence is the same as exact A Problems for: Sensitive A (highly nonnormal) Special starting vector / right-hand side Superlinear convergence as for A (Simoncini & Szyld, SIREV 2005) 18
Flexible preconditioning AP 1 x = b x = P 1 x Flexible: P 1 v i P 1 i v i, bx m span{v 1, AP 1 1 v 1, AP 1 2 v 2,..., AP 1 m 1 v m 1} Directly recover x m (Saad, 1993): [P 1 1 v 1, P 1 2 v 2,..., P 1 m v m ] = Z m x m = Z m y m Inexact framework but exact residual 19
A practical example A B B T 0 x y = f g P = I 0 0 B T B Application of P 1 corresponds to solves with B T B P Use CG to solve systems with B T B Variable inner tolerance: At each outer iteration m, r inner k l m r outer m 1 ε 20
Outer tolerance: 10 8 Electromagnetic 2D problem r inner k l m r outer m 1 ε 0 ε Elapsed Time Pb. Size Fixed Inner Tol Var. Inner Tol. Var. Inner Tol. ε = 10 10 ε = 10 10 / r ε = 10 12 / r 3810 17.0 (54) 11.4 (54) 14.7 (54) 9102 82.9 (58) 62.8 (58) 70.7 (58) 14880 198.4 (54) 156.5 (54) 170.1 (54) 21
Structural Dynamics (A + σb)x = b Solve for many σ s simultaneously (AB 1 + σi) x = b (Perotti & Simoncini 2002) Inexact solutions with B at each iteration: Prec. Fill-in 5 Prec. Fill-in 10 e-time [s] # outer its e-time [s] # outer its Tol 10 6 14066 296 13344 289 Dynamic Tol 11579 301 11365 293 20 % enhancement with tiny change in the code (see also van den Eshof, Sleijpen and van Gijzen, JCAM 2005) 22
Inexactness when A symmetric A symmetric A + E i nonsymmetric Assume V T m V m = I H m upper Hessenberg Wise implementation of short-term recurr. /truncated methods (V m non-orth. W m, H m tridiag./banded T m ) - Inexact short-term recurrence system solvers (Golub-Overton 88, Golub-Ye 99, Notay 00, Sleijpen-van den Eshof 04,... ) - Inexact symmetric eigensolvers (Lai-Lin-Lin 1997, Golub-Ye 2000, Golub-Zhang-Zha 2000, Notay 2002,... ) - Truncated methods (Simoncini - Szyld, Numer.Math. 2005) 23
Ax = b A sym. (2D Laplacian) Preconditioner: P nonsymmetric perturbation (10 5 ) of Incomplete Cholesky 10 0 10 2 10 2 10 0 10 2 ^ T m+1,m T m+1,m 10 4 PCG 10 4 norm of residual 10 6 magnitude 10 6 10 8 * I W W m+1 m+1 m+1 10 10 10 8 10 12 10 10 k = k = 2, 3 10 14 0 2 4 6 8 10 12 14 16 18 20 number of iterations 10 16 0 2 4 6 8 10 12 14 16 18 number of iterations 24
One more application: Approximation of the exponential A symmetric negative semidefinite (large dimension), v s.t. v = 1, exp(a)v x m = V m y m, y m = exp(h m )e 1 Problem: Find acceleration process for A to speed up convergence Hochbruck & van den Eshof (SISC 2006): Determine x m exp(a)v as x m = V m y m K m ((γi A) 1, v) for some scalar γ y m = exp(h m )e 1 has a structured decreasing pattern (Lopez & Simoncini, SINUM 2006) 25
Conclusions A may be replaced by A ɛi with increasing ɛ i and still converge Stable procedure for not too sensitive (e.g. non-normal) problems Property inherent of Krylov approximation Many more applications for this general setting References at: www.dm.unibo.it/~ simoncin 26