Approximation of functions of large matrices. Part I. Computational aspects V. Simoncini Dipartimento di Matematica, Università di Bologna and CIRSA, Ravenna, Italy valeria@dm.unibo.it 1
The problem Given A R n n, v R n and f sufficiently smooth function, approximate x = f(a)v A large dimension, v = 1 A sym. pos. (semi)def., or A positive real f(a)v vs f(a) Projection-type methods K approximation space, m = dim(k) V R n m s.t. K = range(v ) x = f(a)v x m = V f(v AV )(V v) 2
The problem Given A R n n, v R n and f sufficiently smooth function, approximate A large dimension, v = 1 x = f(a)v A sym. pos. (semi)def., or A positive real f(a)v vs f(a) Projection-type methods K approximation space, m = dim(k) V R n m s.t. K = range(v ) x = f(a)v x m = V f(v AV )(V v) 3
Numerical approximation. f(a)v x Important issues: Role of f in the approximation quality Role of A in the approximation quality Efficiency? Measures/Estimates of accuracy? 4
Standard Krylov subspace approximation K = K m (A, v) For H m = V AV, v = V e 1 and V V = I m : x m = V m f(h m )e 1 Polynomial approximation: x m = p m 1 (A)v (p m 1 interpolates f at eigenvalues of H m ) (alternatives: other polyns or interpolation points; e.g., Moret & Novati, 01-04 ) Note: Procedure valid for A symmetric and nonsymmetric Numerical and theoretical results since mid 80s. 5
Typical convergence estimates in K m Approximation of f(λ) = exp( λ) (Hochbruck & Lubich 97) A sym. semidef. σ(a) [0, 4ρ], x m = V m exp( H m )e 1, f(a)v x m 10e m2 /(5ρ), 4ρ m 2ρ f(a)v x m 10 ( eρ ) m ρ e ρ, m 2ρ m see also Tal-Ezer 89, Druskin & Knizhnerman 89, Stewart & Leyk 96 Approximation of f(λ) = λ 1/2, exp( λ),... : f(a)v V m f( H m )e 1 = O(exp( 2m λmin λ max )) 6
Typical convergence estimates in K m Approximation of f(λ) = exp( λ) (Hochbruck & Lubich 97) A sym. semidef. σ(a) [0, 4ρ], x m = V m exp( H m )e 1, f(a)v x m 10e m2 /(5ρ), 4ρ m 2ρ f(a)v x m 10 ( eρ ) m ρ e ρ, m 2ρ m see also Tal-Ezer 89, Druskin & Knizhnerman 89, Stewart & Leyk 96 Approximation of f(λ) = λ 1/2, exp( λ),... : f(a)v V m f( H m )e 1 = O(exp( 2m λmin λ max )) 7
...When things are not so easy exp( A)v V m exp( H m )e 1 A R 400 400, A = 10 5 10 0 10 2 10 4 10 6 error norm 10 8 10 10 10 12 10 14 0 20 40 60 80 100 120 140 Krylov subspace dimension exp( A)v V m exp( H m )e 1 10e m2 /(5ρ), where σ(a) [0, 4ρ] 4ρ m 2ρ 8
Acceleration Procedures: Shift-Invert Lanczos A symmetric pos. (semi)def. Choose γ s.t. (I + γa) is invertible, and construct K = K m ((I + γa) 1, v), van den Eshof-Hochbruck 06, Moret-Novati 04 with T m = V (I + γa) 1 V, v = V e 1 and V V = I m x m = V m f( 1 γ (T 1 m I m ))e 1 Rational approximation: x m = p m 1 ((I + γa) 1 )v Choice of γ: γ = 1/ λ min λ max (Moret, tr 2005) 9
Acceleration Procedures: Extended Krylov For A nonsingular, K = K m1 (A, v) + K m2 (A 1, A 1 v), Druskin-Knizhnerman 1998, A sym. Note: K = A (m 2 1) K m1 +m 2 1(A, v) Algorithm (augmentation-style) - Fix m 2 m 1 - Run m 2 steps of Inverted Lanczos - Run m 1 steps of Standard Lanczos + orth. 10
Extended Krylov: a new implementation m 1 = m 2 = m not fixed a priori K = K m (A, v) + K m (A 1, A 1 v) Arnoldi-type recurrence: - U 1 [v, A 1 v] + orth - U j+1 [AU j (:, 1), A 1 U j (:, 2)] + orth j = 1, 2,... Recurrence to cheaply compute T m = UmAU m, U m = [U 1,..., U m ] Compute x m = U m f(t m )e 1 Simoncini, 2007 11
Extended Krylov: Convergence theory I f satisfying f(z) = 0 (with convenient measure dµ(ζ)) 1 dµ(ζ), z C\], 0] z ζ Druskin-Knizhnerman 1998: A sym: x x m = O(m 2 exp( 2m 4 λ min λ max )) 12
Extended Krylov: Convergence theory II A new approximation result for nonsingular A: Let f = f 1 + f 2, a [0, ) f 1 (z) f 2 (z) m 1 k=0 m 1 k=0 γ 1,k F 1,k (z) c 1 Φ 1 ( a) m, γ 2,k F 2,k (z 1 ) c 2 Φ 2 ( 1 a ) m, Φ 1, F 1,k conformal mapping and Faber Polynomial w.r.to W(A) Φ 2, F 2,k conformal mapping and Faber Polynomial w.r.to W(A 1 ) From this, for A symmetric: x x m = O(exp( 2m 4 λ min λ max )) Currently working on A nonsymmetric 13
Convergence rate. A R 400 400 symmetric. f(λ) = λ 1/2 10 0 10 0 10 2 10 2 10 4 10 4 norm of error 10 6 norm of error 10 6 10 8 10 8 10 10 10 10 10 12 10 12 0 5 10 15 20 25 30 dimension of approximation space 10 14 0 5 10 15 20 25 30 dimension of approximation space σ(a) = [0.01, 0.9] σ(a) = [1, 50] Uniform spectral distribution 14
Experiment. A R 400 400 normal. f(λ) = λ 1/2 10 0 ellipse a1=0.097, a2=0.01, c=0.1 10 2 true a= sqrt(l 1 l n ) a optimal 10 4 norm of error 10 6 10 8 10 10 10 12 10 14 0 5 10 15 20 25 30 35 40 45 50 dimension of approximation space σ(a) on an elliptic curve in C + with center on real axis 15
Log-uniform spectral distribution. f (3) (z) = exp( z), f (5) (z) = z 1/2, f (7) (z) = z 1/4 case 3 1 case 5 1 case 7 1 space dim. at convergence 400 300 200 100 space dim. at convergence 400 300 200 100 space dim. at convergence 400 300 200 100 0 10 0 10 5 case 3 2 0 10 0 10 5 case 5 2 0 10 0 shift invert Lanczos(0.1) 10 5 SI Lanczos(γ) Standard Krylov case Extended 7 2 Krylov space dim. at convergence 400 300 200 100 space dim. at convergence 400 300 200 100 space dim. at convergence 400 300 200 100 0 0 0 10 0 10 0 10 0 cases *-1: λ min = 10 1 cases *-2: λ min = 10 4 x-axis: λ max = 10 k 16
SI-Lanczos and dependence on parameter γ 10 0 f(x) = exp( sqrt(x)), λ min = 1e 1, λ max = 1e6 10 0 f(x) = exp( sqrt(x)), λ min = 1e 4, λ max = 1e3 10 2 Lanczos 10 1 10 2 Lanczos 10 3 SI(0.01) 10 4 10 6 10 8 EK 10 4 10 5 10 6 10 7 SI(γ) EK SI(0.1) 10 10 SI(0.1)SI(1) SI(100) SI(10) SI(0.01) SI(γ) 10 8 10 9 0 50 100 150 200 250 300 10 10 0 50 100 150 200 250 300 Log-uniform spectral distribution 17
Comparisons: CPU Time in Matlab A R 4900 4900 : L(u) = 1 10 u xx 100u yy, in [0, 1] 2, hom.b.c. σ(a) [9.6 10 2, 1.96 10 6 ], f(λ) = λ 1/2 Method space dim. CPU Time Standard Krylov 185 16.02 Rational (Zolotarev) 0.50 SI-Lanczos(0.001) 62 1.00 SI-Lanczos (1e-5) 49 0.60 SI-Lanczos (γ=2e-5) 33 0.32 Extended Krylov 32 0.20 No reorthogonalization. Direct solves. Tol = 10 12. 18
A nonsymmetric matrix. f(z) = z A R 900 900 : L(u) = 100u xx u yy + 10xu x in [0, 1] 2, hom.b.c. 10 2 10 0 10 2 norm of error 10 4 SI Arnoldi(1e 3) 10 6 10 8 Extended Krylov SI Arnoldi(2e 5) SI Arnoldi(1e 5) 10 10 0 10 20 30 40 50 60 70 80 90 100 dimension of approximation space σ(a) R, λ min = 9.2 10 2, λ max = 3.6 10 5 Full orthogonalization 19
Conclusions and work in progress Great potential of using f(a)v in application problems Exploit low cost of using A instead of f(a) Further developments in acceleration techniques The case of A nonsymmetric (preliminary encouraging tests) New implementation of the Extended Krylov method Improved convergence bounds for A symmetric New convergence results for A nonsymmetric (to be completed) Performance: - Does not depend on parameters - Competitive for A sym, and nonsym. (preliminary tests) 20
Conclusions and work in progress Great potential of using f(a)v in application problems Exploit low cost of using A instead of f(a) Further developments in acceleration techniques The case of A nonsymmetric (preliminary encouraging tests) New implementation of the Extended Krylov method Improved convergence bounds for A symmetric New convergence results for A nonsymmetric (to be completed) Performance: - Does not depend on parameters - Competitive for A sym, and nonsym. (preliminary tests) 21