Computing the Action of the Matrix Exponential

Transcription

1 Computing the Action of the Matrix Exponential Nick Higham School of Mathematics The University of Manchester Joint work with Awad H. Al-Mohy 16th ILAS Conference, Pisa, June 2010

2 The f(a)b Problem Given matrix function f : C n n C n n, A C n n, b C n, compute f(a)b without first computing f(a). Most important cases f(x) = x 1, f(x) = e x. Application : y (t) = Ay(t), y(0) = b y(t) = e At b. MIMS Nick Higham Computing the Action of the Matrix Exponential 2 / 29

3 Second Order ODE d 2 y dt + Ay = 0, 2 y(0) = y 0, y (0) = y 0 has solution y(t) = cos( At)y 0 + ( A ) 1 sin( At)y 0. MIMS Nick Higham Computing the Action of the Matrix Exponential 3 / 29

4 Second Order ODE d 2 y dt + Ay = 0, 2 y(0) = y 0, y (0) = y 0 has solution y(t) = cos( At)y 0 + ( A ) 1 sin( At)y 0. But [ ] y = exp y ([ 0 ta t I n 0 ]) [ ] y 0. y 0 MIMS Nick Higham Computing the Action of the Matrix Exponential 3 / 29

5 Themes Simple, direct algorithm for computing e A B with arbitrary A using only matrix products. Many problems can be reduced to e A B. Backward error viewpoint avoids consideration of conditioning in algorithm design. MIMS Nick Higham Computing the Action of the Matrix Exponential 4 / 29

6 Exponential Integrators u (t) = Au(t) + g(t, u(t)), u(0) = u 0, t 0, Solution can be written u(t) = e ta u 0 + ( ) ϕ k ta t k u k, k=1 where u k = g (k 1) (t, u(t)) t=0 and ϕ l (z) = k=0 zk /(k + l)!. MIMS Nick Higham Computing the Action of the Matrix Exponential 5 / 29

7 Exponential Integrators u (t) = Au(t) + g(t, u(t)), u(0) = u 0, t 0, Solution can be written u(t) = e ta u 0 + ( ) ϕ k ta t k u k, k=1 where u k = g (k 1) (t, u(t)) t=0 and ϕ l (z) = k=0 zk /(k + l)!. u(t) û(t) = e ta u 0 + p ( ) ϕ k ta t k u k. k=1 Exponential time differencing (ETD) Euler (p = 1): y n+1 = e ha y n + hϕ 1 (ha)g(t n, y n ). MIMS Nick Higham Computing the Action of the Matrix Exponential 5 / 29

8 Saad s Trick (1992) exp ϕ 1 (z) = ez 1. z ([ ]) A b = 0 0 [ ] e A ϕ 1 (A)b 0 1 MIMS Nick Higham Computing the Action of the Matrix Exponential 6 / 29

9 Theorem (Al-Mohy & H, 2010) Let A C n n, U = [u 1, u 2,...,u p ] C n p, τ C, and define [ ] [ ] A U 0 B = C (n+p) (n+p) Ip 1, J = C p p. 0 J 0 0 Then for X = e τb we have X(1: n, n + j) = jk=1 τ k ϕ k (τa)u j k+1, j = 1: p.

10 Theorem (Al-Mohy & H, 2010) Let A C n n, U = [u 1, u 2,...,u p ] C n p, τ C, and define [ ] [ ] A U 0 B = C (n+p) (n+p) Ip 1, J = C p p. 0 J 0 0 Then for X = e τb we have X(1: n, n + j) = jk=1 τ k ϕ k (τa)u j k+1, j = 1: p. Completely removes the need to evaluate ϕ k functions!

11 Implementation of Exponential Integrators We compute û(t) = e ta u 0 + as, with U = [u p,...,u 1 ], û(t) = [ I n p ( ) ϕ k ta t k u k k=1 0 ] ( [ A ηu exp t 0 J ]) [ u 0 η 1 e p ]. Choose to avoid overscaling. η = 2 log 2 ( U 1) MIMS Nick Higham Computing the Action of the Matrix Exponential 8 / 29

12 Formulae for e A, A C n n Power series Limit Scaling and squaring I + A + A2 2! + A3 3! + lim (I + A/s)s (e s A/2s ) 2s Cauchy integral Jordan form Interpolation 1 n i 1 e z (zi A) 1 dz Zdiag(e J k )Z 1 f[λ 1,..., λ i ] (A λ j I) 2πi Γ i=1 j=1 Differential system Schur form Padé approximation Y (t) = AY(t), Y(0) = I Qe T Q p km (A)q km (A) 1 Krylov methods: Arnoldi fact. AQ k = Q k H k + h k+1,k q k+1 e T k with Hessenberg H: e A b Q k e H k Q k b. MIMS Nick Higham Computing the Action of the Matrix Exponential 9 / 29

13 Scaling and Squaring Method B A/2 i so B 1 r m (B) = [m/m] Padé approximant to e B X = r m (B) 2i e A MATLABexpm uses alg of H (2005). Improved algorithm: Al-Mohy & H (2009). Can we adapt this approach for e A B? MIMS Nick Higham Computing the Action of the Matrix Exponential 10 / 29

14 Computing e A B }{{} A, }{{} B, n 0 n. Exploit, for integer s, n n n n 0 e A B = (e s 1A ) s B = e s 1A e s 1A e s 1 A }{{} B. s times Choose s so T m (s 1 A) = m (s 1 A) j j=0 j! e s 1A. Then B i+1 = T m (s 1 A)B i, i = 0: s 1, B 0 = B yields B s e A B. MIMS Nick Higham Computing the Action of the Matrix Exponential 11 / 29

15 Computing e A B }{{} A, }{{} B, n 0 n. Exploit, for integer s, n n n n 0 e A B = (e s 1A ) s B = e s 1A e s 1A e s 1 A }{{} B. s times Choose s so T m (s 1 A) = m (s 1 A) j j=0 j! e s 1A. Then B i+1 = T m (s 1 A)B i, i = 0: s 1, B 0 = B yields B s e A B. How to choose s and m? MIMS Nick Higham Computing the Action of the Matrix Exponential 11 / 29

16 Backward Error Analysis Lemma (Al-Mohy & H, 2009) T m (s 1 A) s B = e A+ A B, where A = sh m+1 (s 1 A) and h m+1 (x) = log(e x T m (x)) = k=m+1 c k x k. Moreover, h m+1 (A) k=m+1 c k α p (A) k if m + 1 p(p 1), where α p (A) = max(d p, d p+1 ), d p = A p 1/p. MIMS Nick Higham Computing the Action of the Matrix Exponential 12 / 29

17 Why Use d p = A p 1/p? A k A pk 1/p ( A p 1/p) k. ρ(a) A p 1/p A. With A = [ ] : k A k 1 2.0e2 2.2e2 1.2e2 A k 1 2.5e5 3.1e13 9.8e26 d2 k = ( A 2 1/2 ) k 1 2.0e2 5.7e5 3.2e11 d3 k = ( A 3 1/3 ) k 1 4.5e1 1.3e4 1.9e8 Cheaply estimate A k, for a few k (H & Tisseur, 2001). MIMS Nick Higham Computing the Action of the Matrix Exponential 13 / 29

18 Why Use d p = A p 1/p? cont. d p = A p 1/p provide information about the nonnormality of A. Their use helps avoid overscaling. What other uses do they have? MIMS Nick Higham Computing the Action of the Matrix Exponential 14 / 29

19 Choice of s and m α p (A) := max ( d p, d p+1 ) θ m := max { θ : k=m+1 c k θ k 1 ǫ } A A ǫ if m + 1 p(p 1) and s 1 α p (A) θ m. Computational cost for B s e A B is matrix products. C m (A) = m max ( α p (A)/θ m, 1 ) Cost decreases with m. Restrict 2 p p max, p(p 1) 1 m m max. Minimize cost over p, m MIMS Nick Higham Computing the Action of the Matrix Exponential 15 / 29

20 Size of Taylor Series Argument Constants θ m (via symbolic, high precision): m single 1.0e0 3.6e0 6.3e0 9.1e0 1.3e1 double 1.4e-1 1.4e0 3.5e0 6.0e0 9.9e0 MIMS Nick Higham Computing the Action of the Matrix Exponential 16 / 29

21 Preprocessing Expand the Taylor series about µ C: e µ (A µi) k /k! k=0 Choose µ so A µi A. Alg is based on 1-norm, but minimizing A µi F does better empirically at minimizing d p (A µi). Recover e A from e µ[ T m (s 1 (A µi)) ] s, [ e µ/s T m (s 1 (A µi)) ] s. First expression prone to overflow, so prefer second. Balancing is an option. MIMS Nick Higham Computing the Action of the Matrix Exponential 17 / 29

22 Termination Criterion In evaluating T m (s 1 A)B i = m (s 1 A) j j=0 j! B i we accept T k (A)B i for the first k such that A k 1 B i (k 1)! + Ak B i k! ǫ T k (A)B i. MIMS Nick Higham Computing the Action of the Matrix Exponential 18 / 29

23 Algorithm for F = e ta B 1 µ = trace(a)/n 2 A = A µi 3 [m, s] = parameters(ta) 4 F = B, η = e tµ/s 5 for i = 1: s 6 c 1 = B 7 for j = 1: m 8 B = t AB/(sj), c 2 = B 9 F = F + B 10 if c 1 + c 2 tol F, quit, end 11 c 1 = c 2 12 end 13 F = ηf, B = F 14 end MIMS Nick Higham Computing the Action of the Matrix Exponential 19 / 29

24 George Forsythe Pitfalls (1970) MIMS Nick Higham Computing the Action of the Matrix Exponential 20 / 29

25 Conditioning of e A B κ exp (A, B) ea F B F e A B F (1 + κ exp (A)). A 2 κ exp (A) e A 2 A 2 e A 2 Relative forward error due to roundoff bounded by ue A 2 B 2 / e A B F. A normal implies κ exp (A) = A 2. Then instability if e A 2 e A 2. A Hermitian implies spectrum of A n 1 trace(a)i has λ max = λ min (normwise) stability! MIMS Nick Higham Computing the Action of the Matrix Exponential 21 / 29

26 e ta B for Several t Grid: t k = t 0 + kh, k = 0: q, where h = (t q t 0 )/q. Evaluate B k = e tka B, k = 0: q, directly. Form B 0 = e t0a B and then B k = e kha B 0, k = 1: q B 2 = e 2hA B 0 (1) Form B 0 = e t0a B and then B k = e ha B k 1, k = 1: q B 2 = e ha (e ha B 0 ) (2) Suffers from overscaling when h is small enough. MIMS Nick Higham Computing the Action of the Matrix Exponential 22 / 29

27 e ta B for Several t Grid: t k = t 0 + kh, k = 0: q, where h = (t q t 0 )/q. Evaluate B k = e t ka B, k = 0: q, directly. Form B 0 = e t 0A B and then B k = e kha B 0, k = 1: q B 2 = e 2hA B 0 (1) Form B 0 = e t 0A B and then B k = e ha B k 1, k = 1: q B 2 = e ha (e ha B 0 ) (2) Suffers from overscaling when h is small enough. x e x (1 + x) e x (1 + x/2) 2 9.9e-9 2.2e e e e-16 MIMS Nick Higham Computing the Action of the Matrix Exponential 22 / 29

28 e ta B for Several t Grid: t k = t 0 + kh, k = 0: q, where h = (t q t 0 )/q. Evaluate B k = e tka B, k = 0: q, directly. Form B 0 = e t0a B and then B k = e kha B 0, k = 1: q B 2 = e 2hA B 0 (1) Form B 0 = e t0a B and then B k = e ha B k 1, k = 1: q B 2 = e ha (e ha B 0 ) (2) Suffers from overscaling when h is small enough. Use (1) when no cost penalty (no scaling), else (2). Opportunity to save and re-use some matrix products. MIMS Nick Higham Computing the Action of the Matrix Exponential 22 / 29

29 The Sixth Dubious Way (1) Moler & Van Loan (1978, 2003) MIMS Nick Higham Computing the Action of the Matrix Exponential 23 / 29

30 The Sixth Dubious Way (2) Advantages of our method over the one-step ODE integrator: Fully exploits the linearity of the ODE. Backward error based; ODE integrator control local (forward) errors. Overscaling avoided. MIMS Nick Higham Computing the Action of the Matrix Exponential 24 / 29

31 Experiment 1 Trefethen, Weideman & Schmelzer (2006): A R , 2D Laplacian, -2500*gallery( poisson,99). Compute e αta b for 100 equally spaced t [0, 1]. tol = u d. α = 0.02 α = 1 speed cost diff speed cost diff Alg AH expv e e-14 phipm e e-14 rational e e-12 MIMS Nick Higham Computing the Action of the Matrix Exponential 25 / 29

32 Experiment 2 A =-gallery( triw,20,4.1), b i = cos i, tol = u d e ta b Alg AH phipm rational expv t MIMS Nick Higham Computing the Action of the Matrix Exponential 26 / 29

33 Experiment 3 Harwell Boeing matrices: orani678, n = 2529, t = 100, b = [1, 1,...,1] T ; bcspwr10, n = 5300, t = 10, b = [1, 0,...,0, 1] T. 2D Laplacian matrix,poisson. tol = u s. Alg AH ode15s time cost error time cost error orani e e-6 bcspwr e e-5 poisson e e-6 4 poisson e e-1 MIMS Nick Higham Computing the Action of the Matrix Exponential 27 / 29

34 Comparison with Krylov Methods Alg AH Most time spent in matrix vector products. Direct method, cost predictable. No parameters to estimate. Storage: 2 vectors Evaluation of e At at multiple points on interval. Can handle mult col B. Cost tends to with A. Krylov Methods Krylov recurrence and e H can be significant. Iterative method; needs stopping test. Select Krylov subspace size. Storage: Krylov basis Need block Krylov method. Some A dependence. MIMS Nick Higham Computing the Action of the Matrix Exponential 28 / 29

35 Conclusions e A B = (e s 1A ) s B T m (s 1 A)...T m (s 1 A) B. }{{} s times Key ideas: s and m to achieve b err bound; terminate Taylor series prematurely; shifting and balancing; compute e t ka B on grid, avoiding overscaling. Alg AH is best method in our experiments. Very easy to implement. Exploits optimized matrix products. Attractive for exponential integrators using theorem on avoidance of ϕ functions. A. H. Al-Mohy and N. J. Higham, MIMS EPrint , March Paper and MATLAB code. MIMS Nick Higham Computing the Action of the Matrix Exponential 29 / 29

36 References I A. H. Al-Mohy and N. J. Higham. A new scaling and squaring algorithm for the matrix exponential. SIAM J. Matrix Anal. Appl., 31(3): , G. E. Forsythe. Pitfalls in computation, or why a math book isn t enough. Amer. Math. Monthly, 77(9): , N. J. Higham. The Matrix Function Toolbox. http: // MIMS Nick Higham Computing the Action of the Matrix Exponential 26 / 29

37 References II N. J. Higham. The scaling and squaring method for the matrix exponential revisited. SIAM J. Matrix Anal. Appl., 26(4): , N. J. Higham. Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, ISBN xx+425 pp. N. J. Higham and A. H. Al-Mohy. Computing matrix functions. Acta Numerica, 19: , MIMS Nick Higham Computing the Action of the Matrix Exponential 27 / 29

38 References III N. J. Higham and F. Tisseur. A block algorithm for matrix 1-norm estimation, with an application to 1-norm pseudospectra. SIAM J. Matrix Anal. Appl., 21(4): , C. B. Moler and C. F. Van Loan. Nineteen dubious ways to compute the exponential of a matrix. SIAM Rev., 20(4): , C. B. Moler and C. F. Van Loan. Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM Rev., 45(1):3 49, MIMS Nick Higham Computing the Action of the Matrix Exponential 28 / 29

39 References IV L. N. Trefethen, J. A. C. Weideman, and T. Schmelzer. Talbot quadratures and rational approximations. BIT, 46(3): , MIMS Nick Higham Computing the Action of the Matrix Exponential 29 / 29