Research CharlieMatters

Transcription

1 Research CharlieMatters Van Loan and the Matrix Exponential February 25, 2009 Nick Nick Higham Director Schoolof ofresearch Mathematics The School University of Mathematics of Manchester nickhigham.wordpress.com Numerical Linear and Multilinear Algebra: Celebrating Charlie Van Loan SIAM Annual Meeting, Boston July 13, / 6

2

3 Charlie and MATLAB Pioneer of the use of MATLAB in teaching. Popularized the colon notation in numerical linear algebra. Nick Higham Charlie Van Loan 3 / 22

4 Laguerre (1867): Peano (1888):

5 e A in Applied Mathematics Frazer, Duncan & Collar, Aerodynamics Division of NPL: aircraft flutter, matrix structural analysis. Elementary Matrices & Some Applications to Dynamics and Differential Equations, Emphasizes importance of e A. Arthur Roderick Collar, FRS ( ): First book to treat matrices as a branch of applied mathematics. Nick Higham Charlie Van Loan 5 / 22

6 University of Manchester Period, Science Research Council Research Fellow Charles F. Van Loan. A study of the matrix exponential. Numerical Analysis Report No. 10, University of Manchester, Manchester, UK, August I became interested in the matrix exponential during the preparation of a talk I gave on the subject in 1974 here at Manchester. Since then I have been motivated by the work of B.N. Parlett [20] and by C. B. Moler with his n bad ways to compute the matrix exponential (n 9). Nick Higham Charlie Van Loan 6 / 22

7

8 Contents Defines f (A) by Cauchy integral. Gives explicit, divided difference-based formula for f (T ), T triangular. Bounds and perturbation bounds for e ta. Backward error result for Padé approximants. Extensive use of Schur decomposition. one of the most basic tenets of numerical algebra, namely, anything that the Jordan decomposition can do, the Schur decomposition can do better! Nick Higham Charlie Van Loan 8 / 22

9 A Bibliography of the Matrix Exponential Dear Exponential Freak, I collect facts, algorithms, and references about e At. If you have one that is missing from these references, please write to me. (Undated.) Nick Higham Charlie Van Loan 9 / 22

10 Sensitivity of the Exponential SINUM, Jordan and Schur-based bounds. Bounded the condition number. Showed condition number is minimal for normal A. Contemporary work by Kågström. Open Question When is the condition number of e A large? Nick Higham Charlie Van Loan 10 / 22

11 Integrals ([ ]) A11 A exp 12 = 0 A 22 ea e A11(1 s) A 12 e A22s ds. 0 e A 22 CVL, IEEE Trans. Automat. Control (1978): use in reverse direction to evaluate integrals involving e A via exponentials of bock triangular matrices. 4th most cited paper! Nick Higham Charlie Van Loan 11 / 22

12 Nineteen Dubious Ways (1) Moler and Van Loan, SIREV, A C n n : Power series Limit Scaling and squaring I + A + A2 2! + A3 3! + lim (I + A/s)s (e s A/2s ) 2s Cauchy integral Jordan form Interpolation 1 n i 1 e z (zi A) 1 dz Z diag(e J k )Z 1 f [λ 1,..., λ i ] (A λ j I) 2πi Γ i=1 j=1 Differential system Schur form Padé approximation Y (t) = AY (t), Y (0) = I Qdiag(e T )Q p km (A)q km (A) 1 Nick Higham Charlie Van Loan 12 / 22

13 Nineteen Dubious Ways (2) Scaling and squaring: e A = ( e A/s) s. Hump phenomenon: e A e A/s s, instability. Backward error analysis for Padé approximants. S&S: Padé more efficient than Taylor. How to choose Padé degree to achieve given accuracy at min cost. Nick Higham Charlie Van Loan 13 / 22

14 Nineteen Dubious Ways (2) Scaling and squaring: e A = ( e A/s) s. Hump phenomenon: e A e A/s s, instability. Backward error analysis for Padé approximants. S&S: Padé more efficient than Taylor. How to choose Padé degree to achieve given accuracy at min cost. Pros and cons of various other methods explained; emphasis on numerical stability. Republished in SIREV 2003 with Method 20: Krylov space methods and new sections. Taken together, CVL s most cited-paper: 2800 citations. Nick Higham Charlie Van Loan 13 / 22

15 History of the S&S Method (1) J. D. Lawson (1967): exponential integrators for ODE. Suggests using ( e ) 2 s A 2 s = e A with diagonal Padé approximants (uncond. stable). Ward (1977): m = 8, 2 s A 1 1. MATLAB expm up to 2005: m = 6, 2 s A 1/2. In 2004 I started the e A chapter of my book Functions of Matrices: Theory and Computation... Nick Higham Charlie Van Loan 14 / 22

16 History of the S&S Method (2) H (2005): choose (s, m) at run-time based on sharper, non-explicit error bounds. Algorithm m θ m for m = [ ] if A 1 θ m, X = r m (A), quit, end end A A/2 s with s 0 minimal s.t. A/2 s 1 θ 13 = X = r s 13 by repeated squaring. Nick Higham Charlie Van Loan 15 / 22

17 History of the S&S Method (2) H (2005): choose (s, m) at run-time based on sharper, non-explicit error bounds. Algorithm m θ m for m = [ ] if A 1 θ m, X = r m (A), quit, end end A A/2 s with s 0 minimal s.t. A/2 s 1 θ 13 = X = r s 13 by repeated squaring. Faster, and in practice more accurate, than previous alg. Nick Higham Charlie Van Loan 15 / 22

18 History of the S&S Method (3) Overscaling (Kenney & Laub, 1998; Dieci & Papini, 2000) : large A causes larger than necessary s to be chosen, with harmful effect on accuracy. Al-Mohy & H (2009) avoid via sharper bounds. Lemma If k = pm 1 + qm 2 with p, q N and m 1, m 2 N {0}, A k 1/k max ( A p 1/p, A q 1/q). Take {p, q} = {r, r + 1} for k r(r 1). Improved expm in MATLAB 2015b. Nick Higham Charlie Van Loan 16 / 22

19 Software Scene Al-Mohy & H (2009) scaling & squaring algorithm for e A is in Julia, MATLAB, NAG Library, R, SciPy. H & Deadman, A Catalogue of Software for Matrix Functions (2016). Nick Higham Charlie Van Loan 17 / 22

20 Action of the Exponential Compute e A b without first computing e A. What was originally wanted by Lawson (1967), but problems small there! Krylov methods commonly used. Nick Higham Charlie Van Loan 18 / 22

21 The Sixth Dubious Way Moler & Van Loan (1978, 2003) Nick Higham Charlie Van Loan 19 / 22

22 Al-Mohy & Higham (2011) expmv Exploit, for integer s, e A B = (e s 1A ) s B = e s 1A e s 1A e s 1 A }{{} B. s times Choose s so T m (s 1 A) = m (s 1 A) j j=0 j! e s 1A. Then B i+1 = T m (s 1 A)B i, i = 0: s 1, B 0 = B yields B s e A B. Nick Higham Charlie Van Loan 20 / 22

23 Al-Mohy & Higham (2011) expmv Exploit, for integer s, e A B = (e s 1A ) s B = e s 1A e s 1A e s 1 A }{{} B. s times Choose s so T m (s 1 A) = m (s 1 A) j j=0 j! e s 1A. Then B i+1 = T m (s 1 A)B i, i = 0: s 1, B 0 = B yields B s e A B. Choose s and m using b err & A k 1/k approach. Nick Higham Charlie Van Loan 20 / 22

24 Advantages of expmv Versus one-step ODE integrators: Fully exploits the linearity of the ODE. Variable order, up to m = 55. Backward error based; ODE integrator controls local (forward) errors. Overscaling avoided. Versus Krylov methods: Very competitive in cost and storage. Cost dominated by matrix vector multiplications. Black box: no need to choose/tune parameters. Nick Higham Charlie Van Loan 21 / 22

25 Charlie in Three Bullets Aficionado of the Matrix Exponential. Nick Higham Charlie Van Loan 22 / 22

26 Charlie in Three Bullets Aficionado of the Matrix Exponential. Master of Matrix Computations. Nick Higham Charlie Van Loan 22 / 22

27 Charlie in Three Bullets Aficionado of the Matrix Exponential. Master of Matrix Computations. Baseball nut and family man. "Reproduced with the permission of the Commissioner of Major League Baseball". c 1988 Nick Higham Charlie Van Loan 22 / 22

28 References I A. H. Al-Mohy and N. J. Higham. Improved inverse scaling and squaring algorithms for the matrix logarithm. SIAM J. Sci. Comput., 34(4):C153 C169, M. Aprahamian and N. J. Higham. The matrix unwinding function, with an application to computing the matrix exponential. SIAM J. Matrix Anal. Appl., 35(1):88 109, L. Dieci and A. Papini. Padé approximation for the exponential of a block triangular matrix. Linear Algebra Appl., 308: , Nick Higham Charlie Van Loan 1 / 5

29 References II R. A. Frazer, W. J. Duncan, and A. R. Collar. Elementary Matrices and Some Applications to Dynamics and Differential Equations. Cambridge University Press, Cambridge, UK, xviii+416 pp printing. N. J. Higham. Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, ISBN xx+425 pp. Nick Higham Charlie Van Loan 2 / 5

30 References III N. J. Higham and A. H. Al-Mohy. Computing matrix functions. Acta Numerica, 19: , N. J. Higham and E. Deadman. A catalogue of software for matrix functions. Version 2.0. MIMS EPrint , Manchester Institute for Mathematical Sciences, The University of Manchester, UK, Jan pp. Updated March Nick Higham Charlie Van Loan 3 / 5

31 References IV N. J. Higham and F. Tisseur. A block algorithm for matrix 1-norm estimation, with an application to 1-norm pseudospectra. SIAM J. Matrix Anal. Appl., 21(4): , R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, Cambridge, UK, ISBN X. viii+607 pp. C. S. Kenney and A. J. Laub. A Schur Fréchet algorithm for computing the logarithm and exponential of a matrix. SIAM J. Matrix Anal. Appl., 19(3): , Nick Higham Charlie Van Loan 4 / 5

32 References V C. B. Moler and C. F. Van Loan. Nineteen dubious ways to compute the exponential of a matrix. SIAM Rev., 20(4): , C. B. Moler and C. F. Van Loan. Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM Rev., 45(1):3 49, Multiprecision Computing Toolbox. Advanpix, Tokyo. Nick Higham Charlie Van Loan 5 / 5