Krylov Subspace Methods for the Evaluation of Matrix Functions. Applications and Algorithms

Transcription

1 Krylov Subspace Methods for the Evaluation of Matrix Functions. Applications and Algorithms 2. First Results and Algorithms Michael Eiermann Institut für Numerische Mathematik und Optimierung Technische Universität Bergakademie Freiberg, Germany Wintersemester 2010/11 Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 1 / 20

2 Outline 1 Further applications 2 Properties of matrix functions 3 Computational methods Scaling and squaring The Schur-Parlett algorithm An aside: Functions of 2-by-2 triangular matrices Newton s method Trapezoidal rule + conformal maps Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 2 / 20

3 Further applications We saw that exp(a) is important for solving differential equations. But there are other functions involved, e.g., y (t) = Ay(t), y(0) = b, y (0) = c, is solved by y(t) = cos(t A)b + ( A) 1 sin(t A)c. But for now we will stay with exp(a): Numerical simulation of transient electromagnetic (TEM) geophysical explorations. Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 3 / 20

4 Time-evolution of electrical field E(x, t) can be modelled via σe t + (µ 1 E) = 0 on Ω R 3, E(x, t = 0) = E (0) (x) + boundary conditions. Discretization in space leads to homogeneous linear initial value problem with time-independent coefficients. Solution has the form 2000 u(t) = exp( ta)u 0, where A discretizes σ 1 (µ ) and is large 2000 and sparse z x y Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 4 / 20

5 Properties of matrix functions Let A C n n and f, f j, g functions defined on Λ(A), then (f + g)(a) = f (A) + g(a), (fg)(a) = f (A)g(A). f (A T ) = f (A) T, but f (A H ) = f (A) H is false in general. If B commutes with A then B commutes with f (A). Let P(z 1,..., z l ) be a polynomial in l variables. If f (z) := P(f 1 (z),..., f l (z)) = 0 on Λ(A), i.e., f (ν) (λ µ ) = 0 for µ = 1,..., k and ν = 0,..., n µ 1, then f (A) = P(f 1 (A),..., f l (A)) = O. Examples: sin 2 (A) + cos 2 (A) = I, exp( A) = exp(a) 1. f (A I) = f (A) I, f (I A) = I f (A). If AB = BA then exp(a + B) = exp(a) exp(b). exp(a I + I B) = exp(a) exp(b). Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 5 / 20

6 Computational methods As the applications discussed so far show, it is very often not f (A) but f (A)b which needs to be computed. We are interested in problems, where the action of a matrix function on a vector is required and where the matrix is large and sparse or structured. Here, evaluating f (A)b by first computing (the usually dense matrix) f (A) is unfeasible. But since we are discussing projection methods we also need to compute f (A)b for small or medium-sized matrices A. This is why we also discuss methods that compute f (A). Many methods rely on the following observation: Let {s m } be a sequence of "simple" analytic functions (simple means that s m (A) can be computed without major difficulties) which converges uniformly on a compact set Ω to f. If Λ(A) Ω then lim m s m (A) = f (A). Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 6 / 20

7 Scaling and squaring The MATLAB function expm is based on this idea (and some clever tricks). Ingredients: exp(a) = exp(a/t) t. We use it for t = 2 s, s N. exp(z) can be well approximated by a Padé fraction if z is small. The (k, m) Padé fraction of f (analytic in 0) is a rational function r k,m (z) = p k,m (z)/q k,m (z) of type (k, m), i.e., p k,m P k and q k,m P m, with f (z) r k,m (z) = O ( z k+m+1) as z 0. In other words, if we expand r k,m in a Taylor series at z 0 = 0 then its coefficients coincide with the Taylor coefficients of f up to the index k + m. The (k, m) Padé fraction is uniquely determined if it exists (provided we normalize the denominator by q k,m (0) = 1). Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 7 / 20

8 For f (z) = exp(z) the Padé fractions of all types exist and are known explicitly. Here, only diagonal Padé fractions, i.e., k = m, will be applied. We denote them by r m = p m /q m. Method: Choose scaling parameter s. Approximate exp(a/2 s ) by r m (A/2 s ) = p m (A/2 s )[q m (A/2 s )] 1 for some m. Then exp(a) is approximated by r m (A/2 s ) 2s (repeated squaring). s and m are chosen such that exp(a) is approximated with backward error bounded by the unit roundoff and with minimal cost. More tricks involved. Costs: at most 5 + log 2 ( A 1 /2) matrix multiplications, one solution of a system q m (B)X = p m (A). [Higham (2005)] Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 8 / 20

9 The Schur-Parlett algorithm Important ingredient: If [ ] A1,1 A A = 1,2 O A 2,2 with square diagonal blocks then [ f (A1,1 ) X f (A) = O f (A 2,2 ) where X solves the Sylvester equation ], A 1,1 X X A 2,2 = f (A 1,1 )A 1,2 A 1,2 f (A 2,2 ). Proof. Compare blocks in Af (A) = f (A)A. Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 9 / 20

10 More general, if A 1,1 A 1,2 A 1,b O A 2,2 A 2,b A =..... O O A b,b is block triangular with square diagonal blocks then F 1,1 F 1,2 F 1,b O F 2,2 F 2,b f (A) =..... O O F b,b (partitioning conformal to that of A), where F j, j = f (A j,j ) (j = 1, 2,..., b), F i, j solves A i,i F i,j F i,j A j,j = F i,i A i,j A i,j F j,j + (F i,k A k,j A i,k F k,j ) k=1 (i = j 1, j 2,..., 1) [Parlett (1976)]. Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 10 / 20 j 1

11 An aside: Functions of 2-by-2 triangular matrices Let [ ] λ1 α A = 0 λ 2 and f be defined on Λ(A). Then f (A) = f (λ 1) α f (λ 2) f (λ 1 ) λ 2 λ 1 0 f (λ 2 ) if λ 1 λ 2, and f (A) = f (λ) αf (λ) 0 f (λ) if λ 1 = λ 2 = λ. Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 11 / 20

12 Problem: Solution of AX XB = C is ill-conditioned if Λ(A) and Λ(B) are not well separated. The Schur-Parlett algorithm: Compute the Schur form of A: T = U H AU. Reorder Schur form (swapping) to T = [ T i,j ] = V H TV such that min{ λ µ : λ Λ( T i,i ), µ Λ( T j,j ), i j} > δ (usually, δ = 0.1) (separation between blocks), for every block T i,i of size bigger than 1: For each λ Λ( T i,i ) there is µ Λ( T i,i ) such that λ µ δ (separation within blocks). Approximate f ( T i,i ) by a truncated Taylor series. Use the Parlett recursion to compute the off-diagonal blocks of f ( T ). Transform back. [Davies & Higham (2005)] Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 12 / 20

13 Newton s method The sign function is defined by { +1, if Real(z) > 0, sign(z) = 1, if Real(z) < 0. (The sign function is not defined on the imaginary axis). For a matrix A C n n without purely imaginary eigenvalues the matrix sign function sign(a) can then be determined as follows: First compute the Jordan canonical form of A, [ ] J+ A = T T 1, where J + [J ] collects all Jordan blocks belonging to eigenvalues with positive [negative] real parts. Then [ ] I sign(a) = T T 1. I J Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 13 / 20

14 The matrix sign function has important applications. Assume that we want to solve the Sylvester equation AX + XB = C, A C m m, B C n n, C C m n. Additionally, we suppose that sign(a) = I m and sign(b) = I n which is fulfilled, e.g., if A is positive real (i.e., Real(λ) > 0 for all λ Λ(A)) and if the Lyapunov equation AX + XA H = C is solved. Now [ ] [ ] [ ] [ A C Im X A O Im X = O B O O B O and thus, I n I n ] 1 [ A C sign O B ] [ Im X = O I n ] [ Im O O I n ] [ Im X O I n ] 1 [ Im 2X = O I n ]. Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 14 / 20

15 A similar approach leads to the solution of the algebraic Riccati equation XFX A H X XA = G [Roberts (1980)]. Since sign(a) solves X 2 I = O, the Newton s method ( ) X m+1 = 1 2 X m + Xm 1, X 0 = A, is one way to compute sign(a): If A has no purely imaginary eigenvalues then {X m } converges quadratically to sign(a), (see [Higham (2008)]). X m+1 sign(a) Xm X m sign(a) 2 Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 15 / 20

16 Trapezoidal rule + conformal maps Under the usual assumptions on f, A and Γ f (A)b = 1 f (ζ)(ζi A) 1 b dζ =: 2πi Γ 1 g(ζ) dζ. 2πi Γ Here, Γ = {ζ : ζ α = ρ}, i.e., ζ(θ) = α + ρ exp(iθ), 0 θ 2π, f (A)b = 1 2π 2π 0 (ζ(θ) α)g(ζ(θ)) dθ (integrand is periodic function of θ with period 2π). Apply p-point trapezoidal rule, p 1 f (A)b 1 (ζ k α)g(ζ k ), with ζ k α = ρ exp(2πki/m). p k=0 Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 16 / 20

17 Apply this to f analytic in C \ (, 0], A with Λ(A) [m, M] R: Figure from [Hale, Higham & Trefethen (2008)] Does not work, i.e., slow convergence, unless m 0. Idee: Change of variables, goes back to [Iri, Moriguti & Takasawa (1970)], [Takahasi & Mori (1973)],..., technique described here due to [Hale, Higham & Trefethen (2008)]. Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 17 / 20

18 Construct conformal map z = z(s) from annulus onto C \ ((, 0] [m, M]) and integrate with respect to s. p 1 Figure from [Hale, Higham & Trefethen (2008)] f (A)b f p (A, b) := 1 (ζ(s k ) α)g(ζ(s k ))ζ (s k ). p k=0 Theorem [Hale, Higham & Trefethen 2008] f (A)b f p (A, b) = O(exp( π 2 p/(log(m/m) + 3))). Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 18 / 20

19 Figure from [Hale, Higham & Trefethen (2008)] Method can be enhanced for f without singularities in (, 0) and algebraic branch points in 0. Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 19 / 20

20 Pointers to the literature P. I. Davies and N. J. Higham. A Schur Parlett algorithm for computing matrix functions. SIAM J. Matrix Anal. Appl. 25, (2003). P. I. Davies and N. J. Higham. Computing f (A)b for matrix functions f. In: QCD and Numerical Analysis III (A. Boriçi, A. Frommer, B. Joó, A. Kennedy, and B. Penleton, eds.), Lecture Notes in Computational Science and Engineering vol. 47, Springer-Verlag, Berlin 2005, N. Hale, N. J. Higham, and L. N. Trefethen. Computing A α, log(a), and related matrix functions by contour integrals. SIAM J. Numer. Anal. 46, (2008). N. J. Higham. The scaling and squaring method for the matrix exponential revisited. SIAM J. Matrix Anal. Appl. 26, (2005). M. Iri, S. Moriguti and Y. Takasawa. On a certain quadrature formula (in Japanese). Kokyuroku RIMS, Kyoto Univ (1970). Translated into English in J. Comp. Appl. Math. 17, 2 20 (1987). B. N. Parlett. A recurrence among the elements of functions of triangular matrices. Linear Algebra Appl. 14, (1976). J. D. Roberts. Linear model reduction and solution of the agebraic Riccati equation by use of the sign function. Int. J. Control 32, (1980). H. Takahasi and M. Mori. Quadrature formulas obtained by variable transformation. Numer. Math. 12, (1973). Michael Eiermann (TU Freiberg) Matrix Functions WS 2010/11 20 / 20