Krylov methods for the computation of matrix functions

Transcription

1 Krylov methods for the computation of matrix functions Jitse Niesen (University of Leeds) in collaboration with Will Wright (Melbourne University) Heriot-Watt University, March 2010

2 Outline Definition of matrix functions via series via diagonalization via contour integration Motivation Centrality measure Exponential integrators Direct methods for computation of matrix exponential Padé approximation with scaling and squaring Krylov method for computation of matrix functions The basic idea Implementation (sketch) Leja point interpolation Experiment: Heston equation for prizing derivatives Conclusions

3 Matrix functions via series Given a scalar function f : C C with Taylor series f (x) = a 0 + a 1 x a 2x a 3x 3 +, the matrix function f : C n n C n n is defined by f (X ) = a 0 + a 1 X a 2X a 3X 3 +. This talk concentrates on the matrix exponential exp(x ) = n=0 1 n! X n. Matrix functions inherit many properties of the corresponding scalar functions, e.g., solution of X = AX is X (t) = exp(at) X (0). Definition needs to be adapted if scalar function f has singularities.

4 Matrix functions via diagonalization Scalar function: f (x) = a 0 + a 1 x a 2x a 3x 3 + Matrix function: f (X ) = a 0 + a 1 X a 2X a 3X 3 + Matrix function on diagonal matrix reduces to scalar function d f (d 1 ) f 0 d =. 0 f (d 2 ) d n f (d n ) If X can be diagonalized, X = VDV 1, then f (X ) = a n (VDV 1 ) n = n=0 a n VD n V 1 = Vf (D)V 1. n=0 This yields another definition of matrix functions. Can be extended to non-diagonalizable matrices by continuity.

5 Matrix functions via contour integration The third definition of matrix functions is f (X ) = 1 f (z) (X zi ) 1 dz, 2πi where integral is along contour encircling the eigenvalues of X. Equivalent to previous definition by residue theorem; the integrand has poles at the eigenvalues. This definition is theoretically convenient because it always works. References 1. Gantmacher, The Theory of Matrices, Horn & Johnson, Topics in Matrix Analysis, N. Higham, Functions of Matrices, 2008.

6 Outline Definition of matrix functions via series via diagonalization via contour integration Motivation Centrality measure Exponential integrators Direct methods for computation of matrix exponential Padé approximation with scaling and squaring Krylov method for computation of matrix functions The basic idea Implementation (sketch) Leja point interpolation Experiment: Heston equation for prizing derivatives Conclusions

7 Centrality measures of a network A graph is a collection of nodes, some of which are connected by edges. We want to know which nodes are central. One centrality measure is the degree: the number of edges of a node. This is a local measure; wish to extend this to incorporate global information. Let X be the adjacency matrix of a graph: x ij is 1 if there is an edge between nodes i and j and 0 otherwise. The (i, j) entry of X 2 is k x ikx kj. This counts the number of paths of length 2 from i to j. The (i, i) entry of X n counts the number of n-cycles that i is on. In particular, the (i, i) entry of X 2 is the degree. Estrada & Rodríguez-Velázquez (2005) propose to use the diagonal entries of exp(x ) = n 1 n! X n as a centrality measure.

8 Matrix functions solve differential equations The solution of x = ax, x(0) = x 0 is x(t) = exp(at)x 0. The solution of x = Ax, x(0) = x 0 is x(t) = exp(ta)x 0. The solution of x = ax + b, x(0) = 0 is x(t) = t where ϕ 1 (z) = ( exp(z) 1 ) /z. 0 exp(aτ)b dτ = exp(aτ) 1 b = tϕ 1 (at)b. a The solution of x = Ax + b, x(t) = 0 is x(t) = tϕ 1 (ta)b. The solution of x = Ax + ct, x(0) = 0 is x(t) = t 2 ϕ 2 (ta)c, where ϕ 2 (z) = ( exp(z) 1 z ) /z 2. These results can be combined by superposition.

9 Exponential Euler method Solution of x (t) = Lx(t) + v 0 + tv t2 v 2 +, x(0) = x 0, where x(t), x 0, v 0, v 1,... are vectors and L is a matrix, is x(t) = exp(tl)x 0 + hϕ 1 (tl)v 0 + h 2 ϕ 2 (tl)v 1 + h 3 ϕ 3 (tl)v 2 + Consider x = Lx + N(x) (L = linear, N = nonlinear) Replace the nonlinear term with the constant N(x(0)) N(x(t)) (for small t) and use the results from the previous slide: x(t) = exp(tl)x(0) + ϕ 1 (tl) N(x(0)). This leads to the exponential Euler method x n+1 = exp(tl)x n + ϕ 1 (hl)n(x n ). This method is not affected by stiffness in L. (Certaine 1960)

10 Outline Definition of matrix functions via series via diagonalization via contour integration Motivation Centrality measure Exponential integrators Direct methods for computation of matrix exponential Padé approximation with scaling and squaring Krylov method for computation of matrix functions The basic idea Implementation (sketch) Leja point interpolation Experiment: Heston equation for prizing derivatives Conclusions

11 Computation using series The series exp(x ) = I + X X X 3 + is one method to compute the matrix exponential. Series converges slowly away from the origin, so combine with scaling and squaring. Set Y = 2 k X, compute exponential, square exp(y ) k times. This uses the identity exp(x ) = (exp( 1 2 X ))2. Further improvement is to use Padé approximation instead of polynomial approximation: exp(x) c 0 + c 1 X + c 2 X 2 + c 3 X 3 d 0 + d 1 X + d 2 X 2 + d 3 X 3 where the coefficients c i and d i are chosen so that the error is as small as possible for small x (in practice, we use more coefficients). Padé with scaling and squaring is the most popular general-purpose method for computing the matrix exponential. (Lawson 1967; Higham 2005; Al-Mohy & Higham 2009)

12 Computing matrix functions by diagonalization If X = VDV 1 then f (X ) = Vf (D)V 1. So compute matrix function by first diagonalizing the matrix. This works well for some matrices, in particular symmetric matrices. However, it fails if X is (close to) non-diagonalizable. Computing matrix functions by integration Use f (X ) = 1 2πi f (z) (X zi ) 1 dz and evaluate contour integral with trapezium rule. (Kassam & Trefethen 2005; Schmelzer & Trefethen 2007) This has its appeal, but is expensive, especially if eigenvalues are far apart or matrix is very non-normal. (Ashi, Cummings & Matthews 2009)

13 Outline Definition of matrix functions via series via diagonalization via contour integration Motivation Centrality measure Exponential integrators Direct methods for computation of matrix exponential Padé approximation with scaling and squaring Krylov method for computation of matrix functions The basic idea Implementation (sketch) Leja point interpolation Experiment: Heston equation for prizing derivatives Conclusions

14 The idea behind Krylov methods In exponential integrators, and other applications, X is a large matrix. On the other hand, we need not f (X ) but f (X )b. A matrix-free method uses only matrix-vector products. This leads to the Krylov subspace K m (X, b) = span{b, Xb, X 2 b,..., X m 1 b}. This basis is very ill-conditioned, so use Gram Schmidt (here called Arnoldi or Lanczos) to get an orthogonal basis: K m (X, b) = span{v 1, v 2, v 3,..., v m }. Put the vectors v j in an n-by-m matrix V m (where n is size of X ). Basis transformation is encoded in m-by-m matrix H m.

15 The idea behind Krylov methods II Considering the matrices as linear mappings: X is map from big space C n to itself. V m is projection from C n onto small K m (X, b). V T m is extension from K m (X, b) to C n ; V m V T m = Id. H m is projection of action of X on K m (X, b); H m = V m XV T m. Krylov methods use the approximation X Vm T H m V m. In the context of matrix functions, use f (X ) = a k X k a k (Vm T H m V m ) k n=k = k=0 a k Vm T (H m ) k V m = Vm T f (H m )V m. k=0 Krylov approximation replaces big matrix X by small matrix H m.

16 Steps towards a practical method Estimate error in Krylov approximation. Use error estimate to adaptively choose dimension m of Krylov subspace (and to correct approximation). Use recursion relation to combine several terms in one: exp(x )b 0 + ϕ 1 (X )b 1 + ϕ 2 (X )b 2 = w 0 + ϕ 2 (X )w 2. Compute ϕ function using trick ([ ]) X b exp = 0 0 [ exp(x ) ϕ1 (X )b 0 1 Problem: A priori estimate suggests that optimal m is proportional to spectral radius ρ(x ). Thus, introduce time stepping similar to scaling-and-squaring. This is implemented in the matlab code phipm. ].

17 Leja point interpolation Krylov methods project matrix function on Krylov subspace K m (X, b) = span{b, Xb, X 2 b,..., X m 1 b}. They approximate f (X )b by element in K m (X, b). Thus, approximation is of form p(x )b, with p a polynomial. In fact, Krylov method performs polynomial interpolation in Ritz values (eigenvalues of H m ). Good because Ritz values approximate eigenvalues of A. (Saad 1992) But Ritz values may not be the best interpolation points. If bound on eigenvalues of A is known, Leja points may be better and show good performance in experiments. (Caliari & Ostermann 2009; Tambue, Lord & Geiger 2010)

18 Outline Definition of matrix functions via series via diagonalization via contour integration Motivation Centrality measure Exponential integrators Direct methods for computation of matrix exponential Padé approximation with scaling and squaring Krylov method for computation of matrix functions The basic idea Implementation (sketch) Leja point interpolation Experiment: Heston equation for prizing derivatives Conclusions

19 Prizing derivatives The Black Scholes model assumes that the value S t of the underlying asset follows a geometric Brownian motion: ds t = µs t dt + σs t dw t. The no-arbitrage principle (no strategy guarantees a profit) implies that price u of derivative given s (price of asset) satisfies u t σ2 s 2 u u + rs s2 s where r is risk-free interest rate. ru = 0. The Heston model assumes that volatility σ is not constant: ds t = µs t dt + ν t S t dw S t, dν t = κ(η ν t ) dt + λ ν t dw ν t. Parameters are such that ν t > 0.

20 Heston PDE Heston model + no-arbitrage principle + magic yields u t = 1 2 νs2 2 u s 2 +ρλνs 2 u ν s λ2 ν 2 u u +rs ν2 s +κ(η ν) u ν ru. Add boundary conditions for modelling European option. Use standard second-order finite differences and incorporate boundary conditions to get ODE of the form u = Au + v 1, u(0) = v 0. The matrix A has size 5100 and non-zero elements. The ODE can be solved by evaluating ϕ-functions. In t Hout (2007) advocates the use of ADI. We compare his ADI schemes with standard code ode15s and our phipm (in matlab).

21 Error vs time maximum error Phipm Crank Nicolson Douglas Craig Hundsdorfer Ode15s cpu time

22 Krylov dimension m and step size τ m x τ t

23 Error estimate (blue) and actual error (red) 4 x error t

24 Conclusion Our matlab code looks good. But it needs more testing. Current work: Rewrite code in compiled languaged (C++). Investigate instability; compare exp l trick with direct method. Extend test to exotic options (American, Asian, barrier). Other work shows our code can be used in exponential integrators to solve semi-linear PDEs. Excellent with spectral discretiation, promising for FD yielding mildly stiff problems, disappointing for very stiff. Compare with other methods for evaluating ϕ functions, especially Leja point interpolation and also RD-rational approximations (Moret & Novati 2004). For more details, see: Niesen & Wright, A Krylov subspace algorithm for evaluating the ϕ-functions appearing in exponential integrators, arxiv: matlab code is available at my home page.