Functions of Matrices and

Size: px

Start display at page:

Download "Functions of Matrices and"

Roger Butler
6 years ago
Views:

Functions of Matrices and Nearest Research Correlation MattersMatrices February 25, 2009 Nick Higham School Nick Higham of Mathematics The Director University of

1 Functions of Matrices and Nearest Research Correlation MattersMatrices February 25, 2009 Nick Higham School Nick Higham of Mathematics The Director University of Research of Manchester School of nickhigham.wordpress.com 1 / 6

2 Accuracy Estimates Guarantees Precision Single Double Quad Variable Efficiency Speed Parallelism Energy efficiency

3 What is a Matrix Function? It s not det(a) or trace(a), elementwise evaluation: f (a ij ), A T, matrix factor (e.g., A = LU). University of Manchester Nick Higham Matrix functions & correlation matrices 3 / 38

4 What is a Matrix Function? It s not It is det(a) or trace(a), elementwise evaluation: f (a ij ), A T, matrix factor (e.g., A = LU). A 1, A, e A,... University of Manchester Nick Higham Matrix functions & correlation matrices 3 / 38

Matrix algebra developed by Arthur Cayley, FRS (1821 1895).

5 Cayley and Sylvester Term matrix coined in 1850 by James Joseph Sylvester, FRS ( ). Matrix algebra developed by Arthur Cayley, FRS ( ). Memoir on the Theory of Matrices (1858). University of Manchester Nick Higham Matrix functions & correlation matrices 4 / 38

Sylvester (1883) gave first definition of f (A) for general f.

6 Cayley and Sylvester on Matrix Functions Cayley considered matrix square roots in his 1858 memoir. Tony Crilly, Arthur Cayley: Mathematician Laureate of the Victorian Age, Sylvester (1883) gave first definition of f (A) for general f. Karen Hunger Parshall, James Joseph Sylvester. Jewish Mathematician in a Victorian World, University of Manchester Nick Higham Matrix functions & correlation matrices 5 / 38

7 Two Definitions Definition (Taylor series) If f has a Taylor series expansion f (z) = k=0 a kz k with radius of convergence r and ρ(a) < r then f (A) = a k A k. k=0 University of Manchester Nick Higham Matrix functions & correlation matrices 6 / 38

8 Two Definitions Definition (Taylor series) If f has a Taylor series expansion f (z) = k=0 a kz k with radius of convergence r and ρ(a) < r then f (A) = a k A k. k=0 Definition (Cauchy integral formula) f (A) = 1 f (z)(zi A) 1 dz, 2πi Γ where f analytic on and inside closed contour Γ enclosing λ(a). University of Manchester Nick Higham Matrix functions & correlation matrices 6 / 38

10 Matrix Roots in Markov Models Let vectors v 2011, v 2010 represent risks, credit ratings or stock prices in 2011 and Assume a Markov model v 2011 = P v 2010, where P is a transition probability matrix. P 1/2 enables predictions to be made at 6-monthly intervals. University of Manchester Nick Higham Matrix functions & correlation matrices 8 / 38

11 Matrix Roots in Markov Models Let vectors v 2011, v 2010 represent risks, credit ratings or stock prices in 2011 and Assume a Markov model v 2011 = P v 2010, where P is a transition probability matrix. P 1/2 enables predictions to be made at 6-monthly intervals. P 1/2 is matrix X such that X 2 = P. What are P 2/3, P 0.9? P s = exp(s log P). Problem: log P, P 1/k may have wrong sign patterns regularize. University of Manchester Nick Higham Matrix functions & correlation matrices 8 / 38

12 Chronic Disease Example Estimated 6-month transition matrix. Four AIDS-free states and 1 AIDS state observations (Charitos et al., 2008) P = Want to estimate the 1-month transition matrix. Λ(P) = {1, , , , }. H & Lin (2011). Lin (2011, for survey of regularization methods. University of Manchester Nick Higham Matrix functions & correlation matrices 9 / 38

13 MATLAB: Arbitrary Powers >> A = [1 1e-8; 0 1] A = e e e+000 >> A^0.1 ans = >> expm(0.1*logm(a)) ans = e e e+000 University of Manchester Nick Higham Matrix functions & correlation matrices 10 / 38

14 MATLAB Arbitrary Power New Schur algorithm (H & Lin, 2011, 2013) reliably computes A p for any real p. New backward-error based inverse scaling and squaring alg for matrix logarithm (Al-Mohy, H & Relton, 2012) faster and more accurate. Alternative Newton-based algorithms available for A 1/q with q an integer, e.g., for X k+1 = 1 q [ (q + 1)Xk X q+1 k A ], X 0 = A, X k A 1/q. University of Manchester Nick Higham Matrix functions & correlation matrices 11 / 38

15 Solving Ordinary Differential Equations A C n n : dy dt = Ay, y(0) = y 0 y(t) = e At y 0. University of Manchester Nick Higham Matrix functions & correlation matrices 12 / 38

16 Solving Ordinary Differential Equations A C n n : dy dt = Ay, y(0) = y 0 y(t) = e At y 0. d 2 y dt 2 + Ay = 0, y(0) = y 0, y (0) = y 0 has solution y(t) = cos( At)y 0 + ( A ) 1 sin( At)y 0. University of Manchester Nick Higham Matrix functions & correlation matrices 12 / 38

17 Solving Ordinary Differential Equations A C n n : dy dt = Ay, y(0) = y 0 y(t) = e At y 0. d 2 y dt 2 + Ay = 0, y(0) = y 0, y (0) = y 0 has solution y(t) = cos( At)y 0 + ( A ) 1 sin( At)y 0. But also [ ] y = exp y ([ 0 ta t I n 0 ]) [ ] y 0. y 0 University of Manchester Nick Higham Matrix functions & correlation matrices 12 / 38

18 Phi Functions: Definition ϕ 0 (z) = e z, ϕ 1 (z) = ez 1, ϕ 2 (z) = ez 1 z,... z z 2 ϕ k+1 (z) = ϕ k(z) 1/k!. z ϕ k (z) = j=0 z j (j + k)!. University of Manchester Nick Higham Matrix functions & correlation matrices 13 / 38

19 Phi Functions: Solving ODEs y C n, A C n n. dy dt = Ay, y(0) = y 0 y(t) = e At y 0. University of Manchester Nick Higham Matrix functions & correlation matrices 14 / 38

20 Phi Functions: Solving ODEs y C n, A C n n. dy dt = Ay, y(0) = y 0 y(t) = e At y 0. dy dt = Ay + b, y(0) = 0 y(t) = t ϕ 1 (ta)b. University of Manchester Nick Higham Matrix functions & correlation matrices 14 / 38

21 Phi Functions: Solving ODEs y C n, A C n n. dy dt = Ay, y(0) = y 0 y(t) = e At y 0. dy dt dy dt = Ay + b, y(0) = 0 y(t) = t ϕ 1 (ta)b. = Ay + ct, y(0) = 0 y(t) = t 2 ϕ 2 (ta)c.. University of Manchester Nick Higham Matrix functions & correlation matrices 14 / 38

22 Exponential Integrators Consider y = Ly + N(y). N(y(t)) N(y(0)) implies y(t) e tl y 0 + tϕ 1 (tl)n(y(0)). Exponential Euler method: y n+1 = e hl y n + hϕ 1 (hl)n(y n ). Lawson (1967); recent resurgence. University of Manchester Nick Higham Matrix functions & correlation matrices 15 / 38

23 Toolbox of Matrix Functions Want software for evaluating interesting f at matrix args as well as scalar args. MATLAB has expm, logm, sqrtm, funm. The Matrix Function Toolbox (H, 2008). NAG Library: Extensive set of new codes included in Mark 23 (2012), Mark 24 (2013). University of Manchester Nick Higham Matrix functions & correlation matrices 16 / 38

24 Compute e A b Exploit, for integer s, e A b = (e s 1A ) s b = e s 1A e s 1A e s 1 A }{{} b. s times Choose s so T m (s 1 A) = m (s 1 A) j j=0 j! e s 1A. Then b i+1 = T m (s 1 A)b i, i = 0: s 1, b 0 = b yields b s e A b. Al-Mohy & H (2011), SIAM J. Sci. Comp. University of Manchester Nick Higham Matrix functions & correlation matrices 17 / 38

25 Experiment Compute e ta b for Harwell Boeing matrices: orani678, n = 2529, t = 100, b = [1, 1,..., 1] T ; bcspwr10, n = 5300, t = 10, b = [1, 0,..., 0, 1] T. 2D Laplacian matrix, poisson. tol = Alg AH ode15s time error time error orani e e-6 bcspwr e e-5 poisson e e-6 4 poisson 15 9e e-1 University of Manchester Nick Higham Matrix functions & correlation matrices 18 / 38

26 General Functions Schur Parlett algorithm (Davies & H, 2003) computes f (A) given the ability to evaluate f (k) (x) for any k and x. Implemented in MATLAB s funm. Beware unstable diagonalization algorithm: function F = funm_ev(a,fun) %FUNM_EV Evaluate general matrix % function via eigensystem. [V,D] = eig(a); F = V * diag(feval(fun,diag(d))) / V; University of Manchester Nick Higham Matrix functions & correlation matrices 19 / 38

27 Knowledge Transfer Partnership #1 University of Manchester and NAG ( ) funded by EPSRC, NAG and TSB. Developing suite of NAG Library codes for matrix functions. Extensive set of new codes included in Mark 23 (2012), Mark 24 (2013). Improvements to existing state of the art: faster and more accurate. My work also supported by 2M ERC Advanced Grant. University of Manchester Nick Higham Matrix functions & correlation matrices 20 / 38

30 Some NAG Toolbox Timings All ei vals & ei vectors of real symmetric matrix. f08fc: divide and conquer, eig: QR. n f08fc eig Matrix logarithm using the Schur Parlett alg. n f01ej logm e-4 1.0e University of Manchester Nick Higham Matrix functions & correlation matrices 23 / 38

31 Knowledge Transfer Partnership #2 University of Manchester and NAG ( ), funded by NAG and TSB. Lead academic: Jack Dongarra (UT Knoxville, Oak Ridge National Laboratory & U Manchester) Developing, tuning and integrating key components of the Parallel Linear Algebra for Scalable Multicore Architectures (PLASMA) library to support NAG products. University of Manchester Nick Higham Matrix functions & correlation matrices 24 / 38

32 Questions From Finance Practitioners Given a real symmetric matrix A which is almost a correlation matrix what is the best approximating (in Frobenius norm?) correlation matrix? I am researching ways to make our company s correlation matrix positive semi-definite. Currently, I am trying to implement some real options multivariate models in a simulation framework. Therefore, I estimate correlation matrices from inconsistent data set which eventually are non psd. University of Manchester Nick Higham Matrix functions & correlation matrices 25 / 38

33 Correlation Matrix An n n symmetric positive semidefinite matrix A with a ii 1. Properties: symmetric, 1s on the diagonal, off-diagonal elements between 1 and 1, eigenvalues nonnegative. University of Manchester Nick Higham Matrix functions & correlation matrices 26 / 38

34 Correlation Matrix An n n symmetric positive semidefinite matrix A with a ii 1. Properties: symmetric, 1s on the diagonal, off-diagonal elements between 1 and 1, eigenvalues nonnegative. Is this a correlation matrix? University of Manchester Nick Higham Matrix functions & correlation matrices 26 / 38

35 Correlation Matrix An n n symmetric positive semidefinite matrix A with a ii 1. Properties: symmetric, 1s on the diagonal, off-diagonal elements between 1 and 1, eigenvalues nonnegative. Is this a correlation matrix? Spectrum: , , University of Manchester Nick Higham Matrix functions & correlation matrices 26 / 38

36 How to Proceed Make ad hoc modifications to matrix: e.g., shift negative e vals up to zero then diagonally scale. Plug the gaps in the missing data, then compute an exact correlation matrix. Compute the nearest correlation matrix in the weighted Frobenius norm ( A 2 = i,j w iw j a 2 ij ). Given approx correlation matrix A find correlation matrix C to minimize A C. Constraint set is a closed, convex set, so unique minimizer. University of Manchester Nick Higham Matrix functions & correlation matrices 27 / 38

37 Alternating Projections Algorithm von Neumann (1933): for subspaces. Dykstra (1983): corrections for closed convex sets. S 1 S 2 Easy to implement. Guaranteed convergence, at a linear rate. Can add further constraints/projections. University of Manchester Nick Higham Matrix functions & correlation matrices 28 / 38

38 Unexpected Applications Some recent papers: Applying stochastic small-scale damage functions to German winter storms (2012) Estimating variance components and predicting breeding values for eventing disciplines and grades in sport horses (2012) Characterisation of tool marks on cartridge cases by combining multiple images (2012) Experiments in reconstructing twentieth-century sea levels (2011) University of Manchester Nick Higham Matrix functions & correlation matrices 29 / 38

40 Newton Method Qi & Sun (2006): convergent Newton method based on theory of strongly semismooth matrix functions. Globally and quadratically convergent. Algorithmic improvements by Borsdorf & H (2010). Implemented in NAG codes g02aaf (g02aac) and g02abf (weights, lower bound on ei vals Mark 23). University of Manchester Nick Higham Matrix functions & correlation matrices 31 / 38

41 Newton Method Qi & Sun (2006): convergent Newton method based on theory of strongly semismooth matrix functions. Globally and quadratically convergent. Algorithmic improvements by Borsdorf & H (2010). Implemented in NAG codes g02aaf (g02aac) and g02abf (weights, lower bound on ei vals Mark 23). University of Manchester Nick Higham Matrix functions & correlation matrices 31 / 38

42 Alternating Projections vs Newton Matrix n tol Code Time (s) Iters 1. Random 100 1e-10 g02aa alt proj Random 500 1e-10 g02aa alt proj Real-life e-4 g02aa alt proj University of Manchester Nick Higham Matrix functions & correlation matrices 32 / 38

43 Factor Model (1) ξ = }{{} X η }{{} n k k 1 + F }{{} n n }{{} ε n 1 where var(ξ i ) 1, F = diag(f ii ). Implies k j=1, η i, ε i N(0, 1), x 2 ij 1, i = 1: n. Multifactor normal copula model. Collateralized debt obligations (CDOs). Multivariate time series. University of Manchester Nick Higham Matrix functions & correlation matrices 33 / 38

44 Factor Model (2) Yields correlation matrix of form C(X) = D + XX T = D + k j=1 x j x T j, D = diag(i XX T ), X = [x 1,..., x k ]. C(X) has k factor correlation matrix structure. 1 y1 T y 2... y1 T y n y C(X) = 1 T y y T n 1 y n, y i R k. y1 T y n... yn 1 T y n 1 University of Manchester Nick Higham Matrix functions & correlation matrices 34 / 38

46 k Factor Problem min X R n k f (X) := A C(X) 2 F subject to k j=1 x 2 ij 1. Nonlinear objective function with convex quadratic constraints. Some existing algs ignore the constraints. University of Manchester Nick Higham Matrix functions & correlation matrices 36 / 38

47 Algorithms Algorithm based on spectral projected gradient method (Borsdorf, H & Raydan, 2010). Respects the constraints, exploits their convexity, and converges to a feasible stationary point. NAG routine g02aef (Mark 23). Principal factors method (Andersen et al., 2003) has no convergence theory and can converge to an incorrect answer. University of Manchester Nick Higham Matrix functions & correlation matrices 37 / 38

48 Conclusions Matrix functions a powerful and versatile tool, with excellent algs available. Beware unstable/impractical algs in literature! Working with NAG to implement state of the art f (A) algs for NAG Library. Excellent algs available in NAG Library for nearest correlation matrix problems. Further improvements coming. Beware algs in literature that may not converge or converge to wrong solution! Keen to hear about your matrix problems. University of Manchester Nick Higham Matrix functions & correlation matrices 38 / 38

49 References I A. H. Al-Mohy and N. J. Higham. A new scaling and squaring algorithm for the matrix exponential. SIAM J. Matrix Anal. Appl., 31(3): , A. H. Al-Mohy and N. J. Higham. Computing the action of the matrix exponential, with an application to exponential integrators. SIAM J. Sci. Comput., 33(2): , A. H. Al-Mohy and N. J. Higham. Improved inverse scaling and squaring algorithms for the matrix logarithm. SIAM J. Sci. Comput., 34(4):C153 C169, University of Manchester Nick Higham Matrix functions & correlation matrices 1 / 10

50 References II A. H. Al-Mohy, N. J. Higham, and S. D. Relton. Computing the Fréchet derivative of the matrix logarithm and estimating the condition number. MIMS EPrint , Manchester Institute for Mathematical Sciences, The University of Manchester, UK, July pp. Revised December L. Anderson, J. Sidenius, and S. Basu. All your hedges in one basket. Risk, pages 67 72, Nov University of Manchester Nick Higham Matrix functions & correlation matrices 2 / 10

51 References III R. Borsdorf and N. J. Higham. A preconditioned Newton algorithm for the nearest correlation matrix. IMA J. Numer. Anal., 30(1):94 107, R. Borsdorf, N. J. Higham, and M. Raydan. Computing a nearest correlation matrix with factor structure. SIAM J. Matrix Anal. Appl., 31(5): , T. Charitos, P. R. de Waal, and L. C. van der Gaag. Computing short-interval transition matrices of a discrete-time Markov chain from partially observed data. Statistics in Medicine, 27: , University of Manchester Nick Higham Matrix functions & correlation matrices 3 / 10

52 References IV T. Crilly. Arthur Cayley: Mathematician Laureate of the Victorian Age. Johns Hopkins University Press, Baltimore, MD, USA, ISBN xxi+610 pp. P. I. Davies and N. J. Higham. A Schur Parlett algorithm for computing matrix functions. SIAM J. Matrix Anal. Appl., 25(2): , University of Manchester Nick Higham Matrix functions & correlation matrices 4 / 10

53 References V R. A. Frazer, W. J. Duncan, and A. R. Collar. Elementary Matrices and Some Applications to Dynamics and Differential Equations. Cambridge University Press, Cambridge, UK, xviii+416 pp printing. P. Glasserman and S. Suchintabandid. Correlation expansions for CDO pricing. Journal of Banking & Finance, 31: , C.-H. Guo and N. J. Higham. A Schur Newton method for the matrix pth root and its inverse. SIAM J. Matrix Anal. Appl., 28(3): , University of Manchester Nick Higham Matrix functions & correlation matrices 5 / 10

54 References VI N. J. Higham. The Matrix Function Toolbox. http: // N. J. Higham. Accuracy and Stability of Numerical Algorithms. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, second edition, ISBN xxx+680 pp. University of Manchester Nick Higham Matrix functions & correlation matrices 6 / 10

55 References VII N. J. Higham. The scaling and squaring method for the matrix exponential revisited. SIAM J. Matrix Anal. Appl., 26(4): , N. J. Higham. The scaling and squaring method for the matrix exponential revisited. SIAM Rev., 51(4): , N. J. Higham and L. Lin. On pth roots of stochastic matrices. Linear Algebra Appl., 435(3): , University of Manchester Nick Higham Matrix functions & correlation matrices 7 / 10

56 References VIII N. J. Higham and L. Lin. An improved Schur Padé algorithm for fractional powers of a matrix and their Fréchet derivatives. MIMS EPrint , Manchester Institute for Mathematical Sciences, The University of Manchester, UK, Jan pp. J. D. Lawson. Generalized Runge-Kutta processes for stable systems with large Lipschitz constants. SIAM J. Numer. Anal., 4(3): , Sept University of Manchester Nick Higham Matrix functions & correlation matrices 8 / 10

57 References IX L. Lin. Roots of Stochastic Matrices and Fractional Matrix Powers. PhD thesis, The University of Manchester, Manchester, UK, pp. MIMS EPrint , Manchester Institute for Mathematical Sciences. University of Manchester Nick Higham Matrix functions & correlation matrices 9 / 10

58 References X K. H. Parshall. James Joseph Sylvester. Jewish Mathematician in a Victorian World. Johns Hopkins University Press, Baltimore, MD, USA, ISBN xiii+461 pp. University of Manchester Nick Higham Matrix functions & correlation matrices 10 / 10

Rehabilitating Research Correlations, Matters Avoiding Inversion, and Extracting Roots

Rehabilitating Research Correlations, Matters Avoiding Inversion, and Extracting Roots February 25, 2009 Nick Nick Higham Higham Director School of of Research Mathematics The University of Manchester