An Algorithm for the Research Complete Matters Solution of Quadratic February Eigenvalue 25, 2009 Problems Nick Higham Françoise Tisseur Director of Research School of Mathematics The School University of Mathematics of Manchester ftisseur@ma.man.ac.uk http://www.ma.man.ac.uk/~ftisseur/ Joint work with Sven Hammarling and Christopher Munro. 1 / 6
Quadratic Eigenvalue Problem (QEP) Consider Q(λ) = λ 2 M + λd + K C[λ] n n. Assume Q(λ) regular (det(q(λ)) 0). QEP: find nonzero vectors x, y and scalars λ C s.t. Q(λ)x = 0, y Q(λ) = 0. Q(λ) has 2n e vals. Finite e vals are roots of det(q(λ)) = 0. E val at 0 when K is singular and e val at when M is singular. MIMS Françoise Tisseur Quadratic eigenproblem 2 / 28
Example 1 Q(λ) = λ 2 0 8 0 0 6 0 + λ 1 6 0 2 7 0 + 1 0 0 0 1 0. 0 0 1 0 0 0 0 0 1 Regular: det Q(λ) = 6λ 5 + 11λ 4 12λ 3 + 12λ 2 6λ + 1 0. Six eigenpairs (λ k, x k ), k = 1: 6, given by k 1 2 3 4 5 6 λ k 1/3 1/2 1 i i x k 1 1 0 1 1 0 0 1 0 0 0 1 0 0 1 1 0 0 MIMS Françoise Tisseur Quadratic eigenproblem 3 / 28
NLEVP Toolbox Collection of Nonlinear Eigenvalue Problems : T. Betcke, N. J. Higham, V. Mehrmann, C. Schröder, F. T., 2011. Quadratic, polynomial, rational and other nonlinear eigenproblems. Provided in the form of a MATLAB Toolbox. Compatible with GNU Octave. Problems from real-life applications + specifically constructed problems. http://www.mims.manchester.ac.uk/research/ numerical-analysis/nlevp.html MIMS Françoise Tisseur Quadratic eigenproblem 4 / 28
Sample of Quadratic Problems n m quadratic Q(λ) = λ 2 M + λd + K. Speaker box (pep,qep,real,symmetric). n = m = 107. Finite element model of a speaker box. M 2 = 1, D 2 = 5.7 10 2, K 2 = 1.0 10 7. Railtrack (pep,qep,t-palindromic,sparse). n = m = 1005. Model of vibration of rail tracks under the excitation of high speed trains. M = K T, D = D T. Surveillance (pep,qep,real,nonsquare, nonregular). n = 21, m = 16. From calibration of surveillance camera using human body as calibration target. MIMS Françoise Tisseur Quadratic eigenproblem 5 / 28
Standard Solution Process Find all λ and x satisfying Q(λ)x = (λ 2 M + λd + K )x = 0. Commonly solved by linearization: Convert Q(λ)x = 0 into (A λb)ξ = 0, e.g., [ ] [ ] K 0 D M A λb = λ, ξ = 0 I I 0 [ x λx Solve (A λb)ξ = 0 with an eigensolver for generalized eigenproblem (e.g., QZ algorithm). Recover eigenvectors of Q(λ) from those of A λb. Eigensolver often absent from numerical libraries. MIMS Françoise Tisseur Quadratic eigenproblem 6 / 28 ].
Beam Problem /////// L /////// //////////// Transverse displacement u(x, t) governed by ρa 2 u t 2 + c(x) u t + EI 4 u x 4 = 0. u(0, t) = u (0, t) = u(l, t) = u (L, t) = 0. Separation of variables u(x, t) = e λt v(x, λ) yields the eigenvalue problem for the free vibrations: λ 2 ρav(x, λ) + λc(x)v(x, λ) + EI 4 v(x, λ) = 0. x4 MIMS Françoise Tisseur Quadratic eigenproblem 7 / 28
Discretized Beam Problem Finite element method leads to Q(λ) = λ 2 M + λd + K with symmetric M, D, K R n n. M > 0, K > 0, D 0. Roots of x Q(λ)x = 0, x C n \ {0}, λ = (x Dx) ± (x Dx) 2 4(x Mx)(x Kx). 2(x Mx) M > 0, K > 0, D 0 all e vals have Re(λ) 0. D is rank 1. Can show n pure imaginary e vals. MIMS Françoise Tisseur Quadratic eigenproblem 8 / 28
Eigenvalues of Q(λ) = λ 2 M + λd + K When M, K are nonsingular then theoretically [ ] [ ] M 0 D K C 1 (λ) = λ +, 0 I I 0 [ ] [ ] [ ] [ ] M 0 D K 0 M M 0 L 1 (λ) = λ +, L 0 K K 0 2 (λ) = λ + M D 0 K have the same eigenvalues as Q(λ). MIMS Françoise Tisseur Quadratic eigenproblem 9 / 28
Eigenvalues of Q(λ) = λ 2 M + λd + K When M, K are nonsingular then theoretically [ ] [ ] M 0 D K C 1 (λ) = λ +, 0 I I 0 [ D K [ ] M 0 L 1 (λ) = λ + 0 K K 0 ], L 2 (λ) = λ have the same eigenvalues as Q(λ). [ 0 M M D ] [ ] M 0 + 0 K coeffs = nlevp( damped beam,100); K = coeffs{1}; D = coeffs{2}; M = coeffs{3}; I = eye(2*nele); O = zeros(2*nele); eval = eig([d K; -I O],-[M O; O I]; % C 1 %eval = eig([d K; K O],-[M O; O -K]; % L 1 %eval = eig([-m O; O K],-[O M; M D]; % L 2 plot(eval,.r ); MIMS Françoise Tisseur Quadratic eigenproblem 9 / 28
Computed Spectra of C 1, L 1 and L 2 1 2 3 4 x 106 3 2 1 0 4 15 10 5 0 1 2 3 4 x 106 3 2 1 0 4 15 10 5 0 1 2 3 4 x 106 3 2 1 0 4 15 10 5 0 MIMS Françoise Tisseur Quadratic eigenproblem 10 / 28
Sensitivity and Stability of Linearizations Condition number measures sensitivity of the solution of a problem to perturbations in the data. Backward error measures how well the problem has been solved. error in solution < condition number backward error. MIMS Françoise Tisseur Quadratic eigenproblem 11 / 28
Sensitivity and Stability of Linearizations Condition number measures sensitivity of the solution of a problem to perturbations in the data. Backward error measures how well the problem has been solved. error in solution < condition number backward error. For a given Q(λ), infinitely many linearizations exist: can have widely varying eigenvalue condition numbers, computed eigenpairs can have widely varying backward errors. MIMS Françoise Tisseur Quadratic eigenproblem 11 / 28
Objectives To design a general purpose eigensolver for dense QEPs quadeig. Incorporate: Appropriate choice of linearization. Deflation of 0 and eigenvalues. Eigenvalue parameter scaling. Advantageous use of block structure. Careful recovery of the eigenvectors. A MATLAB and Fortran implementation. MIMS Françoise Tisseur Quadratic eigenproblem 12 / 28
Linearization for Q(λ) = λ 2 M + λd + K Opt for C(λ) = [ D I K 0 Simple block structure. ] [ ] M 0 (companion form, + λ 0 I Fiedler pencil) Always a linearization, i.e., there exist E(λ) and F(λ) with constant, nonzero determinants s.t. [ ] Q(λ) 0 = E(λ)C(λ)F(λ). 0 I Left/right e vecs of Q(λ) are easily recovered from those of companion linearizations. Deflation of 0 and e vals easy to implement. "Good" backward error and conditioning properties. MIMS Françoise Tisseur Quadratic eigenproblem 13 / 28
Eigenvalue Condition Numbers κ Q (λ) Q(λ)x = 0, y Q(λ) = 0, Q(λ) = λ 2 M + λ D + K. For λ simple, nonzero and finite, { λ κ Q (λ) = lim sup ɛ 0 ɛ λ : [ (Q + Q)(λ + λ) ] (x + x) = 0, M 2 ɛ M 2, D 2 ɛ D 2, K 2 ɛ K 2 }. MIMS Françoise Tisseur Quadratic eigenproblem 14 / 28
Eigenvalue Condition Numbers κ Q (λ) Q(λ)x = 0, y Q(λ) = 0, Q(λ) = λ 2 M + λ D + K. For λ simple, nonzero and finite, { λ κ Q (λ) = lim sup ɛ 0 ɛ λ : [ (Q + Q)(λ + λ) ] (x + x) = 0, M 2 ɛ M 2, D 2 ɛ D 2, K 2 ɛ K 2 }. Can show that [T. 01] κ Q (λ) = ( ) λ 2 M 2 + λ D 2 + K 2 y 2 x 2. λ y (2λM + D)x When λ = 0 or, use homogeneous form of Q. MIMS Françoise Tisseur Quadratic eigenproblem 14 / 28
Eigenvalue Condition Numbers κ Q (α, β) Homogeneous form: Q(α, β) = α 2 M + αβd + β 2 K. E vals are pairs (α, β) (0, 0) s.t. det Q(α, β) = 0, λ = α/β. MIMS Françoise Tisseur Quadratic eigenproblem 15 / 28
Eigenvalue Condition Numbers κ Q (α, β) Homogeneous form: Q(α, β) = α 2 M + αβd + β 2 K. E vals are pairs (α, β) (0, 0) s.t. det Q(α, β) = 0, λ = α/β. For (α, β) simple with right/left e vecs x, y, [Dedieu, T., 03] κ Q (α, β) = ( α 4 M 2 2 + α 2 β 2 D 2 2 + ) β 4 K 2 1/2 y 2 2 x 2 y ( β D α Q ᾱd β Q) (α,β) x. Angle between original and perturbed eigenvalues satisfies θ ( (α, β), ( α, β) ) κq (α, β) Q + o( Q ). Here Q = ( M, D, K ). Similar expression for κ C (α, β), C(λ) = [ D K ] [ I 0 + λ M ] 0 0 I. MIMS Françoise Tisseur Quadratic eigenproblem 15 / 28
Eigenvalue Condition Numbers κ Q (α, β) Homogeneous form: Q(α, β) = α 2 M + αβd + β 2 K. E vals are pairs (α, β) (0, 0) s.t. det Q(α, β) = 0, λ = α/β. For (α, β) simple with right/left e vecs x, y, [Dedieu, T., 03] κ Q (α, β) = ( α 4 M 2 2 + α 2 β 2 D 2 2 + ) β 4 K 2 1/2 y 2 2 x 2 y ( β D α Q ᾱd β Q) (α,β) x. Angle between original and perturbed eigenvalues satisfies θ ( (α, β), ( α, β) ) κq (α, β) Q + o( Q ). Here Q = ( M, D, K ). Similar expression for κ C (α, β), C(λ) = [ ] [ D I K 0 + λ M ] 0 0 I. Want κ Q (α, β) κ C (α, β) for all (α, β). MIMS Françoise Tisseur Quadratic eigenproblem 15 / 28
Backward Error η Q (x, α, β) For an approximate (right) eigenpair (x, α, β) of Q(α, β), η Q (x, α, β) = min{ ɛ : (Q(α, β) + Q(α, β))x = 0, M 2 ɛ M 2, D 2 ɛ D 2, K 2 ɛ K 2 }, where Q(α, β) = α 2 M + αβ D + β 2 K. Can show that [T.01] η Q (x, α, β) = Q(α, β)x 2 ( α 2 M 2 + α β D 2 + β 2 K 2 ) x 2. Similar expression for η C (z, α, β), C(λ) = [ D K ] [ I 0 + λ M ] 0 0 I. MIMS Françoise Tisseur Quadratic eigenproblem 16 / 28
Backward Error η Q (x, α, β) For an approximate (right) eigenpair (x, α, β) of Q(α, β), η Q (x, α, β) = min{ ɛ : (Q(α, β) + Q(α, β))x = 0, M 2 ɛ M 2, D 2 ɛ D 2, K 2 ɛ K 2 }, where Q(α, β) = α 2 M + αβ D + β 2 K. Can show that [T.01] η Q (x, α, β) = Q(α, β)x 2 ( α 2 M 2 + α β D 2 + β 2 K 2 ) x 2. Similar expression for η C (z, α, β), C(λ) = [ D K Want η Q (z 1, α, β) η C (z, α, β), z = [ z 1 z 2 ]. ] [ I 0 + λ M ] 0 0 I. MIMS Françoise Tisseur Quadratic eigenproblem 16 / 28
Eigenvalue Parameter Scaling Let λ = µγ, γ 0 and convert Q(λ) = λ 2 M + λd + K to δq(µγ) = µ 2 (γ 2 δm) + µ(γδd) + δk = µ 2 M + µ D + K =: Q(µ). Choose γ such that the linearization process does not affect the eigenvalue condition numbers, i.e., κ Q (α, β) κ C (α, β) for all e vals (α, β). the standard solution process in numerically stable, i.e., η Q (z[ 1, α, ] β) η C (z, α, β) for all e vals (α, β), where z1 z =. z 2 MIMS Françoise Tisseur Quadratic eigenproblem 17 / 28
Try γ = exp(r), where r is a tropical root of a tropical scalar quadratic (proposed by Gaubert & Sharify, 09). MIMS Françoise Tisseur Quadratic eigenproblem 17 / 28 Eigenvalue Parameter Scaling Let λ = µγ, γ 0 and convert Q(λ) = λ 2 M + λd + K to δq(µγ) = µ 2 (γ 2 δm) + µ(γδd) + δk = µ 2 M + µ D + K =: Q(µ). Choose γ such that the linearization process does not affect the eigenvalue condition numbers, i.e., κ Q (α, β) κ C (α, β) for all e vals (α, β). the standard solution process in numerically stable, i.e., η Q (z[ 1, α, ] β) η C (z, α, β) for all e vals (α, β), where z1 z =. z 2
Tropical Scalar Polynomials Let (R { },, ) be the tropical semiring with a b = max(a, b), a b = a+b for all a, b R { }. The piecewise affine function p(x) = d k=0 p k x k = max 0 k d (p k + kx), p k R { } is a tropical polynomial of degree d. The tropical roots of p(x) are the points of nondifferentiability of p(x). MIMS Françoise Tisseur Quadratic eigenproblem 18 / 28
Computation of Tropical Roots Let p(x) = d k=0 p k x k = max(p 0, p 1 + x,..., p d + dx). Upper boundary of convex hull of (k, p k ), k = 0: d. Tropical roots are minus the slopes of the segments (Legendre-Fenchel duality). Horizontal width of segment gives multiplicity. p(x) = max(2, 1+x, 3+2x, 3+3x, 1+4x, 1 2 +5x, 2+6x, 1+7x) has roots 1 2, 1 2, 0, 1 3, 1 3, 1 3, 3. p k 3 2 1 0 1 2 3 4 5 6 7 k MIMS Françoise Tisseur Quadratic eigenproblem 19 / 28
Tropical Roots (cont.) Tropical roots can be computed in linear time. Classical roots of p(x) = a 0 + a 1 x + + a n x n can be bounded in terms of tropical roots of p trop (x) = d k=0 p k x k ( Sharify, 11). Let r 1, r 2 be the tropical roots of p trop (r) = max(log( K ), log( D ) + r, log( M ) + 2r). When r 1 r 2 and M, D, K are well conditioned, e r 1 λmax (Q) and e r 2 λmin (Q), where Q(λ) = λ 2 M + λd + K (Gaubert & Sharify, 09). MIMS Françoise Tisseur Quadratic eigenproblem 20 / 28
Tropical Scaling Let p trop (r) = max(log( K ), log( D ) + r, log( M ) + 2r). Convert Q(λ) to δq(µγ), where γ = e r, r is a tropical root of p trop, δ = e ptrop(r). If D 2 2 M 2 K 2 then r 1 = r 2 (one double root). Can show that, for all e vals. η Q η C κ Q κ C Otherwise, two distinct tropical roots r 1 > r 2. Can show { r = r1 and λ γ, that, if η Q η C κ Q κ C r = r 2 and λ γ. MIMS Françoise Tisseur Quadratic eigenproblem 21 / 28
Beam Pb: Λ(C), Λ(L 2 ) before/after Scaling 3 2 1 0 1 2 3 4 x 106 4 15 10 5 0 1 2 3 4 x 106 3 2 1 0 3 2 1 0 1 2 3 4 x 106 4 15 10 5 0 3 2 1 0 1 2 3 4 x 106 4 4 15 10 5 0 15 10 5 0 MIMS Françoise Tisseur Quadratic eigenproblem 22 / 28
Modified Hospital Problem Damping added s.t. D 2 2 M 2 K 2. γ FLV = ( K 2 M 2 ) 1/2 56. Two tropical roots: γ + 3.7 10 3 and γ 0.8. Backward error 10 12 10 14 10 16 no scaling FLV 10 1 10 0 10 1 10 2 10 3 λ Backward error 10 12 10 14 10 16 tropical γ tropical γ + 10 1 10 0 10 1 10 2 10 3 λ MIMS Françoise Tisseur Quadratic eigenproblem 23 / 28
Deflation of 0 and eigenvalues Transform C(λ) = [ γδd δk S 11 S 12 S 13 0 S 22 S 23 0 0 0 n rk ] [ I 0 λ γ 2 δm 0 λ where r M = rank(m) and r K = rank(k ). 0 I ] into T 11 T 12 T 13 0 0 n rm T 23 0 0 I n rk 2 QR fact with col piv and 1 COD if r M < n and r K < n. Make use of block structure of C(λ). Call QZ on S 11 λt 11, with T 11 usually upper triang. Q(λ) nonregular if S 22 singular. Cost of deflation negligible compared with overall cost. MIMS Françoise Tisseur Quadratic eigenproblem 24 / 28,
Some QEPs from NLEVP Collection η Q (x, λ): backward error of computed eigenpair (x, λ). polyeig quadeig Problem n τ Q η Q (x, λ) η Q (x, λ) η Q (y, λ) power plant 8 7e-1 1e-8 4e-16 5e-17 cd player 60 9e+3 2e-10 7e-16 2e-15 speaker box 107 2e-5 2e-11 2e-16 4e-16 damped beam 200 2e-4 3e-9 1e-15 1e-15 shaft 400 1e-6 9e-8 9e-16 6e-16 railtrack 1005 2e1 2e-8 2e-15 8e-15 quadeig is backward stable for τ Q < 1, where τ Q = A 1 / A 2 A 0. MIMS Françoise Tisseur Quadratic eigenproblem 25 / 28
Some Timings polyeig quadeig Problem n Λ (Λ, X) Λ (Λ, X) acoustic wave 2D 870 118s 202s 114s 191s damped beam 1000 92s 156s 97s 163s spring 1000 182s 272s 94s 170s railtrack2 1410 129s 306s 79s 113s railtrack2: QEPs with singular M, K. MIMS Françoise Tisseur Quadratic eigenproblem 26 / 28
Extension to Higher Degree Polynomials P(λ)x = (λ l A l + + λa 1 + A 0 )x = 0. Use Fiedler linearization: F(λ) = λ A l I...... I A l 1 A l 2 A 1 I I... I Stability/conditioning similar to companion forms. Deflation of 0 and e vals easy to implement. A 0 Work in progress: scaling strategy (Betcke s scaling, tropical roots, tropical eigenvalues,... ).. MIMS Françoise Tisseur Quadratic eigenproblem 27 / 28
Summary quadeig is backward stable for not too heavily damped problems. Tropical mathematics useful for e val scaling. Deflation strategy can produce significant speedups. Returns e vals, right & left e vecs, e val condition numbers, b errs of right/left approx. eigenpairs. MATLAB function available. Fortran implementation available soon. Extension to polynomial eigenproblem of degree l > 2. For papers, eprints and codes, http://www.ma.man.ac.uk/~ftisseur/ MIMS Françoise Tisseur Quadratic eigenproblem 28 / 28
References I T. Betcke, N. J. Higham, V. Mehrmann, C. Schröder, and F. Tisseur. NLEVP: A collection of nonlinear eigenvalue problems. MIMS EPrint 2011.116, Manchester Institute for Mathematical Sciences, The University of Manchester, UK, Dec. 2011. 27 pp. To appear in ACM Trans. Math. Software. J.-P. Dedieu and F. Tisseur. Perturbation theory for polynomial eigenvalue problems in homogeneous form. Linear Algebra Appl., 358(1-3):71 94, 2003. MIMS Françoise Tisseur Quadratic eigenproblem 24 / 28
References II H.-Y. Fan, W.-W. Lin, and P. Van Dooren. Normwise scaling of second order polynomial matrices. SIAM J. Matrix Anal. Appl., 26(1):252 256, 2004. S. Gaubert and M. Sharify. Tropical scaling of polynomial matrices. volume 389 of Lecture Notes in Control and Information Sciences, pages 291 303. Springer-Verlag, Berlin, 2009. S. Hammarling, C. J. Munro, and F. Tisseur. An algorithm for the complete solution of quadratic eigenvalue problems. To appear in ACM Trans. Math. Software, 2012. MIMS Françoise Tisseur Quadratic eigenproblem 25 / 28
References III N. J. Higham, R.-C. Li, and F. Tisseur. Backward error of polynomial eigenproblems solved by linearization. SIAM J. Matrix Anal. Appl., 29(4):1218 1241, 2007. N. J. Higham, D. S. Mackey, and F. Tisseur. The conditioning of linearizations of matrix polynomials. SIAM J. Matrix Anal. Appl., 28(4):1005 1028, 2006. N. J. Higham, D. S. Mackey, F. Tisseur, and S. D. Garvey. Scaling, sensitivity and stability in the numerical solution of quadratic eigenvalue problems. Internat. J. Numer. Methods Eng., 73(3):344 360, 2008. MIMS Françoise Tisseur Quadratic eigenproblem 26 / 28
References IV M. Sharify. Scaling Algorithms and Tropical Methods in Numerical Matrix Analysis: Application to the Optimal Assignment Problem and to the Accurate Computation of Eigenvalues. PhD thesis, Ecole Polytechnique, 2011. 146 pp. F. Tisseur. Backward error and condition of polynomial eigenvalue problems. Linear Algebra Appl., 309:339 361, 2000. MIMS Françoise Tisseur Quadratic eigenproblem 27 / 28
References V F. Tisseur and K. Meerbergen. The quadratic eigenvalue problem. SIAM Rev., 43(2):235 286, 2001. MIMS Françoise Tisseur Quadratic eigenproblem 28 / 28