s Iterative s for symmetric eigenvalue problems, PhD McMaster University School of Computational Engineering and Science February 11, 2008
s 1 The power and its variants Inverse power Rayleigh quotient iteration Subspace iteration 2 s Rayleigh-Ritz Lanczos Implicitly restarted Lanczos Band Lanczos 3
Inverse power Rayleigh quotient iteration Subspace iteration s Find the dominating eigenvalue/eigenvector v k+1 = y k / y k 2 y k+1 = Av k+1 λ k+1 = v T k+1 y k+1 Only multiplication is involved Converges unless v 0 v max Convergence rate: λ max /λ max 1 Problems multiple/close largest eigenvalues only the largest eigenvalue is computed
Inverse power Rayleigh quotient iteration Subspace iteration s Inverse power Inner eigenvalues: λ ( (A σi) 1) = 1 λ(a) σ Apply the power to (A σi) 1 v k+1 = y k / y k 2 y k+1 = (A σi) 1 v k+1 λ k+1 = v T k+1 y k+1 Converges to the dominating eigenvalue of (A σi) 1 Converges unless v 0 v max Convergence rate: (λ max σ)/(λ max 1 σ), linear Viable only if (A σi)y = v is easily solvable
Inverse power Rayleigh quotient iteration Subspace iteration s Rayleigh quotient iteration Change the shift in each iteration v k+1 = y k / y k 2 σ k+1 = v T k+1 Av k+1/ v k+1 2 y k+1 = (A σ k+1 I) 1 v k+1 λ k+1 = v T k+1 y k+1 Convergence properties are unclear Finds an eigenvalue faster than inverse iteration Cubic convergence Does not necessarily find λ max May not converge to an eigenvalue A σ k+1 I will become singular New factorization in every iteration
Inverse power Rayleigh quotient iteration Subspace iteration s Subspace iteration Invariant subspaces are robust, eigenvectors are not 0 1 ε 2 0 0 0 ε 1 0 1 0 2 0 0 0 0 1 + ε It is better to identify the invariant subspaces QR factorize Y k = V k+1 R k+1 Y k+1 = AV k+1 H k+1 = V T k+1y k+1 Y, V R n p, H R p p The eigenvalues of H are the largest eigenvalues of A Clustered (not multiple) eigenvalues Choosing p smartly Can also be applied to (A σi) 1 Software: EA12 in HSL
s Rayleigh-Ritz Lanczos Implicitly restarted Lanczos Band Lanczos s Problems with power iteration based s Extremal eigenvalues only Internal eigenvalues require solution of a linear system A k v is used as the best guess for an eigenvector : span { v, Av,..., A k v } Find the best approximate eigenvector (Ritz vectors) Columns of Q k are orthogonal, span Krylov space λ(q T k AQ k) approximates λ(a) Choose a Q k to simplify the structure of Q T k AQ k }{{} T k
s Rayleigh-Ritz Lanczos Implicitly restarted Lanczos Band Lanczos Lanczos Gradually build the Maintain an orthogonal basis Q k, T k tridiagonal Find the corresponding Ritz vectors v 0 = 0, β 1 = 0, v 1 random unit repeat q j = Av j β j v j 1 α j = qj T v j q j = q j α j v j β j+1 = q j v j+1 = q j /β j+1 Extreme eigenvalues converge first Can also be applied to (A σi) 1 Memory consumption increases
s Rayleigh-Ritz Lanczos Implicitly restarted Lanczos Band Lanczos Implicitly restarted Lanczos Prevents k growing too much Applies shifts µ i to the algorithm Equivalently, changes v 0 How to choose the shifts? Flexible eigenvalue configurations Locking/purging eigenvalues Software: ARPACK (also in Matlab)
s Rayleigh-Ritz Lanczos Implicitly restarted Lanczos Band Lanczos Band Lanczos Multiple starting vectors, finds more eigenvalues span { V, AV,..., A k V } Suitable for multiple/clustered eigenvalues T is block tridiagonal
s Problems with Lanczos only efficient if the eigenvalues are well separated needs (A σi) 1 y for internal eigenvalues Build a different set of orthogonal vectors spanning the Galerkin vectors Interior eigenvalues without inversion Very good if A has multiple eigenvalues Software: JDQR (Matlab)
s Z. Bai, J. Demmel, J. Dongarra, A. Ruhe, and H. van der Vorst, editors. Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide. SIAM, Philadelphia, 2000. James W. Demmel. Applied Numerical Linear Algebra. SIAM, Philadelphia, 1997.