Numerical methods for eigenvalue problems

Transcription

1 Numerical methods for eigenvalue problems D. Löchel Supervisors: M. Hochbruck und M. Tokar Mathematisches Institut Heinrich-Heine-Universität Düsseldorf GRK 1203 seminar february 2008

2 Outline Introduction Power Method Inverse Power Method Inverse Power Method with Shift Rayleigh Quotient Iteration Simultaneous Iteration QR Algorithm QR Algorithm with Shift Summary

3 Introduction Linear Algebra: Definition A C n n, eigenvector x C n \ {0}, eigenvalue λ C A x = λ x Theorem λ eigenvalue of A P A (λ) := det(λi A) = 0 P A characteristic polynomial deg(p A ) 5: no explicit formula numerical algorithm required using P A (λ) numerical instable

4 Introduction Definition Rayleigh quotient: ρ A ( x) = x H A x x H x Theorem x eigenvector of A = ρ A ( x) is eigenvalue of A Definition numbering of eigenvalues: λ 1 λ 2... λ n x j eigenvector for λ j

5 Power Method Given: A C n n, y 0 C n \ {0}, y 0 2 = 1 Assume: y 0 H x 1 0 Idea: n y 0 = β j x j = A k y 0 = = A y 0 = j=1 n β j A x j = j=1 n β j λ j x j j=1 ( n β j λ k j x j = β 1 λ k 1 x 1 + O j=1 λ 2 ) k λ 1

6 Power Method Initial guess y 0 2 = 1. for k = 0, 1,... do z k+1 = A y k // power ρ k = y k H z k+1 // Rayleigh quotient y k HA y k y k H y k y k+1 = z k+1 // avoid over and underflow z k+1 2 end for Convergence if η := λ 2 λ 1 < 1 and x 1 H y 0 0. Approximation ρ( y k ) = λ 1 + O(η k ), slow if η 1. Finds λ 1 only.

7 Power Method

8 Interior Eigenvalues Recall: λ 0 eigenvalue of A 1 λ eigenvalue of A 1. Idea: apply power method to A 1 inverse power method. Definition LU decomposition: A = LU, L lower, U upper triangular matrix A z = y L(U z) = y two triangular linear systems L t = y, U z = t

9 Inverse Power Method Initial guess y 0 2 = 1. Calculate LU decomposition of A. for k = 0, 1,... do Solve A z k+1 = y k with LU decomposition. // z k+1 = A 1 y k y k+1 = z k+1 // avoid over and underflow z k+1 2 ρ k = ρ A ( y k+1 ) = y k+1 H A y k+1 // Rayleigh quotient end for Finds eigenvalue closest to zero. Convergence if η := λn λ n 1 < 1 and x n H y 0 0. Approximation ρ( y k ) = λ n + O(η k ), slow if η 1.

10 Inverse Power Method

11 Shifts What about the remaining eigenvalues? Smallest eigenvalue of A µi is eigenvalue from A closest to µ. Definition µ is called Shift and A µi shifted matrix.

12 Inverse Power Method with Shift Initial guess y 0 2 = 1. µ 0 initial guess for desired eigenvalue. Calculate LU decomposition of (A µ 0 I). for k = 0, 1,... do Solve (A µ 0 I) z k+1 = y k with LU decomposition. y k+1 = z k+1 // avoid over and underflow z k+1 2 ρ k = ρ A ( y k+1 ) = y k+1 H A y k+1 // Rayleigh quotient end for Finds eigenvalue λ j closest to µ 0. λ Convergence if η := max j µ 0 m j λ m µ 0 < 1 and x j H y 0 0. Approximation ρ( y k ) = λ j + O(η k ), slow if η 1.

13 µ 0 = 2.6 µ 0 = 2.6 Inverse Power Method with Shift

14 Rayleigh Quotient Iteration Initial guess y 0 2 = 1. µ 0 initial guess for desired eigenvalue. for k = 0, 1,... do Calculate LU decomposition of (A µ k I). Solve (A µ k I) z k+1 = y k with LU decomposition. y k+1 = z k+1 // avoid over and underflow z k+1 2 ρ k = ρ A ( y k+1 ) = y k+1 H A y k+1 // Rayleigh quotient µ k := ρ k end for Finds eigenvalue λ j closest to µ 0. λ Convergence if η 0 := max j µ 0 m j λ m µ 0 < 1 and x j H y 0 0. η k := max m j λ j µ k λ m µ k 1, if η0 < 1: lim k η k = 0. Approximation ρ( y k ) = λ j + O(η 2k 0 ). LU decompostion in each step expensive Convergence fast

15 Rayleigh Quotient Iteration µ 0 = 2.6 µ 0 = 2.6

16 Simultaneous Iteration Until now: calculating only one eigenvalue Next step: calculating many or all eigenvalues m n, U 0 C n,m unitary, U H 0 U 0 = I m, A 0 := A for k = 0, 1,... do Y k+1 := A k U k // power of each column of U k Calculate QR decomposition Y k+1 = U k+1 R k+1. A k+1 = U H k+1 AU k+1 // similarity transformation end for Orthogonalisation is necessary to avoid that all columns converge against x 1.

17 Simultaneous Iteration

18 QR Algorithm m = n: QR algorithm (equivalent to simultaneous iteration) A 0 := A for k = 0, 1,... do Calculate QR Decomposition A k = Q k R k. A k+1 := R k Q k end for A k+1 = R k Q k = Q H k A kq k = (Q k 1 Q k ) H A k 1 (Q k 1 Q k ) = Q H k A 0 Q k, unitary similarity transformation. A k upper triangular matrix, eigenvalues on diagonal (proof: schur decomposition). This way QR algorithm can be very slow.

19 QR Algorithm

20 QR Algorithm with Shift A 0 := A for k = 0, 1,... do Choose a shift µ k. Calculate QR decomposition A k µ k I = Q k R k. A k+1 := R k Q k +µ k I end for this way convergence is fast in general

21 QR Algorithm with Shift

22 QR Implementation complexity of one QR decomposition: 2 3 n3 operations complexity to calculate all eigenvalues: at least O(n 4 ) reduce complexity for QR decomposition by A H := U H AU with H upper Hessenberg. h 1,1... h 1,n 1 h 1,n U H. AU = H = h.. 2, hn 1,n 1 h n 1,n 0 h n,n 1 h n,n U product of Givens-Rotations [ ] cos(α) sin(α) G = sin(α) cos(α)

23 Summary power method for biggest eigenvalue inverse power method for eigenvalue closest to zero. inverse power method with shift for desired eigenvalue Rayleigh quotient iteration, fast simultaneous iteration for many eigenvalues QR algorithm (with shift) for all eigenvalues (fast)

24

25 Thank you for your attention!