Introduction to Arnoldi method

Transcription

1 Introduction to Arnoldi method SF Matrix Computations for Large-scale Systems KTH Royal Institute of Technology (Elias Jarlebring) KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

2 Agenda lecture 2 Introduction to Arnoldi method Gram-Schmidt - efficiency and roundoff errors Derivation of Arnoldi method (Convergence characterization) KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

3 Main eigenvalue algorithms in this course Fundamental eigenvalue techniques (Lecture 1) Arnoldi method (Lecture 2-3). QR-method (Lecture 8-9). KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

4 Main eigenvalue algorithms in this course Fundamental eigenvalue techniques (Lecture 1) Arnoldi method (Lecture 2-3). Typically suitable when we are interested in a small number of eigenvalues, the matrix is large and sparse Currently solvable size on desktop m 10 6 (depending on structure) QR-method (Lecture 8-9). Typically suitable when we want to compute all eigenvalues, the matrix does not have any particular easy structure. Solvable size on desktop m KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

5 Finite-element method example (slide 1/2) Laplace eigenvalue problem u = λu with Dirichlet boundary conditions outside wing profile KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

6 Finite-element method example (slide 2/2) Problem: Compute eigenvalues of matrix A R m m where m large but few non-zero elements: nz = 3550 KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

7 Finite-element method example (slide 2/2) Problem: Compute eigenvalues of matrix A R m m where m large but few non-zero elements: nz = 3550 In MATLAB with very efficient QR-method: tic; eig(full(a)); toc KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

8 Finite-element method example (slide 2/2) Problem: Compute eigenvalues of matrix A R m m where m large but few non-zero elements: nz = 3550 In MATLAB with very efficient QR-method: tic; eig(full(a)); toc More than 10 hours we need a different method. KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

9 Idea of Arnoldi method (slide 1/3) Eigenvalue problem Ax = λx. TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

10 Idea of Arnoldi method (slide 1/3) Eigenvalue problem Ax = λx. Suppose Q = (q 1,..., q n ) R m n and x span(q 1,..., q n ) TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

11 Idea of Arnoldi method (slide 1/3) Eigenvalue problem Ax = λx. Suppose Q = (q 1,..., q n ) R m n and x span(q 1,..., q n ) There exists z R n such that AQz = λqz TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

12 Idea of Arnoldi method (slide 1/3) Eigenvalue problem Ax = λx. Suppose Q = (q 1,..., q n ) R m n and x span(q 1,..., q n ) There exists z R n such that AQz = λqz and Q T AQz = λq T Qz KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

13 Idea of Arnoldi method (slide 1/3) Eigenvalue problem Ax = λx. Suppose Q = (q 1,..., q n ) R m n and x span(q 1,..., q n ) There exists z R n such that AQz = λqz and Q orthogonal Q T AQz = λq T Qz = λz KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

14 Idea of Arnoldi method (slide 2/3) Rayleigh-Ritz procedure TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

15 Idea of Arnoldi method (slide 2/3) Rayleigh-Ritz procedure Let Q R m n be orthogonal basis of some subspace. TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

16 Idea of Arnoldi method (slide 2/3) Rayleigh-Ritz procedure Let Q R m n be orthogonal basis of some subspace. Solutions (µ, z) to the eigenvalue problem Q T AQz = µz are called Ritz pairs. TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

17 Idea of Arnoldi method (slide 2/3) Rayleigh-Ritz procedure Let Q R m n be orthogonal basis of some subspace. Solutions (µ, z) to the eigenvalue problem Q T AQz = µz are called Ritz pairs. The procedure (Rayleigh-Ritz) returns an exact eigenvalue if x span q 1,..., q n, TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

18 Idea of Arnoldi method (slide 2/3) Rayleigh-Ritz procedure Let Q R m n be orthogonal basis of some subspace. Solutions (µ, z) to the eigenvalue problem Q T AQz = µz are called Ritz pairs. The procedure (Rayleigh-Ritz) returns an exact eigenvalue if x span q 1,..., q n, and approximates eigenvalues of A well if x x span(q 1,..., q n ). TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

19 Idea of Arnoldi method (slide 2/3) Rayleigh-Ritz procedure Let Q R m n be orthogonal basis of some subspace. Solutions (µ, z) to the eigenvalue problem Q T AQz = µz are called Ritz pairs. The procedure (Rayleigh-Ritz) returns an exact eigenvalue if x span q 1,..., q n, and approximates eigenvalues of A well if What is a good subspace span(q)? x x span(q 1,..., q n ). TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

20 Idea of Arnoldi method (slide 2/3) Rayleigh-Ritz procedure Let Q R m n be orthogonal basis of some subspace. Solutions (µ, z) to the eigenvalue problem Q T AQz = µz are called Ritz pairs. The procedure (Rayleigh-Ritz) returns an exact eigenvalue if x span q 1,..., q n, and approximates eigenvalues of A well if What is a good subspace span(q)? x x span(q 1,..., q n ). TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

21 Idea of Arnoldi method (slide 3/3) Recall: Power method approximates the largest eigenvalue well. KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

22 Idea of Arnoldi method (slide 3/3) Recall: Power method approximates the largest eigenvalue well. Definition: Krylov matrix / subspace K n (A, q 1 ) := (q 1, Aq 1,... A n 1 q 1 ) K n (A, q 1 ) := span(q 1, Aq 1,... A n 1 q 1 ). KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

23 Idea of Arnoldi method (slide 3/3) Recall: Power method approximates the largest eigenvalue well. Definition: Krylov matrix / subspace K n (A, q 1 ) := (q 1, Aq 1,... A n 1 q 1 ) K n (A, q 1 ) := span(q 1, Aq 1,... A n 1 q 1 ). Justification of Arnoldi method Use Rayleigh-Ritz on Q = (q 1,..., q n ) where span(q 1,..., q n ) = K n (A, q 1 ) KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

24 Idea of Arnoldi method (slide 3/3) Recall: Power method approximates the largest eigenvalue well. Definition: Krylov matrix / subspace K n (A, q 1 ) := (q 1, Aq 1,... A n 1 q 1 ) K n (A, q 1 ) := span(q 1, Aq 1,... A n 1 q 1 ). Justification of Arnoldi method Use Rayleigh-Ritz on Q = (q 1,..., q n ) where span(q 1,..., q n ) = K n (A, q 1 ) Arnoldi method is a clever procedure to construct H n = V T AV. KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

25 Idea of Arnoldi method (slide 3/3) Recall: Power method approximates the largest eigenvalue well. Definition: Krylov matrix / subspace K n (A, q 1 ) := (q 1, Aq 1,... A n 1 q 1 ) K n (A, q 1 ) := span(q 1, Aq 1,... A n 1 q 1 ). Justification of Arnoldi method Use Rayleigh-Ritz on Q = (q 1,..., q n ) where span(q 1,..., q n ) = K n (A, q 1 ) Arnoldi method is a clever procedure to construct H n = V T AV. Clever : (1) We exploit sparsity (we will only need matrix-vector product) KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

26 Idea of Arnoldi method (slide 3/3) Recall: Power method approximates the largest eigenvalue well. Definition: Krylov matrix / subspace K n (A, q 1 ) := (q 1, Aq 1,... A n 1 q 1 ) K n (A, q 1 ) := span(q 1, Aq 1,... A n 1 q 1 ). Justification of Arnoldi method Use Rayleigh-Ritz on Q = (q 1,..., q n ) where span(q 1,..., q n ) = K n (A, q 1 ) Arnoldi method is a clever procedure to construct H n = V T AV. Clever : (1) We exploit sparsity (we will only need matrix-vector product) (2) We expand V with one row in each iteration Iterate until we are happy. KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

27 Idea of Arnoldi method (slide 3/3) Recall: Power method approximates the largest eigenvalue well. Definition: Krylov matrix / subspace K n (A, q 1 ) := (q 1, Aq 1,... A n 1 q 1 ) K n (A, q 1 ) := span(q 1, Aq 1,... A n 1 q 1 ). Justification of Arnoldi method Use Rayleigh-Ritz on Q = (q 1,..., q n ) where span(q 1,..., q n ) = K n (A, q 1 ) Arnoldi method is a clever procedure to construct H n = V T AV. Clever : (1) We exploit sparsity (we will only need matrix-vector product) (2) We expand V with one row in each iteration Iterate until we are happy. KTH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

28 Arnoldi method graphically Graphical illustration of algorithm: Q = q 1 q 2 q 3 q 4 = H

29 Arnoldi method graphically Graphical illustration of algorithm: A Q = q 1 q 2 q 3 q = H 4 w

30 Arnoldi method graphically Graphical illustration of algorithm: A Orthogonalize Q = q 1 q 2 q 3 q = H 4 w w

31 Arnoldi method graphically Graphical illustration of algorithm: A Orthogonalize Q = q 1 q 2 q 3 q = H 4 w w TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

32 Arnoldi method graphically Graphical illustration of algorithm: A Orthogonalize Q = q 1 q 2 q 3 q = H 4 w w TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

33 Arnoldi method graphically Graphical illustration of algorithm: A Orthogonalize Q = q 1 q 2 q 3 q = H 4 w w TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

34 Arnoldi method graphically Graphical illustration of algorithm: A Orthogonalize Q = q 1 q 2 q 3 q = H 4 w w After iteration: Take eigenvalues of H as approximate eigenvalues. TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

35 Arnoldi method graphically Graphical illustration of algorithm: A Orthogonalize Q = q 1 q 2 q 3 q = H 4 w w After iteration: Take eigenvalues of H as approximate eigenvalues. * show arnoldi.m and Hessenberg matrix in matlab * TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9

36 Arnoldi method graphically Graphical illustration of algorithm: A Orthogonalize Q = q 1 q 2 q 3 q = H 4 w w After iteration: Take eigenvalues of H as approximate eigenvalues. * show arnoldi.m and Hessenberg matrix in matlab * We will now... (1) derive a good orthogonalization procedure: variants of Gram-Schmidt, (2) show that Arnoldi generates a Rayleigh-Ritz approximation, (3) characterize the convergence. TH Royal Institute of Technology (Elias Jarlebring)Introduction to Arnoldi method / 9