On the Ritz values of normal matrices

Transcription

1 On the Ritz values of normal matrices Zvonimir Bujanović Faculty of Science Department of Mathematics University of Zagreb June 13, 2011 ApplMath11 7th Conference on Applied Mathematics and Scientific Computing

2 Contents 1 Convergence of the restarted Arnoldi method 2 Interlacing for normal matrices 3 Characterization of Ritz values using a Cauchy matrix On the Ritz values of normal matrices

3 Contents 1 Convergence of the restarted Arnoldi method 2 Interlacing for normal matrices 3 Characterization of Ritz values using a Cauchy matrix On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

4 The Arnoldi method Task: determine a few eigenvalues λ 1,λ 2,... of a large, sparse matrix A C n n. The Arnoldi method: choose an initial vector v; iteratively build an m-dimensional Krylov subspace K m (v;a) = span{v,av,...,a m 1 v}; this is represented as AV m = V m H m + (β m v m+1 )em; approximate eigs of A with the Ritz values θ 1,θ 2,...,θ m (the eigenvalues of H m ). On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

5 The Arnoldi method Task: determine a few eigenvalues λ 1,λ 2,... of a large, sparse matrix A C n n. The Arnoldi method: choose an initial vector v; iteratively build an m-dimensional Krylov subspace K m (v;a) = span{v,av,...,a m 1 v}; this is represented as AV m = V m H m + (β m v m+1 )em; approximate eigs of A with the Ritz values θ 1,θ 2,...,θ m (the eigenvalues of H m ). On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

6 The restarted Arnoldi method The restarted Arnoldi method (ARPACK, Matlab eigs): 1 initialization: build K 1 (v;a),k 2 (v;a),...,k m p (v;a); 2 augmentation: build K m p+1 (v;a),k m p+2 (v;a),...,k m (v;a); 3 restart: choose ṽ = (A σ 1 I)(A σ 2 I)...(A σ p I)v; ṽ is such that K m p (ṽ;a) is already computed; set v = ṽ, go to 2. Roots σ 1,...,σ p are called shifts. Exact shifts = unwanted Ritz values from K m (v;a). Methods of restart: Sorensen (IRAM), Stewart (Krylov Schur) On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

7 The restarted Arnoldi method The restarted Arnoldi method (ARPACK, Matlab eigs): 1 initialization: build K 1 (v;a),k 2 (v;a),...,k m p (v;a); 2 augmentation: build K m p+1 (v;a),k m p+2 (v;a),...,k m (v;a); 3 restart: choose ṽ = (A σ 1 I)(A σ 2 I)...(A σ p I)v; ṽ is such that K m p (ṽ;a) is already computed; set v = ṽ, go to 2. Roots σ 1,...,σ p are called shifts. Exact shifts = unwanted Ritz values from K m (v;a). Methods of restart: Sorensen (IRAM), Stewart (Krylov Schur) On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

8 The restarted Arnoldi method The restarted Arnoldi method (ARPACK, Matlab eigs): 1 initialization: build K 1 (v;a),k 2 (v;a),...,k m p (v;a); 2 augmentation: build K m p+1 (v;a),k m p+2 (v;a),...,k m (v;a); 3 restart: choose ṽ = (A σ 1 I)(A σ 2 I)...(A σ p I)v; ṽ is such that K m p (ṽ;a) is already computed; set v = ṽ, go to 2. Roots σ 1,...,σ p are called shifts. Exact shifts = unwanted Ritz values from K m (v;a). Methods of restart: Sorensen (IRAM), Stewart (Krylov Schur) On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

9 Convergence: Hermitian matrices Theorem (Sorensen) Let: A Hermitian; Shifts = smallest p Ritz values. Then under some mild conditions: Ritz values converge to m p largest eigenvalues. The main tool: Cauchy interlacing property λ j+(n m) θ j λ j monotonicity of Ritz values: θ (1) j θ (2) j θ (3) j... bounded from above: θ (k) j λ j restarting preserves wanted Ritz values On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

10 Convergence: Hermitian matrices Theorem (Sorensen) Let: A Hermitian; Shifts = smallest p Ritz values. Then under some mild conditions: Ritz values converge to m p largest eigenvalues. The main tool: Cauchy interlacing property λ j+(n m) θ j λ j monotonicity of Ritz values: θ (1) j θ (2) j θ (3) j... bounded from above: θ (k) j λ j restarting preserves wanted Ritz values On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

11 Convergence: the general case Embree: Convergence fails for a whole class of matrices, any dimension. The wanted Ritz vectors are removed from the Krylov subspace. Example: matrix A On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

12 Convergence: the general case Embree: Convergence fails for a whole class of matrices, any dimension. The wanted Ritz vectors are removed from the Krylov subspace. Example: matrix A, 10-dim. Krylov subspace On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

13 Convergence: the general case Embree: Convergence fails for a whole class of matrices, any dimension. The wanted Ritz vectors are removed from the Krylov subspace. Example: non-normal matrix A, 10-dim. Krylov subspace On the Ritz values of normal matrices Convergence of the restarted Arnoldi method

14 Contents 1 Convergence of the restarted Arnoldi method 2 Interlacing for normal matrices 3 Characterization of Ritz values using a Cauchy matrix On the Ritz values of normal matrices Interlacing for normal matrices

15 Reduction to the Hermitian case Definition Let ϕ [0,2π. For a C, denote a (ϕ) = 1 2 ( e iϕ a + e iϕ a). For a matrix A, denote: A (ϕ) = 1 2 ( e iϕ A + e iϕ A ). On the Ritz values of normal matrices Interlacing for normal matrices

16 Restarting the Arnoldi method and A (ϕ) Interlacing does not hold for θ (ϕ). Interlacing for θ (A arbitrary): λ (ϕ) (n m)+j θ j λ (ϕ) j. θ θ (ϕ) < 1 2 r. H = X AX, X + = [X x], H + = X + AX +, then θ j+1,+ θ j θ j,+. Restart preserves θ (ϕ), does not preserve θ (no such shifts exist!) On the Ritz values of normal matrices Interlacing for normal matrices

17 Restarting the Arnoldi method and A (ϕ) Interlacing does not hold for θ (ϕ). Interlacing for θ (A arbitrary): λ (ϕ) (n m)+j θ j λ (ϕ) j. θ θ (ϕ) < 1 2 r. H = X AX, X + = [X x], H + = X + AX +, then θ j+1,+ θ j θ j,+. Restart preserves θ (ϕ), does not preserve θ (no such shifts exist!) On the Ritz values of normal matrices Interlacing for normal matrices

18 Contents 1 Convergence of the restarted Arnoldi method 2 Interlacing for normal matrices 3 Characterization of Ritz values using a Cauchy matrix

19 Geometry of the Ritz values Let A C n n. When does a set {θ 1,...,θ m } of complex numbers represent Ritz values for A, using any m dimensional subspace? A normal, Krylov subspaces.

20 Geometry of the Ritz values Notation: Λ = (λ 1,...,λ n ), Θ = (θ 1,...,θ m ) Cauchy matrix P(Λ,Θ) C m n, C(Λ,Θ) = λ 1 θ 1 λ 2 θ... 1 λ n θ λ 1 θ 2 λ 2 θ... 2 λ n θ λ 1 θ m λ 2 θ m... λ n θm P(Λ,Θ) i,j = (λ i θ j ) m q=1 q j Cm n λ i θ q 2 x = (x 1,...,x n ) > 0 if each x i 0 and at least one x i > 0. x = (x 1,...,x n ) 0 if all x i > 0.

21 Geometry of the Ritz values Notation: Λ = (λ 1,...,λ n ), Θ = (θ 1,...,θ m ) Cauchy matrix P(Λ,Θ) C m n, C(Λ,Θ) = λ 1 θ 1 λ 2 θ... 1 λ n θ λ 1 θ 2 λ 2 θ... 2 λ n θ λ 1 θ m λ 2 θ m... λ n θm P(Λ,Θ) i,j = (λ i θ j ) m q=1 q j Cm n λ i θ q 2 x = (x 1,...,x n ) > 0 if each x i 0 and at least one x i > 0. x = (x 1,...,x n ) 0 if all x i > 0.

22 Geometry of the Ritz values Notation: Λ = (λ 1,...,λ n ), Θ = (θ 1,...,θ m ) Cauchy matrix P(Λ,Θ) C m n, C(Λ,Θ) = λ 1 θ 1 λ 2 θ... 1 λ n θ λ 1 θ 2 λ 2 θ... 2 λ n θ λ 1 θ m λ 2 θ m... λ n θm P(Λ,Θ) i,j = (λ i θ j ) m q=1 q j Cm n λ i θ q 2 x = (x 1,...,x n ) > 0 if each x i 0 and at least one x i > 0. x = (x 1,...,x n ) 0 if all x i > 0.

23 Geometry of the Ritz values Theorem (B.) Let A C n n be normal with eigenvalues Λ = (λ 1,..., λ n ). Let Θ = (θ 1,...,θ m ) denote an m-tuple of distinct complex numbers such that λ i θ j, for all i = 1,...,n and j = 1,...,m. The following claims are equivalent: (a) There exists a vector v C n such that θ 1,...,θ m are Ritz values for A from K m (v;a). (b) Linear system P(Λ,Θ)ˆv = 0 has a solution ˆv > 0. (c) Linear system C(Λ,Θ)w = 0 has a solution w > 0. The main idea of the proof: The minimum π(a)v for π P m is obtained at π = κ(h m). The proof is constructive.

24 Geometry of the Ritz values Theorem (B.) Let A C n n be normal with eigenvalues Λ = (λ 1,..., λ n ). Let Θ = (θ 1,...,θ m ) denote an m-tuple of distinct complex numbers such that λ i θ j, for all i = 1,...,n and j = 1,...,m. The following claims are equivalent: (a) There exists a vector v C n such that θ 1,...,θ m are Ritz values for A from K m (v;a). (b) Linear system P(Λ,Θ)ˆv = 0 has a solution ˆv > 0. (c) Linear system C(Λ,Θ)w = 0 has a solution w > 0. The main idea of the proof: The minimum π(a)v for π P m is obtained at π = κ(h m). The proof is constructive.

25 Feasibility problem in LP Linear programming: solving the existence Ax = b,x > 0 is as hard as optimizing c τ x max, under condition Ax b.

26 The dual problem Theorem The following is equivalent to (a)(b)(c): (d) For each u C m there exists an eigenvalue λ i such that ) Re ( m j=1 u j λ i θ j 0. Farkas lemma: for M R n m it holds that: (i) Either Mv = 0 for some v 0, or u M > 0 for some u R m. (ii) Either Mv = 0 for some v > 0, or u M 0 for some u R m.

27 Geometry of the Ritz values

28 Geometry of the Ritz values

29 Geometry of the Ritz values

30 Geometry of the Ritz values

31 Corollary: Interlacing for the Hermitian matrices Corollary (compare: Parlett, The Symmetric Eigenvalue Problem) Suppose A is Hermitian. Let θ 1 >... > θ m denote the Ritz values from a m-dimensional Krylov subspace; θ j λ i. Then for all j = 1,...,m 1 there exists an eigenvalue λ such that θ j+1 < λ < θ j.

32 Corollary: Ritz values = convex combination of eigenvalues Using P(Λ,Θ)ˆv = 0 we get: θ j = n i=1 λ i ( ) P i,j = m q=1 λ i θ q 2 ˆv i 2 q j P i,j P 1,j + P 2,j P n,j.

33 Restarting Arnoldi: an example An unwanted Ritz value cannot equal the extreme λ. Can it equal the second largest eigenvalue? λ 1 = 8, λ 2 = 4. 4 A = Is there an initial vector v such that for K 3 (v;a) it holds θ 3 λ 2?

34 Restarting Arnoldi: an example An unwanted Ritz value cannot equal the extreme λ. Can it equal the second largest eigenvalue? λ 1 = 8, λ 2 = 4. 4 A = Is there an initial vector v such that for K 3 (v;a) it holds θ 3 λ 2?

35 Restarting Arnoldi: an example θ = λ =

36 Restarting Arnoldi: an example Optimization (θ 1,θ 2 max,θ 3 λ 2 ): v = The Ritz values from K 3 (v;a) are: θ 1 = i, θ 2 = i, θ 3 = Restarting to dimension 2 erases the component in direction u 2.

37 Bibliography D. C. Sorensen Implicit Application of Polynomial Filters in a k-step Arnoldi Method, SIAM J. Matrix Anal. Appl. (vol. 13, 1992.) G. W. Stewart A Krylov Schur Algorithm for Large Eigenproblems, SIAM J. Matrix Anal. Appl. (vol. 23, 2001.) M. Embree The Arnoldi Eigenvalue Iteration with Exact Shifts Can Fail, SIAM J. Matrix Anal. Appl. (vol. 31, 2009.) S. M. Malamud Inverse spectral problem for normal matrices and the Gauss Lucas theorem, Transactions of the American Mathematical Society (vol. 357, 2005.) J. F. Queiró, A. L. Duarte Imbedding conditions for normal matrices, Linear Algebra and its Applications (vol. 430, 2009.) Z. Bujanović Krylov Type Methods for Large Scale Eigenvalue Computations, PhD Thesis, Univ. of Zagreb (2011.)