A Tuned Preconditioner for Inexact Inverse Iteration Applied to Hermitian Eigenvalue Problems

Transcription

1 A Tuned Preconditioner for Applied to Eigenvalue Problems Department of Mathematical Sciences University of Bath, United Kingdom IWASEP VI May 22-25, 2006 Pennsylvania State University, University Park Joint work with: Alastair Spence

2

3

4 Problem and Inverse Find an eigenvalue and eigenvector of a positive definite A: Ax = λx, Inverse : A large, sparse. (A σi)y = x Inverse iteration with preconditioned iterative

5

6 for i = 1 to i max do choose τ (i), σ (i) solve (A σ (i) I)y (i) x (i) τ (i), Rescale x (i+1) = y(i) y (i), Update λ (i+1) = x (i+1)t Ax (i+1), possibly: update the shift σ (i) Test: eigenvalue residual r (i+1) = (A λ (i+1) I)x (i+1). end for

7 Error indicator Error indicator (Orthogonal decomposition for symmetric A, Parlett) Eigenvalue residual x (i) = cos θ (i) x 1 + sin θ (i) x (i), x(i) x 1. sin θ (i) λ 2 λ (i) r (i) sin θ (i) λ n λ 1

8 rates of inexact inverse iteration Decreasing tolerance τ (i) = C r (i) = O(sin θ (i) ) 1 For decreasing tolerance τ (i) C r (i) = O(sin θ (i) ) the inexact method recovers the rate of convergence achieved by exact. 2 Fixed shift σ: linear convergence. 3 Rayleigh quotient shift σ (i) = ρ(x (i) ) = x(i)t Ax (i) x (i)t x : cubic (i) convergence for A = A. Fixed tolerance τ (i) = τ 1 Rayleigh quotient shift: quadratic convergence

9 MINRES (A σi)y = x when A is symmetric Solving a linear system (A σi)y = x standard MINRES for y 0 = 0: x (A σi)y k 2 where κ is the condition number of A σi. Number of inner iterations: then k 1 + κ ( ) k 1 κ 1 x. κ + 1 { log 2 + log x } τ x (A σi)y k τ.

10 Unpreconditioned with MINRES rates for with MINRES for simple eigenvalue If A is positive definite and has a simple eigenvalue then x (i) (A σi)y (i) k 2 2 λ 1 λ n λ 1 σ ( )k 2 κ1 1 Qx (i) 2. κ where Q is the orthogonal projection onto span{x 2,..., x n } and κ 1 is the reduced condition number κ 1 = max i=2,...,n λ i σ min i=2,...,n λ i σ. Number of inner for each i for x (i) (A σ (i) I)y (i) τ (i) ( ) k (i) 2 + κ 1 log 2 λ 1 λ n + log Qx(i) 2 λ 1 σ τ (i)

11 Unpreconditioned with MINRES rates for with MINRES for simple eigenvalue If A is positive definite and has a simple eigenvalue then x (i) (A σi)y (i) k 2 2 λ 1 λ n λ 1 σ ( )k 2 κ1 1 sin θ (i). κ where Q is the orthogonal projection onto span{x 2,..., x n } and κ 1 is the reduced condition number κ 1 = max i=2,...,n λ i σ min i=2,...,n λ i σ. Number of inner for each i for x (i) (A σ (i) I)y (i) τ (i) ( k (i) 2 + κ 1 log 2 λ 1 λ n + log sin ) θ(i) λ 1 σ τ (i)

12

13 Incomplete Cholesky A = LL T + E symmetric of (A σi)y (i) = x (i) : Remarks L 1 (A σi)l T ỹ (i) = L 1 x (i), 1 changes number of inner iterations k (i) 2 + κ 1 (log 2 λ 1 λ n + log 2 k (i) increases with i for τ (i) = C r (i). y (i) = L T ỹ (i) L 1 ) λ 1 σ τ (i)

14 Incomplete Cholesky A = LL T + E symmetric of (A σi)y (i) = x (i) : Remarks L 1 (A σi)l T ỹ (i) = L 1 x (i), 1 changes number of inner iterations k (i) 2 + κ 1 (log 2 λ 1 λ n + log 2 k (i) increases with i for τ (i) = C r (i). y (i) = L T ỹ (i) L 1 ) λ 1 σ τ (i)

15 Derivation Aims 1 modify L L L 1 (A σi)l T ỹ (i) = L 1 x (i), 2 minor extra computation cost for L y (i) = L T ỹ (i) 3 nice right hand side L 1 x (i) (same behaviour as unpreconditioned, e.g. for fixed shifts k (i) does not increase with i)

16 Derivation Aims 1 modify L L L 1 (A σi)l T ỹ (i) = L 1 x (i), 2 minor extra computation cost for L y (i) = L T ỹ (i) 3 nice right hand side L 1 x (i) (same behaviour as unpreconditioned, e.g. for fixed shifts k (i) does not increase with i)

17 Choice of L Condition MINRES indicates that L 1 x (i) should be close to eigenvector of L 1 (A σi)l T Holds if Justification of LL T x (i) = Ax (i) If x (i) = x 1 then LL T x 1 = λ 1 x 1 LL T x (i) = Ax (i) L 1 (A σi)l T L 1 x 1 = λ 1 σ λ 1 L 1 x 1 L 1 (A σi)l T L 1 x (i) = λ 1 σ L 1 x (i) + C r (i) λ 1

18 Choice of L Condition MINRES indicates that L 1 x (i) should be close to eigenvector of L 1 (A σi)l T Holds if Justification of LL T x (i) = Ax (i) If x (i) = x 1 then LL T x 1 = λ 1 x 1 LL T x (i) = Ax (i) L 1 (A σi)l T L 1 x 1 = λ 1 σ λ 1 L 1 x 1 L 1 (A σi)l T L 1 x (i) = λ 1 σ L 1 x (i) + C r (i) λ 1

19 How do we achieve LL T x (i) = Ax (i)? Theorem Let x (i) current eigenvector approximation, e (i) = Ax (i) LL T x (i) (known) and L chosen such that L = L + α (i) e (i) (L 1 e (i) ) T with α (i) root of quadratic function we get LL T x (i) = Ax (i). Implementation 1 Note: LL T = LL T 1 + e (i)t x (i) e(i) e (i)t 2 L is a rank-one update of L.

20 How do we achieve LL T x (i) = Ax (i)? Theorem Let x (i) current eigenvector approximation, e (i) = Ax (i) LL T x (i) (known) and L chosen such that L = L + α (i) e (i) (L 1 e (i) ) T with α (i) root of quadratic function we get LL T x (i) = Ax (i). Implementation 1 Note: LL T = LL T 1 + e (i)t x (i) e(i) e (i)t 2 L is a rank-one update of L.

21 Implementation General positive definite For MINRES implementation only the evaluation of P 1 is necessary Sherman-Morrison formula where z (i) = P 1 Ax (i). P = P + γ (i) e (i) e (i)t P 1 = P 1 (z(i) x (i) )(z (i) x (i) ) T (z (i) x (i) ) T Ax (i)

22 rates The 1 outer convergence rate is retained 2 cheap inner are provided ( sin θ k (i) (i) ) C 1 + C 2 log λ 1 σ τ (i) 3 only a single extra back substitution with P = LL T per outer iteration needed

23 Example SPD matrix from the Matrix Market library (nos5: 3 story building with attached tower) seek eigenvalue near fixed shift σ = 100 A LL T, incomplete Cholesky factorisation (drop tol. = 0.1) compare standard and

24 Fixed shift with standard incomplete Cholesky total number of inner iterations using standard : 2026 total number of inner iterations using : 779

25 Comparison of LL T with LL T Spectral properties of preconditioned matrix Let Theorem If σ / Λ(A) then µ, ξ 0 and L 1 (A σi)l T w = µw L 1 (A σi)l T ŵ = ξŵ min µ Λ(L 1 (A σi)l T ) where γ = 1/(e T x) and v = L 1 e. µ ξ ξ γv v,

26 Comparison of LL T with LL T Spectral properties of preconditioned matrix Let L 1 (A σi)l T w = µw L 1 (A σi)l T ŵ = ξŵ Interlacing property Rewrite second equation Dt = ξ(i + γzz T )t where L 1 (A σi)l T = QDQ T, z = Q T v, (I + αvv T )Qt = ŵ. Interlacing property If γ > 0 eigenvalues are moved towards the origin. If γ < 0 eigenvalues are moved away from the origin.

27 Comparison of LL T with LL T Spectral properties of preconditioned matrix Let L 1 (A σi)l T w = µw L 1 (A σi)l T ŵ = ξŵ Interlacing property Rewrite second equation Dt = ξ(i + γzz T )t where L 1 (A σi)l T = QDQ T, z = Q T v, (I + αvv T )Qt = ŵ. Interlacing property If γ > 0 eigenvalues are moved towards the origin. If γ < 0 eigenvalues are moved away from the origin.

28 Comparison of LL T with LL T Interlacing property µ and ξ interlace each other depending on the sign of γ Clustering properties are preserved reduced condition number κ 1 L κ1 L κ1 L (1 + γvt v)

29 Comparison of LL T with LL T Interlacing property µ and ξ interlace each other depending on the sign of γ Clustering properties are preserved reduced condition number κ 1 L κ1 L κ1 L (1 + γvt v)

30 Changing the right hand side Approach by Simoncini/Eldén [3] Instead of solving L 1 (A σ (i) I)L T ỹ (i) = L 1 x (i), change the right hand side L 1 (A σ (i) I)L T ỹ (i) = L T x (i), y (i) = L T ỹ (i) y (i) = L T ỹ (i)

31 Comparison Tuned and Simoncini & Eldén Example nos5.mtx from Matrix Market. Solves to fixed tolerance τ = Rayleigh quotient shift. Quadratic convergence for both methods. Simoncini & Eldén Tuned Drop Tolerances Outer total

32

33 example Ax = λm x with bcsstk08 (Structural engineering) Figure: Fixed Shift Figure: Rayleigh Quotient Shift

34 example for the eigenproblem Ax = λm x with bcsstk08 (Structural engineering) Figure: Fixed Shift Figure: Rayleigh Quotient Shift

35 M. A. Freitag and A. Spence, rates for inexact inverse iteration with application to preconditioned iterative, Submitted to BIT., A for inexact inverse iteration applied to eigenvalue problems, Submitted to IMA J. Numer. Anal. V. Simoncini and L. Eldén, Inexact Rayleigh quotient-type methods for eigenvalue computations, BIT, 42 (2002), pp