An Explanation of the MRRR algorithm to compute eigenvectors of symmetric tridiagonal matrices

Transcription

1 An Explanation of the MRRR algorithm to compute eigenvectors of symmetric tridiagonal matrices Beresford N. Parlett Departments of Mathematics and Computer Science Division, EECS dept. University of California, Berkeley Inderjit S. Dhillon Department of Computer Sciences University of Texas, Austin

2 Difficulties All eigenvalues of T are easily computed in O(n 2 ) time. Given ˆλ, inverse iteration computes the eigenvector: (T ˆλI)x i+1 = x i, i = 0, 1, 2,.... Costs O(n) per iteration. Typically, 1-3 iterations are enough. BUT, inverse iteration only guarantees T ˆv ˆλˆv = O(ε T ).

3 Gap Theorem : Fundamental Limitations T ˆv ˆλˆv sin (v, ˆv). Gap(ˆλ) Assume all off-diagonals not negligible, so Gap(ˆλ) not zero, but can be small: 1 ε 1 ε 1 1 ε 2 ε 2 1 ε 3 ε 3 1 When eigenvalues are close, independently computed eigenvectors WILL NOT be mutually orthogonal.

4 Plot eigenvalues : Eigenvalues of Biphenyl Matrix Eigenvalue Eigenvalue Index Plot Absgap(i) = log 10 (min(λ i+1 λ i, λ i λ i 1 )/ T ) : 0 2 Absolute Eigenvalue Gaps Eigenvalue Index LAPACK one big cluster λ 1, λ 2,..., λ 939. Tridiagonal solution takes 80% of Total Time.

5 Ideal Solution Fundamental Limitation: sin (v, ˆv) T ˆv ˆλˆv. Gap(ˆλ) Get smallest possible residual norm. Compute eigenvalue to greatest accuracy possible : ˆλ λ = O(ε ˆλ ). Compute eigenvector to high relative accuracy : T ˆv ˆλˆv = O(ε ˆλ ). Gap Theorem implies : sin (v, ˆv) = O(ε λ ) Gap(ˆλ) = O(ε) Relgap(ˆλ). Can we achieve the above?

6 Key Idea 1. Discard Tridiagonals, Embrace Bidiagonals

7 Factored Forms yield Better Representations Tridiagonals DO NOT determine their eigenvalues to high relative accuracy. Bidiagonals determine their singular values to high relative accuracy. T + µi = L L T x x x x x x x x = x x x x.... x x x x x x x.... x x x Bidiagonal Factors are better since they allow us to compute eigenvalues to high relative accuracy, compute eigenvectors to high relative accuracy. High accuracy Orthogonality. For interior eigenvalues, extends to indefinite factorization LDL T.

8 Algorithm Outline 1. Choose µ such that T + µi is positive definite. 2. Compute the factorization : T + µi = LDL T. 3. Compute eigenvalues of LDL T to high relative accuracy (by dqds or bisection). 4. Given eigenvalues, compute accurate eigenvectors of LDL T. HOW?

9 Key Idea 2. Shift with Differential QD How do we get an eigenvector such that T ˆv ˆλˆv = O(ε ˆλ )?

10 Differential Transformations Inverse iteration Solve for z : LDL T ˆλI = L + D + L T +. L + D + L T + z = random vector. Simple qd : D + (1) := d 1 ˆλ for i = 1, n 1 L + (i) := (d i l i )/D + (i) D + (i + 1) := d i li 2 + d i+1 L + (i)d i l i ˆλ end for Differential qd (dqds) : s 1 := ˆλ for i = 1, n 1 D + (i) := s i + d i L + (i) := (d i l i )/D + (i) end for s i+1 := L + (i)l i s i ˆλ D + (n) := s n + d n

11 Computing an Eigenvector Compute the appropriate Twisted Factorization : where D r is diagonal, N r = T ˆλI = N r D r N T r, x x x.. x γ r x.. x x x x x and r is chosen to minimize γ r (it will be O(λ ˆλ)). Solve for z, N r D r Nr T z = γ r e r ( Nr T z = e r ) : 1, i = r, z(i) = L + (i) z(i + 1), i = r 1,...,1, U (i 1) z(i 1), i = r + 1,..., n. Solves an open problem posed by Wilkinson (1965).

12 Main Theorem THEOREM. [Dhillon & Parlett, 2003] Eigenvectors computed by twisted factorization are numerically orthogonal if eigenvalues of LDL T have large relative gaps. In particular, where (ˆv i, ˆv j ) = Relsep(λ i, λ j ) = O(ε) Relsep(λ i, λ j ), λ i λ j max( λ i, λ j ). Example of Large Relsep : λ 1 = 10 16, λ 2 = Relsep(λ 1, λ 2 ) 1 Above Theorem Automatic Orthogonality. Example of Small Relsep : λ 1 = , λ 2 =

13 Proof of Correctness Desired Relationship: LDL T ˆλI = N r D r N T r, and N r D r N T r z = γ r e r. computed v, LDL T ˆNr ˆDr ˆNT r, ẑ 3 ulps in L 3 ulps in D 4 ulps in Ñr 2 ulps in D r v, L D LT exact Ñ r Dr Ñ r T z = γr e r Exact Mathematical relationship holds : L D LT ˆλI = Ñr D r Ñ r T. Key step in proof is to relate ẑ to v in 3 steps : 1. ẑ is close to z, (only multiplications), 2. sin ( v, z) = O(ε λ )/gap(ˆλ), ( γ r = O(ε λ )), 3. sin ( v, v) = O(ε)/relgap(ˆλ) (relative perturbation theory). sin (ẑ, v) = O(ε) Relgap(ˆλ).

14 Key Idea 3. Shift for Separation, again differentially

15 Algorithm MR 3 (Multiple RRRs) 1. Choose µ such that T + µi is positive definite. 2. Compute the factorization : T + µi = LDL T. 3. Compute eigenvalues of LDL T to high relative accuracy (by dqds or bisection). 4. Group eigenvalues according to their Relative Gaps : a) isolated (agree in < 3 digits). Compute eigenvector using a twisted factorization. b) clustered (agree in > 3 digits). Pick µ near cluster to form LDL T µi = L 1 D 1 L T 1 (by dqds). Refine eigenvalues in cluster to high relative accuracy. Set L L 1, D D 1. Repeat step 4 for eigenvalues in cluster.

16 Key Idea 4. Analyze the Representation Tree Step 4 of the MR 3 algorithm may be represented as a tree: At the root is the original factorization. At each internal node is a factorization for another shift µ. Each child of the node for µ corresponds to an isolated eigenvalue or a cluster.

17 Small Example Eigenvalues: ε, 1 + ε, ε, 2. Extra representation needed at σ = 1: LDL T I = L 1 D 1 L T 1. The following Representation Tree captures the steps of the algorithm: ({L, D}, {1, 2, 3, 4}) ε 1 2 ( ) {N (1) 1, 1}, {1} ({L 1, D 1 }, {2, 3}) ε 2 ε ) ( ( {N (4) 2, 2}, {2} ) 3, 3}, {3} {N (4) ( {N (3) 4, 4}, {4} )

18 Wilkinson s Matrix W 21 + : Wilkinson s matrix. λ 20 and λ 21 are identical to working precision. What happens in this case? LDL T ˆλ 21 I = L 1 D 1 L T 1. λ 20 (L 1 D 1 L T 1 ) & λ 21(L 1 D 1 L T 1 ) no digits in common! & (ˆv 20, ˆv 21 ) = Computed Eigenvectors ˆv 20 and ˆv 21 :

19 Worst Case / Large Depth matrix with eigenvalues: 0, 1, 1 ± 10 15, 1 ± 10 12, 1 ± 10 9, 1 ± 10 6, 1 ± 10 3, 2. ({L, D}, {1,..., 13}) ε ({N 1, 1}, {1}) ({L 1, D 1}, {2,..., 12}) ({N 13, 13}, {13}) ({N 2, 2}, {2}) ({L 2, D 2}, {3,..., 11}) ({N 12, 12}, {12}) ({N 3, 3}, {3}) ({L 3, D 3}, {4,..., 10}) ({N 11, 11}, {11}) 0 ({N 4, 4}, {4}) ({L 4, D 4}, {5,..., 9}) ({N 10, 10}, {10}) 0 ({N 5, 5}, {5}) ({L 5, D 5}, {6,7, 8}) ({N 9, 9}, {9}) ({N 6, 6}, {6}) ({N 7, 7}, {7}) ({N 8, 8}, {8})

20 Performance of MRRR on Biphenyl Matrix For the biphenyl matrix (n = 966), the root node had 805 leaf children and 63 internal node children. all nodes at the next level were leaf nodes. 49 clusters had 2 eigenvalues, 13 clusters had 3-8 eigenvalues. one cluster had 9 eigenvalues,

21 Residual Norms for computed z - Part 1 Paper 1 guarantees small residuals at the bottom of the representation tree : a leaf and its parent (leaf δλi)z = γ = O(εδλ). Our problem : (root λi)z = γ = O(ε spdiam(root))??? In exact arithmetic, root residual is also O(εδλ).

22 Residual Norms for computed z - Part 2 Compare child residual with parent residual at each interval node from leaf to root. parent λ + τ r p computed child r c λ r c = (T c λi)z T c = L c D c L T c r p exact τ r c By design also r p = r c r p = r p + δt p z, r c = r c + δt c z, exact Tp = T p + δt p Tc = T c + δt c T + δt = (L + δl)(d + δd)(l + δl) T, T = LDL T

23 Technical Lemma If then Dz c spdiam(t 0 ) LD L T z c spdiam(t 0 ), L = L I, ( ) δtz (2c )(9ε) spdiam(t 0) + O(nε 2 ) NOTE: large values in D and LDL T can be neutralized by small entries in z. (*) gives a bound on increase in residual norm at each internal node on path from leaf to root.

24 Orthogonality j j Γ α LDL T, Γ Γ α Γ β = k Γ β k By Paper 1, eigenvectors with same parent are orthogonal to working accuracy. Our problem : z T j z k = O(nε)???? S Γ = S LDLT Γ = subspace invariant under LDL T for eigenvalues in Γ.

25 Two Angles Ψ k,γ := (z k, S Γ ) Φ Γα := (SΓ parent α, SΓ child α ), SΓ parent α S parent Γ Lemma 1. sin Ψ j,γ sin Ψ j,γα + sin Φ Γα Lemma 2. sin Φ Γα Rnε, R depends on tolerance for relgap.

26 Theorem Let (LDL T, Γ) be the least common ancestor of z j and z k,j k. If all internal nodes on the paths from leaves < j > and < k > to Γ, (in the representation tree) are RRRs, then cos (z j,z k ) 2 leafbound+{depth(γ,j)+depth(γ,k) 2}Rnε.