Accurate eigenvalue decomposition of arrowhead matrices and applications

Transcription

1 Accurate eigenvalue decomposition of arrowhead matrices and applications Nevena Jakovčević Stor FESB, University of Split joint work with Ivan Slapničar and Jesse Barlow IWASEP9 June 4th, /31

2 Introduction We present a new algorithm (aheig) for computing eigenvalue decomposition of real symmetric arrowhead matrix. 2/31

3 Introduction We present a new algorithm (aheig) for computing eigenvalue decomposition of real symmetric arrowhead matrix. Under certain conditions, the aheig algorithm computes all eigenvalues and all components of corresponding eigenvectors with high rel. acc. in O(n 2 ) operations. 2/31

4 Introduction We present a new algorithm (aheig) for computing eigenvalue decomposition of real symmetric arrowhead matrix. Under certain conditions, the aheig algorithm computes all eigenvalues and all components of corresponding eigenvectors with high rel. acc. in O(n 2 ) operations. 2/31

5 Introduction We present a new algorithm (aheig) for computing eigenvalue decomposition of real symmetric arrowhead matrix. Under certain conditions, the aheig algorithm computes all eigenvalues and all components of corresponding eigenvectors with high rel. acc. in O(n 2 ) operations. The algorithm is based on shift-and-invert technique and limited finite O(n) use of double precision arithmetic when necessary. 2/31

6 Introduction We present a new algorithm (aheig) for computing eigenvalue decomposition of real symmetric arrowhead matrix. Under certain conditions, the aheig algorithm computes all eigenvalues and all components of corresponding eigenvectors with high rel. acc. in O(n 2 ) operations. The algorithm is based on shift-and-invert technique and limited finite O(n) use of double precision arithmetic when necessary. Orthogonality of eigenvectors computed by aheig algorithm is consequence of their accuracy (no need for follow-up orthogonalization). 2/31

7 Introduction We present a new algorithm (aheig) for computing eigenvalue decomposition of real symmetric arrowhead matrix. Under certain conditions, the aheig algorithm computes all eigenvalues and all components of corresponding eigenvectors with high rel. acc. in O(n 2 ) operations. The algorithm is based on shift-and-invert technique and limited finite O(n) use of double precision arithmetic when necessary. Orthogonality of eigenvectors computed by aheig algorithm is consequence of their accuracy (no need for follow-up orthogonalization). Each eigenvalue and corresponding eigenvector can be computed independently, which makes the algorithm adaptable for parallel computing. 2/31

8 Introduction We present a new algorithm (aheig) for computing eigenvalue decomposition of real symmetric arrowhead matrix. Under certain conditions, the aheig algorithm computes all eigenvalues and all components of corresponding eigenvectors with high rel. acc. in O(n 2 ) operations. The algorithm is based on shift-and-invert technique and limited finite O(n) use of double precision arithmetic when necessary. Orthogonality of eigenvectors computed by aheig algorithm is consequence of their accuracy (no need for follow-up orthogonalization). Each eigenvalue and corresponding eigenvector can be computed independently, which makes the algorithm adaptable for parallel computing. We also present the applications of aheig algorithm to hermitian arrowhead, symmetric tridiagonal matrices and diagonal plus rank one matrices. 2/31

9 Introduction Outline 1 Introduction 2 The idea of the aheig algorithm 3 Example 4 Accuracy of the aheig algorithm 5 Application to hermitian arrowhead matrices 6 Application to tridiagonal symmetric matrices 7 Application to diagonal + rank-one matrices (D +uu T ) 3/31

10 Introduction Let A = [ D z z T α be n n real symmetric arrowhead matrix, where D = diag(d 1,d 2,...,d n 1 ), z = [ ζ 1 ζ 2 ζ n 1 ] T, α R and ] 4/31

11 Introduction Let A = [ D z z T α be n n real symmetric arrowhead matrix, where D = diag(d 1,d 2,...,d n 1 ), z = [ ζ 1 ζ 2 ζ n 1 ] T, α R and A = VΛV T eigenvalue decomposition of A, where ] Λ = diag(λ 1,λ 2,...,λ n ) and V = [ v 1 v n ]. 4/31

12 Introduction We assume that A is irreducible: ζ i 0, i, d i d j, i j. Without loss of generality we can assume that ζ i > 0, i and that diagonal elements of D are ordered, that is d 1 > d 2 > > d n 1. 5/31

13 Introduction We assume that A is irreducible: ζ i 0, i, d i d j, i j. Without loss of generality we can assume that ζ i > 0, i and that diagonal elements of D are ordered, that is d 1 > d 2 > > d n 1. The assumptions on D imply the interlacing property λ 1 > d 1 > λ 2 > d 2 > > d n 2 > λ n 1 > d n 1 > λ n where λ i,i = 1,...,n, are eigenvalues of matrix A. 5/31

14 Introduction The eigenvalues of A are the zeros of function: n 1 ζi 2 ϕ A (λ) = α λ d i λ = α λ zt (D λi) 1 z. i=1 6/31

15 Introduction The eigenvalues of A are the zeros of function: n 1 ζi 2 ϕ A (λ) = α λ d i λ = α λ zt (D λi) 1 z. i=1 and the eigenvectors are given by: v i = x [ i (D λi I) 1 z, x i = x i 2 1 ], i = 1,...,n. 6/31

16 Introduction The eigenvalues of A are the zeros of function: n 1 ζi 2 ϕ A (λ) = α λ d i λ = α λ zt (D λi) 1 z. i=1 and the eigenvectors are given by: v i = x [ i (D λi I) 1 z, x i = x i 2 1 ], i = 1,...,n. Problem: λ i is not exact = v i may not be orthogonal. 6/31

17 Introduction The existing algorithms obtain orthogonal eigenvectors with the following procedure. Compute eigenvalues of A. Construct new matrix  (inverse problem) with prescribed eigenvalues λ, and diagonal matrix D (that is compute new ẑ and α). Compute eigenvectors of  with the previous formula. This way computed, eigenvectors are not the exact eigenvectors of starting matrix A, they are exact eigenvectors of matrix [ D ẑ  = ẑ T α ) ) ẑ i = ( d i λ ) i ( λj d n 1 i ( λj d i n )( λ1 d i ( ) ( ), j=2 dj 1 d i j=i+1 dj d i ], n 1 α = λ ) n + ( λj d j. j=1 7/31

18 The idea of the aheig algorithm Our algorithm has a different concept. Accuracy of the eigenvectors and their orthogonality follow from accuracy of computed eigenvalues. Thus, there is no need for follow-up orthogonalization. 8/31

19 The idea of the aheig algorithm Our algorithm has a different concept. Accuracy of the eigenvectors and their orthogonality follow from accuracy of computed eigenvalues. Thus, there is no need for follow-up orthogonalization. Let d i be the diagonal element (pole) in A which is closest to λ. 8/31

20 The idea of the aheig algorithm Our algorithm has a different concept. Accuracy of the eigenvectors and their orthogonality follow from accuracy of computed eigenvalues. Thus, there is no need for follow-up orthogonalization. Let d i be the diagonal element (pole) in A which is closest to λ. From the interlacing property it follows that either λ = λ i or λ = λ i+1. 8/31

21 The idea of the aheig algorithm Let A i be the shifted matrix, A i = A d i I = D z ζ i 0 0 D 2 z 2 z T 1 ζ i z T 2 a, 9/31

22 The idea of the aheig algorithm Let A i be the shifted matrix, A i = A d i I = D z ζ i 0 0 D 2 z 2 z T 1 ζ i z T 2 a, where D 1 = diag(d 1 d i,...,d i 1 d i ) positive definite, D 2 = diag(d i+1 d i,...,d n 1 d i ) negative definite, z 1 = [ ζ 1 ζ 2 ] ζ T i 1, z 2 = [ ζ i+1 ζ i+2 ] ζ T n 1, a = α d i. 9/31

23 The idea of the aheig algorithm Let A i be the shifted matrix, A i = A d i I = D z ζ i 0 0 D 2 z 2 z T 1 ζ i z T 2 a, where D 1 = diag(d 1 d i,...,d i 1 d i ) positive definite, D 2 = diag(d i+1 d i,...,d n 1 d i ) negative definite, z 1 = [ ζ 1 ζ 2 ] ζ T i 1, z 2 = [ ζ i+1 ζ i+2 ] ζ T n 1, a = α d i. Obviously µ = λ d i is an eigenvalue of A i iff λ is an eigenvalue of A. 9/31

24 The idea of the aheig algorithm Now A 1 i = D 1 1 w w T 1 b w T 2 1/ζ i 0 w 2 D /ζ i 0 0, 10/31

25 The idea of the aheig algorithm Now where A 1 i = D 1 1 w w T 1 b w T 2 1/ζ i 0 w 2 D /ζ i 0 0, w 1 = D 1 1 z 1 1, ζ i w 2 = D 1 2 z 1 2, ζ i b = 1 ζ 2 i ( a + z T 1 D 1 1 z 1 + z T ) 2 D 1 2 z 2. 10/31

26 The idea of the aheig algorithm λ is the eigenvalue of matrix A which is closest to the pole d i. 11/31

27 The idea of the aheig algorithm λ is the eigenvalue of matrix A which is closest to the pole d i. µ is the eigenvalue of matrix A i which is closest to zero. 11/31

28 The idea of the aheig algorithm λ is the eigenvalue of matrix A which is closest to the pole d i. µ is the eigenvalue of matrix A i which is closest to zero. 1/ µ = A 1 i 2 11/31

29 The idea of the aheig algorithm λ is the eigenvalue of matrix A which is closest to the pole d i. µ is the eigenvalue of matrix A i which is closest to zero. 1/ µ = A 1 i 2 µ is well behaved by the standard perturbation theory. 11/31

30 Example Example 1/2 Let A = , where D = [ ], z = [ ] and α = /31

31 Example Example 2/2 Eigenvalues computed by Matlab eig, aheig and Mathematica (100 digits) are: λ eig λ aheig λ Math λ 3 and λ 4 computed by aheig are accurate (to the machine precision). 13/31

32 Example Example 2/2 Eigenvalues computed by Matlab eig, aheig and Mathematica (100 digits) are: λ eig λ aheig λ Math λ 3 and λ 4 computed by aheig are accurate (to the machine precision). 13/31

33 Example Example 2/2 Eigenvalues computed by Matlab eig, aheig and Mathematica (100 digits) are: λ eig λ aheig λ Math λ 3 and λ 4 computed by aheig are accurate (to the machine precision). Eigenvectors computed by aheig are accurate and therefore, orthogonal. For example, let us look at U 4 U 4(eig) U 4(aheig) U 4(Math) /31

34 Example Example 2/2 Eigenvalues computed by Matlab eig, aheig and Mathematica (100 digits) are: λ eig λ aheig λ Math λ 3 and λ 4 computed by aheig are accurate (to the machine precision). Eigenvectors computed by aheig are accurate and therefore, orthogonal. For example, let us look at U 4 U 4(eig) U 4(aheig) U 4(Math) /31

35 Accuracy of the aheig algorithm We will use the following notation: MATRIX exact eigenvalue computed eigenvalue A λ λ A i µ Ã i = fl(a i ) µ µ = fl( µ) A 1 i ν (A 1 i ) = fl(a 1 i ) ν ν = fl( ν) 14/31

36 Accuracy of the aheig algorithm Let Ãi = fl(a i ) Ã i = D 1 (I +E 1 ) 0 0 z ζ i 0 0 D 2 (I +E 2 ) z 2 z T 1 ζ i z T 2 a(1+ε a ) where E 1,E 2 are diagonal matrices: (E 1 ) ii ε M, (E 2 ) ii ε M and ε a ε M. Also, ( ) ( ) A 1 i = fl A 1 i. 15/31

37 Accuracy of the aheig algorithm Let Ãi = fl(a i ) Ã i = D 1 (I +E 1 ) 0 0 z ζ i 0 0 D 2 (I +E 2 ) z 2 z T 1 ζ i z T 2 a(1+ε a ) where E 1,E 2 are diagonal matrices: (E 1 ) ii ε M, (E 2 ) ii ε M and ε a ε M. Also, ( ) ( ) A 1 i = fl A 1 i. All elements of A 1 i are computed with high relative accuracy except possibly b. Weather b is computed accurately (or we need extra precision) is monitored by condition number. 15/31

38 Accuracy of the aheig algorithm Q: What is the accuracy of computed A 1 i? Let s recall A 1 i = D 1 1 w w T 1 b w T 2 1/ζ i 0 w 2 D /ζ i 0 0, 16/31

39 Accuracy of the aheig algorithm Q: What is the accuracy of computed A 1 i? Let s recall A 1 i = D 1 1 w w T 1 b w T 2 1/ζ i 0 w 2 D /ζ i 0 0, where w 1 = D 1 1 z 1 ζ k 1, fl((w 1 ) k ) = (1 + ε 2 + ε 3 ), ζ i (d k d i )(1 + ε 1 )ζ i w 2 = D 1 2 z 1 2, ζ i b = 1 ζ 2 i ( a + z T 1 D 1 1 z 1 + z T ) 2 D 1 2 z 2. 16/31

40 Accuracy of the aheig algorithm Q: What is the accuracy of computed A 1 i? Let s recall A 1 i = D 1 1 w w T 1 b w T 2 1/ζ i 0 w 2 D /ζ i 0 0, where w 1 = D 1 1 z 1 ζ k 1, fl((w 1 ) k ) = (1 + ε 2 + ε 3 ), ζ i (d k d i )(1 + ε 1 )ζ i w 2 = D 1 2 z 1 2, ζ i b = 1 ζ 2 i ( a + z T 1 D 1 1 z 1 + z T ) 2 D 1 2 z 2. A : (A 1 i )ij = fl( A 1 i )ij = ( A 1 ) i (1+ε ij ij), ε ij 3ε M for all elements of matrix A 1 i with possible exception of element b. 16/31

41 Accuracy of the aheig algorithm Q: When b is not computed accurately and how to fix it? 17/31

42 Accuracy of the aheig algorithm Q: When b is not computed accurately and how to fix it? Condition K 1 K 1 (λ i ) = a + z1 TD 1 1 z 1 + z2 T D 1 A 1 i 2 ζi 2 2 z 2. 17/31

43 Accuracy of the aheig algorithm Q: When b is not computed accurately and how to fix it? Condition K 1 K 1 (λ i ) = a + z1 TD 1 1 z 1 + z2 T D 1 A 1 i 2 ζi 2 2 z 2. If K 1 >> 1 we have to compute b in double of standard precision arithmetic. 17/31

44 Accuracy of the aheig algorithm Q: When b is not computed accurately and how to fix it? Condition K 1 K 1 (λ i ) = a + z1 TD 1 1 z 1 + z2 T D 1 A 1 i 2 ζi 2 2 z 2. If K 1 >> 1 we have to compute b in double of standard precision arithmetic. Also K 1 (n 2) 1 ζ i max k=1,...,n 1 k i ζ k. 17/31

45 Accuracy of the aheig algorithm Example (double precision) 1/3 Let A = Condition numbers of matrix A eigenvalues are: K /31

46 Accuracy of the aheig algorithm Example (double precision) 2/3 A 2 = Inverse computed by aheig is A 1 2 = b = , by aheig, b = , by Matlab inv, b = , by aheig quad. 19/31

47 Accuracy of the aheig algorithm Example (double precision) 2/3 A 2 = Inverse computed by aheig is A 1 2 = b = , by aheig, b = , by Matlab inv, b = , by aheig quad. 19/31

48 Accuracy of the aheig algorithm Example (double precision) 3/3 Eigenvalues computed by aheig, aheig quad and Mathematica (100 digits) are: λ aheig λ aheig quad λ Math /31

49 Accuracy of the aheig algorithm Example (double precision) 3/3 Eigenvalues computed by aheig, aheig quad and Mathematica (100 digits) are: λ aheig λ aheig quad λ Math /31

50 Accuracy of the aheig algorithm Example (double precision) 3/3 Eigenvalues computed by aheig, aheig quad and Mathematica (100 digits) are: λ aheig λ aheig quad λ Math Eigenvectors computed by aheig quad are accurate and therefore, orthogonal. For example, let us look at U 6 U 6(eig) U 6(aheig quad) U 6(Math) /31

51 Accuracy of the aheig algorithm Example (double precision) 3/3 Eigenvalues computed by aheig, aheig quad and Mathematica (100 digits) are: λ aheig λ aheig quad λ Math Eigenvectors computed by aheig quad are accurate and therefore, orthogonal. For example, let us look at U 6 U 6(eig) U 6(aheig quad) U 6(Math) /31

52 Accuracy of the aheig algorithm Q: Is µ i eigenvalue of A i closest to zero and if not how far is it from the closest one? Condition K 2 K 2 (λ i ) = A 1 2. µ i. i A: If K 2 >> 1 µ i is not eigenvalue of matrix A i which is closest to zero. 21/31

53 Accuracy of the aheig algorithm Q: Is µ i eigenvalue of A i closest to zero and if not how far is it from the closest one? Condition K 2 K 2 (λ i ) = A 1 2. µ i. i A: If K 2 >> 1 µ i is not eigenvalue of matrix A i which is closest to zero. Two different cases: for λ 1 or λ n, we can compute them from the starting matrix A. for inside eigenvalues (only open problem), possible solutions is to send this eigenvalue to the other pole. 21/31

54 Accuracy of the aheig algorithm Example Arrowhead matrix applications in quantum optics 1/3 The research is about quantum dots excited states decay in real photonic crystals. Matrix has the following structure g 1 g 2 g n g 1 ω 1 A = g 2 ω 2,.... where g n quantum dot transition frequency, ω i is frequency of optical mode, g i interaction constant of quantum dot with optical modes. At this point our task is to compute the eigenvalues of matrix A. ω n 22/31

55 Accuracy of the aheig algorithm Example Arrowhead matrix applications in quantum optics 2/3 The size of the matrix is changeable but in realistic case it is approximately n 10 3 to For example for n = g is vector with components from interval [ , ]. ω is vector with components from interval [ , ]. = /31

56 Accuracy of the aheig algorithm Example Arrowhead matrix applications in quantum optics 3/3 The components of vector g are of the same order of magnitude = we can guarantee all eigenvalues will be computed with high relative [ ] accuracy. (K , ). Let { y(λ) = 0, d i > λ i+1 > d i+1 1, λ i+1 > d i or λ i+1 < d i+1 24/31

57 Accuracy of the aheig algorithm Example Arrowhead matrix applications in quantum optics 3/3 The components of vector g are of the same order of magnitude = we can guarantee all eigenvalues will be computed with high relative [ ] accuracy. (K , ). Let { y(λ) = 0, d i > λ i+1 > d i+1 1, λ i+1 > d i or λ i+1 < d i MATLAB 24/31

58 Accuracy of the aheig algorithm Example Arrowhead matrix applications in quantum optics 3/3 The components of vector g are of the same order of magnitude = we can guarantee all eigenvalues will be computed with high relative [ ] accuracy. (K , ). Let { y(λ) = 0, d i > λ i+1 > d i+1 1, λ i+1 > d i or λ i+1 < d i MATLAB AHEIG 24/31

59 Application to hermitian arrowhead matrices Algorithm herm2ahig Let [ D r H = r T α ], r = [ ρ 1 ρ 2 ρ n 1 ] T,ρi C hermitian arrowhead matrix transform [ A = D z H = z T α ]. where = diag( ρ 1 ρ 1,..., ρ n 1 ρ n 1,1). 25/31

60 Application to hermitian arrowhead matrices Algorithm herm2ahig Let [ D r H = r T α ], r = [ ρ 1 ρ 2 ρ n 1 ] T,ρi C hermitian arrowhead matrix transform [ A = D z H = z T α ]. where = diag( ρ 1 ρ 1,..., ρ n 1 ρ n 1,1). A = VΛV T = H = UΛU,U = V. 25/31

61 Application to hermitian arrowhead matrices Algorithm herm2ahig Let [ D r H = r T α ], r = [ ρ 1 ρ 2 ρ n 1 ] T,ρi C hermitian arrowhead matrix transform [ A = D z H = z T α where ]. = diag( ρ 1 ρ 1,..., ρ n 1 ρ n 1,1). A = VΛV T = H = UΛU,U = V. Accuracy of EVD of A = Accuracy of EVD of H. (If aheig quad is needed, we also need to compute z in double of standard precision.) 25/31

62 Application to tridiagonal symmetric matrices Algorithm dc t2a T is symmetric tridiagonal matrix α 1 β 2 T = β 2 α 2 β β n 1 α n 1 β n β n α n T 1 β k+1 e k 0 β k+1 e T k α k+1 β k+2 e T 1. 0 β k+2 e 1 T 2 where 1 < k < n, T 1 and T 2 are k k and (n k 1) (n k 1) submatrices of T, respectively, and e j is the j th unit vector of appropriate dimension. Usually k is taken to be [n/2]. 26/31

63 Application to tridiagonal symmetric matrices Algorithm dc t2a Let Q i D i Q T i = T i be an eigenvalue decomposition of T i. T = = = = QAQ T, T 1 β k+1 e k 0 β k+1 e T k α k+1 β k+2 e T 1 0 β k+2 e 1 T 2 Q 1 D 1 Q T 1 β k+1 e k 0 β k+1 e T k α k+1 β k+2 e T 1 0 β k+2 e 1 Q 2 D 2 Q T 2 0 Q Q 2 α k+1 β k+1 l T 1 β k+2 f T 2 β k+1 l 1 D 1 0 β k+2 f 2 0 D Q T Q T 2 where l T 1 is last row of Q 1 and f T 2 is first row of Q 2. Thus T is reduced to symmetric arrowhead matrix A by orthogonal transformation Q. 27/31

64 Application to tridiagonal symmetric matrices Algorithm dc t2a is used for computing an eigenvalue decomposition of symmetric tridiagonal matrices in a way that: We transform a symmetric tridiagonal matrix to symmetric arrowhead matrix by orthogonal transformation. We compute eigenvalue decomposition of symmetric arrowhead matrix using aheig algorithm. We can guarantee high relative accuracy of eigenvalues and orthogonality of eigenvectors of tridiagonal symmetric matrix only when we can guarantee high relative accuracy of eigenvalue decompositions of corresponding symmetric arrowhead matrices emerging during algorithm dc t2a. 28/31

65 Application to tridiagonal symmetric matrices Algorithm dc t2a Example Wilkinson matrix 21 1/1 λ Math λ dc t2a(t) /31

66 Application to tridiagonal symmetric matrices Algorithm dc t2a Example Wilkinson matrix 21 1/1 λ Math λ dc t2a(t) /31

67 Application to diagonal + rank-one matrices (D + uu T ) Algorithm dpr1 2a Let where M = D +uu T d 1 +u 2 1 u 1 u 2 u 1 u n u 2 u 1 d 2 +u 2 2 u 2 u n =..... u n u 1 u n u 2 d n +u 2 n d i R, i = 1,...,n, u i R,, i = 1,...,n. Now, for x 1 = 0, x j = u j /u 1, j = 2,...,n ( G = I + e 1 x T) M (I e 1 x T) is arrowhead matrix. 30/31

68 Application to diagonal + rank-one matrices (D + uu T ) Algorithm dpr1 2a Under assumptions we form d 1 < d j, j = 2,...,n ( d2 d 1 = diag 1,,, u 1 dn d 1 u 1 ). A = G 1 is symmetric arrowhead matrix of form d 1 +u u 2 n u 2 d2 d 1 u n dn d 1 u 2 d2 d 1 d 2 0 A =..... u n dn d 1 0 d n. Algorithm aheig is now used on A. If aheig quad is needed, we also need to compute α and z in double of standard precision. 31/31