EIGENVALUE PROBLEMS. EIGENVALUE PROBLEMS p. 1/4

EIGENVALUE PROBLEMS EIGENVALUE PROBLEMS p. 1/4

EIGENVALUE PROBLEMS p. 2/4 Eigenvalues and eigenvectors Let A C n n. Suppose Ax = λx, x 0, then x is a (right) eigenvector of A, corresponding to the eigenvalue λ.

EIGENVALUE PROBLEMS p. 2/4 Eigenvalues and eigenvectors Let A C n n. Suppose Ax = λx, x 0, then x is a (right) eigenvector of A, corresponding to the eigenvalue λ. Q: Prove that if x is an eigenvector, so is αx, for α 0. So we often take x = 1. For an eigenpair (λ,x), (λi A)x = 0, x 0. Q: This is only possible if? λi A is singular. For general λ, π(λ) det (λi A) is called the characteristic polynomial of A π(λ) = λ n (a 11 + + a nn )λ n 1 + + det( A). π(λ) = 0 is called the characteristic equation. π(λ) has exact degree n, and so has n zeros λ 1,...,λ n, say.

EIGENVALUE PROBLEMS p. 3/4 Eigenvalues and eigenvectors, ctd Spectrum of A: the set Λ(A) = {λ 1,...,λ n }.

EIGENVALUE PROBLEMS p. 3/4 Eigenvalues and eigenvectors, ctd Spectrum of A: the set Λ(A) = {λ 1,...,λ n }. An eigenvalue λ has algebraic multiplicity r if it is repeated exactly r times.

EIGENVALUE PROBLEMS p. 3/4 Eigenvalues and eigenvectors, ctd Spectrum of A: the set Λ(A) = {λ 1,...,λ n }. An eigenvalue λ has algebraic multiplicity r if it is repeated exactly r times. Q: Can real A have complex eigenvalues? Q: Let x i be an eigenvector of A coresponding to λ i. Are x 1,...,x n linearly independent? [ ][ ] [ ] 1 1 ξ ξ =, 1 η η It has 2 eigenvalues 1 and 1, but 1 independent eigenvector.

EIGENVALUE PROBLEMS p. 4/4 Similarity Transformations (ST) Similarity transformations: B X 1 AX. A and B are said to be similar.

EIGENVALUE PROBLEMS p. 4/4 Similarity Transformations (ST) Similarity transformations: B X 1 AX. A and B are said to be similar. Thm. Similar matrices have the same characteristic polynomial.

EIGENVALUE PROBLEMS p. 4/4 Similarity Transformations (ST) Similarity transformations: B X 1 AX. A and B are said to be similar. Thm. Similar matrices have the same characteristic polynomial. Cor. Similarity transformations preserve eigenvalues and algebraic multiplicities.

EIGENVALUE PROBLEMS p. 5/4 Jordan Canonical Form (JCF) k k Jordan block: J (k) λ = λ 1 λ 1 λ It has one distinct eigenvalue λ with algebraic multiplicity k and geometric multiplicity 1.

EIGENVALUE PROBLEMS p. 5/4 Jordan Canonical Form (JCF) k k Jordan block: J (k) λ = λ 1 λ 1 λ It has one distinct eigenvalue λ with algebraic multiplicity k and geometric multiplicity 1. Thm. Let A C n n, then there exist unique numbers λ 1,λ 2,...,λ s (complex, not necessarily distinct), and unique positive integers m 1,m 2,...,m s, and nonsingular X C n n giving the JCF of A X 1 AX = J diag[j (m 1) λ 1,...,J (m s) ]. λ s

EIGENVALUE PROBLEMS p. 6/4 Jordan Canonical Form, ctd We see AX = XJ, X = [x 1,,x n ]. Q: Which of x 1,...,x n are eigenvectors?

EIGENVALUE PROBLEMS p. 6/4 Jordan Canonical Form, ctd We see AX = XJ, X = [x 1,,x n ]. Q: Which of x 1,...,x n are eigenvectors? Q: Can we find the algebraic multiplicity and geometric multiplicity of an eigenvalue of A from its JCF? Q: What is the relationship between the algebraic multiplicity and geometric multiplicity of an eigenvalue? If all the Jordan blocks corresponding to the same eigenvalue have dimension 1, the eigenvalue is nondefective or semisimple, else defective.

EIGENVALUE PROBLEMS p. 7/4 Jordan Canonical Form, ctd If J is diagonal, A is diagonalizable.

EIGENVALUE PROBLEMS p. 7/4 Jordan Canonical Form, ctd If J is diagonal, A is diagonalizable. Q. Is nondefective = diagonalizabl?

EIGENVALUE PROBLEMS p. 7/4 Jordan Canonical Form, ctd If J is diagonal, A is diagonalizable. Q. Is nondefective = diagonalizabl? Q: If a matrix A C n n has n distinctive eigenvalues, is A diagonalizable? If two or more Jordan blocks have the same eigenvalue, the eigenvalue is derogatory, else nonderogatory. If a matrix has a derogatory eigenvalue, the matrix is derogatory, else nonderogatory.

EIGENVALUE PROBLEMS p. 8/4 Jordan Canonical Form, ctd The JCF is classical theory important in describing properties of various linear differential equations, and so is important in control theory etc. Two problems with JCF: For a general A, X in X 1 AX = J can be very ill-conditioned. Rounding errors in the JCF computation can be greatly magnified: fl(x 1 AX) = X 1 AX+E, E 2 = O(u)κ 2 (X) A 2.

EIGENVALUE PROBLEMS p. 9/4 Unitary similarity transformations Unitary transformations preserve size. If Q C n n is unitary, B Q H AQ is a unitary similarity transformation of A.

EIGENVALUE PROBLEMS p. 9/4 Unitary similarity transformations Unitary transformations preserve size. If Q C n n is unitary, B Q H AQ is a unitary similarity transformation of A. Q: What are the relations between the eigenvalues and singular values of B and those of A?

EIGENVALUE PROBLEMS p. 10/4 Schur Decomposition Schur s Theorem. There exists a similarity transformation with unitary Q such that Q H AQ = R, R upper triangular.

EIGENVALUE PROBLEMS p. 10/4 Schur Decomposition Schur s Theorem. There exists a similarity transformation with unitary Q such that Q H AQ = R, R upper triangular. Pf. A has at least one eigenvector x, so suppose Ax = λx, x H x = 1, Q H 1 x = e 1, with Q 1 unitary, gives Q 1 = [x,f] say, then [ ] x Q H 1 AQ 1 = H Ax x H AF F H Ax F H AF = [ λ x H AF O F H AF ], and the same argument can be applied to F H AF, etc.

EIGENVALUE PROBLEMS p. 11/4 Real Schur decomposition If A R n n, then there exists real orthogonal Q, giving Q T AQ = R, R is quasi upper triangular, with blocks on the diagonal corresponding to complex conjugate pairs, e.g., R = two complex conjugate pairs, and one real eigenvalue.

EIGENVALUE PROBLEMS p. 12/4 Real Schur decomposition, ctd Q: If A = A H, the Schur form has real diagonal R?

EIGENVALUE PROBLEMS p. 12/4 Real Schur decomposition, ctd Q: If A = A H, the Schur form has real diagonal R? Hermitian matrices have real eigenvalues, and a complete set of unitary eigenvectors Q.

EIGENVALUE PROBLEMS p. 12/4 Real Schur decomposition, ctd Q: If A = A H, the Schur form has real diagonal R? Hermitian matrices have real eigenvalues, and a complete set of unitary eigenvectors Q. Any matrix having a complete set of unitary eigenvectors is called a normal matrix.

EIGENVALUE PROBLEMS p. 13/4 Algorithms for the EVP If A is real, A = A T, A = Q diag(λ i )Q T, Q T Q = I, the eigendecomposition of A, is the same as the SVD (apart from signs).

EIGENVALUE PROBLEMS p. 13/4 Algorithms for the EVP If A is real, A = A T, A = Q diag(λ i )Q T, Q T Q = I, the eigendecomposition of A, is the same as the SVD (apart from signs). Q: What is one way of computing this? Jacobi s method (1845) For general A C n n, there exists unitary Q such that Q H AQ = R, R upper triangular. Also Q H AQ 2,F = A 2,F, numerically desirable. So we seek such Q and R.

EIGENVALUE PROBLEMS p. 14/4 Algorithms for the EVP, ctd Q: Why must methods be iterative for the EVP?

EIGENVALUE PROBLEMS p. 14/4 Algorithms for the EVP, ctd Q: Why must methods be iterative for the EVP? Finding zeros of π(λ) = 0 is a nonlinear problem cannot be solved on a fixed number of steps.

EIGENVALUE PROBLEMS p. 14/4 Algorithms for the EVP, ctd Q: Why must methods be iterative for the EVP? Finding zeros of π(λ) = 0 is a nonlinear problem cannot be solved on a fixed number of steps. The form of such iterative algorithms for computing the Schur form is: A 1 = A A k+1 = Q H k A k Q k, Q k is unitary, k = 1, 2,... = Q H k Q H k 1 Q H 1 AQ 1 Q k 1 Q k, We try to design the Q k so that A k upper triangular.

EIGENVALUE PROBLEMS p. 15/4 The QR algorithm Basic algorithm: A 1 = A for k = 1, 2,... until convergence do QR factorization of A k : A k = Q k R k, Q k unitary, R k upper triangular recombine in the reverse order: A k+1 = R k Q k end

EIGENVALUE PROBLEMS p. 16/4 The QR algorithm, ctd 1. A k+1 = Q H k A k Q k = Q H k Q H 1 AQ 1 Q k is a unitary similarity transformation of A. With refinements this is one of the most effective matrix computation algorithms for transforming A to upper triangular form using unitary similarity transformations. 2. If A is real, then Q k are orthogonal, A k+1 = Q T k A k Q k = Q T k Q T 1 AQ 1 Q k. With some refinements, the algorithm will transform A to quasi-upper triangular form.

EIGENVALUE PROBLEMS p. 17/4 Hessenberg Reduction: Each iteration (called a QR step) costs O(n 3 ) flops very expensive.

EIGENVALUE PROBLEMS p. 17/4 Hessenberg Reduction: Each iteration (called a QR step) costs O(n 3 ) flops very expensive. Q. Can we reduce A to a matrix with special structure so that each QR step costs O(n 2 ) flops? Reduce the matrix A to the Hessenberg form:, the closest we can get to upper triangular form using a fixed number of unitary similarity transformations.

EIGENVALUE PROBLEMS p. 18/4 Hessenberg Reduction: Q. How to do the Hessenberg reduction?

EIGENVALUE PROBLEMS p. 18/4 Hessenberg Reduction: Q. How to do the Hessenberg reduction? Q. What is its cost if A is real? 10n 3 /3 flops.

EIGENVALUE PROBLEMS p. 18/4 Hessenberg Reduction: Q. How to do the Hessenberg reduction? Q. What is its cost if A is real? 10n 3 /3 flops. Q. Do we lose the upper Hessenberg form in the iterations?

EIGENVALUE PROBLEMS p. 18/4 Hessenberg Reduction: Q. How to do the Hessenberg reduction? Q. What is its cost if A is real? 10n 3 /3 flops. Q. Do we lose the upper Hessenberg form in the iterations? No if we use Givens rotations in the QR factn. Q. What is the cost of each QR step now if A is real? 6n 2 flops.

EIGENVALUE PROBLEMS p. 19/4 QR algorithm with Hessenberg reduction: Compute Hessenberg reduction A 1 := Q H 0 AQ 0 where Q 0 := H 1...H n 2 for k = 1, 2,... until convergence do QR factorization of A k : A k = Q k R k, Q k unitary, R k upper triangular recombine in the reverse order: A k+1 = R k Q k end

EIGENVALUE PROBLEMS p. 20/4 Shifting Let λ 1 λ n. If λ i > λ i+1, we can show a (k) i+1,i 0, as λ i+1 k λ i 0 so it has linear convergence.

EIGENVALUE PROBLEMS p. 21/4 Shifting, ctd Q: What are the eigenvalues of A µi, and what is the new measure of convergence for a (k) i+1,i 0?

EIGENVALUE PROBLEMS p. 21/4 Shifting, ctd Q: What are the eigenvalues of A µi, and what is the new measure of convergence for a (k) i+1,i 0? Eigenvalues of A µi: λ i µ, i = 1 : n.

EIGENVALUE PROBLEMS p. 21/4 Shifting, ctd Q: What are the eigenvalues of A µi, and what is the new measure of convergence for a (k) i+1,i 0? Eigenvalues of A µi: λ i µ, i = 1 : n. Suppose we renumber the eigenvalues of A so that λ 1 µ λ n µ New measure of convergence: λ i+1 µ λ i µ.

EIGENVALUE PROBLEMS p. 22/4 Shifting, ctd Suppose λ n µ < λ n 1 µ. If we apply the QR iterations to Ã1 = A 1 µi, then ã (k) n,n 1 will converge to zero quickly. Suppose after k 0 1 QR steps, ã (k 0) n 1,n is small enough, we regard it as zero and add the shift back on: Ã k0 + µi = 0 λ Then λ is an eigenvalue of A.

EIGENVALUE PROBLEMS p. 23/4 Shifting, ctd The above process can be written as Ã 1 := A 1 µi for k = 1 : k 0 1 Ã k = Q k Rk Ã k+1 = R k Qk end A k0 := Ãk 0 + µi

EIGENVALUE PROBLEMS p. 23/4 Shifting, ctd The above process can be written as Ã 1 := A 1 µi for k = 1 : k 0 1 Ã k = Q k Rk Ã k+1 = R k Qk end A k0 := Ãk 0 + µi It is easy to show the above process is equivalent to for k = 1 : k 0 1 A k µi = Q k R k A k+1 = R k Q k + µi end

EIGENVALUE PROBLEMS p. 24/4 Shifting, ctd But there is no reason to use the same shift µ in all QR steps.

EIGENVALUE PROBLEMS p. 24/4 Shifting, ctd But there is no reason to use the same shift µ in all QR steps. Also we usually can only find a good approximation to an eigenvalue during the QR iterations. So we should use different shifts in the algorithm for different QR steps. Shifted QR Algorithm with Hessenberg Reduction: Compute the Hessenberg reduction A 1 = Q H 0 AQ 0, where Q 0 := H 1...H n 2 for k = 1, 2,... until convergence do A k µ k I = Q k R k A k+1 = R k Q k + µ k I end

EIGENVALUE PROBLEMS p. 25/4 Shifting, ctd Remarks: A k+1 = Q H k (A k µ k I)Q k + µ k I = Q H k A kq k. Now Q k depends on µ k.

EIGENVALUE PROBLEMS p. 25/4 Shifting, ctd Remarks: A k+1 = Q H k (A k µ k I)Q k + µ k I = Q H k A kq k. Now Q k depends on µ k. With correct choice of shift we get quadratic convergence, and if A = A H we can get approximately cubic convergence. If a (k) n,n 1 is small, then a(k) nn is close to an eigenvalue. So one possible choice for µ is a (k) nn. This shift is called the Rayleigh quotient shift.

Example: no shifting >> A = [8 1; -2 1]; >> A1 = A; >> [Q1,R1] = qr(a1); >> A2 = R1 * Q1 A2 = 7.8235e+00-2.7059e+00 2.9412e-01 1.1765e+00 >> [Q2,R2] = qr(a2); >> A3 = R2 *Q2 A3 = 7.7236e+00-4.7985e-02 2.9520e+00 1.2764e+00 >> [Q3,R3] = qr(a3); >> A4 = R3 *Q3 EIGENVALUE PROBLEMS p. 26/4

EIGENVALUE PROBLEMS p. 27/4 Example: shifting >> A1 = A; >> I = eye (2); >> [Q1,R1] = qr(a1 - A1(2,2)*I); >> A2 = R1 * Q1 + A1(2,2)*I A2 = 7.7358e+00-2.9245e+00 7.5472e-02 1.2642e+00 >> [Q2,R2] = qr(a2 - A2(2,2)*I); >> A3 = R2 * Q2 + A2(2,2)*I A3 = 7.7017e+00 2.9996e+00-3.9768e-04 1.2983e+00

EIGENVALUE PROBLEMS p. 28/4 Deflation Technique When a (k) n,n 1 row and column. is small enough, we deflate by ignoring the last 0 Note the remaining eigenvalues of A are those of A k (1:n 1, 1:n 1). Apply the QR algorithm to A k (1:n 1, 1:n 1).

EIGENVALUE PROBLEMS p. 29/4 Deflation Technique, ctd Note: During iterations, it may happen that some a (k) i+1,i other than a(k) n 1,n becomes very small. Then we just regard it as zero, and work with A k (1:i, 1:i), A k (i + 1:n,i + 1:n)

EIGENVALUE PROBLEMS p. 30/4 Special case A = A H Q: What does the upper Hessenberg form become?

EIGENVALUE PROBLEMS p. 30/4 Special case A = A H Q: What does the upper Hessenberg form become? Tridiagonal, costs 4n 3 /3 flops if A is real.

EIGENVALUE PROBLEMS p. 30/4 Special case A = A H Q: What does the upper Hessenberg form become? Tridiagonal, costs 4n 3 /3 flops if A is real. Q: Is it preserved in the QR algorithm?

EIGENVALUE PROBLEMS p. 30/4 Special case A = A H Q: What does the upper Hessenberg form become? Tridiagonal, costs 4n 3 /3 flops if A is real. Q: Is it preserved in the QR algorithm? Yes Q: What is the cost per QR step if A is real? 12n flops. It has cubic convergence and the eigenvectors are immediately available. Q H AQ D diagonal, so the eigenvectors are the columns of Q.

EIGENVALUE PROBLEMS p. 31/4 Numerical Difficulty with Shifting Form A µi with e.g. µ = a nn. Suppose 10 6 10 4 A = 10 4 1 10 2 10 2 10 16 fl(a 10 16 I) = It loses information unnecessarily. 10 16 10 4 0 10 4 10 16 10 2 0 10 2 0

EIGENVALUE PROBLEMS p. 32/4 Numerical Difficulty with Shifting After one QR step, 0 10 4 3.9 10 34 A 2 10 0 0 0 10 26 10 16 leading to the eignevalues 10 4, 10 4, 10 16. More accuare computed eignevluaes by MATLAB are: 9.9 10 7, 1, 10 16. We can use implicit shifts in order to avoid this loss.

EIGENVALUE PROBLEMS p. 33/4 Implicitly shifted QR alg. for real unsym A A real unsymmetric A usually has complex conjugate pairs of eigenvelues.

EIGENVALUE PROBLEMS p. 33/4 Implicitly shifted QR alg. for real unsym A A real unsymmetric A usually has complex conjugate pairs of eigenvelues. A complex µ in Q H (A µi) requires complex arithmetic. But for a real matrix, we would like to avoid complex arithmetic as much as possible. When the QR algorithm converges to a complex conjugate pair, we find the (n 1,n 2) entry converges to 0, and eventually we can deflate.

EIGENVALUE PROBLEMS p. 34/4 Implicitly shifted QR alg. for real unsym A When the (n 1,n 2) entry is small, the eigenvalues µ 1,µ 2 of the bottom right hand corner 2 2 block are good approximations to eigenvalues of A. If they are a complex conjugate pair, we could do one QR step with µ 1, the next with µ 2 = µ 1.

EIGENVALUE PROBLEMS p. 35/4 Implicitly shifted QR alg. for real unsym A Suppose A 1 is real upper Hessenberg. One step of double QR with explicit shifts µ 1 and µ 2 is A 1 µ 1 I = Q 1 R 1, A 2 = R 1 Q 1 + µ 1 I, A 2 µ 2 I = Q 2 R 2, A 3 = R 2 Q 2 + µ 2 I.

EIGENVALUE PROBLEMS p. 35/4 Implicitly shifted QR alg. for real unsym A Suppose A 1 is real upper Hessenberg. One step of double QR with explicit shifts µ 1 and µ 2 is Two drawbacks: A 1 µ 1 I = Q 1 R 1, A 2 = R 1 Q 1 + µ 1 I, A 2 µ 2 I = Q 2 R 2, A 3 = R 2 Q 2 + µ 2 I. Although A 1 is real, complex arithmetic is involved if µ 1 and µ 2 are complex. Explicit shifting may cause numerical difficulties.

EIGENVALUE PROBLEMS p. 36/4 Implicitly shifted QR alg. for real unsym A Q. Show that 1. A 3 = Q H 2 A 2 Q 2 = (Q 1 Q 2 ) H A 1 (Q 1 Q 2 ).

EIGENVALUE PROBLEMS p. 36/4 Implicitly shifted QR alg. for real unsym A Q. Show that 1. A 3 = Q H 2 A 2 Q 2 = (Q 1 Q 2 ) H A 1 (Q 1 Q 2 ). 2. N (A 1 µ 1 I)(A 1 µ 2 I) is real, N = Q 1 Q 2 R 2 R 1.

EIGENVALUE PROBLEMS p. 36/4 Implicitly shifted QR alg. for real unsym A Q. Show that 1. A 3 = Q H 2 A 2 Q 2 = (Q 1 Q 2 ) H A 1 (Q 1 Q 2 ). 2. N (A 1 µ 1 I)(A 1 µ 2 I) is real, N = Q 1 Q 2 R 2 R 1. 3. N ij = 0 for i j + 3, i.e., N has two nonzero subdiagonals. Since N is real, we can choose the Q-factor Q 1 Q 2 of its QR factorization to be real. So we can obtain the QR factorization of N to get Q 1 Q 2 and then use it to obtain A 3 avoiding complex arithmetic.

EIGENVALUE PROBLEMS p. 37/4 One step of double QR iteration We can avoid these two problems by the following algorithm.

EIGENVALUE PROBLEMS p. 37/4 One step of double QR iteration We can avoid these two problems by the following algorithm. Do the 1st step of Householder QR of N: N = Q 1 Q 2 R 2 R 1.

EIGENVALUE PROBLEMS p. 37/4 One step of double QR iteration We can avoid these two problems by the following algorithm. Do the 1st step of Householder QR of N: N = Q 1 Q 2 R 2 R 1. Compute n 1 = Ne 1 = [τ,σ,ν, 0,, 0] T.

One step of double QR iteration We can avoid these two problems by the following algorithm. Do the 1st step of Householder QR of N: N = Q 1 Q 2 R 2 R 1. Compute n 1 = Ne 1 = [τ,σ,ν, 0,, 0] T. Design a real Householder transformation H 0 such that H T 0 n 1 = 1... 1 0. 0 = 0. 0 EIGENVALUE PROBLEMS p. 37/4

EIGENVALUE PROBLEMS p. 38/4 One step of double QR iteration, ctd Apply H0 T & H 0 to A 1 from left and right, respectively: H0 T A 1 H 0 =

EIGENVALUE PROBLEMS p. 38/4 One step of double QR iteration, ctd Apply H0 T & H 0 to A 1 from left and right, respectively: H0 T A 1 H 0 = Then use real Householder transformations H 1,...,H n 2 to transform H T 0 A 1H 0 back to upper Hessenberg form: Ã 3 = H T n 2 H T 1 H T 0 A 1 H 0 H 1 H n 2

EIGENVALUE PROBLEMS p. 39/4 One step of double QR iteration, ctd Remember what we want is A 3. But Ã3 is essentially just A 3 due to the following result:

EIGENVALUE PROBLEMS p. 39/4 One step of double QR iteration, ctd Remember what we want is A 3. But Ã3 is essentially just A 3 due to the following result: The Implicit Q-Theorem. Suppose Q and P are two real orthogonal matrices such that Q T AQ and P T AP are both upper Hessenberg matrices, one of which is unreduced (i.e., all of its subdiagonal entries are nonzero). If Qe 1 = ±Pe 1, then and Qe j = ±Pe j, j = 2,...,n Q T AQ = D 1 P T APD, D = diag(±1,...,±1)

EIGENVALUE PROBLEMS p. 40/4 One step of double QR iteration, ctd Two Hessenberg reductions: A 3 = (Q 1 Q 2 ) T A 1 (Q 1 Q 2 ) Ã 3 = (H 0 H 1 H n 2 ) T A 1 (H 0 H 1 H n 2 )

EIGENVALUE PROBLEMS p. 40/4 One step of double QR iteration, ctd Two Hessenberg reductions: A 3 = (Q 1 Q 2 ) T A 1 (Q 1 Q 2 ) Ã 3 = (H 0 H 1 H n 2 ) T A 1 (H 0 H 1 H n 2 ) Q. Show (Q 1 Q 2 )e 1 = (H 0 H 1 H n 2 )e 1.

EIGENVALUE PROBLEMS p. 40/4 One step of double QR iteration, ctd Two Hessenberg reductions: A 3 = (Q 1 Q 2 ) T A 1 (Q 1 Q 2 ) Ã 3 = (H 0 H 1 H n 2 ) T A 1 (H 0 H 1 H n 2 ) Q. Show (Q 1 Q 2 )e 1 = (H 0 H 1 H n 2 )e 1. Q. Suppose A 1 is unreduced Hessenberg and µ 1 and µ 2 are not its eigenvalues. Show that A 3 is unreduced Hessenberg.

EIGENVALUE PROBLEMS p. 41/4 QR Algorithm Given A R n n and a tolerance tol greater than the unit roundoff, this algorithm computes the real Schur decomposition Q T AQ = R. If Q and R are desired, then R is stored in A. If only the eigenvalues are desired, then diagonal blocks in R are stored in the corresponding positions in A.

1. Compute the Hessenberg reduction A := Q T AQ where Q = H 1 H n 2. If the final Q is desired, form Q := H 1 H n 2. 2. until q = n Set to zero all subdiagonal entries that satisfy: a i,i 1 tol( a ii + a i 1,i 1 ). Find the largest non-negative q and the smallest non-negative p such that p n p q q A = A 11 A 12 A 13 p 0 A 22 A 23 n p q 0 0 A 33 q where A 33 is upper quasi-triangular and A 22 is unreduced. (Note: either p and q may be zero.) if q < n Perform one step of double QR iteration on A 22 : A 22 := Z T A 22 Z if Q is desired Q := Qdiag(I p, Z, I q ), A 12 := A 12 Z, A 23 := Z T A 23 end end end 3. Upper triangularize all 2-by-2 diagonal blocks in A that have real eigenvalues and accumulate the transformations if necessary.

EIGENVALUE PROBLEMS p. 42/4 Inverse iteration for eigenvectors Inverse Iteration: Given A C n n. Let 0 < λ j µ λ i µ (i j). Choose q 0 with q 0 2 = 1. for k = 1, 2,... Solve (A µi)z k = q k 1 q k = z k / z k 2 Stop if (A µi)q k 2 cu A 2. end

EIGENVALUE PROBLEMS p. 43/4 Inverse Iteration for eigenvectors, ctd 1. Solve (A µi)z k = q k 1 by LU with PP, & replace any pivot < u A by u A, this works even when A µi is singular. Ill-condition of A µi does not spoil computation. 2. When it stops µ and q k is an exact eigenpair for a nearby problem: (A + E k )q k = µq k, where E k r k q T k with r k (A µi)q k.

EIGENVALUE PROBLEMS p. 44/4 Convergence Analysis Assume A is nondefective, AX = Xdiag(λ i ).

EIGENVALUE PROBLEMS p. 44/4 Convergence Analysis Assume A is nondefective, AX = Xdiag(λ i ). Let q 0 = Xa = n i=1 a ix i, where we assume a j 0.

EIGENVALUE PROBLEMS p. 44/4 Convergence Analysis Assume A is nondefective, AX = Xdiag(λ i ). Let q 0 = Xa = n i=1 a ix i, where we assume a j 0. Thus n n (A µi) k q 0 = (A µi) k 1 a i x i = a i (λ i µ) kx i = i=1 1 [ a (λ j µ) k j x j + i j i=1 a i ( λj µ λ i µ Since λ j µ λ i µ, q k = (A µi) k q 0 ± x j (A µ) k as k, q 0 2 x j 2 ) kxi ].

EIGENVALUE PROBLEMS p. 45/4 Computing an eigenvector after QR alg 1. QR algorithm {λ i } 2. Apply the inverse iteration to Hessenberg A 1 = Q T 0 AQ 0, to find an eiegnvector y i of A 1 corresponding to λ i. Inverse iteration with A 1 is economical: solving (A 1 λ i I)z k = q k 1 costs O(n 2 ) flops. 3. Let x i = Q 0 y i. Then Ax i = λ i x i, x i 2 = 1, i.e. x i is a unit eigenvector of A corresponding to λ i.