Computation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated.

Transcription

1 Math 504, Homework 5 Computation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated 1 Find the eigenvalues and the associated eigenspaces for [ A = and B = For each one of A and B, indicate whether the matrix has an eigenvalue decomposition 2 Suppose A C n n has distinct eigenvalues Denote these distinct eigenvalues of A by λ 1, λ 2,, λ n and the associated eigenvectors by v (1), v (2),, v (n), respectively Since A has distinct eigenvalues, λ j λ k for all j, k such that j k Show that the set of eigenvectors {v (1), v (2),, v (n) } is linearly independent, that is the vector equation c 1 v (1) + c 2 v (2) + + c n v (n) = 0 (where c 1, c 2,, c n denote scalars) holds only for c 1 = c 2 = = c n = 0 [ Let v (1) and v (2) be two linearly independent eigenvectors of B = 3 5 Suppose also that {q (k) } denotes the sequence of vectors generated by the inverse iteration with shift µ = 2, and starting with an initial vector q 0 = α 1 v (1) + α 2 v (2) C 2 where α 1, α 2 are nonzero scalars Determine the subspace that span{q (k) } is approaching as k 4 Suppose that the power iteration is applied to a Hermitian matrix A C n n such that λ 1 = λ 2 > λ 3 where λ 1, λ 2, λ 3 denote the largest three eigenvalues of A in absolute value Would you expect the power iteration to converge in exact arithmetic? What happens to the vectors in the sequence in the limit, and how quickly? Explain 5 Implement a Matlab routine rayleigh iterm to compute an eigenvalue λ and an associated eigenvector v of an n n matrix A by Rayleigh iteration Your routine must return two output arguments, the computed eigenvalue λ and the associated eigenvector v It must take one input argument, the n n matrix A for which an eigenvalue and an eigenvector are sought You can choose a randomly generated vector (by typing the command randn(n,1)) as your initial estimate q (0) C n for the eigenvector Display the estimate for the eigenvalue at each iteration (by typing display(q *A*q) where q is the current estimate for the eigenvector with 2-norm equal to one) so that you can observe the rate of convergence You should terminate when the eigenvector estimates q (k) and q (k+1) at two consecutive iterations are sufficently close to each other, eg q (k) q (k+1) 10 15

2 Test your implementations with the following matrices: 1 0 1/10 B 1 = and B 2 = 1/ / / Run your routine to compute an eigenvalue and an associated eigenvector for B 1, B 2 Run it several times (eg six, seven times) Does it always converge to the same eigenvalue? What kind of rate of convergence do you observe in practice? 6 In class, it was shown that a unit vector q C n n that is an estimate for a unit eigenvector v of a given matrix A C n n associated with an eigenvalue λ satisfies Here, r(q) := q Aq is the Rayleigh quotient λ r(q) 2 A 2 v q 2 When A is Hermitian, better estimates can be deduced This is due to the orthogonality of the eigenvectors of A Denote the orthonormal set of eigenvectors of A with {v (1), v (2),, v (n) }, and the associated eigenvalues with λ 1, λ 2,, λ n The question concerns, given an estimate q C n for the eigenvector v = v (1), how good of an estimate r(q) is for the associated eigenvalue λ = λ 1 (a) The vector q can be expanded as q = c 1 v (1) + c 2 v (2) + + c n v (n) for scalars c 1,, c n Show that n k=1 c k 2 = q 2 2 = 1 (b) Show that n k=2 c k 2 v (1) q 2 2 (c) Derive an expression for the Rayleigh quotient r(q) = q Aq in terms of the coefficients c j and the eigenvalues λ j (d) Show that λ 1 r(q) κ v (1) q 2 2, where κ := max k=2,,n λ 1 λ k starting with λ 1 = n k=1 λ 1 c k 2 7 The QR algorithm is the standard approach to compute the eigenvalues of a dense matrix A C n n Below pseudocodes are provided for the QR algorithm without and with shifts The shifted version is employed in practice (but implicitly), since appropriately chosen shifts speed-up the convergence Algorithm 1 The QR Algorithm without Shifts for k = 0, 1, do Compute a QR factorization A k = Q k+1 R k+1 A k+1 R k+1 Q k+1 (a) Apply one iteration of the QR algorithm without shifts to the matrix A = [

3 Algorithm 2 The QR Algorithm with Shifts for k = 0, 1, do Choose a shift µ k Compute a QR factorization A k µ k I = Q k+1 R k+1 A k+1 R k+1 Q k+1 + µ k I (b) Apply one iteration of the QR algorithm to the matrix A given in (a) with the shift µ = 6 8 In class, it was shown that the normalized simultaneous iteration is equivalent to the QR algorithm In this question you are expected to establish the equivalence of the QR algorithm and unnormalized simultaneous iteration A pseudocode for the QR Algorithm without shifts (Algorithm 1) is provided in the previous question A pseudocode for the unnormalized simultaneous iteration is given below Algorithm 3 Unnormalized Simultaneous Iteration for k = 1,, m do Compute a QR factorization A k = ˆQ k ˆRk ˆΛ k ˆQ k A ˆQ k Show that a QR factorization for A k is given by Here, Q k, R k are as defined in Algorithm 1 A k = Q 1 Q 2 Q }{{ k R } k R 2 R }{{} 1 ˆQ k ˆR k 9 Let A C n n be a Hermitian matrix, µ R be a given shift, and r < n be a given positive integer Furthermore, let v (j) denote an eigenvector of A associated with an eigenvalue that is jth closest to µ among all eigenvalues of A Write down a pseudocode that generates a sequence of matrices {P k } in C n r such that, under mild conditions, Col(P k (:, j)) span{v j } as k for j = 1,, r State precisely the mild conditions that ensure this convergence 10 Let A C m n, and B = [ 0 A A 0 (a) Show that if σ is a singular value of A, then σ and σ are the eigenvalues of B (b) Show that if λ is an eigenvalue of B, then λ is a singular value of A Write down also the left and right singular vectors of A corresponding to the singular value λ in terms of the eigenvector v of B associated with the eigenvalue λ

4 11 To compute the singular values of a given matrix A C m n with m n, the first stage is the Golub-Kahan bidiagonalization, which constructs unitary matrices U C m m and V C n n such that UAV is bidiagonal (that is all entries of B = UAV are zero excluding its diagonal entries b kk and superdiagonal entries b k(k+1) ) Specifically, at the kth step of this bidiagonalization procedure, a unitary matrix U k is applied from left to make the entries on the k column and rows k +1 : m zero, and a unitary matrix V k is applied from right to introduce zeros on the kth row and columns k + 2 : n The unitary matrices U and V are given by U = U n 1 U 1 and V = V 1 V m 2 Write down a pseudocode for Golub-Kahan bidiagonalization making sure that the number of flops required is O(mn 2 ) Do not form the unitary matrices U and V 12 Algorithm 4 The QR Algorithm for Singular Values for k = 0, 1, do Choose a shift µ k Compute a QR factorization A k A k µ k I = Q k+1 R k+1 Compute a QR factorization A k A k µ ki = P k+1 S k+1 A k+1 P k+1 A kq k+1 Implement the QR algorithm to compute singular values, based on the pseudocode above, in a naive fashion (ie, without deflations, form A k A k and A k A k explicitly) 13 A pseudocode for the QR Algorithm for singular value computation is given above in question 12 If the matrix A is initially bidiagonal (eg, via Golub-Kahan bidiagonalization), then the sequence {A k } remains bidiagonal, which you are expected to show in this question This reduces the cost of each iteration of the QR algorithm significantly, indeed linear with respect to m and n (a) Exploiting the identities (A k A k µ ki)a k = A k (A k A k µ k I) and (A k A k µ k I)A k = A k (A ka k µ ki), show that A k+1 = S k+1 A k R 1 k+1 and A k+1 = R k+1a k S 1 k+1 (b) Show that if A k is upper triangular, then so is A k+1 (c) Show that if A k is Hessenberg, then so is A k+1 Parts (b)-(c) above combined imply that if A k is bidiagonal, then so is A k+1 14 Let A C m m be a matrix whose eigenvalues are sought Recall the recurrence for the Arnoldi iteration where AQ n = Q n+1 Hn (1) Q n = [ q (1) q (2) q (n) C m n, m n

5 has orthonormal columns that span the Krylov subspace K n := span{b, Ab,, A n 1 b}, (2) and H n C (n+1) n is a Hessenberg matrix The eigenvalues of H n = Q naq n C n n are supposedly good estimates for the extreme eigenvalues of A, and called the Ritz values Suppose that at the nth Arnoldi iteration the Hessenberg matrix H n is such that h (n+1)n = 0 (a) Simplify the Arnoldi iteration (1) (b) Show that the Krylov subspace K n is an invariant subspace of A, ie, AK n K n (c) Show that K n = K j for all j > n (d) Show that each eigenvalue of H n is an eigenvalue of A 15 Let us consider the Krylov subspace K n once again defined as in (2) for a given matrix A C m m and a vector b C m Throughout this question, assume dim (K n ) = n In this case the Arnoldi iteration generates an orthonormal basis Q n := {q (1),, q (n) } for K n Especially, letting Q n = [ q (1) q (2) q (n), the eigenvalues of the matrix H n = Q naq n capture some of the eigenvalues of A well in certain occasions To generate the orthonormal basis Q n and the matrix H n, we rely on the recurrence (1), and the fact that H n = H n (1 : n, 1 : n) (a) Suppose A C m m is Hermitian Simplify the recurrence (1) (b) Write down an efficient pseudocode to generate Q n+1 and H n for an Hermitian matrix A C m m based on the simplified recurrence in part (a)