13-2 Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2

Transcription

1 The QR algorithm The most common method for solving small (dense) eigenvalue problems. The basic algorithm: QR without shifts 1. Until Convergence Do: 2. Compute the QR factorization A = QR 3. Set A := RQ 4. EndDo Note: A new = RQ = Q H (QR)Q = Q H AQ A new is similar to A throughout the algorithm. Above basic algorithm is never used in practice. Two variations: (1) Use shift of origin and (2) Start by transforming A into an Hessenberg matrix Until Convergence means Until A becomes close enough to an upper triangular matrix 13-1 Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2 Practical QR algorithms: Shifts of origin Observation: (from theory): Last row converges fastest. Convergence is dictated by λ n λ n 1 We will now consider only the real symmetric case. Eigenvalues are real. A (k) remains symmetric throughout process. As k goes to infinity the last column and row (except a (k) nn ) converge to zero quickly.,, and a (k) nn converges to lowest eigenvalue. A (k) = a a a a a a Idea: Apply QR algorithm to A (k) µi with µ = a (k) nn. Note: eigenvalues of A (k) µi are shifted by µ, and eigenvectors are the same Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2

2 QR with shifts 1. Until row a in, 1 i < n converges to zero DO: 2. Obtain next shift (e.g. µ = a nn ) 3. A µi = QR 5. Set A := RQ + µi 6. EndDo Convergence (of last row) is cubic at the limit! [for symmetric case] Result of algorithm: A (k) = λ n Next step: deflate, i.e., apply above algorithm to (n 1) (n 1) upper triangular matrix Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2 Practical algorithm: Use the Hessenberg Form Recall: Upper Hessenberg matrix is such that a ij = 0 for j < i 1 Observation: The QR algorithm preserves Hessenberg form (tridiagonal form in symmetric case). Results in substantial savings. Transformation to Hessenberg form Want H 1 AH T 1 = H 1AH 1 to have the form shown on the right Consider the first step only on a 6 6 matrix Choose a w in H 1 = I 2ww T to make the first column have zeros from position 2 to n. So w 1 = 0. Apply to left: B = H 1 A Apply to right: A 1 = BH 1. Main observation: the Householder matrix H 1 which transforms the column A(2 : n, 1) into e 1 works only on rows 2 to n. When applying the transpose H 1 to the right of B = H 1 A, we observe that only columns 2 to n will be altered. So the first column will retain the desired pattern (zeros below row 2). Algorithm continues the same way for columns 2,...,n Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2

3 QR for Hessenberg matrices Need the Implicit Q theorem Suppose that Q T AQ is an unreduced upper Hessenberg matrix. Then columns 2 to n of Q are determined uniquely (up to signs) by the first column of Q. In other words if V T AV = G and Q T AQ = H are both Hessenberg and V (:, 1) = Q(:, 1) then V (:, i) = ±Q(:, i) for i = 2 : n. Implication: To compute A i+1 = Q T i AQ i we can: Compute 1st column of Q i [== scalar A(:, 1)] Choose other columns so Q i = unitary, and A i+1 = Hessenberg Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2 Wll do this with Givens rotations A = Example: With n = 6 : Choose G 1 = G(1, 2, θ 1 ) so that (G T 1 A 0) 21 = 0 A 1 = G T 1 AG 1 = Choose G 2 = G(2, 3, θ 2 ) so that (G T 2 A 1) 31 = 0 A 2 = G T 2 A 1G 2 = Choose G 3 = G(3, 4, θ 3 ) so that (G T 3 A 2) 42 = 0 A 3 = G T 3 A 2G 3 = Choose G 4 = G(4, 5, θ 4 ) so that (G T 4 A 3) 53 = 0 A 4 = G T 4 A 3G 4 = 0 0 Process known as Bulge chasing Similar idea for the symmetric (tridiagonal) case Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2

4 The symmetric eigenvalue problem: Basic facts Consider the Schur form of a real symmetric matrix A: A = QRQ H Since A H = A then R = R H Eigenvalues of A are real and There is an orthonormal basis of eigenvectors of A In addition, Q can be taken to be real when A is real. (A λi)(u + iv) = 0 (A λi)u = 0 & (A λi)v = 0 Can select eigenvector to be either u or v Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2 The min-max theorem (Courant-Fischer) Label eigenvalues decreasingly: λ 1 λ 2 λ n The eigenvalues of a Hermitian matrix A are characterized by the relation λ k = max min S, dim(s)=k x S,x 0 (x, x) Proof: Preparation: Since A is symmetric real (or Hermitian complex) there is an orthonormal basis of eigenvectors u 1, u 2,, u n. Express any vector x in this basis as x = n i=1 α iu i. Then : /(x, x) = [ λ i α i 2 ]/[ α i 2 ]. (a) Let S be any subspace of dimension k and let W = span{u k, u k+1,, u n }. A dimension argument (used befire) shows that S W {0}. So there is a non-zero x w in S W. Express this x w in the eigenbasis as x w = n i=k α iu i. Then since λ i λ k for i k we have: (Ax w, x w ) n (x w, x w ) = i=k λ i α i 2 n i=k α λ i 2 k So for any subspace S we have min x S,x 0 /(x, x) λ k. (b) We now take S = span{u 1, u 2,, u k }. Since λ i λ k for i k: for this particular subspace we have: min x S (x, x) = min x S k i=1 λ i α i 2 n i=k α = λ i 2 k (c) The results of (a) and (b) imply that the max over all subspaces S of dim. k of min x S,x 0 /(x, x) is equal to λ k Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2 Consequences: λ 1 = max x 0 (x, x) λ n = min x 0 (x, x) Actually 4 versions of the same theorem. 2nd version: λ k = min S, dim(s)=n k+1 max x S,x 0 (x, x) Other 2 versions come from ordering eigenvalues increasinly instead of decreasingly. Write down all 4 versions of the theorem Use the min-max theorem to show that A 2 = σ 1 (A) - the largest singular value of A Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2

5 Interlacing Theorem: Denote the k k principal submatrix of A as A k, with eigenvalues {λ [k] i } k i=1. Then λ [k] 1 λ [k 1] 1 λ [k] 2 λ [k 1] 2 λ [k 1] k 1 λ[k] k Many uses. For example: interlacing theorem for roots of orthogonal polynomials The Law of inertia Inertia of a matrix = [m, z, p] with m = number of < 0 eigenvalues, z = number of zero eigenvalues, and p = number of > 0 eigenvalues. Sylvester s Law of inertia: then A and X T AX have the same inertia. If X is an n n nonsingular matrix, Suppose that A = LDL T where L is unit lower triangular, and D diagonal. How many negative eigenvalues does A have? Assume that A is tridiagonal. How many operations are required to determine the number of negative eigenvalues of A? Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2 Devise an algorithm based on the inertia theorem to compute the i-th eigenvalue of a tridiagonal matrix. What is the inertia of the matrix ( ) I F F T 0 where F is m n, with n < m, and of full rank? [Hint: use a block LU factorization] Bisection algorithm for tridiagonal matrices: Goal: to compute i-th eigenvalue of A (tridiagonal) Get interval [a, b] containing spectrum [Gershgorin]: Let σ = (a + b)/2 = middle of interval a λ n λ 1 b Calculate p = number of positive eigenvalues of A σi If p i then λ i (σ, b] set a := σ a λ λ σ n n 1 λ i λ 1 b Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2 Else then λ i [a, σ] set b := σ Repeat until b a is small enough Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2

6 The QR algorithm for symmetric matrices Most important method used : reduce to tridiagonal form and apply the QR algorithm with shifts. Householder transformation to Hessenberg form yields a tridiagonal matrix because HAH T = A 1 is symmetric and also of Hessenberg form it is tridiagonal symmetric. Tridiagonal form preserved by QR similarity transformation Practical method How to implement the QR algorithm with shifts? It is best to use Givens rotations can do a shifted QR step without explicitly shifting the matrix.. Two most popular shifts: s = a nn and s = smallest e.v. of A(n 1 : n, n 1 : n) Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2 Jacobi iteration - Symmetric matrices Main idea: Rotation matrices of the form c s 0 J(p, q, θ) = s c c = cos θ and s = sin θ are so that J(p, q, θ) T AJ(p, q, θ) has a zero in position (p, q) (and also (q, p)) Frobenius norm of matrix is preserved but diagonal elements become larger convergence to a diagonal. p q Let B = J T AJ (where J J p,q,θ ). Look at 2 2 matrix B([p, q], [p, q]) (matlab notation) Keep in mind that a pq = a qp and b pq = b qp ( ) bpp b pq b qp b qq = = ( ) ( ) ( ) c s app a pq c s s c a qp a qq s c ( c s ) [ ] capp sa pq sa pp + ca pq ca qp sa qq sa pq + ca qq s c = [ ] c 2 a pp + s 2 a qq 2sc a pq (c 2 s 2 )a pq sc(a qq a pp ) c 2 a qq + s 2 a pp + 2sc a pq Want: (c 2 s 2 )a pq sc(a qq a pp ) = Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2

7 c 2 s 2 2sc Letting t = s/c (= tan θ) t = τ ± 1 + τ 2 = = a qq a pp 2a pq t 2 + 2τ t 1 = 0 1 τ ± 1+τ 2 τ quad. equation Select sign to get a smaller t so θ π/4. Then : c = t 2 ; s = c t Implemented in matlab script jacrot(a,p,q) see matlab webpage of class Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2 Define: A O = A Diag(A) A with its diagonal entries replaced by zeros Observations: (1) Unitary transformations preserve. F. (2) Only changes are in rows and columns p and q. Let B = J T AJ (where J J p,q,θ ). Then, a 2 pp + a2 qq + 2a2 pq = b2 pp + b2 qq + 2b2 pq = b2 pp + b2 qq because b pq = 0. Then, a little calculation leads to: B O 2 F = B 2 F b 2 ii = A 2 F b 2 ii = A 2 F a 2 ii + a 2 ii b 2 ii = A O 2 F + (a2 pp + a2 qq b2 pp b2 qq ) = A O 2 F 2a2 pq Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2 A O F will decrease from one step to the next. Let A O I = max i j a ij. Show that A O F n(n 1) A O I Use this to show convergence in the case when largest entry is zeroed at each step Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2