SEC POWER METHOD Power Method

Transcription

1 SEC..2 POWER METHOD Power Method We now describe the power method for computing the dominant eigenpair. Its extension to the inverse power method is practical for finding any eigenvalue provided that a good initial approximation is known. Some schemes for finding eigenvalues use other methods that converge fast, but have limited precision. The inverse power method is then invoked to refine the numerical values and gain full precision. To discuss the situation, we will need the following definitions. Definition.0. If λ is an eigenvalue of A that is larger in absolute value than any other eigenvalue, it is called the dominant eigenvalue. An eigenvector V corresponding to λ is called a dominant eigenvector.

2 600 CHAP. EIGENVALUES AND EIGENVECTORS Definition.. An eigenvector V is said to be normalized if the coordinate of largest magnitude is equal to unity (i.e., the largest coordinate in the vector V is the number ). It is easy to normalize an eigenvector [ v v 2 v n ] by forming a new vector V = (/c) [ v v 2 v n ], where c = v j and v j =max i n { v i }. Suppose that the matrix A has a dominant eigenvalue λ and that there is a unique normalized eigenvector V that corresponds to λ. This eigenpair λ, V can be found by the following iterative procedure called the power method. Start with the vector () X 0 = [ ]. Generate the sequence {X k } recursively, using (2) Y k = AX k, X k+ = c k+ Y k, where c k+ is the coordinate of Y k of largest magnitude (in the case of a tie, choose the coordinate that comes first). The sequences {X k } and {c k } will converge to V and λ, respectively: (3) lim k X k = V and lim k c k = λ. Remark. If X 0 is an eigenvector and X 0 = V, then some other starting vector must be chosen. Example.5. for the matrix Use the power method to find the dominant eigenvalue and eigenvector 0 5 A = Start with X 0 = [ ] and use the formulas in (2) to generate the sequence of vectors {X k } and constants {c k }.Thefirst iteration produces = 8 = 2 2 = c 3 X The second iteration produces = = = c 2 X 2.

3 SEC..2 POWER METHOD 60 Table. Power Method Used in Example.5 to Find the Normalized Dominant Eigenvector V = [ ] and Corresponding Eigenvalue λ = 4 AX k = Y k = c k+ X k+ AX 0 =[ ] = [ ] = c X AX =[ ] = [ ] = c 2 X 2 AX 2 =[ ] = [ ] = c 3 X 3 AX 3 =[ ] = [ ] = c 4 X 4 AX 4 =[ ] = [ ] = c 5 X 5 AX 5 =[ ] = [ ] = c 6 X 6 AX 6 =[ ] = [ ] = c 7 X 7 AX 7 =[ ] = [ ] = c 8 X 8 AX 8 =[ ] = [ ] = c 9 X 9 AX 9 =[ ] = [ ] = c 0 X 0 AX 0 =[ ] = [ ] = c X Iteration generates the sequence {X k } (where X k is a normalized vector): , 6 6, 9 2, , , , The sequence of vectors converges to V = [ ], and the sequence of constants converges to λ = 4 (see Table.). It can be proved that the rate of convergence is linear. Theorem.8 (Power Method). Assume that the n n matrix A has n distinct eigenvalues λ, λ 2,..., λ n and that they are ordered in decreasing magnitude; that is, (4) λ > λ 2 λ 3 λ n. If X 0 is chosen appropriately, then the sequences { X k = [ x (k) {c k } generated recursively by (5) Y k = AX k and (6) X k+ = Y k, c k+ where x (k) (7) c k+ = x (k) j and x (k) j = max i n { x(k) i }, 2... x n (k) ] } and

4 602 CHAP. EIGENVALUES AND EIGENVECTORS will converge to the dominant eigenvector V and eigenvalue λ, respectively. That is, (8) lim X k = V and lim c k = λ. k k Proof. Since A has n eigenvalues, there are n corresponding eigenvectors V j,for j =, 2,..., n, that are linearly independent, normalized, and form a basis for n- dimensional space. Hence the starting vector X 0 can be expressed as the linear combination (9) X 0 = b V + b 2 V 2 + +b n V n. Assume that X 0 = [ ] x x 2... x n was chosen in such a manner that b = 0. Also, assume that the coordinates of X 0 are scaled so that max j n { x j } =. Because {V j } n j= are eigenvectors of A, the multiplication AX 0, followed by normalization, produces Y 0 = AX 0 = A(b V + b 2 V 2 + +b n V n ) = b AV + b 2 AV 2 + +b n AV n (0) = b λ V + b 2 λ 2 V 2 + +b n λ n V n ( ( ) ( ) ) = λ b V + b 2 V 2 + +b n V n λ λ and X = λ ( ( ) ( ) ) b V + b 2 V 2 + +b n V n. c λ λ After k iterations we arrive at () Y k = AX k ( λ k ( ) k ( ) ) k = A b V + b 2 V 2 + +b n V n c c 2 c k λ λ ( λ k ( ) k ( ) ) k = b AV + b 2 AV 2 + +b n AV n c c 2 c k λ λ ( λ k ( ) k ( ) ) k = b λ V + b 2 λ 2 V 2 + +b n λ n V n c c 2 c k λ λ ( λ k ( ) k ( ) ) k = b V + b 2 V 2 + +b n V n c c 2 c k λ λ and X k = λ k c c 2 c k ( ( ) k ( ) ) k b V + b 2 V 2 + +b n V n. λ λ

5 SEC..2 POWER METHOD 603 Since we assumed that λ j / λ < for each j = 2, 3,..., n, wehave (2) lim k b j Hence it follows that ( ) λ k j V j = 0 for each j = 2, 3,..., n. λ (3) lim X b λ k k = lim V. k k c c 2 c k We have required that both X k and V be normalized and their largest component be. Hence the limiting vector on the left side of (3) will be normalized, with its largest component being. Consequently, the limit of the scalar multiple of V on the right side of (3) exists and its value must be ; that is, b λ k (4) lim =. k c c 2 c k Therefore, the sequence of vectors {X k } converges to the dominant eigenvector: (5) lim k X k = V. Replacing k with k in the terms of the sequence in (4) yields lim k b λ k c c 2 c k =, and dividing both sides of this result into (4) yields λ lim = lim k c k k b λ k /(c c 2 c k ) b λ k /(c c 2 c k ) = =. Therefore, the sequence of constants {c k } converges to the dominant eigenvalue: (6) lim k c k = λ, and the proof of the theorem is complete. Speed of Convergence In the light of equation (2) we see that the coefficient of V j in X k goes to zero in proportion to (λ j /λ ) k and that the speed of convergence of {X k } to V is governed by the terms (λ 2 /λ ) k. Consequently, the rate of convergence is linear. Similarly, the

6 604 CHAP. EIGENVALUES AND EIGENVECTORS Table.2 Comparison of the Rate of Convergence of the Power Method and Acceleration of the Power Method Using Aitken s 2 Technique c k Y k ĉ k X k c X = [ ] ; [ ] = ĉ X c 2 X 2 = [ ] ; [ ] = ĉ 2 X 2 c 3 X 3 = [ ] ; [ ] = ĉ 3 X 3 c 4 X 4 = [ ] ; [ ] = ĉ 4 X 4 c 5 X 5 = [ ] ; [ ] = ĉ 5 X 5 c 6 X 6 = [ ] ; [ ] = ĉ 6 X 6 c 7 X 7 = [ ] ; [ ] = ĉ 7 X 7 c 8 X 8 = [ ] ; [ ] = ĉ 8 X 8 c 9 X 9 = [ ] ; [ ] = ĉ 9 X 9 c 0 X 0 = [ ] ; [ ] = ĉ 0 X 0 convergence of the sequence of constants {c k } to λ is linear. The Aitken 2 method can be used for any linearly convergent sequence {p k } to form a new sequence, { } (p k+ p k ) 2 p k =, p k+2 2p k+ + p k that converges faster. In Example.4 this Aitken 2 method can be applied to speed up convergence of the sequence of constants {c k }, as well as the first two components of the sequence of vectors {X k }. A comparison of the results obtained with this technique and the original sequences is shown in Table.2. Shifted-Inverse Power Method We will now discuss the shifted inverse power method. It requires a good starting approximation for an eigenvalue, and then iteration is used to obtain a precise solution. Other procedures such as the QM and Givens method are used first to obtain the starting approximations. Cases involving complex eigenvalues, multiple eigenvalues, or the presence of two eigenvalues with the same magnitude or approximately the same magnitude will cause computational difficulties and require more advanced methods. Our illustrations will focus on the case where the eigenvalues are distinct. The shifted inverse power method is based on the following three results (the proofs are left as exercises). Theorem.9 (Shifting Eigenvalues). Suppose that λ, V is an eigenpair of A. If α is any constant, then λ α, V is an eigenpair of the matrix A α I. Theorem.20 (Inverse Eigenvalues). Suppose that λ, V is an eigenpair of A. If λ = 0, then /λ, V is an eigenpair of the matrix A.

7 SEC..2 POWER METHOD 605 x λ λ j λ j α λ j + λ n Figure.2 The location of α for the shifted-inverse power method. Theorem.2. Suppose that λ, V is an eigenpair of A. Ifα = λ, then /(λ α), V is an eigenpair of the matrix (A α I). Theorem.22 (Shifted-Inverse Power Method). Assume that the n n matrix A has distinct eigenvalues λ, λ 2,..., λ n and consider the eigenvalue λ j. Then a constant α can be chosen so that µ = /(λ j α) is the dominant eigenvalue of (A α I). Furthermore, if X 0 is chosen appropriately, then the sequences { X k = [ (k) x x (k) 2... x n (k) ] } and {ck } are generated recursively by (7) Y k = (A α I) X k and (8) X k+ = Y k, c k+ where (9) c k+ = x (k) j and x (k) j = max j n { x(k) i } will converge to the dominant eigenpair µ, V j of the matrix (A α I). Finally, the corresponding eigenvalue for the matrix A is given by the calculation (20) λ j = µ + α. Remark. For practical implementations of Theorem.22, a linear system solver is used to compute Y k in each step by solving the linear system (A α I)Y k = X k. Proof. Without loss of generality, we may assume that λ <λ 2 < <λ n. Select a number α (α = λ j ) that is closer to λ j than any of the other eigenvalues (see Figure.2), that is, (2) λ j α < λ i α for each i =, 2,..., j, j +,..., n. According to Theorem.2, /(λ j α), V is an eigenpair of the matrix (A α I). Relation (2) implies that / λ i α < / λ j α for each i = j so that µ = /(λ j α) is the dominant eigenvalue of the matrix (A α I). The shifted-inverse power method uses a modification of the power method to determine the eigenpair µ, V j. Then the calculation λ j = /µ + α produces the desired eigenvalue of the matrix A.

8 606 CHAP. EIGENVALUES AND EIGENVECTORS Table.3 Shifted-Inverse Power Method for the Matrix (A 4.2I) in Example.6: Convergence to the Eigenvector V = [ ] and µ = 5 (A α I) X k = c k+ X k+ (A α I) X 0 = [ ] = c X (A α I) X = [ ] = c 2 X 2 (A α I) X 2 = [ ] = c 3 X 3 (A α I) X 3 = [ ] = c 4 X 4 (A α I) X 4 = [ ] = c 5 X 5 (A α I) X 5 = [ ] = c 6 X 6 (A α I) X 6 = [ ] = c 7 X 7 (A α I) X 7 = [ ] = c 8 X 8 (A α I) X 8 = [ ] = c 9 X 9 Example.6. matrix Employ the shifted-inverse power method to find the eigenpairs of the 0 5 A = Use the fact that the eigenvalues of A are λ = 4, λ 2 = 2, and λ 3 =, and select an appropriate α and starting vector for each case. Case (i): For the eigenvalue λ = 4, we select α = 4.2 and the starting vector X 0 = [ ]. First, form the matrix A 4.2I, compute the solution to Y 0 = X 0 =, and get the vector Y 0 = [ ]. Then compute c = and X = [ ]. Iteration generates the values given in Table.3. The sequence {c k } converges to µ = 5, which is the dominant eigenvalue of (A 4.2I), and {X k } converges to V = [ ]. The eigenvalue λ of A is given by the computation λ = /µ +α = /( 5)+4.2 = = 4. Case (ii): For the eigenvalue λ 2 = 2, we select α = 2. and the starting vector X 0 = [ ]. Form the matrix A 2.I, compute the solution to Y 0 = X 0 =, and obtain the vector Y 0 = [ ]. Then c = and vector X = [ ]. Iteration produces the

9 SEC..2 POWER METHOD 607 Table.4 Shifted-Inverse Power Method for the Matrix (A 2.I) in Example.6: Convergence to the Dominant Eigenvector V = [ 4 2 ] and µ = 0 (A α I) X k = c k+ X k+ (A α I) X 0 = [ ] = c X (A α I) X = [ ] = c 2 X 2 (A α I) X 2 = [ ] = c 3 X 3 (A α I) X 3 = [ ] = c 4 X 4 (A α I) X 4 = [ ] = c 5 X 5 (A α I) X 5 = [ ] = c 6 X 6 (A α I) X 6 = [ ] = c 7 X 7 Table.5 Shifted-Inverse Power Method for the Matrix (A 0.875I) in Example.6: Convergence to the Dominant Eigenvector V = [ 2 2 ] and µ = 8 (A α I) X k = c k+ X k+ (A α I) X 0 = [ ] = c X (A α I) X = [ ] = c 2 X 2 (A α I) X 2 = [ ] = c 3 X 3 (A α I) X 3 = [ ] = c 4 X 4 (A α I) X 4 = [ ] = c 5 X 5 (A α I) X 5 = [ ] = c 6 X 6 (A α I) X 6 = [ ] = c 7 X 7 values given in Table.4. The dominant eigenvalue of (A 2.I) is µ = 0, and the eigenpair of the matrix A is λ 2 = /( 0) + 2. = = 2 and V 2 = [ 4 2 ]. Case (iii): For the eigenvalue λ 3 =, we select α = and the starting vector X 0 = [ 0 ]. Iteration produces the values given in Table.5. The dominant eigenvalue of (A 0.875I) is µ = 8, and the eigenpair of matrix A is λ 3 = / = = and V 3 = [ 2 2 ]. The sequence {Xk } of vectors with the starting vector [ 0 ] converged in seven iterations. (Computational difficulties were encountered when X 0 = [ ] was used, and convergence took significantly longer.)

10 Numerical Methods Using Matlab, 4 th Edition, 2004 John H. Mathews and Kurtis K. Fink ISBN: Prentice-Hall Inc. Upper Saddle River, New Jersey, USA