Lecture Eigenvalue Problem Review of Eigenvalues Some properties Power method Shift method Inverse power method Deflation QR Method
Eigenvalue Eigenvalue ( A I) If det( A I) (trivial solution) To obtain a non-trivial solution, a a L a n a a L a a M n a M n O L n Characteristic equation a a nn det ( AI)
Properties of Eigenvalue ) trace A tr( A) n i a ii n i i ) det n AΠ i i ) If A is symmetric, then the eigenvectors are orthogonal:, G 4) Let the eigenvalues of T i i j then, the eigenvalues of (A - ai) a a,,, j ii i j A,, L, L n a n
Application Eamples If a matri A represents a system, then eigenvalues can be used to system identification, e.g., stability and so on. For infinite -dimension systems, eigenvalueeigenfunction pairs are used, e.g., resonance frequency of an oscillator or cavity, waveguide modes, etc. In image and speech recognition, eigenface, eigenvoice are used for identification. In statistics, principal component analysis (PCA) is used to find relationships among variables. 4
Geometrical Interpretation of Eigenvectors Transformation A A :The transformation of an eigenvector is mapped onto the same line of. Symmetric matri orthogonal eigenvectors Relation to Singular Value if A is singular {eigenvalues} 5
Power Method Any vector can be written as where α m is a constant and φ m denotes an eigenvector. In general, continue the multiplication: A n m α α φ + α φ + α φ + K+ α m φ m where denotes an eigenvalue, and A AA 44 K A n nφ n 6
Power Method Factor the large value term As you continue to multiply the vector by [A] + + + n φ φ φ n n α α α K A as αφ A 7
Power Method The basic computation of the power method is summarized as as φ φ φ α φ α φ α φ α A A u 8
Power Method The basic computation of the power method is summarized as u Au Au - - and lim u φ The equation can be written as: Au u - - Au u - - 9
The Power Method Algorithm ynonzero random vector Initialize A*y vector for,, n y/ Ay ( is the approimate eigenvector) approimate eigenvalue µ (y T )/(y T y) rµy- +
Eample of Power Method Consider the follow matri A A 4 Assume an arbitrary vector { } T
Eample of Power Method Multiply the matri by the matri [A] by 5 4 Normalize the result of the product..6 5 5
Eample of Power Method 45..478 4.74.45.7 4.45.7 4.6. 4.6. 4.6..6 4..4 4.74.45.478 4.74
Eample of Power Method..65 4.4..4 4.5.56 4.4..65 4.4 As you continue to multiply each successive vector yields 4 and the vector u { } T 4
Eample of Power Method Consider the follow matri A 6 6 6 A Assume an arbitrary vector { } T 5 ;.857.574 4 4 8 6 6 6 ;.87.56.857.857.857 6.857.857.574 6 6 6
Eample of Power Method 6 ;.84.59.465.465.465 6.465.87.56 6 6 6 ;.84.5.77.77.77 6.77.84.59 6 6 6 ;.84.5... 6..84.5 6 6 6 Result: and u {.5.8} T
Power method : Advantages Simple, easy to implement. The eigenvector corresponding to the dominant (i.e., maimum) eigenvalue is generated at the same time. The inverse power method solves for the minimal eigenvalue/eigenvector pair. 7
Power Method : Disadvantages The power method provides only one eigenvalue/eigenvector pair, which corresponds to the maimum eigenvalue. Some modifications must be implemented to find other eigenvalues, e.g., shift method, deflation, etc. Also, if the dominant eigenvalue is not sufficiently larger than others, a large number of iterations are required. 8
Shift method It is possible to obtain another eigenvalue from the set equations by using a technique nown as shifting the matri. [ A] Subtracting a vector s from each side, thereby changing the maimum eigenvalue [ A] s[ I] ( s) 9
Shift method Let the eigenvalue, ma, be the maimum value of the matri A, Then the matri is rewritten in a form: [ B] [ A] [ I] ma The Power method can then be applied to obtain the largest eigenvalue of [B].
Eample of Power Method Consider the following matri B 5 4 4 B Assume an arbitrary vector { } T
Eample of Power Method Multiply by the matri [B] 5 5 Normalize the result of the product.6. -5 5
Eample of Power Method..4 5 5.6. 5.6... 5 Continue with the iteration and the final value is -5. However, to get the true you need to shift bac by: 4 5 ma + +
Inverse Power Method The inverse method is similar to the power method, ecept that it finds the smallest eigenvalue. Using the following technique. [ A] [ A] [ A] [ A] [ A] µ [ B] 4
Inverse Power Method The algorithm is the same as the Power method and the eigenvector is not the eigenvector for the smallest eigenvalue. To obtain the smallest eigenvalue from the power method. µ µ 5
Inverse Power Method The inverse algorithm use the technique avoids calculating the inverse matri and uses a LU decomposition to find the vector. [ A] [ L ][ U] Eample [ A] 4 5 4.964 5.55.8 6
Matri Deflation It is possible to obtain eigenvectors one after another Properly assigning the vector is important. e.g. Wielandt s deflation 7
Eample Using Deflation(I) 8
Eample: Using Deflation(II) 9
Wielandt s Deflation Let, be the eigenvector, eigenvalue of A. Consider Clearly, is an eigenvalue of A, since Assume j, j,j,, then ; u u A A T T u A A T j j j j j T j j T j j j T j j y A y u A u u A A i.e.,,
Wielandt s Deflation () Thus, y j, j,j,, : eigenvector, eigenvalue of A. To get u, first find v such that v T, then Thus, and v T y A j v T v T u T v A A j v T T A j A j v T A v j j T
Wielandt s Deflation () If has nonzero first element, normalizing it to mae its first element to be, then we can choose v T e T [ ]. Then A A e A T ( T I e )A
Other methods Rayleigh quotient iteration Bisection method Divide-and-conquer QR Algorithm etc.
Similar Matrices Definition: A and B are similar matrices if and only if there eists a nonsingular matri P such that B P - AP. (or PBP - A) Theorem: Similar matrices have the same set of eigenvalues. Proof: Since Ap p PBP - p p P - (PBP - p) P - (p) B(P - p) (P - p). Then, is an eigenvalue of A (with eigenvector p) is an eigenvalue of B (with eigenvector P - p). QED 4
QR Algorithm Given square matri A we can factor A QR where Q is orthogonal (i.e., Q t QI or Q t Q - ) and R is upper triangular. Algorithm to find eigenvalues: Start: A () A QR (note: R Q t A) A () RQ Q t A Q (A and A are similar) Factor: A ) Q () R () A () R () Q () Q ()t Q t AQQ () (similar to A () ) General: Given A (m) Factor: A (m) Q (m) R (m) A (m+) R (m) Q (m) (similar to A (m,, A ( 5, A)
QR Algorithm Note: If the eigenvalues of A satisfy > > > n Then, the iterates A m converge to an upper triangular matri having the eigenvalues of A on the diagonal. (Proof omitted) n m A L M O M M L L * * * 6
QR Algorithm- Eigenvectors How do we compute the eigenvectors? Note: A (m) Q (m-) t Q () t Q t A Q Q () Q (m-) Let Q * Q Q () Q (m-) Then, A (m) Q *t A Q * If A (m) becomes diagonal, the eigenvectors of A (m) are e, e,, e n. Since A (m) and A are similar, the eigenvectors of A are (Q *t ) - e i Q * e i, i.e., the eigenvectors of A are the columns of Q *. For any symmetric real matri, A (m) converges to a diagonal matri. For more details, see Matri Computations, by Golub, Van Loan, or other boos on matri computations. 7
Eample 8 5 4 5 A.76 -.. -. 6.9.87..87 6.89 () A.76 -... 6..8 -..8 6.884 (5) A.5 -.78 -.4984.77.6.77 -.5 -.76.56 (5) Q
Eample 9 5 4 5 A.76... 6.84.794..794 6.7 () A.76 -... 6.. -.. 6.884 () A.5 -.77 -.5.77..77 -.5 -.77.5 () Q