Eigenvalues and Eigenvectors Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 1 / 14
Introduction We define eigenvalues and eigenvectors. We discuss how to compute them. We present their main properties. We finish with two applications, each could be a final project. Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 2 / 14
Definitions Definition A scalar λ is called an eigenvalue or a characteristic value of A if there is a nontrivial solution x of Ax = λx. Such a vector x is called an eigenvector of A corresponding to the eigenvalue λ. Let us note a few things: 1 Note that the zero vector x is always a solution of Ax = λx, it is why we are looking for nontrivial vectors x. 2 An eigenvalue can be 0, but not an eigenvector. Definition The eigenspace of A corresponding to the eigenvalue λ is the set of all eigenvectors of A corresponding to λ. Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 3 / 14
Computation Recall, we are trying to solve Ax = λx. Two things can happen. Ax = λx Ax λx = 0 Ax λi x = 0 (A λi ) x = 0 1 (A λi ) is invertible that is det (A λi ) 0. The only solution is the trivial solution. This is not of interest to us since we are seeking a nontrivial solution. 2 (A λi ) is not invertible that is det (A λi ) = 0. This is the only case in which we can hope to find a solution. Definition det (A λi ) is a polynomial of degree n. It is called the characteristic polynomial. Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 4 / 14
Computation 1 det (A λi ) is a polynomial of degree n, hence it has n roots, real and/or complex. Some of which may be repeated. 2 This means that if A is an n n matrix then it will have n eigenvalues, call them λ 1, λ 2,..., λ n. Some may be repeated. 3 If an eigenvalue λ appears only once in the list, it is called simple. 4 If an eigenvalue λ appears k > 1 times in the list, we say that λ has multiplicity k. 5 If λ 1, λ 2,..., λ k (k n) are the simple eigenvalues in the list, with corresponding eigenvectors x (1), x (2),.., x (k), then the eigenvectors are linearly independent. 6 If λ is an eigenvalue with multiplicity k > 1 then λ will have anywhere from 1 to k linearly independent eigenvectors. 7 If x is an eigenvector corresponding to λ then kx is also an eigenvector corresponding to λ. This means that eigenvectors are defined up to a constant. Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 5 / 14
Computation Example Find the eigenvalues and eigenvectors of A = Example Find the eigenvalues and eigenvectors of A = ( 1 3 1 5 ) 2 1 1 0 0 2 0 3 5 Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 6 / 14
Eigenvalues and MATLAB The MATLAB function to get the eigenvalues of a matrix is eig. It can be used different ways; we only show a few here. For a complete list, type help eig within MATLAB. 1 Given an n n matrix A, eig (A) will display the eigenvalues of A. Each eigenvalue will be printed as many times as its multiplicity. 2 Given an n n matrix A, s = eig (A) will find the eigenvalues of A and store them into the n 1 vector s. As above, each eigenvalue will appear as many times as its multiplicity. 3 Given an n n matrix A, [V D] = eig (A) will find the eigenvalues and eigenvectors of A. The eigenvectors of A will be stored in V as column vectors. So, V is in fact a matrix. The eigenvalues of A will be stored on the diagonal of D, the remaining entries of D being zeros. The eigenvalues will appear in the same order as the eigenvectors. Note that MATLAB will find eigenvectors which are unit vectors (magnitude 1). Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 7 / 14
Eigenvalues and MATLAB Example Test the eig function with A = 0 0 2 1 2 1 1 0 3. Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 8 / 14
Properties of Eigenvalues and Eigenvectors We list some important properties. Their proof can be found in any linear algebra book. It will be useful to remember some properties of determinants. det ( A 1) 1 = det (A) det (AB) = det (A) det (B) det ( A ) T = det (A) det (ca) = c n det (A) Powers of a matrix: If Ax = λx then A 2 x = A (Ax) = A (λx) = λax = λ 2 x. In general, A n x = λ n x Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 9 / 14
Properties of Eigenvalues and Eigenvectors Eigenvalues of a triangular or diagonal matrix. Remembering that the determinant of a triangular or a diagonal matrix is the product of its entries on the diagonal, we see that the characteristic polynomial n of such a matrix is (a ii λ) hence the eigenvalues are a ii for i = 1..n. i=1 Similar Matrices: Recall that two matrices A and B are similar if there exists an invertible matrix P of the same size such that B = PAP 1. Two similar matrices have the same eigenvalues. λ is an eigenvalue of A if and only if 1 λ is an eigenvalue of A 1. If λ = a + bi is an eigenvalue of A with eigenvector v then λ = a bi is also an eigenvalue of A and its corresponding eigenvector is the conjugate of v. Symmetric Matrices always have real eigenvalues. Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 10 / 14
Properties of Eigenvalues and Eigenvectors Diagonalization. Suppose that v 1, v 2,..., v n are linearly independent vectors and λ 1, λ 2,..., λ n are their corresponding eigenvalues. Define P = [v 1 v 2... v n ] (note this is an n n matrix) and λ 1 0 0. D = 0 λ.. 2........ 0 then AP = PD that is A = PDP 1 or 0 0 λ n D = P 1 AP which means that the eigenvalues of A and D are the same. We can make the following important conclusions: If P is as defined, then P 1 AP is a diagonal matrix. det D = det ( P 1 AP ) = det ( P 1) det (A) det (P) = det (A) hence the determinant of a matrix is the product of its eigenvalues. Similarly, the trace of a matrix is the sum of its eigenvalues. The above implies that a matrix is invertible if and only if none of its eigenvalues is zero. Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 11 / 14
Properties of Eigenvalues and Eigenvectors Powers Matrix Revisited: If A = PDP 1 then A 2 = PDP 1 PDP 1 = PD 2 P 1. Similarly, A n = PD n P 1. Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 12 / 14
Applications An important application of eigenvalues and eigenvectors is with solving systems of first order differential equations. Google s page ranking algorithm uses a lot of linear algebra, including eigenvalues and eigenvectors. Here is a paper by Bryan and Leise on Google s PageRank algorithm. Eigenvalues for face recognition (eigenfaces). Here is the paper which started it all by Turk and Pentland. Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 13 / 14
Assignment 1 In the last section of this document, read and understand the paper on Google page ranking. This could be a potential project. 2 In the last section of this document, read and understand the paper on eigenfaces. This could be a potential project. Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 14 / 14