EIGENVALUES AND EIGENVECTORS

EIGENVALUES AND EIGENVECTORS Diagonalizable linear transformations and matrices Recall, a matrix, D, is diagonal if it is square and the only non-zero entries are on the diagonal This is equivalent to D e i = λ i e i where here e i are the standard vector and the λ i are the diagonal entries A linear transformation, T : R n R n, is diagonalizable if there is a basis B of R n so that [T ] B is diagonal This means [T ] is similar to the diagonal matrix [T ] B Similarly, a matrix A R n n is diagonalizable if it is similar to some diagonal matrix D To diagonalize a linear transformation is to find a basis B so that [T ] B is diagonal To diagonalize a square matrix is to find an invertible S so that S AS = D is diagonal Fix a matrix A R n n We say a vector v R n is an eigenvector if () v (2) A v = λ v for some scalar λ R The scalar λ is the eigenvalue associated to v or just an eigenvalue of A Geometrically, A v is parallel to v and the eigenvalue, λ counts the stretching factor Another way to think about this is that the line L := span( v) is left invariant by multiplication by A An eigenbasis of A is a basis, B = ( v,, v n ) of R n so that each v i is an eigenvector of A Theorem The matrix A is diagonalizable if and only if there is an eigenbasis of A Proof Indeed, if A has eigenbasis B = ( v,, v n ), then the matrix S = [ v v n ] satisfies λ S AS = λ 2 λ n where each λ i is the eigenvalue associated to v i Conversely, if A is diagonalized by S, then the columns of S form an eigenbasis of A EXAMPLE: The the standard vectors e i form an eigenbasis of I n Their eigenvalues are More generally, if D is diagonal, the standard vectors form an eigenbasis with associated eigenvalues the corresponding entries on the diagonal EXAMPLE: If v is an eigenvector of A with eigenvalue λ, then v is an eigenvector of A 3 with eigenvalue λ 3 EXAMPLE: is an eigenvalue of A if and only if A is not invertible Indeed, is an eigenvalue there is a non-zero v so A v = true v ker A so ker A is non-trivial A not invertible

2 EIGENVALUES AND EIGENVECTORS EXAMPLE: If v is an eigenvector of Q which is orthogonal, then the associated eigenvalue is ± Indeed, v = Q v = λ v = λ v as v dividing, gives λ = EXAMPLE: If A 2 = I n, then there are no eigenvectors of A To see this, suppose v was an eigenvector of A Then A v = λ v As such v = I n v = A 2 v = λ 2 v That is, λ 2 = There are no real numbers whose square is negative, so there is no such v This means A has no real eigenvalues (it does have have a comples eigenvalues see Section 75 of the textbook This is beyond scope of this course) 2 Characteristic Equaiton One of the hardest (computational) problems in linear algebra is to determine the eigenvalues of a matrix This is because, unlike everything else we have considered so far, it is a non-linear problem That being said, it is still a tractable problem (especially for small matrices) To understand the approach Observe that if λ is an eigenvalue of A, then there is a non-zero v so thata v = λ v That is, A v = (λi n ) v (A λi n ) v = v ker(a λi n ) From which we conclude that A λi n is not invertible and so det(a λi n ) = In summary, Theorem 2 λ is an eigenvalue of A if and only if det(a λi n ) = The equation det(a λi n ) = is called the characteristic equation of A EXAMPLE: Find the eigenvalues of 2 3 A = 3 2 The characteristic equation is 2 λ 3 det(a λi 2 ) = det = λ 2 4λ 5 = (λ + )(λ 5) 3 2 λ Hence, the eigenvalues are λ = and λ = 5 To find corresponding eigenvectors we seek non-trivial solutions to 2 ( ) 3 x 2 (5) 3 x = and = 3 2 ( ) x 2 3 2 (5) x 2 By inspection the non-trivial solutions are and Hence, So we have diagonalized A 2 3 3 2 = 5

EIGENVALUES AND EIGENVECTORS 3 So EXAMPLE: Find eigenvalues of 2 3 A = 2 3 λ 2 3 det(a λi 3 ) = det 2 λ = ( λ)(2 λ)(3 λ) 3 λ Hence, eigenvalues are, 2, 3 This example is a special case of a more general phenomena Theorem 22 If M is upper triangular, then the eigenvalues of M are the diagonal entries of M EXAMPLE: When n = 2, the eigenvalues of a b A = c d so the characteristic equation is a λ b det(a λi 2 ) = det = λ 2 (a + d)λ + (ad bc) = λ 2 traλ + det A c d λ Using the quadratic formula we have the following: () When tr(a) 2 4 det A >, then two distinct eigenvalues (2) When tr(a) 2 4 det A =, exactly one eigenvalue 2 tra (3) When tr(a) 2 4 det A <, then no (real) eigenvalues 3 Characteristic Polynomial As we say for a 2 2 matrix, the characteristic equation reduces to finding the roots of an associated quadratic polynomial More generally, for a n n matrix A, the characteristic equation det(a λi n ) = reduces to finding roots of a degree n polynomial o fthe form f A (λ) = ( ) n λ n + ( ) n (tra)λ n + + det A this is called the characteristic polynomial of A To see why this is true observe that if P is the diagonal pattern of A λi n, then prod(p ) = (a λ) (a nn λ) = ( ) n λ n + ( ) n (tra)λ n 2 + R P (λ) where R P (λ) is a polynomial of degree at most n 2 that depends on P (and of course also on A and λ) If P is some other pattern of A λi n, then at least two entries are not on the diagonal Hence, prod(p ) = R P (λ) for some R P (λ) that is a polynomial of degree at most n 2 that depends on P Hence, f A (λ) = det(a λi n ) = ( ) n λ n + ( ) n (tra)λ n + R(λ) where R(λ) has degree at most n 2 Finally, f A () = det(a) and so the constant term of f A is det A as claimed

4 EIGENVALUES AND EIGENVECTORS EXAMPLE: If n is odd, there is always at least one real eigenvalue Indeed, in this case lim f A(λ) = lim n ± n λ3 = That is for λ >>, f A (λ) < and for λ <<, f A (λ) > and so by the intermediate value theorem from calculus (and the fact that polynomials are continuous), f A (λ ) = for some λ This may fail when n is even An eigenvalue λ has algebraic multiplicity k if f A (λ) = (λ λ) k g(λ) where g is a polynomial of degree n k with g(λ ) Write almu(λ ) = k in this case EXAMPLE: If 2 A = 2 then f A (λ) = (2 λ) 2 ( λ)( λ) and so almu(2) = 2, while almu() = almu( ) = Strictly speaking, almu() =, as is not an eigenvalue of A and it is sometimes convenient to follow this convention We say an eigenvalue, λ, is repeated if almu(λ) 2 Algebraic fact, counting algebraic multiplicity, a n n matrix has at most n real eigenvalues If n is odd, then there is at least one real eigenvalue The fundamental theorem of algebra ensures that, counting multiplicity, such a matrix always has exactly n complex eigenvalues We conclude with a simple theorem Theorem 3 If A R n n has eigenvalues λ,, λ n (listed counting multiplicity): () det A = λ λ 2 λ n (2) tra = λ + λ 2 + + λ n Proof If the eigenvalues are λ,, λ n, then we have the complete factorization of f A (λ) f A (λ) = (λ λ) (λ n λ) This means det(a) = f A () = λ λ n (λ λ) (λ n λ) = ( ) n λ n + ( ) n (λ + +λ n )λ n +R(λ) where R(λ) of degree at most n 2 Comparing the coefficient of the λ n 2 term gives the result 4 Eigenspaces Consider an eigenvalue λ of A R n n We define the eigenspace associated to λ to be E λ = ker(a λi n ) = { v R n : A v = λ v} R n Observe that dim E λ All non-zero elements of E λ are eigenvectors of A with eigenvalue λ EXAMPLE: A = has repeated eigenvalue Clearly, E = ker(a I 2 ) = ker( 2 2 ) = R 2

EIGENVALUES AND EIGENVECTORS 5 2 Similarly, the matrix B = has one repeated eigenvalue However, [ 2 ker(b I 2 ) = ker = span( ) ] Motivated by this example, define the geometric multiplicity of an eigenvalue λ of A R n n tobe gemu(λ) = null(a λi n ) = n rank(a λi n ) 5 Diagonalizable Matrices We are now ready to give a computable condition that will allow us to determine an answer to our central question in this part of the course: When is A R n n diagonalizable? Theorem 5 A matrix A R n n is diagonalizable if and only if the sum of the geometric multiplicities of all of the eigenvalues of A is n EXAMPLE: For which k is the following diagonalizable k? 2 As this is upper triangular, the eigenvalues are with almu() = 2 and 2 with almu(2) = It is not hard to see that gemu() = when k and gemu() = 2 when k = We always have gemu(2) = Hence, according to the theorem the matrix is diagonalizable only when it is already diagonal (that is k = ) and is otherwise not diagonalizable To prove this result we need the following auxiliary fact Theorem 52 Fix a matrix A R n n and let v,, v s be a set of vectors formed by concatenating a basis of each non-trivial eigenspace of A This set is linearly independent (and so s n) To explain what I mean by concatenating Suppose A R 5 5 has exactly three distinct eigenvalues λ = 2 and λ 2 = 3 and λ 3 = 4 If gemu(2) = 2 and while gemu(3) = gemu(4) = and E 2 = span( a, a 2 ) E 3 = span( b ) and E 4 = span( c ), then their concatenation is the set of vectors ( v, v 2, v 3, v 4 ) = ( a, a 2, b, c ) According to the auxiliary theorem this list is linearly independent As s = 4 < 5 = n, if this was the complete set of eigenvalues, A would not be diagonalizable by the main theorem It would be if we had omitted one eigenvalue from our list Let us now prove the auxiliary theorem

6 EIGENVALUES AND EIGENVECTORS Proof If the vectors v i are not linearly independent, then they at least one is redundant Let v m be the first redundant vector on the list That is for some m s we can write v m = m i= and cannot do this for any smaller m This means v,, v m are linearly independent Let λ m be the eigenvalue associated to v m Observe, there must be some k m so that λ k λ m and c k as otherwise we would have a non-trivial linear relation for of a set of linearly independent vectors in E λm (which is impossible) Clearly, = (A λ m I n ) v m = c i v i m i= c i (λ i λ m ) v i Hence, c i (λ i λ m ) = for each i This contradicts, λ k λ m and c k and proves the claim The main theorem follows easily form this Indeed, the hypotheses gives n lin indep vectors all which are eigenvectors of A That is, an eigenbasis of A 6 Strategy for diagonalizing A R n n We have the following strategy for diagonalizing a given matrix: () Find eigenvalues of A by solving f A (λ) = (this is a non-linear problem) (2) For each eigenvalue λ find a basis of the eigenspace E λ (this is a linear problem) (3) The A is diagonalizable if and only if the sum of the dimensions of the eigenspaces is n In this case, obtain an eigenbasis, v,, v n, by concatenation (4) As A v i = λ i v i, setting λ S = v v n, S AS = D = λ 2 λ n EXAMPLE: Diagonalize (if possible) 2 A = 2 We compute 2 λ f A (λ) = det(a λi 3 ) = det 2 λ 2 λ = ( λ) det 2 λ λ f A (λ) = ( λ)(λ 2 4λ + 3) = ( λ)(λ 3)(λ ) = ( λ) 2 (3 λ)

EIGENVALUES AND EIGENVECTORS 7 Hence, eigenvalues are λ = and λ = 3 Have almu() = 2 and almu(3) = We compute E = ker(a I 3 ) = ker = span(, ) Hence, gemu() = 2 Likewise, E 3 = ker(a 3I 3 ) = ker = span( ) 2 Hence, gemu(3) = As gemu()+gemu(3) = 2+ = 3 the matrix is diagonalizable Indeed, setting S = We have S AS = 3 7 Eigenvalues of linear transformations Fix a linear space V and consider a linear transformation T : V V A scalar λ is an eigenvalue of T, if T (f) = λf for some nonzero (nonneutral) element f V In general we refer to such f as a eigenvector If V is a space of functions, then it is customary to call f an eigenfunction, etc If B = (f,, f n ) is a basis of V and each f i is an eigenvector, then we say B is an eigenbasis of T If B is an eigenbasis of T, then T is diagonalizable as λ [T ] B = λ n EXAMPLE: If V = C, and D(f) = f, then D(e kx ) = d dx ekx = ke kx so each f k (x) = e kx is an eigenfunction and every scalar k R is an eigenvalue EXAMPLE: Consider the map T : P 2 P 2 given by T (p) = p(2x + ) Is T diagonalizable? As usual it is computationally more convenient to work in some basis To that end, let U = (, x, x 2 ) be the usual basis of P 2 As [T ()] U = [] U = [T (x)] U = [2x + ] U = 2

8 EIGENVALUES AND EIGENVECTORS [T (x 2 )] U = [4x 2 + 4x + ] U = 4, 4 the associated matrix is A = [T ] U = 2 4 4 This matrix is upper triangular, with distinct eigenvalues, 2 and 4 This means T is also diagonalizable and has the same eigenvalues We compute (for A) E = ker(a I 3 ) = ker 4 = span( ) 3 E 2 = ker(a 2I 3 ) = 4 = span( ) 2 3 E 3 = ker(a 4I 3 ) = 2 4 = span 2 Hence, A can be diagonalized by S = 2 Going back to T we check T () = T ( + x) = + (2x + ) = 2(x + ) T ( + 2x + x 2 ) = + 2(2x + ) + (2x + ) 2 = 4( + 2x + x 2 ) In particular, B = (, + x, + 2x + x 2 ) is an eigenbasis and [T ] B = 2 4 8 Eigenvalues and Similarity There is a close relationship between similar matrices and their eigenvalues This is clearest for diagonalizable matrices but holds more generally Theorem 8 If A is similar to B () f A (λ) = f B (λ) (2) rank(a) = rank(b) and null(a) = null(b) (3) A and B have the same eigenvalues with the same algebraic and geometric multiplicities (the eigenvectors are in general different) (4) tra = trb and det(a) = det(b) Proof Proof of claim (): A is similar to B means B = S AS for some invertible S Hence, f B (λ) = det(b λi n ) = det(s AS λi n ) = det(s (A λi n )S) = det(a λi n ) = f A (λ) Claim (2) was shown in a previous handout (the one on similar matrices)

EIGENVALUES AND EIGENVECTORS 9 Claim (3) can be shown as follows: By claim (), f A (λ) = f B (λ) and so A and B have the same eigenvalues with the same algebraic multiplicities Furthermore, if λ is an eigenvalue of both A and B, then A λi n is similar to B λi n Hence, by claim (2) gemu A (λ) = null(a λi n ) = null(b λi n ) = gemu B (λ) so the geometric multiplicities are the same as well Notice E λ (A) E λ (B) in general Finally, claim (4) Follows from claim () and the observation that the characteristic polynomial encodes the trace and determinant 2 3 3 2 EXAMPLE: The matrix is not similar to as their traces are 5 7 8 5 different Even if a matrix is not diagonalizable, it is still similar to one of a list of canonical matrices To do this is full generality is beyond the scope of this course To illustrate the idea we present the list for 2 2 matrices Theorem 82 Any A R 2 2 is similar to one of the following: λ () for λ λ, λ 2 R This occurs when A is diagonalizable 2 λ (2) This occurs when A has repeated eigenvalue λ with gemu(λ) = λ Such [ A is] sometimes called defective λ λ (3) 2 = cos θ sin θ λ λ 2 λ 2 + λ2 2 where λ sin θ cos θ 2 > and π > θ > This corresponds to an A with no real eigenvalues (in this case the complex eigenvalues of A are λ ± iλ 2 )