Diagonalization We have seen that diagonal and triangular matrices are much easier to work with than are most matrices For example, determinants and eigenvalues are easy to compute, and multiplication is much more straightforward Diagonal matrices are particularly nice For example, the product of two diagonal matrices can be computed by simply multiplying their corresponding diagonal entries: a b a b a 2 b 2 a 2 b 2 a n b n a n b n Computationally, diagonal matrices are the easiest to work with With this idea in mind, we introduce similarity: Definition An n n matrix A is similar to a matrix B if there is an invertible matrix P so that B P AP, and the function c P defined by is called a similarity transformation c P (A) P AP As an example, matrices are similar; with you should check that A P ( ) 4 ( 4 ) 3 and B and P ( 8 ) 3 36 3 ( ), 3 4 ( 8 ) 3 36 3 so that A and B are indeed similar B P AP ( ) ( 3 4 4 ) ( 4 3 We are concerned with similarity not for its own sake as an interesting phenomenon, but because of quantities known as similarity invariants We can get a feel for what similarity invariants are by considering data about the matrices from the previous example: in particular, let s calculate the determinant, trace and eigenvalues of A and B: ),
property A B determinant 4 4 trace 5 5 eigenvalues, 4, 4 As you may have guessed from the table above, certain matrix properties, such as the determinant, trace, and eigenvalues, are shared among similar matrices; this is what we mean when we use the phrase similarity invariant In other words, the determinant of the matrix A is also the determinant of any matrix similar to A In fact, it is quite easy to check that the determinant is a similarity invariant; to do so, we recall two rules: det(ab) det(a) det(b), and det(p ) det P Any matrix similar to a given matrix A must have form P AP, with P an invertible matrix Let s calculate the determinant of this similar matrix, using the rules above: det(p AP ) (det P )(det A)(det P ) ( ) (det A)(det P ) det P det A We have just shown that similar matrices share determinants that is, det A det(p AP ) for any invertible matrix P In general, if matrices A and B are similar, so that B P AP, then they share: determinant trace eigenvalues rank nullity invertibility characteristic polynomial eigenspace corresponding to a particular (shared) eigenvalue Based on the example above, you may have already guessed the reason that we care about the idea of similarity: If A and B are similar matrices, and if A is diagonal, then it is much easier to calculate data such as determinant and eigenvalues about A than it is about B With this in mind, we introduce the idea of diagonalizability: Definition 2 An n n matrix A is diagonalizable if it is similar to a diagonal matrix That is, A is diagonalizable if there is an invertible matrix P so that P AP is diagonal The matrix B ( 8 ) 3 36 3 2
is diagonalizable, since it is similar to the diagonal matrix ( ) A 4 Key Point It is important to note that not every matrix is diagonalizable Indeed, there are many matrices which are simply not similar to a diagonal matrix We will examine a few later in this section Criteria for Diagonalizability Given the information we have collected so far, it should be clear that, while diagonal matrices are arguably the easiest type of matrix to work with, diagonalizable matrices are almost as easy If I want to know the determinant, trace, etc of a matrix B that is similar to a diagonal matrix A, then I simply need to make the (easier) calculations for A This leads to a few interesting questions: how can we be certain that a given matrix is diagonalizable? And if we know that a matrix is diagonlizable, how do we find the diagonal matrix to which it is similar? The following theorem answers the first question: Theorem 52 If A is an n n matrix, then the following statements are equivalent: (a) A is diagonalizable (b) A has n linearly independent eigenvectors In other words, we can check that matrix A is diagonalizable by looking at its eigenvectors: if A has n linearly independent eigenvectors, then it is diagonalizable Example In Section 5, we saw that the matrix 2 A 4 2 4 3 has repeated eigenvalue λ λ 3 2, with two corresponding linearly independent eigenvectors x and x 3 4 3
In addition, the eigenvalues λ 2 and λ 4 3 have eigenvectors x 2 4 and x 4, 3 respectively You should check that the four eigenvectors above are linearly independent inspecting the linear combination ax + bx 2 + cx 3 + dx 4 ; indeed, it is easy to see that the corresponding system a b 4 3 b + c 4a + d has only the trivial solution a b c d Since A is 4 4 and has 4 distinct eigenvectors, A is diagonalizable Example Determine if the matrix is diagonalizable A ( ) 4 2 We need to check the eigenvectors of A; if A is diagonalizable, then it has two linearly independent eigenvectors Accordingly, we begin by finding the eigenvalues of A, using the characteristic equation: ( ) λ 4 det(λi A) det λ 2 (λ 4)(λ 2) + λ 2 6λ + 8 + λ 2 6λ + 9, 4
so that the characteristic equation for A is λ 2 6λ + 9 By factoring the equation, we see that its roots are λ 3, so that A has a single repeated eigenvalue Any eigenvector x corresponding to λ 3 must satisfy Ax 3x ( ) ( ) 4 x 2 x 2 ( ) 4x x 2 x + 2x 2 ( ) 3x 3x 2 ( ) 3x 3x 2 Thus we see that both of which amount to the single equation 4x x 2 3x x + 2x 2 3x 2, x x 2 or x x 2 Parameterizing x as x t, we see that any eigenvector of A must have form ( ) ( ) t t t Thus A has only one linearly independent eigenvector; since A is 2 2, it is not diagonalizable The following theorem on eigenvalues and their associated eigenvectors will give us a quick way to check some matrices for diagonalizability: Theorem 522 If λ, λ 2,, λ k are distinct eigenvalues of an n n matrix A, and x, x 2,, x k are eigenvectors corresponding to λ, λ 2,, λ k respectively, then the set is a linearly independent set {x, x 2,, x k } The theorem says that, for each distinct eigenvalue of a matrix A, we are guaranteed another linearly independent eigenvector For example, if a 4 4 matrix A has eigenvalues, 2,, and 5, then since it has four distinct eigenvalues, A automatically has four linearly independent eigenvectors Taken together with Theorem 52 on diagonlizability, we have the following corollary: Corollary Any n n matrix with distinct eigenvalues is diagonalizable 5
Key Point We must be extremely careful to note that we can only use the corollary to draw conclusions about an n n matrix if the matrix has n distinct eigenvalues If the matrix does not have n distinct eigenvalues, then it may or may not be diagonalizable In fact, we have seen two matrices 2 ( ) 4 4 2 and 2 4 3 with repeated eigenvalues: the first matrix has eigenvalues, 2, 2, and 3, and the second has eigenvalues 3 and 3 The first matrix is diagonalizable, while the second is not Finding the Similar Diagonal Matrix Earlier, we asked how we could go about finding the diagonal matrix to which a diagonalizable matrix A is similar If A is diagonalizable, with P AP the desired diagonal matrix, then we can rephrase the question above: How do we find P? The answer to this question turns out to be quite interesting: Theorem Let the n n matrix A be diagonalizable with n linearly independent eigenvectors x, x 2,, x n Set P x x 2 x n In other words, P is the matrix whose columns are the n linearly independent eigenvectors of A Then P AP is a diagonal matrix whose diagonal entries are the eigenvalues λ, λ 2,, λ n that correspond to the eigenvectors forming the successive columns of P Example Find the diagonal matrix to which is similar Earlier, we saw that matrix 2 A 4 2 4 3 2 A 4 2 4 3 6
has linearly independent eigenvectors x corresponding to eigenvalues 4, x 2 4 3, x 3, and x 4 λ 2, λ 2, λ 3 2, and λ 4 3 respectively The matrix P from the theorem above is given by you should check that P 4 ; 3 4 P 4 3 4 is its inverse According to the theorem, P AP is diagonal let s verify this: 2 P AP 4 4 2 4 3 3 4 4 3 4 2 8 2 4 3 3 2 3 4 2 2 3 7
Powers of a Diagonalizable Matrix Matrix multiplication with diagonal matrices is remarkably simple As a quick example, we can calculate the product below simply by multiplying corresponding diagonal entries: 5 4 2 2 3 2 2 6 As you might have guessed, there is a simple formula for calculating powers of diagonal matrices: if d d 2 D, d n then D k exists if k is any integer and each d i is nonzero, or one or more of the d i is, and k is a positive integer If D k exists, then d k D k d k 2 d k n We can use these facts to our advantage if we know that matrix A is diagonalizable To see why, suppose we wished to calculate A k, where A is diagonalizable but not itself diagonal Since A is diagonalizable, there is a diagonal matrix D and invertible matrix P so that thus we rewrite A k as A k (P DP ) k A P DP ; (P DP )(P DP )(P DP ) (P DP )(P DP ) }{{} k copies P D(P P )D(P P )D(P P )D(P P )DP ) P DDD }{{ DD} P k copies P D k P 8
In other words, in order to calculate A k (which might be hard), we can first calculate D k (easy), then finish off with a similarity transformation: A k P D k P Key Point The idea behind the observations above are actually true in a more general context than the one presented above: if A and B (not necessarily diagonal) are similar by an invertible matrix P, then A k and B k are also similar by P Example Given find the eigenvalues of A 5 2 A 4 2, 4 3 In an earlier example, we saw that A is diagonalizable via P 4 3 4 to the diagonal matrix 2 D 2 3 Now since A and D are similar, they share eigenvalues; A 5 and D 5 are also similar (via the same matrix P ), so that they share eigenvalues as well The eigenvalues of D 5 are easy to find; D 5 is diagonal, so its eigenvalues are its diagonal entries Let s make the calculation: 2 5 D 5 ( ) 5 2 5 3 5 32 32 243 9
Thus D 5 has eigenvalues λ 32, λ 2, λ 3 32, and λ 4 243, which it shares with A 5 since they are similar Geometric and Algebraic Multiplicity Given an n n matrix A, there are two possibilities for the types of eigenvalues A could have: n distinct eigenvalues 2 some repeated eigenvalues In the first case, we automatically know that A is diagonalizable; however, in the second case, A may or may not be diagonalizable Indeed, we saw two different examples of repeated eigenvalues: 2 A 4 2 4 3 has eigenvalues, 2, 2, and 3 and is diagonalizable, while ( ) 4 B 2 has eigenvalues 3 and 3 but is not diagonalizable There are apparently different types of repeated eigenvalues those, like the 2 from matrix A above, that do not affect diagonalizability, and those such as the 3 from matrix B which do We would like to have some terminology to help us differentiate between good eigenvalues and bad ones, and so we will momentarily introduce the ideas of algebraic and geometric multiplicity Before we do so, recall that eigenvalues are nothing but roots of the characteristic equation of the associated matrix For example, since matrix B above has eigenvalue 3 repeated twice, its characteristic equation must be (λ 3)(λ 3) Definition Let λ be an eigenvalue of the matrix A The number of times that the factor (λ λ ) appears as a factor of the characteristic polynomial det(λi A) of A is called the algebraic multiplicity of λ The dimension of the eigenspace associated with λ is called the geometric multiplicity of λ Let s think about the ideas of algebraic and geometric multiplicity in terms of the two examples above Starting with 2 A 4 2, 4 3
we can calculate that its characteristic equation is (λ 2)(λ 2)(λ + )(λ 3) Since λ 2 shows up twice as a factor, the eigenvalue 2 has algebraic multiplicity 2; the other two eigenvalues have algebraic multiplicity To get the geometric multiplicities of the eigenvalues, we need to check the eigenspaces corresponding to each distinct eigenvalue: earlier, we saw that A has linearly independent eigenvectors x 4, x 2 4 3, x 3, and x 4 ; x and x 3 correspond to the repeated eigenvalue λ 2, x 2 corresponds to the distinct eigenvalue λ 2 and x 4 corresponds to the distinct eigenvalue λ 4 3 Thus the repeated eigenvalue 2 corresponds to an eigenspace with a basis consisting of 2 vectors, whereas each of the eigenvalues and 3 have eigenspaces with a basis consisting of vector Thus the geometric multiplicity of the eigenvalue 2 is 2, and the eigenvalues and 3 both have geometric multiplicities We record all of the data in a table: Distinct Geometric Algebraic Eigenvalue Multiplicity Multiplicity 2 2 2-3 Next, let s examine matrix B It is easy to calculate the characteristic equation ( ) 4 2 (λ 3)(λ 3) ; since the factor (λ 3) shows up twice in the equation, the repeated eigenvalue λ 3 has algebraic multiplicity 2 Earlier, we saw that the eigenspace corresponding to eigenvalue λ 3 is generated by the single vector ( ) x ; thus λ 3 has geometric multiplicity Let s compare all of the data that we have generated:
Distinct Geometric Algebraic Distinct Geometric Algebraic Eigenvalue of A Multiplicity Multiplicity Eigenvalue of B Multiplicity Multiplicity 2 2 2 3 2-3 Notice in the example above that the algebraic and geometric multiplicities of each distinct eigenvalue of the diagonalizable matrix A matched up, whereas the geometric multiplicity of the only distinct eigenvalue of the non-diagonalizable matrix B was deficient compared with its algebraic multiplicity This idea is actually a general rule, made precise in the following theorem: Theorem 524 Let A be an n n matrix (a) The geometric multiplicity of a distinct eigenvalue of A is less than or equal to its algebraic multiplicity (b) A is diagonalizable if and only if the algebraic and geometric multiplicities of each of its distinct eigenvalues are equal 2