MAT 1302B Mathematical Methods II Alistair Savage Mathematics and Statistics University of Ottawa Winter 2015 Lecture 19 Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 1 / 28
Announcements Review class: April 14 (10:00am) The last class is open for review. We can go over theory, do extra examples, or go through an old exam. Think of topics that you d like to go over I ll collect requests next class. Third Midterm Handed back in DGDs this week. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 2 / 28
Eigenvectors, eigenvalues, and eigenspaces Definition (eigenvectors and eigenvalues) Suppose A is a square matrix. If x is a nonzero vector and λ is a scalar such that A x = λ x, x 0 then λ is an eigenvalue of A, and x is an eigenvector of A (an eigenvector corresponding to the eigenvalue λ). If λ is an eigenvalue, then the set of solutions to A x = λ x (or (A λi ) x = 0) is the eigenspace corresponding to λ. Note: Since the eigenspaces are null spaces of A λi, they are subspaces. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 3 / 28
Recap: finding eigenvalues and eigenvectors/eigenspaces Procedure for finding eigenvalues, eigenvectors and eigenspaces 1 To find the eigenvalues, find the solutions to the characteristic equation det(a λi ) = 0. 2 For each each eigenvalue, solve the equation (A λi ) x = 0 to find the corresponding eigenspace. 3 The nonzero vectors in each eigenspace are the eigenvectors corresponding to the given eigenvalue. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 4 / 28
Diagonal matrices Recall: a diagonal matrix is a square matrix whose only nonzero entries lie on the main diagonal. Examples: 1 0 0 0 6 0 1 0 0 2 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 5 Diagonal matrices are easy to work with for many reasons: They are both upper and lower triangular. Their eigenvalues are just the entries on the diagonal. Their determinant is simply the product of the entries on the diagonal. Easy to see if they are invertible (and find inverses). Easy to multiply (just multiply corresponding diagonal entries). Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 5 / 28
Example Powers of arbitrary matrices can be difficult to compute: 2 10 12 1 A = 3 1 6 3 4 8 5 4, A4 is a lot of work to compute. 7 5 7 4 Powers (or products) of diagonal matrices are relatively simple: 10 0 0 0 10000 0 0 0 D = 0 1 0 0 0 0 2 0, D4 = 0 1 0 0 0 0 16 0 0 0 0 1 0 0 0 1 Goal: Somehow relate arbitrary matrices to diagonal matrices. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 6 / 28
Example D = [ ] 2 0, P = 0 1 Then [ ] 2 7, so P 1 = 1 4 A def = PDP 1 = [ ] 23 42. 12 22 [ 4 ] 7 1 2 Suppose we want to compute A 4. Directly, this would be a lot of work. However, notice A 4 = (PDP 1 ) 4 = (PDP 1 )(PDP 1 )(PDP 1 )(PDP 1 ) = PD 4 P 1. Therefore A 4 = [ ] [ ] 4 [ ] [ ] [ ] [ ] 2 7 2 0 4 7 2 7 16 0 4 7 = 1 4 0 1 1 2 1 4 0 1 1 2 [ ] [ ] [ ] 2 7 64 112 121 210 = =. 1 4 1 2 60 104 For higher powers, save even more work: A k = PD k P 1. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 7 / 28
Some definitions Definition (similar matrices) If A = PBP 1 for some invertible matrix P, we say A and B are similar. Goal For a matrix A, try to find a diagonal matrix similar to A. Definition (diagonalizable) If A is similar to a diagonal matrix, we say A is diagonalizable. So A is diagonalizable if there is some diagonal matrix D and some invertible matrix P such that A = PDP 1. To diagonalize a matrix A means to find a diagonal matrix D and an invertible matrix P such that A = PDP 1. Note: Not all matrices are diagonalizable! Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 8 / 28
Example Let A = [ ] 2 3. Let s find the eigenvalues of A. 4 1 0 = det(a λi ) = 2 λ 3 4 1 λ = (2 λ)(1 λ) 12 = λ2 3λ 10 = (λ 5)(λ + 2) = λ = 2 or 5 Now we find the eigenspace for each eigenvalue. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 9 / 28
Example (cont.) λ = 2 : [ A + 2I 0 ] = x 1 = 3 4 x 2 x 2 free So the eigenspace is Span λ = 5 : [ A 5I 0 ] = x 1 = x 2 x 2 free So the eigenspace is Span = x = {[ ]} 3 4 1 [ 4 3 0 4 3 0 [ 3 4 x ] 2 x 2 = Span [ 3 3 0 4 4 0 [ ] x2 = x = {[ ]} 1. 1 x 2 ] [ row reduce 1 3/4 0 0 0 0 [ 3 = x 4 2 1 {[ 3 4 ] = x 2 [ 1 1 ]}. ], x 2 R [ row reduce 1 1 0 0 0 0 ], x 2 R Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 10 / 28 ] ]
Example (cont.) Form a diagonal matrix D by putting the eigenvalues on the diagonal: [ ] 2 0 D = 0 5 Then form a matrix P by putting eigenvectors as columns (in the same order as the corresponding eigenvalues appear in D): Then P = [ ] 3 1 4 1 So we have diagonalized A! = P 1 = 1 7 [ 1 ] 1 4 3 [ ] [ ] [ ] PDP 1 3 1 2 0 1/7 1/7 = 4 1 0 5 4/7 3/7 [ ] [ ] [ ] 3 1 2/7 2/7 2 3 = = = A. 4 1 20/7 15/7 4 1 Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 11 / 28
Example (cont.) Suppose we now want to compute A 100. Directly computing [ ] 100 2 3 4 1 would be a lot of work (99 matrix multiplications). However, since we ve diagonalized A, we can compute this in an easier way: A 100 = (PDP 1 ) 100 = PD 100 P 1 [ ] [ ] 100 [ ] 3 1 2 0 1/7 1/7 = 4 1 0 5 4/7 3/7 [ ] [ ] [ ] 3 1 ( 2) 100 0 1/7 1/7 = 4 1 0 5 100 4/7 3/7 This involves only two matrix multiplications much less! Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 12 / 28
Another example Suppose Then 4 2 2 A = 2 4 2. 2 2 4 det(a λi ) = (2 λ) 2 (8 λ). (exercise) So there are two eigenvalues: The eigenvalue 8 has multiplicity one. The eigenvalue 2 has multiplicity two. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 13 / 28
Another example (cont.) λ = 8: [ A 8I 0 ] = 4 2 2 0 2 4 2 0 2 2 4 0 row reduce x 1 = x 3 x 3 1 x 2 = x 3 = x = x 3 = x 3 1, x 3 free x 3 1 1 So the eigenspace is Span 1. 1 1 0 1 0 0 1 1 0 0 0 0 0 x 3 R Note: The eigenvalue 8 has multiplicity one and the eigenspace has dimension one. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 14 / 28
Another example (cont.) λ = 2: [ A 2I 0 ] = 2 2 2 0 2 2 2 0 2 2 2 0 row reduce 1 1 1 0 0 0 0 0 0 0 0 0 x 1 = x 2 x 3 x 2 x 3 1 1 x 2 free = x = x 2 = x 2 1 +x 3 0, x 2, x 3 R x 3 free x 3 0 1 1 1 So the eigenspace is Span 1, 0. 0 1 Note: The eigenvalue 2 has multiplicity two and the eigenspace has dimension two. So for each eigenvalue, the multiplicity was equal to the dimension of the eigenspace. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 15 / 28
Another example (cont.) Form a diagonal matrix by putting the eigenvalues on the diagonal (the number of times equal to its multiplicity): 8 0 0 D = 0 2 0 0 0 2 Then form a matrix P by putting the basis vectors of the eigenspaces as columns (in the same order as the corresponding eigenvalues appear in D): 1 1 1 P = 1 1 0 1 0 1 Exercise: Check that A = PDP 1. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 16 / 28
Comments 1 Can reorder the eigenvalues/eigenvectors just make sure the order of eigenvalues matches the order of eigenvectors. Example: In the previous example, we had 1 1 1 8 0 0 P = 1 1 0, D = 0 2 0 1 0 1 0 0 2 but we could have taken 1 1 1 2 0 0 P = 1 0 1, D = 0 2 0. 0 1 1 0 0 8 Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 17 / 28
Comments (cont.) 2 For a given eigenvalue, can choose any basis of eigenvectors for the corresponding eigenspace. 3 A matrix is diagonalizable iff the dimension of each eigenspace equals the multiplicity of the corresponding eigenvalue (the dimension will always be the multiplicity). Equivalently, an n n matrix is diagonalizable iff the sum of the dimensions of the eigenspaces is n. Definition (geometric multiplicity) The geometric multiplicity of an eigenvalue is the dimension of the corresponding eigenspace. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 18 / 28
Warnings! Diagonalizable invertible! Diagonalizable is not the same thing as invertible! A matrix can be invertible but not diagonalizable or diagonalizable but not invertible. However...remember that a matrix is invertible iff it does not have zero as an eigenvalue. So if you ve diagonalized a matrix (hence found its eigenvalues), you can easily tell if it s invertible or not. Similar row equivalent! Similar is not the same this as row equivalent! A is row equivalent to B if you can get from A to B via a sequence of row operations. A is similar to B if there is some invertible matrix P such that A = PBP 1. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 19 / 28
Example 3 1 0 A = 0 3 0 0 0 2 Characteristic polynomial: det(a λi ) = (3 λ) 2 (2 λ) So the eigenvalues are 2 (multiplicity 1) and 3 (multiplicity 2). For λ = 3, we find the eigenspace: [ A 3I 0 ] 0 1 0 0 = 0 0 0 0 0 0 1 0 row reduce 0 1 0 0 0 0 1 0 0 0 0 0 1 free variable. So the dimension of the eigenspace is 1 (the geometric multiplicity is 1). But the (algebraic) multiplicity of the eigenvalue is 2. Therefore, A is not diagonalizable. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 20 / 28
Note Remember that λ is an eigenvalue iff (A λi ) x = 0 has some nontrivial solution (i.e. there is at least one free variable in the solution set). Thus, the dimension of the eigenspace corresponding to any eigenvalue is always at least one. Theorem An n n matrix with n distinct eigenvalues is always diagonalizable. Reason An n n matrix has n eigenvalues counting multiplicity. If it has n distinct eigenvalues, each one must have multiplicity one. Since the dimension of each eigenspace is at least one and can be at most one (since the dimension of the eigenspace is always the multiplicity), it is exactly one. So the matrix is diagonalizable. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 21 / 28
Procedure for diagonalizing matrices Suppose we want to diagonalize the n n matrix A. 1 Find the eigenvalues of A by solving det(a λi ) = 0. 2 For each eigenvalue, find the corresponding eigenspace: Row reduce [ A λi 0 ]. Write the solution in vector parametric notation: x = x i1 v 1 + x i2 v 2 + + x ik v k Eigenspace = Span{ v1,..., v k }. If dimension k < the multiplicity of the eigenvalue, then A is not diagonalizable. 3 Construct P from the eigenvectors found in Step 2: P = [ v 1 v 2 ] 1 4 Construct D from the eigenvalues: D = λ λ 2 λ i = eigenvalue of v i. where... 5 Check by verifying A = PDP 1 or AP = PD (easier, since don t need to compute P 1 ). Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 22 / 28
The Diagonalization Theorem Definition (eigenvector basis) If A is an n n matrix, then an eigenvector basis for A is a basis of R n consisting of eigenvectors of A. The Diagonalization Theorem 1 An n n matrix A is diagonalizable iff A has n linearly independent eigenvectors. In other words, A is diagonalizable iff there is an eigenvector basis for A. 2 A = PDP 1 for some diagonal matrix D iff the columns of P are an eigenvector basis. In this case, the diagonal entries of D are the corresponding eigenvalues of A. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 23 / 28
Example 1 Is diagonalizable? 3 10 8 3 B = 0 4 7 5 0 0 1 4 0 0 0 0 Solution: B is triangular (so the diagonal entries are the eigenvalues). The eigenvalues of B are 3, 4, 1, 0. Since B is 4 4 and has 4 distinct eigenvalues, it is diagonalizable. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 24 / 28
Example 2 Suppose 0 4 6 A = 1 0 3. 1 2 5 If the eigenvalues of A are 2 and 1, find matrices P and D with D diagonal such that A = PDP 1. Solution: We first find the eigenspace for each eigenvalue. For λ = 2: [ A 2I 0 ] = x = x 2 2 4 6 0 1 2 3 0 1 2 3 0 row reduce 1 2 3 0 0 0 0 0 0 0 0 0 2 3 2 3 1 +x 3 0, x 2, x 3 R = Eigenspace = Span 1, 0 0 1 0 1 Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 25 / 28
Example 2 (cont.) For λ = 1: [ A I 0 ] = 1 4 6 0 1 1 3 0 1 2 4 0 row reduce 1 0 2 0 0 1 1 0 0 0 0 0 2 2 x = x 3 1, x 3 R = Eigenspace = Span 1 1 1 Since the sum of the dimensions of the eigenspaces is 3 and A is 3 3, it is diagonalizable. Let 2 3 2 2 0 0 P = 1 0 1, D = 0 2 0. 0 1 1 0 0 1 Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 26 / 28
Example 2 (cont.) Check your answer! 0 4 6 2 3 2 4 6 2 AP = 1 0 3 1 0 1 = 2 0 1 1 2 5 0 1 1 0 2 1 2 3 2 2 0 0 4 6 2 PD = 1 0 1 0 2 0 = 2 0 1 0 1 1 0 0 1 0 2 1 Note: Could also take 3 2 2 2 0 0 P = 0 1 1, D = 0 1 0, 1 1 0 0 0 2 etc. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 27 / 28
Next time Next time: Markov chains Applications (Google PageRank). Difference equations. Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture 19 28 / 28