12.4 The Diagonalization Process

Transcription

1 Chapter - More Matrix Algebra.4 The Diagonalization Process We now have the background to understand the main ideas behind the diagonalization process. Definition: Eigenvalue, Eigenvector. Let A be an nµ n matrix over R. l is an eigenvalue of A if for some nonzero column vector x œ R n we have A x l x. x is called an Eigenvectors corresponding to the eigenvalue l. Example.4.. Find the eigenvalues and corresponding eigenvectors of the matrix A K O. We want to find nonzero vectors x K x O x and real numbers l such that A X l X ñ K O K x x O l K x x O ñ K O K x x O - l K x x O K O ñ K O K x x O - l K O K x x O K O ñ K K O - l K O O K x x O K O ñ K - l - l O K x O K x O H.4 al The last matrix equation will have nonzero solutions if and only if det K - l - l O or H - ll H - ll -, which simplifies to l - 5 l + 4. Therefore, the solutions to this quadratic equation, l and l 4, are the eigenvalues of A. We now have to find eigenvectors associated with each eigenvalue. Case. For l, Equation.4a becomes: K - - O K x x O K O K O K x O K x O which reduces to the single equation, x + x. From this, x -x. This means the solution set of this equation is (in column notation) E : K -c c O c œ R > So any column vector of the form K -c c O where c is any nonzero real number is an eigenvector associated with l. The reader should verify that, for example, K O - - so that - is an eigenvector associated with eigenvalue. Case. For l 4 equation.4.a becomes: K O K x x O K O K - - O K x O K x O which reduces to the single equation - x + x, so that x x. The solution set of the equation is c E :K - c O c œ R > Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

2 Chapter - More Matrix Algebra c Therefore, all eigenvectors of A associated with the eigenvalue l 4 are of the form K O, where c can be any nonzero number. - c The following theorems summarize the more important aspects of this example: Theorem.4.. Let A be any nµ n matrix over R. Then l œ R is an eigenvalue of A if and only if detha - l IL. The equation detha - l IL is called the characteristic equation and the left side of this equation is called the characteristic polynomial of A. Theorem.4.. Nonzero eigenvectors corresponding to distinct eigenvalues are linearly independent. The solution space of HA - l IL x is called the eigenspace of A corresponding to l. This terminology is justified by Exercise of this section. We now consider the main aim of this section. Given an nµn (square) matrix A, we would like to "change" A into a diagonal matrix D, perform our tasks with the simpler matrix D, and then describe the results in terms of the given matrix A. Definition: Diagonalizable Matrix. An n µ n matrix A is called diagonalizable if there exists an invertible n µ n matrix P such that P - A P is a diagonal matrix D. The matrix P is said to diagonalize the matrix A. Example.4.. We will now diagonalize the matrix A of Example.4.. Form the matrix P as follows: Let P HL be the first column of P. Choose for P HL any eigenvector from E. We may as well choose a simple vector in E so P HL K - O is our candidate. Similarly, let PHL be the second column of P, and choose for P HL any eigenvector from E. The vector P HL K O is a reasonable choice, thus So that P K - O and P- K - O - P - A P K - O K O K - O K 4 O Notice that the elements on the main diagonal of D are the eigenvalues of A, where D i i is the eigenvalue corresponding to the eigenvector P HiL. Remarks: () The first step in the diagonalization process is the determination of the eigenvalues. The ordering of the eigenvalues is purely arbitrary. If we designate l 4 and l, the columns of P would be interchanged and D would be K 4 O (see Exercise b of this section). Nonetheless, the final outcome of the application to which we are applying the diagonalization process would be the same. () If A is an nµn matrix with distinct eigenvalues, then P is also an nµn matrix whose columns P HL, P HL,, P HnL are n linearly independent vectors. Example.4.. Diagonalize the matrix -8 A l -8 detha - l IL det -l l -l - 8 H - ll det K -6 - l O H - ll HH-l - L H - ll + 8L H - ll Hl + l - L Hence, the equation detha - l IL becomes H - ll Hl + l - L - Hl - L Hl + L Therefore, our eigenvalues for A are l - and l. We note that we do not have three distinct eigenvalues, but we proceed as in the previous example. Case. For l - the equation HA - l IL x becomes Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

3 Chapter - More Matrix Algebra x x x Using Mathematica, we can row reduce the matrix: RowReduceB In equation form, the matrix equation is then equivalent to x - x x x F Therefore, the solution, or eigenspace, corresponding to l - consists of vectors of the form Therefore - - x x x x - is an eigenvector corresponding to the eigenvalue l -, and can be used for our first column of P: P HL - Before we continue we make the observation: E is a subspace of R with basis 8P HL < and dim E. Case. If l, then the equation HA - l IL x becomes x x x Without the aid of any computer technology, it should be clear that all three equations that correspond to this matrix equation are equivalent to x - x, or x x. Notice that x can take on any value, so any vector of the form x x x x will solve the matrix equation. + x We note that the solution set contains two independent variables, x and x. Further, note that we cannot express the eigenspace E as a linear combination of a single vector as in Case. However, it can be written as E :x + x x, x œ R >. We can replace any vector in a basis is with a nonzero multiple of that vector. Simply for aesthetic reasons, we will multiply the second vector that generates E by. Therefore, the eigenspace E is a subspace of R with basis :, > and so dim E. What this means with respect to the diagonalization process is that l gives us both Column and Column the diagonalizing matrix. The order is not important. Let P HL and P HL and so P - Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

4 Chapter - More Matrix Algebra The reader can verify (see Exercise 5 of this section) that P and P - A P - In doing Example.4., the given µ matrix A produced only two, not three, distinct eigenvalues, yet we were still able to diagonalize A. The reason we were able to do so was because we were able to find three linearly independent eigenvectors. Again, the main idea is to produce a matrix P that does the diagonalizing. If A is an n µ n matrix, P will be an nµn matrix, and its n columns must be linearly independent eigenvectors. The main question in the study of diagonalizability is When can it be done? This is summarized in the following theorem. Theorem.4.. Let A be an n µ n matrix. Then A is diagonalizable if and only if A has n linearly independent eigenvectors. Outline of a proof: (ì) Assume that A has linearly independent eigenvectors, P HL, P HL,, P HnL, with corresponding eigenvalues l, l,, l n. We want to prove that A is diagonalizable. Column i of the n µn matrix A P is A P HiL (see Exercise 7 of this section). Then, since the P HiL is an eigenvector of A associated with the eigenvalue l i we have A P HiL l i P HiL for i,,..., n. But this means that A P P D, where D is the diagonal matrix with diagonal entries l, l,, l n. If we multiply both sides of the equation by P - we get the desired P - A P D. (ï) The proof in this direction involves a concept that is not covered in this text (rank of a matrix); so we refer the interested reader to virtually any linear algebra text for a proof. We now give an example of a matrix which is not diagonalizable. Example.4.4. Let us attempt to diagonalize the matrix A A detha - l IL det - l - l l H - ll det K - l l O H - ll HH - ll H4 - ll + L H - ll Hl - 6 l + 9L H - ll Hl - L detha - l IL l or l - 4 Therefore there are two eigenvalues, l and l. Since l is an eigenvalue of degree it will have an eigenspace of dimension. Since l is a double root of the characteristic equation, the dimension of its eigenspace must be in order to be able to diagonalize. Case. For l, the equation HA - l IL x becomes - x x x A quick Mathematica evaluation make the solution to this system obvious RowReduce@A - IdentityMatrix@DD 4 There is one free variable, x, and Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

5 Chapter - More Matrix Algebra Hence, : -4 - x x x -4 x -x x x -4 - > is a basis for the eigenspace of l. Case. For l, the equation HA - l IL x becomes x x x RowReduce@A - IdentityMatrix@DD - Once again there is only one free variable in the row reduction and so the dimension of the eigenspace will be one: x x x x x x Hence, : > is a basis for the eigenspace of l. This means that l produces only one column for P. Since we began with only two eigenvalues, we had hoped that one of them would produce a vector space of dimension two, or, in matrix terms, two linearly independent columns of P. Since A does not have three linearly independent eigenvectors A cannot be diagonalized. Mathematica Note Diagonalization can be easily done with a few built-in functions of Mathematica. Here is a µ matrix we've selected because the eigenvalues are very simple, and could be found by hand with a little work. A ; The set of linearly independent eigenvectors of A can be computed: Eigenvectors@AD - - The rows of this matrix are the eigenvectors, so we transpose the result to get our diagonalizing matrix P whose columns are eigenvectors. P Transpose@Eigenvectors@ADD - - We then use P to diagonalize. The entries in the diagonal matrix are the eigenvalues of A. Inverse@PD.A.P 6 4 We could have gotten the eigenvalues directly this way: Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

6 Chapter - More Matrix Algebra Eigenvalues@AD 86, 4, < Most matrices that are selected at random will not have "nice" eigenvalues. Here is a new matrix A that looks similar to the one above. A ; Asking for the eigenvalues first, we see that the result is returned symbolically as the three roots to a cubic equation. The default for Mathematica is to leave these non-computed. Since the entries of A are exact numbers, Mathematica is capable of giving an exact solution, but it's very messy. The easiest way around the problem is to make the entries in A approximate. The following expression redefines A as approximate. A N@AD Now we can get approximate eigenvalues, and the approximations are very good for most purposes. Eigenvalues@AD 88.77, 7.789, 4.489< We can verify that the matrix can be diagonalized although due to round-off error some of the off-diagonal entries of the "diagonal" matrix are nonzero. P Transpose@Eigenvectors@ADD Inverse@PD.A.P µ µ µ µ µ The Chop function will set small numbers to zero. The default thresh hold for "small" is - but that can be adjusted, if desired. Diag Chop@Inverse@PD.A.PD We can't use the name D here because Mathematica reserves it for the differentiation function. If you experiment with more matrices, you will undoubtedly encounter situations where some eigenvalues are complex. The process is the same, although we've avoided these just for simplicity. Sage Note We start by defining the same matrix as we did in Mathematica. We also declare D and P to be variables. A Matrix (QQ, [[4,, ], [, 5, ], [,, 4]]);A [4 ] [ 5 ] [ 4] var (' D, P') (D, P) We have been working with "right eigenvectors" since the x in A x l x is a column vector to the right of A. It's not so common but still desirable in some situations to consider "left eigenvectors," so Sage allows either one. The right_eigenmatrix method returns a pair of matrices. The diagonal matrix, D, with eigenvalues and the diagonalizing matrix, P, which is made up of columns that are eigenvectors Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

7 Chapter - More Matrix Algebra corresponding to the eigenvectors of D. (D,P)A.right_eigenmatrix();(D,P) ( [6 ] [ ] [ 4 ] [ -] [ ], [ - ] ) We should note here that P is not unique because even if an eigenspace has dimension one, any nonzero vector in that space will serve as an eigenvector. For that reason, the P generated by Sage isn't identical to the one generated by Mathematica, but they both work. Here we verify the result for our Sage calculation. Recall that an asterisk is used for matrix multiplication in Sage. P.inverse()*A*P [6 ] [ 4 ] [ ] Here is a second matrix, again the same as we used with Mathematica. AMatrix(QQ,[[8,,],[,5,],[,,7]]);A [8 ] [ 5 ] [ 7] Here we've already specified that the underlying system is the rational numbers. Since the eigenvalues are not rational, Sage will revert to approximate number by default. We'll just pull out the matrix of eigenvectors this time and display rounded entries. Here the diagonalizing matrix looks very different from the result from Mathematica, but this is because he eigenvalues are not in the same order in the two calculations. They both diagonalize but with a different diagonal matrix. PA.right_eigenmatrix()[] P.numerical_approx(digits) [...] [ ] [ ] D(P.inverse()*A*P);D.numerical_approx(digits) [ 4.5..] [. 7.7.] [.. 8.8] EXERCISES FOR SECTION.4 A Exercises. (a) List three different eigenvectors of A K O, the matrix of Example.4., associated with the two eigenvalues and 4. Verify your results. ((b) Choose one of the three eigenvectors corresponding to and one of the three eigenvectors corresponding to 4, and show that the two chosen vectors are linearly independent.. (a) Verify that E and E in Example.4. are vector spaces over R. Since they are also subsets of R, they are called subvector-spaces, or subspaces for short, of R. Since these are subspaces consisting of eigenvectors, they are called eigenspaces. (b) Use the definition of dimension in the previous section to find dim E and dim E. Note that dim E + dim E dim R. This is not a coincidence.. (a) Verify that P - A P is indeed equal to K O, as indicated in Example (b) Choose P HL K O and PHL K - O and verify that the new value of P satisfies P- A P K 4 O (c) Take any two linearly independent eigenvectors of the matrix A of Example.4. and verify that P - A P is a diagonal matrix. Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

8 Chapter - More Matrix Algebra 4. (a) Let A be the matrix in Example.4. and P P - A P (b) If you choose the columns of P in the reverse order, what is P - A P? 5. Diagonalize the following, if possible: (a) K - O (b) K -7 6 O (c) K 4 O. Without doing any actual matrix multiplications, determine the value of (d) (e) Diagonalize the following, if possible: (f) (a) K O (b) K - O (c) K 4 O (d) B Exercise (e) (f) Let A and P be as in Example.4.. Show that the columns of the matrix A P can be found by computing A P HL, A P HL,, A P HnL. 8. Prove that if P is an nµn matrix and D is a diagonal matrix with diagonal entries d, d,, d n, then P D is the matrix obtained from P, but multiplying column i of P by d i, i,,, n. C Exercise 9. (a) There is an option to the Mathematica functions Eigenvectors and Eigenvalues called Cubics that will use the cubic equation 8 to find exact eigenvalues of a matrix like 5. Use that option to find the exact eigenvalues of the matrix. Diagonalize the matrix using 7 the Cubics option and then convert the result to a matrix of approximate numbers to compare your result with the approximate result we found in the Mathematica Note..5 Some Applications A large and varied number of applications involve computations of powers of matrices. These applications can be found in science, the social sciences, economics, the analysis of relationships with groups, engineering, and, indeed, any area where mathematics is used and, therefore, where programs are to be developed. We will consider a few diverse examples here. To aid your understanding of the following examples, we develop a helpful technique to compute A m, m >. If A can be diagonalized, then there is a matrix P such that P - A P D, where D is a diagonal matrix and A m P D m P - for all m. (.5 a) You are asked to prove this equation in Exercise 9 of Section 5.4. The condition that D be a diagonal matrix is not necessary but when it is, the calculation on the right side is particularly easy to perform. Although the formal proof of equation.4a is done by induction, the reason why it is true is easily seen by writing out an example such as m : A m HP D P - L m To get this, solve P - A P D for A and substitute HP D P - L HP D P - L HP D P - L P D HP - PL D HP - P L D P - by associativity of matrix mult. P D I D I D P - P D D D P - P D P - Example.5.: Recursion. Consider the computation of terms of the Fibonacci sequence, which we examined in Example 8..5: F, F F k F k- + F k- for k. Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

9 Chapter - More Matrix Algebra In order to formulate the calculation in matrix form, we introduced the "dummy equation" F k- F k- so that now we have two equations F k F k- + F k- F k- F k- These two equations can be expressed in matrix form as K F k F k- O K O K F k- F k- O if k We can use induction to prove that if k, K F k F k- O A k- K O A K F k- F k- O if A K O A K F k- O if k F k- etc. if k is large enough Next, by diagonalizing A and using the fact that A m P D m P -. we can show that F k k k See Exercise la of this section. Comments: () An equation of the form F k a F k- + b F k-, where a and b are given constants, is referred to linear homogeneous second-order difference equation. The conditions F c and F c, where c and c are constants, are called initial conditions. Those of you who are familiar with differential equations may recognize that the this language parallels what is used in differential equations. Difference (AKA recurrence) equations move forward discretely that is, in a finite number of positive steps while a differential equation moves continuously that is, takes an infinite number of infinitesimal steps. () A recurrence relationship of the form F k a F k- + b, where a and b are constants, is called a first-order difference equation. In order to write out the sequence, we need to know one initial condition. Equations of this type can be solved similarly to the method outlined in Example.5. by introducing the superfluous equation F k- + to obtain in matrix equation: K F k O K a b O K F k- O K F k O K a b k O K F O Example.5.: Graph Theory. Consider the graph in Figure.5.. a b c Figure.5. From the procedures outlined in Section 6.4, the adjacency matrix of this graph is A Recall that A k is the adjacency matrix of the relation r k, where r is the relation 8Ha, al, Ha, bl, Hb, al, Hb, cl, Hc, bl, Hc, cl< of the above graph. Also recall that in computing A k, we used Boolean arithmetic. What happens if we use "regular" arithmetic? For example, A How can we interpret this? We note that A and that there are two paths of length two from c (the third node) to c. Also, A, and there is one path of length from a to c. The reader should verify these claims from the graph in Figure.5.. Theorem.5.. The entry IA k M i j is the number of paths, or walks, of length k from node v i, to node v j. Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

10 Chapter - More Matrix Algebra How do we find A k for possibly large values of k? From the discussion at the beginning of this section, we know that A k P D k P - if A is diagonalizable. We leave to the reader to show that l,, and - are eigenvalues of A with eigenvectors respectively, so that where P A k P - - -,, and k P - H-L k and P See Exercise 5 of this section for the completion of this example. 6 Example.5.: Matrix Calculus. Those who have studied calculus recall that the Maclaurin series is a useful way of expressing many common functions. For example, x k k! k x Indeed, calculators and computers use these series for calculations. Given a polynomial f HxL, we defined the matrix-polynomial f HAL for square matrices in Chapter 5. Hence, we are in a position to describe A for an n µ n matrix A as a limit of polynomial. Formally, we write A I + A + A! + A! A k + º k! k Again we encounter the need to compute high powers of a matrix. Let A be an nµn diagonalizable matrix. Then there exists an invertible nµn matrix P such that P - A P D, a diagonal matrix, so that A P D P- IP D P - M k k! k D k P P - k! k The infinite sum in the middle of this final expression can be easily evaluated if D is diagonal. All entries of powers off the diagonal are zero and the i th entry of the diagonal is D k k! k i i k Di i D i i k! k For example, if A K O, the first matrix we diagonalized in Section., we found that P K - O and D K 4 O. Therefore, A K - O º K O Comments on Example.5.: () Many of the ideas of calculus can be developed using matrices. For example, if AHtL t t + 8 t e t Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

11 Chapter - More Matrix Algebra then d AHtL d t t 6 t + 8 e t () Many of the basic formulas in calculus are true in matrix calculus. For example, d HAHtL+BHtLL d t d AHtL d t and if A is a constant matrix, d A t d t A A t + d BHtL d t () Matrix calculus can be used to solve systems of differential equations in a similar manner to the procedure used in ordinary differential equations. Mathematica Note Mathematica's matrix exponential function is MatrixExp. MatrixExpBK OF H + 4 L H- + 4 L H- + 4 L H + 4 L Sage Note Sage's matrix exponential method is called exp. AMatrix(QQ,[[,],[,]]); A.exp() [ /*e + /*e^4 -/*e + /*e^4] [-/*e + /*e^4 /*e + /*e^4] EXERCISES FOR SECTION.5 A Exercises. (a) Write out all the details of Example.5. to show that the formula for F k given in the text is correct. (b) Use induction to prove the assertion made in Example.5. that K F k O A k- K F k- O. (a) Do Example 8..8 of Chapter 8 using the method outlined in Example.5.. Note that the terminology characteristic equation, characteristic polynomial, and so on, introduced in Chapter 8, comes from the language of matrix algebra, (b) What is the significance of Algorithm 8.., part c, with respect to this section?. Solve S HkL 5 S Hk - L + 4, with S HL, using the method of this section. 4. How many paths are there of length 6 between vertex and vertex in Figure.5.? How many paths from vertex to vertex of length 6 are there? Hint: The characteristic polynomial of the adjacency matrix is l 4. Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.

12 Chapter - More Matrix Algebra 4 Figure Use the matrix A of Example.5. to: (a) Determine the number of paths of length that exist from vertex a to each of the vertices in Example.5.. Verify using the graph. Do the same for vertices b and c. (b) Verify all the details of Example.5.. (c) Use Example.5. to determine the number of paths of length 4 there are from each node in the graph of Figure.5. to every node in the graph. Verify your results using the graph. 6. Let A K - - O (a) Find A (b) Recall that sin x H-L k x k H k+l! k and compute sin A. (d) Formulate a reasonable definition of the natural logarithm of a matrix and compute ln A. 7. We noted in Chapter 5 that since matrix algebra is not commutative under multiplication, certain difficulties arise. Let A K O and B K O. (a) Compute A, B, and A+B. Compare A B, B A and A+B. (b) Show that if is the µ zero matrix, then I. (c) Prove that if A and B are two matrices that do commute, then A+B A B, thereby proving that A and B commute. (d) Prove that for any matrix A, H A L - -A. 8. Another observation for adjacency matrices: For the matrix in Example.5., note that the sum of the elements in the row corresponding to the node a (that is, the first row) gives the outdegree of a. Similarly, the sum of the elements in any given column gives the indegree of the node corresponding to that column. 4 Figure.5. (a) Using the matrix A of Example.5., find the outdegree and the indegree of each node. Verify by the graph. (b) Repeat part (a) for the directed graphs in Figure.5.. Applied Discrete Structures by Alan Doerr & Kenneth Levasseur is licensed under a Creative Commons Attribution-Noncommercial-ShareAlikes. United States License.