ES803 Topic 6 Notes Jerem Orloff 6 Eigenalues, diagonalization, decoupling This note coers topics that will take us seeral classes to get through We will look almost eclusiel at 2 2 matrices These hae almost all the features of bigger square matrices and the are computationall eas 6 Etmolog: This is from a Wikipedia discussion page: The word eigen in German or Dutch translates as inherent, characteristic, priate So eigenector of a matri is characteristic or inherent to the matri The word eigen is also translated as own with the same sense as the meanings aboe That is the eigenector of a matri is the matri s own ector In English ou sometimes see eigenalues called special or characteristic alues 62 Definition For a square matri M an eigenalue is a number (scalar) λ that satisfies the equation M λ for some non-zero ector () The ector is called a non-zero eigenector corresponding to λ We will call Equation 6 the eigenector equation Comments: Using the smbol λ for the eigenalue is a fairl common practice when looking at generic matrices If the eigenalue has a phsical interpretation we ll often use a corresponding letter For eample, in population matrices the eigenalues are growth rates, so we ll often denote them using r or k 2 Eigenectors are not unique That is, if is an eigenector with eigenalue λ then so an multiple of Indeed, the set of all eigenectors with eigenalue λ is clearl a ector space (You should show this!) 63 Wh eigenectors are special 6 5 Eample 6 Let A 2 We will eplore how A transforms ectors and what makes an eigenector special We will see that A scales and rotates most ectors, but onl scales eigenectors That is, eigenectors lie on lines that are unmoed b A Take u [ 0 Au [ 6 ; Take u 2 We see that A scales and turns most ectors 0 Au 2 5 2
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 2 5 35 Now take A 7 7 This shows that is an eigenector with eigenalue 7 The eigenector is special since A just scales it b 7 Likewise, 2 then A 2 2 So, 2 is an eigenector with eigenalue The eigenector 2 is reall special, it is unmoed b A 2 u 2 Au 2 to A (35, 7) Au u 2 4 6-2 A 2 Eample 6: Action of the matri A on ectors The following eample shows how knowing eigenalues and eigenectors simplifies calculations with a matri In fact, ou don t een need the matri once ou know all of its eigenalues and eigenectors Eample 62 Suppose A is a 2 2 matri that has eigenectors eigenalues 2 and 4 respectiel (a) Compute A 2 answer: Since is an eigenector this follows directl from the definition of eigenectors: 2 2 A 2 2 2 4 ( ) (b) Compute A + 2 3 answer: This uses the definition of eigenector plus linearit: ( ) A + A + A 2 + 4 2 3 2 3 2 3 ( (c) Compute A 3 + 5 2 ) 3 answer: Again this uses the definition of eigenector plus linearit: ( ) A 3 + 5 3A + 5A 6 + 20 2 3 2 3 2 3 2 6 6 26 72 and 3 with
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 3 (d) Compute A 0 answer: We first decompose 0 into eigenectors: 0 3 2 Now we can once again use the definition of eigenector plus linearit: ( ) 0 2 A A A A 4 + 2 3 2 3 2 3 2 8 Eample 63 An rotation in three dimensions is around some ais The ector along this ais is fied b the rotation, ie it is an eigenector with eigenalue 64 Computational approach In order to find the eigenectors and eigenalues we recall the following basic fact about square matrices Fact: The equation M 0 is satisfied b a non-zero ector if and onl if det(m) 0 (Said differentl, the null space of M is non-triial eactl when det(m) 0) First we manipulate the eigenalue equation so that finding eigenectors becomes finding a null ectors: A λ A λ I A λ I 0 (A λ I) 0 Now we can appl our fact to the last equation aboe to conclude that λ is an eigenalue if and onl if det(a λ I) 0 (2) (Recall that an definition of eigenalue requires to be non-zero) We call Equation 2 the characteristic equation (Eigenalues are sometimes called characteristic alues) It allows us to find the eigenalues and eigenectors separatel in a two step process Notation: For simplicit we will use the notation A det(a) Eigenspace: For an eigenalue λ the set of all eigenectors is none other than the nullspace of A λi That is, it is a ector subspace which we call the eigenspace of λ 6 5 Eample 64 Find the eigenalues of the matri A For each eigenalue find 2 a basis of the correpsonding eigenspace answer: Step : Find the eigenalues λ: A λi 0 (characteristic equation) 6 5 λ 0 0 2 0 λ 6 λ 5 2 λ 0
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 4 (6 λ)(2 λ) 5 λ 2 8λ + 7 0 λ 7, Step 2: Find basis eigenectors : We hae (A λi) 0, ie we want Null(A λi) 5 5 λ 7: A λi This has RREF R The nullspace is 5 0 0 5 dimensional, a basis is λ : A λi [ basis is 2 5 5 This has RREF R The nullspace is dimensional, a 0 0 Remember, an scalar multiple of these eigenectors is also an eigenector with the same eigenalue Trick In the 2 2 case we don t hae to write out the RREF to find the eigenector Notice that the entries in our eigenectors come from the entries in one row of the matri You use the right entr of the row and minus the left entr If ou think about this a moment ou ll see that it is alwas the case Matlab: In Matlab the function eig(a) returns the eigenectors and eigenalues of a matri Comple eigenalues If the eigenalues are comple then the eigenectors are comple Otherwise there is no difference in the algebra 3 4 Eample 65 Find the eigenalues and basic eigenectors of the matri A 4 3 answer: Step : Find the eigenalues λ: A λi 0 (characteristic equation) 3 4 λ 0 0 4 3 0 λ 3 λ 4 4 3 λ 0 (3 λ) 2 + 6 0 λ 3 ± 4i Step 2, Find the eigenectors : (A λi) 0 4i 4 λ 3 + 4i: 4 4i 0 Take i λ 2 3 4i: 4i 4 4 4i 2 0 Take 2 i Notice that the eigenalues and eigenectors come in comple conjugate pairs Knowing this there is no need to do a computation to find the second member of each pair 4 Eample 66 Find the eigenalues and basic eigenectors of the matri A 5 5 answer: Step : find λ (eigenalues): A λi 0 (characteristic equation) λ 4 5 5 λ 0 λ2 6λ + 25 0 λ 3 ± 4i
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 5 Step 2, find (eigenectors): (A λi) 0 2 4i 4 4 λ 3 + 4i: 5 2 4i 0 Take 2 4i 4 λ 2 3 4i: take 2 2 + 4i Repeated eigenalues When a matri has repeated eigenalues the eigenectors are not as well behaed as when the eigenalues are distinct There are two main eamples Eample 67 Defectie case Find the eigenalues and basic eigenectors of the matri A 3 0 3 answer: Step : Find the eigenalues λ: A λi 0 (characteristic equation) 3 λ 0 3 λ 0 (λ 3)2 0 λ 3, 3 Step 2, Find the eigenectors : (A λi) 0 Note: because the eigenalue is repeated, we hope to find a two dimensional eigenspace 0 λ 3: 0 0 0 This is alread in RREF It has one free ariable, so the nullspace is dimensional We can take a basis ector: 0 There are no other eigenalues and we can t find another independent eigenector with eigenalue 3 We hae two eigenalues but onl one independent eigenector, so we call this case defectie or incomplete In terms of eigenspaces and dimensions we sa the repeated eigenalue λ 3 has a one dimensional eigenspace It is defectie because we want two dimensions from the double root Eample 68 Complete case Find the eigenalues and basic eigenectors of the matri 3 0 A 0 3 answer: Step : Find the eigenalues λ: A λi 0 (characteristic equation) 3 λ 0 0 3 λ 0 (λ 3)2 0 λ 3, 3 Step 2, Find the eigenectors : (A λi) 0 0 0 λ 3: 0 0 0 This equation shows that eer ector in R 2 is an eigenector That is, the eigenalue λ 3 has a [ two dimensional eigenspace We can pick an two independent ectors as a basis, eg 0 and 0 2 (These are the simplest choices, but an two independent ectors
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 6 would work!) Because we hae as man independent eigenectors as eigenalues we call this case complete 65 Diagonal matrices In this section we will see how eas it is to work with diagonal matrices In later sections we will see how working with eigenalues and eigenectors of a matri is like turning it into a diagonal matri Eample 69 Consider the diagonal matri B Conince ourself that B second coordinate b 3 We can write this as u B 2 0 0 3 2u That is B scales the first coordinate b 2 and the 3 2 0 0 and B 0 0 3 and 0 This is eactl the definition of eigenectors That is, are eigenectors with 0 eigenalues 2 and 3 respectiel That is, for a diagonal matri the diagonal entries are the eigenalues and the eigenectors all point along the coordinate aes 66 Diagonal matrices and uncoupled algebraic sstems Eample 60 An uncoupled algebraic sstem Consider the sstem 7u 3 This barel deseres to be called a sstem The ariables u and are uncoupled and we sole the sstem b finding each ariable separatel: u /7, 3 Eample 6 Now consider the sstem 6 + 5 2 + 2 4 In matri form this is 6 5 2 (3) 2 4 6 5 The matri A is the same matri as in Eamples 6 and 64 aboe In this 2 sstem the ariables and are coupled We will eplain the logic of decoupling later For this eample, we will decouple the equations using some magical choices inoling eigenectors The eamples aboe showed that the eigenalues of A are 7 and and the eigenectors are 5 and We write our ectors in terms of the eigenectors b making the change of
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 7 ariables 5 u + 5u ; u + 2 5 We also note that + 3 Conerting and to u and we get 4 ( ) 6 5 6 5 5 5 u + 7u + 2 2 Thus 6 5 2 2 4 7u 5 + It is eas to see that the last sstem is the same as the equations 7u 3 In u, coordinates the sstem is diagonal and eas to sole 5 + 3 67 Introduction to matri methods for soling sstems of DE s In this section we will sole linear, homogeneous, constant coefficient sstems of differential equations using the matri methods we hae deeloped For now we will just consider matrices with real, distinct eigenalues In the net topic we will look at comple and repeated eigenalues As with constant coefficient DE s we will use the method of optimism to discoer a sstematic technique for soling sstems of DE s We start b giing the general 2 2 linear, homogeneous, constant coefficient sstem of DE s It has the form a + b c + d Here a, b, c, d are constants and (t), (t) are the unknown functions we need to sole for There are a number of important things to note We can write Equation 4 in matri form a b c d a b where A and c d (4) A (5) 2 The sstem is homogeneous You can see this b taking Equation 4 and putting all the and on the left side so that the right side becomes all zeros 3 The sstem is linear You should be able to check directl that a linear combination of solutions to Equation 5 is also a solution We illustrate the method of optimism for soling Equation 5 in an eample Eample 62 Sole the linear, homogeneous, constant coefficient sstem 6 5 A, where and A 2
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 8 answer: Using the method of optimism we tr a solution e λt, where λ is a constant and is a constant ector Substituting the trial solution into both sides of the DE we get λe λt e λt A A λ This is none other than the eigenalue/eigenector equation So soling the sstem amounts to finding eigenalues and eigenectors From our preious eamples we know the eigenalues and eigenectors of A We get two solutions e t, 2 e 7t 5 The general solution is the span of these solutions: c + c 2 2 c e t + c 2 e 7t [ 5 The solutions and 2 are called modal solutions Now that we know where the method of optimism leads we can do a second eample starting directl with finding eigenalues and eigenectors Eample 63 Find the general solution to the sstem 3 4 3 answer: First find eigenalues and basic eigenectors Characteristic equation: 3 λ 4 3 λ 0 λ2 6λ + 5 0 λ, 5 Eigenectors: sole (A λi) 0 [ 2 4 2 λ : 0 Take 2 2 4 λ 5: 0 Take 2 2 2 We hae two modal solutions: e λt and 2 e 5λt 2 2 2 The general solution is c + c 2 2 c e λt + c 2 e 5λt 68 Decoupling sstems of DE s Eample 64 (An uncoupled sstem) Consider the sstem u (t) 7u(t) (t) (t)
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 9 Since u and don t hae an effect on each other we sa the u and are uncoupled It s eas to see the solution to this sstem is u(t) c e 7t (t) c 2 e t In matri form we hae u [ 7 0 0 [ u The coefficient matri has eigenalues are 7 and, with eigenectors general solution to the sstem of DEs is u c e 7t + c 0 2 e t 0 0 and 0 The We see an uncoupled sstem has a diagonal coefficient matri and the eigenectors are the standard basis ectors All in all, it s simple and eas to work with The following eample shows how to decouple a coupled sstem Eample 65 Consider once again the sstem from Eample 62 6 + 5 6 5 A, where, A + 2 2 In this sstem the ariables and are coupled Make a change of ariable that conerts this to a decoupled sstem [ answer: [ From Eample 62 we know the [ eigenalues are 7 and [, the eigenectors are 5 5 and, and the general solution is c e 7t + c 2 e t Notice that c e 7t and c 2 e t in the aboe solution are just u and from the preious eample So we can write (t) 5 u(t) + (t) (7) (t) This [ is a change of ariables Notice that u and are the coefficients in the decomposition of into eigenectors Also, since and depend on t so do u and Let s rewrite the sstem in Equation 6 in terms of u, Using Equation 7 we get ( ) u 5 + 5 5 and A A u + 7u + The last equalit follows because [ 5 T and [ T are eigenectors of A Equating the two sides we get u [ 5 + 7u 5 + (6)
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 0 Comparing the coefficients of the eigenectors we get u 7u u 7 0 u 0 That is, in terms of u and the sstem is uncoupled Note that the eigenalues of A are precisel the diagonal entries of the uncoupled sstem 68 Decoupling in general To end this section we ll redo the preious eample to show a sstematic method for decoupling a sstem Eample 66 Decouple the sstem in Eample 65 5 answer: Let S matri with eigenectors of A as columns 7 0 Let Λ diagonal matri with the eigenalues of A as diagonal entries 0 In the eample we made a change of ariable using 5 u u this is just S Inerting we hae the change of ariables this gies u [ 7 0 0 u 5 + In matri form u S In Eample 65 we found that u Λ u Since u and are uncoupled the change of ariables u S is called decoupling the sstem Abstract form We finish b noting in smbolic form our sstematic method of decoupling Suppose we hae the sstem A If S is a matri with the eigenectors of A as columns and Λ is the diagonal matri with the corresponding eigenalues as diagonal entries Then the change of ariables u S changes the coupled sstem A into an uncoupled sstem u Λu Proof The ke is diagonalization which is discussed in the net section This sas that A SΛS Thus, if Su we hae A Su ASu Su SΛS Su SΛu u Λu 69 Diagonalization Decoupling b changing to coordinates based on eigenectors can be organized into what is called the diagonalization of a matri Theorem Diagonalization theorem Suppose the n n matri A has n independent eigenectors Then, we can write A SΛS,
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING where S is a matri whose columns are the n independent eigenalues and Λ is the diagonal matri whose diagonal entries are the corresponding eigenalues The proof is below We illustrate this first with our standard eample 6 5 Eample 67 We know the matri A has eigenalues 7 and with corresponding eigenectors [ 5 T and 2 [ 2 T We put the eigenectors as the columns of a matri S and the eigenalues as the entries of a diagonal matri Λ S [ 5 7 0 2, Λ 0 The diagonalization theorem sas that A SΛS 5 7 0 /6 /6 0 /6 5/6 This is called the diagonalization of A Note the form: a diagonal matri Λ surrounded b S and S Proof of the diagonalization theorem We will do this for the matri in the eample aboe It will be clear that this proof carries oer to an n n matri with n independent eigenectors We could check directl that the diagonalization equation holds Instead we will gie an argument based on matri multiplication Because the eigenectors form a basis of R 2, superposition tells us that it will be enough to show that A and SΛS hae the same effect on the eigenectors and 2 Recall that multipling a matri times a column ector results in a linear combination of the columns In our case S 0 0 2 0, likewise S 2 Inerting we hae S 0 and S 2 0 Now we can check that SΛS A and SΛS 2 A 2 : SΛS 7 0 7 SΛ S S 7 0 0 0 0 A The last equalit follows because is an eigenector of A with eigenalue 7 The equation of 2 is just the same Since the two matrices hae the same effect when multiplied b a basis, 2 of R 2 the are the same matri QED The eample shows the general method for diagonalizing an n n matri A The steps are: Find the eigenalues λ,, λ n and corresponding eigenectors,, n 2 Make the matri of eigenectors S [ 2 n
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 2 λ 0 0 0 0 λ 2 0 0 3 Make the diagonal matri of eigenalues Λ 0 0 0 0 λ n The diagonalization is: A SΛS Note: Diagonalization requires that A hae a full complement of eigenectors If it is defectie it can t be diagonalized We hae the following important formula det(a) product of its eigenalues This follows easil from the diagonalization formula det(a) det(sλs ) det(s) det(λ) det(s ) det(λ) product of diagonal entries 60 Smmetric matrices This section is optional We won t ask about it on psets or tests The first eample in this section is a nice eercise in thinking about matri multiplication as a wa to transform ectors Eample 68 Geometr of smmetric matrices [ This is a fairl comple eample a 0 showing how we can use the diagonal matri Λ and the rotation matri R 0 b cos θ sin(θ) to conert a circle to an ellipse as shown in the figures below sin(θ) cos(θ) To do this we think of matri multiplication as a linear transformation The diagonal matri Λ transforms the circle b scaling the and directions b a and b respectiel This creates the ellipse in (b) which is oriented with the aes The rotation matri R then rotates this ellipse to the general ellipse in (c) j Λj bj i u Λi ai (a) Unit circle (b) Ellipse made b scaling the aes b a and b respectiel
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 3 RΛi 2 RΛj θ (c) Ellipse in (b) rotated b θ In coordinates RΛ maps the unit circle u 2 + 2 to the ellipse shown in (c) That is, RΛ u or u Λ R Eample 69 Spectral theorem The preious eample transforms the unit circle in u-coordinates into an ellipse in -coordinates In terms of inner products and transposes this becomes [ u, u Λ R, Λ R T (Λ R ) T Λ R T RΛ 2 R The last equalit uses the facts that for a rotation matri R T R and for a diagonal matri Λ T Λ Call the matri occuring in the last two lines aboe We then hae the equation of the ellipse is A RΛ 2 R (Λ R ) T Λ R The matri A has the following properties T A
6 EIGENVALUES, DIAGONALIZATION, DECOUPLING 4 It is smmetric 2 Its eigenalues are a 2 and b 2 3 Its eigenectors are the the ectors and 2 along the aes of the ellipse (see figure (c) aboe) 4 Its eigenectors are orthogonal Proof This is clear from the formula A B T B where B Λ R 2 This is clear from the diagonalization A RΛ 2 R (Remember the eigenalues are in the diagonal matri Λ 2 3 We need to show that A transforms to a multiple of itself This also follows b considering the action of each term in the diagonalization in turn (see the figures): R moes to the -ais; then Λ 2 scales the -ais b a 2 ; and finall R rotates the -ais back the line along Using smbols A RΛ 2 R RΛ 2 ai R(a 2 ai) a 2 The properties of A are general properties of smmetric matrices Spectral theorem A smmetric matri A has the following properties It has real eigenalues 2 Its eigenectors are mutuall orthogonal Because of the connection to the aes of ellipses this is also called the principal ais theorem