D. Richard Brown III Worcester Polytechnic Institute 07-Oct-2008 Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 1 / 35
Lecture 6 Major Topics We are still in Part II of ECE504: Quantitative and qualitative analysis of systems mathematical description results about behavior of system Our focus today is on linear time-invariant systems. In this case, recall that the DT-STM is Φ[k, j] = A k j and the CT-STM is Φ(t, s) = exp{(t s)a}. We will discuss 1. Properties of A k and exp{ta}. 2. Eigenvalues and eigenvectors. 3. Computation of A k and exp{ta} when A is diagonalizable. 4. Computation of A k and exp{ta} when A is not diagonalizable. You should be finishing Chen Chapter 4 now (and referring back to Chapter 3 as necessary). You should also look over Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later by Moler and Van Loan (link on course web page). Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 2 / 35
Basic Properties of A k and exp{ta} Recall that the matrix exponential exp{ta} is not performed element-by-element. Nevertheless, the matrix exponential has many of the same properties as the usual scalar exponential. Specifically: 1. For any A R n n lim t 0 exp{ta} = I n This can be seen directly from the definition of exp{ta}. 2. For any A R n n exp{(t 1 + t 2 )A} = exp{t 1 A}exp{t 2 A} This is a consequence of the semigroup property of Φ(t,s). 3. Given A R n n and à Rn n, does exp{t(a + Ã)} = exp{ta}exp{tã}? Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 3 / 35
Basic Properties of A k and exp{ta} (cont.) 4. Given any A R n n such that 2 a 11 0 0 3 6 A = 4 0... 0 0 0 a nn 7 5 is diagonal, then exp{ta} = 2 6 4 0 0 3 0... 0 7 5 0 0 e annt e a 11t This can be seen directly from the definition of exp{ta}. 5. Given any A R n n such that we can find some invertible V C n n satisfying then V 1 AV = Λ = λ 11 0 0 0... 0 0 0 λ nn A k = (V ΛV 1 )... (V ΛV 1 ) = V Λ k V 1 }{{} k fold product Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 4 / 35
Basic Properties of A k and exp{ta} (cont.) 6. Given any A R n n such that we can find some invertible V C n n satisfying λ 11 0 0 V 1 AV = Λ =. 0.. 0 0 0 λ nn then exp{ta} = k=0 t k k! (V ΛV 1 ) k = V [ k=0 = V exp{tλ}v 1 e λ 11t 0 0 = V. 0.. 0 V 1 0 0 e λnnt ] t k k! Λk V 1 Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 5 / 35
Diagonalizability of Square Matrices Diagonalizability makes the computation of A k and exp{ta} easy! Question: Is every matrix A R n n diagonalizable? Question: Is there any connection between invertibility and diagonalizability? Question: What procedure can we use to diagonalize square matrices? Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 6 / 35
Eigenvalues Definition Given A R n n, λ 0 is an eigenvalue of A if and only if (A I n λ 0 ) is not invertible. Equivalently, based on what we already know about invertibility, we can say λ 0 is an eigenvalue of A det(a I n λ 0 ) 0 Definition The characteristic polynomial of A is det(λi n A) where λ is a variable. What is deg(det(λi n A))? It is not too hard to show that the eigenvalues of A are equivalent to the roots of the characteristic polynomial of A. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 7 / 35
Some Consequences of What We Know About Eigenvalues 1. There can be at most n different eigenvalues of A. 2. Even if A is real, its eigenvalues can be complex. 3. If A is real and λ 0 is a complex eigenvalue of A, then λ 0 is also an eigenvalue of A where the notation () means complex conjugate, i.e. z = a + jb z = a jb 4. If λ 0 is an eigenvalue of A, then dim(null(a λ 0 I n )) 1 since A λ 0 I n is not invertible. This last consequence is of particular importance. Let v 0 be any vector (except for the zero vector) in the nullspace of A λ 0 I n. Then we can say that (A λ 0 I n )v 0 = 0 which can be rewritten as Av 0 = λ 0 v 0. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 8 / 35
Eigenvectors Definition Given A R n n, v 0 is an eigenvector of A corresponding to the eigenvalue λ 0 if and only if Av 0 = λ 0 v 0. Intuition: The matrix A scales its eigenvectors by its eigenvalues. Note that there is nothing unique about an eigenvector. If v 0 is an eigenvector of A corresponding to the eigenvalue λ 0, then so is αv 0 for any α 0 since A(αv 0 ) = α(av 0 ) = α(λ 0 v 0 ) = λ 0 (αv 0 ) Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 9 / 35
Linear Independence of Eigenvectors Fact: If v 1,v 2,...,v s R n are eigenvectors of A R n n corresponding respectively with different eigenvalues λ 1,λ 2,...,λ s C, then {v 1,v 2,...,v s } is a linearly independent set. Intuitively: Eigenvectors corresponding to different eigenvalues are linearly independent. Geometrically: Let V j = null(a λ j I n ). If λ j λ m, then V j Vm = {0}. Important special case: If the matrix A R n n has n distinct eigenvalues, then there must exist n linearly independent eigenvectors {v 1,v 2,...,v n } R n. Let V = [v 1 v 2... v n ]. What can we say about the invertibility of V? Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 10 / 35
When is A is Diagonalizable? We now know that, when A has n distinct eigenvalues, A is diagonalizable and we can write A = V ΛV 1 since V = [v 1 v 2... v n ] is invertible in this case. In this case, computing exp{ta} and A k is easy. The main difficulty is finding the eigenvalues (finding the roots of a degree n polynomial). What if A does not have n distinct eigenvalues? Does this mean that A is not diagonalizable? Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 11 / 35
Example 1 0 2 A = 0 1 0 0 0 2 To determine the eigenvalues, we need to compute the roots of the characteristic polynomial, i.e. solve det(λi 3 A) = 0... Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 12 / 35
Algebraic Multiplicity of an Eigenvalue We can always write the characteristic polynomial of A in terms of its roots, i.e. det(λi n A) = (λ λ 1 ) r 1 (λ λ 2 ) r2 (λ λ s ) rs where {λ 1,...,λ s } is the set of distinct eigenvalues of A with 1 s n. Definition The algebraic multiplicity of the eigenvalue λ j of the matrix A R n n is the number of times the root λ j appears in the characteristic polynomial of A and is denoted as r j. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 13 / 35
Eigenspace and Geometric Multiplicity of an Eigenvalue Definition Given A R n n, if λ 0 C is an eigenvalue of A, then the eigenspace corresponding with λ 0, denoted as E(λ 0 ), is the subspace of C n spanned by the eigenvectors corresponding to the eigenvalue λ 0, i.e. E(λ 0 ) = null(a λ 0 I n ) Definition The geometric multiplicity of the eigenvalue λ j of the matrix A R n n is the dimension of the eigenspace of λ j and is denoted as m j, i.e. m j = dim(null(a λ j I n )) Fact: For each j {1,...,s}, 1 m j r j. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 14 / 35
When is A Diagonalizable? Theorem If, for each j {1,...,s}, m j = r j, then A is diagonalizable. This should be obvious when A has distinct eigenvalues since m j = r j = 1 for all j. Proof sketch for the case when A does not have distinct eigenvalues: Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 15 / 35
A Procedure to Know When A is Diagonalizable Does A have n distinct eigenvalues? no Compute the algebraic multiplicities of each eigenvalue { r j } yes Compute the geometric multiplicities of each eigenvalue { m j } Does m j = r j for all j? no yes A is diagonalizable A is not diagonalizable Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 16 / 35
Summary of Diagonalization 1. Compute the eigenvalues of A and denote the distinct values as {λ 1,...,λ s }. 2. If A is diagonalizable (see procedure on previous slide), then for each j {1,...,s}, find a basis for the eigenspace E(λ j ) = null(a λ j I n ). You can do this with Gaussian elimination and echelon form. Let be a basis for E(λ j ). B j = {v j1,v j2,...,v jrj } 3. Form V by stringing bases together. Note that V will be a square matrix since s j=1 r j = s j=1 m j = n. 4. Now A = V ΛV 1 (you should check it to be sure). Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 17 / 35
What Can We Do If A is Not Diagonalizable? Some options: 1. It might be possible to just compute exp{ta} by the definition, e.g. [ ] 0 1 A = 0 0 2. You can use the fundamental matrix method to compute the CT-STM Φ(t, s), and hence compute exp{(t s)a}. 3. You can still use the eigenvalue/eigenvector method except you have to work with generalized eigenvectors. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 18 / 35
Generalized Eigenvectors Definition Given A R n n and λ 0 C an eigenvalue of A, we say that v 0 C n is a generalized eigenvector corresponding with λ 0 if v 0 0 and for some integer k 1. (A λ 0 I n ) k v 0 = 0 Question: Are all regular eigenvectors also generalized eigenvectors? Question: Are all generalized eigenvectors also regular eigenvectors? Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 19 / 35
Generalized Eigenspace Definition The generalized eigenspace of the eigenvalue λ 0 is the subspace of C n spanned by all of the generalized eigenvalues corresponding to λ 0. We will use the notation F(λ 0 ) to denote the generalized eigenspace of the eigenvector λ 0. Examples... Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 20 / 35
Some Basic Properties of Generalized Eigenspaces 1. E(λ 0 ) F(λ 0 ), i.e., the regular eigenspace of the eigenvalue λ 0 is a subset of the generalized eigenspace of the eigenvalue λ 0. Why? 2. If {v 1,...,v k } is a set of generalized eigenvectors corresponding to different eigenvalues, then {v 1,...,v k } is a linearly independent set. 3. If v 0 F(λ 0 ), then Av 0 F(λ 0 ). In other words, the subspace F(λ 0 ) is invariant under A. 4. Given the characteristic polynomial of A det(λi n A) = (λ λ 1 ) r 1 (λ λ 2 ) r2 (λ λ s ) rs where λ 1,...,λ s are all distinct and r 1,...,r s are the respective algebraic multiplicities, it can be shown that dim(f(λ j )) = r j. When combined with property #1, this implies that 1 dim(e(λ j )) dim(f(λ j )) = r j. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 21 / 35
Some Basic Properties of Generalized Eigenspaces (cont.) 5. A consequence of properties #2 and #4. If {v 11,...,v 1r1 } is a basis for F(λ 1 ). {v s1,...,v 1rs } is a basis for F(λ s ) then we can string all of these sets of generalized eigenvectors together into a big set {v 11,...,v 1r1,...,v s1,...,v 1rs }. How many vectors are in this set? This set of generalized eigenvectors is a basis for C n. Why?.. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 22 / 35
Some Basic Properties of Generalized Eigenspaces (cont.) 6. From property #3, if v jk F(λ j ), then Av jk F(λ j ). Av jk can be expressed as a linear combination of the vectors comprising a basis for F(λ j ). If the basis for F(λ j ) is {v j1,...,v jrj } then Av j1 = α 11 v j1 + + α 1r1 v jrj Av jrj. = α j1 v j1 + + α jr1 v jrj We can rewrite these r j equations as one big matrix equation: α 11 α 21... α rj1 A [ ] [ ] α 12 α 22... α rj2 v j1... v jrj = vj1... v jrj }{{}}{{}...... V j V j α 1rj } α 2rj... {{ α rjr j } Q j Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 23 / 35
Some Basic Properties of Generalized Eigenspaces (cont.) Property #6 continued... We now have AV j = V j Q j. What are the dimensions of A, V j, and Q j? Let V = [V 1 V 2... V s ] and Q = Q 1... Q s What are the dimensions of V and Q? (block diagonal form) We now have AV = V Q. From property #5, what can we say about the invertibility of V? Hence, we can write A = V QV 1. Note that Q is not diagonal, but block diagonal. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 24 / 35
Some Basic Properties of Generalized Eigenspaces (cont.) 7. By the definition of generalized eigenvectors and generalized eigenspaces, the statement v F(λ j ) is equivalent to (A λ j I n ) k v = 0 (1) for some integer k 1. Note that, if (1) is true when k = k 0, then it is also true for all k k 0. This implies that F(λ j ) = null((a λ j I n ) r j ). In other words, to determine the generalized eigenspace for the eigenvalue λ j, you don t need to compute the nullspace of (A λ j I n ) k for k = 1,2,...,r j. You can just compute the nullspace of the matrix (A λ j I n ) r j by doing the standard Gaussian elimination and putting the result in echelon form. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 25 / 35
Some Basic Properties of Generalized Eigenspaces (cont.) 8. From properties #6 and #7, we can say that Why? (Q j λ j I rj ) r j = 0 Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 26 / 35
Nilpotent Matrices Definition A nilpotent matrix is a square matrix ˆN with the property that ˆN m = 0 for some positive integer m. An equivalent definition for a nilpotent matrix is a square matrix with eigenvalues all equal to zero. Examples: Which of these matrices are [ ] [ ] 0 1 1 1 A 1 = A 0 0 2 = 0 0 A 3 = [ ] 0 1 0 1 nilpotent? Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 27 / 35
Some Basic Properties of Generalized Eigenspaces (cont.) 9. From property #8 we know that Q 1 λ 1 I r1... = ˆN is nilpotent. If we let Λ = λ 1 I r1... Q s λ s I rs λ s I rs then we have Q = Λ + ˆN where Λ is diagonal and ˆN is nilpotent. We can show that Λ ˆN = ˆNΛ In other words, ˆN and Λ commute. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 28 / 35
Computing A k When A is Not Diagonalizable We now know everything we need to compute A k when A is not diagonalizable. In general, we can always write A = V (Λ + ˆN)V 1 This implies that A k = V (Λ + ˆN) k V 1 By the binomial expansion theorem and property #9, we can write k ( ) A k = V k Λ k j ˆNj V 1 j j=0 But ˆN is nilpotent. Hence, for j max{r 1,...,r s }, ˆNj = 0. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 29 / 35
Computing exp{ta} When A is Not Diagonalizable By property #9, we can write exp{ta} 1 = V exp{t(λ + ˆN)}V 1 = V exp{tλ}exp{t ˆN}V The term exp{tλ} is easy to compute because Λ is diagonal. What about the term exp{t ˆN}? Look at the definition: exp{t ˆN} = k=0 t k k! ˆN k. But ˆN is nilpotent. So the sum will only have a finite number of terms: exp{t ˆN} = max{r 1,...,r s} 1 k=0 t k k! ˆN k. In typical cases, there are only a few terms to compute. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 30 / 35
Examples Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 31 / 35
Putting it All Together A procedure for finding A k and/or exp{ta} for arbitrary A R n n : 1. Find all of the eigenvalues of A. Usually you do this by computing the roots of the characteristic polynomial, i.e. det(λi n A) = (λ λ 1 ) r 1 (λ λ 2 ) r2 (λ λ s ) rs = 0 2. For each j {1,...,s}, find a basis for E(λ j ) = null(a λ j I n ). If dim(e(λj )) = r j then good! Move on to next eigenvalue. If dim(e(λj )) > r j then you ve done something wrong. If dim(e(λj )) < r j then you need to find a basis for the generalized eigenspace F(λ j ) = null(a λ j I n ) rj. This basis must contain r j linearly independent vectors. 3. Form V C n n by stringing together all of the bases. 4. Compute V 1 AV = Q = Λ + ˆN where Λ is diagonal and ˆN is nilpotent. Note that ˆN = 0 when A is diagonalizable. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 32 / 35
Putting it All Together (cont.) 5. Compute A k via A k = V k ( ) k Λ k j ˆNj V 1 j j=0 where the nilpotent property of ˆN implies that the sum will have at most max{r 1,...,r s } 1 terms for any k. 6. Compute exp{ta} via 1 exp{ta}v exp{tλ}exp{t ˆN}V where the term exp{tλ} is easy to compute because Λ is diagonal and the term exp{t ˆN} = max{r 1,...,r s} 1 k=0 is also not too difficult since the sum is finite. t k k! ˆN k. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 33 / 35
Remarks Note that exp{tλ} will have elements that look like e λ 1t,e λ 2t,... What will the elements of exp{t ˆN} look like? Hence, when A is diagonalizable, exp{ta} will only have terms that look like e λt. When A is not diagonalizable, exp{ta} will also have terms that look like t m e λt. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 34 / 35
Conclusions This concludes our quantitative analysis of systems. We will be moving on to qualitative analysis (e.g. stability) after the midterm exam. 1. Solution to LTI or LTV discrete-time state-space difference equations (existence and uniqueness). 2. Solution to LTI or LTV continuous-time state-space differential equations (existence and uniqueness). 3. Important special case: DT LTI systems with Φ[k,j] = A k j and CT LTI systems with Φ(t, s) = exp{(t s)a}. 4. Linear algebraic tools: Subspaces Nullspace, range, rank, nullity Matrix invertibility Eigenvalues, eigenvectors, eigenspaces, and nilpotent matrices 5. Properties of A k and exp{ta}. 6. Computation of A k and exp{ta} when A is diagonalizable. 7. Computation of A k and exp{ta} when A is not diagonalizable. Worcester Polytechnic Institute D. Richard Brown III 07-Oct-2008 35 / 35