Eigenwerte, Eigenvektoren, Diagonalisierung Vorlesungsnotizen zu Mathematische Methoden der Physik I J. Mark Heinzle Gravitational Physics, Faculty of Physics University of Vienna Version 5/5/2
2 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter. Basics Basics Consider a vector space V of dimension dimv = n over the field R (or C). A basis { b,b 2,...,b n } is a set of n linearly independent vectors. Every vector v V admits a unique decomposition w.r.t. a chosen basis, i.e., it is represented as a unique linear combination of the basis vectors v = v b + v 2 b 2 + + v n b n = n v i b i. i= An obvious but important observation is that different bases lead to different decompositions and thus to different vector components. Suppose we have a basis {ˆb,...,ˆb n } and a different basis {ˇb,...,ˇb n }. Then v = n ˆv iˆbi = i= n ˇv iˇbi, where the components (ˆv i ) i=,...,n and (ˇv i ) i=,...n are in general completely different but represent one and the same vector (w.r.t. two different bases, however). Once a basis, say { } b,b 2,...,b n, has been chosen (or if it is clear which basis is used), it is customary to collect the components of a vector v (w.r.t. that basis) into a column vector. We thus write that v is represented by w.r.t. the chosen basis. v i= v 2 v =. v n version 5/5/2 (J. Mark Heinzle, SoSe 2) 3
Chapter. Basics An endomorphism is a linear map A of the vector space V onto itself, i.e., A : V V. Such a linear map takes vectors v V and maps these to vectors A(v) V. It is customary to write Av instead of A(v) because of the linearity of A. Once a basis has been chosen (or if it is clear which basis is used), the linear map A is represented by a matrix, which we will denote by the same letter; its components are ( A ij )i,j=,...,n (w.r.t. the basis {b,b 2,...,b n }). Let us elaborate. Take an arbitrary vector v; application of the map A yields a different vector; we choose to denote this vector by v, i.e., v = Av. The map A is linear, hence v = Av = A ( n j= v jb j ) = n j= v j Ab }{{} j. b j Now, b j = Ab j is a vector in V and can thus be decomposed w.r.t. the basis, It follows that v = Av = n j= v j b j = n b j = n j= v j i= A ijb i. n i= A ijb i = n i= ( n j= A ijv j ) } {{ } v i b i. Therefore, the i =,...,n components of the transformed vector v, i.e., v,... v n, are given by v i = n j= A ijv j. We collect the components A ij into a matrix (which we choose to denote by the same letter as the linear map), i.e., A A 2 A n A = ( A ij )i,j=,...,n = A 2 A 22 A 2n.... A n A n2 A nn 4 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter. Basics This matrix representation of the linear map is very useful. Using the column vector representations of the vectors involved, i.e., we find that v v v 2 v =., v v 2 =., v n v n v A A 2 A n v v 2. = A 2 A 22 A 2n v 2...... A n A n2 A nn v n v n In other words, using column vectors and the matrix representation of A, the components of v = Av are obtained from the components of v through matrix multiplication. Although, admittedly, it is a common source of confusion to denote the linear map A and the matrix representation of A (w.r.t. a chosen basis) by the same letter, we will nonetheless stick to this convention at least in connection with those problems where the basis is regarded as fixed (i.e., when there is no change of basis to consider). It is useful to always keep in mind that while a linear map is an abstract (and fixed) entity, its matrix representation is not at all fixed but depends on the basis we choose. Let us elaborate. Consider two bases, {ˆb,...,ˆb n } and {ˇb,...,ˇb n }. The matrix representation of the linear map A w.r.t. the basis {ˆb,...,ˆb n } is   2  n  = (  ij )i,j=,...,n =  2  22  2n....,  n  n2  nn where the components of this matrix are determined through the relation Aˆb j = n i=  ijˆbi. version 5/5/2 (J. Mark Heinzle, SoSe 2) 5
Chapter. Basics The matrix representation of the linear map A w.r.t. the basis {ˇb,...,ˇb n } Ǎ Ǎ 2 Ǎ n Ǎ = ( Ǎ ij )i,j=,...,n = Ǎ 2 Ǎ 22 Ǎ 2n...., Ǎ n Ǎ n2 Ǎ nn where the components of this matrix are determined by Aˇb j = n i= Ǎ ijˇbi. The relation between the two matrices  and Ǎ is obtained by considering the change of basis, which is represented by a matrix S = (S ij ) i,j=,...,n through the relation ˇbj = n S ijˆb i, i= which is merely the decomposition of the vector ˇb j w.r.t. the basis {ˆb,...,ˆb n }. From we thus obtain Aˇb j = n i= On the other hand, Ǎ ijˇbi = n i= Aˇb j = n i= Ǎ ijˇbi Ǎ ij n k= S kiˆb k = n ( n ) Aˇb j = A S ijˆb i = n S ij Aˆb i = n S ij i= i= i= = n ( n  ki S ij )ˆbk. k= i= k= ( n n k= i= S kiǎij  kiˆbk )ˆbk. Since the two expressions are equal, we may equate the components, which yields n for all j and k, or, in matrix notation i= S kiǎij = n SǍ = ÂS. Therefore, the matrix Ǎ is obtained from  by i=  ki S ij Ǎ = S ÂS, (.) 6 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter. Basics where S is the matrix that encodes the change of basis. This switch matrix contains the basis vectors {ˇb,...,ˇb n }, represented as column vectors w.r.t. the original basis {ˆb,...,ˆb n }, as its columns, i.e., S = ˇb ˇb2 ˇbn. (.2) version 5/5/2 (J. Mark Heinzle, SoSe 2) 7
Chapter. Basics 8 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 2. Eigenvalues and eigenvectors 2 Eigenvalues and eigenvectors A vector v o of a vector space V is an eigenvector of a linear map A if it is merely stretched, compressed, or inverted by the map A, i.e., if there is a number λ (of the underlying field) such that Av = λv. Eigenvectors are the linear map s pampered children. While the linear map might have some nasty effect on a general vector (rotate, reflect,...), the map is rather kind to an eigenvector: The effect of the map on an eigenvector is to simply multiply it by a number. The number λ is called the eigenvalue that is associated with the eigenvector v. Consider the vector space R 2 (with the standard basis) and the linear map represented by the matrix ( ). 2 The vector ( ) is an eigenvector, since ( )( ) 2 The associated eigenvalue is 2. = 2 ( ). version 5/5/2 (J. Mark Heinzle, SoSe 2) 9
Chapter 2. Eigenvalues and eigenvectors The set of C functions x f(x) forms a vector space, and is a linear map. The function d dx e x is an eigenvector ( eigenfunction ) since d dx e x = e x. The associated eigenvalue is ( ). (Since the vector space of C functions is not finite dimensional, we cannot operate with matrices.) A vector v is an eigenvector of A if and only if there exists λ such that Av = λv; we write (A λ)v = o. Reinterpreting this equation we see that the set of eigenvalues of A is the set of numbers λ such that the system of linear equations possesses a non-trivial solution v. (A λ)v = o This implies a number of equivalent statements along the following lines: λ is an eigenvalue of A (A λ)v = o has non-trivial solutions v ker(a λ) { o} (A λ) is not invertible det(a λ) = Consequently, to obtain the set of eigenvalues of A, we solve the equation A λ A 2 A n A 2 A 22 λ A 2n det(a λ) = =..... A n A n2 A nn λ version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 2. Eigenvalues and eigenvectors The expression det(a λ) is called the characteristic polynomial of A. It is a polynomial of degree n, det(a λ) = ( ) n( ) λ n + c n λ n + + c λ + c, where the coefficients (c i ) i=,...,n are (complicated) expression of the components (A ij ) i,j=,...,n of A. The case of a two-dimensional vector space is particularly simple. The characteristic polynomial of A is det(a λ) = A λ A 2 A 22 λ = (A λ)(a 22 λ) A 2 A 2 A 2 = λ 2 (A + A 22 ) λ + A } {{ } A 22 A 2 A } {{ 2 } c which is a polynomial of degree 2. c, The eigenvalues of A are the zeros of the characteristic polynomial, i.e., the solutions of the equation det(a λ) = ( ) n( λ n + c n λ n + + c λ + c ) =. How many eigenvalues does a linear map A possess, then? To answer this question it is crucial to distinguish real vector spaces and complex vector spaces. Consider a vector space over the field R. Then the linear map A is represented by a real matrix (i.e., a matrix whose entries are real) and the coefficients of the characteristic polynomial are real. The eigenvalues of A are the (real!) zeros of the characteristic polynomial, which is a polynomial of degree n. Therefore, recalling basic algebra, we find that, if n is even, the number of eigenvalues of the map A can be anything between and n; if n is odd, the number of eigenvalues of the map A can be anything between and n. version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 2. Eigenvalues and eigenvectors Consider the vector space R 2 and the linear maps represented by the matrices ( ) ( ) ( ) A =, A =, A 2 =. The characteristic polynomials are det(a λ) = λ λ = λ2 + =, det(a λ) = λ λ = ( λ)2 = λ 2 2λ + =, det(a 2 λ) = λ λ = λ2 =. Therefore, A has no eigenvalue λ, A has one eigenvalue λ =, A 2 has two eigenvalues λ =, λ 2 =. We see that real (2 2) matrices can have,, or 2 eigenvalues. Consider a vector space over the field C. Then the linear map A is represented by a complex matrix (i.e., a matrix whose entries are complex) and the coefficients of the characteristic polynomial are complex. Note that this does not necessarily mean that a complex matrix features an i somewhere (but it could); e.g., both 4 + i i i and 2 2 5 are complex matrices. (Since R C the real numbers are automatically complex numbers.) The eigenvalues of A are the (complex) zeros of the characteristic polynomial. Since this is a polynomial of degree n, the number of eigenvalues of the map A can be anything between and n. (The fundamental theorem of algebra states that every polynomial has at least one (complex) zero.) 2 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 2. Eigenvalues and eigenvectors Consider the vector space C 2 and the linear maps represented by the matrices ( ) ( ) + i 3 i i 2 B =, B + i 2 =, The characteristic polynomials are det(b λ) = + i λ 3 i + i λ = ( + i λ)2 = λ 2 2( + i)λ + ( + i) 2 =, det(b 2 λ) = i λ 2 λ = (i λ)λ + 2 = λ2 iλ + 2 =. Therefore, B has one eigenvalue λ = + i, B 2 has two eigenvalues λ = 2i, λ 2 = i. We see that complex (2 2) matrices can have or 2 eigenvalues. version 5/5/2 (J. Mark Heinzle, SoSe 2) 3
Chapter 2. Eigenvalues and eigenvectors Consider the vector space C 2 and the linear maps represented by the matrices ( ) ( ) ( ) A =, A =, A 2 =. The characteristic polynomials are det(a λ) = λ λ = λ2 + =, det(a λ) = λ λ = ( λ)2 = λ 2 2λ + =, det(a 2 λ) = λ λ = λ2 =. Therefore, A has two eigenvalues λ = i, λ 2 = i, A has one eigenvalue λ =, A 2 has two eigenvalues λ =, λ 2 =. We see that complex (2 2) matrices can have or 2 eigenvalues. Like every polynomial, the characteristic polynomial can be factorized by using its roots. Suppose that there are r roots (which correspond to eigenvalues) {λ,λ 2,...,λ r }. (We know that r n when the underlying field is C; in the case of R, the set of roots might be the empty set.) Then det(a λ) = ( ) n( λ n + c n λ n + + c λ + c ) where = ( ) n( λ λ ) m ( λ λ2 ) m2 (λ λ r ) mr, m + m 2 + m r n in the case of R, m + m 2 + m r = n in the case of C. This is straightforward consequence of the fundamental theorem of algebra. 4 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 2. Eigenvalues and eigenvectors The integer numbers m,...,m r are the multiplicities of the roots λ,...,λ r ; in our present context we say that m,m 2,...,m r are the algebraic multiplicities of the eigenvalues λ,λ 2,...,λ r. Consider a linear map A of the vector space V onto itself. The field can be R or C. Suppose that λ is an eigenvalue of A. By definition, the eigenvectors associated with λ are obtained by solving the equation Av = λv, which corresponds to (A λ)v = o. Since this is a system of linear equations, the existence of non-trivial solutions v is guaranteed by the fact that det(a λ) = (which in turn follows from the fact that λ is an eigenvalue). Applying the theory of systems of linear equations we see that the solutions of (A λ)v = o form a (non-trivial) linear subspace E λ in V. Each vector v E λ satisfies the equation (A λ)v = o and is thus an eigenvector of A with eigenvalue λ. We may write E λ as or, equivalently, as E λ = { v V Av = λv }, ker(a λ). We call the space E λ the eigenspace of the map A associated with the eigenvalue λ. version 5/5/2 (J. Mark Heinzle, SoSe 2) 5
Chapter 2. Eigenvalues and eigenvectors Consider the vector space C 3 and the linear map represented by the matrix 2 A = 5 2 2. 5 2 2 Computing the eigenvalues we obtain 2 λ A λ = 2 λ 5 2 5 2 2 λ = (2 λ) 2 λ 5 2 5 2 2 λ (λ ) ) = (2 λ)( + 2 2 25 4 = (2 λ) ( λ 2 + λ 6 ) =. Accordingly, one eigenvalue is 2; the remaining eigenvalue(s) are obtained by solving the quadratic equation λ 2 + λ 6 =, ( ) 2 ± + 24 = { 3,2}. The eigenvalue 2 appears again and the number ( 3) is one more eigenvalue. Accordingly, in the present example the eigenvalues of A are λ = 2 and λ 2 = 3. The algebraic multiplicities of the eigenvalues are m = 2 and m 2 =, because the characteristic polynomial reads (λ 2) ( λ 2 + λ 6 ) = (λ 2)(λ 2)(λ + 3) = (λ 2) 2 (λ + 3). Let us compute the eigenspace E (which is the set of eigenvectors) associated with λ = 2. We solve v (A λ }{{} )v = o 5 5 2 2 v 2 =. 5 2 2 5 2 v 3 for v = (v,v 2,v 3 ) t. The only equation we get is 5 2 v 2 + 5 2 v 3 = ; hence the solution is { v E = v = v 2 } v2 = v 3. v 3 To be continued... 6 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 2. Eigenvalues and eigenvectors...and now the continuation. The eigenspace E is a two-dimensional subspace of V ; it is the space of eigenvectors of A w.r.t. the eigenvalue λ = 2. Every vector in E is an eigenvector of A w.r.t. λ ; examples are,,, 2. 2 Since E is two-dimensional, it is spanned by any two (linearly independent) vectors in E ; for instance we can write E =,, where denotes the linear span. (Recall that the linear span v,...,v n is defined as {c v + +c n v n } with constants c,...,c n.) Analogously, we compute the eigenspace E 2 associated with λ 2 = 3. 5 v (A λ }{{} 2 )v = o 5 5 2 2 v 2 =. 3 v 3 5 2 We obtain two independent equations: 5v = and 5 2 v 2 + 5 2 v 3 = ; hence the solution is { v E 2 = v = v 2 } v = v 2 = v 3. v 3 This eigenspace is spanned by one vector and thus one-dimensional; we write E 2 =. Every vector in E 2 is an eigenvector associated with the eigenvalue λ 2 = 3. To be continued... 5 2 version 5/5/2 (J. Mark Heinzle, SoSe 2) 7
Chapter 2. Eigenvalues and eigenvectors... and now the conclusion. Let us summarize. The linear map represented by the matrix 2 A = 5 2 2. 5 2 2 possesses two eigenvectors: λ = 2 and λ 2 = 3. The associated spaces of eigenvectors (eigenspaces) E and E 2 are E =,, E 2 =. The algebraic multiplicity of λ = 2 is m = 2; the algebraic multiplicity of λ 2 = 3 is m 2 =. Let us define the geometric multiplicity d λ of an eigenvalue λ as the dimension of the associated eigenspace E λ. In our example we get d = dim E = 2 and d 2 = dim E 2 =. Comparing the algebraic multiplicities with the geometric multiplicities we see that d = m = 2, d 2 = m 2 =. An obvious question to ask is whether this statement generalizes: Does the geometric multiplicity always coincide with the algebraic multiplicity? Unfortunately, as we will see in the subsequent example, the answer is no. 8 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 2. Eigenvalues and eigenvectors Consider the vector space C 3 and the linear map represented by the matrix A = 2. To compute the eigenvalues we calculate the characteristic polynomial; we obtain λ A λ = 2 λ = ( λ) λ λ 2 λ = ( λ) ( λ(2 λ) + ) = (λ + ) ( λ 2 2λ + ) = (λ + )(λ ) 2 =. Accordingly, there exist two eigenvalues, λ = and λ 2 = ; the algebraic multiplicities are m = and m 2 = 2. Computing the eigenvectors (eigenspace) associated with λ = we obtain ( A λ ) v = ( A + ) v v = 3 v 2 =, v 3 hence v v 2 =, v + 3v 2 and therefore v = and v 2 =. Accordingly, E =. Analogously, we compute the eigenvectors (eigenspace) associated with λ 2 =. We obtain ( A λ ) v = ( A ) v v = v 2 =, 2 v 3 hence v + v 2 = and v 3 =. To be continued... version 5/5/2 (J. Mark Heinzle, SoSe 2) 9
Chapter 2. Eigenvalues and eigenvectors... and now the conclusion. Accordingly, E 2 =. We see that the geometric multiplicities are d = dim E =, d 2 = dim E 2 =. In particular, we conclude that the geometric multiplicity of the eigenvalue λ 2 = is less than its algebraic multiplicity. d = m =, d 2 = < m 2 = 2. This statement is true in general. The geometric multiplicity of an eigenvalue is less than or equal to its algebraic multiplicity. Consider a linear map A of V onto itself. Let λ be an eigenvalue of A with algebraic multiplicity m λ ; let E λ = ker(a λ) denote the space of eigenvectors (eigenspace) associated with λ. We define the geometric multiplicity d λ of λ as the dimension of the associated eigenspace E λ, d λ = dim E λ. Then there is the following important statement: d λ m λ, i.e., the geometric multiplicity of an eigenvalue λ is less than or equal to its algebraic multiplicity. (The proof is not particularly difficult and thus omitted.) Suppose the linear map A possesses the eigenvalues λ,λ 2,...,λ r with geometric multiplicities d,d 2,...,d r, i.e., the associated eigenspaces E,E 2,...,E r satisfy dim E i = d i i =,...,r. So, how many linearly independent eigenvectors does the map A have? The answer is d + d 2 + + d r. Since d i m i and m + + m r = n (or n in the case of real vector spaces) we obtain d + d 2 + + d r n. 2 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 2. Eigenvalues and eigenvectors The fact that there are d + d 2 + + d r linearly independent eigenvectors is a non-trivial statement, which is intimately connected with the statement that eigenvectors associated with different eigenvalues are always linearly independent. Let us give a proof. Suppose the linear map A possesses the eigenvalues λ,λ 2,...,λ r and associated eigenspaces E,E 2,...,E r with dim E i = d i i =,...,r. Let us choose, separately, in each E i, i =,...,r, a set {v i;,...,v i;d } of linearly independent vectors, E = v ;,...,v ;d } {{ } d vectors, E2 = v 2;,...,v 2;d2 } {{ } d 2 vectors,..., Er = v r;,...,v r;dr. } {{ } d r vectors To prove that the collection of these vectors is linearly independent, we need to show that every linear combination of the kind µ ; v ; + + µ ;d v ;d + + µ r; v r; + + µ r;dr v r;dr = o (2.) is trivial, i.e., µ ; =,...,µ r;dr =. Let us apply the linear map A to (2.). Since the vectors are eigenvectors, we obtain λ ( µ; v ; + + µ ;d v ;d ) + + λr ( µr; v r; + + µ r;dr v r;dr ) = o. Dividing by λ we get µ ; v ; + + µ ;d v ;d + λ 2 λ ( ) + + λ r λ ( ) = o, (2.2) which we may subtract from (2.) to obtain ( λ 2 λ )( µ2; v 2; + +µ 2;d2 v 2;d2 ) + + ( λ r λ )( µr; v r; + +µ r;dr v r;dr ) = o. We choose to write this relation as µ 2;v 2; + + µ 2;d 2 v 2;d2 + + µ r;v r; + + µ r;d r v r;dr = o (2.3) with constants µ 2;,...,µ r;d r that are multiples of µ 2;,...,µ r;dr. Equation (2.3) has the same structure as (2.), and we may repeat the entire procedure to obtain µ 3;v 3; + + µ 3;d 3 v 3;d3 + + µ r;v r; + + µ r;d r v r;dr = o with constants µ 3;,...,µ r;d r that are multiples of µ 3;,...,µ r;dr. After a finite number of iterations we thus arrive at the equation µ r;v r; + + µ r;d r v r;dr = o version 5/5/2 (J. Mark Heinzle, SoSe 2) 2
Chapter 2. Eigenvalues and eigenvectors with constants µ r;,...,µ r;d r that are multiples of µ r;,...,µ r;dr. However, the vectors v r;,...,v r;dr are linearly independent; therefore, we find that every constant vanishes, i.e., µ r; =,...,µ r;d r =, and thus µ r; =,...,µ r;dr =. Insertion into (2.) yields µ ; v ; + +µ ;d v ;d + +µ r ; v r ; + +µ r ;dr v r ;dr = o. (2. ) Repeating the entire procedure we find that µ r ; =,...,µ r ;dr =. After a finite number of iterations we thus arrive at the conclusion that µ ; =,...,µ ;d =,...,µ r; =,...,µ r;dr =, i.e., every single constant in (2.) is zero. In other words, linear combinations of the kind (2.) are trivial which entails that the vectors are linearly independent, as claimed. Remark. There is a slight subtlety which we overlooked. One eigenvalue might be zero which makes the division impossible. However, the argument can readily be modified to this case. An alternative way of stating that there are d +d 2 + +d r linearly independent eigenvectors is E + + E r = E E r ( V ), or dim ( E E r ) = dim E + + dime r = d + + d r ( n). In brief, every eigenspace E i adds d i linearly independent eigenvectors to the set of eigenvectors (and we never have to worry that we get a linearly dependent one). We conclude this section with some useful remarks and observations. Triangular matrices Consider an (upper or lower) triangular matrix, i.e., A A 2 A 3 A n A 22 A 23 A 2n A = A 33 A 3n..... A nn. 22 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 2. Eigenvalues and eigenvectors To compute the eigenvalues of this matrix we search for the zeros of the characteristic polynomial, i.e., A λ A 2 A 3 A n A 22 λ A 23 A 2n A λ = A 33 λ A 3n.... A nn λ A 22 λ A 23 A 2n A 33 λ A 3n = (A λ).... A nn λ A 33 λ A 3n = (A λ)(a 22 λ).... A nn λ = (A λ)(a 22 λ)(a 33 λ) (A nn λ). It follows that the eigenvalues coincide with the diagonal elements of the triangular matrix, i.e., the set of eigenvalues is {A,A 22,A 33,...,A nn }. Eigenvalues, determinant, and trace Recall that the trace of a matrix is the sum of its diagonal elements, i.e., tr A = A + A 22 + + A nn = n A ii. Consider for consistency an (n-dimensional) vector space V over the field C. Let A be a linear map of V onto itself. Then there exist r n eigenvalues of A, {λ,λ 2,...,λ r }, with algebraic multiplicities m, m 2,..., m r, such that m + m 2 + + m r = n. The eigenvalues {λ,...,λ r } of a linear map A are intimately connected with its determinant and its trace: The determinant is the product of the eigenvalues, the i= version 5/5/2 (J. Mark Heinzle, SoSe 2) 23
Chapter 2. Eigenvalues and eigenvectors trace is the sum of the eigenvalues. However, we must take care of the algebraic multiplicities; an eigenvalue λ i with algebraic multiplicity m i appears m i times in the product or sum. Therefore, deta = λ λ } {{ } λ 2 λ 2 m times } {{ } m 2 times λ r λ } {{ } r = λ m λm 2 2 λ mr r = m r times r i= λ m i i, tr A = λ + + λ } {{ } + λ 2 + + λ 2 + + λ } {{ } r + + λ } {{ } r m times m 2 times m r times r = m λ + m 2 λ 2 + + m r λ r = m i λ i. The proof of these relations is not difficult if one uses the characteristic polynomial and its decomposition into its roots, i.e., i= A λ = ( ) n( λ λ ) m ( λ λ2 ) m2 (λ λ r ) mr. We restrict ourselves to the simple example of a (2 2) matrix, i.e., ( ) A A A = 2. A 2 A 22 Let λ and λ 2 denote the eigenvalues of A. (Either λ λ 2 or λ = λ 2 ; in the latter case there exists only one eigenvalue whose multiplicity is 2). We obtain A λ = A λ A 2 A 22 λ = (A λ)(a 22 λ) A 2 A 2 and, on the other hand, A 2 = λ 2 ( A + A 22 ) λ + A A 22 A 2 A 2 = λ 2 (tr A)λ + det A, (λ λ )(λ λ 2 ) = λ 2 (λ + λ 2 )λ + λ λ 2. Comparing the coefficients of the polynomials we are led to the result det A = λ λ 2, tr A = λ + λ 2. 24 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 2. Eigenvalues and eigenvectors Consider the linear map on C 2 represented by (2 2) matrix ( ) 3 A =. 3 We use det A and tr A to compute the eigenvalues of A. Accordingly, deta = 6, tr A = 4. λ λ 2 = det A = 6, λ + λ 2 = tr A = 4, which leads to a quadratic equation that can be solved to yield λ = 2 + i 2,λ 2 = 2 i 2. A simple corollary of the relation deta = is the statement that a linear map is singular (i.e., not invertible) if and only if zero is an eigenvalue of A. r i= λ m i i version 5/5/2 (J. Mark Heinzle, SoSe 2) 25
Chapter 2. Eigenvalues and eigenvectors 26 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 3. Diagonalization 3 Diagonalization Before we begin let us reiterate: A vector v of a vector space V can be represented by a column vector once a basis of V has been chosen. The column vector representation depends on the choice of basis. Likewise, a linear map A of a vector space V onto itself can be represented as a matrix once a basis of V has been chosen. The matrix representation of A depends on the choice of basis. version 5/5/2 (J. Mark Heinzle, SoSe 2) 27
Chapter 3. Diagonalization Consider the vector space R 2 and the standard basis vectors ( ) ( ) e =, e 2 =. We shall analyze the linear map A that describes a reflection at the straight line with slope 45 (i.e., a reflection at < e + e 2 >). Under this reflection, the standard basis vectors e and e 2 are mapped to Ae = A ( ) = ( ) = e 2, Ae 2 = A ( ) = ( ) = e, which is immediate from the geometry of the problem. The matrix representation of A is obtained by using the images of e and e 2, i.e., Ae and Ae 2, as columns; hence ( ) A = (3.) w.r.t. {e,e 2 }. Now let us choose a different basis {b,b 2 } and represent the linear map A w.r.t. {b,b 2 }. Choose b = e + e 2, b 2 = e e 2. We obtain (from purely geometric considerations, i.e., by applying the reflection) Ab = b, Ab 2 = b 2. Note that b and b 2 are eigenvectors of A (associated with λ = and λ 2 =, respectively). Since we have chosen a (non-standard) basis, column vectors do not quite represent what we are used to. For instance, the vector ( ) 2 v = now means v = 2 b + ( ) b 2, i.e., this vector v points along the 45 2 line. (It corresponds to w.r.t. the old standard basis.) 2 To be continued... 28 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 3. Diagonalization... and now the conclusion. Likewise, the vector ( ) v = now means v = ( ) b + b 2, i.e., this vector ( ) v points in the direction 2 of the negative x-axis. (It corresponds to w.r.t. the old standard basis.) To obtain the matrix representation of A w.r.t. the new basis {b,b 2 }, we write b and b 2 as columns vectors, ( ) ( ) b =, b 2 =. (Column vectors are w.r.t. {b,b 2 }; in particular b = b + b 2 and b 2 = b + b 2.) The transformation A, which is described by Ab = b, Ab 2 = b 2, then looks like Ab = A ( ) = ( ) = b Ab 2 = A ( ) ( ) = = b 2. The matrix representation of A is obtained by using Ab and Ab 2 as columns; hence ( ) A = (3.2) w.r.t. the basis {b,b 2 }. We obtain a different matrix representation for the same linear map. This matrix representation is preferred to the original one since the matrix is diagonal. version 5/5/2 (J. Mark Heinzle, SoSe 2) 29
Chapter 3. Diagonalization It is straightforward to make a connection between the previous example and the considerations of chapter. Set {ˆb,ˆb 2 } = {e,e 2 }, {ˇb,ˇb 2 } = {b,b 2 }. Denote by  the matrix representation of A w.r.t. the standard basis {ˆb,ˆb 2 }; in the previous example we have seen that ( )  =, see (3.). On the other hand, the matrix representation of A w.r.t. the second basis {ˇb,ˇb 2 }, which we denote by Ǎ, is ( ) Ǎ =, see (3.2). The switch matrix S is the matrix that contains the basis vectors {ˇb,ˇb 2 }, represented as column vectors w.r.t. the original basis {ˆb,ˆb 2 }, as its columns, i.e., S = ˇb ˇb2, see (.2). In our example we have ˇb = ˆb +ˆb 2 and ˇb 2 = ˆb ˆb 2, hence ( ) ( ) ˇb =, ˇb2 = w.r.t. {ˆb,ˆb 2 }, and the switch matrix becomes ( ) S =. From equation (.) we see that Ǎ = S ÂS is supposed to hold. To be continued... 3 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 3. Diagonalization...and now the conclusion. Indeed, since S = ( ) 2, we find S ÂS = ( )( )( ) = 2 which is in fact Ǎ. ( ), We call a linear map A diagonalizable, if A possesses n linearly independent eigenvectors, i.e., a basis of eigenvectors. When does this happen? Suppose that {λ,λ 2,...,λ r } are the eigenvalues of the linear map A. The associated eigenspaces (spaces of eigenvectors) are E,E 2,...,E r. The geometric multiplicity of the eigenvalue λ i is d i = dime i. We know that there exist d + d 2 + + d r ( n) linearly independent eigenvectors. Therefore, if and only if d + d 2 + + d r = n, then there exist n linearly independent eigenvectors. Alternatively we can use the condition E E r = V. An important case of diagonalizability is the case of a linear map A that possesses n different eigenvalues λ,λ 2,...,λ n (i.e., r = n). Then, automatically, there exist n linearly independent eigenvectors. (This is simply because d i = dime i i; Hence, if there exist n different eigenvalues, then d i = dim E i = i, and by the general considerations on linear independence, these eigenspaces/-vectors are linearly independent.) version 5/5/2 (J. Mark Heinzle, SoSe 2) 3
Chapter 3. Diagonalization Let us suppose that the map A is diagonalizable and let us choose a basis of (i.e., n linearly independent) eigenvectors. We do this by successively choosing bases {v i;,...,v i;di } in the eigenspaces E i, i.e., E E { }} { 2 E { }} { { r }} { V = v;,...,v ;d v2;,...,v 2;d2 vr;,...,v r;dr. } {{ }} {{ }} {{ } d vectors d 2 vectors d r vectors Since v i;j is in E i, it is an eigenvector associated with the eigenvalue λ i, i.e., Av i;j = λ i v i;j. Let us consider the matrix representation of the diagonalizable map A w.r.t. this basis of eigenvectors. Let us denote the matrix we obtain by D (instead of A). We straightforwardly obtain D = diag ( ) λ,...,λ } {{ },λ 2,...,λ 2,...,λ } {{ } r,...,λ } {{ } r d times d 2 times d r times λ... = λ λ 2... λ2... λr... λr. It therefore follows that a diagonalizable linear map can be represented by a diagonal matrix, whose entries are the eigenvalues. We call the diagonal matrix D = diag ( ) λ,...,λ } {{ },λ 2,...,λ 2,...,λ } {{ } r,...,λ } {{ } r d times d 2 times d r times the eigenvalue matrix of the linear map A. Conversely, if a map A can be represented by a diagonal matrix, then the eigenvalues are the entries of this matrix (so that the matrix is automatically the 32 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 3. Diagonalization eigenvalue matrix) and the eigenvectors of A are represented by the column vectors.,.,...,.. Hence, there exists a basis of eigenvectors and thus A is diagonalizable. Summing up, we see that a linear map A is diagonalizable if and only if it can be represented by a diagonal matrix. Consider the vector space R 3 and the linear map A represented by the Sudoku matrix 2 3 A = 4 5 6. 7 8 9 (The basis is tacitly assumed to be the standard basis.) The characteristic polynomial is A λ = λ 3 + 5λ 2 + 8λ. The eigenvalues are the zeros of the characteristic polynomial, i.e., λ =, λ 2 = 3 ( ) 2 5 + 33, λ3 = 3 ( ) 2 5 33. Since the map A has three different eigenvalues, it must have three linearly independent eigenvectors and thus a basis of eigenvectors. Therefore, the Sudoku map is diagonalizable and it can be represented by the diagonal eigenvalue matrix D = diag (, 3 2 (5 + 33), 3 2 (5 33) ). version 5/5/2 (J. Mark Heinzle, SoSe 2) 33
Chapter 3. Diagonalization Consider the vector space C 2 and the linear map A represented by matrix ( ) A =. (The basis is tacitly assumed to be the standard basis.) The eigenvalues can be read off directly, since this is a triangular matrix: There is only one eigenvalue, λ =. (Its algebraic multiplicity must be m λ = 2.) Let us compute the space of eigenvectors E λ. From we deduce that (A λ)v = ( )( v v 2 ) = ( ) E λ = ( ), dλ = dim E λ =. In particular, there is only one eigenvector (and not two linearly independent ones). There does not exist a basis of eigenvectors; therefore, the map A is not diagonalizable. In connection with the previous example we consider a rather trivial example: The identity map has one eigenvalue, λ = (with algebraic multiplicity m λ = 2). Every vector is an eigenvector for, hence E λ = C 2 and g λ = dime λ = 2. The identity map is diagonalizable (and the standard matrix representation of is already diagonal). We see that it is not a problem if an eigenvalue appears multiple times (i.e., if its algebraic multiplicity is greater than ). A problem occurs if the geometric multiplicity is strictly less than the algebraic multiplicity, d λ < m λ. In that case, r i= m i = n but r i= d i < n, whence diagonalizability is ruled out. 34 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 3. Diagonalization Consider the vector space R 2 and the linear map represented by the matrix ( ) A =. The characteristic polynomial is λ 2 + =, hence there do not exist any eigenvalues. If we consider the same map as a map on the vector space C 2, then there exist two eigenvalues: λ = i, λ 2 = i. The map is not diagonalizable as a real map, but it is in fact diagonalizable regarded as a complex map. (Since there exist two different eigenvalues, there exist two linearly independent eigenvectors.) In practice, a linear map is given in its matrix representation w.r.t. some (standard) basis, A A 2 A n A 2 A 22 A 2n A =..... A n A n2 A nn We know that, if and only if A is diagonalizable, then we can switch to a matrix representation in terms of a diagonal matrix (the eigenvalue matrix D). How do we switch in practice? We need a switch matrix S. The switch matrix is supposed to transform the standard basis to a basis of eigenvectors. On the basis of eigenvectors, the linear map then acts as a diagonal matrix (the eigenvalue matrix D). Having applied the map in this simple form, we then switch back to the standard basis. Hence, D = S AS. The switch matrix contains the eigenvectors of A as columns, i.e., S = v ; v ;2 v r;dr. To prove that D = S AS we show that SDw = ASw for all w V. Due to linearity, if we aim at proving a statement for all w V, it suffices to show this version 5/5/2 (J. Mark Heinzle, SoSe 2) 35
Chapter 3. Diagonalization statement for all basis vectors. Consider the standard basis vector e =.. We obtain On the other hand, Se = v ; ASe = λ v ;. De = λ e SDe = λ Se = λ v ;. We conclude that SDe = ASe ; analogously, we obtain SDe i = ASe i for all standard basis vectors e i and thus SDw = ASw for all w V. This completes the proof of the claim. 36 version 5/5/2 (J. Mark Heinzle, SoSe 2)
Chapter 3. Diagonalization Consider the map A on C 3 given by 3 + 2i 2 2i 4 A = + i i 2. + i i The characteristic polynomial is A λ = λ 3 + (2 + i)λ 2 ( + 2i)λ + i ; it is not difficult to convince oneself that the factorization into roots is Therefore, the eigenvalues are A λ = (λ i)(λ ) 2. λ = i (m =, d = ), λ 2 = (m 2 = 2). To see whether A is diagonalizable there must exist two linearly independent eigenvectors associated with the eigenvalue λ 2 (i.e., d 2 = dim E 2 = 2 is required). A straightforward computation shows that 2 E = i, E 2 =,, hence d 2 = dim E 2 = 2 indeed; accordingly, there exist 3 linearly independent eigenvectors and A is diagonalizable. The switch matrix S is 2 i S =, its inverse is It is straightforward to check that i i + i S = i i i. i i i S AS = D = diag(i,,). version 5/5/2 (J. Mark Heinzle, SoSe 2) 37
Chapter 3. Diagonalization 38 version 5/5/2 (J. Mark Heinzle, SoSe 2)