# Eigenvectors and Hermitian Operators

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 7 71 Eigenvalues and Eigenvectors Basic Definitions Let L be a linear operator on some given vector space V A scalar λ and a nonzero vector v are referred to, respectively, as an eigenvalue and corresponding eigenvector for L if and only if L(v) = λv Equivalently, we can refer to an eigenvector v and its corresponding eigenvalue λ, or to the eigen-pair (λ, v) Do note that an eigenvector is required to be nonzero 1 An eigenvalue, however, can be zero! Example 71: If L is the projection onto the vector i, L(v) = pr i (v), then L(i) = pr i (i) = i = 1 i and L(j) = pr i (j) = = j So, for this operator, i is an eigenvector with corresponding eigenvalue 1, and j is an eigenvector with corresponding eigenvalue! Example 72: Suppose V is a two-dimensional vector space with basis A = {a 1, a 2 }, and let L be the linear operator whose matrix with respect to A is [ ] 1 2 L = 3 2 Letting v = 2a 1 + 3a 2, we see that [ ][ ] L(v) = L v = = [ ] = [ ] 8 12 = 4 [ ] 2 3 = 4 v = 4v This shows that L(v) = 4v, so v is an eigenvector for L with corresponding eigenvalue 4 1 This simply is to avoid silliness After all, L() = = λ for every scalar λ 1/15/213

2 Chapter & Page: 7 2! Example 73: Let V be the vector space of all infinitely-differentiable functions, and let be the differential operator ( f ) = f Observe that (sin(2π x)) = d2 dx 2 sin(2π x) = 4π 2 sin(2π x) Thus, for this operator, 4π 2 is an eigenvalue with corresponding eigenvector sin(2π x) 2? Exercise 71: Find other eigenpairs for In practice, eigenvalues and eigenvectors are often associated with square matrices, with a scalar λ and a column matrix V being called an eigenvalue and corresponding eigenvector for a square matrix L if and only if LV = λv For example, in example 72 we saw that [ ][ ] = 4 [ ] 2 3 Thus, we would say that matrix [ ] has eigenvalue λ = 4 and corresponding eigenvector V = [ ] 2 3 This approach to eigenvalues and eigenvectors is favored by instructors and texts whose main interest is in getting their students to compute a bunch of eigenvalues and eigenvectors Unfortunately, it also obscures the basis-independent nature of the theory, as well as the basic reasons we may be interested in eigen-thingies It also pretty well limits us to the cases where the operators are only defined on finite-dimensional vector spaces Admittedly, every linear operator on a finite-dimensional space can be described in terms of a square matrix with respect to some basis, and every square matrix can be viewed as a matrix with respect to some basis for some linear operator on some finite-dimensional vector space So associating eigenpairs with matrices could be considered the same as associating eigenpairs with operators provided we only consider finite-dimensional spaces Ultimately, though, we want to consider infinite-dimensional spaces (when solving partial differential equations and generating generalized Fourier series) Also, it really is a good idea to associate eigenpairs with the single operator generating the matrices, rather than individual matrices, if only so that we can avoid having to prove such facts as Assume L and L are two square matrices related by a similarity transform Then L and L have the same eigenvalues, and their eigenvectors are related by the matrices used in the similarity transform If we associate eigenpairs with operators, then the above will not need be proven It will follow automatically from the change of basis formulas for the matrices of linear operators 2 When the eigenvector is actually a function, we often use the term eigenfunction instead of eigenvector

3 Eigenvalues and Eigenvectors Chapter & Page: 7 3 All that said, you will often (even in this class) be asked to compute the eigenvalues and corresponding eigenvectors for some N N matrix Just remember that we are really looking for the eigenpairs for the operator described by that matrix using some basis for some N-dimensional vector space, with the column matrix being called an eigenvector really being the components of a eigenvector with respect to that basis And unless otherwise specified, assume the scalars for this unspecified vector space are complex Eigenspaces and Multiple Eigenvalues Now suppose we have two eigenvectors v and w for some linear operator L, with both corresponding to the same eigenvalue λ If α and β are any two scalars, then L(αv + βw) = αl(v) + βl(w) = αλv + βλw = λ [αv + βw] This shows that any linear combination of eigenvectors corresponding to a single eigenvalue is also an eigenvector corresponding to that eigenvalue (provided the linear combination doesn t happen to yield the zero vector) Consequently, the set of all eigenvectors corresponding to a single eigenvalue is a vector space (after tossing in the zero vector) We call this space the eigenspace corresponding to the given eigenvalue? Exercise 72: Consider the projection operator in example 71 What is the eigenspace corresponding to eigenvalue 1? What is the eigenspace corresponding to eigenvalue? In practice, this means that we can describe all eigenvectors for a given eigenvalue λ by simply giving a basis for its corresponding eigenspace This also means that we can associate with each eigenvalue the dimension of its eigenspace This leads to more terminology, namely: 1 The geometric multiplicity of an eigenvalue λ is the dimension of the corresponding eigenspace 3 2 An eigenvalue λ is (geometrically) simple if its corresponding eigenspace is one dimensional Otherwise, λ is called a multiple or repeated eigenvalue If λ is a simple eigenvalue, then a basis for its eigenspace consists of a single vector v, and its eigenspace is the set of all possible scalar multiples αv of this vector If λ is a repeated eigenvalue with multiplicity N, then any basis for its eigenspace will have N vectors {v 1, v 2,,v N } Naturally, if we are actually going to do something with these basis vectors, we will want the basis for each eigenspace to be orthonormal Finding Eigenpairs (Finite-Dimensional Case) The goal is to find every scalar λ and every corresponding nonzero vector v satisfying L(v) = λv (71) where L is some linear transformation Note that this equation is completely equivalent to the equation L(v) λv = (71 ) 3 We ll discuss a slightly different notion of multiplicity for eigenvalues ( algebraic multiplicity ) in a few pages version: 1/15/213

4 Chapter & Page: 7 4 We will assume the underlying vector space is N-dimensional, and that we ve already picked out some basis for this space Letting L be the matrix for L with respect to this basis, we have that L(v) = L v and λv = λ v = λi v where, as you should also recall, I is the N N identity matrix So, in terms of components with respect to our basis, equations (71) and (71 ) become and L v = λ v (72) L v λi v = (72 ) If we have the value of λ, the first matrix equation, L v = λ v, turns out to be nothing more that a system of N linear equations (the rows of the matrix equation) with N unknowns (the components of v ) That is easily solved by various methods you should already know To see how to find the possible values of λ, look at the second matrix equation, equation (72 ) Factoring out the v on the right, this becomes If the matrix L λi were invertible, then [L λi] v = v = [L λi] 1 =, implying that v is the zero vector, contrary to the initial requirement that v be a nonzero vector So, recalling an old test for the invertibility of a matrix (see page 4 11), we see that, for a scalar λ and nonzero vector v, L(v) = λv [L λi] v = L λi is not invertible det(l λi) = Computing out the last equation (involving the determinant) yields an N th degree polynomial equation with λ as the unknown This equation is called the characteristic equation for L, and the polynomial obtained by computing out det(l λi) is called the characteristic polynomial for L The possible values of λ, then, are the solutions to this equation All this leads to the following general procedure for finding all eigenpairs (λ, v) for L : 1 Pick a convenient basis for the space and find the corresponding matrix L for L 2 Solve the characteristic equation det(l λi) = for all possible values of λ Remember, unless otherwise stated, we are assuming scalars can be complex; so don t forget to also find the possible complex values for λ

5 Eigenvalues and Eigenvectors Chapter & Page: For each λ found, set v = v 1 v 2, v N where the v k s are scalars to be determined, and then solve the N N linear system L v = λ v for the possible v k s Equivalently, you could also solve [L λi] v = Once you have found the possible v k s, write out the corresponding column matrices for v (Note 1: Because the matrix L λi is not invertible, the above systems are degenerate, and you will get arbitrary constants in your solution The total number of arbitrary constants will end up being the dimension of the corresponding eigenspace) (Note 2: In doing your computations, you may want to use symbols like x, y and z instead of v 1, v 2 and v 3 ) 4 If desired (or demanded), find a basis or even an orthonormal basis for the eigenspace corresponding to each eigenvalue In practice (in assigned problems, at least), the first step (finding the matrix for the operator with respect to some basis) has usually already been done! Example 74: Assume we have a three-dimensional vector space, a basis A = {e 1, e 2, e 3 } for that space, and a linear transformation on that space whose matrix is L A = L = Suppose, naturally, that we want to find all eigenvalues and corresponding eigenvectors for this operator First, we find the characteristic equation: det [L λi] = det 6 λ 1 = λ det 6 λ = 1 7 λ version: 1/15/213

6 Chapter & Page: λ 5 det 6 λ = 1 7 λ (1 λ)(6 λ)(7 λ) + 5(6 λ) = (6 λ) [(1 λ)(7 λ) + 5] = (6 λ) [ λ 2 8λ + 12 ] = Note that we did not completely multiply out the terms Instead, we factored out the common factor 6 λ to simplify the next step, which is to find all solutions to the characteristic equation Fortunately for us, we can easily factor the rest of the polynomial, obtaining (6 λ) [ λ 2 8λ + 12 ] = (6 λ) [(λ 2)(λ 6)] = (λ 2)(λ 6) 2 as the completely factored form for our characteristic polynomial Thus our characteristic equation det[l λi] reduces to (λ 2)(λ 6) 2 =, which we can solve by inspection We have two eigenvalues, λ = 2 and λ = 6 (with λ = 6 being a repeated root of the characteristic polynomial) To find the eigenvectors corresponding to eigenvalue λ = 2, we first write out [L λi] v = with λ = 2 and v = v A = v 1 v 2 = v 3 x y z Then [L λi] v = x y = z x 6 2 y = z 1 5 x 4 y = 1 5 z

7 Eigenvalues and Eigenvectors Chapter & Page: 7 7 So the unknowns x, y and z must satisfy x + y 5z = x + 4y + z = 1x + y + 5z = Using whichever method you like, this is easily reduced to y = and x + 5z = We can choose either x or z to be an arbitrary constant Choosing z, we then must have x = 5z to satisfy the last equation above Thus, the eigenvectors corresponding to eigenvalue r = 2 are given by v = x y = z 5z 5 = z z 1 where z can be any nonzero value That is, in terms of our basis vectors, the one-dimensional eigenspace corresponding to eigenvalue λ = 2 is the set of all constant multiples of v 1 = 5e 1 + e 3 To find the eigenvectors corresponding to eigenvalue λ = 6, we write out [L λi] v = with λ = 6 and Then v = v A = v 1 v 2 = v 3 x y z [L λi] v = x y = z 5 5 x y =, 1 1 z which reduces to x + z = and y = In this case, let us take x to be an arbitrary constant with z = x so that the first equation is satisfied The second equation is satisfied no matter what y is, so y is another arbitrary constant Thus, the eigenvectors corresponding to eigenvalue λ = 2 are given by x x x 1 v = y = y = + y = x + y 1 z x x 1 version: 1/15/213

8 Chapter & Page: 7 8 where x and y are arbitrary constants And thus, also, we see that the eigenspace corresponding to eigenvalue λ = 6 is two-dimensional, and has basis {v 2, v 2 } with v 2 = e 1 e 3 and v 3 = e 2 72 More Basic Eigen-Theory Basis Independence of the Characteristic Polynomial Before looking at the special cases where the operators are self adjoint, it is worthwhile to note that the characteristic polynomial and the corresponding characteristic equation described above are actually basis independent To see this, let A and B be any two bases for our (finitedimensional) vector space V, and let, as usual, L A and L B be the matrices of a linear operator L with respect to these two respective bases From our discussion on change of basis, we know these two matrices are related by L B = M 1 L A M where M is the matrix for the change of basis formulas ( M = M AB and M 1 = M BA = M AB if A and B are both orthonormal) Observe that M 1 [L A λi] M = M 1 [L A M λm] = M 1 L A M λm 1 M = L B λi Using this, along with properties of determinants and the fact that the value of a determinant is just a number, we get det(l B λi) = det ( M 1 [L A λi] M ) = det ( M 1) det(l A λi) det(m) = det(m) 1 det(l A λi) det(m) Cutting out the distracting middle leaves us with = det(m) 1 det(m) det(l A λi) = det(l A λi) det(l A λi) = det(l B λi) The determinants on the left and right of the last equation are the characteristic polynomials for L A and L B, respectively, and the = between them tells us that these two polynomials are the same What this means is that we have the following theorem: Theorem 71 Each linear operator L on an N-dimensional vector space V has a single associated N th degree polynomial P L, and this polynomial is given by P L (λ) = det(l λi)

9 More Basic Eigen-Theory Chapter & Page: 7 9 where L is the matrix for L with respect to any desired basis for V Moreover, λ is an eigenvalue λ for L if and only if P L (λ) = The polynomial P L (λ) and the equation P L (λ) = are, of course, referred to as the characteristic polynomial and characteristic equation for the operator L Changing the basis, and hence the matrix L, does not change the polynomial P L 4 You should be aware that any N th degree polynomial can be rewritten in completely factored form P(x) = a + a 1 x + a 2 x a N x N P(x) = a N (x r 1 ) m 1 (x r 2 ) m2 (x r K ) m K The r 1, r 2,, and r K are all the different roots of P (ie, solutions to P(r) = ) Some, or all, of these roots may be complex numbers Each m j is a positive integer, called the (algebriac) multiplicity of r j Since each of the above expressions is supposed to represent an N th degree polyomial, the number of roots, K, must be no larger than N with m 1 + m m K = N On occassion, it turns out to be more convenient to not gather like terms and to write the polynomial as P(x) = a N (x x 1 )(x x 2 )(x x 3 ) (x x N ) where m 1 of the x j s are r 1, m 2 of the x j s are r 2, and so on Keeping in mind that L is an N N matrix and I is the N N identity matrix, it is easy to verify that the completely factored form for our characteristic polynomial is P L (λ) = det(l λi) P L (λ) = ±(λ λ 1 ) m 1 (λ λ 2 ) m2 (λ λ K ) m K where the λ j s are all the different eigenvalues for L, and the ± will be + if the dimension N of the space is even, and otherwise (in practice, though, the ± is irrelevant and usually dropped) For example, the completely factored form of the characteristic polynomial from our example is (λ 2)(λ 6) 2 Not gathering like terms together, this is (λ 2)(λ 6)(λ 6) 4 Thus, the polynomial, itself, is a basis-free formula version: 1/15/213

10 Chapter & Page: 7 1 Eigenvectors as Basis Vectors Let us now suppose we were clever (or lucky) enough so that we have a basis B = { b 1, b 2,, b N } in which as least some of the b k s are eigenvectors for our linear operator L, and let us see what we can say about L B = L 11 L 12 L 13 L 1N L 21 L 22 L 23 L 2N L 31 L 32 L 33 L 3N L N1 L N2 L N3 L N N, the matrix for L with respect to basis B Remember, the components of the b k s with respect to B end up simply being b 1 B = 1, b 2 B = 1, b 3 B = 1, Now observe: b 1 is an eigenvector for L with corresponding eigenvalue λ 1 L(b 1 ) = λ 1 b 1 L B b 1 B = λ 1 b 1 B L 11 L 12 L 13 L 1N L 21 L 22 L 23 L 2N L 31 L 32 L 33 L 3N L N1 L N2 L N3 L N N 1 = λ 1 1 L 11 L 21 L 31 L N1 = λ 1

11 More Basic Eigen-Theory Chapter & Page: 7 11 So, Basis vector b 1 is an eigenvector for L with corresponding eigenvalue λ 1 L 11 L 21 L 31 L N1 = λ 1 By similar computations, you can easily derive Basis vector b 2 is an eigenvector for L with corresponding eigenvalue λ 2 L 12 L 22 L 32 L N2 = λ 2, Basis vector b 3 is an eigenvector for L with corresponding eigenvalue λ 3 L 13 L 23 L 33 L N3 = λ 3, and so on In summary, Lemma 72 Let L be a linear operator on a finite dimensional vector space, and let L B be its matrix with respect to some basis B If the k th vector in basis B is an eigenvector for L with corresponding eigenvalue λ k, then the k th column of L B is given by [L B ] jk = λ k δ jk for j = 1, 2,, N All this shows that the matrix for a linear operator L is particularly simple when as many basis vectors as possible are also eigenvectors for the operator Such a basis would be considered a natural basis for the operator If we are very lucky, there will be N independent eigenvectors (with N being the dimension of the vector space) Using those eigenvectors as the basis B, the matrix for L simplifies to the diagonal matrix L B = λ 1 λ 2 λ 3 λ N When this happens, we sometimes say that we have a complete set of eigenvectors This will certainly be the case if we have N different eigenvalues Then each eigenvalue λ k will have a corresponding eigenvector b k, and it is easy to verify that {b 1, b 2,,b N } will be a linearly version: 1/15/213

12 Chapter & Page: 7 12 independent set and hence a basis for our N-dimensional vector space On the other hand, if an eigenvalue λ k is a repeated root of the characteristic equation, then the most one can generally prove is that dimension of the eigenspace corresponding to λ j algebraic multiplicity of λ j As you should have seen in the homework, it is quite possible for dimension of the eigenspace corresponding to λ j algebraic multiplicity of λ j as a root of the characteristic polynomial as a root of the characteristic polynomial So there are linear operators and square matrices that do not have complete sets of eigenvectors (such operators or matrices are sometimes cruelly said to be defective) At this point, it is worth noting that things simplify even more if our operator/matrix is self adjoint (ie, Hermitian) Using the above lemma and the basic idea of self adjointness, you can easily verify the following corollary to the above lemma: Corollary 73 Let H be a self-adjoint linear operator on a finite dimensional vector space, and let H B be its matrix with respect to any orthonormal basis B If the k th vector in basis B is an eigenvector for H with corresponding eigenvalue λ k, then the k th column of H B is given by and the k th row of H B is given by [H B ] jk = λ k δ jk for j = 1, 2,, N, [H B ] k j = λ k δ k j for j = 1, 2,, N Since we will be using this corollary in a few pages, you really should verify it? Exercise 73: Verify the above corollary Diagonalizing a Matrix It is a fairly common exercise to diagonalize a given N N matrix L Here are the basic ideas: You view the given matrix as the matrix of some linear operator L with respect to whatever orthonormal basis A we are using as a default for our N-dimensional vector space V So, in our more explicit notation L = L A We then find a linearly independent set of N eigenvectors B = {b 1, b 2,,b N } along with their corresponding eigenvalues λ 1, λ 2,, and λ N Then, by the above, we know L B is a diagonal matrix, λ 1 λ 2 L B = λ 3, λ N

13 Self-Adjoint/Hermitian Operators and Matrices Chapter & Page: 7 13 which, for possibly obvious reasons, we may wish to denote by D Part of the exercise is to also find the change of basis matrix M for computing D from L (ie, computing L B from L A ) via D = M 1 LM This gives us 1 a natural basis for the operator, 2 the simplest possible matrix for this operator, 3 and the matrices for converting from the original basis to this natural basis and back again Of course, getting that diagonal matrix requires that the original matrix/operator has a complete set of eigenvectors to serve as that basis of eigenvectors for the operator If we are lucky, the matrix/operator will have a complete set If we are very lucky, we can even find an orthonormal set Then the change of basis matrix M will simply be the unitary matrix M AB (and M 1 = M BA = M AB ) Just why we would want all this depends on the application If, for example, this operator is the moment of inertia tensor for some physical object, then the natural basis gives you the principle axes ( axes of symmetry ) through the center of mass of the object In any case, of course, it s just plain easier to do matrix computations with diagonal matrices 73 Self-Adjoint/Hermitian Operators and Matrices Some General Background Stuff Let us quickly recall a few definitions and facts: 1 If A is a matrix, then its adjoint A is the transpose of its conjugate So [ A ] jk = ( [A] k j ) 2 Any matrix A that equals its own adjoint (ie, A = A ) is said to be self adjoint or, equivalently, Hermitian 3 More generally, if L is a linear operator on some vector space V, then its adjoint L is defined to be the linear operator satisfying L (v) a = v L(a) for every a, v V It should be noted that, since b a = a b, the above equation is completely equivalent to a L (v) = L(a) v for every a, v V If V is finite dimensional and B is an orthonormal basis, then the matrix for L with respect to B is simply L where L is the matrix for L with respect to B version: 1/15/213

14 Chapter & Page: Any linear operator L that equals its own adjoint (ie, L = L ) is said to be self adjoint or, equivalently, Hermitian 5 If L is a linear operator on some finite dimensional vector space, and L is its matrix with respect to some orthonormal basis, then L is a Hermitian operator L is a Hermitian matrix 6 If the scalars involved are simply the real numbers, then adjoints reduce to transposes, and Hermitian/self-adjoint matrices and operators can be referred to as being symmetric There are several reasons to be interested in Hermitian operators and matrices Here are a few: 1 There are some nice (and useful) mathematics we can derive regarding the eigenvalues and eigenvectors of Hermitian operators and matrices 2 Matrices and transformations arising in many applications are Hermitian (or even symmetric) because of natural symmetries As a pseudo-example, consider a matrix A defined by [A] jk = 2 φ x j x k where φ is some (sufficiently differentiable) real-valued function of several variables (x 1, x 2,, x N ) Because the order in which partial derivatives are computed does not affect the final result, we have that [A] jk = 2 φ x j x k = 2 φ x k x j = [A] k j A better example is given by the moment of inertia tensor/matrix for any given object see example 2 on page 299 of Arfken, Weber and Harris Unfortunately, they treat the moment of inertia as a matrix (and use unfortunate notation) While reading this example, try to mentally replace their moment of inertia matrix I with an operator I such that, if the given body is rotating about its center of mass with angular velocity ω, then its angular momentum and the energy in the rotation are given by I(ω ) and ω I(ω ), respectively The matrix I they have is the matrix of I with respect to some convenient orthonormal basis Change the basis (say, to one based on the major axes of rotation of the object) and you have to change the matrix, but not the operator 3 The Sturm-Liouville theory we will develop for solving partial differential equations is the infinite dimensional version of what we will develop here In fact, we will use some of what we develop here Eigenpairs for Hermitian Operators and Matrices Let H be an Hermitian operator on some vector space V Remember, this means H = H So, for every pair of vectors v and w, v H(w) = H (v) w = H(v) w

15 Self-Adjoint/Hermitian Operators and Matrices Chapter & Page: 7 15 Cutting out the middle, gives v H(w) = H(v) w, (73) an equation important enough to be given a number In particular, now, let s assume v is an eigenvector with corresponding eigenvalue λ Replacing w in the last equation with v, using the fact that H(v) = λv, and recalling the properties of inner products, we have the following: v H(v) = H(v) v v λv = λv v λ v v = λ v v λ = λ But the only way a scalar can be equal to its complex conjugate is for that scalar to be just a real number So we have just (rigorously) derived Lemma 74 ( realness of Hermitian eigenvalues) All the eigenvalues for a Hermitian operator must be real values Now let us suppose v and w are two eigenvectors corresponding to two different eigenvalues λ and µ with H(v) = λv and H(w) = µw This, combined with equation (73) gives the following: v H(w) = H(v) w v µw = λv w µ v w = λ v w (µ λ ) v w = Since we just saw that the eigenvalues of H are real, λ = λ and the last equation becomes which means that either (µ λ) v w =, µ λ = or v w = The first must be ruled out because we are assuming µ and λ are different values This leaves us with v w = verifying Lemma 75 (orthogonality of eigenvectors for Hermitian operators) Any pair of eigenectors corresponding to different eigenvalues for a Hermitian operator must be orthogonal For future applications, note that we did not assume the vector space is finite dimensional in the above version: 1/15/213

16 Chapter & Page: 7 16 The Completeness of the Set of Eigenvectors Now let consider a Hermitian operator H on a vector space V of finite dimension N If H has N different eigenvalues, then it has N different corresponding eigenvectors { v 1, v 2,, v N } As we just saw (lemma 75), this will be an orthogonal set of nonzero vectors Hence, this set will be a basis for our N-dimensional vector space Dividing each v j by its length then yields an orthonormal basis, { u 1, u 2,, u N } where u j = v j v j But what if one or more of the eigenvalues are repeated? Two-Dimensional Case Assume H is an Hermitian operator on a two-dimensional vector space V The characteristic polynomial for H can be written P H (λ) = (λ λ 1 )(λ λ 2 ) where λ 1 and λ 2 are the eigenvalues of H Let v 1 be an eigenvector corresponding to λ 1 Since V is two dimensional, we can find another vector v 2 perpendicular to v 1, and B = { u 1, u 2 } where u 1 = v 1 v 1 and u 2 = v 2 v 2 will be an orthonormal basis for our two-dimensional space, with (λ 1, u 1 ) being an eigenpair for H Now let H B be the matrix for H with respect to this basis Since (λ 1, u 1 ) is an eigenpair for H we know (corollary 73 on page 7 12) that [ ] [ ] H11 H 12 λ1 H B = = H 21 H 22 H 22 which means that (λ λ 1 )(λ λ 2 ) = P H (λ) = det [ ] λ1 λ H 22 λ = (λ λ 1 )(λ H 22 ) This tells us that H 22 = λ 2 So, in fact, H B = [ ] λ1 λ 2, which, in turn, means that H(u 2 ) B = H B u 2 B = [ λ1 λ 2 ][ ] 1 = λ 2 [ 1 ] = λ 2 u 2 B, telling us that u 2 is an eigenvector corresponding to λ 2

17 Self-Adjoint/Hermitian Operators and Matrices Chapter & Page: 7 17 Thus, If H is a Hermitian operator on a two-dimensional vector space V, then P H (λ) = (λ λ 1 )(λ λ 2 ) where λ 1 and λ 2 are the two eigenvalues for H (not necessarily different), and V has an orthonormal basis { u 1, u 2 } with each u k being an eigenvector for H corresponding to eignevalue λ k Now let us verify that a similar statement can be made when V is three-dimensional Many of the details will be left to you Three-Dimensional Case Assume H is an Hermitian operator on a three-dimensional vector space V We know the characteristic polynomial for H can be written P H (λ) = (λ λ 1 )(λ λ 2 )(λ λ 3 ) where λ 1, λ 2 and λ 3 are the (not necessarily different) eigenvalues of H Let v 3 be an eigenvector corresponding to λ 3 By a straightforward modification of the Gram-Schmidt procedure, we can find an orthonormal basis for V with B = { w 1, w 2, u 3 } u 3 = v 3 v 3 So (λ 3, u 3 ) will be an eigen-pair for H Now let H B be the matrix for H with respect to this basis? Exercise 74: Show that the matrix for H with respect to B is H 11 H 12 H B = H 21 H 22, λ 3 with being a 2 2 Hermitian matrix H = [ ] H11 H 12 H 21 H 22 Now let V be the two-dimensional subspace of V with basis B = { w 1, w 2 }, and let H be the linear operator on V whose matrix with respect to B is H Since H is Hermitian, so is H version: 1/15/213

18 Chapter & Page: 7 18? Exercise 75: Starting with the fact that P H (λ) = det(h B λi), show that the characteristic polynomial for H is? Exercise 76: P H (λ) = (λ λ 1 )(λ λ 2 ) Let a be any vector in the subspace V Show that: a: The set {a, u 3 } is orthogonal b: H(a) = H (a) (Hint: Use the matrices H B and H along with the components of a with respect to basis B ) c: If a is an eigenvector for H with eigenvalue λ, then a is also an eigenvector for H with eigenvalue λ Now, by our discussion of the two-dimensional case, above, we know there is an orthonormal basis for V { u 1, u 2 } such that (λ 1, u 1 ) and (λ 2, u 2 ) are eigen-pairs for H As you just verified in the last exercise, (λ 1, u 1 ) and (λ 2, u 2 ) are then eigen-pairs for H as well Moreover, from the first part of the last exercise, it also follows that { u 1, u 2, u 3 } is an orthonormal basis for V And since we started with (λ 3, u 3 ) being an eigen-pair for H, it should be clear that we have the following: If H is a Hermitian operator on a three-dimensional vector space V, then P H (λ) = (λ λ 1 )(λ λ 2 )(λ λ 3 ) where λ 1, λ 2 and λ 3 are the three eigenvalues for H (not necessarily different), and V has an orthonormal basis { u 1, u 2, u 3 } with each u k being an eigenvector for H corresponding to eignevalue λ k The General Case Continuing the above yields: Lemma 76 If H is a Hermitian operator on an N-dimensional vector space V, then P H (λ) = ±(λ λ 1 )(λ λ 2 )(λ λ 3 ) (λ λ N ) where λ 1, λ 2, and λ N an orthonormal basis are the N eigenvalues for H (not necessarily different), and V has { u 1, u 2,, u N }

19 Self-Adjoint/Hermitian Operators and Matrices Chapter & Page: 7 19 with each u k being an eigenvector for H corresponding to eigenvalue λ k It is worth noting that if, say, m of the above λ j s are the same value, then the lemma assures us that there are m corresponding u j s in a basis for V Hence: Corollary 77 If H is a Hermitian operator on an N-dimensional vector space V with characteristic polynomial P H (λ) = ±(λ λ 1 ) m 1 (λ λ 2 ) m 2 (λ λ 3 ) m3 (λ λ K ) m K where λ 1, λ 2, and λ K are all the different eigenvalues H, then the dimension of the eigenspace for each λ j is the algebraic multiplicity, m j, of that eigenvalue In other words, if λ is an eigenvalue for a Hermitian operator, then The dimension of the eigenspace corresponding to λ = the algebraic multiplicity of λ as a root of the characteristic polynomial Compare this to the corresponding statement made a few pages ago (page 7 12) for the case where λ is an eigenvalue for an arbitrary linear operator on some vector space: The dimension of the eigenspace corresponding to λ j the algebraic multiplicity of λ j as a root of the characteristic polynomial Summary of the Big Results We have too many lemmas regarding Hermitian operators Let s summarize them in one theorem so that, in the future, we can forget the lemmas and just refer to the one theorem Theorem 78 (Big Theorem on Hermitian Operators) Let H be a Hermitian (ie, self-adjoint) operator on a vector space V Then: 1 All eigenvalues of H are real 2 Any pair of eigenvectors corresponding to different eigenvalues is orthogonal Moreover, if V is finite dimensional, then 1 If λ is an eigenvalue of algebraic multiplicity m in the characteristic polynomial, then we can find an orthonormal set of exactly m eigenvectors whose linear combinations generate all other eigenvectors corresponding to λ 2 V has an orthonormal basis consisting of eigenvectors for H version: 1/15/213

20 Chapter & Page: Diagonalizing Hermitian Matrices and Operators General Ideas and Procedures The above big theorem on Hermitian operators, along with what we know know about change of basis (especially theorem 61 on page 6 8) and the matrix of an operator with respect to a basis of eigenvectors (lemma 72 on page 7 11 or corollary 73 on page 7 12), gives us the following result on diagonalizing Hermitian operators/matrices Corollary 79 Let H be a Hermitian operator on a vector space V of finite dimension N, and let H A be the matrix for H with respect to some orthonormal basis A Let U = { u 1, u 2,, u N } be an orthonormal basis for V consisting of eigenvectors for H Then the matrix for H with respect to this basis of eigenvectors is given by H U = M UA H A M AU = λ 1 λ 2 λ 3 λ N where each λ j is the eigenvalue corresponding to eigenvector u j, and M AU and M UA are the unitary change of bases matrices described in theorem 53 on page 5 9 In light of this corollary, diagonalizing a Hermitian operator H simply means the finding of an orthonormal basis of eigenvectors U so that the matrix of this operator with respect to this basis is the diagonal matrix H U given in the corollary If on the other hand, we are asked to diagonalize a Hermitian matrix H, then we are to find a unitary matrix U and a diagonal matrix D so that D = U HU The above corollary tells us that, treating H as H A, the matrix for a corresponding operator with respect to the original basis, we can use D = H U and U = M AU Let us now consider the mechanics of diagonalizing a Hermitian matrix/operator Assume we have either a Hermitian operator H on an N-dimensional vector space V, or an N N Hermitian matrix H If it is an operator H that we have, then the first thing is to find its matrix H = H A with respect some convenient orthonormal basis A = { a 1, a 2,, a N }

21 Diagonalizing Hermitian Matrices and Operators Chapter & Page: 7 21 If, instead, we are starting with a matrix H, then the first thing is to view it as H = H A, the matrix of a Hermitian operator H with respect to some convenient orthonormal basis A = { a 1, a 2,, a N } In practice, of course, the vectors in our convenient orthonormal basis A will usually be a 1 = i, a 2 = j, For example, let s suppose we ve been asked to diagonalize the Hermitian matrix H = Here, we have a 3 3 Hermitian matrix H which we will view as the matrix H A of a Hermitian operator H with respect to A = { a 1, a 2, a 3 } = { i, j, k } The first step after getting the matrix H is to find all of its eigenvalues (and their multiplicities) by solving det(h λi) = (If N 3, this can be a challenge) For our example, det(h λi) = det λ 1 = λ 2 2 det 2 5 λ 4 = λ = [ λ 3 18λ λ ] = λ(λ 9) 2 = (λ )(λ 9) 2 = So, and λ = is a simple eigenvalue for H, λ = 9 is an eigenvalue of multiplicity 2 for H version: 1/15/213

22 Chapter & Page: 7 22 For each simple eigenvalue λ (ie, each eigenvalue with multiplicity 1 ), we merely need to solve either H v A = λ v A or [H λi] v A = A (our choice) for the components with respect to A of any eigenvector v corresponding to that eigenvalue λ Since this eigenvalue has multiplicity 1, its eigenspace will be one-dimensional, which means that you should get an expression for v A involving exactly one arbitrary constant Set that constant equal to any convenient (nonzero) value to get a eigenvector v, and then normalize v (ie, divide by its length) to get a unit eigenvector u corresponding to eigenvalue λ For our example, the only simple eigenvalue is λ = Let v 1, v 2 and v 3 be the components with respect to basis A of any corresponding eigenvector v Then v 1 v 1 H v = λ v v 2 = v Multiplying through then yields the system 8v 1 2v 2 2v 3 = 2v 1 + 5v 2 4v 3 = v 3 2v 1 4v 2 + 5v 3 =, which, after appropriately multiplying the equations by appropriate scalars and adding/subtracting various equations, reduces to the system 2v 1 + v 2 1v 3 = 1v 2 1v 3 = = As noted above, there will be exactly one arbitrary constant since λ = is a simple eigenvalue We can use v 1 or v 2 or v 3 as that constant in this case Choosing v 1 will be convenient Then, by the last set of equations, we have that So v 3 = 2v 1 and v 2 = v 3 = 2v 1 v = v 1 v 2 v 3 In particular, (with v 1 = 1 ) we have that v 1 1 = 2v 1 = v 1 2 2v 1 2 v = 1i + 2j + 2k is an eigenvector corresponding to λ = Normalizing this eigenvector then gives our first unit eigenvector, u 1 = 1 [ ] 1i + 2j + 2k 3 Remember, it corresponds to eigenvalue λ 1 = (It s a good idea to start indexing the unit eigenvectors and corresponding eigenvalues) v 3

23 Diagonalizing Hermitian Matrices and Operators Chapter & Page: 7 23 For each eigenvalue λ with multiplicity m > 1, we still need to solve either H v A = λ v A or [H λi] v A = A (our choice) for the components with respect to A of any eigenvector v corresponding to that eigenvalue λ 5 Since this eigenvalue has multiplicity m, its eigenspace will be m-dimensional Thus, any basis for this eigenspace will contain m vectors This means that you should get an expression for v A involving exactly m arbitrary constants call them α 1, α 2,, and α m, for now Now choose m sets of values for these constants to generate a linearly independent set of m eigenvectors { v 1, v 2,, v m } For example, v 1 could be the result of choosing v 2 could be the result of choosing α k = α k = { 1 if k = 1 otherwise { 1 if k = 2 otherwise,, and so on With great luck (or skill on your part) the resulting set of m eigenvectors will be orthonormal If not, construct an orthonormal set from the v j s using the Gram-Schmidt procedure (or any other legitimate trick) For our example, λ = 9 is an eigenvalue of multiplicity 2 Let v 1, v 2 and v 3 be the components with respect to basis A of any eigenvector v corresponding to this eigenvalue Then [H λi] v A = A v v 2 = v v v 2 = Multiplying through gives the system But each of these equations is just 1v 1 2v 2 2v 3 = 2v 1 4v 2 4v 3 = v 3 2v 1 4v 2 4v 3 = v 1 + 2v 2 + 2v 3 = 5 Unless you can use one or more of the cheap tricks discussed later version: 1/15/213

24 Chapter & Page: 7 24 multiplied by either 1 or 2 Since the eigenvalue here has multiplicity 2, we can choose any two of the components to be arbitrary constants Choosing v 2 and v 3 to be our two arbitrary constants means that v 3 v 1 = 2v 2 2v 3, and that every eigenvector v corresponding to λ = 9 can be written v 1 2v 2 2v v A = v 2 = v 2 = v v 3 1 v 3 Setting (v 2,v 3 ) = (1, ) and then (v 2,v 3 ) = (, 1) gives two eigenvectors v 2 and v 3 with 2 2 v 2 A = 1 and v 3 A = 1 That is, v 2 = 2i + j and v 3 = 2i + k But {v 2, v 3 } is not orthonormal To get a corresponding orthonormal pair {u 2, u 3 } of eigenvectors corresponding to λ = 9, we ll use the Gram-Schmidt procedure Hence, we ll set u 2 = v 2 v 2 = = 1 [ ] 2i + j 5 and where Thus, u 3 = w 3 w 3 w 3 = v 3 u 2 v 3 u 2 = = 1 5 [ 2i 4j + 5k ] u 3 = w 3 w 3 = = 1 [ ] 3 2i 4j + 5k 5 Eventually, you will have used up all the eigenvalues, and found an orthonormal set of N eigenvectors U = { u 1, u 2,, u N } This is a basis for our vector space Let {λ 1, λ 2,, λ N } be the corresponding set of eigenvalues (with λ k being the eigenvalue corresponding to eigenvector u k ) If we are diagonalizing the Hermitian operator H, then, as we already noted, U is the desired basis for our vector space, and λ 1 λ 2 H U = λ 3 λ N

25 Diagonalizing Hermitian Matrices and Operators Chapter & Page: 7 25 is our desired diagonal matrix Writing these facts down (with the values found) completes the task If we are diagonalizing the Hermitian matrix H, then, as we already noted, we have a diagonal matrix D and a unitary matrix U such that by just setting U = M AU and D = H U = D = U HU λ 1 λ 2 λ 3 λ N Computing M AU, and writing out the resulting U and D completes the task Don t forget the true reason for doing all this: H U is the matrix for the operator H with respect to the orthonormal basis U This basis is a natural basis for the operator, and H U is the simplest description for this operator For our example, we have obtained the orthonormal basis of eigenvectors with corresponding eigenvalues B = { u 1, u 2, u 3 } λ 1 =, λ 2 = 9 and λ 3 = 9 We can compute M AU either using the basic definition, [M AU ] jk = a j u k, or, more easily, by observing that our formulas for the u k s can be written as [ u 1, u 2, u 3 ] = [ i, j, k ] 2 1 }{{} [ a 1, a 2, a 3 ] }{{} M AU So D = U HU with λ 1 D = H U = λ 2 = 9 λ 3 9 and U = M AU = version: 1/15/213

26 Chapter & Page: 7 26 Cheap Tricks Don t forget that the u k s discussed above form an orthonormal basis You can often use this fact to shortcut your computations by using one or more of the following cheap tricks : 1 If V can be viewed as a three-dimensional space of traditional vectors, and an orthonormal pair of eigenvectors {u 1, u 2 } has already been found, then the third vector in the orthonormal basis {u 1, u 2, u 3 } can be found via the cross product, u 3 = u 1 u 2 2 If the u k s have been found for all but one value of the eigenvalues, then the remaining eigenvectors can be chosen by just picking vectors orthogonal to those already found By the theory we ve developed, we know these will be eigenvectors corresponding to our last eigenvalue In our example above, we found that the simple eigenvalue λ 1 = has corresponding unit eigenvector u 1 = 1 [ ] 1i + 2j + 2k 3 For {u 2, u 3 }, the orthonormal pair corresponding to the double eigenvalue λ = 9, we can first pick any unit vector orthogonal to u 1, say, u 2 = 1 2 [ j k ] (you should verify that this is a unit vector orthogonal to u 1 ), and then set u 3 = u 1 u 2 = = [ 4i + j + k ] 75 Spectral Decomposition Let V be an N-dimensional vector spacehaving an orthonormal basis B = { e 1, e 2,, e N } We now know that, given any Hermitian operator H on V, there is an orthonormal basis of eigenvectors for H U = { u 1, u 2,, u N } We also know that the matrix of H with respect to U is simply λ 1 λ 2 H U = D = λ 3 λ N

27 Spectral Decomposition Chapter & Page: 7 27 where { λ 1, λ 2,, λ N } is the set of eigenvalues corresponding to U We even know that this matrix is related to the matrix of H with respect to basis B via Equivalently, D = M UB H B M BU H B = M BU DM UB Now, notice that the entries in D are given by [D] mn = λ m δ mn Using this, and various things from previous chapters, [H B ] jk = [M BU DM UB ] jk N N = [M BU ] jm [D] mn [M UB ] mk m=1 n=1 N N = e j u m λm δ mn u n e k m=1 n=1 N = e j u m λm u m e k m=1 Rewritten in terms of the bra and ket matrices of components, this is N [H B ] jk = e j B u m B λ m u m B e k B m=1 ( N ) = e j B u m B λ m u m B e k B = m=1 [ N ] u m B λ m u m B m=1 jk In other words, H B = N u m B λ m u m B (74) m=1 This equation is just another way of writing H B = M BU DM UB Do note that both this and (74) are formulas that are independent of the choice of original orthonormal basis B By the way, the set of eigenvalues of an operator is said to be the spectrum of that operator, and formula (74) is called the spectral decomposition of Hermitian operator H version: 1/15/213

### Definition (T -invariant subspace) Example. Example

Eigenvalues, Eigenvectors, Similarity, and Diagonalization We now turn our attention to linear transformations of the form T : V V. To better understand the effect of T on the vector space V, we begin

### Math Linear Algebra II. 1. Inner Products and Norms

Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,

### Chapter Two Elements of Linear Algebra

Chapter Two Elements of Linear Algebra Previously, in chapter one, we have considered single first order differential equations involving a single unknown function. In the next chapter we will begin to

### Page 52. Lecture 3: Inner Product Spaces Dual Spaces, Dirac Notation, and Adjoints Date Revised: 2008/10/03 Date Given: 2008/10/03

Page 5 Lecture : Inner Product Spaces Dual Spaces, Dirac Notation, and Adjoints Date Revised: 008/10/0 Date Given: 008/10/0 Inner Product Spaces: Definitions Section. Mathematical Preliminaries: Inner

### 22m:033 Notes: 7.1 Diagonalization of Symmetric Matrices

m:33 Notes: 7. Diagonalization of Symmetric Matrices Dennis Roseman University of Iowa Iowa City, IA http://www.math.uiowa.edu/ roseman May 3, Symmetric matrices Definition. A symmetric matrix is a matrix

### 1 Last time: least-squares problems

MATH Linear algebra (Fall 07) Lecture Last time: least-squares problems Definition. If A is an m n matrix and b R m, then a least-squares solution to the linear system Ax = b is a vector x R n such that

### Mathematical Methods wk 2: Linear Operators

John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm

### MAT1302F Mathematical Methods II Lecture 19

MAT302F Mathematical Methods II Lecture 9 Aaron Christie 2 April 205 Eigenvectors, Eigenvalues, and Diagonalization Now that the basic theory of eigenvalues and eigenvectors is in place most importantly

### 4. Linear transformations as a vector space 17

4 Linear transformations as a vector space 17 d) 1 2 0 0 1 2 0 0 1 0 0 0 1 2 3 4 32 Let a linear transformation in R 2 be the reflection in the line = x 2 Find its matrix 33 For each linear transformation

### Topics in linear algebra

Chapter 6 Topics in linear algebra 6.1 Change of basis I want to remind you of one of the basic ideas in linear algebra: change of basis. Let F be a field, V and W be finite dimensional vector spaces over

### 18.06 Problem Set 8 - Solutions Due Wednesday, 14 November 2007 at 4 pm in

806 Problem Set 8 - Solutions Due Wednesday, 4 November 2007 at 4 pm in 2-06 08 03 Problem : 205+5+5+5 Consider the matrix A 02 07 a Check that A is a positive Markov matrix, and find its steady state

### October 25, 2013 INNER PRODUCT SPACES

October 25, 2013 INNER PRODUCT SPACES RODICA D. COSTIN Contents 1. Inner product 2 1.1. Inner product 2 1.2. Inner product spaces 4 2. Orthogonal bases 5 2.1. Existence of an orthogonal basis 7 2.2. Orthogonal

### There are two things that are particularly nice about the first basis

Orthogonality and the Gram-Schmidt Process In Chapter 4, we spent a great deal of time studying the problem of finding a basis for a vector space We know that a basis for a vector space can potentially

### Chapter 4 & 5: Vector Spaces & Linear Transformations

Chapter 4 & 5: Vector Spaces & Linear Transformations Philip Gressman University of Pennsylvania Philip Gressman Math 240 002 2014C: Chapters 4 & 5 1 / 40 Objective The purpose of Chapter 4 is to think

### MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

### BASIC ALGORITHMS IN LINEAR ALGEBRA. Matrices and Applications of Gaussian Elimination. A 2 x. A T m x. A 1 x A T 1. A m x

BASIC ALGORITHMS IN LINEAR ALGEBRA STEVEN DALE CUTKOSKY Matrices and Applications of Gaussian Elimination Systems of Equations Suppose that A is an n n matrix with coefficents in a field F, and x = (x,,

### Homogeneous Constant Matrix Systems, Part II

4 Homogeneous Constant Matrix Systems, Part II Let us now expand our discussions begun in the previous chapter, and consider homogeneous constant matrix systems whose matrices either have complex eigenvalues

### Dot Products. K. Behrend. April 3, Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem.

Dot Products K. Behrend April 3, 008 Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem. Contents The dot product 3. Length of a vector........................

### Eigenvalues and Eigenvectors

LECTURE 3 Eigenvalues and Eigenvectors Definition 3.. Let A be an n n matrix. The eigenvalue-eigenvector problem for A is the problem of finding numbers λ and vectors v R 3 such that Av = λv. If λ, v are

### Linear Algebra Massoud Malek

CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

### Answers in blue. If you have questions or spot an error, let me know. 1. Find all matrices that commute with A =. 4 3

Answers in blue. If you have questions or spot an error, let me know. 3 4. Find all matrices that commute with A =. 4 3 a b If we set B = and set AB = BA, we see that 3a + 4b = 3a 4c, 4a + 3b = 3b 4d,

### Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )

Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O

### Math 314/ Exam 2 Blue Exam Solutions December 4, 2008 Instructor: Dr. S. Cooper. Name:

Math 34/84 - Exam Blue Exam Solutions December 4, 8 Instructor: Dr. S. Cooper Name: Read each question carefully. Be sure to show all of your work and not just your final conclusion. You may not use your

### Mathematical Formulation of the Superposition Principle

Mathematical Formulation of the Superposition Principle Superposition add states together, get new states. Math quantity associated with states must also have this property. Vectors have this property.

### Dot Products, Transposes, and Orthogonal Projections

Dot Products, Transposes, and Orthogonal Projections David Jekel November 13, 2015 Properties of Dot Products Recall that the dot product or standard inner product on R n is given by x y = x 1 y 1 + +

### Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors week -2 Fall 26 Eigenvalues and eigenvectors The most simple linear transformation from R n to R n may be the transformation of the form: T (x,,, x n ) (λ x, λ 2,, λ n x n

### 1 Invariant subspaces

MATH 2040 Linear Algebra II Lecture Notes by Martin Li Lecture 8 Eigenvalues, eigenvectors and invariant subspaces 1 In previous lectures we have studied linear maps T : V W from a vector space V to another

### Stat 159/259: Linear Algebra Notes

Stat 159/259: Linear Algebra Notes Jarrod Millman November 16, 2015 Abstract These notes assume you ve taken a semester of undergraduate linear algebra. In particular, I assume you are familiar with the

### MAT 1302B Mathematical Methods II

MAT 1302B Mathematical Methods II Alistair Savage Mathematics and Statistics University of Ottawa Winter 2015 Lecture 19 Alistair Savage (uottawa) MAT 1302B Mathematical Methods II Winter 2015 Lecture

### Matrices and Vectors. Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A =

30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can

### Singular Value Decomposition

Chapter 5 Singular Value Decomposition We now reach an important Chapter in this course concerned with the Singular Value Decomposition of a matrix A. SVD, as it is commonly referred to, is one of the

### Math 4153 Exam 3 Review. The syllabus for Exam 3 is Chapter 6 (pages ), Chapter 7 through page 137, and Chapter 8 through page 182 in Axler.

Math 453 Exam 3 Review The syllabus for Exam 3 is Chapter 6 (pages -2), Chapter 7 through page 37, and Chapter 8 through page 82 in Axler.. You should be sure to know precise definition of the terms we

### 18.06 Quiz 2 April 7, 2010 Professor Strang

18.06 Quiz 2 April 7, 2010 Professor Strang Your PRINTED name is: 1. Your recitation number or instructor is 2. 3. 1. (33 points) (a) Find the matrix P that projects every vector b in R 3 onto the line

### What is A + B? What is A B? What is AB? What is BA? What is A 2? and B = QUESTION 2. What is the reduced row echelon matrix of A =

STUDENT S COMPANIONS IN BASIC MATH: THE ELEVENTH Matrix Reloaded by Block Buster Presumably you know the first part of matrix story, including its basic operations (addition and multiplication) and row

### The Spectral Theorem for normal linear maps

MAT067 University of California, Davis Winter 2007 The Spectral Theorem for normal linear maps Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (March 14, 2007) In this section we come back to the question

### 12. Perturbed Matrices

MAT334 : Applied Linear Algebra Mike Newman, winter 208 2. Perturbed Matrices motivation We want to solve a system Ax = b in a context where A and b are not known exactly. There might be experimental errors,

### 1. General Vector Spaces

1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

### Real symmetric matrices/1. 1 Eigenvalues and eigenvectors

Real symmetric matrices 1 Eigenvalues and eigenvectors We use the convention that vectors are row vectors and matrices act on the right. Let A be a square matrix with entries in a field F; suppose that

### Lecture 2: Eigenvalues and their Uses

Spectral Graph Theory Instructor: Padraic Bartlett Lecture 2: Eigenvalues and their Uses Week 3 Mathcamp 2011 As you probably noticed on yesterday s HW, we, um, don t really have any good tools for finding

### Math 215 HW #9 Solutions

Math 5 HW #9 Solutions. Problem 4.4.. If A is a 5 by 5 matrix with all a ij, then det A. Volumes or the big formula or pivots should give some upper bound on the determinant. Answer: Let v i be the ith

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### THE MINIMAL POLYNOMIAL AND SOME APPLICATIONS

THE MINIMAL POLYNOMIAL AND SOME APPLICATIONS KEITH CONRAD. Introduction The easiest matrices to compute with are the diagonal ones. The sum and product of diagonal matrices can be computed componentwise

### Diagonalization of Matrices

LECTURE 4 Diagonalization of Matrices Recall that a diagonal matrix is a square n n matrix with non-zero entries only along the diagonal from the upper left to the lower right (the main diagonal) Diagonal

### Roberto s Notes on Linear Algebra Chapter 9: Orthogonality Section 2. Orthogonal matrices

Roberto s Notes on Linear Algebra Chapter 9: Orthogonality Section 2 Orthogonal matrices What you need to know already: What orthogonal and orthonormal bases for subspaces are. What you can learn here:

### Lecture 3: QR-Factorization

Lecture 3: QR-Factorization This lecture introduces the Gram Schmidt orthonormalization process and the associated QR-factorization of matrices It also outlines some applications of this factorization

### Calculating determinants for larger matrices

Day 26 Calculating determinants for larger matrices We now proceed to define det A for n n matrices A As before, we are looking for a function of A that satisfies the product formula det(ab) = det A det

### Determinants and Scalar Multiplication

Invertibility and Properties of Determinants In a previous section, we saw that the trace function, which calculates the sum of the diagonal entries of a square matrix, interacts nicely with the operations

### Review of Linear Algebra Definitions, Change of Basis, Trace, Spectral Theorem

Review of Linear Algebra Definitions, Change of Basis, Trace, Spectral Theorem Steven J. Miller June 19, 2004 Abstract Matrices can be thought of as rectangular (often square) arrays of numbers, or as

### LECTURE VI: SELF-ADJOINT AND UNITARY OPERATORS MAT FALL 2006 PRINCETON UNIVERSITY

LECTURE VI: SELF-ADJOINT AND UNITARY OPERATORS MAT 204 - FALL 2006 PRINCETON UNIVERSITY ALFONSO SORRENTINO 1 Adjoint of a linear operator Note: In these notes, V will denote a n-dimensional euclidean vector

### Eigenvalues, Eigenvectors, and Diagonalization

Week12 Eigenvalues, Eigenvectors, and Diagonalization 12.1 Opening Remarks 12.1.1 Predicting the Weather, Again Let us revisit the example from Week 4, in which we had a simple model for predicting the

### Math 224, Fall 2007 Exam 3 Thursday, December 6, 2007

Math 224, Fall 2007 Exam 3 Thursday, December 6, 2007 You have 1 hour and 20 minutes. No notes, books, or other references. You are permitted to use Maple during this exam, but you must start with a blank

### I. Multiple Choice Questions (Answer any eight)

Name of the student : Roll No : CS65: Linear Algebra and Random Processes Exam - Course Instructor : Prashanth L.A. Date : Sep-24, 27 Duration : 5 minutes INSTRUCTIONS: The test will be evaluated ONLY

### and let s calculate the image of some vectors under the transformation T.

Chapter 5 Eigenvalues and Eigenvectors 5. Eigenvalues and Eigenvectors Let T : R n R n be a linear transformation. Then T can be represented by a matrix (the standard matrix), and we can write T ( v) =

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### Linear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University

Linear Algebra Done Wrong Sergei Treil Department of Mathematics, Brown University Copyright c Sergei Treil, 2004, 2009 Preface The title of the book sounds a bit mysterious. Why should anyone read this

### A linear algebra proof of the fundamental theorem of algebra

A linear algebra proof of the fundamental theorem of algebra Andrés E. Caicedo May 18, 2010 Abstract We present a recent proof due to Harm Derksen, that any linear operator in a complex finite dimensional

### MATH 304 Linear Algebra Lecture 23: Diagonalization. Review for Test 2.

MATH 304 Linear Algebra Lecture 23: Diagonalization. Review for Test 2. Diagonalization Let L be a linear operator on a finite-dimensional vector space V. Then the following conditions are equivalent:

### LINEAR ALGEBRA MICHAEL PENKAVA

LINEAR ALGEBRA MICHAEL PENKAVA 1. Linear Maps Definition 1.1. If V and W are vector spaces over the same field K, then a map λ : V W is called a linear map if it satisfies the two conditions below: (1)

### ALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA

ALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA Kent State University Department of Mathematical Sciences Compiled and Maintained by Donald L. White Version: August 29, 2017 CONTENTS LINEAR ALGEBRA AND

### MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors.

MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors. Orthogonal sets Let V be a vector space with an inner product. Definition. Nonzero vectors v 1,v

### Topic 15 Notes Jeremy Orloff

Topic 5 Notes Jeremy Orloff 5 Transpose, Inverse, Determinant 5. Goals. Know the definition and be able to compute the inverse of any square matrix using row operations. 2. Know the properties of inverses.

### Inner product spaces. Layers of structure:

Inner product spaces Layers of structure: vector space normed linear space inner product space The abstract definition of an inner product, which we will see very shortly, is simple (and by itself is pretty

### Recall the convention that, for us, all vectors are column vectors.

Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists

### Linear Algebra using Dirac Notation: Pt. 2

Linear Algebra using Dirac Notation: Pt. 2 PHYS 476Q - Southern Illinois University February 6, 2018 PHYS 476Q - Southern Illinois University Linear Algebra using Dirac Notation: Pt. 2 February 6, 2018

### MATH 310, REVIEW SHEET 2

MATH 310, REVIEW SHEET 2 These notes are a very short summary of the key topics in the book (and follow the book pretty closely). You should be familiar with everything on here, but it s not comprehensive,

### 5.) For each of the given sets of vectors, determine whether or not the set spans R 3. Give reasons for your answers.

Linear Algebra - Test File - Spring Test # For problems - consider the following system of equations. x + y - z = x + y + 4z = x + y + 6z =.) Solve the system without using your calculator..) Find the

### Review of some mathematical tools

MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical

### MATH 304 Linear Algebra Lecture 33: Bases of eigenvectors. Diagonalization.

MATH 304 Linear Algebra Lecture 33: Bases of eigenvectors. Diagonalization. Eigenvalues and eigenvectors of an operator Definition. Let V be a vector space and L : V V be a linear operator. A number λ

### 1 Matrices and Systems of Linear Equations

March 3, 203 6-6. Systems of Linear Equations Matrices and Systems of Linear Equations An m n matrix is an array A = a ij of the form a a n a 2 a 2n... a m a mn where each a ij is a real or complex number.

### (a) II and III (b) I (c) I and III (d) I and II and III (e) None are true.

1 Which of the following statements is always true? I The null space of an m n matrix is a subspace of R m II If the set B = {v 1,, v n } spans a vector space V and dimv = n, then B is a basis for V III

### MTH 102: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology - Kanpur. Problem Set

MTH 102: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology - Kanpur Problem Set 6 Problems marked (T) are for discussions in Tutorial sessions. 1. Find the eigenvalues

### Maths for Signals and Systems Linear Algebra in Engineering

Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE

### Background Mathematics (2/2) 1. David Barber

Background Mathematics (2/2) 1 David Barber University College London Modified by Samson Cheung (sccheung@ieee.org) 1 These slides accompany the book Bayesian Reasoning and Machine Learning. The book and

### Numerical Linear Algebra

Numerical Linear Algebra The two principal problems in linear algebra are: Linear system Given an n n matrix A and an n-vector b, determine x IR n such that A x = b Eigenvalue problem Given an n n matrix

### SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear Algebra

1.1. Introduction SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear algebra is a specific branch of mathematics dealing with the study of vectors, vector spaces with functions that

### ANSWERS. E k E 2 E 1 A = B

MATH 7- Final Exam Spring ANSWERS Essay Questions points Define an Elementary Matrix Display the fundamental matrix multiply equation which summarizes a sequence of swap, combination and multiply operations,

### Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal

### 0.1 Diagonalization and its Applications

.. DIAGONALIZATION AND ITS APPLICATIONS. Diagonalization and its Applications Exercise.. Let A be a -matrix whose characteristic polynomial is A (t) = t t + t +. a) Express A in term of powers A, A, and

### MA 265 FINAL EXAM Fall 2012

MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators

### CS168: The Modern Algorithmic Toolbox Lecture #8: How PCA Works

CS68: The Modern Algorithmic Toolbox Lecture #8: How PCA Works Tim Roughgarden & Gregory Valiant April 20, 206 Introduction Last lecture introduced the idea of principal components analysis (PCA). The

### No books, no notes, no calculators. You must show work, unless the question is a true/false, yes/no, or fill-in-the-blank question.

Math 304 Final Exam (May 8) Spring 206 No books, no notes, no calculators. You must show work, unless the question is a true/false, yes/no, or fill-in-the-blank question. Name: Section: Question Points

### Math 443 Differential Geometry Spring Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook.

Math 443 Differential Geometry Spring 2013 Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook. Endomorphisms of a Vector Space This handout discusses

### After we have found an eigenvalue λ of an n n matrix A, we have to find the vectors v in R n such that

7.3 FINDING THE EIGENVECTORS OF A MATRIX After we have found an eigenvalue λ of an n n matrix A, we have to find the vectors v in R n such that A v = λ v or (λi n A) v = 0 In other words, we have to find

### Volume in n Dimensions

Volume in n Dimensions MA 305 Kurt Bryan Introduction You ve seen that if we have two vectors v and w in two dimensions then the area spanned by these vectors can be computed as v w = v 1 w 2 v 2 w 1 (where

### Further Mathematical Methods (Linear Algebra) 2002

Further Mathematical Methods (Linear Algebra) 00 Solutions For Problem Sheet 0 In this Problem Sheet we calculated some left and right inverses and verified the theorems about them given in the lectures.

### Math 110 Linear Algebra Midterm 2 Review October 28, 2017

Math 11 Linear Algebra Midterm Review October 8, 17 Material Material covered on the midterm includes: All lectures from Thursday, Sept. 1st to Tuesday, Oct. 4th Homeworks 9 to 17 Quizzes 5 to 9 Sections

### Linear Algebra- Final Exam Review

Linear Algebra- Final Exam Review. Let A be invertible. Show that, if v, v, v 3 are linearly independent vectors, so are Av, Av, Av 3. NOTE: It should be clear from your answer that you know the definition.

### 21 Symmetric and skew-symmetric matrices

21 Symmetric and skew-symmetric matrices 21.1 Decomposition of a square matrix into symmetric and skewsymmetric matrices Let C n n be a square matrix. We can write C = (1/2)(C + C t ) + (1/2)(C C t ) =

### Determinants and Scalar Multiplication

Properties of Determinants In the last section, we saw how determinants interact with the elementary row operations. There are other operations on matrices, though, such as scalar multiplication, matrix

### Topic 2 Quiz 2. choice C implies B and B implies C. correct-choice C implies B, but B does not imply C

Topic 1 Quiz 1 text A reduced row-echelon form of a 3 by 4 matrix can have how many leading one s? choice must have 3 choice may have 1, 2, or 3 correct-choice may have 0, 1, 2, or 3 choice may have 0,

### Introduction to Matrix Algebra

Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p

### MATH 304 Linear Algebra Lecture 34: Review for Test 2.

MATH 304 Linear Algebra Lecture 34: Review for Test 2. Topics for Test 2 Linear transformations (Leon 4.1 4.3) Matrix transformations Matrix of a linear mapping Similar matrices Orthogonality (Leon 5.1

### Math 240 Calculus III

Generalized Calculus III Summer 2015, Session II Thursday, July 23, 2015 Agenda 1. 2. 3. 4. Motivation Defective matrices cannot be diagonalized because they do not possess enough eigenvectors to make

### Algebra I Fall 2007

MIT OpenCourseWare http://ocw.mit.edu 18.701 Algebra I Fall 007 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 18.701 007 Geometry of the Special Unitary

### Gaussian elimination

Gaussian elimination October 14, 2013 Contents 1 Introduction 1 2 Some definitions and examples 2 3 Elementary row operations 7 4 Gaussian elimination 11 5 Rank and row reduction 16 6 Some computational

### Solutions to Final Practice Problems Written by Victoria Kala Last updated 12/5/2015

Solutions to Final Practice Problems Written by Victoria Kala vtkala@math.ucsb.edu Last updated /5/05 Answers This page contains answers only. See the following pages for detailed solutions. (. (a x. See

### Honors Linear Algebra, Spring Homework 8 solutions by Yifei Chen

.. Honors Linear Algebra, Spring 7. Homework 8 solutions by Yifei Chen 8... { { W {v R 4 v α v β } v x, x, x, x 4 x x + x 4 x + x x + x 4 } Solve the linear system above, we get a basis of W is {v,,,,

### SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear Algebra

SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to 1.1. Introduction Linear algebra is a specific branch of mathematics dealing with the study of vectors, vector spaces with functions that