Lecture Notes for Fall 2015, Michigan State University Matthew Hirn December 11, 2015 Beginning of Lecture 1 1 Vector Spaces What is this course about? 1. Understanding the structural properties of a wide class of spaces which all share a similar additive and multiplicative structure structure = vector addition and scalar multiplication vector spaces 2. The study of linear maps on finite dimensional vector spaces We begin with vector spaces. First two examples: 1. R n = n-tuples of real numbers x = (x 1,..., x n ), x k R vector addition: x+y = (x 1,..., x n )+(y 1,..., y n ) = (x 1 +y 1,..., x n +y n ) scalar multiplication: λ R, λx = λ(x 1,..., x n ) = (λx 1,..., λx n ) 2. C n [on your own: review 1.A on complex numbers] 1.B Definition of Vector Space Scalars: Field F (assume F = R or C unless otherwise stated). So the previous two vector spaces can be written as F n with scalars F Let V be a set (for now). 1
Definition 1 (Vector addition). u, v V, assigns an element u + v V Definition 2 (Scalar multiplication). λ F, v V, assigns an element λv V Definition 3 (Vector space). A set V is a vector space over the field F if vector addition and scalar multiplication are defined, and the following properties hold (u, v, w V, a, b F): 1. Commutativity: u + v = v + u 2. Associativity: (u + v) + w = u + (v + w) and (ab)v = a(bv) 3. Additive Identity: 0 V such that v + 0 = v 4. Additive Inverse: for every v there exists w such that v + w = 0 5. Multiplicative Identity: 1v = v 6. Distributive Properties: a(u + v) = au + av and (a + b)v = av + bv If F = R, real vector space If F = C, complex vector space From here on out V will always denote a vector space Two more examples of vector spaces: 1. F : x = (x 1, x 2,...) just like F n 2. F S = the set of functions f : S F from S to F [check on your own] Now for some important properties... Proposition 1. The additive identity is unique. Proof. Let 0 1 and 0 2 be any two additive identities. Then 0 1 = 0 1 + 0 2 = 0 2 + 0 1 = 0 2 Proposition 2. The additive inverse is unique. 2
Proof. Let w 1 and w 2 be two additive inverses of v. Then: w 1 = w 1 + 0 = w 1 + (v + w 2 ) = (v + w 1 ) + w 2 = 0 + w 2 = w 2 Now we can write v as the additive inverse of v and define subtraction as v w = v + ( w). On the other hand, we still don t know that 1v = v! Notation: We have 0 F and 0 V. In the previous two propositions we dealt with 0 V. Next we will handle 0 F. We just write 0 for either and use the context to determine the meaning. Proposition 3. 0 F v = 0 V for every v V Proof. 0v = (0 + 0)v = 0v + 0v = 0v = 0 Now the other way around... Proposition 4. λ0 = 0 for every λ F Proposition 5. ( 1)v = v for all v V Proof. v + ( 1)v = 1v + ( 1)v = (1 + ( 1))v = 0v = 0 Now use uniqueness of additive inverse. End of Lecture 1 3
Beginning of Lecture 2 Warmup: Is the empty set a vector space? Answer: No since 0 / 1.C Subspaces A great way to find new vector spaces is to identify subsets of an existing vector space which are closed under addition and multiplication. Definition 4 (Subspace). U V is a subspace of V if U is also a vector space (using the same vector addition and scalar multiplication as V ). Proposition 6. U V is a subspace if and only if: 1. 0 U 2. u, w U = u + w U 3. λ F and u U = λu U Now we can introduce more interesting examples of vector spaces, many of which are subspaces of F S for some set S [you should verify these are vector spaces]: 1. P(F) = {p : F F : p(z) = a } 0 + a 1 z {{ + a m z m }, a k F k, m N} deg(p)=m 2. C(R; R) = real valued continuous functions 3. C m (R n ; R) = real valued functions with continuous partial derivatives up to order m 4. R([0, 1]) = {f : [0, 1] R : 1 0 f(x) dx < }. 5. F m,n = the set of all m n matrices with entries in F 6. S = {x : [0, 1] R n : x (t) is continuous and x (t) = Ax(t), where A R n,n } Another convenient way to get new vector spaces is to add subspaces together (this is like the union of two sets, but for vector spaces!). 4
Definition 5 (Sum of subsets). Suppose U 1,..., U m V. Then: U 1 + + U m := {u 1 + + u m : u 1 U 1,..., u m U m }. Proposition 7. Suppose U 1,..., U m are subspaces of V. Then U 1 + + U m is the smallest subspace of V containing U 1,..., U m. An example: U 1 = {x R 3 : x 1 + x 2 + x 3 = 0} U 2 = {x R 3 : x 3 = 0} U 1 + U 2 = {x R 3 : x = y + z, y 1 + y 2 + y 3 = 0 and z 3 = 0} U 1 + U 2 = {x R 3 : x = a( 1, 0, 1) + b(1, 1, 0) + c(1, 0, 0) + d(0, 1, 0)} (1) U 1 + U 2 = R 3 Note there is redundancy in (1). We will be especially interested in situations that avoid this redundancy, i.e., subspace summations U 1 + + U m when the representation u 1 + + u m is unique. Definition 6 (Direct sum). Suppose that U 1,..., U m are subspaces of V. U 1 + + U m is a direct sum if each element of U 1 + + U m can be written in only one way as u 1 + + u m where u k U k. If U 1 + + U m is a direct sum, then we denote it as U 1 U m Examples: 1. Let U k be the subspace of F n such that only the k th coordinate is nonzero: U k = {(0,..., 0, x, 0,..., 0) : x F} }{{} k 1 Then R n = U 1 U n 2. Recall the previous example with redundancy. That is not a direct sum. We can change U 2 though to get a direct sum: U 1 = {x R 3 : x 1 + x 2 + x 3 = 0} U 2 = {x R 3 : x 1 = x 2 = x 3 } R 3 = U 1 U 2 5
Notice in the second example that U 1 U 2 = {0}. This leads us to the following proposition. Proposition 8. Let U, W be subspaces of V. Then, V = U W U W = {0} The first example makes it tempting to propose the same pairwise intersection property for any number of subspaces, but this is not true! [try to come up with an example, then see the book] Instead we have the following proposition, which we can use to prove Proposition 8. Proposition 9. Suppose U 1,..., U m are subspaces of V. Then U 1 + + U m is a direct sum 0 = u 1 + + u m, u k U k, only when u k = 0 k Proof. The direction is clear. For the direction, let v U 1 + + U m and suppose we have two representations: v = u 1 + + u m = w 1 + + w m Then 0 = (u 1 w 1 ) + + (u m w m ) Since u k w k U k, we must have u k = w k for each k. [try to prove Proposition 8 on your own using Proposition 9, then see the book]. 2 Finite Dimensional Vector Spaces 2.A Span and Linear Independence We saw last time that summing subspaces gives rise to new vector spaces. Now we keep track of each of the vectors that generate these spaces. Definition 7 (Linear combination). w is a linear combination of the vectors v 1,..., v m V if a 1,..., a m F such that w = a 1 v 1 + a m v m 6
Definition 8 (Span). The span of v 1,..., v m V is span(v 1,..., v m ) = {a 1 v 1 + a m v m : a k F k} Analogous to the sum of subspaces, we have the following result. Proposition 10. span(v 1,..., v m ) is the smallest subspace of V containing v 1,..., v m. Nomenclature: If span(v 1,..., v m ) = V then we say that v 1,..., v m spans V. Definition 9 (Finite dimensional vector space). V is finite dimensional if there exists a finite number of vectors v 1,..., v m (a list) such that span(v 1,..., v m ) = V. Definition 10 (Infinite dimensional vector space). V is infinite dimensional if it is not finite dimensional. End of Lecture 2 7
Beginning of Lecture 3 Warmup: Is this a vector space? 1. {f C((0, 1); R) : f(x) = x p for some p > 0} Answer: No (all three properties fail) 2. {f C(R; R) : f is periodic of period σ} Answer: Yes (contains zero function, closed under addition and scalar multiplication) Examples: 1. P(F) is infinite dimensional [see the proof in the book]. 2. P m (F) = {p P(F) : deg(p) m} is finite dimensional: span(1, z, z 2,..., z m ) = P m (F) 3. U = {f C(R; R) : f is periodic of period n for some n N} U is infinite dimensional Proof. Let L = v 1,..., v m be an arbitrary list from U, so that each v k has period n k N. If l = lcm(n 1,..., n m ), then any linear combination from L will have period which is at most l. Therefore if p is a prime number such that p > l, sin( 2π p x) / L, but sin(2π p x) U, and thus span(l) U. Since L was arbitrary we can conclude that no finite list will span U. It will be very useful to record if a list of vectors v 1,..., v m has no redundancy in its span, just as we isolated sums of subspaces with no redundancy by defining the direct sum. Definition 11 (Linear independence). v 1,..., v m V are linearly independent if whenever 0 = a 1 v 1 + + a m v m, then necessarily a 1 = = a m = 0. Definition 12 (Linear dependence). v 1,..., v m V are linearly dependent if a 1,... a m with at least one a k 0 and 0 = a 1 v 1 + + a m v m. The notions of linear independence and linear dependence are extremely important! Examples: 8
1. (1, 0, 0), (0, 1, 0) are linearly independent in F 3 2. 1, z,..., z m are linearly independent in P(F) [Why? Use the fact that a polynomial of degree m has at most m distinct zeros] 3. Recall example from sum of subspaces: ( 1, 0, 1), (1, 1, 0), (1, 0, 0), (0, 1, 0) are linearly dependent ( 1, 0, 1), (1, 1, 0), (1, 1, 1) are linearly independent The following is a very useful lemma... Lemma 1 (Linear Dependence Lemma, LDL). If v 1,..., v m V are linearly dependent and v 1 0, then k {2,..., m} such that 1. v k span(v 1,..., v k 1 ) 2. If the v k is removed from v 1,..., v m then the resulting span is the same as the original. Proof. Let L = v 1,..., v m. For #1, by definition of linear dependence a 1,..., a m not all zero such that 0 = a 1 v 1 + +a m v m. Let k {2,..., m} be the largest index such that a k 0. Then: v k = a 1 a k v 1 a k 1 a k v k 1 (2) For #2, let L = L \ {v k }. Since L L, span(l ) span(l). Let u span(l). Then: u = a 1 v 1 + a k 1 v k 1 + a k v k + a k+1 v k+1 + + a m v m Substitute (2) in for v k and the sum is now in terms of L, i.e., u span(l ). Thus span(l) span(l ). Now for our first theorem. Theorem 1. If V = span(v 1,..., v n ) and w 1,..., w m are linearly independent in V, then m n. 9
Proof. We will use the two lists and make successive reductions and additions using Lemma 1. Note: w 1,..., w m linearly indpendent w k 0 k [why?] Add & reduce: Since V = span(v 1,..., v n ) and w 1 V, then w 1, v 1,..., v n are linearly dependent. So Lemma 1 says at least one of the v k can be removed. Up to a relabeling, we may assume it is v n. So span(w 1, v 1,..., v n 1 ) is the same as span(v 1,..., v n ). Now we can repeat: w 2 V = span(w 1, v 1,..., v n 1 ) so w 2, w 1, v 1,..., v n 1 are linearly dependent. Use Lemma 1 again, which says that one of them can be removed. The question is which? If it is w 1, then w 1 span(w 2 ), which is a contradiction; so it must be one of the v 1,..., v n 1. Without loss of generality (WLOG), we may assume it is v n 1 and so span(w 2, w 1, v 1,..., v n 2 ) = span(w 2, v 1,..., v n 1 ) = V. Keep repeating. At each stage one of the v k must be removed, else Lemma 1 implies that w j span(w 1,... w j 1 ) which is a contradiction. The process stops when either we run out of w s (m n) or we run out of v s (m > n). If m > n, then span(w 1,..., w n ) = V and m > n. Thus w m / span(w 1,..., w n ) = V, but this is a contradiction since w k V k. Proposition 11. If V is finite dimensional and U is a subspace of V, then U is finite dimensional. End of Lecture 3 10
Beginning of Lecture 4 2.B Bases span+ linear independence = basis Definition 13. v 1,..., v n V is a basis of V if span(v 1,..., v n ) = V and v 1,..., v n are linearly independent. Proposition 12. v 1,..., v n V is a basis of V if and only if v V,! a 1,..., a n F such that v = a 1 v 1 + a n v n The notion of a basis is extremely important because it allows us to define a coordinate system for our vector spaces! Examples: 1. (1, 0,..., 0), (0, 1, 0,..., 0),..., (0,..., 0, 1) is the standard basis of F n. 2. 1, z,..., z m is the standard basis for P m (F) 3. Let Z N = {0, 1,..., N 1} (with addition mod N) and let V = {f : Z N C}. The standard (time side) basis for V is δ 0,..., δ N 1 where { 1 n = k δ k (n) = 0 n k Indeed, f(n) = N 1 k=0 f(k)δ k (n) Fourier analysis tells us that another (frequency side) basis for V is e 0,..., e N 1 where e k (n) = 1 e 2πikn/N N and f(n) = N 1 k=0 a k e k (n) 11
with a k = ˆf(k) = 1 N 1 N n=0 f(n)e 2πikn/N The coefficients a k define the function ˆf(k) which is the Fourier transform of f. If v 1,..., v n spans V, it should have enough vectors to make a basis. Indeed: Proposition 13. If L = v 1,..., v n spans V, then L can be reduced to a basis. Proof. If L is linearly independent, then we are done. So assume it is not. We will selectively throw away vectors using the LDL. Step 1: If v 1 = 0 remove v 1 Step 2: If v 2 span(v 1 ), remove v 2 Step k: If v k span(v 1,..., v k 1 ), remove v k Stop at Step n, getting a new list L = w 1,..., w m. We still have span(l ) = V since we only discarded vectors that were in the span of other vectors. We also have the property: w k / span(w 1,..., w k 1 ), k > 1 Thus by the contrapositive of LDL, L is linearly independent, and hence a basis. Corollary 1. If V is finite dimensional, it has a basis. We just removed stuff from a spanning set to get a basis. We can also add stuff to a linearly independent set to get a basis. Proposition 14. If L = u 1,..., u m V is linearly independent, then L can be extended to a basis. Proof. Let w 1,..., w n be a basis of V. Thus L = u 1,..., u m, w 1,..., w n spans V. Apply the procedure in the proof of Proposition 13, and note that none of the u s get deleted [why?]. 12
Now we show that every subspace U has a complementary subspace W that together direct sum to V. Proposition 15. Suppose V is finite dimensional and that U is a subspace of V. Then there exists another subspace W such that V = U W Proof. V finite dimensional U finite dimensional U has a basis u 1,..., u m. By the previous proposition we can extend u 1,..., u m to a basis of V, say L = u 1,..., u m, w 1,..., w n. We show that W = span(w 1,..., w n ) is the answer. We need to show: (1) V = U + W, and (2) U W = {0}. Since L is a basis, for any v V we have: v = a } 1 u 1 + {{ + a m u m} u U Now suppose that v U W. Then which implies + } b 1 w 1 + {{ + b n w n} = u + w U + W w W v = a 1 u 1 + + a m u m = b 1 w 1 + + b n w n a 1 u 1 + + a m u m b 1 w 1 b n w n = 0 But L is linearly independent so a 1 = = a m = b 1 = = b n = 0. 2.C Dimension Since a basis gives a unique representation of each v V, we should be able to say that the number of vectors in basis is the dimension of V. But to do so, we need to make sure every basis of V has the same number of vectors. Indeed: Theorem 2. Any two bases of a finite dimensional vector space have the same length. Proof. Let B 1 = v 1,..., v m and B 2 = w 1,..., w n be two bases of V. Since B 1 is linearly independent and B 2 spans V, m n. Flipping the roles of B 1 and B 2, we get n m. 13
Definition 14. The dimension of V is the length of B for any basis B. Proposition 16. If U is a subspace of V, then dim U dim V Examples: 1. dim F n = n Remark: dim R 2 = 2 and dim C = 1, even though R 2 can be identified with C. The scalar field F cannot be ignored when computing the dimension of V! 2. dim P m (F) = m + 1 Let L = v 1,..., v n. If dim V = n, then we need only check if L is linearly independent OR if span(l) = V to conclude that L is a basis for V. Proposition 17. Suppose dim V = n and let L = v 1,..., v n. 1. If L is linearly independent, then L is a basis 2. If span(l) = V, then L is a basis. Proof. Use Proposition 14 for (1) and Proposition 13 for (2). End of Lecture 4 14
Beginning of Lecture 5 Theorem 3. dim V <, U 1 and U 2 subspaces of V. Then dim(u 1 + U 2 ) = dim U 1 + dim U 2 dim(u 1 U 2 ) Proof. Proof will use 3 objects: 1. B = u 1,..., u m = basis of U 1 U 2 2. L 1 = v 1,..., v j = extension of B so that B L 1 = basis for U 1 3. L 2 = w 1,..., w k = extension of B so that B L 2 = basis for U 2. We will show that L = B L 1 L 2 is a basis for U 1 + U 2. This will complete the proof since if it is true, then dim(u 1 +U 2 ) = m+j+k = (m+j)+(m+k) m = dim U 1 +dim U 2 dim(u 1 U 2 ) Clearly L spans U 1 + U 2 since span(l) contains both U 1 and U 2. Now we show linear independence. Suppose: a i u i + b l v l + i l p Then: c p w p = p i But w p U 2 by assumption, so c p w p U 1 U 2 p p a i u i l c p w p = q c p w p = 0 (3) b l v l U 1 d q u q for some d q Now, (u 1,..., u m, w 1,..., w k ) is a basis for U 2. Thus: c p w p d q u q = 0 c p = 0, d q = 0, p, q p q Therefore (3) reduces to a i u i + i l Repeat the previous argument. b l v l = 0 15
3 Linear Maps V, W always vector spaces. 3.A The Vector Space of Linear Maps Definition 15. Let V, W be vector spaces over the same field F. A function T : V W is a linear map if it has the following two properties: 1. additivity: T (u + v) = T u + T v, u, v V 2. homogeneity: T (λv) = λ(t v) λ F, v V The set of all linear maps from V to W is denoted L(V, W ). Note: You could say T is linear if it preserves the vector space structures of V and W. Examples (read the ones in the book too!): Fix a point x 0 R. Evaluation at x 0 is a linear map: T : C(R; R) R T v = v(x 0 ) The anti-derivative is a linear map: T : C(R; R) C 1 (R; R) (T v)(x) = x 0 v(y) dy Fix b F. Define the forward shift operator as: T : F F T (v 1, v 2, v 3,...) = (b, v 1, v 2, v 3,...) T is a linear map if and only if b = 0 [why?]. Next we show that we can always find a linear map that takes whatever values we want on a basis, and furthermore, that it is completely determined by these values. 16
Theorem 4. Let v 1,..., v n be a basis for V and let w 1,..., w n W. Then there exists a unique linear map T : V W such that T v k = w k, k Proof. Define T : V W as T (a 1 v 1 + a n v n ) = a 1 w 1 + + a n w n Clearly T v k = w k for all k. It is easy to see that T is linear as well [see the book]. For uniqueness, let S : V W be another linear map such that Sv k = w k for all k. Then: S(a 1 v 1 + a n v n ) = S(a k v k ) = a k Sv k = a k w k = T (a 1 v 1 + +a n v n ) The previous theorem is elementary, but highlights the fact that amongst all the maps from V to W, linear maps are very special. Theorem 5. L(V, W ) is a vector space with the following vector addition and scalar multiplication operations: vector addition: S, T L(V, W ), (S + T )(v) = Sv + T v v V scalar mult.: T L(V, W ), λ F, (λt )(v) = λ(t v) v V Theorem 6. L(V, W ) is finite dimensional and dim L(V, W ) = (dim V )(dim W ) Proof. Suppose dim V = n and dim W = m and let B V = v 1,..., v n B W = w 1,..., w m be bases for V and W respectively. Define the linear transform E p,q : V W as { 0 k q E p,q (v k ) =, p = 1,..., m, q = 1,..., n w p k = q 17
By Theorem 4, this uniquely defines each E p,q. We are going to show that these mn transformations {E p,q } p,q form a basis for L(V, W ). Let T : V W be a linear map. For each 1 k n, let a 1,k,..., a m,k be the coordinates of T v k in the basis B W : T v k = m a p,k w p p=1 To prove spanning, we wish to show that: T = m a p,q E p,q (4) p=1 q=1 Let S be the linear map on the right hand side of (4). Then for each k, Sv k = a p,q E p,q v k p q = p a p,k w p = T v k So S = T, and since T was arbitrary, {E p,q } p,q spans L(V, W ). To prove linear independence, suppose that S = a p,q E p,q = 0 p q Then Sv k = 0 for each k, so a p,k w p = 0, p k But w 1,..., w m are linearly independent, so a p,k = 0 for all p and k. End of Lecture 5 18
Beginning of Lecture 6 Warmup: Let U, W be 5-dimensional subspaces of R 9. Can U W = {0}? Answer: No. First note that dim{0} = 0. Then, using Theorem 3 we have: dim R 9 = 9 dim(u 1 + U 2 ) = dim U 1 + dim U 2 dim(u 1 U 2 ) = 10 dim(u 1 U 2 ) dim(u 1 U 2 ) 1 Proposition 18. If T : V W is a linear map, then T (0) = 0. Proof. T (0) = T (0 + 0) = T (0) + T (0) T (0) = 0 Usually the product of a vector from one vector space with a vector from another vector space is not well defined. However, for some pairs of linear maps, it is useful to define their product. Definition 16. If T L(U, V ) and S L(V, W ), then the product ST L(U, W ) is (ST )(u) = S(T u), u U Note: You must make sure the range of T is in the domain of S! Another note: Multiplication of linear maps is not commutative! In other words, in general ST T S. 3.B Null Spaces and Ranges For a linear map T, the collection of vectors that get mapped to zero and the collection of those that do not are very important. Definition 17. For T L(V, W ), the null space of T, null T, is: See examples in the book. null T = {v V : T v = 0} Proposition 19. For T L(V, W ), null T is a subspace of V. 19
Proof. Check if it contains zero, closed under addition, closed under scalar multiplication: T (0) = 0 so 0 null T u, v null T, then T (u + v) = T u + T v = 0 + 0 = 0 u null T, λ F, then T (λu) = λt u = λ0 = 0 Definition 18. A function T : V W is injective if T u = T v implies u = v. Proposition 20. Let T L(V, W ). Then T is injective null T = {0} Proof. For the direction, we already know that 0 null T. Thus T (v) = 0 = T (0), but since T is injective v = 0. For the direction, we have: T u = T v T (u v) = 0 u v = 0 u = v Definition 19. For T : V W, the range of T is: range T = {T v : v V } Proposition 21. If T L(V, W ), then range T is a subspace of W. Definition 20. A function T : V W is surjective if range T = W. Theorem 7 (Rank-Nullity Theorem). Suppose V is finite dimensional and T L(V, W ). Then range T is finite dimensional and dim V = dim(null T ) + dim(range T ) Proof. Let u 1,..., u m be a basis for null T, and extend it to a basis u 1,..., u m, v 1,..., v n of V. So we need to show that dim range T = n. To do so we prove that T v 1,..., T v n is a basis for range T. 20
Let v V and write: v = a 1 u 1 + + a m u m + b 1 v 1 + + b n v n T v = b 1 T v 1 + + b n T v n Thus span(t v 1,..., T v n ) = range T Now we show that T v 1,..., T v n are linearly independent. Suppose c 1 T v 1 + + c n T v n = 0 T (c 1 v 1 + + c n v n ) = 0 c 1 v 1 + + c n v n null T c 1 v 1 + + c n v n = d 1 u 1 + + d m u m But v 1,..., v n, u 1,..., u m are linearly independent, so c j = d k = 0 for all j, k. Thus T v 1,..., T v n are linearly independent. Corollary 2. Suppose V, W are finite dimensional and let T L(V, W ). Then: 1. If dim V > dim W then T is not injective. 2. If dim V < dim W then T is not surjective. Proof. Use the Rank-Nullity Theorem: 1. dim null T = dim V dim range T dim V dim W > 0 2. dim range T = dim V dim null T dim V < dim W End of Lecture 6 21
Beginning of Lecture 7 Very important applications: Homogeneous systems of equations m equations and n unknowns: a 1,k x k = 0. (5) a m,k x k = 0 where a j,k F and x = (x 1,..., x n ) F n. Can you solve all m equations simultaneously? Clearly x = 0 is a solution. Are there any others? Define T : F n F m : ( ) T (x 1,..., x n ) = a 1,k x k,..., a m,k x k (6) Note: T (0) = 0 is equivalent to saying 0 is a solution of (5). Furthermore, Nontrivial solutions exist for (5) dim null T > 0 But by the Rank-Nullity Theorem: Since dim range T m, dim null T > 0 dim F n dim range T > 0 if n > m = Nontrivial solutions exist for (5) Inhomogeneous systems of equations: Let c k F and consider: a 1,k x k = c 1. (7) a m,k x k = c m 22
New question, can you say for all c = (c 1,..., c m ) F m there exists at least one solution to (7)? Using the same T as defined in (6), we have: A solution exists for (6) c F m, x F n s.t. T (x) = c range T = F m dim range T = m dim F n dim null T = m dim null T = n m Since dim null T 0, if n < m then certainly there exists c F m such that no solution exists for (7). 3.C Matrices Definition 21. Let T L(V, W ) and let B V = v 1,..., v n and B W = w 1,..., w m be bases of V and W respectively. The matrix of T with respect to B V and B W is the m n matrix M(T ; B V, B W ) (or just M(T ) when B V and B W are clear) with entries A j,k defined by: T v k = m A j,k w j, j=1 k = 1,..., n Note: Recall the proof of the fact that dim L(V, W ) = mn. In that proof we were implicitly using the matrix representation of T. Another note: Recall the idea that a basis B V = v 1,..., v n for a vector space V gives coordinates for V. That is, for all v V, there exists a 1,..., a n F such that v = a 1 v 1 + + a n v n So the n-tuple (a 1,..., a n ) F n is a coordinate representation of the vector v in the basis B V. If we change the basis, say to B V, we change the coordinate representation of v say to (a 1,..., a n), but we do not change v. Similarly, the matrix M(T ; B V, B W ) can be thought of as a coordinate representation of the linear map T L(V, W ) with respect to the bases B V and B W. If we change the bases, we get a new matrix representation of T, but we do not change T ; it is still the same linear map. [we will come back to this with an example later] 23
Definition 22. F m,n is the set of all m n matrices with entries in F. Proposition 22. F m,m is a vector space with the standard matrix addition and scalar multiplication. Proposition 23. dim F m,n = mn. We will derive matrix multiplication from the desire that M(ST ) = M(S)M(T ) for all S, T for which ST makes sense. Suppose T : U V, S : V W, and that B V = {v r } n r=1 is basis for V, B W = {w j } m j=1 is a basis for W, and B U = {u k } p is a basis for U. Let M(S) = A and M(T ) = C. Then for each 1 k p: ( ) (ST )u k = S nc r,k v r = = = r=1 C r,k Sv r r=1 r=1 C r,k m j=1 A j,r w j ( m ) A j,r C r,k w j j=1 r=1 Thus we define matrix multiplication as: (AC) j,k = A j,r C r,k r=1 [read the rest of 3.C on matrix multiplication on your own] End of Lecture 7 24
Beginning of Lecture 8 3.D Invertibility and Isomorphic Vector Spaces Definition 23. A linear map that is both injective and surjective is called bijective. Definition 24. A linear map T L(V, W ) is invertible if S L(W, V ) such that ST = I V and T S = I W. Such a map S is an inverse of T. Proposition 24. An invertible linear map has a unique inverse. Proof. Let S 1 and S 2 be two inverses of T L(V, W ). Then: S 1 = S 1 I = S 1 (T S 2 ) = (S 1 T )S 2 = IS 2 = S 2 Notation: Thus we can denote the inverse of T as T 1 L(W, V ). Theorem 8. T L(V, W ) is invertible T is bijective Proof. For the = direction: Need to show T is injective and surjective. Suppose: T v 1 = T v 2 T 1 T v 1 = T 1 T v 2 v 1 = v 2 since T 1 T = I. Thus T is injective. Now suppose w W. Then: and so T is surjective. T T 1 w = w T (T 1 w) = w }{{} V Now for the = direction: Need to show T is invertible. To do so we define a map S L(W, V ) and show that ST = I and T S = I. Define S : W V as: Sw := unique v V s.t. T v = w (i.e., Sw = v T v = w) 25
Note S is well defined only because T is bijective! By construction we have T S = I. To show that ST = I, let v V, then: T (ST v) = (T S)(T v) = T v ST = I since T is injective Now we need to show that S L(W, V ). For additivity let w 1, w 2 W : T (Sw 1 + Sw 2 ) = T Sw 1 + T Sw 2 = w 1 + w 2 S(w 1 + w 2 ) = Sw 1 + Sw 2 by definition of S For homogeneity use a similar argument: T (λsw) = λt (Sw) = λw S(λw) = λsw We now want to formalize the notion of when two vector spaces are essentially the same. Definition 25. Two parts: An isomorphism is an invertible linear map (i.e., a bijection) V, W are isomorphic if there exists T L(V, W ) such that T is an isomorphism. We write V = W. Theorem 9. V = W dim V = dim W Proof. For the = direction, we know then there is a bijection T L(V, W ). Thus null T = {0} and range T = W, so by Rank-Nullity Theorem: dim V = dim null T + dim range T = 0 + dim W = dim W For the = direction, let v 1,..., v n be a basis for V and let w 1,..., w n be a basis for W. Define T : V W as: T (c 1 v 1 + + c n v n ) = c 1 w 1 + c n w n It is easy to see T L(V, W ), T is injective, T is surjective. Thus T defines an isomorphism. Corollary 3. If dim V = n, then V = F n. 26
Remark: This proves that we can think of the coordinates of any v V in a basis B V = v 1,..., v n as a unique representation in F n, with the vector space structure of V carried over to F n. Indeed, define the matrix of v V with respect to the basis B V as the n 1 matrix: where M(v; B V ) := c 1. c n v = c 1 v 1 + + c n v n The linear map M(, B V ) : V F n (note F n,1 = F n trivially) is an isomorphism. Corollary 4. If dim V = n and dim W = m, then L(V, W ) = F m,n. Proof. This follows easily since we already proved that dim L(V, W ) = (dim V )(dim W ). Proposition 25. Let B V = v 1,..., v n be a basis of V and let B W = w 1,..., w m be a basis of W. Then M( ; B V, B W ) : L(V, W ) F m,n is an isomorphism. Proposition 26. Let T L(V, W ), let v V, and let B V of V and W respectively. Then: and B W be bases M(T v; B W ) = M(T ; B V, B W )M(v; B V ) [See the book for the proofs of the previous two propositions.] Example: Let D L(P 3 (R), P 2 (R)) be the differentiation operator, defined by Dp = p. Let s compute the matrix M(D) of D with respect to the standard bases B 3 = 1, x, x 2, x 3 of P 3 (R) and B 2 = 1, x, x 2 of P 2 (R). Since Dx n = (x n ) = nx n 1 we have: End of Lecture 8 M(D; B 3, B 2 ) = 0 1 0 0 0 0 2 0 0 0 0 3 27
Beginning of Lecture 9 Example: Let D L(P 3 (R), P 2 (R)) be the differentiation operator, defined by Dp = p. Let s compute the matrix M(D) of D with respect to the standard bases B 3 = 1, x, x 2, x 3 of P 3 (R) and B 2 = 1, x, x 2 of P 2 (R). Since Dx n = (x n ) = nx n 1 we have: M(D; B 3, B 2 ) = 0 1 0 0 0 0 2 0 0 0 0 3 Now lets consider a different basis for P 3 (R), for example B 3 = 1 + x, x + x 2, x 2 + x 3, x 3. Compute: Thus: D(1 + x) = 1 D(x + x 2 ) = 1 + 2x D(x 2 + x 3 ) = 2x + 3x 2 D(x 3 ) = 3x 2 M(D; B 3, B 2 ) = 1 1 0 0 0 2 2 0 0 0 3 3 Now consider the specific polynomial p P 3 (R), p(x) = 2 + x + 3x 2 + 5x 3 = p (x) = 1 + 6x + 15x 2 The coordinates of p in B 3 and B 3, as well as p in B 2, are: 2 2 M(p; B 3 ) = 1 3 M(p; B 3) = 1 4 5 1 M(p ; B 2 ) = 1 6 15 Computing Dp in terms of matrix multiplication with respect to B 3 and B 2 28
we should get back M(p ; B 2 ); indeed: M(Dp; B 2 ) = M(D; B 3, B 2 )M(p; B 3 ) = 0 1 0 0 2 0 0 2 0 1 3 0 0 0 3 5 = 1 6 15 = M(p ; B 2 ) We should also be able to compute Dp in terms of matrix multiplication but with respect to B 3 and B 2 and still get back M(p ; B 2 ); indeed: M(Dp; B 2 ) = M(D; B 3, B 2 )M(p; B 3) = 1 1 0 0 2 0 2 2 0 1 4 0 0 3 3 1 = 1 6 15 = M(p ; B 2 ) Remark: As we said earlier, the choice of bases determines the matrix representation M(T ; B V, B W ) of the linear map T L(V, W ). Later on we will prove important results about the choice of the bases the give the nicest possible matrix representation of T. Definition 26. A linear map T L(V, V ) =: L(V ) is an operator. Remark: For the matrix of an operator T L(V ), we assume that we take the same basis B V for both the domain V and the range V, and thus write it as M(T ; B V ) := M(T ; B V, B V ). Furthermore, M(T ; B V ) F n,n, where dim V = n, and so we see that M(T ; B V ) is a square matrix. Theorem 10. Suppose V is finite dimensional and T L(V ). Then the following are equivalent: 29
1. T is bijective (i.e., invertible) 2. T is surjective 3. T is injective Remark: Not true if V is infinite dimensional! Proof. We prove this by proving that 1 2 3 1. Clearly 1 2 so that part is done. Now suppose T is surjective, i.e., range T = V. Then by the Rank-Nullity Theorem: dim V = dim null T + dim range T dim V = dim null T + dim V dim null T = 0 null T = {0} T is injective So that takes care of 2 3. Now suppose T is injective. Then null T = {0} and dim null T = 0. Once again use the Rank-Nullity Theorem: dim V = dim null T + dim range T dim V = 0 + dim range T range T = V Thus T is surjective. Since we assumed it was injective, this means T is bijective and so we have 3 1 and we are done. 4 Polynomials Read on your own! 5 Eigenvalues, Eigenvectors, and Invariant Subspaces Extremely important subject matter that is the heart of Linear Algebra and is used all over mathematics, applied mathematics, data science, and more. 30
For example, consider a graph G = (V, E) consisting of vertices V and edges E; for example see Figure 1. You can encode this graph with a 6 6 matrix Figure 1: Graph with 6 vertices and 7 edges L so that: degree of vertex k, j = k L j,k = 1, j k and there is an edge between vertices j and k 0, otherwise This matrix is called the graph Laplacian and it encodes connectivity properties of the graph through its eigenvalues and eigenvectors. If the nodes in the graph represent webpages, and the edges represent hyperlinks between the webpages, then a similar type of matrix represents the world wide web, and its eigenvectors and eigenvalues form the foundation of how Google computes search results! 5.A Invariant Subspaces At the beginning of the course we defined a structure on sets V through the notion of a vector space. We then examined this structure further through subspaces, bases, and related notions. We then extended our study through linear maps between vector spaces, culminating in the Rank-Nullity Theorem and the notion of an isomorphism between two vector spaces with the same structure. Now we examine the structure of linear operators. The idea is that we will study the structure of T L(V ) by finding nice structural decompositions of V relative to T. Thought experiment: Let T L(V ) and suppose V = U 1 U m 31
To understand T, we would need only understand T k = T Uk for each k = 1,..., m. However, T k may not be in L(U k ); indeed, T k might map U k to some other part of V. This is a problem, since we would like each restricted linear map T k to be an operator itself on the subspace U k. This leads us to the following definition. Definition 27. Suppose T L(V ). A subspace U of V is invariant under T if T u U for all u U, i.e., T U L(U). Examples: {0}, V, null T, range T Must an operator have any invariant subspaces other than {0} and V? We will see... We begin with the study of one dimensional invariant subspaces. End of Lecture 9 32
Beginning of Lecture 10 Definition 28. Suppose T L(V ). A scalar λ F is an eigenvalue of T if there exists v V, v 0, such that T v = λv Such a v is called an eigenvector of T. Proposition 27. T L(V ) has a one dimensional invariant subspace if and only if T has an eigenvalue. Proof. First suppose that T has a one dimensional invariant subspace, which we denote as U. Since dim U = 1, U must be of the form: U = {λv : λ F} = span(v) for some v V, v 0. Since T is invariant under U, T v U. Thus there exists λ F such that T v = λv. Now suppose that T has an eigenvalue λ F. Then there exists v V, v 0, such that T v = λv. Then U = span(v) is an invariant subspace under T. Proposition 28. Suppose V is finite dimensional, T L(V ), and λ F. The following are equivalent: 1. λ is eigenvalue of T 2. T λi is not injective 3. T λi is not surjective 4. T λi is not invertible Example: The Laplacian for V = {f C ([ π, π]; C) : f( π) = f(π)} is defined as: f = d2 f dx 2 The eigenvalues and eigenvectors of are: λ = k 2, k Z, v(x) = e ikx = cos kx + i sin kx Notice the similarity between the eigenvectors of and the Fourier Transform defined earlier on Z N... 33
Theorem 11. Let T L(V ). If λ 1,..., λ m are distinct eigenvalues of T and v 1,..., v m are corresponding eigenvectors, then v 1,..., v m are linearly independent. Proof. Proof by contradiction. Suppose v 1,..., v m are linearly dependent. Using the LDL, let k be the smallest index such that Thus We also can conclude: v k span(v 1,..., v k 1 ) (8) v k = a 1 v 1 + + a k 1 v k 1 T v k = a 1 T v 1 + + a k 1 T v k 1 λ k v k = a 1 λ 1 v 1 + + a k 1 λ k 1 v k 1 v k = a 1 v 1 + + a k 1 v k 1 λ k v k = a 1 λ k v 1 + + a k 1 λ k v k 1 Combining the two expansions of λ k v k yields: 0 = a 1 (λ k λ 1 )v 1 + + a k 1 (λ k λ k 1 )v k 1 Since k is the smallest index satisfying (8), v 1,..., v k 1 must be linearly independent. Thus a 1 = = a k 1 = 0 since λ k λ j 0 for all k j. But then v k = 0, which is a contradiction. Corollary 5. Suppose V is finite dimensional. Then T L(V ) has at most dim V distinct eigenvalues. End of Lecture 10 34
Beginning of Lecture 11 5.B Eigenvectors and Upper-Triangular Matrices One of the main differences between operators and general linear maps is that we can take powers of operators! This will lead to many interesting results... Definition 29. Let T L(V ) and let m Z, m > 0. T m = T T (composition m times) T 0 = I If T is invertible, then T m = (T 1 ) m Definition 30. Suppose T L(V ) and let p P(F) be given by: Then p(t ) L(V ) is defined as: p(z) = a 0 + a 1 z + a 2 z 2 + + a m z m p(t ) = a 0 I + a 1 T + a 2 T 2 + + a m T m Theorem 12. Let V {0} be a finite dimensional vector space over C. Then every T L(V ) has an eigenvalue. Proof. Suppose dim V = n > 0 and choose v V, v 0. Then: L = v, T v, T 2 v,..., T n v is linearly dependent because the length of L is n + 1. Thus there exists a 0,..., a n C, not all zero, such that 0 = a 0 v + a 1 T v + a 2 T 2 v + + a n T n v Consider the polynomial p P(C) with coefficients given by a 0,..., a n. By the Fundamental Theorem of Algebra, p(z) = a 0 + a 1 z + + a n z n = c(z λ 1 ) (z λ m ), z C, where m n, c C, c 0, and λ k C. Thus: 0 = a 0 v + a 1 T v + + a n T n v = (a 0 I + a 1 T + + a n T n )v = c(t λ 1 I) (T λ m I)v Thus (T λ k I)v = 0 for at least one k, which means T λ k I is not injective, which implies that λ k is eigenvalue of T. 35
Example: Theorem 12 is not true for real vector spaces! Take for example the following operator T L(F 2 ) defined as: T (w, z) = ( z, w) If F = R, then T is a counterclockwise rotation by 90 degrees. Since a 90 degree rotation of any nonzero v R 2 will never equal a scalar multiple of itself, T has no eigenvalues! On the other hand, if F = C, then by Theorem 12 T must have at least one eigenvalue. Indeed it has two, λ = i and λ = i [see the book p. 135]. Recall we want a nice decomposition of V as V = U 1 U m, where each U k is an invariant subspace of T, so that to understand T L(V ) we only need to understand T Uk. We will accomplish this by finding bases of V that yield matrices M(T ) with lots of zeros. As a first baby step, let V be a complex vector space. Then T L(V ) must have at least one eigenvalue λ and a corresponding eigenvector v. Extend v to a basis of V : B V = v, v 2,..., v n Then: M(T ; B V ) = λ 0. 0 (9) Furthermore, if we define U 1 = span(v ) and U 2 = span(v 2,..., v n ), then V = U 1 U 2. The subspace U 1 is a one dimensional invariant subspace of V under T, but U 2 is not necessarily. It is a start though! Now let s try to do better... Definition 31. A matrix is upper triangular if all the entries below the diagonal equal 0: λ 1... 0 λ m There is a useful connection between upper triangular matrices and invariant subspaces: 36
Proposition 29. Suppose T L(V ) and B V = v 1,..., v n is a basis for V. Then the following are equivalent: 1. M(T ; B V ) is upper triangular 2. T v k span(v 1,..., v k ) for each k = 1,..., n 3. span(v 1,..., v k ) is invariant under T for each k = 1,..., n Proof. First we prove 1 2. Let A = M(T ; B V ). Then by the definition of A we have: T v k = A j,k v j But then Clearly 3 = 2 j=1 T v k span(v 1,..., v k ) A j,k = 0 j > k }{{} A is upper triangular We finish the proof by showing 2 = 3. Fix k. From 2 we have: T v 1 span(v 1 ) span(v 1,..., v k ) T v 2 span(v 1, v 2 ) span(v 1,..., v k ). T v k span(v 1,..., v k ) Thus if v span(v 1,..., v k ), then T v span(v 1,..., v k ) as well. Now can improve upon our baby step (9) above by showing that given an eigenvector v with eigenvalue λ, we can extend it to a basis B V such that M(T ; B V ) is upper triangular. Theorem 13. Suppose V is a finite dimensional complex vector space and T L(V ). Then there exists a basis B V such that M(T ; B V ) is upper triangular. End of Lecture 11 37
Beginning of Lecture 12 Warmup: Suppose T L(V ) and 6I 5T + T 2 = 0. What are the possible eigenvalues of T? Answer: 6I 5T + T 2 = 0 implies that (T 2I)(T 3I) = 0. Now let v 0 be an eigenvector of T with eigenvalue λ. Then 0 = (T 2I)(T 3I)v = (λ 2)(λ 3)v, which implies that λ = 2 or λ = 3. Theorem 14. Suppose V is a finite dimensional complex vector space and T L(V ). Then there exists a basis B V such that M(T ; B V ) is upper triangular. Proof. Induction on dim V. Clearly the result is true when dim V = 1. Now suppose the result is true for all complex vector spaces with dimension n 1 or less, and let V be a complex vector space with dim V = n. We know that V has one eigenvalue λ. Define: U = range (T λi) Since T λi is not surjective, dim U < dim V. Furthermore, U is invariant under T ; indeed, let u U: T u = (T λi)u }{{} U + λu }{{} U Thus T = T U L(U), and we can apply the induction hypothesis to T and U. In particular, there exists a basis B U = u 1,..., u m of U such that M( T ; B U ) is upper triangular. Extend B U to a basis for V : B V = u 1,..., u m, v 1,..., v l, l + m = n Since M( T ; B U ) is upper triangular, by Proposition 29 we have: Furthermore, T v j = (T λi)v }{{} j U T u k = T u k span(u 1,..., u k ) for all k = 1,..., m. + λv }{{} j span(u 1,..., u m, v j ) span(u 1,..., u m, v 1,..., v j ) span(v j ) 38
Thus T and B V satisfy condition 2 of Proposition 29, and so M(T ; B V ) is upper triangular. Upper triangular matrices a very useful for determining if T L(V ) is invertible... Proposition 30. Let T L(V ) and let B be a basis for which M(T ; B) is upper triangular. Then T is invertible all diagonal entries of M(T ; B) are nonzero Proof. Let B = v 1,..., v n and let A = M(T ; B). Easier to prove not (a) not (b). First suppose T is not invertible; we want to show that some entry of M(T ; B) is zero. T not invertible T not injective there exists v 0 such that T v = 0. Expand v in B: v = c j v j j=1 Let k be the index satisfying the following: c k 0 and c j = 0 for all j > k (note that possibly k = n). If k = 1, then v = c 1 v 1 T v 1 = 0 A 1,1 = 0. If k > 1 then: v = T v = k c j v j j=1 k c j T v j j=1 k 1 0 = c j T v j + c k T v k j=1 k 1 T v k = j=1 ( cj c k ) T v j span(v 1,..., v k 1 ), where in the last line we used Proposition 29. But also by Proposition 29, k 1 b j v j = T v k = j=1 k A j,k v j j=1 39
and since B is a basis we must have A k,k = 0. Now suppose some entry on the diagonal of M(T ; B) is zero. If A 1,1 = 0 then T v 1 = 0 and so T is not injective, and hence not invertible. If A k,k = 0 for k > 1, then by Proposition 29 we have: T v k = k k 1 A j,k v j = A j,k v j span(v 1,..., v k 1 ) (10) j=1 j=1 Consider now the linear map T = T span(v1,...v k ). By (10), T L(span(v 1,..., v k ), span(v 1,..., v k 1 )) Thus T cannot be injective since it maps a k-dimensional vector space to a (k 1)-dimensional vector space. In particular, there exists v span(v 1,..., v k ) such that T v = 0. But then T v = 0, and so T is not injective, and hence not invertible. End of Lecture 12 40
Beginning of Lecture 13 Not only can upper triangular matrices tell us when T L(V ) is invertible, they also tell us precisely what the eigenvalues of T are! Proposition 31. Let T L(V ) and suppose A = M(T ) is upper triangular. Then: λ is an eigenvalue of T λ = A k,k for some k Proof. Let A = M(T ) have diagonal entries given by A k,k = λ k : λ 1 A = M(T ) =... 0 λ m Let λ F. Then M(T λi) = λ 1 λ... 0 λ m λ Thus by Proposition 30 T λi is not invertible (and hence λ is an eigenvalue) if and only if λ = λ k for some k. 5.C Eigenspaces and Diagonal Matrices Definition 32. A diagonal matrix is a square matrix that is 0 everywhere except possibly the diagonal: λ 1 0... 0 λ m Note: If M(T ; B) is upper triangular, then the diagonal entries are precisely the eigenvalues of T (since diagonal matrices are upper triangular). Definition 33. Suppose T L(V ) and λ F. The eigenspace of T corresponding to λ is: E(λ, T ) = null (T λi) Note: T E(λ,T ) = λi (so eigenspaces are invariant subspaces) 41
Proposition 32. Suppose V is finite dimensional and T L(V ). Suppose also that λ 1,..., λ m are distinct eigenvalues of T. Then: is a direct sum and furthermore E(λ 1, T ) + + E(λ m, T ) (11) dim E(λ 1, T ) + + dim(λ m, T ) dim V Proof. Let u k E(λ k, T ) and suppose that u 1 + + u m = 0 Since eigenvectors corresponding to distinct eigenvalues are linearly independent, each u k = 0 and so (11) is a direct sum. Furthermore, by #16 of 2.C (HW1), dim E(λ 1, T )+ +dim E(λ m, T ) = dim(e(λ 1, T ) E(λ m, T )) dim V End of Lecture 13 42
Beginning of Lecture 14 Definition 34. An operator T L(V ) is diagonalizable if there exists a basis B such that M(T ; B) is diagonal. Proposition 33. Suppose V is finite dimensional and T L(V ). Then: T is diagonalizable V has a basis of eigenvectors of T. Proof. An operator T L(V ) has a diagonal matrix with respect to a basis B = v 1,..., v n if and only if T v k = λ k v k for each k. Example: Not every operator is diagonalizable, even over complex vector spaces! Consider T L(C 2 ) defined as: T (w, z) = (z, 0) Then T 2 = 0. Now let v 0 be an eigenvector with eigenvalue λ. Then 0 = T 2 v = T (T v) = λt v = λ 2. Thus λ = 0. Even though dim E(0, T 2 ) = 2, we see that E(0, T ) = {(w, 0) : w C} and so dim(0, T ) = 1. Therefore V does not have a basis of eigenvectors of T, and so T is not diagonalizable. We will address examples like this much later with the notion of generalized eigenvectors... On the other hand, if we have enough distinct eigenvalues, we know that T is diagonalizable: Proposition 34. If T L(V ) has dim V < distinct eigenvalues, then T is diagonalizable. Proof. Let dim V = n and suppose T L(V ) has distinct eigenvalues λ 1,..., λ n with corresponding eigenvectors v 1,..., v n. The eigenvectors are linearly independent because they correspond to distinct eigenvalues, and thus they form a basis for V. Thus T is diagonalizable. Note: The converse is not true! Take any diagonal matrix with non-unique entries on the diagonal. Finally, our main result for this chapter. Namely, if T is diagonalizable, then we can achieve our stated goal of decomposing V as V = U 1 U n, where each U k is an invariant subspace of V under T and dim U k = 1. 43
Theorem 15. Suppose V is finite dimensional and T L(V ). Let λ 1,..., λ m denote distinct eigenvalues of T. Then the following are equivalent: 1. T is diagonalizable 2. V has a basis consisting of eigenvectors of T 3. There exist one dimensional invariant subspaces U 1,..., U n of V such that V = U 1 U n 4. V = E(λ 1, T ) E(λ m, T ) 5. dim V = dim E(λ 1, T ) + + dim E(λ m, T ) Proof. Many parts. The plan is: 1 2 3, 2 = 4 = 5 = 2 1 2: Simply Proposition 33. 2 = 3: Let B = v 1,..., v n be basis of eigenvectors of V. Define U k = span(v k ). Then each U k is a 1-dimensional invariant subspace of V under T, and since B is a basis it is clear V = U 1 U n. 3 = 2: For each k, let v k U k, v k 0. Since U k is a 1-dimensional invariant subspace under T, each v k is an eigenvector of T. Furthermore each v V can be written uniquely as: v = u 1 + + u n, where u k U k and therefore u k = a k v k for some a k F. Thus v 1,..., v n is a basis for V. 2 = 4: Let v 1,..., v n be a basis of eigenvectors for V, and subdivide the list according to the unique eigenvalues of T, so that: v (l) 1,..., v(l) k l corresponds to λ l, for l = 1,..., m and k 1 + k 2 + + k m = n. Then any v V can be written as: v = m k l l=1 j=1 a j,l v (l) j } {{ } E(λ l,t ) E(λ 1, T ) E(λ m, T ) 44
4 = 5: This is simply 2.C #16, which you did for homework! 5 = 2: Choose a basis for each E(λ l, T ), say v (l) 1,..., v(l) k l, where k 1 + + k m = n by assumption. Let L be the list of all of these vectors concatenated together. To show L is linearly independent, suppose: m k l l=1 j=1 a j,l v (l) j = 0 }{{} u l E(λ l,t ) m u l = 0 l=1 Each u l is eigenvector of T corresponding to a distinct eigenvalue λ l ; thus u 1,..., u m must be linearly independent and so u l = 0 for all l. But then a j,l = 0 for all j = 1,..., k l and for each l, since v (l) 1,..., v(l) k l are linearly independent. End of Lecture 14 45
Beginning of Lecture 15 6 Inner Product Spaces We now introduce geometrical aspects such as length and angle into the setting of abstract vector spaces. 6.A Inner Products and Norms We begin by looking at R n. Definition 35. The norm of x = (x 1,..., x n ) R n is: x = x 2 1 + + x2 n Definition 36. For x, y R n, the dot product of x and y is: Notice that x 2 = x x. x y = x 1 y 1 + + x n y n. Example: In R 2, x = x 2 1 + x2 2 which is just the length of x, and where θ is the angle between x and y. Properties of the dot product: x x 0 x R n x x = x 2 = 0 x = 0 x y = y x x y = x y cos θ, Fix y R n. Then T y (x) = x y is a linear map, i.e., T y L(R n, R). Now we want to generalize the dot product to abstract vector spaces. First lets consider C n. Let λ = a + ib C be a complex scalar. Recall that: λ = a 2 + b 2 46
λ 2 = λ λ For z C n, the norm is defined as: Note that: z = z 1 2 + + z n 2 z 2 = z 1 z 1 + + z n z n If we want z z = z 2, then the previous line implies that we should define the dot product on C n as: w z = w 1 z 1 + + w n z n This leads us to the generalization of the dot product to abstract vector spaces: Definition 37. An inner product on V is a function, : F 2 F that has the following properties: 1. Positive Definitness: v, v 0 v V v, v = 0 v = 0 2. Linearity in the first argument: u + v, w = u, w + v, w u, v, w V λu, v = λ u, v λ F, u, v V 3. Conjugate Symmetry: u, v = v, u u, v V Examples: 1. Euclidean inner product on F n. Let w = (w 1,..., w n, z = (z 1,..., z n ) F n : w, z = w 1 z 1 + + w n z n 2. Weighted Euclidean inner product on F n. Fix c = (c 1,..., c n ) R n with c k 0. Then for w, z F n, w, z c = c 1 w 1 z 1 + + c n w n z n 47
3. Define V = L 2 (R) as: L 2 (R) = {f : R R : f(x) 2 dx < } One can verify this is a real vector space. Since it is a subset of the vector space of all functions mapping R to R, we need to show (1) it contains an additive identity (zero), (2) it is closed under addition, and (3) it is closed under scalar multiplication. Indeed, f 0 L 2 (R), and furthermore if f L 2 (R) then λf L 2 (R) for any λ R since λf(x) 2 dx = λ 2 f(x) 2 dx < The trickiest part is that it is closed under addition; i.e., if f, g L 2 (R), then f + g L 2 (R). First note: f(x)+g(x) 2 dx = + g(x) 2 dx +2 f(x) 2 dx } {{ } I } {{ } II f(x)g(x) dx } {{ } III Since f, g L 2 (R), we know that the first two terms are finite. That leaves the third term. That this is finite follows from what s known in Real Analysis as Hölder s Inequality. However, we can in fact prove it with more elementary tools. First let a, b R and note that: (a b) 2 0 a 2 2ab + b 2 0 ab a2 2 + b2 2 Now let f(x) = a and g(x) = b. Then: f(x)g(x) dx f(x) 2 2 + g(x) 2 2 dx < (12) Thus L 2 (R) is a vector space! We can add an inner product to it by defining the inner product as: f, g = f(x)g(x) dx By what we just showed in (12), the inner product is well defined. Furthermore, it is easy to verify that all of the properties of an inner 48
product hold, except for definiteness property: f, f = 0 f = 0. This is a bit technical but follows from Real Analysis. Now L 2 (R) is what we call an inner product space. Any inner product can always be used to define the norm of a vector. In this case, we get the L 2 -norm: f 2 = ( f, f = ) 1/2 f(x) 2 dx In fact L 2 (R) is a special inner product space called a Hilbert space, but we leave that for more advanced math classes... Definition 38. An inner product space is a vector space V along with an inner product on V. Important Note: For the rest of chapter 6, we assume V is an inner product space. Definition 39. For v V an inner product space, the norm of v is: End of Lecture 15 v = v, v, 49
Beginning of Lecture 16 Proposition 35. The following basic properties hold: 1. For each fixed u V, the function T u (v) = v, u is linear, i.e., T u L(V, F). 2. 0, v = 0 v V 3. v, 0 = 0 v V 4. u, v + w = u, v + u, w u, v, w V 5. u, λv = λ u, v λ F and u, v V 6. v = 0 v = 0 7. λv = λ v λ F Proof. The proofs are all very simple and in the book. Definition 40. u, v V are orthogonal if u, v = 0. In plane geometry, two vectors are orthogonal if they are perpendicular, see Figure 2. Figure 2: Orthogonal line segments It is easy to see the following two basic facts: 0 is orthogonal to every v V 0 is the only vector in V orthogonal to itself Theorem 16 (Pythagorean Theorem). Suppose u and v are orthogonal vectors in V. Then: u + v 2 = u 2 + v 2 50