A Do It Yourself Guide to Linear Algebra

Transcription

1 A Do It Yourself Guide to Linear Algebra Lecture Notes based on REUs, Instructor: László Babai Notes compiled by Howard Liu Vector Spaces 1.1 Basics Definition A vector space (over the reals) is a set V with (a) addition V V V, (x, y) x + y, and (b) scalar multiplication R V V, (α, x) αx, satisfying following axioms (a) (V, +) is an abelian group, i. e., (a1) ( x, y V )(! x + y V ), (a2) ( x, y V )(x + y = y + x) (commutative law), (a3) ( x, y, z V )((x + y) + z = x + (y + z)) (associative law), (a4) ( 0 V )( x)(x + 0 = 0 + x = x) (existence of zero), (a5) ( x V )( ( x) V )(x + ( x) = 0), b) (b1) ( x V )( α R)(! αx V ) (b2) ( α, β R)( x V )(α(βx)) = (αβ)x ( associativity linking two operations), (b3) ( α, β R)( x V )((α + β)x = αx + βx) (distributivity over scalar addition), (b4) ( α R)( x, y V )(α(x + y) = αx + αy) (distributivity over vector addition), 1

2 (c) ( x V )(1 x = x) (normalization). Exercise Show ( x V )(0x = 0). (The first 0 is a number, the second a vector.) Exercise Show ( α R)(α0 = 0). Exercise Show ( α R)( x V )(αx = 0 (α = 0 or x = 0)) Example Examples of vector spaces 1. R n - the set of n-tuples of real numbers 2. geometric vectors in 2 or 3 dimensions 3. k l matrices 4. C[0, 1], the continuous functions from [0,1] to R 5. the space of infinite sequences of real numbers 6. R[x] = {polynomials with real coefficients}. 7. R(x) = {rational functions with real coefficients} (see Definition below). 8. R Ω, the functions from Ω to R Ω = {1,..., n} is example 1 above Ω = {1,..., k} {1,..., l} is example 3 Ω = [0, 1] contains example 4 Ω = N is example 5 Definition Rational functions are equivalence classes of formal fractions p q q R[x], q 0, under the equivalence relation p 1 q 1 = p 2 q 2 iff p 1 q 2 = p 2 q 1. with p, Definition A linear combination of vectors v 1,..., v k V is a vector α 1 v 1 + +α k v k where α 1,... α k R. Definition The span of v 1,..., v k V is the set of all linear combinations of v 1,..., v k, i. e., Span(v 1,..., v k ) = {α 1 v α k v k α 1,..., α k R}. Remark We let Span( ) = {0}, by virtue of the fact that an empty sum is always zero. Definition A linear combination of an infinite list of vectors is a linear combination of a finite sublist. The span of an infinite list S of vectors is, as defined before, as the set of all linear combinations of S. 2

3 Note that 0 Span(T ) for any list T of vectors, because the trivial linear combination (all coefficients are 0) is 0. Exercise Prove: ( S V )(Span(Span(S)) = Span(S)). Definition A list S of vectors generates V if Span(S) = V. Definition U V is a subspace if U = Span(U). Notation: U V. Exercise Prove: a subset U V is a subspace exactly if it is closed under linear combinations. Exercise Prove: a subset U V is a subspace exactly if (a) 0 U; (b) ( x, y U)(x + y U); (c) ( x U)( α R)(αx U). Exercise Prove: a subspace is a vector space (under the same operations). Exercise Show S V, Span(S) V, i.e. that Span(S) is a subspace of V. (Recall that Span( ) = {0}.) Exercise Show that Span(S) is the smallest subspace containing S, i.e., (a) S Span(S) V and (b) If S W V then Span(S) W. Exercise Show that the intersection of any set of subspaces is a subspace. Remark Note that this doesn t hold true for unions. Exercise If U 1, U 2 V then U 1 U 2 is a subspace if and only if U 1 U 2 or U 2 U 1. Exercise Span(S) = S W V W. 1.2 Linear Independence Definition Vectors v 1,..., v k V are linearly independent if only their trivial linear combination gives 0, i. e., ( α 1,..., α k R)(α 1 v α k v k = 0 α 1 = = α k = 0.) Definition An infinite list of vectors is linearly independent if all finite sublists are linearly independent. Exercise Prove: The empty list is linearly independent. 3

4 Exercise Prove: A one-element lists {v} of vectors is linearly independent v 0. Exercise A list containing 0 is never linearly independent. Exercise A list of vectors with repetitions is never linearly independent. Exercise Show that if T S V and S is linearly independent then T is linearly independent. Exercise Find a curve in R n, i.e., a continuous function f : R R n, such that any n points on the curve are linearly independent, i.e., for any n distinct real numbers α 1,..., α n, the vectors f(α 1 ),..., f(α n ) are linearly independent. (Give a simple explicit formula.) Definition We say that vectors u, v V are parallel if u, v 0 and ( α R)(u = αv). Exercise Show that vectors u, v V are linearly dependent if and only if u = 0 or v = 0 or u, v are parallel. Remark We say that a property P is a finitary property if a set S has the property P if and only if all finite subsets of S have property P. Note that linear independence is, by Definition 1.2.2, a finitary property. Exercise (Erdős debruijn) Show that 3-colorability of a graph is a finitary property. (The same holds for 4-colorability, etc.) (Hints for three different proofs. (1) Use Zorn s Lemma. (2) Use the Compactness Theorem of first-order logic. (3) Use Tikhonoff s Theorem (compactness of the product of compact topological spaces). 1 1 Exercise Let α 1,..., α n be distinct real numbers. Prove: x α 1,..., independent rational functions. x α n are linearly Exercise Prove: for all α, β R, sin(x), sin(x + α), sin(x + β) are linearly dependent functions R R. Exercise Prove: 1, cos(x), cos(2x), cos(3x),..., sin(x), sin(2x),... are linearly independent functions R R. 1.3 Rank, Dimension Definition A maximal linearly independent subset of a set S V is a subset T S such that (a) T is linearly independent, and (b) if T T S then T is linearly dependent. Definition A maximum linearly independent subset of a set S V is a subset T S such that 4

5 (a) T is linearly independent, and (b) if T S is linearly independent then T T. Exercise (Independence of vertices in a graph.) Find a graph which has a maximal independent set of vertices which is not maximum. We shall see that this cannot happen with linear independence: every maximal linearly independent set is maximum. Exercise Let S V. Then there exists T S such that T is a maximal independent subset of S. Exercise Let L S V. Assume L is linearly independent. Then there exists a maximal linearly independent subset T S such that L T. (Every linearly independent subset of S set can be extended to a maximal linearly independent subset of S.) Remark It is easy to prove Exx , by successively adding vectors until our set becomes maximal as long as all linearly independent subsets of S are finite. For the infinite case, we need transfinite induction or a result from set theory called Zorn s Lemma (a version of the Axiom of Choice). Definition A vector v V depends on S V if v Span(S), i. e. v is a linear combination of S. Definition A set of vectors T V depends on S V if T Span(S). Exercise Show that dependence is transitive: if R Span(T ) and T Span(S) then R Span(S). ( A linear combination of linear combinations is a linear combination. ) Definition A vector v depends on a list S of vectors if v Span(S). A list T of vectors depends on a list S of vectors if every member of T depends on S. Exercise Prove: dependence is transitive, i. e., if R, S, T are lists of vectors, R depends on S and S depends on T then R depends on T ( linear combinations of linear combinations are linear combinations). Exercise Suppose that α i v i is a nontrivial linear combination. Then ( i) such that v i depends on the rest (i. e. on {v j j i}). Indeed, this will be the case whenever α i 0. Exercise A list S of vectors is linearly dependent iff some member of the list depends on the rest. Exercise If v 1,..., v k are linearly independent and v 1,..., v k, v k+1 are linearly dependent then v k+1 depends on v 1,..., v k. The next fundamental result asserts the impossibility of boosting linear independence. 5

6 Theorem (First Miracle of Linear Algebra). If v 1,..., v k are linearly independent and they all depend on w 1,..., w l then k l. We shall prove this in Section 1.6. Now we derive some corollaries. Corollary All maximal linearly independent subsets of a set S V are maximum. Exercise If T S, T is a maximal independent subset of S then S Span(T ). Exercise Prove Corollary from Theorem and Exercise Definition For S V, the rank of S is the common cardinality of all the maximal independent subsets of S. Notation: rk(s). Definition The dimension of a vector space is dim(v ) := rk(v ). Exercise rk(s) = rk(span(s)) = dim(span(s)). Exercise Show that dim(r n ) = n. Exercise Let P k be the space of polynomials of degree k. Show that dim(p k ) = k+1. Exercise Let T = {sin(x + α) α R}. Prove rk(t ) = 2. Notation: [n] = {1, 2,..., n}. Exercise Let us say that the set system A 1,..., A m [n] is k-intersecting if ( i j)( A i A j = k). Show that for every n there exists a 1-intersecting family of n distinct subsets of [n]. ( Family is just another word for set system, i.e., a list of sets.) Exercise (a) Construct a 1-intersecting family of 7 3-subsets of [7]. (A 3-subset is a set of 3 elements.) (b) For every prime p, construct a 1-intersecting family of p 2 + p + 1 (p + 1)-subsets of [p 2 + p + 1]. Exercise (Generalized Fisher Inequality) Prove: if k 1 and A 1,..., A m is a k-intersecting family of distinct subsets of [n] then m n. (Hint. First Miracle.) Exercise (a) Construct a set of n points in R n such that each pair is at the same distance. (b) Construct a set of n + 1 points in R n such that each pair is at the same distance. (c) Prove: there is no set of n + 2 points in R n such that each pair is at the same distance. A 2-distance set is a set of points with only two distinct distances between them. Example: the regular pentagon in the plane. Exercise Prove: a 2-distance set in the plane has at most 5 points. Exercise Find a 2-distance set of quadratic size (at least cn 2 for some positive constant c) in R n. Exercise Prove: the size of a 2-distance set in R n is at most quadratic ( Cn 2 for some constant C). (Hint. First Miracle.) 6

7 1.4 Basis Definition A basis of V is a linearly independent set of generators of V, i.e., it is a linearly independent set that spans V. Definition A basis of S V is a linearly independent subset of S on which S depends. In other words, a basis B of S is a linearly independent set satisfying B S Span(B). Exercise A list B of vectors is a basis of V iff every vector can be written uniquely as a linear combination of B. Definition Let B = (b 1,..., b n ) be a basis of V. Write x V as x = n i=1 β ib i. The coefficients in this unique linear combination are called the coordinates of x with respect to B. Definition For a basis B of V, regarded as a list of vectors, we associate with each x V the column vector [x] B := β 1. β k where the β i are the coordinates of x with respect to B. Exercise B is a basis of S if and only if B is a maximal independent subset of S. In view of Ex we have the following corollary. Corollary Every vector space has a basis. In fact, every subset of a vector space has a basis. (Note: in the infinite dimensional case the proof requires Zorn s Lemma.) Exercise Show that if the vectors of B S are linearly independent, then B can be extended to a basis of S. Exercise Show that every set of generators of V contains a basis. In fact this is true for any subset S V : if T S such that T generates S, i.e., S Span(T ), then there exists a basis B of S such that B T. Exercise Prove: All bases for S have equal size. Show that the this statement is equivalent to the First Miracle, Theorem Exercise Prove: if B is a basis of V then dim(v ) = B. Exercise A Fibonacci-type sequence is a sequence (a 0, a 1, a 2,... ) such that ( n)(a n+2 = a n+1 + a n ). (a) Prove that the Fibonacci-type sequences form a 2-dimensional vector space. (b) Find a basis in this space consisting of two geometric progressions. (c) Express the Fibonacci sequence (0, 1, 1, 2, 3, 5, 8, 13,... ) as a linear combination of the basis found in item (b). 7

8 1.5 Linear maps, isomorphism of vector spaces Definition Let V and W be vector spaces. homomorphism or a linear map if We say that a map f : V W is a (a) ( x, y V )(f(x + y) = f(x) + f(y)) (b) ( x V )( α R)(f(αx) = αf(x)) Exercise Show that if f is a linear map then f(0) = 0. Exercise Show that f( k i=1 α iv i ) = k i=1 α if(v i ). Definition We say that f is an isomorphism if f is a bijective homomorphism. Definition Two spaces V and W are isomorphic if there exists an isomorphism between them. Exercise Show the relation of being isomorphic is an equivalence relation. Exercise Show that an isomorphism maps bases to bases. Therefore, isomorphic vector spaces have the same dimension. Theorem If dim(v ) = n then V = R n. Proof: Choose a basis, B of V, now map each vector to its coordinate vector, i.e., v [v] B. Exercise Prove: two vector spaces are isomorphic if and only if they have the same dimension. Exercise (Degree of freedom in choosing a linear map) Let V, W be vector spaces. Let B = (b 1,..., b k ) be a basis of V. Let w 1,..., w k be arbitrary vectors in W. Prove: there is a unique linear map f : V W such that ( i)(f(b i ) = w i ). Exercise Show that the linear maps V W form a vector space. This space is called Hom(V, W ). Prove that its dimension is dim(v ) dim(w ). Definition The image of f is the set Im(f) = {f(x) : x V } Definition The kernel of f is the set Ker(f) = {x V : f(x) = 0} Exercise For a linear map f : V W show that Im(f) W and Ker(f) V. Theorem For a linear map f : V W we have dim Ker(f) + dim Im(f) = dim V. Lemma If U V and A is a basis of U then A can be extended to a basis of V. Exercise Prove Theorem Hint: apply Lemma setting U = Ker(f). 8

9 1.6 Proof of the First Miracle Lemma (Exchange Principle). If v 1,..., v k are linearly independent and v 1,..., v k Span(w 1,..., w l ) then j, 1 j l such that w j, v 2,..., v k are linearly independent. (Note in particular that w j v 2,..., v k.) Exercise Prove the Exchange Principle. Exercise If a 1,..., a k are linearly independent and ( i)(a i Span{b 1,..., b l }) then ( i)( j)(a 1,..., a i 1, b j, a i+1,..., a k ) are linearly independent. Exercise Prove the First Miracle, Theorem , using the preceding exercises. Here is another, more commonly used exchange principle from which the equality of bases and therefore the First Miracle follows. Theorem (Steinitz Exchange Principle). Let B be a basis of V, and let L be a linearly independent set of vectors in V. Then there exists a subset M S such that M = L and (S \ M) L is also a basis of V. Exercise Prove the Steinitz Exchange Principle. Do not use consequences of the First Miracle. Exercise Use the Steinitz Exchange Principle to prove that all bases have equal size. Infer the First Miracle. 1.7 Matrix rank; the Second Miracle Definition An m n matrix is an m n array of numbers {α ij } which we write as α 11 α 1n..... α m1 α mn Definition The row rank (or column rank) of a matrix is the rank of the list of its row vectors (column vectors, respectively). Theorem (Second Miracle of Linear Algebra). The row rank of a matrix is equal to its column rank. Definition The following actions on a set of vectors {v 1,..., v k } are called elementary operations: (a) Replace v i by v i αv j where i j. (b) Replace v i by αv i where α 0. 9

10 (c) Switch v i and v j. Exercise Show that the rank of a list of vectors doesn t change under elementary operations. Exercise Let {v 1,..., v k } have rank r. Show that by a sequence of elementary operations we can get from {v 1,..., v k } to a set {w 1,..., w k } such that w 1,..., w r are linearly independent and w r+1 = = w k = 0. Consider a matrix. An elementary row-operation is an elementary operation applied to the rows of the matrix. Elementary column operations are defined analogously. Exercise shows that elementary row-operations do not change the row-rank of A. Exercise Show that elementary row-operations do not change the column-rank of a matrix. Exercise Use Exercises and to prove the Second Miracle. 10