Linear Algebra (Part II) Vector Spaces, Independence, Span and Bases A vector space, or sometimes called a linear space, is an abstract system composed of a set of objects called vectors, an associated field of scalars, together with the operations of vector addition and scalar multiplication Let V denote the set of vectors and F denote the field of scalars Here I ll use bold lowercase Roman letters to signify vectors, ie x V, and lowercase Greek letters to signify scalars, ie α F I m going to list out now what properties vector addition and scalar multiplication are required to satisfy on a given vector space (a ) For every x and y V we have x+y V (a ) For every x, y and z V we have (x+y)+z = x+(y+z) (a ) For every x and y V we have x+y = y+x (a 3) There is a vector V such that x+ = x for every x V (a 4) For every x V there is a vector x V such that x+x = (m ) For every α F and x V we have αx V (m ) For every α and β F we have α(βx) = (αβ)x for every x V (m ) If F is the scalar field s multiplicative identity then x = x for any x V (d ) For every α F and x and y V we have α(x+y) = αx+αy (d ) (α+β)x = αx+βx Let me say some words about these items (a ) says V is closed under vector addition (m ) says V is also closed under scalar multiplication (a ) and(a ) say vector addition must be associative and commutative (a 3) says V must contain the additive identity (a 4) says every vector in V has its additive inverse in V (d ) and (d ) are the scalar vector distributive properties When F = R one often calls the vector space a real vector space, and it s often called a complex vector space when F = C In this course we will study only one particular type of vector space The vectors themselves are column matrices x = x x m,
and the scalar field will be either the real or complex numbers For now, let s assume all numbers are real Vector addition and scalar multiplication is defined exactly as done for matrices on your previous homework, x y x =, y = x m y m, α R x+y x +y x m +y m and αx αx αx m I will use the symbol R m to denote the vector space whose vectors are composed of all such real m matrices Please check on your own that R m satisfies all requirements (a ) thru (d ) listed above Suppose V is a vector space, ie it is composed of a given set of vectors V with associated scalar field F, and its notion of vector addition and scalar multiplication satisfies requirements (a ) thru (d ) A subspace S of V, denoted by S V, shares the same scalar field F with the parent space V, and inherits the notion of vector addition and scalar multiplication from the parent, but its vectors are composed of a subset S V of V s vectors However, a subspace is not just a subset of vectors from a vector space, it is more If we say S is a subspace of V, then it must also be a vector space on its own Here are three examples to help clarify what a subspace is Consider a subset of vectors from the vector space R S = {x R : x = } Does this set define a subspace of R? The answer is no S is neither closed under vector addition nor scalar multiplication For example x = S, y = S, but x+y = S, and also x = Here s a second example Consider S, α = R but αx = S = {x R : x > } S This set is closed under vector addition since for arbitrary x S and y S x y x +y x = S, y = S x x y >, y > x+y = S, x +y since x +y > But it s not closed under scalar multiplication since for example x = S, α = R but αx = S Therefore, S = {x R : x > } does not define as subspace of R
Here s a third example Consider S = {x R : x +x = } This set is closed ( under ) vector addition since x y x = S, y = S x x y +x =, y +y = x +y x+y = S x +y Note x+y S because (x +y )+(x +y ) = x +x +y +y = + = Moreover x αx x = S x x +x = α R, αx = S, αx since (αx )+(αx ) = α(x +x ) = α = So, S isalsoclosedunderscalarmultiplication As you ll see in a moment, this is enough to conclude S = {x R : x +x = } does define a subspace of R On their own, the nine requirements I listed above, (a ) thru (d ), are independent of eachother However, for a subset of vectors which inherits its structure (addition, etc) from a parent vector space, some requirements become redundant Clearly, all requirements except (a ), (a 3), (a 4) and (m ) are automatically satisfied for such a subsystem In fact, as you will show in an exercise, if S is a nonempty subset of vectors from a parent vector space and S is closed under the parent s addition and scalar multiplication, ie (a ) and (m ) are true, then (a 3) and (a 4) must also be true This allows us to state the following If S is a nonempty subset of vectors from a parent vector space and S is closed wrt the parent s vector addition and scalar multiplication, then S defines a subspace of the parent space Suppose S is a nonempty subset of vectors from a (real) vector space Also, suppose S is closed wrt the parent s vector addition and scalar multiplication (a) Show the additive identity is in S (b) Show that for any x S there is a x S such that x+x = Hint: (a) (+)x = x (b) x x S Determine whether or not the following sets define a subspace of R (a) {x R : x = } (c) {x R : x +x } (b) {x R : x x = } (d) {x R : x +x = } Either prove the set is closed under both vector addition and scalar multiplication or give an example to show one is not 3
Consider a set of n vectors {x,,x n } This set is called a dependent set if there are n scalars, α,,α n, which are not all zero such that α x + +α n x n = A set of vectors that is not dependent is called an independent set Given that the vectors in the set above come from the vector space R m, we can use matrix elimination to determine whether the set is independent or not The problem can be recast as follows α x + +α n x n = x, x,n x m, x m,n α α n = Make sure you work this out on your own Check that the jth column of the m n matrix on the right is the column vector x j R m The zero matrix on the right has size m If the only solution to this linear system is α = = α n =, then the set is independent If the system has a nontrivial solution however, the set is dependent Consider the following four vectors from R 3 x, x 3 4 6, x 3 I m going to use these in the next two examples 7 8, x 4 Is the set {x,x,x 3 } an independent set? The augmented matrix to consider is [ 4 7 [ 4 7 [ 4 7 8 3 6 3 6 6 Back substitution tells us α 3 = α, α = α and α = 4( α) 7(α) = α for any real number α WLOG take α = to see x x +x 3 =, and conclude {x,x,x 3 } is not an independent set of vectors Is the set {x,x,x 4 } an independent set? The augmented matrix to consider here is [ 4 7 [ 4 7 [ 4 7 [ 4 7 8 3 6 3 3 6 6 8 3 7 8 3 This time back substitution tells us α 3 =, α = and α = Therefore α x +α x +α 3 x 4 = α = α = α 3 =, and we conclude {x,x,x 4 } is an independent set of vectors 4
3 Prove the following A set of vectors {x,,x n } (assume n ) is dependent if and only if at least one its vectors can be written as a linear combination of the others Hint: Consider x i = k i α k x k for some index i n Consider the following vectors from R 4 7 4 x =, x =, x 4 3 = 3 4 Is {x,x,x 3 } an independent set of vectors? Is {x,x,x 4 } an independent set of vectors? 6 Is {x,x 3,x 4 } an independent set of vectors? 4, x 4 = 4 Considerafinitesetofvectorsfrom R m, {x,,x n } Thespanofthissetisthesubspace of R m defined by That is, span{x,,x n } { n k= α kx k : each α k R} y span{x,,x n } y = α x + +α n x n for some set of real numbers α,,α n In other words, a vector is in span{x,,x n } when it can be written as a linear combination of the specified vectors x,,x n Clearly span{x,,x n } is closed under vector addition and scalar multiplication and is therefore a subspace of R m regardless of what the set {x,,x n } is If {x,,x n } is an independent set of vectors and y span{x,,x n } then the decomposition y = α x + +α n x n is unique Let me show you why Suppose there are two ways to decompose y, say y = α x + +α n x n and y = β x + +β n x n = (α β )x + +(α n β n )x n (α β ) = = (α n β n ) = This last step follows from the fact that {x,,x n } is an independent set So, since we have α k = β k for each k =,,n, the two decompositions above are in fact identical
It s not hard to show the following If {x,,x n } is a dependent set of vectors and y span{x,,x n } then the decomposition y = α x + +α n x n is not unique You are asked to show this in exercise 7 below Now, how do we compute whether or not a given vector is in a span? We ll use elimination of course Consider the subspace S span{x,x,x 3 } R 4 where 7 4 x =, x =, x 4 3 = 3 4 Is y 7 7 span{x,x,x 3 }? The linear system we have to solve is 7 4 4 3 4 α α α 3 = 7 7 and we eliminate the augmented matrix to obtain 7 7 7 4 4 3 4 7, 7 Now, use back substitution See that α 3 is a free variable, so let α 3 = α where α is any real number Then, α = (+α) and α = ( 3α) So we get 7 y = = ( 3α)x + (+α)x +αx 3 S 7 Therefore we see y S Moreover, since the decomposition is not unique, ie α here can be any real number, we also conclude the set of vectors {x,x,x 3 } is not independent (Look back at exercise above) Let me change y by a little bit and ask the same question 7 Is y 6 span{x,x,x 3 }? The augmented matrix to consider here is 7 4 4 3 4 7 6 7 6 / 7 /
However, the third row in the right above says α + α + α 3 = /, and this is impossible Therefore, this time y span{x,x,x 3 } 7 Suppose {x,,x n } is a dependent set and y span{x,,x n } Prove there are an infinite number decompositions such that y = α x + +α n x n Hint Since {x,,x n } is a dependent set, there are numbers β,,β n which are not all zero such that β x + +β n x n = 8 Let {x,x,x 3 } come from exercise 4 above Determine if the given vector y is in span{x,x,x 3 } Ifitis, writedownandcheckthedecomposition y = α x +α x +α 3 x 3 7 (a) y = (b) y = 3 3 4 8 A basis for a vector space is a linearly independent spanning set That is, {b,,b n } is a basis for a vector space V if: () {b,,b n } is an independent set () V = span{b,,b n } The dimension of a vector space is the number of basis vectors needed to span it It s not obvious, but this number is independent of any particular spanning basis Clearly, R = span{e,e }, where e =, e =, and so R is two dimensional (Duh) Not as obvious, here s another basis for R R = span{b,b }, where b =, b = 3 The basis {e,e } is called the standard basis for R The standard basis for R m is e =, e =,, e m =, e m = One might think that the standard basis for R m is the most useful of all of its bases But it really depends on the application Later in this course we will consider others 7
Let me close out this assignment by showing you, by example, how to convert a given basis for a subspace of R m to its standard basis Recall from exercise 4 you showed 4 x =, x = 3 7 4, x 3 = 4 is an independent set Therefore {x,x,x 3 } is a basis for S span{x,x,x 3 } To determine S s standard basis, write out an augmented matrix using these three column vectors as rows 4 3 7 4 4 Notice there s no vertical bar ( ) here Now, row reduce to row echelon form 4 3 8 8 4 3 8 8 4 3 /8 Notice on the right I ve scaled all pivots to one Finally, starting from the right most pivot, use backward elimination to get 4 This is called the row canonical form or sometimes the reduced row echelon form for the augmented matrix See https://wikipediaorg/wiki/row echelon form The standard basis for the subspace S span{x,x,x 3 } can now be read off as follows S = span{e,e,e 3 }, where e =, e =, e 3 = BTW I checked my calculation by observing x = e +4e 3e 3 x = 7e +e e 3 x 3 = e +e 4e 3, Find the standard basis for span{x,x 3,x 4 } from exercise 6 The set {x,x,x 4 } from exercise is not independent However, it s still possible to determine the standard basis for span{x,x,x 4 } as just done You ll get a zero row when 8
eliminating to row canonical form Disregard the zero row when you read off your basis What is the dimension of span{x,x,x 4 }? Answer: two