4.3 - Linear Combinations and Independence of Vectors

Transcription

1 - Linear Combinations and Independence of Vectors De nitions, Theorems, and Examples De nition 1 A vector v in a vector space V is called a linear combination of the vectors u 1, u,,u k in V if v can be written in the form v = c 1 u 1 + c u + + c k u k, where c 1 ; c ; :::; c k are scalars Example For the set of vectors in R, S = f(1; ; 1) ; (; 1; ) ; (1; ; 5)g, v 1 = v + v = (; 1; ) + (1; ; 5) = (1; ; 1) Example For the set of vectors in M ;, (the set of matrices) 8 1 S = ; ; ; v 1 is a linear combination of v ; v, and v because Example How to find a linear combination 1 = Given the set S = f(1; ; ) ; (; 1; ) ; ( 1; ; 1)g, write the vector (1; 1; 1) as a linear combination of vectors in the set S Solution 5 We need to nd scalars c 1 ; c, and c such that, (1; 1; 1) = c 1 (1; ; ) + c (; 1; ) + c ( 1; ; 1) = (c 1 c ; c 1 + c ; c 1 + c + c ) which leads to the system 8 < : c 1 c = 1 c 1 + c = 1 c 1 + c + c = 1 with augmented coe cient matrix and subsequent reduced-row echelon form rref : 1 1! The resulting system has a free variable c = t such that c 1 = 1 + t; c = 1 t, and c = t Since c is free, one such linear combination is therefore when t =, so that c 1 = 1, c = 1, c =, and (1; 1; 1) = (1) (1; ; ) + ( 1) (; 1; ) + () ( 1; ; 1) Write the vector (1; ; ) as a linear combination of vectors from the same set S as above Solution Setting up and solving the system as done above, we end up with the reduced-row echelon form whereby we conclude that the system is inconsistent Hence we cannot write (1; ; ) as a linear combination of the vectors in S (meaning, as we ll see below, that (1; ; ) is linearly independent of the vectors in S), or that (1; ; ) is not in the span of S, or in symbols, that (1; ; ) = span(s) (see the de nition below) 1

2 De nition Let S = fv 1 ; v ; :::; v k g be a subset of the vector space V The set S is called a spanning set of V if every vector in V can be written as a linear combination of vectors in S In such cases it is said that "S spans V " Example 8 Examples of Spanning Sets 1 The set S = f(1; ; ) ; (; 1; ) ; (; ; 1)g = fe 1 ; e ; e g spans R because any vector u = (u 1 ; u ; u ) can be written as u = u 1 e 1 + u e + u e The set S = 1; x; x spans the set of all second-degree (or less) polynomials (denoted P ) because any polynomial p (x) = a + bx + cx can be written as p (x) = a (1) + b (x) + c x The set S = f(1; ; ) ; (; 1; ) ; ( ; ; 1)g also spans R because for any vector u = (u 1 ; u ; u ) in R, we have 8 < c 1 c = u 1 c 1 + c = u : c 1 + c + c = u whose coe cient matrix reduces to rref! and hence every system Ax = b has a unique solution x, regardless of b The set S = f(1; ; ) ; (; 1; ) ; ( 1; ; 1)g from our previous examples does not span R Although this set is hardly any di erent from set S, the vectors in S lead to a row-reduced augmented coe cient matrix of 1 1 a 1 b 1 c 5 rref! a 1 b c In this case, the presence of a free variable will guarantee in nitely many solutions if c turns out to equal upon row reduction, but the system will have no solutions for any c = Therefore, given the system Ax = b, not all vectors b will be expressible as a linear combination of the vectors in S, ie, the set S does not span all of R : 5 De nition 9 If S = fv 1 ; v ; :::; v k g is a set of vectors in a vector space V, then the span of S is the set of all linear combinations of the vectors in S, span (S) = fc 1 v 1 + c v + + c k v k : c i are all real numbersg : The span of S is denoted by span(s) or span(fv 1 ; v ; :::; v k g) by fv 1 ; v ; :::; v k g, or that S spans V If span(s) = V, it is said that V is spanned Theorem 1 If S = fv 1 ; v ; :::; v k g is a set of vectors in a vector space V, then span(s) is a subspace of V Moreover, span(s) is the smallest subspace of V that contains S, in the sense that every other subspace of V that contains S must contain span(s) Proof We would have to show that the set of all linear combinations of v 1 ; v ; :::; v k is closed under addition and scalar multiplication Consider any two vectors in S: u = c 1 v 1 + c v + + c k v k v = d 1 v 1 + d v + + d k v k

3 where the c i and d i are scalars for all i Then u + v = (c 1 v 1 + c v + + c k v k ) + (d 1 v 1 + d v + + d k v k ) = (c 1 + d 1 ) v 1 + (c + d ) v + (c k + d k ) v k and au = a (c 1 v 1 + c v + + c k v k ) = (ac 1 ) v 1 + (ac ) v + + (ac k ) v k, hence u+v and au are also in span(s) because they too can be written as linear combinations of v 1 ; v ; :::; v k Therefore, span(s) is a subspace of V De nition 11 A set of vectors S = fv 1 ; v ; :::; v k g in a vector space V is called linearly independent if the vector equation c 1 v 1 + c v + + c k v k = has only the trivial solution, c 1 = c = = c k = linearly dependent If there are any nontrivial solutions, then S is called Remark 1 It is important to note that the coe cients in a linear combination of linearly independent vectors v 1 ; v ; :::; v k are unique Proof If vector w can be written as both and then and w = c 1 v 1 + c v + + c k v k w = d 1 v 1 + d v + + d k v k ; c 1 v 1 + c v + + c k v k = d 1 v 1 + d v + + d k v k (c 1 d 1 ) v 1 + (c d ) v + + (c k d k ) v k = Then, because the vectors v 1 ; v ; :::; v k are linearly independent, it must be true that each of the coe cients is zero, so that c i = d i for all i = 1:::k Theorem 1 A set S = fv 1 ; v ; :::; v k g, k, is linearly dependent if and only if at least one of the vectors v j can be written as a linear combination of the other vectors in S Proof ) Assume that S is a linearly dependent set Then there exist scalars c 1 ; c ; :::; c k (not all zero) such that c 1 v 1 + c v + + c k v k = Because one of the coe cients must be nonzero, we can assume without loss of generality that c 1 = Then we can solve for v 1 in terms of the other vectors, producing v 1 = c c 1 v c c 1 v hence v 1 is a linear combination of the other vectors c k c 1 v k, ( Suppose the vector v 1 in S is a linear combination of the other vectors In other words, v 1 = c v + c v + + c k v k Then the equation v 1 + c v + c v + + c k v k = has at least one coe cient, 1, that is nonzero, and you can conclude that S is linearly dependent

4 Linear Independence in R n Theorem 1 The n vectors v 1 ; v ; :::; v n in R n are linearly independent if and only if the n n matrix A = v 1 v v n having the v i as column vectors has nonzero determinant Remark 15 Any set of more than n vectors in R n is always linearly dependent Proof If k > n, then the system c 1 v 1 + c v + + c k v k = is a homogeneous system that has more variables than equations and therefore has in nitely many nontrivial solutions - Bases and Dimension for Vector Spaces De nitions, Theorems, and Examples De nition 1 A nite set S of vectors in a vector space V is called a basis for V provided that (a) the vectors in S are linearly independent, and (b) the vectors in S span V In other words, a basis for a vector space V is a linearly independent spanning set of vectors in V De nition 1 The standard basis for R n consists of the unit vectors e 1 = (1; ; ; :::; ) ; e = (; 1; ; :::; ) ; :::, e n = (; ; ; :::; 1) : Claim 18 Any set of more than n vectors in R n is linearly dependent Claim 19 Any set of n linearly independent vectors in R n is a basis for R n Claim A basis for any vector space V contains the largest possible number of linearly independent vectors in V De nition 1 Any two bases for a vector space consist of the same number of vectors, and this number is called the dimension of the vector space We then call the vector space an "n-dimensional vector space" Example The zero vector space fg has no basis because it contains no linearly independent set of vectors Example The space of all polynomials } has no nite basis (ie, we cannot add two nd-degree polynomials to obtain all other degrees of polynomials) We therefore say that } is in nite dimensional Claim An n-dimensional vector space contains proper subspaces of each dimension k = 1; ; :::; n 1 (Recall that a proper subspace is one that is not "trivial," ie, is not just the fg space or not the entire space itself) Theorem 5 Let V be an n-dimensional vector space and let S be a subspace of V (a) If S is linearly independent and consists of n vectors, then S is a basis for V (b) If S spans V and consists of n vectors, then S is a basis for V (c) If S is linearly independent, then S is contained in a basis for V (d) If S spans V, then S contains a basis for V Then Bases for Solution Spaces Consider a homogeneous linear system Ax =, in which A is an m n matrix, so the system consists of m equations in the n variables x 1 ; x ; :::; x n Its "solution space" W, as discussed previously, is a subspace of R n We typically want to describe this solution space in terms of an explicit basis for W In the case of a homogeneous system, if the row-echelon form of the matrix has no free variables, the system has only the trivial solution, and W = fg If there are free variables, we parameterize to obtain a basis for the solution vectors as shown in the following example:

5 Example Find a basis for the solution space of the homogeneous linear system 8 < x 1 + x x 5x + 5x 5 = x 1 + x x x + x 5 = : : x 1 + x x x + x 5 = The reduced row-echelon form for this system is given by : Thus the leading variables are x 1 and x while the free variables are x ; x ; and x 5 Setting we nd x = r, x = s, and x 5 = t, (1) x 1 = r + s t and x = s t () Now the equations in (1) and () give us a typical solution vector of the form 1 1 x 1 r + s t x B x x A = r B s t s A ; x 5 t all in terms of the parameters r, s, and t So far, there is nothing new in this approach However, we now want to generate linearly independent solution vectors that span the solution space We do so by in turn selecting one of the parameters to be 1 while all the others are So for r = 1 and s = t =, we have the vector v 1 = ( ; 1; ; ; ), for s = 1 and r = t =, we have and for r = s = and t = 1, we have v = (; ; 1; 1; ), v = ( ; ; ; ; 1) Therefore the solution space of the system is a -dimensional subspace of R 5 with a basis given by fv 1 ; v ; v g Claim When homogeneous systems are solved from the reduced row-echelon form, the resulting spanning set is always linearly independent Claim 8 The dimension of the solution space for the homogeneous linear system is the same as the number of free variables in the reduced row-echelon form 5 - Row and Column Spaces Row Space and Row Rank In the course of solving systems of linear equations, we have often produced entire rows of zeros in our reduction of matrices to echelon form What this really means is that some of the equations in the original system did not give us any additional information about the system In other words, those equations were redundant or unnecessary It turns out that it is a relatively easy determination as to how many equations within a system are not redundant First, we address some de nitions 5

6 De nition 9 Given an m n matrix A, the row vectors of A are the m vectors r 1 = (a 11 ; a 1 ; :::; a 1n ) r = (a 1 ; a ; :::; a n ) r m = (a m1 ; a m ; :::; a mn ) in R n The subspace of R n spanned by these m row vectors is called the row space, denoted Row(A), of the matrix A The dimension of the row space Row(A) is called the row rank of the matrix A Note that if the rst column of an echelon matrix E is not all zero, then its nonzero row vectors have the forms r 1 = (e 11 ; :::; e 1p; :::; e 1q ; :::); r = (; ::; e p ; :::; e q ; :::); r = (; :::; ; ; ; ; : e q ; :::); and so on implies that Then the equation c 1 r 1 + c r + + c k r k = c 1 e 11 = ; c 1 e 1p + c e p = ; c 1 e 1q + c e q + c e q = ; and so forth We can therefore conclude that c 1 = c = ::: = c k =, and thus the row vectors r 1 through r k are linearly independent This result leads to the following theorem Theorem The nonzero row vectors of an echelon matrix are linearly independent and therefore form a basis for its row space Furthermore, if two matrices are row equivalent, they have the same row space Algorithm 1 To nd a basis for the row space of a matrix A, use elementary row operations to reduce A to an echelon matrix E Then the nonzero row vectors of E form a basis for the row space of A, Row(A) Remark The nonzero row vectors of E are not necessarily the same rows that would have been linearly independent in the original matrix A Column Space and Column Rank De nition Given an m n matrix A, the column vectors of A are the n vectors a 11 a 1 c 1 = a m1 5 ; c = a 1 a a m 5 ; :::; c n = in R m The subspace of R m spanned by these n column vectors is called the column space, denoted Col(A), of the matrix A The dimension of the column space Col(A) is called the column rank of the matrix A a 1n a n a mn 5 Realize that an echelon matrix E has the following general form, d 1 d d E = d k ; 5

7 where the d 1 to d k are all nonzero leading entries Then the k pivot columns of E look like d 1 d ; 5 ; 5 d ; :::; 5 The same argument we used to determine the linear independence of the row vectors earlier can be used here We would nd that a 1 c 1 + a c + + a k c k = implies that a 1 = a = = a k = : Therefore, the column rank of the echelon matrix E is equal to the number of its nonzero rows, and hence is equal to its row rank Now, what about the column rank of the original matrix A? Note that the homogeneous linear systems have the same solution set because the two matrices are row equivalent left-hand sides in Eq () are linear combinations of the column vectors: d k : 5 Ax = and Ex = () Ax = x 1 c 1 + x c + + x n c n ; Ex = x 1 c 1 + x c + + x n c n: Because x satis es either both or neither of the above systems, it follows that if and only if x 1 c 1 + x c + + x n c n = x 1 c 1 + x c + + x n c n = : If x = (x 1 ; x ; :::; x n ), then the Therefore, every linear dependence between column vectors of E is mirrored in a linear dependence with the same coe cients between the corresponding column vectors of A Because the k pivot columns of E are linearly independent, we can conclude that the corresponding k columns of A are also linearly independent In addition, because the pivot columns of E span Col(E), the corresponding columns in A span Col(A) Therefore, the latter k vectors form a basis for the column space of A, and the column rank of A is k This leads to the following algorithm Algorithm To nd a basis for the column space of a matrix A, use elementary row operations to reduce A to an echelon matrix E Then the column vectors of A that correspond to the pivot columns of E form a basis for the column space of A, Col(A) Theorem 5 The row rank and column rank of any matrix are equal the rank of the matrix This common value is simply called De nition The solution space of the homogeneous system Ax = is often called the null space of A and is denoted Null(A) The dimension of the null space is sometimes referred to as the nullity of the space, or as dim Null(A) Note that if A is an m n matrix, then Row(A) and Null(A) are subspaces of R n, while Col(A) is a subspace of R m In addition, we have the important identity rank(a) + dim Null(A) = n: Why? The rank of A is the number of leading variables in the reduced echelon matrix E, while the dimension of the null space is the number of free variables in E Therefore, their sum is the total number of variables in the matrix, or in other words, their sum is n It is an (almost) immediate consequence of the de nition of Col(A) that the nonhomogeneous linear system Ax = b is consistent if and only if the vector b is in the column space of A

8 Problems 1-1 The directions state that we are given a set of vectors in R Then " nd a subset of S that forms a basis for the subspace of R spanned by S" This amounts to determining which vectors in the given list are linearly independent We can do this by setting up a matrix with the given vectors as its columns and reducing the matrix to echelon form Doing so will identify the column space of the matrix, which in turn tells us which of the original vectors were linearly independent Problems 1- Here we are given a basis for some subspace W of R n We are then supposed to "extend" the basis so that it spans R n We do so by assuming that whatever basis vectors are lacking in S and would be needed to span R n would be multiples of the "standard unit vectors" e 1 ; e ; :::; e n Realizing this, we set these problems up in exactly the same way as problems 1-1, with the v 1 ; :::; v k as column vectors, along with the e 1 ; :::; e n augmented as columns as well The process of reducing the resulting matrix to reduced row-echelon form will identify which of the original columns were linearly independent, and hence will identify the basis vectors for R n Problems 1- Here we are trying to identify which equations within a system are actually "redundant" Recall that nding a basis for the row space tells us how many independent rows there are in a system, but not which rows those independent rows were However, the basis for the column space corresponds exactly with the linearly independent columns of the original matrix Therefore, if we take the transpose of the original system and nd a basis for the column space, we ve basically identi ed which equations (after re-transposition) in the original system were "irredundant" - Orthogonal Vectors in R n De nitions and Theorems De nition The length, or magnitude, of a vector v = (v 1 ; v ; :::; v n ) in R n is given by q kvk = v1 + v + + v n: This length is also called the norm If kvk = 1, then v is called a unit vector Note that by de nition, the length cannot be negative, ie kvk Also, kvk = if and only if v is the zero vector Theorem 8 The length of a scalar multiple of a vector is given as follows c a scalar Then kcvk = jcj kvk, where jcj is the absolute value of c Theorem 9 We often want to nd a unit vector in the same direction as a given vector vector in R n, then the vector u = v kvk Let v be a vector in R n and If v is a nonzero has length 1 and has the same direction as v This vector is called the unit vector in the direction of v 1 Proof Because v is nonzero, we know that kvk = Therefore kvk >, and we can write 1 u = v, kvk so u is a scalar multiple of v and hence in the same direction as v Additionally, u has length 1 because kuk = 1 kvk v = 1 kvk kvk = 1 The process of nding the unit vector in the direction of v is called normalizing the vector v 8

9 De nition The distance between two vectors u and v in R n is given by q ku vk = (u 1 v 1 ) + (u v ) + + (u n v n ) Note the following properties of this distance: 1) ku vk ) ku vk = if and only if u = v ) ku vk = kv uk Dot Product and the Angle Between Two Vectors To nd the angle between two vectors u and v, use the law of cosines (shown below in R ): kv uk = kuk + kvk kuk kvk cos (v 1 u 1 ) + (v u ) = u 1 + u + v 1 + v kuk kvk cos v1 u 1 v 1 + u 1 + v u v + u = u 1 + u + v1 + v kuk kvk cos u 1 v 1 + u v = cos () kuk kvk The numerator on the left-hand side represents the R version of the dot product, which is de ned in R n as u v = u 1 v 1 + u v + + u n v n This dot product, also called a scalar product, produces a scalar, not a vector! following properties: 1 u v = v u (symmetric) u (v + w) = u v + u w (distributivity) (cu) v = u (cv) = c (u v) (homogeneity) v v, and v v = if and only if v = (positivity) This product exhibits the Given the above derivation of cos in Eq (), we would like to de ne the angle between two vectors as cos = uv uv kukkvk, but how do we know that 1 kukkvk 1? We know this because of the Cauchy-Schwarz Inequality, often considered one of the most important inequalities in all of mathematics: Theorem 1 (Cauchy-Schwarz Inequality): If u and v are vectors in R n, then ju vj kuk kvk Proof If u =, then ju vj = j vj = and kuk kvk = kvk = If u =, we consider tu + v for any t R We then know (by property above) that (tu + v) (tu + v) t (u u) + t (u v) + (v v) Now, let a = (u u), b = (u v), and c = (v v) Then we have at + bt + c, which represents a parabola that is on or above the x-axis Therefore, we know that it has two non-real roots or one repeated real root, and that its discriminant is non-positive We thus have b ac b ac 9

10 and substituting the values for a, b, and c, this leads to (u v) (u u) (v v) or, dividing by, (u v) (u u) (v v) Recall that (u u) = kuk and (v v) = kvk Then we have (u v) kuk kvk, and because both sides are positive, we end up with ju vj kuk kvk (5) Now, dividing both sides of Eq (5) by the right-hand side (which is non-zero for non-zero vectors u and v) leads to ju vj kuk kvk 1; which in turn is equivalent to We therefore de ne the angle between two vectors as cos = 1 u v kuk kvk 1 u v, for kuk kvk Furthermore, because kuk kvk >, we know that the sign of the cos is determined by the dot product u v Therefore, we have the following result regarding the relationship between u and v: Opposite direction u v < u v = u v > Same direction cos = 1 cos < cos = cos > cos = 1 = < < (Q ) = < < (Q 1) = Of particular interest is the case in which u v = The angle between the two vectors is then, and the vectors are considered perpendicular to one another (or orthogonal) This leads to the following de nition De nition Two vectors u and v in R n are orthogonal if their dot product is zero (ie, if u v = ) By this de nition, the zero vector is orthogonal to every other vector (even though cos = uv kukkvk would be unde ned) Another very important theorem in mathematics is the Triangle Inequality In two dimensions, this says that the longest side of a "triangle" is shorter than the sum of the two shorter sides (what happens if it s not?) This theorem is widely used in proofs of important mathematical results, and it generalizes to higher dimensions Here is its formal statement and its proof Theorem (Triangle Inequality): If u and v are vectors in R n, then Proof We have ku + vk kuk + kvk ku + vk = (u + v) (u + v) = u (u + v) + v (u + v) = u u + u v + v u + v v = kuk + (u v) + kvk () kuk + ju vj + kvk : () 1

11 Now, by the Cauchy-Schwarz Inequality, we know that ju vj kuk kvk, so from Eq () we have ku + vk kuk + ju vj + kvk kuk + kuk kvk + kvk = (kuk + kvk) ; and because both sides are, we nally have ku + vk kuk + kvk Note that equality occurs precisely when u = cv In the midst of the proof for the Triangle Inequality (Eq ())we notice the following statement: ku + vk = kuk + (u v) + kvk If the vectors u and v are orthogonal, then u v =, and we end up with the following generalization of the Pythagorean Theorem to R n Theorem If u and v are vectors in R n, then u and v are orthogonal if and only if ku + vk = kuk + kvk Dot Products and Matrix Multiplication We quite often represent a vector v = (v 1 ; v ; :::; v n ) as an n1 matrix With this in mind, we can represent the dot product of two vectors u u C B C as the matrix product u = u n u v = u T v = (u 1 ; u ; :::; u n ) C A and v = v 1 v v n 1 v 1 v v n C A C A = u 1v 1 + u v + + u n v n Orthogonality, Linear Independence, and Matrix Subspaces As one might expect, there is a connection between orthogonality and linear independence Theorem 5 If the nonzero vectors v 1 ; v ; :::; v n are mutually orthogonal - that is, each two of them are orthogonal - then they are linearly independent (see proof in textbook on pages -8) An important implication of this theorem is that any set of n mutually orthogonal vectors in R n constitutes a basis for R n This particular basis is called an orthogonal basis One simple example is the set of standard unit vectors e 1 ; e ; :::; e n in R n Of course, we would now have to ask exactly how we can nd an orthogonal basis for a given space Furthermore, we often would like to have basis vectors with lengths of 1 This leads to what is called an orthonormal basis, and of course, we would like to know how to nd a basis of this type as well More on this later 11

12 Orthogonal Complements Consider the homogeneous linear system Ax = of m equations in n unknowns If v 1 ; v ; :::; v n are the row vectors of the m n coe cient matrix A, then the system can be represented as v 1 v 1 x v v x v n 5 x 5 = v n x 5 = 5 : (8) From this, it should be clear that x is a solution vector of Ax = if and only if x is orthogonal to each row vector of A But then x is orthogonal to every linear combination of row vectors because x (c 1 v 1 + c v + + c k v k ) = c 1 x 1 v 1 + c x v + + c k x k v k = (c 1 ) () + (c ) () + + (c k ) () = : Therefore, the vector x in R n is a solution vector of Ax = if and only if x is orthogonal to each vector in the row space of the matrix A This leads us to the following de nition: De nition (Orthogonal Complement of a Subspace): Given a subspace V of R n, the space of all vectors orthogonal to V is called the orthogonal complement of V It is denoted by V? and read "V perp" It is easy to show that V? is also a subspace of R n This statement is part of the following theorem Theorem Let V be a subspace of R n Then 1 Its orthogonal complement V? is also a subspace of R n ; The only vector that lies in both V and V? is the zero vector; The orthogonal complement of V? is V, or in other words, V?? = V ; If S is a spanning set for V, then the vector u is in V? if and only if u is orthogonal to every vector in S The importance of our discussion of the system in Eq (8) cannot be overstated to the following theorem That discussion leads Theorem 8 Let A be an m n matrix complements in R n Then the row space of A and the null space of A are orthogonal We then have at least one way to nd a basis for the orthogonal complement of a given space example, if we seek a basis for the orthogonal complement V? of a subspace V of R n : For Algorithm 9 Let A be the m n matrix with row vectors v 1 ; v ; :::; v m Then, reduce A to echelon form and nd a basis for the solution space Null(A) of Ax = Because V? = Null(A), this latter basis will be a basis for the orthogonal complement of V - General Vector Spaces Examples of General Vector Spaces The concept of a "vector space" can (and is) easily extended to spaces for which the elements are not necessarily "vectors" in the way we typically view or de ne vectors Here are some examples Example 5 (Function Space): Let F denote the set of all functions f : R! R This notation means that f is de ned for all real numbers on the real line Therefore, functions such as f 1 (x) = x x + and f (x) = sin x are in F, but f (t) = 1=t and f (t) = tan t are not in F because they are not de ned for all real t Addition and scalar multiplication of functions are de ned as (f + g) (x) = f (x) + g (x) and (cf) (x) = cf (x), respectively, and the zero element is the constant function f (x) = for all x R (One can also consider functions de ned on some I R) 1

13 Example 51 (Polynomial Space): Let } be the set of all polynomials in F (ie, polynomials are de ned for all real values of their independent variable) Sums of polynomials and scalar multiples of polynomials are still polynomials, so } is a subspace of F, and hence a vector space in its own right However, a basis for } would have to be in nite-dimensional (why?) Example 5 (Space of polynomials of degree at most n): Let } n be the set of all polynomials of degree at most n (for n ) Again, this set would be closed under addition and scalar multiplication and therefore would be a subspace of both } and F Whereas } is in nite-dimensional, } n is (n + 1)-dimensional (why n + 1 and not just n?), and has basis 1; x; x ; :::; x n NOTE: The fact that the basis "vectors" of this polynomial space are linearly independent leads to the result that the coe cients in a polynomial are unique, which in turn is what allows us to use the method of "partial fractions" that we learned in calculus class Example 5 (Space of Continuously Di erentiable Functions): Let C 1 be the set of all continuously differentiable functions u : R! R A function u is in this space if u and u exist and are continuous for all t R Some functions that would be in C 1 would include all polynomials, functions of the form u (t) = e t for all R, and the trigonometric functions such as v (t) = sin (t) and w (t) = cos (t) for all R Functions not in this space would be those such as f (t) = 1 t 5, g (t) = jtj, or h (t) = csc (t) This space is a subspace (hence a vector space) because and d dx (x (t) + y (t)) = dt dt + dy dx d dx (rx (t)) = r dt dt Example 5 (Space of m n Matrices): It is easy to verify that for given m and n, the set of m n matrices possesses all the properties of a vector space We label this space as M m;n Example 55 A speci c example of the above space would be the set of all matrices, M ; This is a subspace (it is closed under scalar multiplication and matrix addition) of M m;n Furthermore, it is a -dimensional space with basis vectors 1 1 ; ; ; and 1 1 a b Example 5 A subset of M ; might consist of all matrices of the form, which is easily b a veri able as a subspace of M ; This is a -dimensional subspace because its basis is given by 1 1 A = and B = 1 1 Note that B = = and for any linear combinations aa + bb and ca + db given by a b c d and b a d c 1 1, = A, we also have (aa + bb) (ca + db) = = a b c d b a d c ac bd (bc + ad) bc + ad ac bd 1

14 This last result shows that this particular space is also closed under matrix multiplication (in addition to scalar multiplication We know by now that this is not necessarily that case Furthermore, it should not take you long to recognize the above as a matrix representation of complex numbers! The matrix a b = a Re +b Im b a corresponds to the complex number a+bi, and the matrix multiplication above is none other than the familiar (a + bi) (c + di) = (ac bd) + (ad + bc) i The primary convenience of such a representation is that matrices are much easier to program on a computer than are the rules of complex number arithmetic Solution Spaces of Di erential Equations The most applicable "vector space" in our class (apart from R n ) is the space of solutions of di erential equations The concepts regarding vector spaces, such as linear independence, rank, dimension, bases, and so on, come into play in our solution of di erential equations, particularly when we get into systems of di erential equations Here are some more examples of vector spaces in this context Example 5 The rst-order di erential equation y = ky, where k is a constant, has the general solution y = Ce kx Therefore the solution space of the di erential equation consists of all constant multiples of the single function e kx, and is therefore the 1-dimensional function space with basis e kx We will soon study this in more detail, but for now, it is probably not di cult to trust that the set of all solutions of a linear second-order di erential equation of the form ay + by + cy = (9) with constant coe cients a, b, and c is a -dimensional function subspace S The following serve as examples of this concept, and the details behind the derivation of the solutions can be found on pages -8 of the textbook Do not be concerned (for now) with how to solve these di erential equations Just be sure to understand how the solutions can be generated from basis "vectors" in a subspace of functions Example 58 Let a = 1 and b = c = in Eq (9) Then we have y = ; (1) which after two integrations (do this for review!) has the solution y (x) = Ax + B Every function of this form would be a solution to the di erential equation in (1), and every solution to the di erential equation would have this form Therefore the solution space of this di erential equation is the -dimensional subspace } 1 of linear polynomials with basis f1; xg Example 59 Let a = 1, b =, and c = in Eq (9) Then we have y y =, (11) which has solution (verify this!) y (x) = 1 Cex + A Therefore every solution of (11) is of the form y (x) = A + Be x (where B = 1 C), and every function of this form (verify this via substitution) is a solution of the di erential equation The solution space therefore is -dimensional with basis 1; e x 1

15 Example Let a = 1, b =, and c = 1 in Eq (9) Then we have which after quite a bit of algebraic manipulation leads to the solution y y =, (1) y (x) = A cosh x + B sinh x, (1) where A = a sinh b and B = a cosh b Therefore every solution of the di erential equation in (1) is of the form shown in (1), and every function of this form is a solution of the di erential equation Therefore, the solution space of the di erential equation is the -dimensional function space generated by the basis fcosh x, sinh xg Furthermore, because cosh x and sinh x are de ned as linear combinations of e x and e x, ie, cosh x = ex + e x and sinh x = ex e x, we also see that the solution can be written in the form y (x) = Ce x + De x, and so a second basis for the solution space would be the set fe x ; e x g Homework Hints If you are trying to determine if a set is a subspace (or vector space), often an easy check is to determine whether the space contains the zero element (ie, the zero matrix, the zero function, etc) If it does not, it cannot be a vector space, and hence cannot be a subspace Other than checking for the zero element, make sure the set is closed under addition and scalar multiplication In other words, make sure that the sums and scalar multiples have the same characteristics as those "vectors" with which you started To determine linear independence, make sure that one "vector" cannot be written as a multiple (or linear combination) of the others This can often done by inspection (ie, with little or no work) 15