Vectors, Linear Combinations, and Matrix-Vector Mulitiplication In this section, we introduce vectors, linear combinations, and matrix-vector multiplication The rest of the class will involve vectors, and they are objects that appear consistently in more advanced mathematics, physics, differential equations, engineering, and more Taking linear combinations of lists of vectors is a natural and important thing we do and will lead to matrix-vector multiplication Matrix-vector multiplication will be instrumental in this class as it gives rise to linear maps Definition A (column) vector is a list of real numbers arranged in a column We write vectors with arrows over them, as in x x =, where x,,, are real numbers called the components of x An n-vector is a vectors with n components We think of vectors geometrically as arrows starting at the origin, where the jth-component records the amount the arrow goes along the j-th coordinate axis, as the following example illustrates - x To describe -dimensional space, we take three directions for our coordinate axes, say forward, right, and up Then, three dimensional space is all of the points, or vectors, that can be reached by going some amount forward, some amount right, then some amount up, including negative amounts So, in terms of vectors, -dimensional space is the set of all -vectors It works similarly for other dimensions, though it is harder to visualize geometrically Definition The set of all n-vectors is n-dimensional Euclidean space, denoted R n Definition A set is a collection of objects, where multiplicity is not allowed, and order does not matter We use { } to denote sets For example, {,, } is a set, {,, } is NOT a set, and {,, } = {,, } Definition 4 A list is a list of objects, where multiplicity is allowed, and order matters For example, as lists (,, ), (,, ) are lists, and (,, ) (,, ) Since we will be using lists and sets a lot in this class, we introduce some essential concepts and notation Such concepts and notation are ubiquitous in more advanced mathematics and science because sets are a convenient way to store information In addition, we introduce new notation for
describing sets, called set notation, which will make it much easier to describe and specify certain sets in the future Definition We use x A, read as x is in A, to mean x is an element of A, and we say x / A, read as x is not in A, otherwise We use A B, read as A is contained in B or A is a subset of B if every element of A is also an element of B, and we say A B otherwise For example, {,, 4}, / {,, 4}, {,, 4} {,,, 4, 7}, {,, 4} {,, } But what if we wanted to specify more interesting set, like to set S of all positive integers Then, we could write S = {,,, 4, } and assume the reader will intuit what means here Or, we can write S = {x x is a positive integer} Here, the left side of the bar indicates the objects in the set, and the right side of bar indicates the properties these objects need to have in order to be in the set So, S is the set of all positive integers Now, suppose we wanted to specify the set S of all -vectors whose components sum to 0 If we just list elements of S, S = 0 0, 0, 0,, π,, 0 0 π it is not clear what vectors are in S and which are not Instead, we write S = x () x + + x = 0 This means S is the set of all x satisfying x + + x = 0 So, for example, x x S, since + 4 = 0, 4 / S, since + + = 6 0 But R n, the set of all n-vectors, is not just a static object In this class, we will be concerned with the operations of addition and scalar multiplication on R n We will define them exactly as one would expect These operations have many of the same properties that ordinary addition and multiplication do One vector that will show up over and over again is the vector of all zeros, which we call the zero vector Definition 6 Let 0 denote the zero vector, whose components are all zeros: 0 0 =, 0
where the number of components is in context For example, if 0 R, then 0 = numbers x,,, y,, y n, c, define x y x + y + y = + y, y n + y n x cx c = c, c 0 0 For any real In terms of arrows, we add vectors tip to tail, and we scale vectors by scaling arrows as follows Note that scaling by reverses the direction of the arrow x + y y c v x v v Proposition 7 For any x, y, z R n and scalars c, d, (a) x + y = y + x (b) ( x + y) + z = x + ( y + z) (c) (c(d x)) = (cd) x (d) (c + d) x = c x + d x (e) c( x + y) = c x + c y (f) x + 0 = x (g) x x = 0 (h) 0 x = 0 Proof Problem Suppose we have some vectors a,, a n in R m and we are allowed to use the operations of addition and scalar multiplication What are all the vectors we can get? These vectors are called the linear combinations of ( a,, a n ) Linear combinations of lists of vectors will appear constantly in this class Definition 8 Let ( a,, a n ) be a list of vectors in R m A linear combination of ( a,, a n ) is a vector of the form c a + c a + + c n a n for some scalars c, c,, c n For example, the following are linear combinations of ( a, a ): (a) a + a, (b) 7 a π a,
4 (c) a = a + 0 a, (d) a = 0 a + a, (e) 0 = 0 a + 0 a Geometrically, the linear combinations of ( a,, a n ) are all of the vectors that can be obtained from ( a,, a n ) using addition and scalar multiplication Below are all the linear combinations of ( a, a ) using coefficients in {,, 0,, }: a a Example 9 (a) Find 4 4 (b) Express as a linear combination of 7 (c) Express Solution: (a) (b) We solve (, ( w as a linear combination of, w 4 = This becomes x x 4 + = x 7 x = ) ) 4 9 + = 8 4 + = 7 x 4 = x + 7 7 = x =4 x + =7 Writing in augmented matrix form and performing EROs, we get 4 0 4/ = x 7 0 / = 4, = ( ) 4 Therefore, is the following linear combination of, : 7 4 7 = 4 +
(c) We solve w = x w + x = x w = x + w = w w as a linear system in x, Solving this linear system by EROs gives w 0 w + w w 0 w + w = x = w + w, = w + w Therefore, w is a linear combination of w 4 7 = and as follows: ( w + ( ) w + w + w ) One can also solve the linear system in Example 9(c) using a multi-augmented matrix as follows For multi-augmented matrices, each augmented column represents the coefficients of variables just like the non-augmented columns do To write the linear system in Example 9(c) as a multi-augmented matrix, we add two augmented columns to keep track of the coefficients of w, w on the right hand side, which gives 0 0 We can solve this linear system with EROs as usual Note that EROs work no differently for multi-augmented matrices Thus, 0 0 0 0 R 0 R 0 R 0 R +R 0 Interpreting these equations gives x = w + w, = w + w as in Example 9(c) Thus, in order to express a vector as a linear combination of a set of vectors, we had to solve a linear system Going in the other direction, any linear system can be written as expressing one vector as a linear combination of a list of vectors We will see this by example as it would add very little insight but much more notation to prove this fact in full generality The interested reader can try to prove this in general in Problelm 7 Example 0 Rewrite the linear system () x + + x + 4x 4 =, 6x + 7 + 8x + 9x 4 = 8, x 4 + x x 4 = 6 as expressing one vector as a linear combination of some list of vectors Solution: Turning both sides into vectors, x + + x + 4x 4 6x + 7 + 8x + 9x 4 = () 8 x 4 + x x 4 6 Next, we employ an algebraic technique that will be used over and over again in this class Split the vector on the left hand side of () into a linear combination of constant vectors with each variable
6 as a coefficient The entries of the vectors simply become the coefficients of each variable in the corresponding component So, () becomes x 6 + 7 + x 8 + x 4 4 9 = (4) 8 4 6 Next, we will define matrix-vector multiplication so that the left hand side of (4) becomes 4 x 6 7 8 4 = 8 6 Notice that one can obtain this matrix from () by simply removing the variables and putting the coefficients into a rectangular array One can also obtain the vector on the right hand side by turning the right side of the equals signs into a vector Given an m n matrix A, we view its n columns, which we often call a,, a n, as vectors in R m For example, A = = a 4 6 a a, where a = R 4, a = R, a = R 6 Definition For an m n matrix A = a a n, and x R n, define x A x = a a n x 4 = x a + + a n Since a,, a n R m, A x R m as well If the number of columns of A does not equal the number of components of x, then A x is not defined Example Perform the following matrix-vector multiplications, if possible (a), (b) (c) (d) 4 6,, 4 6 4 6 Solution: (a) = = 6
7 (b) = 4 6 4 + = 6 (c) 4 6 is not defined since the number of columns of the matrix is not the same as the number of components of the vector (d) 4 = + 4 + = 7 6 6 9 Alternatively, one can compute the i-th entry of A x by multiplying the i-th row of A by x Again, we will see this by example as it would add very little insight but much more notation to prove this fact in full generality The interested reader can try to prove this in general, Problem 8 For example, in Example (c), notice that the first component of 4 is 6 7 = + ( ) + =, You may have seen this notion before as the dot product Thus, matrix-vector multiplication reduces to the dot product when the matrix has one row Let us look at another set of examples so that we notice some remarkable properties Example Let Find A x, A y, A( x + y) and A( x) Solution: () A x = + = 4 7 () A y = + = 4 () A( x + y) = = (4) A( x) = A = 4 = 4, x =, y = 4 8 + = 4 8 + = 4 Notice that A( x + y) = A x + A y, A( x) = A x This is not a coincidence, as we prove in the following Theorem Theorem 4 Let A be a m n matrix, x, y R n, and c a scalar Then, (a) A( x + y) = A x + A y
8 (b) A(c x) = c(a x) Proof We begin by setting Then, we find x A = a a n, x = A( x + y) = x + y a a n + y n, y = y y n = (x + y ) a + + ( + y n ) a n = (x a + + a n ) + (y a + + y n a n ) = A x + A y A(c x) = cx a a n c = (cx ) a + + (c ) a n = c(x a + + a n ) = c(a x) These properties of distributing over addition and scalar multiplication are the properties of linearity, which we will use for the rest of this class Exercises: Express as a linear combination of (, if possible or demonstrate it is not possible 7) Can you do it in multiple ways? If so, how? ( ) 4 Express as a linear combination of,, if possible, or demonstrate it is not possible 6 8 Can you do it in multiple ways? If so, how? ( 4 Express as a linear combination of,, if possible, or demonstrate it is not possible 6) Can you do it in multiple ways? If so, how? 4 Express as a linear combination of 0,, 4, if possible, or demonstrate it is 0 0 not possible Can you do it in multiple ways? If so, how? Compute 4 4 4 using both methods of matrix multiplication Check that your answers agree 6 Write the following vectors as linear combinations of constant vectors (vectors without variables)
x + x (a) x 4 (b) x + 0 (c) x + x + x + x (d) x x + x 4x 4 + x x + 7x 4 ( w 7 When is a linear combination of? Your answer should be some relation(s) between w ) w and w 8 When is w w a linear combination of 4, 6? Your answer should be some relation(s) w 7 among w, w, w 9 When is w w a linear combination of,, 0? Your answer should be some relation(s) w 0 among w, w, w 0 When is w w a linear combination of 4? Your answer should be some relation(s) among w 7 w, w, w Problems: () Prove Proposition 7 () Show that A 0 = 0 for any matrix A () Show that 0, a,, a n are all linear combinations of ( a,, a n ) for any a,, a n R m 4 () Show that if x is a linear combination of ( a,, a n ), then x is a linear combination of any rearrangement of ( a,, a n ) 9 (+) Show that 0 0 x x 0 0 = 0 0 6 Suppose A is an m n matrix, v, w R n, and c is a scalar (a) () Show that there exists x R n so that A x = c(a v) (b) () Show that there exists y R n so that A y = A v + A w
0 7 () Show that every linear system in the variables with m equations variables x,, can be written as A x = b, where A is an m n matrix, x = x and b R m 8 (+) Let r, r, r m be row vectors of size n, and let x R n Show that r r x r x = r x r m x r m 9 Suppose a,, a n R m, and c is a scalar For each of the following claims, prove it or give a counterexample (a) () Every linear combination of ( a + a, a, a, a n ) is a linear combination of ( a,, a n ) (b) () Every linear combination of ( a,, a n ) is a linear combination of ( a + a, a, a n ) (c) () Every linear combination of ( a,, a n ) is a linear combination of ( a + a, a, a, a n ) (d) () Every linear combination of (c a, a, a n ) is a linear combination of ( a,, a n ) (e) () Every linear combination of ( a, a, a n ) is a linear combination of (c a, a,, a n ) 0 Suppose x, y R n (a) () Show that every vector u R n whose endpoint lies on the line through the endpoints of x and y is of the form u = t x + ( t) y for some scalar t (b) () Show that every vector u R n whose endpoint lies on the line segment between the endpoints of x and y is of the form u = t x + ( t) y for some scalar t, 0 t () Show that if A is an m n matrix, v,, v k R n, and c,, c k are scalars, then A(c v + + c k v k ) = c (A v ) + + c k (A v k ) () Suppose A, B are m n matrices, and x R n with x 0 Prove or give a counterexample: If A x = B x, then A = B () Suppose A is an m n matrix with at least one nonzero entry, and x, y R n Prove or give a counterexample: If A x = A y, then x = y x 0 0 4 () Show that every x = 0 Rn is a linear combination of,,, 0 0 0
x () Show that every x = 0 Rn is a linear combination of,,, 0 0 6 () Suppose A is an m n matrix, b R n and x, y R n satisfies A y = b and A x = b Show that A( x y) = 0 A 7 (+) If A is an m n matrix and B is a k n matrix, let denote the (m + k) n matrix B obtained by stacking A on top of B Show that for all m n matrices A, k n matrices B, and x R n, A A x x = B B x 8 (+) If A is an m n matrix and B is a m k matrix, let A B denote the m (n + k) matrix obtained by placing B to the right of A Show that for all m n matrices A, m k matrices B, x R m, and y R k, x A B = A x + B y y 9 () Suppose A is an m n matrix, v,, v k R n, and y is a linear combination of (A v,, A v k ) Show that there exists x R n so that A x = y 0 () Suppose v is a linear combination of ( a,, a n ) but is not a linear combination of ( a,, a n ) Show that a n is a linear combination of ( a,, a n, v)