Math 46, Spring 2 More on Algebraic and Geometric Properties January 2, 2 MORE ON ALGEBRAIC AND GEOMETRIC PROPERTIES Algebraic properties Algebraic properties of matrix/vector multiplication Last time we talked about how one goes about multiplying an r c matrix against a vector from R c I sent a note to the class describing some of the nice algebraic properties that matrix/vector multiplication enjoyed They were If A and B are r c matrices and x is a vector in R c, then (A + B) x = A x + B x The fancy way to say this is matrix/vector multiplication is distributive It s also true that if w is another vector in R c, then A( v + w) = A v + A w If k is a scalar, then A(k v ) = k(a v ) To pick up where we left off, we recalled that matrix multiplication has two equivalent definitions: one in terms of columns of the matrix, and the other in terms of rows of the matrix If w A = v v 2 v c w 2 = w r and x is of the form x = Then the product A an be computed either by dot products with the row vectors of A, w x A w 2 x x = w r x or as a combination of the column vectors of A, x A x = x v + v 2 + + v c, or In particular this latter form shows us that outputs of multiplication by A are all vectors which are in the span of the columns of A Definition The span of a collection of vectors { v,, v c } is the set of all vectors x which are a linear combination of v,, v c ; ie, so that there exists a,, a c so that x = a v + a c vc acs@mathuiucedu http://wwwmathuiucedu/ acs/w/math46 Page of 8
Math 46, Spring 2 More on Algebraic and Geometric Properties January 2, 2 Example Suppose that { v,, v c } is a collection of vectors in R n Then the zero vector in R n (ie, the vector with n coordinates, all of which are ) is a linear combination of { v,, v c }, since = = v + + v c It s also true that the vector This is true because 4 5 6 4 5 6 is in the span of 2,, 7 8 3 9 = 3 2 + 7 8 3 9 2 + 7 8 2 2 3 9 Notice in particular that a vector can occasionally be expressed in multiple ways in terms of the vectors of the collection, and that we re allowed to use as a coefficient when writing one vector as a linear combination of others Equipped with the new terminology of span, we proved the following Theorem For an r c matrix A, suppose that x is a vector in R c so that A x = b, with b some vector in R r Then b is in the span of the columns of A Proof To prove this result, we just need to follow the definitions around First we know that if A = v v 2 v c and x is of the form x = x then the product A an be computed as b = A x = x v + v 2 + + v c This means that b is a linear combination of the columns of A; this is just the definition of what it means for b to be in the span of the columns of A As an interesting side-note, it s worth noting that the the columns of A are themselves outputs of multiplication by A To see this, we ll first establish some notation: acs@mathuiucedu http://wwwmathuiucedu/ acs/w/math46 Page 2 of 8
Math 46, Spring 2 More on Algebraic and Geometric Properties January 2, 2 Definition 2 The vector e i living in R c, called the ith standard basis vector, is that vector whose only nonzero entry is a in the ith position e i := ith position The collection { e,, e c } R c is called the standard basis of R m Now notice that if we compute A e i, we get A x = v + v 2 + + v i + + v c = v i 2 Algebraic Properties of dot products Theorem 2 Let u, v, and w be vectors in R m, and suppose k is a scalar Then u ( v + w) = u v + u w and (k u ) v = k( u v ) = u (k v ) These properties allow us to introduce the following Definition 3 A unit vector is a vector with length The unit vector in the direction of u ( u ) is the vector u u The second definition implies that the stated vector is unit length To see this, notice that ( ) ( ) u = u u u u = u u 2 ( u u 2 u ) = u 2 = 2 Geometry of Vectors, Redux 2 Sums and scalars in graphical depictions In class last period we said we can add two vectors together Today I want to show what addition looks like, which is to say I want to give a geometric interpretation of addition We ll do the same for scaling Example Suppose we are given vectors u and v in Figure What is u + v? Solution To add two vectors geometrically, we simply translate one vector onto the other More precisely, we move the base of the second vector onto the tip of the first vector The sum of the two vectors is then the tip of the translated second vector Notice that since u + v = v + u, we should be able to translate either vector onto the other to perform addition Indeed, both translations produce the same sum, as we see in the second illustration above acs@mathuiucedu http://wwwmathuiucedu/ acs/w/math46 Page 3 of 8
Math 46, Spring 2 More on Algebraic and Geometric Properties January 2, 2 u v Figure Vectors u and v u u u + v u + v v v The sum u + v You can add in either order! Figure 2 Example What is 2 u or u, where u is as above? Solution We can compute 2 u in two ways The first would be to recognize that 2 u = u + u, after which we could apply the reasoning above The second is to notice that 2 u is just u stretched by a factor of 2 Hence we can just draw a vector twice the length of u, but in the same direction The vector u is drawn as a vector in the opposite direction as u but with the same magnitude One of the benefits of this depection scheme is that it gives us a way to parametrize lines Example Suppose that l is a line passing through the origin If v is a vector with tail at the origin and whose tip lands on l, then the line l has the parametric representation l = t v, where t is the parameter (meaning that t can take any real value you like) Likewise, if one has a line l which is parallel to l but which doesn t pass through the origin, than one can give a parametric description for l To do so, find a vector u whose tail is at the origin and whose tip lies on l Then we get l = t v + u 22 Dot Product, again It turns out that dot product can be defined in a different way than how we defined it last period Both are equivalent, but this second one while harder to use in practice has very nice geometric consequences acs@mathuiucedu http://wwwmathuiucedu/ acs/w/math46 Page 4 of 8
Math 46, Spring 2 More on Algebraic and Geometric Properties January 2, 2 2 u u u u The vector 2 u The vector u Figure 3 Theorem 2 The dot product of two vectors u, v R m is given by u v = u v cos(θ), where θ is the angle between u and v This new definition can be used to verify that the length of a vector is given by the square root of the dot product of the vector with itself Indeed this new definition shows us that v v = v v cos() = v 2, and taking square roots gives the result v v = v Dot products can also tell us how to prove that two vectors are perpendicular (or, in linear algebra speak, orthogonal) when their dot product is For if u and v are orthogonal then the angle between them is 9 Hence we have u v = u v cos(9 ) = We can use this property to prove the following Theorem 22 Suppose that u and v are vectors Then we can write u = u + u where u is a vector in the direction of v and u is a vector perpendicular to v In fact, we can construct u as u u v = v v, v and u is therefore u = u u Note: often times u is called the projection of u onto v acs@mathuiucedu http://wwwmathuiucedu/ acs/w/math46 Page 5 of 8
Math 46, Spring 2 More on Algebraic and Geometric Properties January 2, 2 Proof Here s a diagram depicting v and u v? u Notice that to find the vector u, we can play around with some trigonometry: the vector u is the vector that points in the direction of v, but has magnitude given by the length marked by a? in our diagram Now that mystery length is expressible in terms of trig functions on the angle θ; in fact, we can see that mystery quantity Hence we get that our mystery quantity is simply u cos(θ) = u cos(θ) = u v cos(θ) v = u v v To produce the vector in question, we just need to multiply this desired length by a unit vector in the direction of v, and we ve already seen that v v fits the bill for the latter Hence our vector u becomes simply u v v u v v = v v v v All we have left to do is show that u = u u is perpendicular to v, as claimed To prove this, we ll just compute u v = ( u u ) v = ( u v u v ) v = u u v v v v v v = v 3 Matrix multiplication as a function First, let s review what a function is Definition 3 A function f : D C is a rule that assigns to each input d living in D an output c living in C We write f(d) = c or d c The set of all inputs for f, which we ve called D above, is the domain of f The set in which outputs take values, written C above, is the codomain (or target) The range (or image) of D is the collection of all outputs Notice that the range of a function is not necessarily all of the codomain Indeed, it s always an interesting question to ask exactly what the range of a function is Now recall that an r c matrix A can be multiplied on the left of a vector x in R c to produce a vector y R r Hence A defines a function from R c to R r (after all, a function from R c to R r is nothing more than acs@mathuiucedu http://wwwmathuiucedu/ acs/w/math46 Page 6 of 8
Math 46, Spring 2 More on Algebraic and Geometric Properties January 2, 2 a rule for assigning an input from R c to an output in R r ) We also saw that matrix multiplication follows the following rules: () A( v + w) = A v + A w (2) A(k v ) = k(a v ) As it happens, matrix multiplications are the only kinds of functions which obey these two rules We have the following Theorem 3 Suppose that T : R c R r satisfies the two rules above for all vectors v, w in R c Then there is a matrix A so that T( x ) = A x for every vector x in R c In fact, the ith column of A is given by T( e i ) (Recall that e i is the vector in R c that has a in every coordinate except the ith one, where it has coordinate ) Proof Suppose that T is a function that obeys those rules, and let v i = T( e i ) For the matrix A = ( v vc ) we ll show that T( x ) = A x for any x in R c To do this, write x = x Then we have T( x ) = T x = T x + + + Now by rule we know that T x + + + = T x + T + + T Notice then that T x +T + +T = T x +T + +T, and then rule 2 gives T x +T + +T = x T +T + +T acs@mathuiucedu http://wwwmathuiucedu/ acs/w/math46 Page 7 of 8
Math 46, Spring 2 More on Algebraic and Geometric Properties January 2, 2 But this is imply x v + v2 + + vc = ( v ) vc x = A x, as desired This winds up being super useful, because there are times when you know that a function satisfies conditions and 2 without explicitly knowing the corresponding matrix This theorem tells us how we can construct the matrix that gives rise to the function, simply by computing what the function does to the standard basis vectors acs@mathuiucedu http://wwwmathuiucedu/ acs/w/math46 Page 8 of 8