Math 416, Spring 010 Matrix multiplication; subspaces February, 010 MATRIX MULTIPLICATION; SUBSPACES 1 Announcements Office hours on Wednesday are cancelled because Andy will be out of town If you email Andy on Tuesday by 7pm, he ll make sure he responds by Wednesday morning He might not have internet access after Wednesday morning, though, so make sure your emails get sent by Tuesday night Also, we have a quiz in class on Thursday Matrix Multiplication We have seen in previous classes that one can define certain kinds of matrix multiplication: the dot product of two vectors, the product of n m matrix on the left of a m 1 column vector Today we ll introduce the mother of these multiplications 1 Composing linear operators Suppose we have a r p matrix A corresponding to an operator T A and a p c matrix B corresponding to T B Notice that since T B : R c R p and T A : R p R r, the composition T A T B is a function from R c to R r Indeed, T A T B is a linear operator from R c to R r We can see that it satisfies the first property of a linear operator by the equations (T A T B ( v + w = T A (T B ( v + w = T A (T B ( v + T B ( w = T A (T B ( v + T A (T B ( w = (T A T B v + (T A T B w where the second equality holds because T B is a linear operator and the third equality holds because T A is a linear operator Similarly we have (T A T B (k v = T A (T B (k v = T A (kt B ( v = k(t A (T B ( v = k(t A T B v Now that we know T A T B is a linear operator, we ask: what is the matrix corresponding to T A T B? T A R r R p R c T B T A T B = T C acs@mathuiucedu http://wwwmathuiucedu/ acs/w10/math416 Page 1 of 6
Math 416, Spring 010 Matrix multiplication; subspaces February, 010 We know that the matrix should be given by applying T A T B to the standard basis vectors e i Hence we get that the matrix associated to T A T B is AB e 1 AB e AB e c If we define b i to be B e i (in other words, b i is the ith column of the matrix B, then this expression simply becomes A b 1 A b A b c Definition 1 If A is an r p matrix and B is a p c matrix, then there is a r c matrix AB defined as AB := Ab1 A b c, where b i is the ith column of the matrix B Notice that the matrix/vector products are well defined: A has p columns and each of the b i have p entries Since we can compute the product of a matrix with a vector using dot products, this means that Example Let Then we have whereas the entry in ith row, jth column of AB = the ith entry of A b j BA = AB = A = ( 1 3 4 5 6 = the dot product of ith row of A with b j and B = ( 1 1 + 3 + 3 5 1 + 4 + 3 6 4 1 + 5 3 + 6 5 4 + 5 4 + 6 6 1 1 + 4 1 + 5 1 3 + 6 3 1 + 4 4 3 + 4 5 3 3 + 4 6 5 1 + 6 4 5 + 6 5 5 3 + 6 6 1 3 4 5 6 = = ( 8 49 64 9 1 15 19 6 33 9 40 51 Notice that these two matrices aren t just unequal, they don t even have the same dimension! Let s work through a specific geometric example of linear operators and see how matrix multiplication can be interpretted through these geometric actions Example In your last homework assignment, you showed that the linear operator Rot : R R which rotates vectors through 45 in the counterclockwise direction is given by the matrix ( ( cos(45 sin(45 sin(45 cos(45 = Consider the linear operator Ref : R R which reflects across the line y = 0 (ie, the x-axis This operator has matrix form ( 1 0 0 1 acs@mathuiucedu http://wwwmathuiucedu/ acs/w10/math416 Page of 6
Math 416, Spring 010 Matrix multiplication; subspaces February, 010 Now if we draw the action of the operators Rot Ref and Ref Rot on the standard basis { e 1, e }, they will tell us the matrix forms for ( ( ( 1 0 0 1 and ( 1 0 0 1 We show the action of the operator Rot Ref in Figure 1, and the action of the operator Ref Rot in Figure The green vector tracks the images of e 1 and the red vector tracks the images of e before any action after reflection after rotation Figure 1 before any action after rotation after reflection Figure Notice that since the images of e 1 and e are different for the two operators, the given operators are not the same Without using the algebraic definition of matrix multiplication, one can use these pictures to write the matrix products which result from these two compositions Try it out to convince yourself that the different orders of composition really do produce different matrices Notice: The last few examples illustrates a property that matrix multiplication does not have: commutativity That is to say, if you select random matrices A and B so that both AB and BA are defined, then it is likely the case that AB BA Aside from this, though, matrix multiplication has all the properties you would like it to have acs@mathuiucedu http://wwwmathuiucedu/ acs/w10/math416 Page 3 of 6
Math 416, Spring 010 Matrix multiplication; subspaces February, 010 Theorem 1 Matrix multiplication is associative and distributive, which means A(BC = (ABC for matrices A, B and C which make the given products defined, and A(B + C = AB + AC and (B + CA = BA + CA for matrices A, B and C which make the given products defined Matrix multiplication shows that a matrix and its inverse have a particular product: Theorem Suppose that T : R n R n is an invertible linear operator, and let A be the corresponding n n matrix Then we have AA 1 = A 1 A = I n Proof You ll establish these equalities in your homework 3 Subspaces Last class period we defined the kernel of an r c matrix A: ker(a = { x R c : A x = 0 } Today we ll start by showing that ker(a has some nice properties: 0 is an element of ker(a; if x 1 and x are elements of ker(a, then x 1 + x is an element of ker(a; and if x is an element of ker(a, then for any scalar k we have k x is an element of ker(a Proof Actually, you ve already proven all of these statements in an earlier homework assignment But just for review, here s how the proofs work The first statement comes from the fact that A 0 = 0 For the second statement, notice that A( x 1 + x = A x 1 + A x = 0 + 0 = 0 So x 1 + x is in the kernel of A The last statement is proved similarly: A(k x = k(a x = k 0 = 0 There is another collection of vectors associated to a vector A which are very important: the set of all vectors which are in the image of A In set notation, the image of A is written im(a = { b R r : there exists x R c so that A x = b } Theorem 31 For any matrix A, the image of A is the span of the column vectors of A Proof To see this, choose a vector in the image of A, say b Since b is an element of the image of A, there must be a vector x = x 1 x c so that A x = b But recall that A x = A x 1 x c = x 1 v1 + + x c vc, acs@mathuiucedu http://wwwmathuiucedu/ acs/w10/math416 Page 4 of 6
Math 416, Spring 010 Matrix multiplication; subspaces February, 010 where here v i is the ith column of A Hence we have b written as a linear combination of the columns of A as desired, and so b is in the span of the columns of A These steps can be reversed to show that an element in the span of the column vectors is in the image of A The image of a matrix satisfies conditions analogous to those which we saw in the kernel Theorem 3 For an n m matrix A, 0 is an element of im(a; if b 1 and b are elements of im(a, then b 1 + b is an element of im(a; and if b is an element of im(a, then for any scalar k we have k b is an element of im(a Proof To see that 0 is in the image, we must find a vector x so that A x = 0 But of course A 0 = 0, and so we can choose x = 0 Now suppose b 1 and b are elements in the image of A To show that b 1 + b is in the image of A, we need to find a vector x so that A x = b 1 + b Since b 1 and b are in the image of A, this means there exist inputs x 1 and x so that A x 1 = b 1 and A x = b Hence we have A( x 1 + x = A x 1 + A x = b 1 + b, and so we know the sum b 1 + b is in the image of A You can imagine that a similar proof shows the last condition With this terminology, we now have a handful of ways for expressing when a matrix A represents an invertible linear operator Theorem 33 For an n n matrix, the following statements are equivalent; that is, if one of them is true, then all of them are true, and if one is false, then all are false: A is an invertible matrix rref(a = I n A has rank n For any b R n, the system A x = b has a unique solution ker(a = { 0 } im(a = R n Proof We ve already seen these statements, with the exception of the last statement We ll show that im(a = R n is equivalent to the statement that rank(a = n Suppose, then, that im(a = R n Suppose, for the sake of contradiction, that rank(a < n In the last homework assignment you showed that this implies the existence of some b R n so that A x = b has no solutions This would force b im(a, contradicting the assumption that im(a = R n Hence we must conclude that rank(a = n Conversely, suppose that rank(a = n Since A is an n n matrix, a previous theorem tells us that for any b R n, there is a vector x R n with the property that A x = b This is precisely what it means for b to be an element of im(a Since b was arbitrary, this means that im(a must contain all vectors in R n acs@mathuiucedu http://wwwmathuiucedu/ acs/w10/math416 Page 5 of 6
Math 416, Spring 010 Matrix multiplication; subspaces February, 010 The three conditions we have seen in ker(a and im(a endow these sets with special properties which are useful in linear algebra Essentially these properties give ker(a and im(a some stability, in that one can t leave either collection through additions and scalar multiplication We ll be interested in these kinds of stable collections of vectors for the rest of the course Definition 31 A collection of vectors W R n is called a subspace if 0 is an element of W; whenever w 1 and w are elements of W, then w 1 + w are elements of W whenever w is an element of W and k is a scalar, then k w is an element of W Example Suppose that { v 1,, v c } is a collection of vectors in R r Then the span of these vectors is a subspace You can prove this bare-handed, or you can recall that im v 1 v c = span { v 1,, v c } Since we ve already seen that the image of a matrix is a subspace, this means that the given span is also a subspace Non-example The following shaded regions are not subspaces, because they each fail (at least one condition necessary to be a subspace One non-subspace Another non-subspace Figure 3 acs@mathuiucedu http://wwwmathuiucedu/ acs/w10/math416 Page 6 of 6