228 hapter Three Maps etween Spaces IV2 Matrix Multiplication After representing addition and scalar multiplication of linear maps in the prior subsection, the natural next operation to consider is function composition 2 Lemma The composition of linear maps is linear Proof (Note: this argument has already appeared, as part of the proof of Theorem I22) Let h: V W and g: W U be linear The calculation g h c ~v + c 2 ~v 2 = g h(c ~v + c 2 ~v 2 ) = g c h(~v )+c 2 h(~v 2 ) shows that g = c g h(~v )) + c 2 g(h(~v 2 ) = c (g h)(~v )+c 2 (g h)(~v 2 ) h: V U preserves linear combinations, and so is linear QED As we did with the operation of matrix addition and scalar multiplication, we will see how the representation of the composite relates to the representations of the compositors by first considering an example 22 Example Let h: R 4 R 2 and g: R 2 R 3, fix bases R 4, R 2, D R 3, and let these be the representations H = Rep, (h) = 4 6 8 2 G = Rep 5 7 9 3,D (g) = @ A, To represent the composition g h: R 4 R 3 we start with a ~v, representh of ~v, andthenrepresentg of that The representation of h(~v) is the product of h s matrix and ~v s vector v Rep ( h(~v))= 4 6 8 2 v 2 = 4v + 6v 2 + 8v 3 + 2v 4 5 7 9 3 @ v 3 A 5v + 7v 2 + 9v 3 + 3v 4, The representation of g( h(~v)) is the product of g s matrix and h(~v) s vector 4v + 6v 2 + 8v 3 + 2v 4 Rep D ( g(h(~v)) )= @ A 5v + 7v 2 + 9v 3 + 3v 4,D (4v + 6v 2 + 8v 3 + 2v 4 )+ (5v + 7v 2 + 9v 3 + 3v 4 ) = @ (4v + 6v 2 + 8v 3 + 2v 4 )+ (5v + 7v 2 + 9v 3 + 3v 4 ) A (4v + 6v 2 + 8v 3 + 2v 4 )+ (5v + 7v 2 + 9v 3 + 3v 4 ) v 4,D D
Section IV Matrix Operations 229 Distributing and regrouping on the v s gives ( 4 + 5)v +( 6 + 7)v 2 +( 8 + 9)v 3 +( 2 + 3)v 4 = @( 4 + 5)v +( 6 + 7)v 2 +( 8 + 9)v 3 +( 2 + 3)v 4 A ( 4 + 5)v +( 6 + 7)v 2 +( 8 + 9)v 3 +( 2 + 3)v 4 which is this matrix-vector product 4 + 5 6 + 7 8 + 9 2 + 3 = @ 4 + 5 6 + 7 8 + 9 2 + 3A 4 + 5 6 + 7 8 + 9 2 + 3,D @ v v 2 v 3 v 4 A D D The matrix representing g H h has the rows of G combined with the columns of 23 Definition The matrix-multiplicative product of the m r matrix G and the r n matrix H is the m n matrix P, where p i,j = g i, h,j + g i,2 h 2,j + + g i,r h r,j so that the i, j-th entry of the product is the dot product of the i-th row of the first matrix with the j-th column of the second h,j h 2,j GH = @ g i, g i,2 g i,r A @ A = @ p i,j A h r,j 24 Example 2 @ 4 6A 3 2 + 5 2 3 + 7 2 6 = @ 4 + 6 5 4 3 + 6 7A = @ 34 54A 5 7 8 2 8 + 2 5 8 3 + 2 7 8 38 25 Example Some products are not defined, such as the product of a 2 3 matrix with a 2 2, because the number of columns in the first matrix must equal the number of rows in the second ut the product of two n n matrices is always defined Here are two 2 2 s 2 3 4-2 -2 = (-)+2 2 + 2 (-2) 3 (-)+4 2 3 + 4 (-2) = 3-4 5-8
23 hapter Three Maps etween Spaces 26 Example The matrices from Example 22 combine in this way @ A 4 6 8 2 5 7 9 3 4 + 5 6 + 7 8 + 9 2 + 3 = @ 4 + 5 6 + 7 8 + 9 2 + 3A 4 + 5 6 + 7 8 + 9 2 + 3 9 3 7 5 = @ 5 7 9 3A 4 6 8 2 27 Theorem A composition of linear maps is represented by the matrix product of the representatives Proof This argument generalizes Example 22 Leth: V W and g: W X be represented by H and G with respect to bases V, W, andd X, of sizes n, r, andm For any ~v 2 V the k-th component of Rep ( h(~v)) is h k, v + + h k,n v n and so the i-th component of Rep D ( g h (~v)) is this g i, (h, v + + h,n v n )+g i,2 (h 2, v + + h 2,n v n ) Distribute and regroup on the v s + + g i,r (h r, v + + h r,n v n ) =(g i, h, + g i,2 h 2, + + g i,r h r, ) v Finish by recognizing that the coefficient of each v j + +(g i, h,n + g i,2 h 2,n + + g i,r h r,n ) v n g i, h,j + g i,2 h 2,j + + g i,r h r,j matches the definition of the i, j entry of the product GH QED This arrow diagram pictures the relationship between maps and matrices ( wrt abbreviates with respect to ) W wrt V wrt h H g h GH G g X wrt D
Section IV Matrix Operations 23 Above the arrows, the maps show that the two ways of going from V to X, straight over via the composition or else in two steps by way of W, havethe same effect ~v g h 7 g(h(~v)) ~v h 7 h(~v) g 7 g(h(~v)) (this is just the definition of composition) elow the arrows, the matrices indicate that multiplying GH into the column vector Rep (~v) has the same effect as multiplying the column vector first by H and then multiplying the result by G Rep,D (g h) =GH Rep,D (g) Rep, (h) =GH As mentioned in Example 25, because the number of columns on the left does not equal the number of rows on the right, the product as here of a 2 3 matrix with a 2 2 matrix is not defined - 2 2 The definition requires that the sizes match because we want that the underlying function composition is possible dimension n space h dimension r space g dimension m space ( ) Thus, matrix product combines the m r matrix G with the r n matrix F to yield the m n result GF riefly:m r times r n equals m n 28 Remark The order of the dimensions can be confusing In m r times r n equals m n thenumberwrittenfirstism utm appears last in the map dimension description line ( ) above, and the other dimensions also appear in reverse The explanation is that while h is done first, followed by g, wewrite the composition as g h, withg on the left (arising from the notation g(h(~v))) That carries over to matrices, so that g h is represented by GH We can get insight into matrix-matrix product operation by studying how the entries combine For instance, an alternative way to understand why we require above that the sizes match is that the row of the left-hand matrix must have the same number of entries as the column of the right-hand matrix, or else some entry will be left without a matching entry from the other matrix Another aspect of the combinatorics of matrix multiplication, in the sum defining the i, j entry, is brought out here by the boxing the equal subscripts p i,j = g i, h,j + g i, 2 h 2,j + + g i, r h r,j The highlighted subscripts on the g s are column indices while those on the h s are for rows That is, the summation takes place over the columns of G but
232 hapter Three Maps etween Spaces over the rows of H the definition treats left differently than right So we may reasonably suspect that GH can be unequal to HG 29 Example Matrix multiplication is not commutative 2 3 4 5 6 = 7 8 9 22 43 5 5 6 7 8 2 = 3 4 23 34 3 46 2 Example ommutativity can fail more dramatically: 5 6 7 8 2 = 3 4 23 34 3 46 while isn t even defined 2 3 4 5 6 7 8 2 Remark The fact that matrix multiplication is not commutative can seem odd at first, perhaps because most mathematical operations in prior courses are commutative ut matrix multiplication represents function composition and function composition is not commutative: if f(x) = 2x and g(x) = x + then g f(x) =2x + while f g(x) =2(x + ) =2x + 2 Except for the lack of commutativity, matrix multiplication is algebraically well-behaved The next result gives some nice properties and more are in Exercise 25 and Exercise 26 22 Theorem If F, G, and H are matrices, and the matrix products are defined, then the product is associative (FG)H = F(GH) and distributes over matrix addition F(G + H) =FG + FH and (G + H)F = GF + HF Proof Associativity holds because matrix multiplication represents function composition, which is associative: the maps (f g) h and f (g h) are equal as both send ~v to f(g(h(~v))) Distributivity is similar For instance, the first one goes f (g + h)(~v) = f (g + h)(~v) = f g(~v) +h(~v) = f(g(~v)) + f(h(~v)) = f g(~v) +f h(~v) (the third equality uses the linearity of f) Right-distributivity goes the same way QED 23 Remark We could instead prove that result by slogging through indices For
Section IV Matrix Operations 233 example, for associativity the i, j entry of (FG)H is (f i, g, + f i,2 g 2, + + f i,r g r, )h,j +(f i, g,2 + f i,2 g 2,2 + + f i,r g r,2 )h 2,j +(f i, g,s + f i,2 g 2,s + + f i,r g r,s )h s,j where F, G, andh are m r, r s, ands n matrices Distribute f i, g, h,j + f i,2 g 2, h,j + + f i,r g r, h,j and regroup around the f s + f i, g,2 h 2,j + f i,2 g 2,2 h 2,j + + f i,r g r,2 h 2,j + f i, g,s h s,j + f i,2 g 2,s h s,j + + f i,r g r,s h s,j f i, (g, h,j + g,2 h 2,j + + g,s h s,j ) + f i,2 (g 2, h,j + g 2,2 h 2,j + + g 2,s h s,j ) + f i,r (g r, h,j + g r,2 h 2,j + + g r,s h s,j ) to get the i, j entry of F(GH) ontrast the two proofs The index-heavy argument is hard to understand in that while the calculations are easy to check, the arithmetic seems unconnected to any idea The argument in the proof is shorter and also says why this property really holds This illustrates the comments made at the start of the chapter on vector spaces at least sometimes an argument from higher-level constructs is clearer We have now seen how to represent the composition of linear maps The next subsection will continue to explore this operation Exercises X 24 ompute, or state not defined - - 3 5 - (a) (b) @2 3 A -4 2 5 4 3 3 5 2-7 (c) @- A 5 2-2 (d) 7 4 3 3-5 3 8 4
234 hapter Three Maps etween Spaces X 25 Where A = - 2 = 5 2 4 4 compute or state not defined (a) A (b) (A) (c) (d) A() = -2 3-4 26 Which products are defined? (a) 3 2 times 2 3 (b) 2 3 times 3 2 (c) 2 2 times 3 3 (d) 3 3 times 2 2 X 27 Give the size of the product or state not defined (a) a 2 3 matrix times a 3 matrix (b) a 2 matrix times a 2 matrix (c) a 2 3 matrix times a 2 matrix (d) a 2 2 matrix times a 2 2 matrix X 28 Find the system of equations resulting from starting with h, x + h,2 x 2 + h,3 x 3 = d h 2, x + h 2,2 x 2 + h 2,3 x 3 = d 2 and making this change of variable (ie, substitution) x = g, y + g,2 y 2 x 2 = g 2, y + g 2,2 y 2 x 3 = g 3, y + g 3,2 y 2 X 29 onsider the two linear functions h: R 3 P 2 and g: P 2 M 2 2 given as here a @ ba p p- 2q 7 (a + b)x 2 +(2a + 2b)x + c px 2 + qx + r 7 q c Use these bases for the spaces = h@ A, @ A, @ Ai = h + x, - x, x 2 i 2 D = h,,, i 3 4 (a) Give the formula for the composition map g h: R 3 M 2 2 derived directly from the above definition (b) Represent h and g with respect to the appropriate bases (c) Represent the map g h computed in the first part with respect to the appropriate bases (d) heck that the product of the two matrices from the second part is the matrix from the third part 22 As Definition 23 points out, the matrix product operation generalizes the dot product Is the dot product of a n row vector and a n column vector the same as their matrix-multiplicative product? X 22 Represent the derivative map on P n with respect to, where is the natural basis h, x,, x n i Show that the product of this matrix with itself is defined; what map does it represent?
Section IV Matrix Operations 235 222 [leary] Matcheachtypeofmatrixwithallthesedescriptionsthatcouldfit: (i) can be multiplied by its transpose to make a matrix, (ii) can represent a linear map from R 3 to R 2 that is not onto, (iii) can represent an isomorphism from R 3 to P 2 (a) a 2 3 matrix whose rank is (b) a 3 3 matrix that is nonsingular (c) a 2 2 matrix that is singular (d) an n column vector 223 Show that composition of linear transformations on R is commutative Is this true for any one-dimensional space? 224 Why is matrix multiplication not defined as entry-wise multiplication? That would be easier, and commutative too 225 (a) Prove that H p H q = H p+q and (H p ) q = H pq for positive integers p, q (b) Prove that (rh) p = r p H p for any positive integer p and scalar r 2 R X 226 (a) How does matrix multiplication interact with scalar multiplication: is r(gh) =(rg)h? IsG(rH) =r(gh)? (b) How does matrix multiplication interact with linear combinations: is F(rG + sh) =r(fg)+s(fh)? Is(rF + sg)h = rfh + sgh? 227 We can ask how the matrix product operation interacts with the transpose operation (a) Show that (GH) T = H T G T (b) Asquarematrixissymmetric if each i, j entry equals the j, i entry, that is, if the matrix equals its own transpose Show that the matrices HH T and H T H are symmetric X 228 Rotation of vectors in R 3 about an axis is a linear map Show that linear maps do not commute by showing geometrically that rotations do not commute 229 In the proof of Theorem 22 we used some maps What are the domains and codomains? 23 How does matrix rank interact with matrix multiplication? (a) an the product of rank n matrices have rank less than n? Greater? (b) Show that the rank of the product of two matrices is less than or equal to the minimum of the rank of each factor 23 Is commutes with an equivalence relation among n n matrices? 232 (We will use this exercise in the Matrix Inverses exercises) Here is another property of matrix multiplication that might be puzzling at first sight (a) Prove that the composition of the projections x, y : R 3 R 3 onto the x and y axes is the zero map despite that neither one is itself the zero map (b) Prove that the composition of the derivatives d 2 /dx 2,d 3 /dx 3 : P 4 P 4 is the zero map despite that neither is the zero map (c) Give a matrix equation representing the first fact (d) Give a matrix equation representing the second When two things multiply to give zero despite that neither is zero we say that each is a zero divisor 233 Show that, for square matrices, (S + T)(S - T) need not equal S 2 - T 2
236 hapter Three Maps etween Spaces X 234 Represent the identity transformation id: V V with respect to, for any basis Thisistheidentity matrix I Show that this matrix plays the role in matrix multiplication that the number plays in real number multiplication: HI = IH = H (for all matrices H for which the product is defined) 235 In real number algebra, quadratic equations have at most two solutions That is not so with matrix algebra Show that the 2 2 matrix equation T 2 = I has more than two solutions, where I is the identity matrix (this matrix has ones in its, and 2, 2 entries and zeroes elsewhere; see Exercise 34) 236 (a) Prove that for any 2 2 matrix T there are scalars c,,c 4 that are not all such that the combination c 4 T 4 + c 3 T 3 + c 2 T 2 + c T + c I is the zero matrix (where I is the 2 2 identity matrix, with s in its, and 2, 2 entries and zeroes elsewhere; see Exercise 34) (b) Let p(x) be a polynomial p(x) =c n x n + + c x + c If T is a square matrix we define p(t) to be the matrix c n T n + + c T + c I (where I is the appropriately-sized identity matrix) Prove that for any square matrix there is a polynomial such that p(t) is the zero matrix (c) The minimal polynomial m(x) of a square matrix is the polynomial of least degree, and with leading coefficient, suchthatm(t) is the zero matrix Find the minimal polynomial of this matrix p 3/2 p -/2 /2 3/2 (This is the representation with respect to E 2, E 2,thestandardbasis,ofarotation through /6 radians counterclockwise) 237 The infinite-dimensional space P of all finite-degree polynomials gives a memorable example of the non-commutativity of linear maps Let d/dx: P P be the usual derivative and let s: P P be the shift map a + a x + + a n x n s 7 + a x + a x 2 + + a n x n+ Show that the two maps don t commute d/dx s 6= s d/dx; in fact, not only is (d/dx s)-(s d/dx) not the zero map, it is the identity map 238 Recall the notation for the sum of the sequence of numbers a,a 2,,a n nx a i = a + a 2 + + a n i= In this notation, the i, j entry of the product of G and H is this p i,j = rx g i,k h k,j k= Using this notation, (a) reprove that matrix multiplication is associative; (b) reprove Theorem 27