Roberto s Notes on Linear Algebra Chapter 4: Matrix Algebra Section 4 Matrix products What you need to know already: The dot product of vectors Basic matrix operations. Special types of matrices What you can learn here: A procedure to multiply matrices that is the foundation of matrix algebra and leads to many applications in all kinds of fields of mathematics and science. We have seen that the dot product of two vectors is defined as: a a... a b b... b a b a b... a b n n n n We have also seen that the dot product turns out to be an easy and natural way to define a linear equation: a b a b a b c......... n n a a a x x x c n Finally, we introduced the concept of a matrix to help us solve a system of linear equations: ax ax... a nxn c a a... an c ax ax... anxn c a a... an c.................. amx amx... amnxn cm a m am... amn cm Notice that in this original use of matrices the dot product is not employed explicitly, even though it hides behind the presence of linear equations. But there is a way of involving the dot product in the solution of certain linear systems, and it is a way that will open the door to many more uses of matrices. The key to doing this is to notice that if each equation of the system can be written as a dot product, the whole system ought to be written as a dot product involving the whole matrix of coefficients. n And since matrices are vectors, this should be possible! Exactly! Let us start from the simplest situation. Definition The product of an mn matrix A and an n- dimensional vector v is the m-dimensional vector denoted by Av whose i-th row is the dot product of the i-th row of A and v: r r v av av... a nvn av av... anv r r v n Av v......... r r v a v a v... a v m m m m mn n Linear Algebra Chapter 4: Matrix Algebra Section 4: Matrix products Page
Example: 3 3 A 5 6, v 6 0 4 3 4 The 34 matrix can be multiplied by the 4D vector as follows: Wow! Lots of operations! 3 3 Av 5 6 6 0 4 3 4 3 36 4 5 3 6 6 4 45 3 0 46 3 4 9 Yes, but we have calculators and computers for that now, so don t let that scare you. At the same time, if need be, you should be able to perform all these operations by hand if the vectors are small and consist of simple numbers. As in a test? Yes. But the more important task is to think properly about this type of multiplication, so as to generalize it to all kind of matrices. We begin by noticing the following. Knot on your finger In order to multiply a matrix by a vector, the dimensions of the two must fit properly, in that the number of columns of the matrix must equal the dimension of the vector. For convenience of notation, it is better to write the vector to the right of the matrix and in vertical form. I can see that the condition on the dimensions is necessary, but is the vertical position also needed? Not at this point, but this convention will prove very useful later, as it will make many more operation run a lot smoother in terms of notation and clarity. But a matrix can be seen as a vector consisting of columns vectors, so can we use this operation to multiply two matrices? Bingo! Here is the key definition. Definition The product of an mn matrix A and an np matrix B is the mp matrix denoted by AB whose element in position (i, j) is the dot product of the i-th row of A and the j-th column of B, as shown here: Linear Algebra Chapter 4: Matrix Algebra Section 4: Matrix products Page
ra ra AB... c B c B c B ra m... p r( A) c( B) r( A) c( B)... r( A) c( B) p... r A c B r A c B r A c B p...... ra i cb j... ra m cb r( A) m c( B)... r( A) m c( B) p What a mess of symbols! Granted, but the idea and the method are a lot easier than what the symbols suggest. The next example and those following the remaining concepts will give you a starting point. Start practicing now and the continuing use of the matrix product in later sections will make you a master of this skill. Example: 3 3 0 A 5 6, B 6 0 4 3 4 The 34 matrix A can be multiplied by the 4 B matrix as follows: 3 3 5 0 AB 5 6 45 0 6 0 4 3 9 8 4 Notice that the entry of the product is in position (,) and is obtained as the dot product of first row of A and the first column of B. Similarly, the 8 is the product of the third row of A and the second column of B. All other entries are The requirement on dimensions that we saw when multiplying a matrix by a vector holds more generally for the product of two matrices. Knot on your finger In order to multiply two matrices, their dimensions must fit properly, in that the number of columns of the first matrix must equal the number of rows of the second. When this dimension requirement is satisfied, the product matrix has the same number of rows as the first matrix and the same number of columns as the second one: Example: A B C m n n p m p 3 3 A, 0, 4 B C 3 4 4 Given the dimensions of these matrices, there are several products that cannot be computed. For instance: Linear Algebra Chapter 4: Matrix Algebra Section 4: Matrix products Page 3
AC cannot be computed, since A has 3 columns, but C only has rows. However, CA can be computed and it equals: 3 3 5 6 CA 3 4 4 5 3 0 CB cannot be computed, since C only has columns, but B has 3 rows. However, BC can be computed and it equals: 3 9 4 BC 0 3 4 4 0 Notice, however, that AB and BA can both be computed: 3 3 4 AB 0 4 4 4 3 3 BA 0 3 4 4 8 0 6 This is weird: AB and BA can both be computed, but are not the same! You noticed, eh? This is a very important point about matrix product, so let us highlight it. Knot on your finger Matrix product is not a commutative operation in general, that is, given two matrices A and B, the products AB and BA may not both be computable, depending on their dimensions may not be equal, even when they are both computable In some cases, it may be true that AB=BA, but this is an exceptional situation, not the norm. Is there a way of knowing when the two different orders provide the same product? Not in general, unless you are dealing with some very easy cases, like diagonal matrices. They commute? Yes, and you ll verify that in one of the Learning activities. Are there any good properties of the matrix product? Oh yes, and we shall see them in the next section. Here is a pretty obvious fact, but one worth stating. Linear Algebra Chapter 4: Matrix Algebra Section 4: Matrix products Page 4
Knot on your finger If A is a square matrix, its square is defined as: A AA. More generally, if A is a square matrix and n is an integer, the n-th power of A is defined as: n n A AA. Because of the peculiar way in which a matrix product is defined, defining a matrix quotient, as well as fractional and negative powers of a matrix is very difficult, works only in rare cases and is not very useful. Therefore, with the one notable exception of inverse matrices, which will be seen soon, we shall avoid them Good, one less thing to worry about. Maybe, but also one less interesting problem to explore. Fine, but what is this product good for, besides having weird properties? Remember that this section started by my pointing out that the dot product is used to define a linear equation, but was not used in the context of linear systems. Now that we have a matrix product we can close the circle. Proof Any system of the form: Technical fact a x a x... a x c a x a x... a x c... a x a x... a x c n n n n m m mn n m can be written in the much shorter notation Ax=c, where A is the matrix of coefficients, x is the vector of variables and c is the vector of constants. The simpler equation denotes the following: a a... a n x c a a... a n x c Ax c......... a a... a x c If we perform the product on the left, we obtain: m m mn n m ax ax a nxn c ax ax anx n c... a x a x a x c m m mn n m Linear Algebra Chapter 4: Matrix Algebra Section 4: Matrix products Page 5
But for these two vectors to be the same, each pair of components must be the same, which brings us back to the original system. Example: 3x 5y 8z 3 5x y z 8x 4y 6z 5 This system can be written as 3 5 8 x 3 5 y 8 4 6 z 5 Fine, but so what? How does that help us solve the system? Excellent question, but we need to look at a few more interesting things about matrix products before I can show you what the advantages are. Time for your practice. Wait! I have a major concern: it looks like this matrix product involves lots of small computations that are easy to get wrong. Are we expected to be perfect with those computations? Good point again, and worthy some knots of the finger. Knot on your finger Matrix product involves many simple additions and multiplications, which can easily lead to computational errors. For this reason, it is ideally suited for computers and calculators. Make sure that you can perform it correctly in simple cases and learn how to use your calculator to check your answers. Summary Two matrices may be multiplied together to produce a new matrix by using the dot product procedure AND assuming that their dimensions fit suitably. Matrix product may be used to define a matrix power, but only for square matrices. The process of matrix product, or matrix multiplication, has many uses both within linear algebra and for its practical applications. Common errors to avoid Multiplying two matrices involves a large number of simple operations and it is easy to make arithmetical error when performing it. In practice, we should prefer the use of a computer. Matrix product does not have all the properties of the usual product and therefore we should be careful when making assumptions that may prove to be wrong! Linear Algebra Chapter 4: Matrix Algebra Section 4: Matrix products Page 6
Solutions to selected Learning questions for Section LA 4-4: Matrix products Review questions:. Describe how the product of two matrices is performed.. Explain why and how the dimensions of two matrices must match in order for them to be multiplied. 3. Identify and describe the algebraic properties that are true for matrix product. 4. Identify and describe the algebraic properties that are true for scalar product, but not for matrix product. Memory questions:. Which vector operation is used when performing a matrix product?. What condition on the dimensions of two matrices allows them to be multiplied? 3. How is the element AB ij of the product AB computed? 4. Is matrix multiplication associative, commutative, both or neither? Compute the matrix product presented in questions -6 by hand and then by calculator. Computation questions:.. 3. 5 6 3 4 7 8 5 6 7 8 3 4 6 0 4 3 5 4. 5. 0 3 3 4 5 0 3 3 4 5 6. 3 3 5 6 6 0 4 3 4 Linear Algebra Chapter 4: Matrix Algebra Section 4: Matrix products Page 7
7. Given the matrix 8. Given the matrix a) 3 A 5 0 A 0 A b), compute A., compute: 9. Compute the only suitable product of the matrices A 3 and 5 0 B 3 4. 3 A 0. There is only one order that can be used to multiply the following three matrices together. Determine what such order is and compute the resulting product. 0 4 A 3 3 B C 3 3 0 3 0 3. Given the matrices B 4 5 and C 4, determine 7 which of the following products exist, explain why or why not, and compute one of the products that does exist: T T T T T T BC, CB, B C, C B, BC, B C, BB, CC 5 3 3. Given the matrices A,. 3. Given the matrix 0 3 4. Given the matrix 5. Write the linear system 5 0 B 0 5, compute T AB and 3 A 4 5 3, compute the matrix AA T 3I. 4 0 D, compute xy 3 3x y 7 I. D as a matrix equation. AB T Theory questions:. What is meant by the notation 0 3 3 5 4? 3. How should we write the vectors u and v so that their dot product can be computed as a matrix product? 4. What type of matrix must C be so that n C exists?. Construct two matrices that can be multiplied in one order, but not the other. 5. Which matrices commute with all other matrices? Linear Algebra Chapter 4: Matrix Algebra Section 4: Matrix products Page 8
Proof questions: r A c c c 3 and B r, then the product AB, done as a matrix product, provides the same matrix as AB, done as a dot product of vectors, r 3. Show that if provided the columns and rows involved are then multiplied as matrices. To make this question more manageable, you may want to start by proving this fact for a specific example and then retrace the steps with generic 3 3 matrices. This way of multiplying matrices is called an outer product and it has interesting theoretical consequences. Templated questions:. Construct two matrices of suitable dimensions and multiply them. Do this lots of times! What questions do you have for your instructor? Linear Algebra Chapter 4: Matrix Algebra Section 4: Matrix products Page 9
Linear Algebra Chapter 4: Matrix Algebra Section 4: Matrix products Page 0