The Matrix Vector Product and the Matrix Product As we have seen a matrix is just a rectangular array of scalars (real numbers) The size of a matrix indicates its number of rows and columns A matrix with m rows and n columns is said to be of size m n The order is important here The number of rows is alway specified first Thus, is a matrix, while is a matrix We add two matrices simply by adding corresponding entries This means that if A is an m n matrix, then we may add it to any other m n matrix, but not to a matrix of a different size For example, we have + + + + + 7 On the other hand we can t add and because they differ in size We say that matrix addition is performed entrywise because we add corresponding entries If we want to subtract one matrix from another, we simply subtract the corresponding entries The notion of the product of a scalar and a matrix is similar To multiply an matrix A by a scalar c, we simply multiply entrywise by c For example These operations are intuitive and easy to grasp On the other hand the matrix product is not intuitive Moreover, it s complicated and involves enough calculation so that it is easy to make a mistake There are several ways to view the matrix product and so there are several ways to define it Two definitions will be presented here The first one is useful theoretically although it is a bit more difficult to compute The second definition is easier to compute, but it obscures some of the product s theoretical implications For both of the reasons mentioned above, it is useful to define an intermediate notion of this product We shall begin by defining the product of an m n matrix and an n matrix Now an n matrix with entries x, x,, has the form x x
and as such is just a (vertically displayed) list of numbers In other words it is a vector For this reason we shall call n matrices column matrices or column vectors Readers who are put off by calling such objects vectors can just think of them as column matrices Such readers should begin to feel more comfortable when we come to the formal study of vectors In order to emphasize the special nature of column vectors we use a special notation for them They are denoted by lower case letters decorated with a half arrow Thus, we may write x x x, v, u, etc Armed with this new notation, we may use it to view a matrix as being determined by its columns The columns of the matrix are c, c and c and so we may write c c c If we have a general m n matrix then its n columns are c a a a n a A a a n, a m a m a mn a a a m, c a a a m,, c n a n a n a mn With these preparations, we may now define the matrix vector product Just as with matrix addition it is possible to perform this multiplication only when the matrix and column vector have the right respective sizes Specifically, if A is an m n matrix, then the column vector must have size n The formal definition is as follows
Definition If A is an m n matrix with columns c, c,, c n and x is an n column vector with entries x, x,,, then the product of x by A is the m column vector determined by the formula A x c c c n x x x c + x c + + c n Here s an example Suppose A Since A is a matrix, it can multiply only column vectors So if then A x x, + + 8 Now let s discuss the computational definition of the matrix vector product The notion of a row matrix or row vector will be useful here By definition a row matrix or row vector is n matrix Note that if r r r r n and x x x, then r x r r r n x x x r + x r + + r n x r + x r + r n For example + +
The point is that the product of a n matrix and an n column vector is a matrix which is just a real number Note that + + + + + + 8 In our new notation, we may write + + and + + Hence, we have 8 In other words the first entry in A x is the product of the first row of A and x and the second entry in A x is the product of the second row and x In still other words if we define r and r then we have r x and r x and so the matrix product takes the form A x Here are some more examples 8 8 r x r x 8
+ 9 + + 8 8 + + 9 7 + 8 7 It is worth emphasizing that we may only form the product of m n matrices and n column vectors and that the result is an m column vector Another dividend is that the problem of solving a system of linear equations has an equivalent formulation in the language of the matrix vector product To illustrate this let s consider an example Suppose we have the linear system If we write then x x x x + x x + x x A, x x x and b, x A x b is the same as x x or x x x x + x x x + x x Thus the equation A x b is just another way of encoding the information determined by the linear system Since the matrix A is determined by the coefficients of the equations in the linear system, it is called the coefficient matrix for the system In general if we are given a system of m equations in n unknowns of the form a x + a x + + a n b a x + a x + + a n b a m x + a m x + + a mn b n, then the associated coefficient matrix is a a a n a a a n a m a m a mn
If we also write x x x and b b, b n then the equation A x b contains the same information as the original linear system Now we are ready to tackle the product of two matrices We ve done the hard work This next step is relatively easy Definition Suppose A is an m n matrix and B is an m p matrix In this case it is possible to form the product AB as follows Write B in column vector notation B c c c p, where c, c,, c p are the column vectors determined by the columns of B Since each column is an n column vector, we may form the matrix vector products A c, A c,, A c p The product is by definition AB A c c c p A c A c A c p Thus, AB is the m p matrix whose columns are A c, A c,, A c p Here is an example Note that in the product 7 8 8 7 8 7 8 7 + 8 + + 8 + 9 8 the (, ) entry is the product of the first row and the first column, the (, ) entry is the product of the second row and the first column, the (, ) entry is the product of the first row and the second column and the (, ) entry is the product of the second row row and the second column In general, the (i, j) entry in a matrix product is the matrix vector product of the i th row and the j th column
7 Here are some more examples + + + + + + 9 () + + + + + + () 7 7 8 Notice that in () we have the product of a and a matrix and the result is a matrix In (), the product of the same matrices in reverse order is given Now we are calculating the product of a and a and the result is a matrix