MATH 030: MATRICES Matrix Operations We have seen how matrices and the operations on them originated from our study of linear equations In this chapter we study matrices explicitely Definition 01 A matrix is a rectangular array of numbers called the entries or elements of the matrix The following are all matrices, 1 3 1 0 3,, 3 1 3, [3,, 1, 4], 5 5 4 4 7 4 1 6 The size of a matrix is a summary of the numbers of rows and columns in the matrix, we say it is an m n matrix if it has m rows and n columns The above examples are, 3, 3 1, 1 3 and 3 3 respectively We say a 1 n matrix is a row matrix (or row vector) and n 1 a column matrix (a column vector) Alternatively we may describe the matrix with two double subscripts to refer to the matrix A The entries of A in row i and column j are denoted as a ij In the second example above a 1 while a 1 3 Thus we can compactly denote a matrix by A by [a ij ] or [a ij ] n m if it is important to record the size of A as well With this notation, any m n matrix may be described as a 11 a 1 a 1n a 1 a a n A a m1 a m a mn If the columns of A are the vectors a 1, a,,a n ; A is represented as [a 1,, a n ] Alternatively if the rows of A are A 1,, A n A takes the form A A 1 A n The diagonal entries of A are a 11, a, and if m n so that A is an n n matrix, we say it is a sqare matrix as well A square matrix with all entries except the diagonal entries vanishing is called a diagonal matrix If the diagonal matrix have the same entries along the diagonal we say this is a scalar matrix, and if all of the non-zero components in the diagonal matrix are 1s we say this matrix is the identity matrix As we have seen that matrices are a generalization of vectors, many of the conventions and operations for vectors carry through to matrices We say two matrices are equal if they have the same size and if their corresponding entries are equal That is, if A [a ij ] m n and B [b ij ] r s, they are equal if and only if m r, n s and a ij b ij for all i and j 1
MATH 030: MATRICES Example 0 Consider the matrices A [1, 4, 3] and B 1 4 although the components of the matrices agree, these two are not equal as m 1, r 3 and n 3 3 and s 1 these vectors cannot be equal Matrix Addition and Scalar Multiplication As we had vector addition we d like to extend this idea to matrices; we define matrix addition componentwise If A [a ij ] and B [b ij ] are m n matrices, their sum A + B is the m n matrix obtained by adding their entries, ie Example 03 DQ: efining 3 1 A, B 1 5 A + B [a ij + b ij ] 0 1 3, C ; 1 4 3 1 5 A: we cannot compute A + C or B + C, however we can add A and B: 5 A + B 0 9 5 Scalar multiplication is easily defined for matrices If A is an m n matrix and c is a scalar, we say the scalar multiple ca is the matrix obtained by multiplying each entry of A by c That is, ca c[a ij ] [ca ij ] Example 04 Taking the third example C, we may compute 4 6 1 1 3 C, 10 C 3 1, ( 1)C 1 5 5 The matrix ( 1)A will be written as A called the negative or additive inverse of A We may take the difference of two matrices to produce a new matrix If A and B are m n matrices, A B A + ( B) Example 05 Consider C C O 3 C C + 1 5 3 1 5 0 0 0 0 The above matrix with all entries equal to zero is called a zero matrix and will be denoted by O or O m n if it is important to know the size of the matrix Given A [a ij ] m n and O m n we have the familiar properties: A + O A O + A, A A 0 A + A
MATH 030: MATRICES 3 Matrix Multiplication There is one more property of matrices which generalizes the idea of dot product for vectors, a product of sorts that is equivalent to the composition of functions Unlike the definitions of matrix addition and scalar multiplication, this definition is not componentwise Definition 06 If A is an m n matrix and B is an n r matrix, the matrix product C AB is an m r matrix The (i, j) entry of this matrix product may be computed as c ij a i1 b 1j + a i b j + a in b nj Notice that A must have the same number of columns as rows of B, this is related to the fact that the dot product of a row vector with a column vector is computable only if each has the same number of components To see this consider C in the case that A is an n 1 matrix and B is a 1 n matrix Example 07 Q: Compute AB if 4 0 3 1 1 3 1 A, B 5 1 1 1 1 1 0 6 A: As A is an 3 matrix and B is an 3 4, the product is going to be an 4 matrix To calculate the first row, we treat it as a row vector and take the dot product of it with the first,second, third and fourth columns of B treated as vectors c 11 1( 4) + 3(5) + ( 1)( 1) 1 c 1 1(0) + 3( ) + ( 1)() 8 c 13 1(3) + 3( 1) + ( 1)(0) 0 c 14 1( 1) + 3(1) + ( 1)(6) 4 The second row of AB is computed in the same manner, ie by treating it as a row vector and computing its dot product with the four column vectors of B 1 8 0 4 AB 4 5 7 Example 08 Consider the linear system x 1 x + 3x 3 5, x 1 + 3x + x 3 1, x 1 x + 4x 3 14 this may be seen as a matrix product on the left hand side 1 3 1 3 1, x 1 x 5 1 1 4 x 3 4 Equivalently we may write this as Ax b where A is the coefficient matrix and b is the constant vector Every linear system may be described in this manner, and this notation will be immensely useful for expressing systems of linear equations As an example, notice that Ax b has a solution if and only if b is in the span of the column vectors of A
4 MATH 030: MATRICES Theorem 09 Let A be an n m matrix and e i a 1 m standard unit vector, and e j an n 1 standard unit vector Then (1) e i A is the i-th row of A and, () Ae j is the j-th column of A Proof We prove the second part first, supposing a 1,, a n are the column vectors of the matrix A, the product of Ae j can be written as Ae j 0a 1 + + 0a j 1 + 1a j + 0a j+1 + + 0a n a j To prove the second part consider the transpose (defined in the next subsection) of e i A which is A t e t i where et i is now a column vector, and the row vectors of A are now the column vectors of A t The first part of the proof applies and by taking the transpose we prove the result Transpose of a Matrix We now introduce an operation on matrices that has no analogue in the real numbers: Definition 010 The transpose of an m n matrix A is the n m matrix A t obtained by interchanging the rows and columns of A; so that the i-th column of A is now the i-th row of A t Example 011 Let A 1 3, B 5 0 1 a b, C [5, 1, ] c d their transpose are now A t 1 5 3 0, B t a c, C t 5 1 b d 1 Example 01 Define 1 3 A 3 5 0, B 0 4 [ 0 ] 1 1 0 We say that A is symmetric as A t A and B is not symmetric since B t B, since B t B we say this matrix is anti-symmetric Definition 013 A square (ie, n n) matrix is (1) symmetric if and only if A ij A ji for all i and j () anti-symmetric if and only if A ij A ji Matrix Powers When A and B are two n n matrices, their product AB will also be an n n matrix A special case occurs when A B It makes sense to define A AA and generalize that to A k A 1 A k 1 A where the subscript indicates the number of factors multiplied together, when k is a positive integer It will be helpful to impose the condition that A 0 I n How do matrix powers behave like powers of real numbers? Two properties follow immediately from the definition of matrix multiplication Proposition 014 If A is a square matrix and r and s are non-negative integers, then
MATH 030: MATRICES 5 A r A s A r+s (A r ) s A rs 1 1 Example 015 Q: If A then 1 1 1 1 1 1 A 1 1 1 1 and A 3 1 1 1 1 4 4 4 4 Show that A n n 1 n 1 n 1 n 1 A: We will use mathematical induction to prove this example We begin by proving the base case, ie A 1 A which is easily checked Next we assume A k k 1 k 1 k 1 k 1 for some integer k > 1 If we prove this identity holds for the k + 1-th power, we will have proven the identity as k was arbitrary And so, we calculate A k+1 A k A: A k+1 k 1 k 1 1 1 k 1 k 1 1 1 k 1 + k 1 k 1 + k 1 1 1 k 1 + k 1 k 1 + k 1 1 1 k k (k+1) 1 k k (k+1) 1 1 1 (k+1) 1 (k+1) 1 1 1 Thus the formula holds for all n 1 Example 016 Q: If B, B 1 0 are B 3 and B 4? A: The third factor is 1 0 0 1 1 0 Thus B 3 I 3 and B 4 B, [ 1 0 1 0 0 1 ] [ 1 0 ] 1 0, what Partitioned Matrices As we have treated matrices as collections of column or row vectors, it will be helpful to regard a matrix as being composed of a number of smaller submatrices Introducing vertical and horizontal lines into a matrix, we can partition it into blocks Often the form of the matrix suggest how it may be partitioned in a helpful manner As an example consider 1 0 0 1 0 1 0 1 3 A 0 0 1 4 0 0 0 0 1 7 0 0 0 7
6 MATH 030: MATRICES It would seem natural to partition this matrix in the following manner 1 0 0 1 0 1 0 1 3 A 0 0 1 4 0 I B 0 0 0 1 7 O C 0 0 0 7 where I is the 3 3 identity matrix, B is a 3 matrix and O, 3 zero matrix and C is the matrix Seen this way A is a whose entries are matrices themselves Often when the product of large matrices must be determined, there is a computational advantage from seeing them as partitioned matrices, especially when there are large blocks of zeros in the original matrix Thanks to the propeties of matrix multiplication, multiplication of partitioned matrices is just like ordinary matrix multiplication To begin let us consider some special cases of partitioned matrices Each viewpoint will give rise to a different way of computing the product of two matrices Suppose A is an m n matrix and B an n r matrix; if we partition B in terms of its column vectors as B [b 1 b r ] then AB A[b 1 b r ] [[Ab 1 Ab r ] The form on the right side is called the matrix-column representation of the product 4 1 1 3 Example 017 Q: If A, and B 1 compute the product of 1 3 0 A with B s column vectors and determine the matrix product of AB A: Calculating the two new vectors Ab 1 [ 13 ] 5 and Ab we find that the new matrix is then 13 5 AB [Ab 1 Ab ] Alternatively if we treat A and B as before (respectively m n and n r matrices), we may partition A as a column vector of row vectors, ie, A 1 A A A m then the matrix product AB may be seen as A 1 A AB B A m A 1 B A B A m B This is called the row-matrix representation of the product Example 018 Q: Use the row matrix representation to compute AB for the matrices in the previous example
MATH 030: MATRICES 7 A: Calculating A 1 B [13, 5] and A B [, ], thus the matrix product is again, A1 B 13 5 A A B The definition of the matrix product AB uses the natural partition of A into rows and B into columns, and so it is deserving of the name row-column representation of the product Similarly we may partition A into columns and B into rows, B 1 or the column-row representation Defining A [a 1 a n ] and B we B n define AB as AB a 1 B 1 + a B + + a n B n This sum resembles a dot product expansion, except that each individual term are two matrices being multiplied Each a i B j term is the product of an m 1 and a 1 r matrix, thus each term a i B j is an m r matrix We call a i B j the outer products, and the sum in terms of these as the outer product expansion of AB Each of the previous partitions are a special case of partitioning in general A matrix A is said to be partitioned if horizontal and vertical lines have been introduced to sectioning off A into submatrices called blocks Partitioning allows A to be written as a matrix whose entries are blocks If two matrices are the same size and have the same partitioning, ie similar block structure, we may add and scalar multiply block by block Furthermore with an appropriate partitioning, matrices can be multiplied blockwise as well Example 019 Consider the matrices A and B, 1 0 0 3 1 4 3 1 1 0 1 0 1 3 A 0 0 1 4 0 0 0 0 1 7, B 1 1 1 1 5 3 3 1 1 0 0 0 0 0 0 7 0 1 0 0 3 writing this in terms of smaller matrices indicated by the partitioning the product of these two is simplipy: A11 A AB 1 B11 B 1 B 13 A 1 A B 1 B B 3 A11 B 11 + A 1 B 1 A 11 B 1 + A 1 B A 11 B 13 + A 1 B 3 A 1 B 11 + A B 1 A 1 B 1 + A B A 1 B 13 + A B 3 For this calculation to succeed we must check that all of the matrix products are computable, which is the case as the columns of the blocks of A match the numbers of columns in the blocks of B The matrices A and B are said to be partitioned conformably for block multiplication References [1] D Poole, Linear Algebra: A modern introduction - 3rd Edition, Brooks/Cole (01)