chapter 5 INTRODUCTION TO MATRIX ALGEBRA GOALS The purpose of this chapter is to introduce you to matrix algebra, which has many applications. You are already familiar with several algebras: elementary algebra, the algebra of logic, the algebra of sets. We hope that as you studied the algebra of logic and the algebra of sets, you compared them with elementary algebra and noted that the basic laws of each are similar. We will see that matrix algebra is also similar. As in previous discussions, we begin by defining the objects in question and the basic operations. 5. Basic Definitions Definition: Matrix. A matrix is a rectangular array of elements of the form a a 2 a 3 º a n a 2 a 22 a 23 º a 2 n A = a 3 a 32 a 33 º a 3 n ª ª ª ª a m a m2 a m3 º a mn A convenient way of describing a matrix in general is to designate each entry via its position in the array. That is, the entry a 34 is the entry in the third row and fourth column of the matrix A. Depending on the situation, we will decide in advance to which set the entries in a matrix will belong. For example, we might assume that each entry a ij ( i m, j n) is a real number. In that case we would use M mµn HR L to stand for the set of all m by n matrices whose entries are real numbers. If we decide that the entries in a matrix must come from a set S, we use M mµn HSL to denote all such matrices. Definition: Order of a Matrix. The matrix A that has m rows and n columns is called an mµ n (read "m by n") matrix, and is said to have order m µ n. Since it is rather cumbersome to write out the large rectangular array above each time we wish to discuss the generalized form of a matrix, it is common practice to replace the above by A = Aa ij E. In general, matrices are often given names that are capital letters and the corresponding lower case letter is used for individual entries. For example the entry in the third row, second column of a matrix called C would be c 32. Example 5... A = K 2 3 0-5 O, B = 0 2 5, and D = 2 5 6-2 3 4 2 8 are 2 µ2, 3 µ, and 3 µ3 matrices, respectively Since we now understand what a matrix looks like, we are in a position to investigate the operations of matrix algebra for which users have found the most applications. Example 5..2. First we ask ourselves: Is the matrix A = K 2 O equal to 3 4 the matrix B = K 2 O? No, they are not because the corresponding entries in the second row, second column of the two matrices are not 3 5
equal. Next, is A = K 2 3 2 O equal to B = K O? No, although the corresponding entries in the first two columns are identical, B doesn't 4 5 6 4 5 have a third column to compart to that of A. We formalize these observations in the following definition. Definition; Equality of Matrices. A matrix A is said to be equal to matrix B (written A = B) if and only if: () A and B have the same order, and (2) all corresponding entries are equal: that is, a ij = b ij for all appropriate i and j. 5.2 Addition and Scalar Multiplication Example 5.2.. Concerning addition, it seems natural that if A = K 0 2 - O and B = K 3 4-5 2 O, then A + B = K + 3 0 + 4 2-5 - + 2 O = K 4 4-3 O. However, if A = K 2 3 0 2 O and B = K 3 0 O, can we find A + B? No, the orders of the two matrices must be identical. 2 8 Definition: Matrix Addition. Let A and B be mµ n matrices. Then A + B is an mµ n matrix where HA + BL ij = a ij + b ij (read "the ith jth entry of the matrix A + B is obtained by adding the ith jth entry of A to the ith jth entry of B"). If the orders of A and B are not identical, A + B is not defined. It should be clear from Example 5.2. and the definition of addition that A + B is defined if and only if A and B are of the same order. Another frequently used operation is that of multiplying a matrix by a number, commonly called a scalar in this context. Scalars normally come from the same set as the entries in a matrix. For example, if A œ M mµn HR L, a scalar can be any real number. Example 5.2.2. If c = 3 and if A = K -2 O and we wish to find c A, it seems natural to multiply each entry of A by 3 so that 3 5 3 A = K 3-6 O, and this is precisely the way scalar multiplication is defined. 9 5 Definition: Scalar Multiplication. Let A be an m µ n matrix and c a scalar. Then c A is the mµ n matrix obtained by multiplying c times each entry of A; that is HcAL ij = c a ij. 5.3 Multiplication of Matrices - For a video introduction to this section, go to http://faculty.uml.edu/klevasseur/ads2/videos/matrixmultiplication/ A definition that is more awkward to motivate (and we will not attempt to do so here) is the product of two matrices. In time, the reader will see that the following definition of the product of matrices will be very useful, and will provide an algebraic system that is quite similar to elementary algebra. Definition: Matrix Multiplication. Let A be an mµ n matrix and let B be an nµ p matrix. The product of A and B, denoted by AB, is an mµ p matrix whose ith row jth column entry is HA BL i j = a i b j + a i 2 b 2 j + º + a i n b n j n = ai k b k j k= for i m and j p. The mechanics of computing one entry in the product of two matrices is illustrated in Figure 5.3..
Figure 5.3. Computation of one entry in the product of two 3 by 3 matrices The computation of a product can take a considerable amount of time in comparison to the time required to add two matrices. Suppose that A and B are n µn matrices; then HABL ij is determined performing n multiplications and n - additions. The full product takes n 3 multiplications and n 3 n 2 additions. This compares with n 2 additions for the sum of two n µn matrices. The product of two 0 by 0 matrices will require,000 multiplications and 900 additions, clearly a job that you would assign to a computer. The sum of two matrices requires a more modest 00 additions. This analysis is based on the assumption that matrix multiplication will be done using the formula that is given in the definition. There are more advanced methods that, in theory, reduce operation counts. For example, Strassen's algorithm (http://mathworld.wolfram.com/strassenformulas.html) computes the product of two n by n matrices in 7 ÿ7 log 2 n - 6 ÿ 4 log 2 n º 7 n 2.808 operations. There are practical issues involved in actually using the algorithm in many situations. For example, round-off error can be more of a problem than with the standard formula. Example 5.3.. Let A = Remarks: A B = 0 3 2-5 0 3 2-5 K 6 O =, a 3 µ2 matrix, an let B = K 6 O, a 2 µ matrix. Then A B is a 3 µ matrix: ÿ6 + 0 ÿ 2 ÿ + 3 ÿ6-5 ÿ6 + ÿ = 6 20-29 () The product A B is defined only if A is an m µn matrix and B is an n µ p matrix; that is, the two "inner" numbers must be the equal. Furthermore, the order of the product matrix A B is the "outer" numbers, in this case m µ p. (2) It is wise to first determine the order of a product matrix. For example, if A is a 3 µ2 matrix and B is a 2 µ2 matrix, then A B is a 3 µ2 matrix of the form c c 2 A B = c 2 c 22 c 3 c 32 Then to obtain, for example, C 3, we multiply corresponding entries in the third row of A times the first column of B and add the results. Example 5.3.2.
Let A = K 0 3 0 O, and B = K 0 3 2 O. Then ÿ3 + 0 ÿ2 ÿ0 + 0 ÿ A B = K 0 ÿ3 + 3 ÿ2 0 ÿ0 + 3 ÿ O = K 3 0 6 3 O Note: B A = K 3 0 2 3 O ¹ A B Remarks; () An n µn matrix is called a square matrix. (2) If A is a square matrix, A A is defined and is denoted by A 2, and A A A = A 3, etc. (3) The m µn matrices whose entries are all 0 are denoted by 0 mµn, or simply 0, when no confusion arises regarding the order. EXERCISES FOR SECTIONS 5. THROUGH 5.3 A Exercises. Let A = K - 2 3 O, B = K 0 0 - O, and C = K 3-5 3-2 2 O (a) Compute A B and B A. (b) Compute A + B and B + A. (c) If c = 3, show that cha + BL = c A + c B. (d) Show that HA BL C = A HB CL. (e) Compute A 2 C. (f) Compute B + 0 (g) Compute A 0 2µ2 and 0 2µ2 A, where 0 2µ2 is the 2 µ2 zero matrix, (h) Compute 0 A, where 0 is the real number (scalar) zero. (i) Let c = 2 and d = 3. Show that Hc + dl A = c A + d A. 2. Let A = 0 2 2-5 3 2 Compute, if possible;, B = 0 2 3 2-3 -2, and C = 2 2 3 4 0 3-4 (a) A - B (b) A B (c) A C - B C (d) A HB CL (e) C A - C B (f) C x y z w 3. Let A = K 2 0 0 O. Find a matrix B such that A B = I and B A = I, where I = K 0 3 0 O 4. Find A I and B I where I is as in Exercise 3, where A = K 8-2 3 O and B = K 9 5 5-7 O. What do you notice?
5. Find A 3 if A = 0 0 0 2 0 0 0 3. What is A 5 equal to? B Exercises 6. (a) Determine I 2 and I 3 if I = 0 0 0 0 0 0 (b) What is I n equal to for any n? (c) Prove your answer to part (b) by induction.. 7. (a) If A = K 2 - O, X = K x x 2 O, and B = K 3 O, show that A X = B is a way of expressing the system 2 x + x 2 = 3 x - x 2 = (b) Express the following systems of equations using matrices: (i) 2 x - x 2 = 4 x + x 2 = 0 (ii) x + x 2 + 2 x 3 = x + 2 x 2 - x 3 = - x + 3 x 2 + x 3 = 5 (iii) x + x 2 = 3 x 2 = 5 x + 3 x 3 = 6 using matrices.
5.4 Special Types of Matrices We have already investigated one special type of matrix, namely the zero matrix, and found that it behaves in matrix algebra in an analogous fashion to the real number 0; that is, as the additive identity. We will now investigate the properties of a few other special matrices. Definition: Diagonal Matrix. A square matrix D is called a diagonal matrix if d ij = 0 whenever i ¹ j. Example 5.4.. A = 0 0 0 2 0 0 0 5, B = 3 0 0 0 0 0 0 0-5, and I = 0 0 0 0 0 0 are all diagonal matrices. In Example 5.4., the 3 µ3 diagonal matrix I whose diagonal entries are all 's has the singular property that for any other 3 µ3 matrix A we have A I = I A = A. For example: Example 5.4.2. If A = A I = I A = 2 5 6 7-2 3-3 0 2 5 6 7-2 3-3 0 2 5 6 7-2 3-3 0. and, then In other words, the matrix I behaves in matrix algebra like the real number ; that is, as a multiplicative identity. In matrix algebra the matrix I is called simply the identity matrix. Convince yourself that if A is any n µn matrix A I = I A = A. Definition: Identity Matrix. The n µ n diagonal matrix whose diagonal components are all 's is called the identity matrix and is denoted by I or I n. In the set of real numbers we realize that, given a nonzero real number x, there exists a real number y such that x y = y x =. We know that real numbers commute under multiplication so that the two equations can be summarized as x y =. Further we know that y = x - =. Do x we have an analogous situation in M nµn HR L? Can we define the multiplicative inverse of an n µn matrix A? It seems natural to imitate the definition of multiplicative inverse in the real numbers. Definition: Matrix Inverse. Let A be an nµ n matrix. If there exists an nµ n matrix B such that A B = B A = I, then B is the multiplicative inverse of A (called simply the inverse of A) and is denoted by A - (read "A inverse"). When we are doing computations involving matrices, it would be helpful to know that when we find A -, the answer we obtain is the only inverse of the given matrix. Remark: Those unfamiliar with the laws of matrices should go over the proof of Theorem 5.4. after they have familiarized themselves with the Laws of Matrix Algebra in Section 5.5. Theorem 5.4.. The inverse of an nµ n matrix A, when it exists, is unique. Proof: Let A be an n µn matrix. Assume to the contrary, that A has two (different) inverses, say B and C. Then B = B I Identity property of I = B HA C L Assumption that C is an inverse of A = HB AL C Associativity of matrix multiplication = I C Assumption that B is an inverse of A = C Identity property of I Example 5.4.3. Let A = K 2 0 0 3 O. What is A-? Without too much difficulty, by trial and error, we determine that A - = might lead us to guess that the inverse is found by taking the reciprocal of all nonzero entries of a matrix. Alas, it isn't that easy! A = K 2-3 5 O, the "reciprocal rule" would tell us that the inverse of A is B = - 3 2 5 2 0 0 3. This. Try computing A B and you will see that you don't get the identity matrix. So, what is A -? In order to understand more completely the notion of the inverse of a matrix, it would be beneficial to have a formula that would enable us to compute the inverse of at least a 2 µ2 matrix. To do this, we need to recall the definition of the determinant of a 2 µ2 matrix. Appendix A gives a more complete description of the determinant of a 2 µ2 and higher-order matrices. Definition: Determinant of a 2 2 Matrix. Let A = K a b O. The determinant of A is the number det A = a d - b c. c d In addition to det A, common notation for the determinant of matrix A is A. This is particularly common when writing out the whole matrix, If
which case we would write Example 5.4.4. a c b d for the determinant of the general 2 µ2 matrix. If A = K 2 O then det A = ÿ5-2 ÿ H-3L =. -3 5 If B = K 2 O then det B = ÿ4-2 ÿ2 = 0 2 4 Theorem 5.4.2. Let A = K a b c d O. If det A ¹ 0, then A- = K d -b det A -c a O Proof: See Exercise 4 at the end of this section. Example 5.4.5. Can we find the inverses of the matrices in Example 5.4.4? If A = K 2-3 5 O then A- = K 5-2 3 O = The reader should verify that A A - = A - A = I. 5 3-2 The second matrix, B has a determinant equal to zero. We we tried to apply the formula in Theorem 5.4.2, we would be dividing by zero. For this reason, the formula can't be applied and in fact B - does not exist. Remarks: () In general, if A is a 2 µ2 matrix and if det A = 0, then A - does not exist. (2) A formula for the inverse of n µn matrices n 3 can be derived that also involves det A, Hence, in general, if the determinant of a matrix is zero, the matrix does not have an inverse. However the formula for even a 3 µ 3 matrix is very long and is not the most efficient way to compute the inverse of a matrix. (3) In Chapter 2 we will develop a technique to compute the inverse of a higher-order matrix, if it exists. (4) Matrix inversion comes first in the hierarchy of matrix operations; therefore, A B - is AHB - L. EXERCISES FOR SECTION 5.4 A Exercises. For the given matrices A find A - if it exists and verify that A A - = A - A = I If A - does not exist explain why. (a) A = K 3 2 O (b) A = K 6-3 8-4 O (c) A = K -3 0 O (d) A = K 0 0 O (e) Use the definition of the inverse of a matrix to find A - : 3 0 0 A = 0 2 0 0 0-5 2. For the given matrices A find A - if it exists and verify that A A - = A - A = I If A - does not exist explain why. (a) A = K 2 - - 2 O (b) A = K 0 0 2 O (c) A = K c 0 O
(d) A = K a b O, were a > b > 0. b a 3. (a) Let A = K 2 3 3-3 O and B = K 4 2 O. Verify that HA BL- = B - A -. (b) Let A and B be n µn invertible matrices. Prove that HA BL - = B - A -. Why is the right side of the above statement written "backwards"? Is this necessary? Hint: Use Theorem 5.4.. B Exercises 4. Let Let A = K a b c d O. Derive the formula for A-. 5. (a) Let A and B be as in problem 3 above. Show that detha BL = Hdet AL Hdet BL. (b) It can be shown that the statement in part (a) is true for all n µn matrices. Let A be any invertible n µn matrix. Prove that detha - L = Hdet AL -. Note: The determinant of the identity matrix I n is for all n, see Appendix A for details. (c) Verify that the equation in part (b) is true for the matrix in exercise l(a) of this section. 6. Prove by induction that for n, K a 0 0 b O n = K an 0 0 b n O. 7. Use the assumptions in exercise 5 to prove by induction that if n, detha n L = Hdet AL n. 8. Prove: If the determinant of a matrix A is zero, then A does not have an inverse. Hint: Use the indirect method of proof and exercise 5. C Exercise 9. (a) Let A, B, and D be n µn matrices. Assume that B is invertible. If A = B D B -, prove by induction that A m = B D m B - is true for m. -8 5 (b) Given that A = K -6 O = B K 0 0 2 O B- where B = K 5 3 3 2 O what is A0?
5.5 Laws of Matrix Algebra The following is a summary of the basic laws of matrix operations. Assume that the indicated operations are defined; that is, that the orders of the matrices A, B, and C are such that the operations make sense. () A + B = B + A (2) A + HB + CL = HA + BL + C (3) c HA + BL = c A + c B, where c œ R. (4) Hc + c 2 L A = c A + c 2 A, where c, c 2 œ R. (5) c Hc 2 AL = Hc ÿc 2 L A, where c, c 2 œ R. (6) 0 A = 0, where 0 is the zero matrix. (7) 0 A = 0, where 0 on the left is the number 0. (8) A + 0 = A. (9) A + H-L A = 0. (0) A HB + CL = A B + A C. () HB + CL A = B A + C A. (2) AHB CL = HA BL C. (3) I A = A and A I = A. (4) If A - exists, HA - L - = A. (5) If A - and B - exist, HA BL - = B - A - Example 5.5.. If we wished to write out each of the above laws more completely, we would specify the orders of the matrices. For example, Law 0 should read: (0) Let A, B, and C be m µn, n µ p, and n µ p matrices, respectively, then A HB + CL = A B + A C Remarks: () Notice the absence of the "law" A B = B A. Why? (2) Is it really necessary to have both a right (No. ) and a left (No. 0) distributive law? Why? (3) What does Law 8 define? What does Law 9 define? EXERCISES FOR SECTION 5.5 A Exercises. Rewrite the above laws specifying as in Example 5.5. the orders of the matrices. 2. Verify each of the Laws of Matrix Algebra using examples. 3. Let A = K 2 0 - O, B = K 3 7 6 2-5 Matrix Algebra: (a) A B + A C (b) A - (c) A HB + CL 0-2 4 O, and C = K O. Compute the following as efficiently as possible by using any of the Laws of 7
(d) HA 2 L - (e) HC + BL - A - 4. Let A = K 7 4 2 O and B = K 3 5 O. Compute the following as efficiently as possible by using any of the Laws of Matrix Algebra: 2 4 (a) A B (b) A + B (c) A 2 + A B + B A + B 2 (d) B - A - (e) A 2 + A B 5. Let A and B be n µn matrices of real numbers. Is A 2 - B 2 = HA - BL HA + BL? Explain
5.6 Matrix Oddities We have seen that matrix algebra is similar in many ways to elementary algebra. Indeed, if we want to solve the matrix equation A X = B for the unknown X, we imitate the procedure used in elementary algebra for solving the equation a x = b. Notice how exactly the same properties are used in the following detailed solutions of both equations. Solution of a x = b Solution of A X = B a x = b A X = B a - Ha xl = a - b if a ¹ 0 A - HA XL = A - B if A - exists Ha - al x = a - b associative law HA - AL X = A - B associative law x = a - b definition of inverse I X = A - B definition of inverse x = a - b identity property of X = A - B identity property of I Certainly the solution process for A X = B is the same as that of a x = b. The solution of x a = b is x = b a - = a - b. In fact, we usually write the solution of both equations as x = b. In matrix algebra, the solution of a X A = B is X = B A -, which is not necessarily equal to A - B. So in matrix algebra, since the commutative law (under multiplication) is not true, we have to be more careful in the methods we use to solve equations. It is clear from the above that if we wrote the solution of A X = B as X = B, we would not know how to interpret the answer B. Does it mean A A A - B or B A -? Because of this, A - is never written as A. Some of the main dissimilarities between matrix algebra and elementary algebra are that in matrix algebra: () A B may be different from B A. (2) There exist matrices A and B such that A B = 0, and yet A ¹ 0 and B ¹ 0. (3) There exist matrices A where A ¹ 0, and yet A 2 = 0. (4) There exist matrices A where A 2 - A with A ¹ I and A ¹ 0 (5) There exist matrices A where A 2 = I, where A ¹ I and A ¹ -I EXERCISES FOR SECTION 5.6 A Exercises. Discuss each of the above "oddities" with respect to elementary algebra. 2. Determine 2 µ2 matrices which show each of the above "oddities" are true. B Exercises 3. Prove the following implications, if possible: (a) A 2 = A and det A ¹ 0 A = I (b) A 2 = I and det A ¹ 0 A = I or A = -I. 4. Let M nµn HR L be the set of real n µn matrices. Let P Œ M nµn HR L be the subset of matrices defined by A œ P if and only if A 2 = A. Let Q Œ P be defined by A œ Q if and only if det A ¹ 0. (a) Determine the cardinality of Q. (b) Consider the special case n = 2 and prove that a sufficient condition for A œ P Œ M 2µ2 HR L is that A has a zero determinant (i.e., A is singular) and tr HAL = where tr HAL = a + a 22 is the sum of the main diagonal elements of A. (c) Is the condition of part b a necessary condition?
C Exercises 5. Write each of the following systems in the form A X = B, and then solve the systems using matrices. (a) 2 x + x 2 = 3 x - x 2 = (b) 2 x - x 2 = 4 x - x 2 = 0 (c) 2 x + x 2 = x - x 2 = (d) 2 x + x 2 = x - x 2 = - (e) 3 x + 2 x 2 = 6 x + 4 x 2 = - 6. Recall that p HxL = x 2-5 x + 6 is called a polynomial, or more specifically, a polynomial over R, where the coefficients are elements of R and x œ R. Also, think of the method of solving, and solutions of, x 2-5 x + 6 = 0. We would like to define the analogous situation for 2 µ2 matrices. First define where A is a 2 µ2 matrix p HAL = A 2-5 A + 6 I. Discuss the method of solving and the solutions of A 2-5 A + 6 I = 0. 7. (For those who know calculus) (a) Write the series expansion for a centered around a = 0. (b) Use the idea of exercise 6 to write what would be a plausible definion of A where A is an n µ n matrix. (c) If A = K 0 - O and B = K 0 0 0 0 O, use the series in part (b) to show that A = K - O and 0 B = K - 0 O. (d) Show that A B ¹ B A (e) Show that A+B = K 0 0 O (f) Is A B = A+B?
SUPPLEMENTARY EXERCISES FOR CHAPTER 5 Sections 5. through 5.3. Determine x and y in the following: 0-2. Let A = 2 5 3-4 (a) 2 A - 3 B (b) 2 A - 5 A (c) AC + BC, B = 0 0 2 3-2 -, C = 3 4 5 K x + y 5-2 x - y O = K 3 5-2 4 O. Compute: 3. Let A and B be two mµm matrices with AB = BA. Prove by induction on n that AB n = B n A for n greater than or equal to. 4. Prove by induction that if n is a positive integer, and Section 5.4 5. Determine A - A 3 if A = K 2 3 4 O 6. Let A = K 4-2 -2 5 O and B = K 2 0 O A = 0 0 0 0, then A n = n nhn - Lê2 0 n 0 0 Compute A + B, A 2 + AB +BA + B 2, and B - A -. You may save some time by thinking before plunging into the computations. 7. For what real number c will the matrix D have no inverse? Explain your answer. D = K 3 5 4 c O 8. Let P = :K a b c d O œ M 2µ2HR L ad ¹ bc>. Fact: The inverse of a diagonal matrix belonging to P can be found simply by reciprocating the diagonal elements of the matrix. (a) Determine K 3 0 0 6 O -. (b) Suppose K a b c d O œ P and K a b - c d O In general, is K a b c d Section 5.5 = K êa b c êd O O a diagonal matrix? If yes, explain why; if no, give the most general form of such a matrix K a b c d O. 9. (a) Let A and B be nµn matrices. Expand HA + BL 2. (b) Is HA + BL 2 ever equal to A 2 + 2 AB + B 2? Explain. 0. Solve the following matrix equation for X. Be careful to explain under which conditions each step is possible. Section 5.6. Prove or disprove: A - = A and B - = B HABL - = AB. AX + C = BX 2. The following is true for all real numbers a and b: aÿb = 0 if and only if a = 0 or b = 0. Is any part of this statement true for nµn matrices A and B? Explain. Give an example and proof. 3. Let A = K a b 0 b O, where a, b, c, d œ R. Show that the matrices of the form A = ±K O, and A = ±K O are also solutions to the c d c - 0 - equation A 2 = I, confirming that a quadratic matrix equation can have an infinite number of solutions. Are there any others?