An m n matrix is an array A = Matrix Arithmetic a 11 a 12 a 1n a 21 a 22 a 2n a m1 a m2 a mn of real numbers a ij An m n matrix has m rows and n columns a ij is the entry in the i-th row and j-th column We often write A = a ij If A = a ij and B = b ij are two matrices of the same size they are both m n then they can be added The sum A + B is the m n matrix c ij where c ij = a ij + b ij for all rows i and columns j Suppose that c R The scalar product ca is the m n matrix d ij defined by d ij = ca ij 0 m,n is the m n zero matrix all entries are zero Suppose that A, B and C are m n matrices, and c, d R Then we have the following identities: A + B = B + A A + B + C = A + B + C cda = cda 0 m,n + A = A = A + 0 m,n The matrix 1A is an additive identity for A; that is, A + 1A = 1A + A = 0 m,n, so we write A = 1A Two matrices A = a ij and B = b jk can be multiplied to form a new matrix AB if A is m n and B is n l for some m, n, l The restriction is that the number of columns of A must be equal to the number of rows of B The product AB is defined to be the m l matrix e ik where n e ik = a ij b jk For example, 2 3 1 4 5 6 7 1 3 = = 4 2 6 3 8 1 2 4 4 j=1 2 4 + 3 3 + 1 2 2 2 + 3 8 + 1 4 2 6 + 3 1 + 1 4 4 4 + 5 3 + 6 2 4 2 + 5 8 + 6 4 4 6 + 5 1 + 6 4 7 4 + 1 3 + 3 2 7 2 + 1 8 + 3 4 6 7 + 1 1 + 3 4 19 32 19 43 72 53 37 34 55 I n is the n n identity matrix, I n = δ ij where { 1 if i = j δ ij = 0 if i j For instance, I 3 = 0 1
Suppose that A, B, C are matrices which can be multiplied where A is m n and c R Then we have the following identities: ABC = ABC AB + C = AB + AC A + BC = AC + BC cab = cab = AcB 0 l,m A = 0 l,n A0 n,l = 0 m,l I m A = A AI n = A Warning: Some of the familiar properties of multiplication in R are no longer true for matrices If A, B and C are n n with n 2, in general we have that AB BA It is possible for A and B to both be nonzero, but AB = 0 n,n Further, there are nonzero matrices A, B, C such that AB = AC but B C The transpose of an m n matrix A = a ij, written as A T, is the n n matrix b ij where b ij = a ji ; that is, A T is the matrix whose rows are the columns of A For example, T 1 2 3 = 1 5 2 1 5 1 2 3 2 The transpose satisfies the following formulas: A T T = A ca T = ca T A + B T = A T + B T AB T = B T A T Column vectors and row vectors R m is the set of all m 1 column vectors with real coefficients If v R m, then v 1 v 2 v = v m with v 1,, v m R R n is the set of all 1 n row vectors with real coefficients If w R n, then with w 1,, w n R We have a dot product on R m defined by w = w 1,, w n x y = x 1,, x m y 1,, y m = x 1 y 1 + x 2 y 2 + + x m y m R for x = x 1,, x m, y = y 1,, y m R m Similarly, we have a dot product on R n defined by x y = x 1 y 1 + + x n y n R 2
for x = x 1 x n, y = y 1 y n R n Some useful formulas Suppose that A is an m n matrix with coefficients in R, and x = x 1,, x n T R n Let v w = v T w be the dot product of the vectors v, w R n Writing A = A 1, A 2,, A n where A i R m are the columns of A, we obtain the formula 1 Ax = x 1 A 1 + + x n A n Writing 2 Ax = A = A 1 A 2 A m where A j R n are the rows of A, we obtain the formula A 1 x A T 1 A 2 x x A T 2 x A m x = A T m x If A is an m n matrix and B = B 1,, B l is n l with each B j R n an n 1 column vector then the m l matrix 3 AB = AB 1, AB 2,, AB l where each AB i is an m 1 column matrix As a consequence, we obtain the following formula, which we will use later in this note 4 AB C = AB AC where A is m n, B is n s and C is n t, giving us an n s + t matrix B C, and after multiplication, an m s + t matrix AB AC Inverses of square n n matrices Definition 01 Suppose that A is an n n matrix A is invertible if there exists an n n matrix B such that AB = I n and BA = I n Theorem 02 Suppose that an n n matrix A is invertible matrix C such that AC = CA = I n Then there is a unique Proof Suppose that B, C are matrices such that AB = BA = I n and AC = CA = I n Then B = I n B = CAB = CAB = CI n = C Thus B = C is unique 3
Suppose that A is invertible We have seen that the matrix B satisfying AB = BA = I n is unique, so we may call this matrix the inverse of A, and write B = A 1 Example 03 Show, using the definition of invertibility, that 5 2 A = 7 3 is invertible with inverse A 1 = 3 2 7 5 Solution: Let B = 3 2 7 5 we cannot call this matrix A 1 until after we have shown that it really is the inverse to A We compute 5 2 3 2 1 0 AB = = = I 7 3 7 5 2 and BA = 3 2 7 5 Thus A is invertible with inverse A 1 = B 5 2 7 3 = 1 0 Example 04 Show, using the definition of invertibility, that A = 0 0 is not invertible = I 2 Solution: Suppose that A is invertible Let A 1 be the inverse of A We compute A 2 = 0 2,2 Then = A = I 0 0 2 A = A 1 AA = A 1 A 2 = A 0 2,2 = 0 2,2 = 0 0 This is a contradiction to our assumption that A is invertible, so A is not invertible Theorem 05 Suppose that A, B are n n matrices which are invertible Then AB is invertible, with AB 1 = B 1 A 1 Proof We have that ABB 1 A 1 = AI n A 1 = AA 1 = I n and B 1 A 1 AB = B 1 I n B = B 1 B = I n, so that AB is invertible with inverse AB 1 = B 1 A 1 Corollary 06 Suppose that r is a positive integer and A 1, A 2,, A r are invertible n n matrices Then A 1 A 2 A r is invertible with A 1 A 2 A r 1 = A 1 r A 1 r 1 A 1 2 A 1 1 4
Definition 07 An n n elementary matrix is a matrix which is obtained by performing one elementary row operation on the n n identity matrix I n The 3 3 elementary matrix E 1 obtained by interchanging the first and second row of the identity matrix is E 1 = 0 The 3 3 elementary matrix E 2 obtained by multiplying the second row of the identity matrix by 2 is E 2 = 0 2 0 The 3 3 elementary matrix E 3 obtained by adding 4 times the third row of the identity matrix to the first row is E 3 = 1 0 4 0 Theorem 08 Suppose that E is an elementary matrix Then E is invertible and E 1 is an elementary matrix E 1 is obtained by performing the elementary row operation on I n which transforms E back to I n Considering the inverse of the above three examples of elementary matrices, we have that E1 1 is obtained by interchanging the first and second rows of the identity matrix so that E1 1 = 0 2 is obtained by multiplying the second row of the identity matrix by 1 2 so that E2 1 = 1 0 2 0 E 1 E 1 3 is obtained by adding -4 times the third row of the identity matrix to the first row so that E3 1 = 1 0 4 0 Theorem 09 Let A be an m n matrix, and suppose that B is obtained from A by performing a single elementary row operation on A Let E be the m m elementary matrix obtained by performing this row operation on I m Then EA = B An example illustrating the theorem is the case of the matrix A = 1 3 5 2 4 6 3 5 7 5
If we multiply A on the left by E 1, the elementary matrix obtained from interchanging the first and second rows of the 3 3 identity matrix, we obtain E 1 A = 0 1 3 5 2 4 6 = 2 4 6 1 3 5 3 5 7 3 5 7 which is the matrix obtained from A by interchanging the first and second rows We observe that: The RRE form of an n n matrix A is either the identity matrix I n, or the RRE form has a row of zeros for its last row Theorem 010 Suppose that A is an n n matrix Then A is invertible if and only if the RRE form of A is I n Proof First suppose that A is row equivalent to I n Since A is row equivalent to I n, we have by the previous theorem a product I n = E r E r 1 E 2 E 1 A where E 1, E 2,, E r are elementary matrices Since E 1,, E r are invertible, we have that A = E1 1 2 Er 1 is invertible, with inverse 5 A 1 = E r E r 1 E 2 E 1 Now suppose that A is invertible Let C be the RRE form of A We have a factorization C = E r E r 1 E 2 E 1 A where E 1, E 2,, E r are elementary matrices Multiplying both sides of this equation on the right by A 1, we have CA 1 = E r E 1 I n E r E 1 I n is computed by performing a sequence of elementary row operations on I n, so there can be no row consisting entirely of zeros in CA 1 However, if C is not I n, then the last row of C is the zero vector, and then the last row of CA 1 is also the zero vector, which is impossible Thus the RRE form of A is C = I n The proof of the above theorem is constructive and tells us how to a solve a problem like the following Example 011 Write as a product of elementary matrices A = 1 3 4 2 Solution: First transform A into RRE form, keeping careful track of the elementary row operations and the precise order in which they are performed and making this clear from your work 1 3 4 2 Multiply the second row by 1 10 Add -4 times the first row to the second row 1 3 1 3 0 10 Add -3 times the second row to the first row 6 1 0
Let E 1 be the elementary matrix of the first operation, E 2 the elementary matrix of the second operation, E 3 be the elementary matrix of the third operation, so that 1 0 1 0 1 3 E 1 =, E 4 1 2 = 0 1, E 3 = 10 and E 3 E 2 E 1 A = I 2 We compute E1 1 1 0 = 4 1, E2 1 1 0 = 0 10, E3 1 1 3 = Thus we have an expression of A as a product of elementary matrices, A = E1 1 1 3 E 1 2 E 1 3 I 2 = 4 1 0 1 Algorithm to compute the inverse of a matrix Suppose that A is an n n matrix Transform the n 2n matrix A I n into a reduced row echelon form C B by a sequence of elementary row operations A is invertible if and only if C = I n The RRE form of A is I n If A is invertible, then B = A 1 The algorithm is just a restatement of Theorem 010 Let E 1, E 2,, E r be the elementary matrices corresponding to the sequence of elementary row operations which transform A into its RRE form C Then using the formula 4, we have E r E r 1 E 2 E 1 A I n = E r E 1 A E r E 1 = C E r E 1 We have by Theorem 010 that A is invertible if and only if C = I n and if this holds, then A 1 = E r E 1 by 5 Example 012 Let A = 1 2 3 2 5 3 1 0 8 Use the Algorithm to compute the inverse of a matrix to 1 Determine if A is invertible 2 If A is invertible, compute A 1 Solution: A I 3 = 1 2 3 2 5 3 1 0 8 1 2 3 3 0 0 1 1 2 0 0 0 2 1 0 5 2 1 14 6 3 13 5 3 5 2 1 1 2 3 3 0 2 5 1 2 3 3 0 1 A is invertible since it is row equivalent to I 3 7 2 1 0 1 2 1 0 5 2 1 46 9 13 5 3 5 2 1
2 A 1 = 46 9 13 5 3 5 2 1 Example 013 Let A = 1 6 4 2 4 1 1 2 5 Use the Algorithm to compute the inverse of a matrix to 1 Determine if A is invertible 2 If A is invertible, compute A 1 Solution: A I 3 = 2 4 1 1 6 4 1 2 5 0 1 6 4 0 8 9 0 0 0 1 6 4 0 8 9 0 8 9 2 1 1 1 1 1 2 1 0 1 1 A is not invertible since it is row equivalent to a matrix with a row of zeros and thus is not row equivalent to I 3 2 Not applicable the inverse does not exist Systems of Equations A system of m equations in n unknowns, can be written in matrix form as a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2 a m1 x 1 + a m2 x 2 + + a mn x n = b m 6 A x = b where A is the m n matrix A = a ij, x = x 1 x n R n and b = b 1 b m R m If m = n and A is invertible, then 6 has a unique solution; it is x = A 1 b 8