Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the ith row and jth column of A is denoted by a ij and is called the (i, j)-entry of A Each column of A is a list of m real numbers, which identifies a vector in R m A = a 1 a 2 a n The diagonal entries in an m n matrix A = a ij are a 11, a 22, a 33,, and they form the main diagonal of A A diagonal matrix is a square n n matrix whose non-diagonal entries are zero An example is the n n identity matrix, I n An m n matrix whose entries are all zero is a zero matrix and is written as 0 12 Sums and scalar multiples If A and B are m n matrices, then the sum A + B is the m n matrix whose columns are the sums of the corresponding columns in A and B Example: Let A = 4 0 5, B = 1 3 2 1 1 1, C = 3 5 7 2 3 0 1 Then, A + B = 5 1 6 2 8 9 but A + C is not defined because A and C have different sizes The scalar multiple ra is the matrix whose columns are r times the corresponding columns in A Example: If A and B are the matrices in the previous example, then A 2B = 1 1 1 2 2 2 2B = 2 =, 3 5 7 6 10 14 4 0 5 2 2 2 2 2 3 = 1 3 2 6 10 14 7 7 12 1/9
13 Matrix algebra Theorem 1 Let A, B, and C be matrices of the same size, and let r and s be scalars, then A + B = B + A r(a + B) = ra + rb (A + B) + C = A + (B + C) A + 0 = A r(a + B) = ra + rb (r + s)a = ra + sa r(sa) = (rs)a 14 Matrix multiplication When a matrix B multiplies a vector x, it transforms x into the vector Bx If this vector is then multiplied in turn by a matrix A, the resulting vector is A(Bx) Thus A(Bx) is produced from x by a composition of mappings, a linear transformation Our goal is to represent this composite mapping as multiplication by a single matrix, denoted by C = AB, so that A(Bx) = Cx = (AB)x If A is m n, B is n p, and x is in R p, denote the columns of B by b 1,, b p and the entries in x by x 1,, x p Then by definition of matrix-vector multiplication We then have Bx = x 1 b 1 + + x p b p A(Bx) = A(x 1 b 1 + + x p b p ) = x 1 Ab 1 + + x p Ab p, where we used linearity of A The vector A(Bx) is a linear combination of the vectors Ab 1,, Ab p, using the entries in x as weights In matrix notation, this linear combination is written as A(Bx) = Ab 1 Ab 2 Ab p x Thus multiplication by Ab 1, Ab 2,, Ab p transforms x into A(Bx) We found the matrix we are seeking 2/9
Definition 1 If A is an m n matrix, and if B is an n p matrix with columns b 1,, b p, then the product AB is the m p matrix whose columns are Ab 1,, Ab p That is, AB = A b 1 b 2 b p = Ab1 Ab 2 Ab p Multiplication of matrices corresponds to composition of linear transformations 15 Example Compute AB, where A = 2 3 and B = 1 5 Solution Write B = b 1 b 2 b 3, and compute: Therefore, 16 Matrix sizes 4 3 6 1 2 3 2 3 4 11 Ab 1 = =, Ab 1 5 1 1 2 = 2 3 6 21 Ab 3 = = 1 5 3 9 2 3 3 0 =, 1 5 2 13 AB = A 11 0 21 b 1 b 2 b 3 = 1 13 9 Each column of AB in this example is a linear combination of the columns of A using weights from the corresponding columns of B Also, the definition of AB shows that AB has the same number of rows as A and the same number of columns as B For example, if A is a 3 5 matrix and B is a 5 2 matrix, what are the sizes of AB and BA, if they are defined? Solution Since A has 5 columns and B has 5 rows, the product AB is defined and is a 3 2 matrix: The product BA is not defined because the 2 columns of B do not match the 3 rows of A 3/9
17 Example Find the entries in the second row of AB, where 2 5 0 A = 1 3 4 7 6 6 8 7, B = 7 1 3 2 3 0 9 Solution By the row-column rule, the entries of the second row of AB come from row 2 of A (and the columns of B): Notice that since the example above requested only the second row of AB, we could have written just the second row of A to the left of B and computed 7 6 1 3 4 7 1 = 5 1 3 2 This observation about rows of AB is true in general and follows from the row-column rule Let row i (A) denote the ith row of a matrix A Then 18 Properties of matrix multiplication row i (AB) = row i (A)B Theorem 2 Let A be an m n matrix, and let B and C have sizes for which the indicated sums and products are defined 1 A(BC) = (AB)C associative law of multiplication 2 A(B + C) = AB + AC left distributive law 3 (B + C)A = BA + CA right distributive law 4 r(ab) = (ra)b = A(rB) for any scalar r 5 I m A = A = AI n identity for matrix multiplication 19 Warnings 1 In general, AB = BA (left to right order is critical) One side may not even be defined because of size mismatch 2 The cancellation laws do not hold for matrix multiplication That is, if AB = AC, then it is not true in general that B = C If a product AB is the zero matrix, you cannot conclude in general that either A = 0 or B = 0 4/9
110 Powers of a matrix If A is an n n matrix and if k is a positive integer, then A k denotes the product of k copies of A: 111 The transpose of a matrix A k = k times {}}{ A A, A 0 = I n Given an m n matrix A, the transpose of A is the n m matrix, denoted by A T, whose columns are formed from the corresponding rows of A Eg, A = a b c, A T = d e f a b c d e f Theorem 3 Let A and B denote matrices whose sizes are appropriate for the following operations 1 (A T ) T = A 2 (A + B) T = A T + B T 3 For any scalar r, (ra) T = ra T 4 (AB) T = B T A T The transpose of a product of matrices equals the product of their transposes in the reverse order 112 Example What is (ABC) T? Give the formula for (ABx) T 2 Matrix inverse 21 Matrix inverse 22 The Inverse of a matrix Generalizing the notion of inverse from real number a a 1 = 1, an n n matrix A is said to be invertible if there is an n n matrix C such that CA = I and AC = I where I = I n, the n n identity matrix In this case, C is the inverse of A In fact, C is uniquely determined by A, because if B were another inverse of A, then 5/9
This unique inverse is denoted by A 1, so that B = BI = B(AC) = (BA)C = IC = C A 1 A = I, and AA 1 = I A matrix that is not invertible is sometimes called a singular matrix, and an invertible matrix is called a non-singular matrix 23 Example 2 5 If A = and C = 3 7 Therefore, C = A 1 24 2 2 matrix inverse 7 5, then 3 2 2 5 7 5 AC = = 3 7 3 2 CA = 7 5 2 5 = 3 2 3 7 1 0, 0 1 1 0 0 1 a b Theorem 4 Let A = If ad bc = 0, then A is invertible and c d A 1 1 d b = ad bc c a If ad bc = 0, then A is not invertible 25 Solution for a system with invertible matrix Theorem 5 If A is an invertible n n matrix, then for each b in R n, the equation Ax = b has the unique solution x = A 1 b 6/9
26 Properties of the matrix inverse Theorem 6 1 If A is an invertible matrix, then A 1 is invertible and (A 1 ) 1 = A 2 If A and B are n n invertible matrices, then so is AB, and the inverse of AB is the product of the inverses of A and B in the reverse order That is, (AB) 1 = B 1 A 1 3 If A is an invertible matrix, then so is A T, and the inverse of A T is the transpose of A 1 That is, For (2), remember that (A T ) 1 = (A 1 ) T 27 Example If AB = AC and A is invertible, show that B = C For square matrices A, B, C, when AB = BC and B is invertible, what is A? 28 Computing the matrix inverse Theorem 7 An n n matrix A is invertible if and only if A is row equivalent to I n, and in this case, any sequence of elementary row operations that reduces A to I n also transforms I n into A 1 An algorithm for Finding A 1 (recording the row operations): Row reduce the augmented matrix A I If A is row equivalent to I, then A I is row equivalent to I A 1 Otherwise, A does not have an inverse 29 Justification for matrix inversion algorithm Denote the columns of I n by e 1,, e n Then row reduction of A I to I A 1 can be viewed as the simultaneous solution of the n systems: Ax 1 = e 1, Ax 2 = e 2,, Ax n = e n 7/9
where the augmented columns of these systems have all been placed next to A to form A e1 e 2 e n = A I On the other hand, if matrix X is the inverse of A, then AX = I Using the definition of matrix multiplication, we have Ax1 Ax 2 Ax n = e1 e 2 e n This show that the columns of A 1 are precisely the solutions of the n linear systems above with e j on the right-hand-side This observation is useful because some applied problems may require finding only one or two columns of A 1 In this case, only the corresponding systems need to be solved 210 Example Find the inverse of the matrix A, if the inverse exists Solution Since A I, A is invertible, and 211 Numerical Note 0 1 2 A = 1 0 3 4 3 8 0 1 2 1 0 0 1 0 3 0 1 0 A I = 1 0 3 0 1 0 0 1 2 1 0 0 4 3 8 0 0 1 4 3 8 0 0 1 1 0 3 0 1 0 0 1 2 1 0 0 0 3 4 0 4 1 1 0 3 0 1 0 1 0 3 0 1 0 A I = 0 1 2 1 0 0 0 1 2 1 0 0 0 0 2 3 4 1 0 0 1 3/2 2 1/2 1 0 0 9/2 7 3/2 0 1 0 2 4 1 0 0 1 3/2 2 1/2 9/2 7 3/2 A 1 = 2 4 1 3/2 2 1/2 In practice, A 1 is seldom computed, unless the entries of A 1 are needed Computing both A 1 and A 1 b takes about three times as many arithmetic operations as solving Ax = b by row reduction, and row reduction may be more accurate 212 Elementary matrices An elementary matrix is one that is obtained by performing a single elementary row operation on an identity matrix The next example illustrates the three kinds of elementary matrices 8/9
213 Example Let 1 0 0 0 1 0 1 0 0 E 1 = 0 1 0, E 2 = 1 0 0, E 3 = 0 1 0 4 0 1 0 0 1 0 0 5 Compute E 1 A, E 2 A, and E 3 A, for a 3 3 matrix A, and describe how these products can be obtained by elementary row operations on A a b c Let A = d e f then g h i a b c d e f a b c E 1 A = d e f, E 2 A = a b c, E 3 A = d e f g 4a h 4b i 4c g h i 5g 5h 5i If an elementary row operation is performed on an m n matrix A, the resulting matrix can be written as EA, where the m m matrix E is created by performing the same row operation on I m 214 Properties of elementary matrices Each elementary matrix E is invertible The inverse of E is the elementary matrix of the same type that transforms E back into I 1 0 0 Example: Find the inverse of E 1 = 0 1 0 4 0 1 Solution To transform E 1 into I, add +4 times row 1 to row 3 The elementary matrix that does this is 1 0 0 E1 1 = 0 1 0 4 0 1 215 Proof for matrix inverse theorem If A is invertible, then the system Ax = b has unique solution for any RHS b That implies the matrix A has a pivot in each row Since A is n n, it is row equivalent to I n If A I n then there is a series of row operation (elementary matrices) such that E p E 1 A = I n Elementary matrices (and their product) are invertible and (E p E 1 ) 1 = E 1 1 E 1 p Multiplying the first equation with this inverse, we have E p E 1 A = I n, (E p E 1 ) 1 E p E 1 A = (E p E 1 ) 1, A 1 = E p E 1 9/9