A- SUMMER 0 LECTURE 5 NATHANIEL GALLUP Agenda Elimination to the identity matrix Inverse matrices LU factorization Elimination to the identity matrix Previously, we have used elimination to get a system of equations into upper-triangular form, and then used back substitution to find the solution Instead of doing back substitution, we can continue to use elimination to get the matrix into diagonal form: with 0 s above and below the diagonal of the matrix This will make it much easier to read off the solution to the system of equations Let s do an example of this: From last monday, we had the following system of equations: 0 0 7 5 0 We did elimination to get it into upper-triangular form: 0 0 0 0 Then we used back substitution to solve the system Instead of using back substitution this time, let s continue elimination: () Let s clear above the, entry We could subtract / times the third row from the second row, but to avoid using fractions, let s first scale row by / to obtain: 0 0 0 0 Note that this is a new row operation we re using: scaling a row by a nonzero number Doing this also does not change the set of solutions to our system of equations since we ve just multiplied an equation by a nonzero number, and we can recover the original equation by dividing by that number () Now we can clear above the in the, entry We subtract times row from row to obtain: 0 0 0 0 Notice that because the in the, entry is the only nonzero entry in the third row (to the right of the bar), this operation does not affect any of the other entries in the matrix, other than the one we are trying to eliminate Date: July 7th, 0
A- SUMMER 0 LECTURE 5 () Now let s clear the in the,, entry of our matrix by subtracting row from row, to obtain: 0 7 0 0 0 0 () Now we want to clear the in the, entry Again to avoid fractions, let s divide the second equation by / to obtain: 0 7 0 0 0 0 (5) Now we can clear the We subtract times the second row from the third row to obtain: 0 0 0 0 0 0 (6) Now finally let s divide the first row by, to obtain: 0 0 0 0 0 0 (7) Notice that we can immediately read the solution of the system of equations off from this matrix: The third equation tells us that =, the second equation tells us that x =, and the first equation tells us that x = 0 Hence the solution is: x x = (Note that this is the same solution we had before) Let s do another example, this time one where we don t get exactly one solution: Example Here s the system of equations from last Friday: 7 5 0 0 0 We used elimination to get this matrix into the upper-triangular form: 7 0 0 0 0 0 0 Now let s continue to do elimination to clear as much as we can: () We can clear above the in the, entry by adding times Row to Row We get 0 5 0 0 0 0 0 0 () Let s multiply the first row by / to get its pivot to be, and let s multiply the second row by for the same purpose We get
A- SUMMER 0 LECTURE 5 0 5 0 0 0 0 0 0 () This is as much as we can do The third row is missing a pivot, so we can t clear the / in the, entry But the solution is still easy to see: Since there is no pivot in the third column, we call a free variable This means can be any real number t and as long as we choose x and x correctly, we ll get a solution The second equation tells us that x =, and the first equation tells us that if = t then, x = t 5 Therefore our solution is So were done! x = x t 5 t = t 0 + 5 0 This type of elimination (where we do back substitution using elimination) has a special name: First recall that the first nonzero entry of any row in an augmented system is called a pivot Definition The matrix in an augmented system is in reduced row-echelon form if the following conditions are satisfied: () Rows with all 0 s are below rows with nonzero entries () A pivot in row j is in a column strictly to the left of the pivot of any row below j () All pivots are s () All entries below and above pivots are 0 The process of using row operations to reduce a matrix to reduced row-echelon form is called Gauss-Jordan Elimination Let s do one more example, and let s do our first example of a non-square system, and one with more than variables: Example Consider the system of equations: The corresponding augmented system is x + x = 6 x x + x = 0 9x x 7x = 0 6 0 9 7 We ll use Gauss-Jordan Elimination to reduce this to reduced row-echelon form: The general strategy: first clear below the pivots to get the matrix into upper triangular form, then divide each row to get the pivots to be, then clear above the pivots to get the matrix into reduced row-echelon form () The first step is to clear below the pivot in the first row (which is the in the, entry) We subtract times Row from Row, and 9 times Row from Row to obtain:
A- SUMMER 0 LECTURE 5 0 6 0 5 5 0 7 6 50 () The second step is to clear below the pivot in the second row, which is the in the, entry We subtract times Row from Row to obtain: 0 6 0 5 5 0 0 () Now let s divide each row to get the pivots to be The pivot in the first row is already We multiply the second row by and the third row by / to obtain: 0 6 0 5 5 0 0 / 7 () Now we clear above all of the pivots The first two pivots, there s nothing to do They only have 0 s above them For the third pivot, we add 5 times Row to Row, and times Row to Row to obtain: 0 0 0 0 5/ 0 0 / 7 (5) We re done with elimination! Now we just need to interpret our solution Since there is no pivot in the th column, x is free, and can be any real number t Then The third equation tells us that = t 7 The second equation tells us that x = 5 t The first equation tells us that x = t Hence the solution is x t x = 5 t t 7 = t 5 + 7 x t 0 Group Work Homework Question (On WebWork) Circle the pivots on the following matrices Which of the following matrices is in reduced row-echelon form? If the matrix is not, then explain why not () 0 0 0 0 0 () () 0 0 0 0 0 0 0 0 0 0 0 0 0
() A- SUMMER 0 LECTURE 5 5 5 0 0 0 0 0 0 0 0 0 Homework Question (To be turned in on paper) Put the matrices following augmented systems into reduced row-echelon form, then use it to solve the system of equations: () () () x x + = x x + = 5 x x + x = x 7x + x = 0 x + x = 0 x + x + = 0 x + x + x = 0 x + 5x + 5 + 9x = 0 Inverse Matrices Now we re going to talk about a theoretically different way to solve systems of equations (in practice, it turns out we will end up doing the same steps) If we had a single linear equation with one variable like: 5x = 0 We would solve this equation by dividing by multiplying both sides of the equation by 5 We get 5 5x = 5 0 = Now 5 is the multiplicative inverse if 5, in the sense that 5 5 = Therefore our equation becomes and we have solved the equation x = We want to look for the matrix analog of a multiplicative inverse: Definition An n n matrix B is the multiplicative inverse of an n n matrix B if AB = BA = I n the n n identity matrix If A has an inverse, then we call A an invertible matrix, and if B is that inverse then we write B = A Note that invertible matrices are also called nonsingular and noninvertible matrices are called singular Some notes Note We have to require that both AB and BA are equal to the identity matrix since matrix multiplication isn t commutative!
Note Which matrices are invertible? A- SUMMER 0 LECTURE 5 6 Continuing our analogy with numbers, all numbers other than 0 have a multiplicative inverse: If a R and a 0, then a is the multiplicative inverse of a For matrices, the situation is much more complicated Indeed the matrix with all 0 s is not invertible, but there are many other matrices which are not invertible either We ll now see some examples of matrices which are and aren t invertible: Example The matrix Then A is invertible and Let s check this: And similarly Example The matrix AA = A = A = = 0 0 A A = 0 = 0 0 a b is not invertible Indeed for any matrix 0 0 c d 0 a b 0 0 c d = c d 0 0 we have that No matter what c and d are, this can never be the identity matrix since the, entry is always 0, but it would have to be to get I Now let s talk about the point of invertible matrices Suppose that I have a system of n linear equations with n variables: The augmented form of this system is a x + + a n x n = b a x + + a n x n = b a n x + + a nn x n = b n a a a n b a a a n b a n a n a nn b But this augmented system is shorthand for the following matrix equation:
A- SUMMER 0 LECTURE 5 7 which we can write as a a a n x b a a a n x b = a n } a n {{ a nn x n }}{{} b n }{{} A x b A x = b If A is invertible, then we can multiply both sides of this equation on the left by A : A (A x) = A b By associativity of matrix multiplication, A (A x) = (A A) x, so this implies that But A A = I, the identity matrix Hence (A A) x = A b I x = A b Since I x = x (multiplying by the identity matrix does nothing), it follows that x = A b That is the solution to this equation is A b Let s illustrate this with an example Example 5 Consider the following system of equations (from before): This is shorthand for the matrix equation: The inverse of the matrix is Let s check this: 0 0 7 5 0 0 x x = 0 7 5 0 0 7 5 0 7 5 = 0 0 0 0 0 0
And similarly, A- SUMMER 0 LECTURE 5 0 = 0 0 0 0 7 5 0 0 Therefore, let s multiply both sides of our matrix equation by this inverse: 0 x x = 7 5 As we just checked, multiplying out the two matrices on the left, this becomes: 0 0 0 0 x x = 0 0 0 0 But multiplying by the identity matrix doesn t change our vector, hence: x x = 0 = [ ] 0 0 0 This is the same solution as before This is a very powerful tool: If we know the inverse of a matrix A, then we can solve the system of equations A x = b by multiplying by the inverse Now your question should be How did you find the inverse of our matrix? The answer is elimination 5 How to find the inverse of a matrix If a matrix A is invertible, then we are trying to find a matrix B such that If a a a n a a a n A = a n a n a nn Then we are trying to find the entries of B such that AB = I b b b n b b b n B = b n b n b nn a a a n b b b n 0 0 a a a n b b b n = 0 0 a n a n a nn b n b n b nn 0 0 If we look at this multiplication in the column way, this means that a a a n b a a a n b = 0 a n a n a nn 0 and similarly for all of the other columns Therefore we are trying to solve the augmented systems: b n
A- SUMMER 0 LECTURE 5 9 a a a n a a a n 0 a n a n a nn 0 a a a n 0 a a a n a n a n a nn 0 a a a n 0 a a a n 0 a n a n a nn In fact we should solve these using Gauss-Jordan Elimination But we would be doing the same steps n times To save us time, we solve all of these equations in one fell swoop We use Gauss-Jordan elimination on the augmented system: a a a n 0 0 a a a n 0 0 a n a n a nn 0 0 It turns out (we ll prove this later) that when a matrix is invertible, its reduced row-echelon form is the identity matrix Hence after elimination, we get: For some c ij s Therefore this tells us that 0 0 c c c n 0 0 c c c n 0 0 c n c n c nn b b b n c c c n = and similarly for the other columns of B, hence the matrix c c c n c c c n B = c n c n c nn is the inverse of A 6 When does an inverse matrix exist? There are a number of equivalent criteria for this Here s the most important theorem so far (actually the only one that really matters so far): Theorem Let A be an n n matrix The following conditions are equivalent: () A is invertible () The reduced row-echelon form of A is the n n identity matrix () The system of equations A x = b has a unique solution for every vector b ()