Lecture 9: Elementary Matrices Review of Row Reduced Echelon Form Consider the matrix A and the vector b defined as follows: 1 2 1 A b 3 8 5 A common technique to solve linear equations of the form Ax b is to use Gaussian elimination The common method to do this is to write out the matrix: [ ] 1 2 1 [A, b] 3 8 5 and then proceed to rewrite this matrix in row reduced echelon form For convenience, we cover this here with this example First, we wish to eliminate the 3 in Row 2, so to do that, we perform the row operation R 2 3R 1 R 2, this gives us: [ 1 2 1 3 8 5 ] [ R 2 3R 1 R 2 1 2 1 0 2 2 We can also multiply a row by a constant, so for example 5 R 2 R 2 reduces this further: [ ] 1 2 1 R 2 3R 1 R 2 1 2 1 5 R 2 R 2 1 2 1 3 8 5 0 2 2 0 1 1 Remember we can also work backwards too, since we want the right hand side to be 1 s as pivot columns So, if we finally do R 1 2R 2 R 2, we get: [ ] [ ] 1 2 1 R 2 3R 1 R 2 1 2 1 5 R 2 R 2 1 2 1 R 1 2R 2 R 2 1 0 1 3 8 5 0 2 2 0 1 1 0 1 1 This matrix is now in row reduced echelon form Then, from this algorithm, we know that x [ 1, 1] T and our solution is complete This should have been introduced to you already in the Linear Algebra course Row Reduced Echelon Form in MATLAB As we ve seen, solving the equation Ax b in MATLAB is straightforward if we use the command A\b However, we can also compute row reduced echelon form, using the function rref() The function rref() The function does as one might guess It takes a single matrix input and does row reduction on it, like we did above by hand So, for example, we put in our matrix A and b into MATLAB: 1 ]
>> A [ 1, 2 ; 3, 8 ] ; b [ 1 ; 5 ] ; Then, we can write the appended matrix by typing >> Ab [A, b ] Ab 1 2 1 3 8 5 Then, we use the function rref() on Ab, which simplifies it to reduced echelon form: >> r r e f (Ab) ans 1 0 1 0 1 1 We see this is the exact answer we got above by hand, where we can now see the solution x [ 1, 1] T As a second example, consider the matrix >> M magic ( 3 ) ; b [ 1 ; 5 ; 1 0 ] ; I can combine these inside the function if I want, without having to make a new variable So, I type: >> r r e f ( [M, b ] ) ans 10000 0 0 00639 0 10000 0 11056 0 0 10000 01028 and I get my answer Note, row reduction works on any size matrix The function rref() works on any size matrix, although you won t get perfect pivot columns in some cases Elementary Matrices So far, we ve covered a method of solving linear systems using row reduction However, the algorithm doesn t make it clear why it works This lecture will give the explanation, but we must first introduce the concept of elementary matrices These are fairly special matrices and there are three types For simplicity, we will only consider square elementary matrices, so you can see how it relates towards matrix inverses at the end 2
Row Switching Elementary Matrix Consider matrix A again, given at the beginning of the lecture Suppose we wish to switch the first and second row This would usually be denoted R 1 R 2 However, this can be done by multiplying matrix A by an elementary matrix that looks like: [ ] 0 1 E s 1 0 Testing this, you see: E s A 0 1 1 2 1 0 3 8 [ ] 3 8 1 2 This is something commonly done, particularly when re-arranging pivot columns, for example In general, this has the form: 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 E s 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 Where the 1 s that are not on the diagonal are in the locations where rows are being exchanged So, to exchange row i and j, we put a 1 at the (i, j) location on row i, with a 0 at (i, i) And, on row j, we have a 1 at location (j, i) and 0 at (j, j) For example, if we want to swap row 1 and 3 in a 3 3 matrix A, then our elementary matrix would be 0 0 1 E s 0 1 0 1 0 0 Row Multiplication The next row reduction technique used is row multiplication, denoted by cr j R j, where c is the amount you re multiplying the row Let s consider matrix A again and consider multiplying the first row by 2 This would be 2R 1 R 1, but it s also given by the elementary matrix [ ] 2 0 E m 0 1 Testing this, we see: E m A 2 0 1 2 0 1 3 8 3 [ ] 2 4 3 8
and we see the first row was multiplied by 2 The general form of this should be easier to see than the first one If we want to multiply row i by a constant c, then we can premultiply by the matrix E m, where 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 E m 0 0 0 c 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 where c is at the (i, i) position in matrix E m, and the rest looks like the identity matrix For example, if we want to multiply row 4 by 4 in a 5 5 matrix, we want to set: 1 0 0 0 0 0 1 0 0 0 E m 0 0 1 0 0 0 0 0 c 0 0 0 0 0 1 Row Addition Our final elementary matrix is the one most commonly used in doing row reduction This is what allows us to make pivot columns by eliminating columns We use matrix A as an example again Suppose we want to add -3 to row 2, as before, which is given by R 2 3R 1 R 2 This can be accomplished with the matrix [ ] 1 0 E a 3 1 We compute this on matrix A and we get: 1 0 1 2 E a A 3 1 3 8 [ ] 1 2 0 2 4
which is exactly what we expect In general, the formula is: 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 E a 0 0 m 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 where m at the (i, j) location is the amount we want to add to row j from row i As another example, suppose we want to add 2 of row 2 to row 3 in a 4 4 matrix Then, our elementary matrix would look like: 1 0 0 0 E m 0 1 0 0 0 2 1 0 0 0 0 1 Elementary Matrices and Row Reduction Coming back to row reduction, we d like to tie these two concepts together In particular, the idea is as follows: perform a series of row reductions and keep track of these row reductions as matrices Apply these operations to both A and b, and at the end we ll have our result In other words, we re actually doing this: Ax b E 1 Ax E 1 b E 2 E 1 Ax E 2 E 1 b E n E n 1 E 2 E 1 Ax E n E n 1 E 2 E 1 b But, after the nth step, we ve reduced the left hand side to the identity, which implies x E n E n 1 E 2 E 1 b which ends up being the vector of values on the right hand side of our appended matrix In fact, this paradigm tells us more, assuming A is nonsingular Since inverses are unique, and x A 1 b, this tells us that A 1 E n E n 1 E 2 E 1 5
So, by performing a series of elementary row operations (and keeping track of the matrices), we essentially find the inverse of matrix A In fact, you can also check that you can write these same elementary matrices to write matrix A, by noting: (A 1 ) 1 A (E n E n 1 E 2 E 1 ) 1 E1 1 E2 1 En 1E n 1 In fact, if you go on to learn more about abstract algebra, you ll find that the elementary matrices generate what is called the general linear group of invertible matrices The idea being that any invertible matrix can be described as a finite product of elementary matrices As an example, let s go back to solving A Remember our first operation was R 2 3R 1 R 2, so this can be done by pre-multiplying by E 1, which we do to both A and b: 1 0 1 2 1 2 1 0 1 1 E 1 A, E 3 1 3 8 0 2 1 b 3 1 5 2 Next, we want to multiply row two by 5 So, we write the elementary matrix for 5R 2 R 2, and then premultiply again: [ ] [ ] 1 0 1 2 1 2 1 0 1 1 E 2 E 1 A, E 0 5 0 2 0 1 2 E 1 b 0 5 2 1 And our final step is to go backwards to eliminate columns So, we do R 1 2R 2 R 1 as an elementary matrix, which is given by the matrix E 3 : [ ] [ ] 1 2 1 2 1 0 1 2 1 1 E 3 E 2 E 1 A, E 0 1 0 1 0 1 3 E 2 E 1 b 0 0 1 1 By defining A 1 E 3 E 2 E 1, we see that we just have the solution x A 1 b as expected In fact, you could multiply these matrices out to get the inverse, if you wanted: 1 2 1 0 1 0 4 1 A 1 E 3 E 2 E 1 0 1 0 5 3 1 15 5 Elementary Matrices in MATLAB Unfortunately, MATLAB doesn t provide an easy function to compute elementary matrices You could write one, though Maybe I ll make it part of the assignment or as an exercise However, we can outline how you could write an elementary matrix quickly For simplicity, we will only look at square, nonsingular matrices This could be generalized to more general matrices, but we should get a handle on the easier case first We ll start with the initial setup, and then what you d want to type in MATLAB to get what you want So, first, start with the identity These matrices are based on small modifications to the identity matrix, so type it out You can use: 6
>> E eye ( n ) for example, where n is the size of the matrix Then, we need to replace entries using MATLAB commands based on what type of row operation we want to do: Row Swapping This is probably one of the more complicated ones Remember we want a 1 in the (i, j) and (j, i) spot, but now a zero at the (i, i) and (j, j) spot The naive way to do it is one by one, so you could write out: >> E( i, j ) 1 ; E( j, i ) 1 ; E( i, i ) 0 ; E( j, j ) 0 This is perfectly acceptable and probably the method you ll use on your assignments However, remember a matrix is indexed by a pair of indices (it s row and column) or by a single index, which enumerates down the columns Thus, a location (i, j) is given by the single index (j 1)n + i Using this, we can replace elements with each other This can be doing by doing: >> E ( [ ( j 1)n+i ( j 1)n+j ( i 1)n+j ( i 1)n+i ] ) E ( [ ( j 1)n+j ( j 1)n+i ( i 1)n+i ( i 1)n+j ] ) ; I recommend sticking with the first way, but this is here if you want to use it Row Multiplication Row multiplication can also be set, although it s only a single command Remember if we want to multiply row i by constant c, then we just need to replace that part in the diagonal of our matrix E So, you would want to do: >> E( i, i ) c ; and that s it Row Addition Row addition is similar to multiplication, where we just need to modify a single index So, if we re doing the operation R i + cr j R i, then we want to replace the zero with a c at location (i, j) In MATLAB, this is done by: >> E( i, j ) c ; And that s all there is to it Your assignment will ask for you to construct elementary matrices and check that certain properties hold with them, particularly how they relate to a matrix inverse 7
Exercises Will be added 8