Math 344 Lecture #12 2.7 Linear Systems Through a choice of bases S and T for finite dimensional vector spaces V (with dimension n) and W (with dimension m), a linear equation L(v) = w becomes the linear system Ax = b where A M m n (F) is the matrix representation of L in the bases S and T, x = [v] S F n, and b = [w] T F m. A linear system Ax = b has a solution if and only if b R(A), and if a solution exists, there are more than one when N (A) {}. A linear system is solved by row reducing the augmented matrix [A b] when b, or by row reducing A when b =. We will sometimes think of [A b] as a matrix A in what follows. 2.7.1 Elementary Matrices There are three basic row operations on a matrix A: (1) switching rows; (2) multiplying a row through by a nonzero scalar; and (3) adding a multiple of one row to another row. Type I. Swapping rows, symbolic written R i R j (where R i is the i th row of A). Type 2. Multiply a row through by a nonzero scalar α, symbolically written R i αr i. Type 3. Add a scalar multiple of one row to another row, symbolically written R i R i + αr j. Notice here that the j th row remains unchanged. Remark 2.7.1. Each elementary matrix is obtained by performing one of the three basic row operations the identity matrix. Proposition 2.7.2. Each elementary matrix is invertible. Remark 2.7.3. Solving Ax = b is done by applying elementary matrices as left multiplication on both sides. Since each elementary matrix is invertible, multiplication on the left by an elementary matrix is a change of basis for the range of A. One readily confirms that for an elementary matrix any solution x of EAx = Eb is also a solution of Ax = b and vice versa. Definition 2.7.6. A matrix B is row equivalent to a matrix A (both of the same size) if there is a finite collection of elementary matrices E 1, E 2,..., E n such that B = E 1 E 2 E n A. Theorem 2.7.7. Row equivalence is an equivalence relation on M m n (F). It is HW (Exercise 2.4) to prove Theorem 2.7.7. 2.7.2 Row Echelon Form In solving a linear system Ax = b by row reduction on the augmented matrix [A b], we need to know when we have arrived at a row equivalent matrix from which we can read off the solutions, if any, or determine the system has no solution. Definition 2.7.9. A matrix A is in row echelon form (REF) if (i) the first nonzero entry, called the leading entry, of each nonzero row is always strictly to the right of the leading entry of the row above it, and
(ii) all nonzero rows are above any zero rows. A matrix A is in reduced row echelon form (RREF) if (i) it is in REF, (ii) the leading entry of each nonzero row is 1, and (iii) the leading entry in each nonzero row is the only nonzero entry in its column. Remark 2.7.11. REF is a canonical form of a matrix from which a linear system can be solved by back substitution. Example. Here is a row reduction to an REF for an augmented matrix for Ax = b. 2 2 3 5 R 2 R 2 + 2R 1 1 1 3 2 3 R 3 R 3 R 1 1 3 2 1 3 2 1 3 2 R 3 R 3 R 2 1 3 2 R 4 R 4 R 2 1 3 2 Proposition 2.7.13. For each matrix A there exists an RREF matrix B such that A and B are row equivalent. Moreover, if A is a square matrix and B is an REF matrix that is row equivalent to A, then B is upper triangular. Finally, if B is a square upper triangular matrix whose diagonal entries are nonzero, then B is row equivalent to the identity matrix. It is HW (Exercise 2.39) to prove this. Theorem 2.7.14. A square matrix is invertible if and only if it is row equivalent to the identity matrix. Application 2.7.15. Row reduction of a matrix A to REF sometimes produces two matrices L and U for which A = LU. This is called the LU-factorization of A. If the matrix U is a REF for A obtained by only using the Type III elementary matrix R i R i + αr j where i > j, then A has an LU-factorization. To get L from the row reduction of A to U, we write E n E 1 A = U where E 1 is the first Type III elementary matrix applied, and E n is the last Type III elementary matrix applied, and compute L = (E n E 1 ) 1 = E1 1 En 1.
Since the inverse of a Type III elementary matrix is easy to find, the matrix L is easy to find. The matrix L is lower triangular because it is a product of lower triangular matrices. Solving Ax = b by A = LU proceeds in two steps: (1) solve Ly = b via back substitution, and (2) solve for x in Ux = y. Here Ax = LUx = Ly = b. Example (Continued). In row reducing A = 2 2 3 5 1 1 3 2 to B = 1 3 1 3 we only used the Type III row operation R i R i + alphar j for i > j. Thus this matrix has an LU-factorization where U = B and 1 L = 2 1 1 1 1 1 1 which is the product of the inverses of four elementary matrices 1 1 1 1 E1 1 E2 1 E3 1 E4 1 = 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 corresponding the row operations R 2 R 2 + 2R 1, R 3 R 3 R 1, R 3 R 3 R 2, and R 4 R 4 R 2. Do you see what is really happening here? The negative of the scalar multiples in the the Type III row operations appear in the entry of L that was zeroed out during row reduction. Corollary 2.7.16. If a square matrix A is invertible, then the RREF of the augmented matrix [A I] is [I A 1 ]. 2.7.3 Basic and Free Variables There is much being reviewed and stated in this subsection. Rather than detail all of these items, we proceed by way of example. Example (Continued). We have row reduced A = 2 2 3 5 1 1 3 2 to B = 1 3. 1 3
The basic variables for B are associated to the columns with leading entries: these are columns 1 and 3. The free variables for B are associated to the columns without leading entries: these are columns 2 and 4. Since N (A) = N (B), we find N (A) by finding N (B). For x = [x 1, x 2, x 3, x 4 ] T, solutions of Bx = are given by the equations x 1 + x 2 + 2x 3 x 4 =, x 3 + 3x 4 =.. The second equations says that x 3 = 3x 4, which upon back substitution in the first equation gives x 1 + x 2 7x 4 =. Thus a vector x satisfying Bx = is of the form x 1 x 2 + 7x 4 1 x 2 x 3 = x 2 3x 4 = x 1 2 + x 4 x 4 x 4 You should check that each of the two vectors here satisfies Ax =. (They do.) We have found N (B) and hence N (A), and in finding this we have found basis for it. What is the nullity of A? It is 2 which is precisely the number of free variables. What is the rank of A? By the Rank-Nullity Theorem it is 4 2 = 2, which is the number of basic variables. The vector b = [1,, 3, 2] T is in the range of A because row reduction of [A b] produces a solution: 2 2 3 5 1 1 3 2 3 1 3 2. 1 3 2 Putting the last matrix into equation form gives x 1 + x 2 + 2x 3 x 4 = 1, x 3 + 3x 4 = 2.. Solving the second equation for the basic variable gives x 3 = 2 3x 4, and back substitution of this in the first equation and solving for the basic variable gives x 1 = 1 x 2 + 7x 4. Thus solutions of Ax = b are x 1 1 1 x 2 x 3 = 2 + x 1 2 + x 4 x 4 7 3 1 7 3 1..
You should check that the first vector satisfies Ax = b. (It does.) Notice that the linear combination of the other vectors is precisely the kernel of A. If we label the first vector as x, then the set of solutions of Ax = b is precisely the coset x + N (A). Is the range of B the same as the range of A? No, the columns of B have zeroes in the last two entries, while the some columns of A are nonzero in the last two entries. Can we find R(A) from B? Yes, the columns of A associated with the columns of B having leading entries form a basis for R(A). Thus, 1 2 R(A) = span 2 1, 3 3. 1