Solving Systems of Equations Row Reduction

Transcription

1 Solving Systems of Equations Row Reduction November 19, 2008 Though it has not been a primary topic of interest for us, the task of solving a system of linear equations has come up several times. For example, if we want to show that a collection of vectors {v 1,v 2,...,v k } in R n is linearly dependent/independent, then we need to understand the solutions (c 1, c 2,...,c k ) R k of the vector equation k c j v j = 0. (1) j=1 We need to keep in mind that in this equation v j = (v 1j, v 2j,...,v nj ) is known for each j. Thus, rewriting (1) as a system, we find a system of n equations in k unknowns: v 11 c 1 + v 12 c v 1k c k = 0 v 21 c 1 + v 22 c v 2k c k = 0 (2). v n1 c 1 + v n2 c v nk c k = 0. Another instance in which we have encountered systems of linear equations is when we wanted to express a given vector w R n with respect to a given basis, v 1,...,v k. Again we seek the coefficients c 1,..., c k R of a linear combination: k c j v j = w. (3) j=1 1

2 Upon converting to a system, the left side is the same as before: v 11 c 1 + = w 1. v n1 c 1 + = w n. (4) When we think of these questions in terms of systems of linear equations, it is convenient to change the names of the coefficients (the v ij s and the w j s) and the unknowns c 1,...,c k. First of all, let us denote the unknowns by x = (x 1,...,x k ). Next we will call the coefficients of the linear terms a ij and the constant (affine) terms b i, so that, after renaming, systems of the forms (2) and (4) become a 11 x 1 + = b 1. (5) a n1 x 1 + = b n. Finally, making the coefficient vectors (a i1,..., a ik ) the rows of a matrix A and thinking of the unknown x and the constant terms as column vectors, we can write (5) in the compact form Ax = b. (6) 1. Notice that (2) is simply the special case of (4) (or (3)) in which the constant terms are zero. This special case (b = 0) is called the homogeneous case. If b 0, we say the system (6) is nonhomogeneous. (a) Show that the solution set of a homogeneous system is a subspace. (b) Show that the solution set of a nonhomogeneous system is an affine subspace. (Hint: Consider the case where the system has no solution separately.) 2. (This exercise really belongs in the last set of notes, but it was left out.) We know that the determinant of a 2 2 matrix gives the area (up to a sign) of the paralellogram determined by the column vectors. We know also that if v and w are the columns, then the parallelogram 2

3 is {av + bw : 0 a, b 1}. Give a similar interpretation for the determinant of a 3 3 matrix. Be sure to precisely describe any sets to which you refer. As the compact form suggests, one wishes to solve (6) by simply dividing by A, i.e., multiplying both sides of (6) by A 1, so that x = A 1 b. (7) Such a strategy is complicated by the fact that there may be no such thing as A What must be true about the (a) size, (b) determinant, and (c) kernel of the associated linear transformation of the coefficient matrix A in order for the solution (7) to make sense? In view of the last exercise, we see that (7) represents the solution in a rather special case. In particular, this is a case where there is exactly one solution. In an effort to handle all the other cases, it is perhaps a good idea to consider first simply what is likely to happen and what is possible. In all cases, let us denote by L = R k R n the linear transformation associated with A. First of all if k < n, then dim Im(L) < n. This means that Im(L) is a relatively thin set in R n, and (if we know nothing else) it is unlikely that b Im(L) at all. There is most likely no solution. 4. Given an example in which dim Im(L) < k < n and one in which dim Im(L) = k < n. (Draw pictures.) If there is a solution (or solutions) the question arises: How do you find it (or them)? Another question, which is not really obvious algebraically from (6), but is very obvious geometrically once you draw a picture is: If there is no solution, what is the vector x which is closest to being a solution? We 3

4 will come back to this second question. Perhaps the easiest answer to the first question (how to find solutions) is just algebraically manipulate the equations to solve for some of the variables in terms of the others. Let s begin with an example in which k < n. 3x 1 + 2x 2 = 5 2x 1 + 3x 2 = 7 x 1 + x 2 = 6. There s no reason not to do this in an organized way. First, let us get a coefficient of 1 in the a 11 position. Here we can do that by dividing the first equation by 3. We can also accomplish our objective by putting the last equation first: x 1 + x 2 = 6 3x 1 + 2x 2 = 5 2x 1 + 3x 2 = 7. In general, you might need to do a combination of these two elementary row operations. Next, let us use that monic coefficient to eliminate the x 1 terms from the other equations. Multiplying the first equation by 2 and subtracting from the last equation, we get x 1 + x 2 = 6 3x 1 + 2x 2 = 5 x 2 = 5. Multiplying the first equation by 3 and subtracting from the second equation: x 1 + x 2 = 6 x 2 = 13 x 2 = 5. This is clearly inconsistent, i.e., there is no solution, but let us go one step further and get a monic coefficient in the second equation x 1 + x 2 = 6 x 2 = 13 x 2 = 5. 4

5 We ll come back to this unsolvable system. Let s change the constant coefficients so that we know we have a solution x 1 + x 2 = 5 3x 1 + 2x 2 = 12 2x 1 + 3x 2 = 13. (How do you think I come up with b = (5, 12, 13) T?) Let us also note that there is no reason to write down the x s every time. Let us represent the system in single matrix form: (The long vertical line is a concession to the equals signs.) Eliminating the x 1 coefficients in the second two equations we now have This is called Gaussian elimination and it is a convenient computational tool: x 2 = 3, x = 5 x 1 = 2. We can also go one step further using the first nonzero entry (pivot) in the last nonzero row to eliminate the coefficients avove it: In each case, we have boxed the pivots. Let us give one more example in which k < n. 5x 1 + 3x 2 = 11 15x 1 + 9x 2 = 33 20x x 2 = 44. A solution via Gaussian elimination/row reduction: 1 3/5 11/ /5 11/

6 One pivot. We conclude 5x 1 + 3x 2 = 11 or x 1 = 11/5 3x 2 /5. We can take x 2 = t arbitrary in R to obtain the affine line of solutions {(11/5, 0) + t( 3/5, 1) : t R}. (8) 5. Show that the solution set (8) for the last example may also be expressed as {(11/5, 0) + t( 3, 5) : t R}. 6. Draw pictures to illustrate the three systems of equations described above as examples. 7. Describe the solution sets for the following systems of equations { 3x 1 x 2 = 4 2x x 2 = 1 x 1 2x 2 + x 3 + 3x 4 x 5 = 1 3x 1 + 6x 2 4x 3 9x 4 + 3x 5 = 1 x 1 + 2x 2 2x 3 4x 4 3x 5 = 3 x 1 2x 2 + 2x 3 + 2x 4 5x 5 = 1 (There are more systems to practice in Carlen & Carvalho pages ; The number of linearly independent rows does not change after doing an elementary row operation. 9. The number of linearly independent columns does not change after doing an elementary row operation. 10. The number of linearly independent rows is the number of pivots. 11. The number of linearly independent columns is the number of pivots. 12. Now can you prove the dimension theorem? Hint: How does the dimension of the kernel relate to the pivots? 13. The row rank of a matrix is defined to be the number of linearly independent rows, i.e., the dimension of the vector space spanned by the rows. The column rank of a matrix is defined to be the number of 6

7 linearly independent columns, i.e., the dimension of the image of the associated linear transformation. Show that the row rank and the column rank of a matrix are the same number. Given a matrix A, we denote this common number by rank(a). Nonhomogeneous Systems The row reduction technique we have outlined above works for both homogeneous and nonhomogeneous systems Ax = b. Of course, the zero vector (in R n ) is not in the solution set S = {x R n : Ax = b} unless b = 0. For a nonhomogeneous system of equations therefore, the solution set is not a subspace. However, it still has a structure which is worth noting. It is an affine subspace which is confusing terminology since it is not a subspace, but if you look carefully at the following definition, you can see where the terminology comes from. Definition A subset S of a vector space is called an affine subspace if there is a fixed vector x 0 and a subspace Σ of the same vector space such that S = {x + x 0 : x Σ}. So you see, an affine subspace is just an affine shift of a subspace. In order to see that the solution set S = {x : Ax = b} is an affine subspace, let x 0 be any solution of the equation. Now, if y is any other solution of the same equation, then consider x = y x 0 : Ax = A(y x 0 ) = Ay Ax 0 = b b = 0 R m. This means that y = x + x 0 where x is some element of the solution set of the associated homogeneous equation Ax = 0. This notion of an associated 7

8 homogeneous equation is important in many areas of mathematics involving linearity. It makes the following algorithm (which is an alternative to straight row reduction) for solving a nonhomogeneous problem work: Alternative Algorithm for Solving (nonhomogeneous) Linear Systems Ax = b Find a particular solution x 0. Solve the associated homogeneous problem in general to obtain the solution space Σ = {x : Ax = 0}. The solution set for the nonhomogeneous problem is S = {x + x 0 : x Σ}. We have ignored the possibility that there is no solution x 0 for the nonhomogeneous problem, and this can indeed happen. Here are some problems to practice on, and some of them should be nonsolvable you ll see that the inconsistency of the equations is also detected by row reduction. In the next set of notes, we will discuss how to do something anyway, even when you can t solve such a system. 14. Describe the solution sets for the following systems of equations { x 1 + 2x 2 = 5 2x 1 + x 2 = 5 2x 1 3x 2 x 4 = 2 x 2 + x 3 + x 4 = 2 x 1 + 2x 2 + x 3 + x 4 = 1 3x 1 2x 2 + x 4 = Describe the solution sets for the following systems of equations x 1 + x 2 + 2x 3 = 1 x 1 + 2x 2 + 4x 3 = 1 x 1 + x 3 = 2 2x 1 + 5x 2 + x 3 = 1 x 2 + x 3 = 3 x 1 + x 2 + x 3 = 1 8

9 16. Describe the solution sets for the following systems of equations x y = 3 x + y + 2z = 2 2x + y = 1 x + 5y + 4z = 4 x + 5y = 4 2x + 4y + 6z = 1 9