XII. LINEAR ALGEBRA: SOLVING SYSTEMS OF EQUATIONS Tody we re going to tlk bout solving systems of liner equtions. These re problems tht give couple of equtions with couple of unknowns, like: 6 2 3 7 4 How do we solve these things, especilly when they get complicted? How do we know when system hs solution, nd when is it unique? Provided tht the system is firly simple, it might be esiest to solve using successive substitution. Given system tht looks like this: 2 23 3 32 33 For simplicity, most of things I show here will be 3 3 systems, but everything works just s well with more vribles.) You pick ny eqution nd ny vrible, nd solve in terms of tht vrible in terms of the constnts nd the other vribles. Let s sy we pick eqution one nd : ) Then we substitute this vlue of bck into the other two equtions, 2 ) 23 3 ) 32 33 And then we hve two liner equtions in two unknowns: 2 2 2 x 2 2 23 3 32 2 3 x 3 2 33 Once gin, we pick one eqution nd solve it in terms of prticulr vrible: Fll 2007 mth clss notes, pge 90
) b 2 2 23 2 2 After substituting into the remining eqution, we get single expression for the lst of the vribles: 3 2 3 32 ) 2 33 3 ) 23 32 ) 23 2 ) 2 Knowing wht is, we cn find the vlue of nd then. However, this is tiring process, especilly when you strt off with bunch of equtions, nd there re no pprent simple substitutions. It s going to be esier to do this in mtrix form. Let A be the mtrix of coefficients on the system of equtions, nd b the constnts. We cn write this system of equtions s: 2 23 b 3 32 33 2 23 3 32 33 A x And the question is how to solve this system for the vector x of unknowns. There re three wys, more or less. In the first method, we essentilly use Gussin elimintion in mtrix form. First, we write out the ugmented mtrix: 2 23 3 32 33 This is shorthnd for sying the vector x times the left hnd side of the mtrix will equl the right hnd side of the mtrix. Now, if the left-hnd side equls the identity mtrix, 0 0 0 0 0 0 c c 2 c 3 Fll 2007 mth clss notes, pge 9
wht we hve is tht the vector x times the identity mtrix which equls x itself) equls the right hnd side, so x c. Whenever the left-hnd side equls the identity mtrix, the right-hnd side is solution for x. Given the ugmented mtrix corresponding to the system of liner equtions, our mission should we choose to ccept) is to get the left-hnd side into the form of the identity mtrix, using only these three elementry row opertions:. interchnging two rows of the mtrix; 2. dd or subtrct) multiple of one row to nother row; nd 3. multiply ech element in row by the sme nonzero number. We perform these opertions to every element of the row, both on the left hnd side. With the prticulr mtrix given bove, these re wht the permissible elementry row opertions look like: 2 23 3 32 33 2 3 32 23 33 3 32 33 2 23 3 32 33 My strtegy for solving these is usully first to rrnge the equtions in wy tht mkes sense with experience, you ll figure out wht s esiest). Then I divide the first row through by the constnt : 2 2 23 3 32 33 Then I subtrct 2 times the first row off of the second; 3 times the first row off from the third: Fll 2007 mth clss notes, pge 92
0 2 23 2 0 32 3 33 3 2 3 I do similr thing for the second row now, dividing through by the coefficient on the term in the second row: 0 23 2 ) 2 2 ) 0 32 3 33 3 2b ) 2 ) 3 In order to get zeros in the second plces of the first nd third rows, I multiply the second row by the pproprite constnt nd subtrct off: 0 0 23 2 0 0 33 3 32 3 2 ) 23 2 ) 2 2 ) ) 2 2 ) ) 23 2 ) 2 2 ) ) 2b ) 2 2 ) 2b ) 2 2 ) ) 2b ) 2 2 ) 3 32 3 And so on. Though this looks relly nsty when presented this wy, it turns out usully to work pretty well. Let s try n exmple: 7 2 3 4 6 7 8 9 7 2 3 4 6 7 8 9 2 3 7 4 6 7 8 9 The first step is to divide the first row by the coefficient in the top left in this cse, tht turns out to be negtive one. Then we subtrct the top row time four from the second row, nd the top row times seven from the bottom row: 2 3 7 4 6 7 8 9 2 3 7 4 6 7 8 9 2 3 7 0 8 23 0 22 30 38 Then we divide the second row by in order to get leding, nd dd two times the second row to the first row, nd subtrct 22 times the second row from the lst: Fll 2007 mth clss notes, pge 93
2 3 7 0 8 23 0 22 30 38 2 3 7 8 23 0 0 22 30 38 3 0 8 0 6 0 0 4 23 2 Finlly, we divide the lst row by 6, nd subtrct the pproprite bout off from the first nd second rows: 3 0 8 0 6 0 0 4 23 2 3 0 8 0 0 0 4 23 2 0 0 3 0 0 0 0 2 The right-hnd side of the mtrix now tells us wht the vector x should equl. We should now go bck nd verify by multiplying the originl problem) tht this works. Sometimes, you might try to work one of these systems nd end up with very funny contrdictory) result in the end, or n entire row might turn into zeros which leves you with no chnce of turning its digonl element into one). Most likely, this is sign tht you hve mde n rithmetic error but if you go bck nd check your steps nd this is still the outcome, then you hve encountered system without solution or with infinitely mny solutions. I ll tlk more bout these lter. The second wy of solving system of equtions is so simple people often overlook it. Suppose we hve the system: 2 23 3 32 33 b A x Provided tht A is n invertible n n mtrix, we cn solve this by premultiplying both sides by A : x A b And then performing the pproprite mtrix multipliction. exmple gin: Let s look t tht 7 2 3 4 6 7 8 9 2 3 4 6 7 8 9 7 Using the formul for mtrix inversion, we find this: Fll 2007 mth clss notes, pge 94
x A b det A A A 2 A 3 7 A 2 A 22 A 32 3 6 3 7 2 6 30 8 6 A A 23 A 33 3 20 4 Pretty nifty tht we cn do it two wys nd get the sme solution, huh? Of course, this method works only when the mtrix is invertible; lter, I ll show how being singulr corresponds to system with mny or no solutions. If we look t the mtrix inversion method, we observe n interesting pttern rising. In the three by three cse, wht we hve is tht: det A b A b A b A 2 2 3 3) det A b A b A b A 2 2 22 3 32 ) det A A A 23 A 33 ) Wht does this look like? Well, these ber remrkble resemblnce to the formul for determinnts. A A 2 A 3 23 32 33 A A 2 A 3 2 23 3 33 A A 2 A 3 2 3 32 So in fct ll we hve to do to solve this system of equtions much esier thn inverting mtrix) is to sy tht x i equls the determinnt of the mtrix formed by replcing the i-th column of A with the vector b, divided by the determinnt of A. This is known s Crmer s Rule. Theorem: Let A be nonsingulr n n mtrix. Then the system of equtions: Fll 2007 mth clss notes, pge 9
b b 4 hs the unique solution tht: n 2 2n n n2 nn x n Ax x i det B i det A where B i is the mtrix formed by replcing the i-th column of A with the vector b. Provided tht you cn remember this formul, this is usully the most efficient wy to solve system of equtions. Recll tht if we imgine mtrix s bunch of vectors, the determinnt mesures the spn of these vectors. This re is lrgest when the vectors re more t odds with one nother, the closer they re to being orthogonl, the less they hve in common. The first column of A is where does ll of its explining of the outcome: n x n 2 2n x n If is very lrge reltive to the other vribles), then the first column of A should be very similr in direction to the outcome b, right? Only the mgnitudes might differ. In order to test how lrge this effect is, we tke out this first column nd stick in b insted. If it s true tht hs the most effect on the outcome, then this substitution should not chnge the shpe of the re spnned by the mtrix much, only its size. Another wy of thinking of this is tht if vribles other thn hd reltively little effect on the outcome of b, then b would be firly orthogonl to the vectors in A other thn. This would men tht the re spnned by b nd these other vectors would be reltively lrge. It might be useful to mke up some numbers for two-by-two mtrix A, nd to represent its determinnt grphiclly. Then mke up vector for x, nd see wht the implied vlues for b re. Drw the re spnned by B nd B 2. Does it seem tht the reltive size of these res corresponds to the reltive sizes of the two x vribles? Not ll systems of equtions hve unique solution. Some hve infinitely mny, nd some hve none. Here is one simple exmple: Fll 2007 mth clss notes, pge 96
3 2 2 6 4 4 In some sense, the second eqution gives us no more informtion thn the first, since it simple hs ll the constnts doubled. This system cn be fulfilled by lot of points, ll lying long line. In contrst, the system: 3 2 2 6 2 2 hs no solution. Effectively, we hve been given two contrdictory pieces of informtion: by trnsitivity, they imply tht 3 6, which is bsurd. When we hve system of n equtions in n unknowns, the lck of unique solution hppens if nd only if two or more) equtions suggest tht the sme reltionship between vribles produces the sme outcome, or tht they produce different outcomes. In short, the lck of unique solution hppens if nd only if two equtions suggest the sme reltionship between vribles. Here re some exmples of systems of equtions tht suggest the sme reltionship, lso represented in mtrix form: 3 2 2 6 4 4 3 2 2 6 2 2 3 6) * 2 2 4 4 * ) 3 6) * 2 2 2 2 * ) * ) * ) 4 2 2 6 4 3 3 3 4 ) 4 2 ) 2 6 4 3 3* 3 4* ) * In ech cse, one of the following is two: two rows re the sme, one rows is multiple of nother, or one row is liner combintion of two others. If we look t the determinnts of the mtrices on the right hnd side, we ll see something else these equtions hve in common other thn the lck of unique solution): ll these mtrices re singulr. So here s the lw for squre mtrices: Unique solution Full rnk Liner independence Nonsingulr Invertible Fll 2007 mth clss notes, pge 97
I think tht s it. If there re ny other desirble properties of squre mtrices, they re most likely lso equivlent. The old principle bout being ble to solve n equtions in n unknowns works if nd only if these re linerly independent equtions. Wht bout when you hve k equtions in n unknowns? Well, s you probbly knew before, k < n generlly mens tht there is n infinite number of solutions, wheres k > n generlly implies no solution t ll. Systems of inequlities Intersection of lines > intersection of hlfspces Fll 2007 mth clss notes, pge 98