Handout for dequac of Solutions Chapter 04.07 SET ONE The solution to 7.999 4 3.999 Make a small change in the right hand side vector of the equations 7.998 4.00 3.999 4.000 3.999 Make a small change in the coefficient matri of the equations 7.999 4 3.998.00.00.00 0.00388 3.994
SET TWO The solution to 7 4 3 Make a small change in the right hand side vector of the equations. 7.00 4.00 3.00.999 Make a small change in the coefficient matri of the equations. 7 4 3.00.00.00.00 0.997.003
Chapter 04.09 dequac of Solutions fter reading this chapter, ou should be able to:. know the difference between ill-conditioned and well-conditioned sstems of equations,. define the norm of a matri, and 3. relate the norm of a matri and of its inverse to the ill or well conditioning of the matri, that is, how much trust can ou having in the solution of the matri. What do ou mean b ill-conditioned and well-conditioned sstem of equations? sstem of equations is considered to be well-conditioned if a small change in the coefficient matri or a small change in the right hand side results in a small change in the solution vector. sstem of equations is considered to be ill-conditioned if a small change in the coefficient matri or a small change in the right hand side results in a large change in the solution vector. Eample Is this sstem of equations well-conditioned? 4 3.999 7.999 Solution The solution to the above set of equations is Make a small change in the right hand side vector of the equations 04.09.
04.09. Chapter 04.09 7.998 4.00 3.999 gives 4.000 3.999 Make a small change in the coefficient matri of the equations 7.999 4 3.998.00.00.00 gives 0.00388 3.994 This last sstems of equation looks ill-conditioned because a small change in the coefficient matri or the right hand side resulted in a large change in the solution vector. Eample Is this sstem of equations well-conditioned? 7 4 3 Solution The solution to the above equations is Make a small change in the right hand side vector of the equations. 7.00 4.00 3 gives.00.999 Make a small change in the coefficient matri of the equations. 7 4 3.00.00.00.00 gives
dequac of Solution 04.09.3.003 0.997 This sstem of equation looks well conditioned because small changes in the coefficient matri or the right hand side resulted in small changes in the solution vector. So what if the sstem of equations is ill conditioned or well conditioned? Well, if a sstem of equations is ill-conditioned, we cannot trust the solution as much. Revisit the velocit problem, Eample 5. in Chapter 5. The values in the coefficient matri [ ] are squares of time, etc. For eample, if instead of a 5, ou used a 4.99, would ou want this small change to make a huge difference in the solution vector. If it did, would ou trust the solution? Later we will see how much (quantifiable terms) we can trust the solution in a sstem of equations. Ever invertible square matri has a condition number and coupled with the machine epsilon, we can quantif how man significant digits one can trust in the solution. To calculate the condition number of an invertible square matri, I need to know what the norm of a matri means. How is the norm of a matri defined? Just like the determinant, the norm of a matri is a simple unique scalar number. However, the norm is alwas positive and is defined for all matrices square or rectangular, and invertible or noninvertible square matrices. One of the popular definitions of a norm is the row sum norm (also called the uniformmatri norm). For a m n matri [], the row sum norm of [] is defined as ma n a ij < i< m j that is, find the sum of the absolute value of the elements of each row of the matri []. The maimum out of the m such values is the row sum norm of the matri []. Eample 3 Find the row sum norm of the following matri []. 0 7 0 3.099 6 5 5
04.09.4 Chapter 04.09 Solution ma < i< 3 a ij 3 j [( 0 + 7 + 0 ), ( 3 +.099 + 6 ), ( 5 + 5 )] [( 0 + 7 + 0),( 3+.099 + 6),( 5 + 5) ] [ ] ma + ma + ma 7,.099, 7. How is the norm related to the conditioning of the matri? Let us start answering this question using an eample. Go back to the ill-conditioned sstem of equations, 4 3.999 7.999 that gives the solution as Denoting the above set of equations as X C [ ][ ] [ ] X C 7.999 Making a small change in the right hand side, 4.00 3.999 7.998 gives 3.999 4.000 Denoting the above set of equations b [ ][ X '] [ C' ] right hand side vector is found b [ C] [ C' ] [ C] and the change in the solution vector is found b [ X ] [ X '] [ X ] then
dequac of Solution 04.09.5 and then 4.00 4 7.998 7.999 0.00 0.00 [ C ] 3.999 4.000 5.999 3.000 [ ] C 0.00 5.999 The relative change in the norm of the solution vector is 5.999 X.9995 The relative change in the norm of the right hand side vector is C 0.00 C 7.999 4.50 0 4 See the small relative change of.50 0 in the right hand side vector results in a large relative change in the solution vector as.9995. In fact, the ratio between the relative change in the norm of the solution vector and the relative change in the norm of the right hand side vector is / X.9995 4 C / C.50 0 3993 Let us now go back to the well-conditioned sstem of equations. 4 3 7 gives
04.09.6 Chapter 04.09 Denoting the sstem of equations b X C [ ][ ] [ ] X C 7 Making a small change in the right hand side vector 4.00 3 7.00 gives.999.00 Denoting the above set of equations b [ ][ X '] [ C' ] the change in the right hand side vector is then found b [ C] [ C' ] [ C] and the change in the solution vector is [ X ] [ X '] [ X ] then 4.00 4 [ C ] 7.00 7 0.00 0.00 and.999 [ ].00 0.00 0.00 then C 0.00 0.00 The relative change in the norm of solution vector is
dequac of Solution 04.09.7 X 0.00 4 5 0 The relative change in the norm of the right hand side vector is C 0.00 C 7 4.49 0 See the small relative change the right hand side vector of.49 4 small relative change in the solution vector of 5 0. 4 0 results in the In fact, the ratio between the relative change in the norm of the solution vector and the relative change in the norm of the right hand side vector is / X 5 0 4 C / C 49 0 4. 3.5 What are some of the properties of norms?. For a matri [], 0. For a matri [] and a scalar k, k k 3. For two matrices [] and [B] of same order, + B + B 4. For two matrices [] and [B] that can be multiplied as [ ][ B], B B Is there a general relationship that eists between X / X and C / C or between X / X and /? If so, it could help us identif well-conditioned and ill conditioned sstem of equations. If there is such a relationship, will it help us quantif the conditioning of the matri? That is, will it tell us how man significant digits we could trust in the solution of a sstem of simultaneous linear equations? There is a relationship that eists between
04.09.8 Chapter 04.09 C and X C and between and X These relationships are X + and X < C C The above two inequalities show that the relative change in the norm of the right hand side vector or the coefficient matri can be amplified b as much as. This number is called the condition number of the matri and coupled with the machine epsilon, we can quantif the accurac of the solution of [ ][ X ] [ C]. Prove for [ ][ X ] [ C] that. X + Proof Let [ ][ X ] [ C] () Then if [] is changed to [ '], the [X ] will change to [ X '], such that [ '][ X '] [ C] () From Equations () and (), [ ][ X ] [ ' ][ X '] Denoting change in [] and [X ] matrices as [ ] and [ X ], respectivel [ ] [ ' ] [ ] X X ' X [ ] [ ] [ ]
dequac of Solution 04.09.9 then [ ][ X ] ([ ] + [ ] )([ X ] + [ ]) Epanding the above epression [ ][ X ] [ ][ X ] + [ ][ ] + [ ][ X ] + [ ][ ] [ 0] [ ][ ] + [ ] ([ X ] + [ ]) [ ][ ] [ ] ([ X ] + [ ]) [ ] [ ] [ ] ([ X ] + [ ]) ppling the theorem of norms, that the norm of multiplied matrices is less than the multiplication of the individual norms of the matrices, < X + Multipling both sides b < X + < X + How do I use the above theorems to find how man significant digits are correct in m solution vector? The relative error in a solution vector is Cond () relative error in right hand side. The possible relative error in the solution vector is Cond () mach Hence Cond () mach should give us the number of significant digits, m that are at least correct in our solution b finding out the largest value of m for which Cond () m is less than 0.5 0. mach Eample 4 How man significant digits can I trust in the solution of the following sstem of equations? 3.999 4 Solution For [ ] 3.999
04.09.0 Chapter 04.09 it can be shown 3999 000 5.999 5999 000 000 [ ] ( ) Cond 5.999 5999.4 35990 ssuming single precision with 4 bits used in the mantissa for real numbers, the machine epsilon is 4 mach 6 0.909 0 6 Cond ( ) mach 35990 0.909 0 0.490 0 m For what maimum positive value of m, would Cond( ) mach be less than 0.5 0 m 0.490 0 0.5 0 m 0.8580 0 0 m log(0.8580 0 ) log(0 ).067 m m.067 m So two significant digits are at least correct in the solution vector. Eample 5 How man significant digits can I trust in the solution of the following sstem of equations? 4 3 7 Solution For [ ] 3
dequac of Solution 04.09. it can be shown 3 Then 5, [ ] 5. Cond () 5 5 5 ssuming single precision with 4 bits used in the mantissa for real numbers, the machine epsilon 4 mach 6 0.909 0 6 Cond ( ) mach 5 0.909 0 5 0.980 0 m For what maimum positive value of m, would Cond( ) mach be less than 0.5 0 5 m 0.980 0 0.5 0 m 5 So five significant digits are at least correct in the solution vector. Ke Terms: Ill-Conditioned matri Well-Conditioned matri Norm Condition Number Machine Epsilon Significant Digits
Problem Set Chapter 04.09 dequac of Solutions. The adequac of the solution of simultaneous linear equations depends on () Condition number (B) Machine epsilon (C) Product of condition number and machine epsilon (D) Norm of the matri.. If a sstem of equations [ ][ X ] [ C] is ill conditioned, then () det( ) 0 (B) Cond ( ) (C) Cond( ) is large. (D) is large. 4 3. If Cond ( ) 0 and mach 0.9 0-6, then in [ ][ X ] [ C], at least these man significant digits are correct in our solution, () 0 (B) (C) (D) 3 4 4. Make a small change in the coefficient matri of 3.999 7.999 and find / X / Is it a large or small number? How is this number related to the condition number of the coefficient matri? 5. Make a small change in the coefficient matri of 4 3 7 and find / X. / Is it a large or a small number? Compare our results with the previous problem. How is this number related to the condition number of the coefficient matri? 07.03.
07.03. Chapter 07.03 6. Prove X < C C 7. For 0 7 0 [ ] 3.099 6 5 5 gives 0.099 0.333 0.799 [ ] 0.999 0.333 0.3999 5 0.04995 0.666 6.664 0 () What is the condition number of []? (B) How man significant digits can we at least trust in the solution of 6 [ ][ X ] [ C] if mach 0.9 0? (C) Without calculating the inverse of the matri [], can ou estimate the condition number of [] using the theorem in problem#6? 8. Prove that the Cond ( ).
Trapezoidal Rule 07.03.3 nswers to Selected Problems:. C. C 3. C 4. Changing [] to.00.00.00 4.000 Results in solution of 5999 999 X 5999.7 0.00 5.999 8.994 0 6 5. Changing [] to.00.00.00 3.000 Results in solution of.003 0.9970 X 0.003.000 0.00 5 3.75 6. Use theorem that if [ ][ B] [ C] then B C 7.
07.03.4 Chapter 07.03 () 7.033 Cond () 7.56 (B) 5 T (C) Tr different values of right hand side of C [ ± ± ± ] with signs chosen randoml. Then X obtained from solving equation set [ ][ X ] [ C] as C. 8. We know that B B then if [ B ] [ ], I Cond ( )