Handout for Adequacy of Solutions Chapter SET ONE The solution to Make a small change in the right hand side vector of the equations

Transcription

1 Handout for dequac of Solutions Chapter SET ONE The solution to Make a small change in the right hand side vector of the equations Make a small change in the coefficient matri of the equations

2 SET TWO The solution to Make a small change in the right hand side vector of the equations Make a small change in the coefficient matri of the equations

3 Chapter dequac of Solutions fter reading this chapter, ou should be able to:. know the difference between ill-conditioned and well-conditioned sstems of equations,. define the norm of a matri, and 3. relate the norm of a matri and of its inverse to the ill or well conditioning of the matri, that is, how much trust can ou having in the solution of the matri. What do ou mean b ill-conditioned and well-conditioned sstem of equations? sstem of equations is considered to be well-conditioned if a small change in the coefficient matri or a small change in the right hand side results in a small change in the solution vector. sstem of equations is considered to be ill-conditioned if a small change in the coefficient matri or a small change in the right hand side results in a large change in the solution vector. Eample Is this sstem of equations well-conditioned? Solution The solution to the above set of equations is Make a small change in the right hand side vector of the equations

4 Chapter gives Make a small change in the coefficient matri of the equations gives This last sstems of equation looks ill-conditioned because a small change in the coefficient matri or the right hand side resulted in a large change in the solution vector. Eample Is this sstem of equations well-conditioned? Solution The solution to the above equations is Make a small change in the right hand side vector of the equations gives Make a small change in the coefficient matri of the equations gives

5 dequac of Solution This sstem of equation looks well conditioned because small changes in the coefficient matri or the right hand side resulted in small changes in the solution vector. So what if the sstem of equations is ill conditioned or well conditioned? Well, if a sstem of equations is ill-conditioned, we cannot trust the solution as much. Revisit the velocit problem, Eample 5. in Chapter 5. The values in the coefficient matri [ ] are squares of time, etc. For eample, if instead of a 5, ou used a 4.99, would ou want this small change to make a huge difference in the solution vector. If it did, would ou trust the solution? Later we will see how much (quantifiable terms) we can trust the solution in a sstem of equations. Ever invertible square matri has a condition number and coupled with the machine epsilon, we can quantif how man significant digits one can trust in the solution. To calculate the condition number of an invertible square matri, I need to know what the norm of a matri means. How is the norm of a matri defined? Just like the determinant, the norm of a matri is a simple unique scalar number. However, the norm is alwas positive and is defined for all matrices square or rectangular, and invertible or noninvertible square matrices. One of the popular definitions of a norm is the row sum norm (also called the uniformmatri norm). For a m n matri [], the row sum norm of [] is defined as ma n a ij < i< m j that is, find the sum of the absolute value of the elements of each row of the matri []. The maimum out of the m such values is the row sum norm of the matri []. Eample 3 Find the row sum norm of the following matri []

6 Chapter Solution ma < i< 3 a ij 3 j [( ), ( ), ( )] [( ),( ),( 5 + 5) ] [ ] ma + ma + ma 7,.099, 7. How is the norm related to the conditioning of the matri? Let us start answering this question using an eample. Go back to the ill-conditioned sstem of equations, that gives the solution as Denoting the above set of equations as X C [ ][ ] [ ] X C Making a small change in the right hand side, gives Denoting the above set of equations b [ ][ X '] [ C' ] right hand side vector is found b [ C] [ C' ] [ C] and the change in the solution vector is found b [ X ] [ X '] [ X ] then

7 dequac of Solution and then [ C ] [ ] C The relative change in the norm of the solution vector is X.9995 The relative change in the norm of the right hand side vector is C 0.00 C See the small relative change of.50 0 in the right hand side vector results in a large relative change in the solution vector as In fact, the ratio between the relative change in the norm of the solution vector and the relative change in the norm of the right hand side vector is / X C / C Let us now go back to the well-conditioned sstem of equations gives

8 Chapter Denoting the sstem of equations b X C [ ][ ] [ ] X C 7 Making a small change in the right hand side vector gives Denoting the above set of equations b [ ][ X '] [ C' ] the change in the right hand side vector is then found b [ C] [ C' ] [ C] and the change in the solution vector is [ X ] [ X '] [ X ] then [ C ] and.999 [ ] then C The relative change in the norm of solution vector is

9 dequac of Solution X The relative change in the norm of the right hand side vector is C 0.00 C See the small relative change the right hand side vector of.49 4 small relative change in the solution vector of results in the In fact, the ratio between the relative change in the norm of the solution vector and the relative change in the norm of the right hand side vector is / X C / C What are some of the properties of norms?. For a matri [], 0. For a matri [] and a scalar k, k k 3. For two matrices [] and [B] of same order, + B + B 4. For two matrices [] and [B] that can be multiplied as [ ][ B], B B Is there a general relationship that eists between X / X and C / C or between X / X and /? If so, it could help us identif well-conditioned and ill conditioned sstem of equations. If there is such a relationship, will it help us quantif the conditioning of the matri? That is, will it tell us how man significant digits we could trust in the solution of a sstem of simultaneous linear equations? There is a relationship that eists between

10 Chapter C and X C and between and X These relationships are X + and X < C C The above two inequalities show that the relative change in the norm of the right hand side vector or the coefficient matri can be amplified b as much as. This number is called the condition number of the matri and coupled with the machine epsilon, we can quantif the accurac of the solution of [ ][ X ] [ C]. Prove for [ ][ X ] [ C] that. X + Proof Let [ ][ X ] [ C] () Then if [] is changed to [ '], the [X ] will change to [ X '], such that [ '][ X '] [ C] () From Equations () and (), [ ][ X ] [ ' ][ X '] Denoting change in [] and [X ] matrices as [ ] and [ X ], respectivel [ ] [ ' ] [ ] X X ' X [ ] [ ] [ ]

11 dequac of Solution then [ ][ X ] ([ ] + [ ] )([ X ] + [ ]) Epanding the above epression [ ][ X ] [ ][ X ] + [ ][ ] + [ ][ X ] + [ ][ ] [ 0] [ ][ ] + [ ] ([ X ] + [ ]) [ ][ ] [ ] ([ X ] + [ ]) [ ] [ ] [ ] ([ X ] + [ ]) ppling the theorem of norms, that the norm of multiplied matrices is less than the multiplication of the individual norms of the matrices, < X + Multipling both sides b < X + < X + How do I use the above theorems to find how man significant digits are correct in m solution vector? The relative error in a solution vector is Cond () relative error in right hand side. The possible relative error in the solution vector is Cond () mach Hence Cond () mach should give us the number of significant digits, m that are at least correct in our solution b finding out the largest value of m for which Cond () m is less than mach Eample 4 How man significant digits can I trust in the solution of the following sstem of equations? Solution For [ ] 3.999

12 Chapter it can be shown [ ] ( ) Cond ssuming single precision with 4 bits used in the mantissa for real numbers, the machine epsilon is 4 mach Cond ( ) mach m For what maimum positive value of m, would Cond( ) mach be less than m m m log( ) log(0 ).067 m m.067 m So two significant digits are at least correct in the solution vector. Eample 5 How man significant digits can I trust in the solution of the following sstem of equations? Solution For [ ] 3

13 dequac of Solution it can be shown 3 Then 5, [ ] 5. Cond () ssuming single precision with 4 bits used in the mantissa for real numbers, the machine epsilon 4 mach Cond ( ) mach m For what maimum positive value of m, would Cond( ) mach be less than m m 5 So five significant digits are at least correct in the solution vector. Ke Terms: Ill-Conditioned matri Well-Conditioned matri Norm Condition Number Machine Epsilon Significant Digits

14 Problem Set Chapter dequac of Solutions. The adequac of the solution of simultaneous linear equations depends on () Condition number (B) Machine epsilon (C) Product of condition number and machine epsilon (D) Norm of the matri.. If a sstem of equations [ ][ X ] [ C] is ill conditioned, then () det( ) 0 (B) Cond ( ) (C) Cond( ) is large. (D) is large If Cond ( ) 0 and mach , then in [ ][ X ] [ C], at least these man significant digits are correct in our solution, () 0 (B) (C) (D) Make a small change in the coefficient matri of and find / X / Is it a large or small number? How is this number related to the condition number of the coefficient matri? 5. Make a small change in the coefficient matri of and find / X. / Is it a large or a small number? Compare our results with the previous problem. How is this number related to the condition number of the coefficient matri?

15 Chapter Prove X < C C 7. For [ ] gives [ ] () What is the condition number of []? (B) How man significant digits can we at least trust in the solution of 6 [ ][ X ] [ C] if mach 0.9 0? (C) Without calculating the inverse of the matri [], can ou estimate the condition number of [] using the theorem in problem#6? 8. Prove that the Cond ( ).

16 Trapezoidal Rule nswers to Selected Problems:. C. C 3. C 4. Changing [] to Results in solution of X Changing [] to Results in solution of X Use theorem that if [ ][ B] [ C] then B C 7.

17 Chapter () Cond () 7.56 (B) 5 T (C) Tr different values of right hand side of C [ ± ± ± ] with signs chosen randoml. Then X obtained from solving equation set [ ][ X ] [ C] as C. 8. We know that B B then if [ B ] [ ], I Cond ( )