CSC165H, Mathematical expression and reasoning for computer science week 12

Transcription

1 CSC165H, Mathematical exression and reasoning for comuter science week 1 nd December 005 Gary Baumgartner and Danny Hea hea@cs.toronto.edu SF4306A htt// Rounding Most numbers are not exactly reresentable in a oating-oint system using a given base. For examle, when = 10, you cannot reresent 1=3 exactly (no matter how large t is), so we used when t = 3. What should we do with something like the base of natural logarithms, e = ? Two aroaches are used Round to nearest Truncate to zero Overflow There is no way to reresent a number larger than the largest oating-oint number. In our rst examle ( = 10; t = 3; e [ 4; +4]), there is no way to reresent or greater. Underflow There is no way to reresent a ositive number smaller than the smallest ositive oatingoint number. In our rst examle, there is no way to reresent (or a smaller 1

2 ositive number). Absolute rounding error We can calculate the dierence between the true value we're trying to reresent and the value of its oating-oint reresentation. For examle, the absolute error in our reresentation of e is j j = j j. Relative error 100 and are closer than 1 and 1.1, even though the absolute dierence is 0.1 in both cases. Look at the size of the error in terms of the size of the value being reresented. Relative error For x 6= 0, the relative error between the aroximate value x 0 and the real value x is jx For examle, j11 1j=j11j = %. However, j j=j100j = %. jxj Relative error in round-to-nearest When we round numbers to reresent them in a oating-oint system, can we bound relative error for ositive numbers with no overow or underow? 1. In general, a number of the form x 0 j d 0 d 1 d t 1 d t e... gets rounded either u or down to one of d 0 d 1 (d t d 0 d 1 d t 1 + 1) e 1 e The reresentation of the rst number may be dierent (the +1 may cause a number of carries in the addition), but the value is the same. The dierence between these two numbers is simly a 1 in the osition occuied by d t 1, for a dierence of e = e (t 1). Since we round to nearest, our error is at most half this value e (t 1)

3 The relative error can be calculated, since we use the convention that the leading digit, d 0, is non-zero, so the smallest denominator (hence the largest bound) is when d 0 = 1, giving a relative error of j e (t 1) =j j10 0 e j = 1 t This matches our intuition that by increasing t (the number of digits) we get more recision. In our examle of a oating-oint system with = ; t = 3, this gives a bound on the relative error of round-to-nearest of 1 3 = = 1=8. This is also clear from the number line or the 4 values in this reresentation. Accumulation of error Since we can't reresent all real numbers exactly in a oating oint reresentation, what haens when we reeat oerations. For examle, take = 10, t = 3, e [ ; +], and consider the sum [n times] The rst art of the sum, is reresented by As we add additional terms with value 01, the result is still 10010, since our reresentation cannot reresent that additional 01. The relative error can become arbitrarily large, given large enough n. If you lay around with FloatExamle.cumulativeError (on the web age), with say t1 = 1.0, n 10, and t = 10 16, you'll see this demonstrated. The easy way to work around this articular roblem is to add the 01 terms rst, and then add the 100 in other words, addition is not associative for oating-oint numbers (a + b) + c 6= a + (b + c). Catastrohic cancellation Use the same oating-oint system as in the revious examle to comute b 4ac for b = 334, a = 1, and c = 8. The exact value is 009 = 9 10, and this exact value is reresentable in our oating-oint system. Look at how the value is calculated, 3

4 though b = (334) = ac = = = b 4ac = = Comared to our exact answer of 9 10, this has a relative error of j009 01j 009 = = 44 > 40% Subtracting two oating-oint numbers that are very close together leaves very few significant digits a great deal of information is lost. Since the true value is very small, the round-o error becomes much more signicant, and sometimes becomes much larger than the value being comuted (see above). The exression b 4ac cros u in the solution to the quadratic equation ax +bx+c = 0. The general form of the solution, for the two roots x 1 and x is x 1 = b + b 4ac a x = b b 4ac a We may not have to worry about the large relative error in b 4ac, since it may be small in absolute value comared to b. Here's a case where comuting x 1 (using the values that lead to catastrohic cancellation above) gives a fairly accetable value using oating-oint oerations x 1 = = = Comare this to the result if there were no error in the comutation of b 4ac, which is x 1 = for a relative error of less than 5%. Stability = = 130 There is a built-in roblem with the formulas used above to comute the roots of a quadratic equation if b 4ac is close to b, then there will be catastrohic cancellation 4

5 between b and + b 4ac. This is searate from the catastrohic cancellation that may haen if b is close to 4ac. Denition a formula (or algorithm) is called unstable i errors in the inut values get magnied during the comutation (i.e., i the relative error in the nal answer can be larger than the relative error in the inut values). Dealing with instability Our rst examle ( ) was unstable, but there was an easy way to use a dierent algorithm that is stable erform the oerations in a dierent order. Our second examle (b 4ac) was also unstable, because of otential catastrohic cancellation, and there is, unfortunately, no easy x. You could increase the number of signicant digits to make the round-o error smaller, but you will still have the otential for catastrohic cancellation when you subtract numbers that are very close, even with your increased recision. The formula is unstable. Our third examle is also unstable (it includes the instability of examle, lus its own instability), but the ossibility of cancellation between b and + b 4ac can be avoid by changing the formula. Suose b > 0 (otherwise swa the role of x 1 and x below if b < 0). Then avoid the subtraction in the numerator of the quadratic formula x = b b 4ac a Since both b and b 4ac have the same (negative) sign, there will be no catastrohic cancellation. Now comute x 1 using x x 1 = c ax (multily the two roots to see this) This formula involves no subtraction, so there is no catastrohic cancellation. There are two ways, in general, to deal with unstable formulas or algorithms Increase the recision (the number of signicant digits). This does not change the fact that the formula or algorithm is unstable, but can hel minimize the magnitude of the errors,for some inuts. Use a dierent, stable, algorithm or formula to comute the result. When ossible, this is referred. Conditioning In the revious examle we examined the error roduced in calculating the roots of ax + bx + c. The numbers a, b, and c may come from measurement and already have some error 5

6 associated with them. By using a stable algorithm we get a relatively correct answer for slightly incorrect inuts. But, indeendently of the articular algorithm used to comute roots, what can be said about the eect that errors in the measurement of a, b, and c have on values of roots? That is, if the values of a, b, or c change slightly, could the roots change dramatically? To simly the discussion, let's look at the secial case of a quadratic formula x c = 0 (so we're nding c). Suose that c = 05, but we use a bad aroximation c 0 = 036 instead of the true value. Then our answer will be 06 instead of 05. The relative error in the inut is 011=05 = 044, while the relative error in the result is 01=05 = 0. Taking the square root makes the relative error smaller! This isn't something secial about the articular case we chose. Let's work the algebra to see the general case of comuting c using an aroximation c 0 of c. The ratio of the relative error of the result to the relative error of the inut is (assuming c and c 0 are close enough to have the same sign) j c jc c 0j=j cj c 0 j=jcj = c j c jc c 0j c 0 j [ jcj= c = c] = c c + c 0 [ jc c 0 j = ( q c + c 0 )( (c) c This ratio is always less than 1 (so the error imroves), and when c 0 is very close to c, the ratio is close to 1=. So the error is never increased, and as the measurements imrove it is reduced to almost 1=. When comuting the function f(x) using the aroximation x 0 instead of the true value x, we say that the condition number is equal to the ratio of the relative error of the result and the relative error of the inut, i.e. jf(x) jx f(x 0 )j=jf(x)j x 0 j=jxj If you take the limit as x 0! x (and assume that f is dierentiable, and that f(x) 6= 0), this is jf(x) f(x 0 )j=jf(x)j lim x 0!x jx x 0 = jxj j=jxj jf(x)j lim jf(x) f(x 0 )j x 0!x jx x 0 = jxf0 (x)j j jf(x)j For f(x) = x, the condition number is (well, do the derivative)! Not all functions have good condition numbers. Comute the condition number for cos(x), 3 and you'll see that you can a huge condition number by choosing an aroriate x. 4 6

7 Notes 1 In fact, with overow or underow, the error can be arbitrarily large 1/. 3 Did you get jx tan(x)j? 4 Such as x close to = radians. 7