Numerics and Error Analysis

Transcription

1 Numerics and Error Analysis CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Numerics and Error Analysis 1 / 30

2 A Puzzle What is the value of the following expression? abs(3.*(4./3.-1.)-1.) CS 205A: Mathematical Methods Numerics and Error Analysis 2 / 30

3 A Puzzle What is the value of the following expression? abs(3.*(4./3.-1.)-1.) Machine precision ε m : smallest ε m with 1 + ε m = 1 CS 205A: Mathematical Methods Numerics and Error Analysis 2 / 30

4 Prototypical Example double x = 1.0; double y = x / 3.0; if (x == y*3.0) cout << "They are equal!"; else cout << "They are NOT equal."; CS 205A: Mathematical Methods Numerics and Error Analysis 3 / 30

5 Take-Away Mathematically correct Numerically sound CS 205A: Mathematical Methods Numerics and Error Analysis 4 / 30

6 Using Tolerances double x = 1.0; double y = x / 3.0; if (fabs(x-y*3.0) < numeric_limits<double>::epsilon) cout << "They are equal!"; else cout << "They are NOT equal."; CS 205A: Mathematical Methods Numerics and Error Analysis 5 / 30

7 Counting in Binary: Integer 463 = = CS 205A: Mathematical Methods Numerics and Error Analysis 6 / 30

8 Counting in Binary: Fractional = /4 = CS 205A: Mathematical Methods Numerics and Error Analysis 7 / 30

9 Familiar Problem 1 3 = Finite number of bits CS 205A: Mathematical Methods Numerics and Error Analysis 8 / 30

10 Fixed-Point Arithmetic l 2 l k+1 2 k Parameters: k, l Z k + l + 1 digits total Can reuse integer arithmetic (fast; GPU possibility): a + b = (a 2 k + b 2 k ) 2 k CS 205A: Mathematical Methods Numerics and Error Analysis 9 / 30

11 Two-Digit Example = = 0.02 Multiplication and division easily change order of magnitude! CS 205A: Mathematical Methods Numerics and Error Analysis 10 / 30

12 Demand of Scientific Applications Desired: Graceful transition CS 205A: Mathematical Methods Numerics and Error Analysis 11 / 30

13 Observations Compactness matters: = 602, 200, 000, 000, 000, 000, 000, 000 CS 205A: Mathematical Methods Numerics and Error Analysis 12 / 30

14 Observations Compactness matters: = 602, 200, 000, 000, 000, 000, 000, 000 Some operations are unlikely: CS 205A: Mathematical Methods Numerics and Error Analysis 12 / 30

15 Scientific Notation ± sign Store significant digits (d 0 + d 1 b 1 + d 2 b d p 1 b 1 p ) significand Base: b N Precision: p N Range of exponents: e [L, U] b e exponent CS 205A: Mathematical Methods Numerics and Error Analysis 13 / 30

16 Properties of Floating Point Unevenly spaced Machine precision ε m : smallest ε m with 1 + ε m = 1 CS 205A: Mathematical Methods Numerics and Error Analysis 14 / 30

17 Properties of Floating Point Unevenly spaced 1.75 Machine precision ε m : smallest ε m with 1 + ε m = 1 Needs rounding rule (e.g. round to nearest, ties to even ) CS 205A: Mathematical Methods Numerics and Error Analysis 14 / 30

18 Properties of Floating Point Unevenly spaced 1.75 Machine precision ε m : smallest ε m with 1 + ε m = 1 Needs rounding rule (e.g. round to nearest, ties to even ) Can remove leading 1 CS 205A: Mathematical Methods Numerics and Error Analysis 14 / 30

19 Infinite Precision Q = { a /b : a, b Z} Simple rules: a /b + c /d = (ad+cb) /bd Redundant: 1 /2 = 2 /4 Blowup: = Restricted operations: 2 2 CS 205A: Mathematical Methods Numerics and Error Analysis 15 / 30

20 Bracketing Store range a ± ε Special case of Interval Arithmetic Keeps track of certainty and rounding decisions Easy bounds: (x±ε 1 )+(y ±ε 2 ) = (x+y)±(ε 1 +ε 2 +error(x+y)) Implementation via operator overloading CS 205A: Mathematical Methods Numerics and Error Analysis 16 / 30

21 Sources of Error Rounding Discretization Modeling Input CS 205A: Mathematical Methods Numerics and Error Analysis 17 / 30

22 Example What sources of error might affect a financial simulation? CS 205A: Mathematical Methods Numerics and Error Analysis 18 / 30

23 Example Catastrophic cancellation Consider the function near x = 0. f(x) = ex 1 x 1 CS 205A: Mathematical Methods Numerics and Error Analysis 19 / 30

24 Absolute vs. Relative Error Absolute Error The difference between the approximate value and the underlying true value CS 205A: Mathematical Methods Numerics and Error Analysis 20 / 30

25 Absolute vs. Relative Error Absolute Error The difference between the approximate value and the underlying true value Relative Error Absolute error divided by the true value CS 205A: Mathematical Methods Numerics and Error Analysis 20 / 30

26 Absolute vs. Relative Error Absolute Error The difference between the approximate value and the underlying true value Relative Error Absolute error divided by the true value 2 in ± 0.02 in 2 in ± 1% CS 205A: Mathematical Methods Numerics and Error Analysis 20 / 30

27 Relative Error: Difficulty Problem: Generally not computable CS 205A: Mathematical Methods Numerics and Error Analysis 21 / 30

28 Relative Error: Difficulty Problem: Generally not computable Common fix: Be conservative CS 205A: Mathematical Methods Numerics and Error Analysis 21 / 30

29 Computable Measures of Success Root-finding problem For f : R R, find x such that f(x ) = 0. Actual output: x est with f(x est ) 1 CS 205A: Mathematical Methods Numerics and Error Analysis 22 / 30

30 Backward Error Backward Error The amount the problem statement would have to change to make the approximate solution exact CS 205A: Mathematical Methods Numerics and Error Analysis 23 / 30

31 Backward Error Backward Error The amount the problem statement would have to change to make the approximate solution exact Example 1: x CS 205A: Mathematical Methods Numerics and Error Analysis 23 / 30

32 Backward Error Backward Error The amount the problem statement would have to change to make the approximate solution exact Example 1: x Example 2: A x = b CS 205A: Mathematical Methods Numerics and Error Analysis 23 / 30

33 Conditioning Well-conditioned: Small backward error = small forward error Poorly conditioned: Otherwise Example: Root-finding CS 205A: Mathematical Methods Numerics and Error Analysis 24 / 30

34 Condition Number Condition number Ratio of forward to backward error CS 205A: Mathematical Methods Numerics and Error Analysis 25 / 30

35 Condition Number Condition number Ratio of forward to backward error Root-finding example: 1 f (x ) CS 205A: Mathematical Methods Numerics and Error Analysis 25 / 30

36 Theme Extremely careful implementation can be necessary. CS 205A: Mathematical Methods Numerics and Error Analysis 26 / 30

37 Example: x 2 double normsquared = 0; for (int i = 0; i < n; i++) normsquared += x[i]*x[i]; return sqrt(normsquared); CS 205A: Mathematical Methods Numerics and Error Analysis 27 / 30

38 Improved x 2 double maxelement = epsilon; for (int i = 0; i < n; i++) maxelement = max(maxelement, fabs(x[i])); for (int i = 0; i < n; i++) { double scaled = x[i] / maxelement; normsquared += scaled*scaled; } return sqrt(normsquared) * maxelement; CS 205A: Mathematical Methods Numerics and Error Analysis 28 / 30

39 More Involved Example: i x i double sum = 0; for (int i = 0; i < n; i++) sum += x[i]; CS 205A: Mathematical Methods Numerics and Error Analysis 29 / 30

40 Motivation for Kahan Algorithm ((a + b) a) b =? 0 Store compensation value! Details in textbook Next CS 205A: Mathematical Methods Numerics and Error Analysis 30 / 30