Numerical Differentiation & Integration. Numerical Differentiation III

Transcription

1 Numerical Differentiation & Integration Numerical Differentiation III Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011 Brooks/Cole, Cengage Learning

2 Outline 1 Round-Off Error Instability Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 2 / 16

3 Outline 1 Round-Off Error Instability 2 Numerical Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 2 / 16

4 Outline 1 Round-Off Error Instability 2 Numerical Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 3 / 16

5 Concept of Total Error Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 4 / 16

6 Concept of Total Error It is particularly important to pay attention to round-off error when approximating derivatives. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 4 / 16

7 Concept of Total Error It is particularly important to pay attention to round-off error when approximating derivatives. To illustrate the situation, let us examine the three-point midpoint formula: more closely. f (x 0 ) = 1 2h [f(x 0 + h) f(x 0 h)] h2 6 f (3) (ξ 1 ), Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 4 / 16

8 Concept of Total Error It is particularly important to pay attention to round-off error when approximating derivatives. To illustrate the situation, let us examine the three-point midpoint formula: more closely. f (x 0 ) = 1 2h [f(x 0 + h) f(x 0 h)] h2 6 f (3) (ξ 1 ), Suppose that in evaluating f(x 0 + h) and f(x 0 h) we encounter round-off errors e(x 0 + h) and e(x 0 h). Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 4 / 16

9 f (x 0 ) = 1 2h [f(x 0 + h) f(x 0 h)] h2 6 f (3) (ξ 1 ) Concept of Total Error (Cont d) Then our computations actually use the values f(x 0 + h) and f(x0 h), Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 5 / 16

10 f (x 0 ) = 1 2h [f(x 0 + h) f(x 0 h)] h2 6 f (3) (ξ 1 ) Concept of Total Error (Cont d) Then our computations actually use the values f(x 0 + h) and f(x0 h), which are related to the true values f(x 0 + h) and f(x 0 h) by f(x 0 +h) = f(x 0 +h)+e(x 0 +h) and f(x 0 h) = f(x 0 h)+e(x 0 h) Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 5 / 16

11 f (x 0 ) = 1 2h [f(x 0 + h) f(x 0 h)] h2 6 f (3) (ξ 1 ) Concept of Total Error (Cont d) Then our computations actually use the values f(x 0 + h) and f(x0 h), which are related to the true values f(x 0 + h) and f(x 0 h) by f(x 0 +h) = f(x 0 +h)+e(x 0 +h) and f(x 0 h) = f(x 0 h)+e(x 0 h) The total error in the approximation, f (x 0 ) f(x 0 + h) f(x 0 h) 2h = e(x 0 + h) e(x 0 h) 2h h2 6 f (3) (ξ 1 ) is due both to round-off error, the first part, and to truncation error. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 5 / 16

12 f (x 0 ) f(x 0 + h) f(x 0 h) 2h = e(x 0 + h) e(x 0 h) 2h h2 6 f (3) (ξ 1 ) Concept of Total Error (Cont d) If we assume that the round-off errors e(x 0 ± h) are bounded by some number ε > 0 Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 6 / 16

13 f (x 0 ) f(x 0 + h) f(x 0 h) 2h = e(x 0 + h) e(x 0 h) 2h h2 6 f (3) (ξ 1 ) Concept of Total Error (Cont d) If we assume that the round-off errors e(x 0 ± h) are bounded by some number ε > 0 and that the third derivative of f is bounded by a number M > 0, Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 6 / 16

14 f (x 0 ) f(x 0 + h) f(x 0 h) 2h = e(x 0 + h) e(x 0 h) 2h h2 6 f (3) (ξ 1 ) Concept of Total Error (Cont d) If we assume that the round-off errors e(x 0 ± h) are bounded by some number ε > 0 and that the third derivative of f is bounded by a number M > 0, then f (x 0 ) f(x 0 + h) f(x 0 h) 2h ε h + h2 6 M Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 6 / 16

15 f (x 0 ) f(x 0 + h) f(x 0 h) 2h ε h + h2 6 M Concept of Total Error (Cont d) To reduce the truncation error, h 2 M/6, we need to reduce h. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 7 / 16

16 f (x 0 ) f(x 0 + h) f(x 0 h) 2h ε h + h2 6 M Concept of Total Error (Cont d) To reduce the truncation error, h 2 M/6, we need to reduce h. But as h is reduced, the round-off error ε/h grows. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 7 / 16

17 f (x 0 ) f(x 0 + h) f(x 0 h) 2h ε h + h2 6 M Concept of Total Error (Cont d) To reduce the truncation error, h 2 M/6, we need to reduce h. But as h is reduced, the round-off error ε/h grows. In practice, then, it is seldom advantageous to let h be too small because, in that case, the round-off error will dominate the calculations. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 7 / 16

18 Outline 1 Round-Off Error Instability 2 Numerical Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 8 / 16

19 Consider using the values in the following table x sin x x sin x to approximate f (0.900), where f(x) = sin x. The true value is cos = Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 9 / 16

20 f (x 0 ) f(x 0 + h) f(x 0 h) 2h ε h + h2 6 M Solution (1/4) The formula f (0.900) f( h) f(0.900 h) 2h with different values of h, gives the approximations in the following table. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 10 / 16

21 Solution (2/4) Approximation h to f (0.900) Error The optimal choice for h appears to lie between and Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 11 / 16

22 Solution (3/4) We can use calculus to verify that a minimum for e(h) = ε h + h2 6 M, occurs at h = 3 3ε/M, Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 12 / 16

23 Solution (3/4) We can use calculus to verify that a minimum for occurs at h = 3 3ε/M, where M = e(h) = ε h + h2 6 M, max f (x) = max cos x = cos x [0.800,1.00] x [0.800,1.00] Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 12 / 16

24 Solution (3/4) We can use calculus to verify that a minimum for occurs at h = 3 3ε/M, where M = e(h) = ε h + h2 6 M, max f (x) = max cos x = cos x [0.800,1.00] x [0.800,1.00] Because values of f are given to five decimal places, we will assume that the round-off error is bounded by ε = Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 12 / 16

25 Solution (4/4) Therefore, the optimal choice of h is approximately h = 3 3ε/M = 3 3( ) , which is consistent with the results in the earlier table. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 13 / 16

26 Solution (4/4) Therefore, the optimal choice of h is approximately h = 3 3ε/M = 3 3( ) , which is consistent with the results in the earlier table. In practice, we cannot compute an optimal h to use in approximating the derivative, since we have no knowledge of the third derivative of the function. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 13 / 16

27 Solution (4/4) Therefore, the optimal choice of h is approximately h = 3 3ε/M = 3 3( ) , which is consistent with the results in the earlier table. In practice, we cannot compute an optimal h to use in approximating the derivative, since we have no knowledge of the third derivative of the function. But we must remain aware that reducing the step size will not always improve the approximation. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 13 / 16

28 Concluding Remarks (1/2) Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 14 / 16

29 Concluding Remarks (1/2) We have considered only the round-off error problems that are presented by the three-point midpoint formula, but similar difficulties occur with all the differentiation formulas. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 14 / 16

30 Concluding Remarks (1/2) We have considered only the round-off error problems that are presented by the three-point midpoint formula, but similar difficulties occur with all the differentiation formulas. The reason can be traced to the need to divide by a power of h. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 14 / 16

31 Concluding Remarks (1/2) We have considered only the round-off error problems that are presented by the three-point midpoint formula, but similar difficulties occur with all the differentiation formulas. The reason can be traced to the need to divide by a power of h. Division by small numbers tends to exaggerate round-off error, and this operation should be avoided if possible. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 14 / 16

32 Concluding Remarks (1/2) We have considered only the round-off error problems that are presented by the three-point midpoint formula, but similar difficulties occur with all the differentiation formulas. The reason can be traced to the need to divide by a power of h. Division by small numbers tends to exaggerate round-off error, and this operation should be avoided if possible. In the case of numerical differentiation, we cannot avoid the problem entirely, although the higher-order methods reduce the difficulty. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 14 / 16

33 Concluding Remarks (2/2) As approximation methods, numerical differentiation is unstable, since the small values of h needed to reduce truncation error also cause the round-off error to grow. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 15 / 16

34 Concluding Remarks (2/2) As approximation methods, numerical differentiation is unstable, since the small values of h needed to reduce truncation error also cause the round-off error to grow. This is the first class of unstable methods we have encountered, and these techniques would be avoided if it were possible. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 15 / 16

35 Concluding Remarks (2/2) As approximation methods, numerical differentiation is unstable, since the small values of h needed to reduce truncation error also cause the round-off error to grow. This is the first class of unstable methods we have encountered, and these techniques would be avoided if it were possible. However, in addition to being used for computational purposes, the formulas are needed for approximating the solutions of ordinary and partial-differential equations. Numerical Analysis (Chapter 4) Numerical Differentiation III R L Burden & J D Faires 15 / 16

36 Questions?