NUMERICAL DIFFERENTIATION James T Smit San Francisco State University In calculus classes, you compute derivatives algebraically: for example, f( x) = x + x f ( x) = x x Tis tecnique requires your knowing te formula for f and te appropriate differentiation rule Wat if you don t know a rule? One way is to refer to te definition of te derivative as te limit of te differential quotient f ( x+ ) f( x) f ( x) = lim 0 and evaluate te quotient for smaller and smaller until successive values don t cange significantly You can accept te last value as an adequate approximation to te derivative Te function f (x) = arctan x is a good example You don t learn its derivative very early in calculus classes, so you migt want to use tis metod On te oter and, tere is a formula, d arctan x =, + dx x so you can ceck te accuracy of te differential quotient approximation Figure gives te results for a typical argument x, wit a succession of smaller and smaller values Te error seems closely tied to te size of, until tat gets too small Te metod works well ere, but tere seems to be a limit to te accuracy Consider a less successful case: π x = 0 f( x) = sin x = 00 π π π f ( x) = cos = cosπ = 0π 346 x x 00 π π sin sin f( x+ ) f( x) sin0π sinπ = 0 0 = = 0 00 00 Clearly, you must take care wen you approximate a derivative tis way! In tese notes, you ll see ow to predict tis metod s beavior, so you can avoid taking too small, and avoid catastropes like te second example 6 December 00
Page NUMERICAL DIFFERENTIATION f (x) = arctan x x = 345678 f r(x) = 039678 f ( x+ ) f( x) Approximating f r(x) by Error = Approximation f r(x) Approximation Error 0E+0 060044 4E 0E 037759 9E 0E 039445 9E 3 0E 3 0395978 9E 4 0E 4 039654 9E 5 0E 5 0396698 9E 6 0E 6 039670 7E 7 0E 7 039668 36E 6 0E 8 0396375 34E 5 Figure Approximating a derivative by a differential quotient A major tool for analyzing approximation errors is te mean-value teorem, considered in your first calculus course Te teorem relates a difference in function arguments to te resulting difference in function values, as follows Suppose function f is defined and continuous at eac point x in a closed interval a # x # b, and differentiable at eac interior point Ten for some interior point ξ, f( b) f( a) = f ( ξ ) b a Te geometric interpretation is striking: te teorem says tat at some point <ξ, f (ξ)> te grap of f as te same slope as te secant troug points <a, f (a)> and <b, f (b)> Since te derivative isn t based on geometry, te proof of te teorem, found in calculus texts, requires detailed analysis In some cases you can actually calculate ξ: for example, if f (x) = x, ten ξ = ) (a + b) In general, owever, te teorem just asserts existence of ξ, and gives no information about its exact value To use te mean-value teorem to analyze te error f( x+ ) f( x) ε = f ( x)
NUMERICAL DIFFERENTIATION Page 3 resulting from approximating f r(x) by te differential quotient, first let a, b = x, x + so tat b a = Te teorem yields ξ between a and b suc tat ε = f r(ξ) f r(x) Now apply it again wit f replaced by f r, and a, b = x, ξ You get η between x and ξ suc tat *ε * = * f r(ξ) f r(x)* = *(ξ x) f O(η)* # M, provided M is an upper bound for all values * f O(η)* wit η between x and x + Tus, te approximation error doesn t exceed M From Taylor s teorem, considered in later calculus classes, you can derive a smaller error bound ) M Te following paragraps use tis version to analyze te beavior of te differential quotients in te earlier examples If f (x) = arctan x, ten x f ( x) =, + x and you can verify tat * f O(η)* # for all 0 between x = 345678 and x + Tus you can take M =, and te approximation error souldn t exceed ) Tat s true in figure for > 0 6, but it fails afterward Tere s no contradiction in te matematics tis analysis neglected someting tat appened in producing figure Before turning to tat, owever, consider te second example f (x) = sin(π/x): π π ( π ) π f ( x) = cos + sin 3 4 x x x x From te grap of f O you can see tat te largest value M 0000 of * f O(x)* on te interval 00 # x # 0 occurs at x 004 Wit = 00 you get te error bound ) M > 500, wic agrees wit te example and suggests tat te approximation may be very inaccurate Wy did te error ε in figure stop decreasing wit in spite of te bound ε # ) M? Te analysis tat led to tis inequality assumed tat te difference quotient was computed exactly But tat s not true In particular, calculating f wit any computer involves roundoff error Te two f values in te quotient are almost equal, so wen you subtract tem, roundoff errors may form a more significant part of te result Consider a simpler situation wit a different function f :
Page 4 NUMERICAL DIFFERENTIATION 00000005003 0000000400 = 00@0 8 ))))0))))-N ))))0))))- N N f (x + ) * f (x) * * ))))))0)))) ))- * Roundoff errors are in te t digit of te f values Resulting error is in te 4t digit of te difference Tis effect, subtractive cancellation error, is bad by itself But in te difference quotient, it s multiplied by a large factor, because is tiny Tus you can expect tat someting will go wrong wit tis approximation Can you predict wen? It s not difficult to use elementary calculus to determine wen te error stops decreasing Suppose te function values actually computed are f * (x + ) = f (x + ) + e f * (x) = f (x) + e, were f (x + ), f (x) are exact values and e, e are roundoff errors Ten te total error in te approximation is * * f ( x + ) f ( x) f( x + ) f( x) e e f ( x) = f ( x) + + f ( x ) f ( x ) f ( x) e e Wit Turbo Pascal Real aritmetic, used to compute Figure, *e *, *e * # 0 Tus te total error is bounded by M 0 0 E = + = + E() is smoot and approaces 4 as approaces 0+ or 4, ence it as a minimum value at some > 0 were 0 0 = E =, ie = 4 0, = 0 6 Tat te total approximation error is bounded by a function E() wit a minimum value at = 0 6 suggests tat te error will stop decreasing between = 0 5 and 0 6 Figure confirms tat Te previous discussion was devoted to computing a derivative in a situation were you could calculate te function at will, but ad no rule for differentiating it Wat if you only ave a table of function values? If you need an approximation to f r(x) for a
NUMERICAL DIFFERENTIATION Page 5 tabulated argument x, and x + is te next one, ten you can use te differential quotient f ( x+ ) f( x), and from te formula ) M predict te size of te approximation error (as long as is not so small as to cause subtractive cancellation roundoff error buildup) Of course, you d ave to know M If you don t, you souldn t trust approximate derivatives calculated tis way After all, you could construct anoter function g agreeing wit f at arguments x and x + By making go large enoug (increasing M ) you could make gr differ wildly from f r Te differential quotient formula would still yield an approximation to f r(x), not gr(x) If you re designing a table of values f (x) to facilitate calculating f r(x) by differential quotient approximation, and you know M, ten you can ensure tat te approximation error will not exceed a specified tolerance T by selecting so tat ) M < T You souldn t select small enoug, toug, to cause subtractive cancellation roundoff error buildup Many derivative approximation formulas more accurate tan te differential quotient are studied in numerical analysis Tese two are used in te commercial software package MatCAD : f + ( x ) f ( x f x ) f + + + ( x ) 8 f ( x ) 8 f ( x ) f ( x f x ) ) 6 M 3 ) 30 M 5 4 Te expressions at rigt are error bounds, disregarding roundoff error In general, M k stands for an upper bound of te values * f ( k ) (x)* wit x in te region of interest You sould be able to construct corresponding examples and error analyses analogous to te ones in tese notes Tat would test your understanding and sow you ow muc more accurate tese formulas are Reference Ricard L BURDEN and J Douglas FAIRES, Numerical analysis, 6t ed, Brooks/Cole, 997 4