derivatives of a function at a point. The formulas will be in terms of function data; in our case

Transcription

1 Math 220 Spring 200 Prof. Diamond Approximation of derivatives Here we develop formulas, and accompanying error estimates, for approximating derivatives of a function at a point. The formulas will be in terms of function data; in our case the data will just be point values but one could imagine other types of data in the form of integrals or derivatives, etc. Most of the examples will be for f Ÿx ' or f Ÿx ' but the extension to higher derivatives will be obvious. (Here, x ' merely represents some value of x at which we would like to calculate a derivative.) We begin by stating an alternate form of Taylor s theorem that will be useful for us: fÿx 0 h fÿx 0 f Ÿx 0 h 2! f Ÿx 0 h 2... k! f Ÿk Ÿx 0 h k Ÿk! f Ÿk Ÿ8 h k This formula can be obtained from Taylor s theorem: fÿa Ÿx " a fÿx fÿa f Ÿa Ÿx " a... k! f Ÿk Ÿa Ÿx " a k Ÿk! f Ÿk Ÿ8 Ÿx " a k If we replace a by x 0 and replace x " a by h the above formula is obtained. Error analysis of formulas using Taylor s theorem: Given a formula, we can analyze its error via Taylor expansion in a Tayor polynomial about the point at which the derivative is to be approximated. We begin with the forward difference formula familiar to us from calculus. (Here we use x 0 rather than x ' ): f Ÿx 0 X fÿx 0 h " fÿx 0 h We expand about x 0 as explained above: fÿx 0 h " fÿx 0 h h fÿx 0 h fÿx 0 f Ÿx 0 h 2! f Ÿ8 h 2 " fÿx 0 fÿx 0 f Ÿx 0 2! f Ÿ8 h Numerical Analysis Spring 200 Prof Diamond

2 So for the error, we have f Ÿx 0 " fÿx 0 h " fÿx 0 h " 2! f Ÿ8 h This is our sharpest form for the error. We can also say that the error in the formula is approximately " 2! f Ÿx 0 h, since 8 is between x 0 and x 0 h and so is close to x 0 if h is small, as we customarily assume. We also say that the error is OŸh, read as order h or big oh of h, meaning that the error is approximately proportional to h when h is small. An method with OŸh error is also sometimes referred to as a first order method. We apply the same error analysis to the so-called central difference formula: f Ÿx 0 X fÿx 0 h " fÿx 0 " h 2h fÿx 0 h " fÿx 0 " h 2h 2h fÿx 0 h fÿx 0 f Ÿx 0 h 2! f Ÿx 0 h 2 3! f Ÿ3 Ÿ8 h 3 " fÿx 0 " h fÿx 0 " f Ÿx 0 h 2! f Ÿx 0 h 2 " 3! f Ÿ3 Ÿ8 " h 3 2h 2hf Ÿx 0 3! f Ÿ3 Ÿ8 h 3 3! f Ÿ3 Ÿ8 " h 3 f Ÿx 0 3! 2 f Ÿ3 Ÿ8 f Ÿ3 Ÿ8 " h 2 f Ÿx 0 3! f Ÿ3 Ÿ8 h 2 And so we have for the error f Ÿx 0 " fÿx 0 h " fÿx 0 " h 2h " 3! f Ÿ3 Ÿ8 h 2 Above, we have distinguished between the two values 8 and 8 " that appear in the Taylor expansions of fÿx 0 h and fÿx 0 " h respectively; then 2 f Ÿ3 Ÿ8 f Ÿ3 Ÿ8 ", as the average of these two third derivatives at 8 and 8 ", must be equal to f Ÿ3 Ÿ8, where 8 is a point lying between 8 and 8 " (assuming, as we always do, continuity of f Ÿ3 Ÿx ). And finally we can say that 8 lies between x 0 h and x 0 " h. Numerical Analysis Spring 200 Prof Diamond 2

3 The central difference formula is thus seen to be an OŸh 2 (or second order) method for calculating an approximation of f Ÿx 0, and the error will be smaller than that of the forward difference formula for small values of h. You can see in the derivation of the error that we got lucky in that the h 2 " terms in the Taylor series canceled, squeezing out an additional power of h in the error. Derivation of approximation formulas via the exact-for-polynomials property: Here we consider general derivative approximation formulas of the form f Ÿx ' X h a 0fŸx 0... a m fÿx m where the a i are fixed coefficients and the points x 0,..,x m are close to x ' : x i x ' c i h. (not necessarily equally spaced). In principle, to analyze the error we can then expand the right-hand side in a Taylor polynomial about x ' in powers of h ; these expansions will take the form a 0 fÿx 0... a m fÿx m f Ÿx ' terms of form f Ÿk Ÿ8 h k for some value k u, with the coefficients a i, i 0,.., m chosen so as to ensure that the missing terms with powers h j for t j k vanish. (In our examples we make k as large as possible, so that m k is generally the case, but this is not strictly necessary.) Note that the terms involving f Ÿk Ÿ8 will appear with powers of h k in formulas for f Ÿx ' because of the form of our formula, with the leading factor of h. That this form is appropriate can be seen by noting the first two terms in the expansion: h a 0fŸx 0... a m fÿx m h! m 0 a i f Ÿx! ' m Ÿx 0 a i " x ' i h... so that! 0 m a i 0 must be true and! 0 m c i a i must also be true. The other equations that must be satisfied by the a i to make the coefficients vanish are homogeneous equations (zero right hand side), so that the a i satisfy equations independent of h. Numerical Analysis Spring 200 Prof Diamond 3

4 Here we make an important observation: If an approximation method has error OŸh k then its Taylor polynomial expansion has the form f Ÿx ' terms of form f Ÿk Ÿ8 h k. That method is therefore seen to be exact for all polynomials of degree k or less (this simply means that the approximation method will give the exact value of f Ÿx ' if f happens to be a polynomial of degree k or less). This observation is obvious since, by assumption, the error terms involve f Ÿk Ÿ8, which is zero for a polynomial of degree k or less. The converse is also true: If we have a method for f Ÿx exact for polynomials of degree k or less, then the Taylor expansion of the method has the form f Ÿx ' terms of form f Ÿk Ÿ8 h k and so the method is an OŸh k method. This converse is a bit tedious to write down but basically follows from the fact that the terms in the expansion before those of the form f Ÿk Ÿ8 h k are completely determined by the action of the formula on polynomials of degree k or less, and so if we assume a formula exact for such polynomials, then f Ÿx ' is the only surviving term before those of the form f Ÿk Ÿ8 h k, i.e. the error is OŸh k. To summarize these observations: A formula h a 0fŸx 0... a m fÿx m for f Ÿx ' has error OŸh k if and only if it is exact for polynomials of degree k. We now note the following: Any distinct k points x 0,.., x k, with x i x ' c i h, can be used to produce a formula for f Ÿx ' that is exact for polynomials of degree k or less. (Though it s possible that one or more of the coefficients may be zero in the formula.) This formula is given by the result of interpolating the function values fÿx i with a polynomial p k of degree k, and then calculating p k Ÿx '. (Clearly this method will produce the exact answer if fÿx is a polynomial of degree k or less, for in that case p kÿx fÿx will be true identically.) The Numerical Analysis Spring 200 Prof Diamond 4

5 formula thus obtained has i) error OŸh k ii) an error that can be bounded by CM kh k where C is a constant and M k max f Ÿk Ÿt where t ranges between the smallest and largest of the values x ', and x i, i 0,.., k. ii) an error which, for small values of h, is approximately given by C f Ÿk Ÿx ' h k, where C is a constant equal to Ÿk!h k error in f Ÿx ' for fÿx Ÿx " x ' k (which, since it is a constant can be evaluated for any convenient value of h ) One should note that this process - whereby we interpolate function values and then take the derivative of the interpolating polynomial - does lead to a formula of the assumed form f Ÿx ' X h a 0fŸx 0... a m fÿx m. For recall that p kÿx, in terms of the Lagrange fundamental polynomials, is given by p kÿx k! i0 fÿx i L i Ÿx, and then p k Ÿx ' k! i0 fÿx i L i Ÿx '. An examination of L i Ÿx ', along with the fact that x i x ' c i h, will show that L i Ÿx ' h a i for some constant a i independent of h, which of course we can then identify as the coefficient of fÿx i in the approximation formula. We now provide some examples and methods of deriving derivative approximation formulas using the above ideas. Method : Developing formulas using polynomial interpolation, either Newton form or Lagrange form Ex: 3-point forward difference formula for f Ÿx 0 using fÿx 0,fŸx 0 h,fÿx 0 2h - rename the points x 0,x x 0 h,x 2 x 0 2h Newton form: Numerical Analysis Spring 200 Prof Diamond 5

6 p 2 Ÿx fÿx 0 f x 0,x Ÿx " x 0 f x 0,x,x 2 Ÿx " x 0 Ÿx " x p 2 Ÿx 0 f x 0,x f x 0,x,x 2 Ÿx 0 " x ; expand the divided differences to get the explicit form f Ÿx 0 X p 2 Ÿx 0 p 2 Ÿx 0 h " 3 2 fÿx 0 2fŸx " 2 fÿx 2 Lagrange form: If we write the interpolating polynomial in Lagrange form, the explicit form of the derivative approximation formula is directly obtained p 2 Ÿx fÿx 0 Ÿx " x Ÿx " x 2 Ÿx 0 " x Ÿx 0 " x 2 fÿx Ÿx " x 0 Ÿx " x 2 Ÿx " x 0 Ÿx " x 2 fÿx 0 Ÿx " x 0 Ÿx " x Ÿx 2 " x 0 Ÿx 2 " x Then Recall that x x 0 h,x 2 x 0 2h so, for instance, x " x 0 h,x 0 " x 2 "2h, etc. p 2 Ÿx " 2h 2 fÿx 0 Ÿx " x Ÿx " x 2 " h 2 fÿx Ÿx " x 0 Ÿx " x 2 2h 2 fÿx 0 Ÿx " x 0 Ÿx " x Then using the product rule on each term, we obtain p 2 Ÿx 0 " 2h 2 fÿx 0 "2h " h " h 2 fÿx "2h 2h 2 fÿx 0 "h f Ÿx 0 X p 2 Ÿx 0 h " 3 2 fÿx 0 2fŸx " 2 fÿx 2 is, once again, our three point forward difference formula. Method 2: sing undetermined coefficients, the power functions, and h to determine the formula. Here we make another useful observation: If F Ÿ0 a 0 FŸc 0 a FŸc... a k FŸc k is true for all polynomials of degree k or less (where a i s and c i s are fixed numbers) then the derivative approximation formula f Ÿx ' X h a 0fŸx ' c 0 h a fÿx ' c h... a k fÿx ' c k h Numerical Analysis Spring 200 Prof Diamond 6

7 is exact for all polynomials of degree k. Thus we only need a derivative formula that works for the specific choice of x ' 0 and a specific set of c i s in order to obtain a formula that works for a general x' and a general h. To prove this result, given a function fÿx, define the function FŸt fÿx ' ht.iffÿx is a polynomial of degree k in x, then FŸt is a polynomial of degree k in t and so F Ÿ0 a 0 FŸc 0 a FŸc... a k FŸc k will be true. Replacing the terms in this expression by their equivalents using f, we obtain F Ÿ0 hf Ÿx ' a 0 fÿx ' c 0 h a fÿx ' c h... a k fÿx ' c k h whence we obtain the derivative formula exact for any polynomial f of degree k. We use this in connection with the so-called method of undetermined coefficients to rederive the three point forward difference formula. If we want a formula of the form f Ÿx 0 X h a 0fŸx 0 a fÿx 0 h a 2 fÿx 0 2h to be exact for all polynomials of degree 2, then we need only find a formula F Ÿ0 a 0 FŸ0 a FŸ a 2 FŸ2 that is exact for F any polynomial of degree 2. In turn, this will be the case if we make the formula work for FŸx x k, k 0,, 2. Substituting in to our desired form, we obtain: x 0 : 0 a 0 a a 2 x : a 2a 2 x 2 : 0 a 4a 2 and solving gives a 0 " 3 2,a 2,a 2 " 2 and our formula is rederived. Finally we given an example of the calculation of the leading term in the error expansion. Recall that if our formula is exact for polynomials of degree k, then the leading term in the error expansion (that is, the first nonzero term in the Taylor polynomial about x x ' of the error ) has the form Cf Ÿk Ÿx ' h k. It is easy to determine the value of C:itis Numerical Analysis Spring 200 Prof Diamond 7

8 independent of all three of x ',f,h ; if we pick x ' 0,h, and fÿx x k then the error is exactly equal to Cf Ÿk Ÿ0 CŸk! (since all higher derivatives of f are identically zero). In other words, choosing x ' 0, h and fÿx x 3, we have e Ÿk!C. Applying this idea to the 3-pt forward difference formula just derived, with k 2, we have e 0 " " " C, soc 3 and the error in the forward difference formula satisfies eÿh X 3 f Ÿ3 Ÿx ' h 2. Self-validating error estimation: In many applications the estimation of the error using the derivative formula may be infeasible either because the appropriate derivative is complicated to estimate or is unknown. However, as we ll see, simply the knowledge that the error is approximately proportional to h k for small h will be able to give us an estimate of the error. This idea works in many situations, not just for approximation of derivatives. To treat the problem in general, suppose the exact value we are trying is approximate is y ' and that AŸh represents our computed approximation at h. We can estimate the error if we have the approximations AŸh and AŸh/2. We have y ' X AŸh Ch k and y ' X AŸh/2 CŸh/2 k. Eliminating y ', we can then obtain the approximation AŸh/2 " AŸh 2 k " X CŸh/2 k as an estimate of the error in the approximation AŸh/2 (the better approximation). This is a more sophisticated/refined/sharper error criterion than the usual seat of the pants criterion which would roughly say that we keep halving the size of h until two successive calculations, namely AŸh and AŸh/2 differ by less than some preset tolerance. But it also shows that the rough criterion is fairly reasonable. Here is an example using the 3-point forward difference formula to calculate the derivative of sinÿx at x =/3, along with the self-validating error estimate. Numerical Analysis Spring 200 Prof Diamond 8

9 Note: Exact error decreases by factor of about /4 every time h is halved. This reflects the fact that we are using an order 2 method. Note: If we did not know the order of the method, we could estimate it from the ratio ŸAŸh/4 " AŸh/2 /ŸAŸh/2 " AŸh X /4 obtained from three approximations of fÿx ' at successively halved h s. h 3pt fwd diff. Error estimate exact error NaN We see that our error estimate gives a fairly reliable order-of-magnitude estimate and is quite accurate at small values of h. Second and higher-order derivatives: For second derivatives the appropriate formula takes the form f Ÿx ' h 2 a 0fŸx 0...a k fÿx k. Here we can say: A second derivative formula exact for polynomials of degree k or less has error OŸh k" and leading error term of the form Cf Ÿk Ÿx h k". We also have the following: If a formula Numerical Analysis Spring 200 Prof Diamond 9

10 F Ÿ0 a 0 FŸc 0 a FŸc... a k FŸc k is correct for all polynomials of degree k or less, then the formula f Ÿx ' X a 0 fÿx ' c 0 h a fÿx ' c h... a k fÿx ' c k h is exact for all polynomials of degree k or less. The most basic formula here is the formula based on the central second difference : f Ÿx 0 X h 2 fÿx 0 h " 2fŸx 0 fÿx 0 " h 2f x 0 " h,x 0,x 0 h If we consider the formula for x 0 0,h, then the formula F Ÿ0 FŸ " 2FŸ0 FŸ" is easily verified to be exact for all polynomials of degree 3 (one higher than expected - a bonus due to the symmetry of the formula - the formula can be considered a formula on four points, say ", 0,, 2 where the coefficient of FŸ2 turns out to be zero). The leading error term in f Ÿx 0 X h 2 fÿx 0 h " 2fŸx 0 fÿx 0 " h is Cf Ÿ4 Ÿx 0 h 2 and using x 0 0, h and fÿx x 4, we calculate e 0 " 2 Ÿ4! C so C " 2 and eÿh " 2 f Ÿ4 Ÿx 0 h 2... gives the leading error term in powers of h. Analogous theorems are easily stated for higher derivatives. Numerical Analysis Spring 200 Prof Diamond 0