Lecture 21. Numerical differentiation. f ( x+h) f ( x) h h

Transcription

1 Lecture Numerical differentiation Introduction We can analytically calculate te derivative of any elementary function, so tere migt seem to be no motivation for calculating derivatives numerically. However we may need to estimate te derivative of a numerical function, or we may only ave a fixed set of sampled function values. In tese cases we need to estimate te derivative numerically. Finite difference approximation Te definition of te derivative of a function f ( x ) tat you will most often find in calculus textbooks is f ( x+) f (x ) df lim dx () Tis immediately suggests te approximation f ( x+) f ( x) df Δ f dx Δ x () were te step size is small but not zero. Tis is called a finite difference. Specifically it's a forward difference because we compare te function value at x wit its value at a point forward of tis along te x axis, x+. How small sould be? Because of round-off error, smaller is not always better. Let's use Scilab to estimate d x e dx x e e (3) for various values. Te absolute error in te estimate vs. is graped in Fig.. As decreases from down to 8 te error decreases also. However for 9 and smaller te error actually increases! Te culprit is round-off error in te form of te small difference of large numbers. Double precision aritmetic provides about 6 digits of precision. If 6 ten e e to 6 digits and te difference e e will be very inaccurate. Wen 8 te difference e e will be accurate to about 8 digits or about 8, te point at wic teoretical improvement in numerical accuracy is offset by iger round-off error. We typically ignore round-off error wen estimating numerical accuracy, but round-off error needs to be kept in mind wen implementing any algoritm. Let's investigate ow te numerical accuracy of our estimate varies wit step size. Assume we want to estimate te derivative of f (x ) at x. Write f as a power series f (x) f ()+ n EE Numerical Computing (n ) f () x n n! (4) 5-8-8

2 Lecture : Numerical differentiation / x Fig. : Error in forward-difference estimation of derivative of e at x vs. step size. 8 As decreases from to, numerical error decreases proportionally. As decreases furter, round-off error begins to exceed numerical error. Ten f () f () (n) f () n f ()+ f ()+ f () + n! 6 n (5) For small terefore f () f () f ()+ f () + (6) and we say tat te approximation is first-order accurate since te error (te second term on te rigt) varies as te first power of. Decreasing by a factor / will decrease te error by /. However, as we see in Fig. tis is only true up to te point tat round-off error begins to be significant. For double precision 8 is optimal. We can sift (6) along te x axis and rearrange to obtain te general forward-difference formula f (x ) f ( x+) f ( x) +O() (7) 3 Higer-order formulas Te forward-difference approximation (6) uses two samples of te function, namely f () and f (). Using tree samples we migt be able to get a better estimate of f (). Suppose we ave te samples f ( ), f (), f (). In terms of te power series representation of te function tese are EE Numerical Computing 5-8-8

3 Lecture : Numerical differentiation 3/ f ( ) f ( ) f () + f () f ( ) f () f ( )+ f ( ) + f () f (3) ( ) 3+ f (4) () 4 f (5 )( )5+ (8) f () + f (3) () 3+ f (4 ) ()4 + f (5) ()5+ Let's use a linear combination of tese values to estimate f () as f ()a f ( )+b f ( )+c f ( ) (9) were a, b, c are unknown coefficients tat we will coose to get te best possible estimate. Te sum a f ( )+b f ()+c f ( ) will include a term wit a factor of f (). We want tis to vanis. Tis requires (a+b+c) f ( ) a+b+c () wic is one equation in tree unknowns. Terms wit a factor of f () sould combine to give te derivative value f (). Tis requires ( a+c) f () f () a+c () We now ave two equations in tree unknowns. To get a tird equation we can require te next term, wic contains a factor of f (), to vanis. Tis gives us te equation (a+c) f ( ) a+c () Our tree equations in tree unknowns can be written as ( )( ) ( ) a b c Te solution is a, b, c and our approximation reads f () f ( ) f ( ) Using (8) we ave f () f ( ) f ()+ f (3) () + 6 so tis approximation is second-order accurate. Decreasing by a factor of / sould decrease te numerical error by a factor of /. Rearranging and writing tis for an arbitrary value of x we ave te formula EE Numerical Computing 5-8-8

4 Lecture : Numerical differentiation 4/ x Fig. : Error in central-difference estimate of derivative of e vs.. f (x ) f ( x+) f ( x ) +O( ) (3) Tis type of finite difference is called a central difference since it uses bot te forward sample f (x+) and te backward sample f (x ). Scilab code is given in te Appendix. Te error in te central-difference approximation d x e dx x e e is plotted in Fig.. Note ow te error reduces more rapidly wit decreasing. Tis allows te approximation to reac a greater accuracy before round-off error starts to become significant. Wit 5 te error is only about. We extend tis idea by using even more function samples. If we ave te five samples f ( ), f ( ), f (), f ( ), f ( ) we can form an estimate f ()a f ( )+b f ( )+c f ()+d f ()+e f ( ) (4) Tis as five unknowns, so we need to form five equations. In terms of te Taylor series representation of f ( x ) our five samples ave te form EE Numerical Computing 5-8-8

5 Lecture : Numerical differentiation f ( ) 8 (3) 6 (4 ) 3 (5) f () 3+ f ()4 f () 5+ f ( ) f () + f () f (3) () 3+ f (4) () 4 f (5) ( ) 5+ (5) f ( ) f ( )+ f () + f () + f (3) () 3+ f (4) () 4 + f (5) () (4) 3 (5) f ( )+ f () + f ( ) + f (3) ()3+ f () 4+ f () 5+ f ( ) f () f ( ) f ( ) f ()+ f () f () 5/ To get te f () terms in (4) to vanis requires (a+b+c+d +e) f () a+b+c+d +e (6) To get te f () terms to produce te value f () requires ( a b+d + e ) f () f ( ) a+b+d + e (7) Te remaining tree equations are obtained by requiring te f (), f (3) () and f (4 )( ) terms to vanis: ( ) b d a+ + + e f () 4 a+b+d +4 e ( 8 a b+d +8 e ) f (3) ()3 8 a b+d +8 e ( 6 a+b+d +6 e ) f (4) () 4 6 a+b+d +6 e ( 6+b+d +6 e ) f (4) ( ) 4 6 a+b+d +6 e Our five equations in five unknowns form te system ( )( ) ( ) a b c d e (8) wic as te solution a 8 8, b, c, d, e (9) Our approximation is terefore f () EE Numerical Computing f ( ) 8 f ( )+8 f () f ( ) () 5-8-8

6 Lecture : Numerical differentiation 6/ x Fig. 3: Fourt-order accurate central-difference approximation to te derivative of e. Using (5) te f (5) () terms are f (5) 5 () ( ) f (5) () 4 3 () so f ( ) 8 f ( )+8 f () f ( ) (5) 4 f () f () + 3 () We see tat tis is a fourt-order accurate approximation. Decreasing by a factor of / sould decrease te numerical error by a factor of /,. Tis is illustrated in Fig. 3 for x 3 4 f (x )e. For te error is only about. For arbitrary x value our fourt-order central difference approximation is f ( x) f (x ) 8 f ( x )+8 f ( x+) f ( x+ ) 4 +O( ) (3) Scilab code is given in te Appendix. 4 Te numderivative function (Scilab) Scilab as a numerical derivative function named numderivative. To calculate te derivative of te function f ( x ) at xx we execute fp numderivative(f,x); If f is a function several variables f (x), ten numderivative will return te gradient of f. It is also possible to specify te step size and te order of te approximation (, or 4). fp numderivative(f,x,,order); EE Numerical Computing 5-8-8

7 Lecture : Numerical differentiation 7/ Te default is second order (central difference) and Scilab cooses an optimal value of. Some examples: -->deff('yf(x)','yexp(-x)'); -->numderivative(f,) //default central difference wit optimal ans //exact value is -exp(-) >(f(.)-f())/. //forward difference. ans >numderivative(f,,.,) //forward difference. ans >(f(.)-f(.9))/. //central difference. ans >numderivative(f,,.,) //central difference. ans In most cases te default suffices. 5 Second derivative Wit reference to (8), if we want to approximate te second derivative f () as f ()a f ( )+b f ( )+c f ( ) (4) (a+b+c) f ( ) a+b+c (5) we would require ( a+c) f ( ) a+c (6) and (a+c) f () f () a+c (7) Te solution to tese tree equations is ac, b (8) and we find f () f ()+ f ( ) f ()+ (4) f ( ) + (9) so te approximation is second-order accurate. For arbitrary x value we ave EE Numerical Computing 5-8-8

8 Lecture : Numerical differentiation 8/ f (x) f ( x+) f ( x)+ f ( x ) +O ( ) (3) 6 Partial derivatives For a function f (x, y ) te partial derivative can be defined as x f ( x+, y ) f (x, y) lim x tat is, we old y fixed and compute te derivative as if f was only a function of x. A centraldifference approximation is f ( x+, y) f ( x, y) x (3) f ( x, y+) f ( x, y ) y (3) f f ( x+, y) f ( x, y )+ f ( x, y ) x (33) Likewise and f Mixed partial derivative approximations suc as can be developed in steps suc as y x f y x [ ] [ ] x y+ x y f (x+, y+) f ( x, y+) f (x+, y ) f ( x, y ) f ( x+, y+) f (x, y+) f (x+, y )+ f ( x, y ) 4 (34) 7 Differential equations 7. Ordinary differential equations An ordinary differential equation (ODE) relates a single independent variable, e.g., x, to a function f (x ) and its derivatives f (x ), f (x),. Most pysical laws are expressed in terms of differential equations, ence teir great importance. Certain classes of ODEs can be solved analytically but many cannot. In eiter case our derivative formulas can be used to develop numerical solutions. Suppose a pysical problem is described by a differential equation of te form EE Numerical Computing 5-8-8

9 Lecture : Numerical differentiation 9/ f + f +7 f (35) x (36) One can verify tat f ( x )e cos(4 x) solves (35) by taking derivatives and substituting into te equation. A numerical approximation to (35) is given by (using (3) and (3)) f ( x+) f ( x)+ f ( x ) f (x+) f ( x ) + +7 f ( x) (37) Solving tis for f (x+) we obtain f (x+) ( 7 ) f (x ) ( ) f ( x ) + (38) Let's use tis to calculate f (x ) for x,,, 3,. To get started we need te first two values f ( x ), f ( x )e cos (4 ) (39) Ten we can apply (38) to get f (x 3x +), f ( x 4 x3 +) and so on as long as we wis. In Scilab tis looks someting like.; x ::5; f() ; f() exp(-)*cos(4*); for i:n- y(i+) ((-7*^)*y(i)-(-)*y(i-))/(+); end Te resulting numerical solution and te exact solution are sown in Fig. 4. Te agreement is excellent. Function odecentdiff in te Appendix uses tis idea to numerically solve a second-order equation of te form y + p( x ) y +q ( x) yr ( x) Given and initial x value x, a step size and te two function values y ( x ) and y (x +). Fig. 4 compares te numerical solution of (35) using odecentdiff wit te exact solution y f ( x )e x cos (4 x ) (4) for a step size.. 7. Partial differential equations A partial differential equation (PDE) relates two or more independent variables, e.g., x,y, to a f f f f,,,,. One of te most function f ( x, y ) and its partial derivatives x y x x y important PDEs is Laplace's equation EE Numerical Computing 5-8-8

10 Lecture : Numerical differentiation / Fig. 4: Solid line: numerical solution of (35); circles: exact solution. f f + x y (4) Numerically we can write f f + x y f (x+, y ) f ( x, y )+ f ( x, y ) f ( x, y+) f (x, y)+ f ( x, y ) (4) + f ( x+, y)+ f ( x, y)+ f (x, y+)+ f ( x, y ) 4 f ( x, y) Te last expression is zero wen f (x, y ) [ f ( x+, y )+ f (x, y )+ f ( x, y +)+ f ( x, y ) ] 4 (43) wic relates te value f (x, y ) to its neigboring values. Specifically f (x, y ) is equal to te average of its neigbor's values. EE Numerical Computing 5-8-8

11 Lecture : Numerical differentiation / 8 Appendix Scilab code 8. nd order central difference // derivsecondorder.sci // 4--5,, for pedagogic purposes // Numerical estimation of derivative of f(x) using nd-order // accurate central difference and "optimum" step size. function ypderivsecondorder(f, x) e-5*(+abs(x)); //step size scales wit x, no less tan e-5 yp (f(x+)-f(x-))/(*); endfunction 8. 4t order central difference // derivfourtorder.sci // 4--5,, for pedagogic purposes // Numerical estimation of derivative of f(x) using 4t-order // accurate central difference and "optimum" step size. function ypderivfourtorder(f, x) e-3*(+abs(x)); //step size scales wit x, no less tan e-3 yp (f(x-*)-8*f(x-)+8*f(x+)-f(x+*))/(*); endfunction 8.3 Differential equation solver using nd order central difference // odecentdiff.sci // 4--5,, for pedagogic purposes // Uses nd-order accurate central difference approximation to // derivatives to solve ode y''+p(x)y'+q(x)yr(x) // approximations are // y' (y(x+)-y(x-))/() and y'' (y(x+)-y(x)+y(x-))/^ // p,q,r are functions, x is te initial x value, is step size, // n is number of points to solve for, yy(x), yy(x+). function [x, y]odecentdiff(p, q, r, x,, n, y, y) x zeros(n,); y zeros(n,); x() x; x() x()+; y() y; y() y; *; for i:n- p *p(x(i)); x(i+) x(i)+; y(i+) (**r(x(i))+(4-**q(x(i)))*y(i)+(p-)*y(i-))/(+p); end endfunction EE Numerical Computing 5-8-8