NUMERICAL DIFFERENTIATION

Transcription

1 NUMERICAL IFFERENTIATION FIRST ERIVATIVES Te simplest difference formulas are based on using a straigt line to interpolate te given data; tey use two data pints to estimate te derivative. We assume tat we ave function values at x ( or xi ), x ( or xi) and x + ( or xi+ ) ; we let f( x ) = y( x ) ( or f( xi ) = yi ), f( x) = y( x) ( or f( xi) = yi) and f( x+ ) = y( x+ ) ( or f( xi+ ) = yi+ ). Te spacing between te values of x is constant (see figure below), so tat xi xi = and xi xi+ =. x ( x ) x ( x ) x + ( x + ) i i i Fig. Forward difference formula (first derivative) x x + Consider te Taylor series expansion f( x+ ) = f( x) + f ( x) + f ( x) + f ( x) +...!! were, f( x+ ) f( x) f ( x) = f( x+ ) f( x) f ( x) f ( x) +... f ( x) = f ( x) f ( x) +...!!!! f( x+ ) f( x) f ( x) = + Etrunc( f, ) were forward difference formula for first derivative and truncation error as follows f( x+ ) f( x) f ( x) = and Etrunc( f, ) = f ( x) = O( ) (order )!

2 Backward difference formula (first derivative) x x Consider te Taylor series expansion f( x ) = f( x) f ( x) + f ( x) f ( x) +...!! were, f( x) f( x ) f ( x) = f( x) f( x ) + f ( x) f ( x) +... f ( x) = + f ( x) f ( x) +...!!!! f( x) f( x ) f ( x) = + Etrunc( f, ) were backward difference formula for first derivative and truncation error as follows f( x) f( x ) f ( x) = and Etrunc( f, ) = f ( x) = O( ) (order )! Central difference formula Teorem: (centered formula for O ( )). f C a, b and tat x, x + a, b ten Assume tat [ ] [ ] f ( x+ ) f( x ) f ( x) = (central difference formula for first derivative) Furtermore, tere exist a number c c( x) [ a, b] = suc tat () f( x+ ) f( x ) f ( c) f ( x) = + Etrunc( f, ) were Etrunc( f, ) = = O( ) Te term E(f,) is called truncation error.

3 Proof Start wit second degree Taylor expansions f ( x) = P( x) + E( x), about x, for f ( x + ) and f ( x ) () f( x+ ) = f( x) + f ( x) + f ( x) + f ( x) + f ( x) +...!!! () f( x ) = f( x) f ( x) + f ( x) f ( x) + f ( x) +...!!! f( x+ ) f( x ) = f ( x) + f ( x) +...! f ( x) = f ( x + ) f ( x ) f ( x) +... ten f( x+ ) f( x ) f ( x) = f ( x) +... f ( x+ ) f( x ) Central difference formula f ( x) = Truncation error E( f, ) = f ( x) = O( ) Teorem : (centered formula of order O ( ) ). f C 5 a, b and tatx, x, x+, x+ a, b ten Assume tat [ ] [ ] f ( x+ ) + 8 f ( x+ ) 8 f ( x ) + f ( x ) f ( x) = (*) Furtermore tere exists a number c c( x) [ a, b] = suc tat f( x+ ) + 8 f( x+ ) 8 f( x ) + f( x ) f ( x) = + Etrunc ( f, ) Were (5) E( f, ) = f ( c) = O( ) 0

4 Proof One way to derive formula (*) is as follows. Start wit te difference between fourt degree Taylor expansions f ( x) = P( x) + E( x), about x, for f ( x+ ) and f ( x ) 5 () (5) f( x+ ) = f( x) + f ( x) + f ( x) + f ( x) + f ( x) + f ( x)!!! 5! 5 () (5) f( x ) = f( x) f ( x) + f ( x) f ( x) + f ( x) f ( x)!!! 5! 5 (5) f( x+ ) f( x ) = f ( x) + f ( x) + f ( x)...( ) 5! Ten use step size, instead of, and write down te following approximation 5 ( ) ( ) (5) f( x+ ) f( x ) = ( ) f ( x) + f ( x) + f ( x) 5! 5 (5) f( x+ ) f( x ) = f ( x) + f ( x) + f ( x)...( ) 5! Next multiply te terms in equation (**) by 8 and subtract (***) from it 5 (5) 8 f ( x+ ) f( x ) = f ( x) + f ( x) + f ( x) 5! 5 (5) f( x+ ) f( x ) = f ( x) + f ( x) + f ( x) 5! 8 5 (5) f ( x+ ) + 8 f ( x+ ) 8 f ( x ) + f ( x ) = f ( x) f ( x) ten 5! f( x+ ) + 8 f( x+ ) 8 f( x ) + f( x ) f ( x) = + Etrunc ( f, ) and (5) E( f, ) = f ( c) = O( ) 0

5 ERROR ANALYSIS AN OPTIMUM STEP SIZE An important topic in te study of numerical differentiation is te effect of te computer s round-off error. Let us examine te formulas more closely. Assume tat a computer is used to make numerical computation and tat f ( x ) = y + e and f ( x + ) = y + e 0 0 were f ( x0 ) and f ( x0 + ) are approximated by te numerical values y and y, and e and e are te associated round-off errors, respectively. Te following result indicates te complex nature of error analysis for numerical differentiation. COROLLARY: Assume tat f satisfies te ypoteses of and use computational formula f f ( x + ) f( x ) 0 0 ( x0) = () were y y f x E f ( 0) + (, ) () () () e e f c E( f, ) = Eround ( f, ) + Etrunc ( f, ) = () were te total error term E( f, ) as part due to round-off error plus a part due to truncation error. () COROLLARY: If e ε, e ε and M max { f ( x) } = ten (from equation ()) a x b e + e M ε + ε M ε M E( f, ) + = + = + let total error ε M Ψ ( ) = E( f, ) () and setting Ψ ( ) = 0 and te value of tat minimizes te rigt and side of () ε M M ε ε Ψ ( ) = 0 + = 0 = = M ε = M is te optimum step size 5

6 Example (E.Q) :erive te numerical differentiation formula f0 + f f f( x0) + f( x0 + ) f( x0 + ) f ( x0 ) = = If f ( x) is given by f ( x) = sinx, find te optimum step size to estimate te derivative at te point π x =, provided tat te upper bound of te rounding error is given by e k 8 0 compare te π result wit exact value of te derivative at x =. Solution x 0 (0) x 0 + () x0 + () Taylor series expansion for f ( x0 + ) and f ( x0 + ) is as follows () ( f( x + ) = f( x ) + f ( x ) + f ( x ) + f ( x ) + f ( x ) +... )!!! 8 () ( f( x + ) = f( x ) + f ( x ) +! f ( x ) +! f ( x ) +! f ( x ) +...) + f( x + ) f( x + ) = f( x ) + f ( x ) f ( x) +...! ten f( x0) + f( x0 + ) f( x0 + ) 0 = + 0 f ( x ) f ( x ) Etrunc = f ( x0) were e0 + e e E( f, ) = Eround ( f, ) + Etrunc( f, ) = + f ( x)

7 e0 + e + e E( f, ) + f ( x) find M, f ( x) = sin x, f ( x) = cos x, f ( x) = sin x and f ( x) = cosx { } { } M = max f ( x) = max cos x so M = and 0 x π 0 x π ε + ε + ε 8ε E( f, ) + = + 8ε ε let Ψ ( ) = + and Ψ ( ) = + = 0 ε = = ε and = ( ε ) were ε = 8 0 ten ( ( )) = 8 0 = 0.0 Ten find π π π f( ) + f( + 0.0) f( + (0.0)) π f ( ) = (0.0) π π π sin + sin sin + (0.0) = 0.07 π f ( ) = 0.70 Exact value: f ( x) = sin x and f ( x) = cosx ten π π f = cos =

8 Example: erive te numerical differentiation formula f ( x) = f ( x) + f( x+ ) f( x+ ) Evaluate te derivative at te point x= using te above formula for te function f ( x) = x x suc tat te rounding error sould not exceed ε = 0 and te total error bounded by 0 and verify te result. Solution From previous example f( x) + f( x+ ) f( x+ ) f ( x) = + f ( x) Etrunc = f ( x) e0 + e e E( f, ) = Eround ( f, ) + Etrunc( f, ) = + f ( x) e0 + e + e E( f, ) + f ( x) 0 and ε + ε + ε 8ε E( f, ) + f ( x) 0 E( f, ) + f ( x) 0 Find M, were f( x) = x x, f ( x) = x, f ( x) = and f ( x) = 0 tat is M = 0 8ε E( f, ) were ε = E( f, ) 0 0. f() + f( + 0.) f( + (0.)) () + (.08). f () = = = (0.) 0.8 Exact value : f ( x) = x f () = = 8

9 EXERCISE : Te local truncation error of f + f 0 f + f f f ( x ) = () is Etrunc( f, ) = f ( c) Find te best value of to approximate f ( x) bounded by x, for f ( x) e over [,] = wit rounding error EXERCISE : erive te numerical differentiation formula f0 f f f( x0) f( x0 ) f( x0 ) f ( x0 ) + + = = and determine te truncation error EXERCISE : erive te following numerical differentiation formula / f ( x+ ) + 8 f ( x+ ) 8 f ( x ) + f ( x ) f ( x) = and define te order of approximation. 9

10 RICARSON EXTRAPOLATION Te basic idea beind extrapolation is tat wenever te leading term in te error for an approximation formula is known, we can combine two approximations obtained from tat formula using different values of te parameter to obtain a iger order approximation. Tis process will be illustrated in tis section for finite difference approximations to derivatives, te tecnique is known as RICARSON EXTRAPOLATION. Let s consider te second order central difference formula for first derivative f( x0 + ) f( x0 ) 0 f ( x ) = f ( ξ ) () For notational convenience, let denote te true value of te derivative, and let denote te approximation obtained using a step size. Were x0 < ξ < x0 + and from squeeze teorem guarantees ξ 0 as 0 terefore, as 0 = f ( ξ ) f ( x 0) ten = + O( ) Let s now look at te error term more precisely. Since f ( ξ) = f ( x0) + f ( ) f ( x0) we see tat 0 lim lim 0 0 [ ξ ] f ( x ) = [ f ( ξ ) f ( x0) ] = 0 terefore = + K + O( ) were K = f ( x0) As started earlier, te process of extrapolation uses two approximations computed from te same formula, but wit different values of, to obtain a iger order approximation. For te second order approximation (central difference) to f we ave just establised. = + K + O( ) ence = + K + O( ) Upon subtracting tese two expressions, we obtain 0

11 0 = + K + O( ) Wic can easily be solved for K te leading term in te error of our original approximation. K = + ( ) O We now establis for K in te original approximation to obtain = + K + O( ) = + + ( ) O = + O( ) Te formula is a iger order approximation of for te first derivative in te sense tat it converges faster tan, as opposed to at te same rate as. Example: Let s consider te approximation of te derivative function f ( x) = lnx using te second order central difference approximation wit x0 = and = 0.. f( x0 + ) f( x0 ) = f ( x0) =, tat is f( + 0. ) f( 0. ) ln(. ) ln(. 9) = f ( ) = = = = (. ) and f( x0 + ) ( 0 ) ( 0) f x = f x =, tat is ( ) f( ) f( 0. 05) ln(. 05) ln(. 95) = f ( ) = = = = (. ) Extrapolation formula ( ) ( ) ( ) ( ) = = = Exact value: f( x) = ln x, f ( x) = ten f ( ) = = 05. x Exact value : 0.5 Central difference approximation : error: Extrapolation : error:

12 REPEATE EXTRAPOLATION t In general, if a p order formula is extrapolated by cutting te step size by a factor b, te extrapolation formula will take te form p b b b p Example: from previous example We found nd order approximation ( ) ( ) ( ) 0. ( ) = = = If we apply t order approximation ( ) ( ) ( ) ( ) ( ) = = fourt order approximation 5 etermine ( ) 005. ten () () () () ( ) = = ( ) f( ) f( 0. 05) ln. 05 ln. 975 = = = ( 0. 05) () () ( ) ( ) = = = ( ) ( ) ( ) ( ) = = = wit error error = = 0 0

13 Wen performing repeated extrapolation, it is convenient to organize te calculations into an extrapolation table like te following one, Step size p ( p O ) ( p O ) ( p O ) O ( ) ( ) ( ) () ( ) () () 8 ( ) 8 () () () Example: Te first order forward difference approximation is = f ( x0 + ) f( x0) Starting =, approximate te value of te first derivative of f( x) = tan x at x0 = applying extrapolation. Use four rows your extrapolation table and find te error for final approximation Solution: Calculate first column for =, =0.5, =0.5 and =0.5 respectively. Were f( x0 + ) f( x0) = and x = () () f( + ) f( ) tan tan = = = = ( ) f( ) f( ) tan 5. tan = = = = ( ) f( ) f( ) tan. 5 tan = = = = = 05. = f ( ) f ( ) tan 5. tan = =

14 Extrapolation formula p ( k ) ( k ) b p b ( k) = were b= b Find (), () and () using te previous calculation (use first column) p = () () () (0.8) = = = () () () (0.897) = = = () () () ( ) = = = Find p = (), () using te previous calculation (use second column) () () () ( ) = = = ( ) = = = () () () Finally find () p = () () () 8( ) = = =

15 Step size O ( ) p= O ( ) p= O ( ) p= O ( ) p= = ( ) = = = 0.5 ( ) 05. )=0.8 () = ( ) 05. = = 0.5 =0.897 () = () = ( ) 05. = = = () = () = () =