NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the dervatve of y wth respect to x. In more precse language, the dependence of y upon x means that y s a functon of x. Ths functonal relatonshp s often denoted y = f(x), where f denotes the functon. If x and y are real numbers, and f the graph of y s plotted aganst x, the dervatve measures the slope of ths graph at each pont. When the functonal dependence s gven as a smple mathematcal expresson, the dervatve can be determned analytcally. When analytcal dfferentaton of the expresson s dffcult or not possble, numercal dfferentaton has to be used. When the functonal dependence s specfed as a set of dscrete ponts, dfferentaton s completed usng a numercal method. For a gven set of ponts two approaches can be used to calculate a numercal approxmaton of the dervatve at one of the ponts Fnte dfference approxmaton: In ths approach we approxmate the dervatve based on values of ponts n the neghborhood of the pont. The accuracy of a fnte dfference approxmaton depends on the accuracy of the data ponts, the spacng between the pont, and the specfc formula used for the approxmaton. In the example below, the frst dervatve at x s approxmated by the slope of the lne connectng the ponts adjacent to x. Ths approxmaton s called Two ponts central dfference approxmaton. df dx x=x = f(x +1) f(x 1 ) x +1 x 1 Two addtonal two ponts fnte dfference approxmatons: Two ponts forward dfference approxmaton where the frst dervatve x s approxmated by the slope of the lne connectng x and x + 1 df dx x=x = f(x +1) f(x ) x +1 x 1
Two ponts backward dfference approxmaton where the frst dervatve x s approxmated by the slope of the lne connectng x and x 1 df dx x=x = f(x ) f(x 1 ) x x 1 The second approach s to approxmate the ponts wth an analytcal expresson that can be easly dfferentated and then to calculate the dervatve by dfferentatng the analytcal expresson. The approxmate analytcal expresson can be derved usng curve fttng. Note: Sometmes when the measured data contan scatter as shown n the fgure below, usng one of the two ponts fnte dfference approxmaton may be erroneous and better results may be obtaned usng a hgher order fnte dfference approxmaton as wll be dscussed later n ths chapter or by usng a curve ft. Ths elmnates the problem of wrongly amplfed slopes between successve ponts. 2
2 Fnte dfference formulas usng Taylor seres expanson 2.1 Taylor seres expanson of a functon Taylor seres expanson of a functon s a way to fnd the value of a functon near a known pont, that s, a pont where the value of the functon s known. The functon s represented by a sum of terms of a convergent seres. In some cases (f the functon s a polynomal), the Taylor seres can gve the exact value of the functon. In most cases, however, a sum of an nfnte number of terms s requred for the exact value. If only a few terms are used, the value of the functon that s obtaned from the Taylor seres s an approxmaton. Gven a functon that s dfferentable (n+1) tmes n an nterval contanng a pont x o, Taylor s theorem states that for each x n the nterval, there exsts a value η between x and x o such that: f(x) = f(x o ) + (x x o ) df dx x=x o + (x x o) 2 d 2 f 2! dx 2 x=x o +... + (x x o) n d n f n! dx n x=x o + R n (η) where the remander R n (x) s gven by R n (η) = (x x o) n+1 (n + 1)! d n+1 f dx n+1 x=η 2.2 Fnte dfference formulas of frst dervatves Several fnte dfference formulas, ncludng the two ponts fnte dfference approxmatons can be derved from the Taylor seres expanson of the functon. These approxmatons vares n the number of ponts nvolved n the approxmaton, the spacng and accuracy of the measured values. We assume the spacng between adjacent ponts to be fxed and equal to h Two ponts forward dfference approxmaton f(x +1 ) = f(x ) + (x +1 x ) df dx x=x + (x +1 x ) 2 d 2 f 2! dx 2 x=η f(x +1 ) = f(x ) + hf (x ) + h2 2! f (η) f (x ) = f(x +1) f(x ) h h 2! f (η) If the second term s gnored, the prevous expresson reduces to the two ponts forward dfference approxmaton. 3
Ignorng the second term ntroduces a truncaton error that s proportonal to h. The truncaton error s sad to be of on the order of h Two ponts backward dfference approxmaton truncaton error = h 2! f (η) = O(h) f(x 1 ) = f(x ) + (x 1 x ) df dx x=x + (x 1 x ) 2 d 2 f 2! dx 2 x=η f(x 1 ) = f(x ) hf (x ) + h2 2! f (η) f (x ) = f(x ) f(x 1 ) h + h 2! f (η) If the second term s gnored, the prevous expresson reduces to the two ponts backward fnte dfference approxmaton. Ignorng the second term ntroduces a truncaton error that s proportonal to h. The truncaton error s sad to be of on the order of h Two ponts central dfference approxmaton truncaton error = h 2! f (η) = O(h) f(x +1 ) = f(x ) + (x +1 x ) df dx x=x + (x +1 x ) 2 d 2 f 2! dx 2 x=x + (x +1 x ) 3 d 3 f 3! dx 3 x=η 1 f(x 1 ) = f(x ) + (x 1 x ) df dx x=x + (x 1 x ) 2 d 2 f 2! dx 2 x=x + (x +1 x ) 3 d 3 f 3! dx 3 x=η 2 f(x +1 ) = f(x ) + hf (x ) + h2 2! f (x ) + h3 3! f (η 1 ) f(x 1 ) = f(x ) hf (x ) + h2 2! f (x ) h3 3! f (η 2 ) f (x ) = f(x +1) f(x 1 ) h + h2 3! f (η 1 ) h2 3! f (η 2 ) If the second term s gnored, the prevous expresson reduces to the two ponts central fnte dfference approxmaton. Ignorng the second term ntroduces a truncaton error that s proportonal to h 2. The truncaton error s sad to be of on the order of h 2 truncaton error = h3 3! f (η 1 ) h2 3! f (η 2 ) = O(h 2 ) A comparson between the last three approxmatons show that for small h, the central dfference approxmaton gves better approxmaton that the forward or backward approxmatons. The central dfference approxmaton s useful only for nteror ponts and not for the end ponts x 1 or x n 4
Three ponts forward dfference approxmaton f(x +1 ) = f(x ) + hf (x ) + h2 2! f (x ) + h3 3! f (η 1 ) f(x +2 ) = f(x ) + 2hf (x ) + (2h)2 2! f (x ) = 3f(x ) + 4f(x +1 ) f(x +2 ) 2h f (x ) + (2h)3 f (η 2 ) 3! 2h2 3! f (η 1 ) + 4h2 3! f (η 2 ) If the second term s gnored, the prevous expresson reduces to the three ponts forward fnte dfference approxmaton. Ignorng the second term ntroduces a truncaton error that s proportonal to h 2. The truncaton error s sad to be of on the order of h 2 truncaton error = 2h2 3! f (η 1 ) + 4h2 3! f (η 2 ) = O(h 2 ) The three ponts forward dfference approxmaton s only useful for ponts = 1 to = n 2 Three ponts backward dfference approxmaton f(x 1 ) = f(x ) hf (x ) + h2 2! f (x ) h3 3! f (η 1 ) f(x 2 ) = f(x ) 2hf (x ) + (2h)2 2! f (x ) = 3f(x ) 4f(x 1 ) + f(x 2 ) 2h f (x ) (2h)3 f (η 2 ) 3! 2h2 3! f (η 1 ) + 4h2 3! f (η 2 ) If the second term s gnored, the prevous expresson reduces to the three ponts forward fnte dfference approxmaton. Ignorng the second term ntroduces a truncaton error that s proportonal to h 2. The truncaton error s sad to be of on the order of h 2 truncaton error = 2h2 3! f (η 1 ) 4h2 3! f (η 2 ) = O(h 2 ) The three ponts backward dfference approxmaton s only useful for ponts = n to = 2 The same approach can be used to derve the four ponts central dfference approxmaton f (x ) = f(x 2) 8f(x 1 ) + 8f(x +1 ) f(x +2 ) 12h + O(h 4 ) 5
2.3 Fnte dfference formulas of second dervatves Three ponts forward dfference approxmaton f(x +1 ) = f(x ) + hf (x ) + h2 2! f (x ) + h3 3! f (η 1 ) f(x +2 ) = f(x ) + 2hf (x ) + (2h)2 2! f (x ) = f(x +2) 2f(x +1 ) + f(x ) h 2 f (x ) + (2h)3 f (η 2 ) 3! 8h 3! f (η 2 ) + h 3! f (η 1 ) If the second term s gnored, the prevous expresson reduces to the three ponts forward fnte dfference approxmaton. Ignorng the second term ntroduces a truncaton error that s proportonal to h. The truncaton error s sad to be of on the order of h Three ponts backward dfference approxmaton truncaton error = 8h 3! f (η 2 ) + h 3! f (η 1 ) = O(h) f(x 1 ) = f(x ) hf (x ) + h2 2! f (x ) h3 3! f (η 1 ) f(x 2 ) = f(x ) 2hf (x ) + (2h)2 2! f (x ) = f(x 2) 2f(x 1 ) + f(x ) h 2 f (x ) (2h)3 f (η 2 ) 3! + 8h 3! f (η 2 ) h 3! f (η 1 ) If the second term s gnored, the prevous expresson reduces to the three ponts backward fnte dfference approxmaton. 6
Ignorng the second term ntroduces a truncaton error that s proportonal to h. The truncaton error s sad to be of on the order of h Three ponts central dfference approxmaton truncaton error = 8h 3! f (η 2 ) h 3! f (η 1 ) = O(h) f(x +1 ) = f(x ) + hf (x ) + h2 2! f (x ) + h3 3! f (x ) + h4 4! f (η 1 ) f(x 1 ) = f(x ) hf (x ) + h2 2! f (x ) h3 3! f (x ) h4 4! f (η 2 ) f (x ) = f(x +1) + f(x 1 ) 2f(x ) h 2 + h2 4! f (η 2 ) h2 4! f (η 1 ) If the second term s gnored, the prevous expresson reduces to the three ponts central dfference approxmaton. Ignorng the second term ntroduces a truncaton error that s proportonal to h 2. The truncaton error s sad to be of on the order of h 2 truncaton error = h2 4! f (η 2 ) h2 4! f (η 1 ) = O(h 2 ) The same procedure can be used to develop hgher order fnte dfference approxmatons. The result s summarzed n the table below 7
2.4 3 Fnte dfference formulas of thrd and fourth Rchardson s extrapolaton To ths pont, we have seen that there are two ways to mprove dervatve estmates when employng fnte dvded dfferences: (1) decrease the step sze or (2) use hgher order formula that employs more ponts. A thrd approach, based on Rchardson extrapolaton uses two dervatve estmates to compute a thrd, more accurate approxmaton. In general terms, consder the value, D, of a dervatve (unknown) that s calculated by the dfference formula: D = D(h) + k2 h2 + k4 h4 8
where D(h) s a functon that approxmates the value of the dervatve and k 2 h 2 and k 4 h 4 are error terms n whch the coeffcents, k 2 and k 4 are ndependent of the spacng h. Usng the same formula for calculatng the value of D but usng a spacng of h/2 gves: D = D(h/2) + k 2 ( h 2 Combnng the last two equatons we get D = 1 ( ( ) ) h h 4 4D D(h) k 4 3 2 4 = 1 3 ) 2 ( ) 4 h + k 4 2 ( 4D ( ) ) h D(h) + O(h 4 ) 2 Ths means that an approxmated value of D wth error O(h 4 ) s obtaned from two lower-order approxmatons (D(h) and D(h/2)) that were calculated wth an error O(h 2 ). Example: f(x +1 ) = f(x ) + hf (x ) + h2 2! f (x ) + h3 3! f (x ) + h4 4! f v (x ) + h5 5! f v (η 1 ) f(x 1 ) = f(x ) hf (x ) + h2 2! f (x ) h3 3! f (x ) + h4 4! f v (x ) h5 5! f v (η 2 ) f(x +1 ) f(x 1 ) = 2hf (x ) + 2 h2 2! f (x ) + h3 3! f (x ) + h5 5! f v (η 1 ) + h5 5! f v (η 2 ) f (x ) = f(x +1) f(x 1 ) 2h f (x ) = 1 3 h2 3! f (x ) + 1 2 (f v (η 1 ) + f v (η 2 )) h4 5! D(h) = f(x + h) f(x h) 2h [ 4 f(x + h/2) f(x h/2) f(x ] + h) f(x h) + O(h 4 ) h 2h 4 Error n numercal dfferentaton Throughout ths chapter, expressons have been gven for the truncaton error, also known as the dscretzaton error. These expressons are generated by the partcular numercal scheme used for dervng a specfc fnte dfference formula to estmate the dervatve. In each case, the truncaton error depends on h (the spacng between the ponts) rased to some power. Clearly, the mplcaton s that as h s made smaller and smaller, the error could be made arbtrarly small. When the functon to be dfferentated s specfed as a set of dscrete data ponts, the spacng s fxed, and the truncaton error cannot be reduced by reducng the sze of h. In ths case, a smaller truncaton error can be obtaned by usng a fnte dfference formula that has a hgher-order truncaton error. When the functon that s beng dfferentated s gven by a mathematcal expresson, the spacng h for the ponts that are used n the fnte dfference formulas can be defned by the user. It mght appear then that h can be made arbtrarly small and there s no lmt to how small the error can be made. Ths, however, s not true because the total error s composed of two parts. One s the truncaton error arsng from the numercal method (the specfc fnte dfference formula) that s used. The second part s a round-off error arsng from the fnte precson of the partcular computer used. Therefore, even f the truncaton error can be made vanshngly small by choosng smaller and smaller values of h, the round-off error stll remans, or can even grow as h s made smaller and smaller. 9
5 Numercal partal dfferentaton For a functon of several ndependent varables, the partal dervatve of the functon wth respect to one of the varables represents the rate of change of the value of the functon wth respect to ths varable, whle all the other varables are kept constant. For a functon wth two ndependent varables, the partal dervatves wth respect to x and y at the pont (a, b) are defned as: f(x, b) f(a, b) x=a,y=b = lm x a x a f(a, y) f(a, b) x=a,y=b = lm y y b y b Ths means that the fnte dfference formulas that are used for approxmatng the dervatves of functons wth one ndependent varable can be adopted for calculatng partal dervatves. The formulas are appled for one of the varables, whle the other varables are kept constant. Two ponts forward dfference approxmaton Two ponts backward dfference approxmaton Two ponts central dfference approxmaton f(x +1, y ) f(x, y ) x +1 x y f(x, y +1 ) f(x, y ) y +1 y f(x, y ) f(x 1, y ) x x 1 y f(x, y ) f(x, y 1 ) y y 1 f(x +1, y ) f(x 1, y ) x +1 x 1 y f(x, y +1 ) f(x, y 1 ) y +1 y 1 The second partal dervatves wth the three-pont central dfference formula are: 2 f(x, y) 2 f(x 1, y ) 2f(x, y ) + f(x +1, y ) (x +1 x ) 2 2 f(x, y) y 2 f(x, y 1 ) 2f(x, y ) + f(x, y +1 ) (y +1 y ) 2 Mxed dervatves 2 f(x, y) y = = ( y ( f(x, y+1 ) f(x, y 1 ) y +1 y 1 ) ) = f(x +1, y +1 ) f(x +1, y 1 ) f(x 1, y +1 ) f(x 1, y 1 ) (y +1 y 1 )(x +1 x 1 ) 10