Numerical Differentiation Finite Difference Formulas for te first derivative (Using Taylor Expansion tecnique) (section 8.3.) Suppose tat f() = g() is a function of te variable, and tat as 0 te function g() remains bounded. Tis means tat tere exists some M suc tat g() M wen 0. Equivalently, we ave f() M as 0, equivalently f() M as 0. By taking to te oter side, we ave f() M as 0. By tis means tat f() = O(). Similarly, if f() = g() 2, were g remains bounded as 0, ten we ave f() = O( 2 ), and so on. Let f be any function defined on a vicinity of te fixed x, and let > 0 representing a positive number close to zero (we are going to let 0 later on). Using te Taylor expansion of te function f about te base point x, we ave f(x + ) = f(x) + f (x) + 2 2! f (x) + 3 3! f (x) + Move f(x) to te left-and side: f(x + ) f(x) = f (x) + 2 2! f (x) + 3 3! f (x) + Divide by : f(x+) f(x) = f (x) + 2! f () + 2 3! f (x) + { } = f (x) + 2! f (x) + 2 3! f (x) + Te expression inside te braces tends to 2 f (x) as 0, terefore remains bounded as 0. So we can simply write te above equality as: f(x + ) f(x) = f (x) + O()
By moving O() to te oter side we get f (x) = f(x + ) f(x) O() But multiplying an O() by some constant does not affect te status of O(), terefore O() is also is an O(), terefore tis last equality can be written as f (x) = f(x + ) f(x) + O() () in oter words: f (x) f(x+) f(x) wit truncation error of order O() (2) Te expression f(x+) f(x) derivative. Te difference f (x) f(x+) f(x) is called a two-point forward difference formula for te first wic te truncation error is of order O(). So in te forward difference formula te truncation error is of order O(). We will see later on ow to find approximations for f (x) in wic te truncation error is of iger order of accuracy. Example. Te following data set of a values of a function f is given: x 3 4 5 f(x) 27 64 25 Wat is te forward difference approximation for f (4)? Solution. f (4) f(5) f(4) 5 4 = 25 64 = 6 Now in te equation () let us cange to to get: f (x) = f(x ) f(x) + O( ) 2
But we ave O( ) = O(), terefore we may rewrite tis above equality as: f f(x ) f(x) (x) = + O() equivalently f (x) = f(x) f(x ) + O() (3) Te approximation f (x) f(x) f(x ) is called te two-point backward difference formula for te first derivative Example. Te following data set of a values of a function f is given: x 3 4 5 f(x) 27 64 25 Wat is te backward difference approximation for f (4)? Solution. f (4) f(4) f(3) 4 3 = 64 27 = 37 Here we want to find a central difference formula. Let us start wit te expansion f(x + ) = f(x) + f (x) + 2 2! f (x) + 3 3! f (x) + (4) In tis equality, replace to f(x ) = f(x) f (x) + ( )2 2! Equivalently: f (x) + ( )3 f (x) + 3! f(x ) = f(x) f (x) + 2 2! f (x) 3 3! f (x) + (5) 3
By subtracting (5) from (4) we get: f(x + ) f(x ) = f (x) + 3 6 f (x) + Upon dividing by : f(x + ) f(x ) = f (x) + 2 6 f (x) + So: f(x + ) f(x ) = f (x) + O( 2 ) Equivalently: f (x) = f(x + ) f(x ) + O( 2 ) So: f (x) f(x+) f(x ) truncation error being of order of O( 2 ) Tis is called te two-point central difference formula for te first derivative. Note tat te denominator is just te difference between te endpoints x + and x. Tis approximating formula is better tan te forward difference and te backward difference because te truncation error is smaller for small s. Example. Te following data set of a values of a function f is given: x 3 4 5 f(x) 27 64 25 Wat is te central difference approximation for f (4)? Solution. Here eac step is unit, terefore = 2. Hence: 4
f f(5) f(3) 25 27 (4) = = 49 2 2 We can create forward difference formula of order O( 2 ). For tis, let us start wit te expansions: f(x + ) = f(x) + f (x) + 2 2! f (x) + 3 3! f (x) + (6) In tis equality replace by : f(x + ) = f(x) + f (x) + ()2 2! Equivalently f (x) + ()3 f (x) + 3! f(x + ) = f(x) + f (x) + 42 2! f (x) + 83 3! f (x) + (7) By multiplying te identity (6) by 4 it becomes: 4f(x + ) = 4f(x) + 4f (x) + 42 2! f (x) + 43 3! f (x) + (6 ) Subtracting te identity (6 ) from te identity (7) gets rid of te term 2 : f(x + ) 4f(x + ) = 3f(x) f (x) + 43 3! f (x) + wic can be written as : f(x + ) 4f(x + ) = 3f(x) f (x) + O( 3 ) By moving 3f(x) to te left-and side we get: f(x + ) 4f(x + ) + 3f(x) = f (x) + O( 3 ) Multiply bot sides by and reorder terms to get: 5
3f(x) + 4f(x + ) f(x + ) = f (x) + O( 3 ) Dividing bot sides by gives us: 3f(x) + 4f(x + ) f(x + ) = f (x) + O( 2 ) By taking O( 2 ) to te oter side and ignoring a minus sign (wic does not affect te O-notation): f (x) = 3f(x) + 4f(x + ) f(x + ) + O( 2 ) f (x) 3f(x)+4f(x+) f(x+) truncation error being O( 2 ) Tis is called a tree-point forward difference formula for te first derivative. Example. Te following data set of a values of a function f is given: x.7 2.2 2.7 f(x) 27 64 25 Wat is te tree-point forward difference approximation for f (.7)? Solution. Here eac step is 0.5 units, terefore =. Hence: f (3) = 3f(.7) + 4f(2.2) f(2.7) = 3(27) + 4(64) (25) = 50 Example (exercise 8.8 of te textbook). A particular finite difference formula for te first derivative of a function is were te points x i f (x i ) f(x i+3) + 9f(x i+ ) 8f(x i ) 6, x i+, x i+2, x i+3 are all equally spaced wit step size. Wat is te order of truncation or discretization error. 6
Note. I am going to give you a question like tis in your next omework and in your midterm test, so learn tis carefully. Solution. Since te points are equally spaced, we set x i = x x i+ = x + x i+3 = x + 3 Wit tese conventions, we ten want to find te order of truncation error for estimating f (x) by f(x + 3) + 9f(x + ) 8f(x) 6 For tis, we use te unterminated form of te Taylor expansion: f(x + ) = f(x) + f (x) + 2 2! f (x) + 3 3! f (x) + Simplify to get: f(x + ) = f(x) + f (x) + 2 2 f (x) + 3 6 f (x) + () Cange to 3 in () to get: f(x + 3) = f(x) + 3f (x) + (3)2 f (x) + (3)3 f (x) + 2 6 Simplify: f(x + 3) = f(x) + 3f (x) + 92 2 f (x) + 93 2 f (x) + (2) Multiply te equality () by 9 and te equality (2) by - : 9f(x + ) = 9f(x) + 9f (x) + 92 2 f (x) + 93 6 f (x) + f(x + 3) = f(x) 3f (x) 92 2 f (x) 93 2 f (x) + 7
Now add te two equalities to get: f(x + 3) + 9f(x + ) = 8f(x) + 6f (x) 8 6 3 f (x) + Move 8f(x) to te left-and side: f(x + 3) + 9f(x + ) 8f(x) = 6f (x) 8 6 3 f (x) + Now divide by 6 f(x + 3) + 9f(x + ) 8f(x) 6 f(x + 3) + 9f(x + ) 8f(x) 6 = f (x) 3 6 2 f (x) + = f (x) + O( 2 ) So f (x) f(x+3)+9f(x+) 8f(x) 6 wit truncation error of order O( 2 ) Here I sowed on te screen and discussed te First Derivative table on pages 37 and 38 8
2 Finite Difference Formulas for te second derivative (Using Taylor Expansion tecnique) (section 8.3.2) f(x + ) = f(x) + f (x) + 2 2! f (x) + 3 3! f (x) + 4 4! f (4) (x) + f(x ) = f(x) f (x) + ( )2 2! f (x) + ( )3 3! f (x) + ( )4 4! f (4) (x) + Add togeter: = f(x) f (x) + 2 2! f (x) 3 3! f (x) + 4 4! f (4) (x) + f(x + ) + f(x ) = 2f(x) + 2 f (x) + 4 2 f (4) (x) + Move 2f(x) to te left-and side: f(x + ) + f(x ) 2f(x) = 2 f (x) + 4 2 f (4) (x) + Divide by 2 f(x+)+f(x ) 2f(x) 2 = f (x) + 2 2 f (4) (x) + = f (x) + 2 { 2 f (4) (x) + } So: = f (x) + O( 2 ) f (x) f(x ) 2f(x)+f(x+) 2 wit truncation error of order O( 2 ) Tis is called a tree-point central difference formula for te second derivative. Here I sowed on te screen and discussed te Second Derivative and te Tird Derivative tables on page 37 9
0
3 Ricardson s Extrapolation (section 8.8) Consider a procedure to approximate a value A wit some value D() tat depends on. For example we ad f(x + ) = f(x) + f (x) + 2 2! f (x) + 3 3! f (x) + 4 4! f (4) (x) + 5 5! f (5) (x) + Cange to f(x ) = f(x) f (x) + 2 2! f (x) 3 3! f (x) + 4 4! f (4) (x) 5 5! f (5) (x) + Subtract: f(x + ) f(x ) = f (x) + 3 3! f (x) + 5 5! f (5) (x) + Divide by : f(x + ) f(x ) = f (x) + 2 3! f (x) + 4 5! f (5) (x) + Move around : f (x) = f(x+) f(x ) A D() in sort we can write: 2 3! f (x) 4 5! f (5) (x) A = D() + γ 2 + γ 2 4 + () in wic te coefficients γ and γ 2,... do not depend on and terefore do not cange wen we replace.
Now let us cange to 2 : A = D( 2 ) + γ ( ) 2 ( ) 4 + γ 2 + 2 2 equivalently : A = D( 2 ) + γ multiply by 4 : 2 4 + γ 4 2 6 + 4A = 4D( 2 ) + γ 2 + γ 2 4 4 + (2) so in tis discussion we ave ad tese two: A = D() + γ 2 + γ 2 4 + () 4A = 4D( 2 ) + γ 2 + γ 2 4 4 + (2) subtract () from (2) : 3A = {4D( 2 ) D() } 3 4 γ 2 4 + divide by 3 : A = 3 {4D( 2 ) D() } 4 γ 2 4 + so now we ave found an approximation for A wose error is of a better order of O( 4 ) A 3 { 4D( 2 ) D()} wit truncation error of order O( 4 ) Tis tecnique of improving te approximation is called Ricardson extrapolation. We will use it in numerical integration as well. In te particular case of D() = f(x+) f(x ) we get: 2
D( 2 ) = f(x + 2 ) f(x 2 ) 2( 2 ) = f(x + 2 ) f(x 2 ) so ten : {4 D( 2 } 3 ) D() = 3 } 4 {f(x + 2 ) f(x 2 ) f(x + ) f(x ) so : = 8f(x + 2 ) 8f(x 2 ) f(x + ) + f(x ) 6 A f(x ) 8f(x 2 ) + 8f(x + 2 ) f(x + ) + O( 4 ) 6 Example. Te derivative of te function f(x) = x + + ln(x2 + ) is f (x) = ( + x) 2 + 2x x 2 + wose value at x = 0 is -. We want to approximate tis value using te following metods, one is te original metod and te second one is te improved metod. Metod : f(x + ) f(x ) of order O( 2 ) Metod 2 : f(x ) 8f(x 2 ) + 8f(x + 2 ) f(x + ) 6 of order O( 4 ). We start wit = 0.5 and ten using a wile loop you reduce in eac step by 0.025. We continue until is larger tan or equal to 0.0. 3
metod metod 2 error of metod error of metod 2 0.500 -.3333333333-0.9777777778 0.3333333333 0.0222222222 0.475 -.293640032-0.982582809 0.293640032 0.074789 0.450 -.253984953-0.986459909 0.253984953 0.035400909 0.425 -.220442404-0.9895748885 0.220442404 0.004255 0.400 -.90476905-0.992063492 0.90476905 0.0079365079 0.375 -.636363636-0.994037544 0.636363636 0.0059624586 0.350 -.3960396-0.995589648 0.3960396 0.00440359 0.325 -.8099233-0.9967968592 0.8099233 0.003203408 0.300 -.098900989-0.997723504 0.098900989 0.0022764959 0.275 -.08820352-0.9984234343 0.08820352 0.005765657 0.250 -.0666666667-0.998947989 0.0666666667 0.0005820 0.225 -.0533245556-0.999364600 0.0533245556 0.0006835400 0.200 -.046666667-0.999579246 0.046666667 0.0004208754 0.75 -.03592520-0.9997562536 0.03592520 0.0002437464 0.50 -.023079028-0.999869799 0.023079028 0.00030208 0.25 -.05873059-0.9999377529 0.05873059 0.000062247 0.00 -.00000-0.9999746842 0.00000 0.000025358 0.075 -.005656896-0.9999920339 0.005656896 0.000007966 0.050 -.0025062657-0.9999984326 0.0025062657 0.000005674 0.025 -.0006253909-0.9999999023 0.0006253909 0.0000000977 As we see, in te improved metod we get smaller errors for small s. Now we discuss te general form of te Ricarson s extrapolation metod. Generally if we ave an approximation metod of order O( m ) A = D() + γ m + O ( m+) (3) 4
ten by canging to 2 : A = D( 2 ) + γ ( 2 )m + O (( 2 ) )m+ equivalently A = D( 2 ) + γ m ( ) 2 m + O m+ 2m+ equivalently A = D( 2 ) + γ m 2 m + O ( m+) te O notation is not affected by constants multiply by 2 m 2 m A = 2 m D( 2 ) + γ m + 2 m O ( m+) equivalently 2 m A = 2 m D( 2 )+γ m + O ( m+) te O notation is not affected by constants (4) So we ave tese two: A = D() + γ m + O ( m+ ) (3) 2 m A = 2 m D( 2 ) + γ m + O ( m+ ) (4) Subtracting (3) from (4) gives: (2 m )A = (2 m D( 2 ) D() ) + O ( m+) Dividing by 2 m gives: A = (2 m 2 m D( 2 ) ) D() + O ( m+) 5
So te combination (2 m 2 m D( 2 ) ) D() gives a better approximation tan D(). Example. Te metod f(x) f(x 4 ) 4 as truncation order O( 4 ). Use te Ricardson extrapolation metod to improve it. Wat is te improved metod?. Solution. 6
4 Differentiation Formulas Using Lagrange Polynomials (section 8.5) Tere are two main questions tat we want to address in tis section. Question. We want to know ow te book as figured out ow to take particular combination in getting te numerical differentiation formulas in te Table 8- on pages 37, 38, and 39. Question 2. How can we find a formula wen te nodes are not equally spaced. Bot of tese questions can be answered by means of Lagrange metod for numerical differentiation. Supposing we ave four unequally-spaced nodes {2, 4, 5, 7} and we want to find a numerical approximation for te derivatives f (0) and f (.2) based on tese nodes (a forward difference approximation) and te values of f at tese nodes, for instance x 2 4 5 7 f(x) 5-5 -40 0 x k f[ ] = f() f[, ] f[,, ] f[,,, ] 2 5 4-5 5-40 7 0 5 5 4 2 40 ( 5) 5 4 = 5 = 35 0 ( 40) 7 5 = 25 35 ( 5) 5 2 = 0 25 ( 35) 7 4 = 20 20 ( 0) 7 2 = 6 and ten p(x) = 5 5(x 2) 0(x 2)(x 4) + 6(x 2)(x 4)(x 5) 7
Use te function expand() in Matlab to collect terms p(x) = 6x 3 76x 2 + 283x 305 For tis, you put tese two line on te command line: >> syms x; >> expand(5 5 (x 2) 0 (x 2) (x 4) + 6 (x 2) (x 4) (x 5)) Now differentiate tis polynomial twice. p (x) = 8x 2 52x + 283 p (x) = 36x 52 Now we can use p (x) to approximate any value of f (x) and not just f (0). Tis is an advantage of te Lagrange metod over te Taylor expansion. f (0) p (0) = 52 f (.2) p (.2) = 26.52 Note. Advantage of Lagrange Metod over Taylor Metod:. It can be used for unequally-spaced nodes. 2. It gives approximation for te derivative (of any order) at any point (and not just te nodes). Note 2.: If one applies te Lagrange metod for equally spaced noted, te e gets te difference formulas we ave learned before. Note 3.: Based on te Lagrange Metod, if some nodes {x,..., x n } and some values {f(x ),..., f(x n )} of a function f are given, ten to approximate f (x) at any point x (tat 8
could by oter tan te nodes), we fist find te interpolating polynomial p(x) associated wit tose values and ten f (x) p (x) f (x) p (x). Now ere, in te following example, is anoter way of doing te same problem but witout actually finding te interpolating polynomial p(x). Example. Find a, b, and c suc tat te approximation f (0) af(0) + bf() + cf(2) is exact for polynomials of degree less tan or equal to 2. Solution. We force te above approximation to be exact for te polynomials, x, x 2 tat generate all polynomials of degree less tan or equal to 2 troug linear combinations: af(0) + bf() + cf(2) = f (0) for f(x) =, f(x) = x, f(x) = x 2 f(x) = a + b + c = 0 () f(x) = x 0 + b + 2c = (2) f(x) = x 2 0 + b + 4c = 0 (3) (2) b = 2c (4) (3), (4) ( 2c) + 4c = 0 + 2c = 0 c = 2 b = 2c = + = 2 () a = b c = 2 + 2 = 3 2 9
So: a = 3 2 b = 2 c = 2 f (0) 3 2 f(0) + 2f() 2 f(2) Example. Now ere we see ow to get te formula tree-point forward difference g (x) 3g(x) + 4g(x + ) g(x + ) by referring to te approximation in te last example. In fact let f(t) = g(x + t) and note tat from te cain rule we ave f (t) = g (x + t). Especially f (0) = g (x). Ten f (0) 3 2 f(0) + 2f() 2 f(2) g (x) = 3 2 g(x) + 2g(x + ) divide g(x + ) 2 by g (x) 3g(x) + 4g(x + ) g(x + ) Note. As we ave seen, 20