Taylor Series and Numerical Approximations Hilary Weller h.weller@reading.ac.uk August 7, 05 An introduction to the concept of a Taylor series and how these are used in numerical analysis to find numerical approximations and estimate their accuracy. This is a series of four short videos to accompany the printed notes. You can download the printed notes and fill parts in as we go along. Alternatively, you can work through the notes without the videos. Taylor Series Many functions can be expressed as Taylor series Taylor series are infinite polynomials For example an exponential: 6 5 4 e x +x +x + x! +x + x +x + x! + x!! + x! + x4 4! 0.0.5.0 0.0.0.5.0
For example a sine wave:.5.0 0.0 sinx x sinx x sinx sinx x x x x x x x x!!! x x x + x x5 + x5!! 5! 5! x x! + x5 x7 5! 7!.0.5 6 4 0 4 6 The more terms you include, the more accurate it should get. Functions with discontinuities cannot be expressed as Taylor series:.5 square wave.0 0.0.0.5 6 4 0 4 6 A polynomial cannot jump between and - 4
Functions with discontinuities in some of their derivatives cannot be expressed as Taylor series: { / ( + cosx) x < π Eg. cosine bell, f (x) = 0 otherwise.5.0 cosine bell x! x! x! + + x 4 4! x 4 4! x 6 6! 0.0 6 4 0 4 6 A polynomial cannot be uniform in one place and non-uniform in another 5 The Taylor Series: A function, f, can be represented as a Taylor series about position a if: is continuous near a and all of its derivatives are continuous near a Using the notation x = x a: f (x) = f (a) + x f (a) + x! f (a) + x! f (a) + + x j j! f ( j) (a) + where f (x) = d f dx (x), f (x) = d f (x),... dx If infinitely many terms are used then this approximation is exact near a. If all terms of order n and above are discarded then the error is approximately proportional to x n (assuming that x is small). Then the approximation is said to be n th order accurate A third order accurate approximation for f (x) has error proportional to x : f (x) = f (a) + x f (a) + x! f (a) + O ( x ). We say that the error is of order x or O ( x ). If x is small, then higher order accuracy generally means higher accuracy 6
. Exercises (answers at end). Write down the infinite Taylor series for function f at position x + x about x. Write the third order approximation for f (x x) in terms of f (x), f (x) and f (x). Write the error term using the O( x n ) notation.. Write the third order approximation for f (x + x) in terms of f (x x), f (x x) and f (x x). 7 Numerical Differentiation Consider a set of points, x 0,x, x j, x n where x j = j x (the points are distance x apart). Assume that we know the value of the function f (x) at these points, as shown in figure. f f j f j fj+ x 0 x x x x j x j x j+ x Figure : Values of a function f at points x 0, x,,x j,. Some possible estimate of f j = f (x j ) are: forward difference backward difference centred difference f j f j+ f j x f j f j f j x f j f j+ f j x Taylor series can be used to derive estimates of derivatives and to find their order of accuracy. 8
Taylor Series to find Finite Difference Gradients In order to use a Taylor series (below) to find an approximation for f f (x + x) = f (x) + x f (x) + x! f (x) + x! f (x) + + x j j! f ( j) (x) +. write down the knowns. consider where we want to find f. consider what order of accuracy we want 4. write down Taylor series for some of the knowns 5. eliminate the additional unknowns to find f. Example. Assume that we know f j = f (x j ) f j = f (x j ) = f (x x) f j+ = f (x j+ ) = f (x + x). and we want to find f j.. For knowns we wonder if we can get second order accuracy 4. We do not want to generate too many unknowns. We don t know f j or f j+ so no Taylor series about x j or x j+. So let s try Taylor series for f j+ and f j about x j f j+ = f j + x f j + x! f j + x! f + O( x 4 ) f j = f j x f j + x! f j x! f + O( x 4 ) 9 5. Eliminate f j by taking the difference of the two equations Rearrange to get f j f j+ f j = x f j + x f j + O( x 4 ) We cannot eliminate f j f j = f j+ f j x so this is part of the error: f j = f j+ f j x x! f j + O( x ) + O( x ) The error, ε, is proportional to x (ε x ) so this approximation is second order accurate. This is a worked example on my YouTube page: https://www.youtube.com/channel/uco0ywmerbcvw-brr8kcjlzw called TaylorSeries 0
. Exercises (answers at the end). Use the Taylor series to find an approximation for f j in terms of f j and f j. What order accuracy is it?. Derive an uncentred, second order difference formula for f j that uses f j, f j+ and f j+. (And show that it is second order accurate). Find an uncentred approximation for f j using f j, f j+ and f j+. What order accurate is it? 4. Derive a second order approximation for f b from f a, f b and f c at x locations a < b < c when the grid spacing is not regular. (And show that it is second order accurate) 4 Order of Accuracy of Numerical Solutions In order to demonstrate the order of accuracy (or order of convergence) of a numerical method, we can calculate the error of solving a problem that has an analytic solution. For example our numerical method calculates the gradient of sinx and gives these results: x numerical gradient of sinx at x = 0 Error, ε (Difference from cos(0)) 0.4 0.9755-0.0645 0. 0.995-0.00666 0. 0.998-0.0067 Assume that ε = A x n where n is the order of accuracy and A is unknown. From the data, eliminate A and find two possible values for n. 0 - error 0-0 - 0-0 - 0 0 x Plot ε as a function of x on log paper. Then the order of accuracy is the gradient. Does this value agree with the values found from calculating n directly from the data above? (both should give about n =. See YouTube video for a worked example.
5 Interpolation f An Example: Function f is known at points x and x (values f and f ) We want to estimate the value of f at point x i in between x and x x x i x β x ( β) x x Exercise: Use linear interpolation (ie assume that f i lies on a straight line between f and f ): to find f at x i Hint: First write down expressions for x, β and the gradient, f between x and x. Then find an expression for f at x i along the straight line between x and x. x = x x β = x i x x x f = f f x x = f i = ( β) f + β f If f is known at n points then a polynomial of degree n can be fit to estimate f Eg. Cubic Lagrange interpolation for constant grid spacing, x: ˆf (x) = 6 β( β)( β) f k + ( + β)( β)( β) f k f i f x + ( + β)β( β) f k+ 6 ( + β)β( β) f k+ 5. Finding Finite Difference Formulae for Interpolation using Taylor Series An Example: Assume that we know f j = f (x j ) and f j+ = f (x j+ ) and we want to find the interpolated value, f, mid-way between x j and x j+. Start by writing down Taylor series for f j and f j+ about f f j+ = f + x f + ( ) x f! + ( x! f j = f x f + ( ) x f! ( x! ) f + O( x) 4 ) f + O( x) 4 Eliminate the largest unknown, f by adding the two equations f j + f j+ = f + x 4 f + O( x) 4 Rearrange to find f and express the error based on the largest unknown f = ( f j + f j+ )/ + O( x) So this is a second-order accurate approximation 4
5. Exercises (answers at the end). Derive a centred, second order difference interpolation formula for f j that uses f j and f j+. (And show that it is second order accurate). Derive a centred fourth order difference formula for f that uses f j, f j, f j+ and f j+. (And show that it is fourth order accurate). Show that the first order forward difference formula for f j is exact for linear functions, f (x) = ax + b. 4. Show that the centred second order difference formula for f j is exact for quadratic functions f (x) = ax + bx + c. 5 Solutions Solutions to Exercises.. Write down the infinite Taylor series for function f at position x + x about x f (x + x) = f (x) + x f (x) + x! f (x) + x! f (x) + + x j j! f ( j) (x) +. Write the third order approximation for f (x x) in terms of f (x), f (x) and f (x). Write the error term using the O( x n ) notation. f (x x) = f (x) x f (x) + x! f (x) + O ( x ). Write the third order approximation for f (x + x) in terms of f (x x), f (x x) and f (x x). f (x + x) = f (x x) + x f (x x) + x f (x x) + 4 x f (x x) + O ( x ) Solutions to Exercises.. Use the Taylor series to find an approximation for f j in terms of f j and f j. What order accuracy is it? Write the Taylor series for f j in terms of f j : f j = f j x f j + O( x ) Rearrange to find f j: f j = ( f j f j )/ x + O( x) Note dividing O( x ) by x gives O( x) so the approximation is first order accurate. Derive an uncentred, second order difference formula for f j that uses f j, f j+ and f j+. (And show that it is second order accurate) 6
Taylor approximations for f j+ and f j+ about f j : f j+ = f j + x f j + x! f j + x! f j f j+ = f j + x f j + x f j + 4 x Eliminate the largest unknown, f + x4 4! f j + O( x 5 ) f j + x4 f j + O( x 5 ) j by calculating f j+ 4 f j+ : f j+ 4 f j+ = f j x f j + O( x ) Rearrange to find f j: f j = ( f j+ + 4 f j+ f j ) /( x) + O( x ). Find an uncentred approximation for f j using f j, f j+ and f j+. What order accurate is it? Taylor approximations for f j+ and f j+ about f j : f j+ = f j + x f j + x! f j + x! f j f j+ = f j + x f j + x f j + 4 x + x4 4! f j + O( x 5 ) f j + O( x 5 ) Eliminate the largest unknown, f j by calculating f j+ f j+ : f j+ f j+ = f j x f j + O( x ) Rearrange to find f j : f j = ( ) f j+ + f j+ f j / x + O( x) 4. Derive a second order approximation for f b from f a, f b and f c at x locations a < b < c when the grid spacing is not regular. (And show that it is second order accurate) Define x = b a and x = c b and x = max( x, x ) Taylor approximations for f a and f c about f b : f j + x4 f a = f b x f b + x! f b x! f b + x4 4! f b + O( x 5 ) 7 f c = f b + x f b + x! f b + x! f b + x4 4! f b + O( x 5 ) Eliminate the largest unknown, f b : x f c x f a = x { fb + x f b + O( x )} x { fb x f b + O( x )} Rearrange to find f b : x f c x f a = x f b + x x f b + O( x x ) x f b + x x f b + O( x x ) = x f c x f a = ( x x) fb + x x ( x + x ) f b + O( x5 ) = f b = x f c x f a ( x x x x ( x + x ) ) fb + O( x ) Solutions to Exercises 5.. Use Taylor series about f j so that they both contain f j so that we can eliminate it: f j = f j x f j +! x f j + O( x ) f j+ = f j + x f j +! x f j + O( x ) Eliminate f j by adding the two equations: f j + f j+ = f j + x f j + O( x ) Rearrange for f j and express the unknowns as the order of accuracy: f j = ( f j + f j+ )/ + O( x ). Taylor series about f : f j = f x f + 9! 4 x f 7! 8 x f + 8 4! 6 x4 f + O( x 5 ) f j = f x f +! 4 x f! 8 x f + 4! 6 x4 f + O( x 5 ) f j+ = f + x f +! 4 x f +! 8 8 x f + 4! 6 x4 f + O( x 5 )
f j+ = f + x f + 9! 4 x f + 7! 8 x f + 8 4! 6 x4 f + O( x 5 ) First eliminate f j, the biggest unknown (we only need to leave two equations): f j+ f j = x f + 9 8 x f j + O( x 5 ) f j+ f j = x f + 8 x f j + O( x 5 ) Next eliminate f j leaving one equation for f j: f j+ f j 7 f j+ + 7 f j = 4 x f + O( x 5 ) = f = f j 7 f j + 7 f j+ f j+ 4 x + O( x 4 ). If f (x) = ax+b and we use the approximation f j = f j+ f j between any two points x j+ x and x j then we get: f j = (ax j+ + b) (ax j + b) = a which is the exact solution. x j+ x j 4. If f (x) = ax +bx+c and we use the approximation f j = f j+ f j for f j at x j between x two points x j+ and x j then we get: f j = (ax j+ + bx j+ + c) (ax j + bx j + c) = a(x j + x) a(x j x) + b x x j+ x j x = ax j + b which is the exact solution. 9