Making Models with Polynomials: Taylor Series

Making Models with Polynomials: Taylor Series

Why polynomials? i a 3 x 3 + a 2 x 2 + a 1 x + a 0 How do we write a general degree n polynomial? n a i x i i=0 Why polynomials and not something else? We can add, multiply, maybe divide (grade school, computer HW) 1

Why polynomials? ii More complicated functions (e x, sin x, x) have to be built from those parts at least approximately Easy to work with as a building block. General recipe for numerics: Model observation with a polynomial, perform operation on that, evaluate. (e.g. calculus, root finding) 2

Reconstructing a Function From Derivatives i Given f ( ), f ( ), f ( ),... can we reconstruct a polynomial f? f (x) =??? +???x +???x 2 + For simplicity, let us consider = 0 and a degree 4 polynomial f f (x) = a 0 + a 1 x + a 2 x 2 + a 3 x 3 + a 4 x 4 Note that a 0 = f (0). Now, take a derivative f (x) = a 1 + 2a 2 x + 3a 3 x 2 + 4a 4 x 3 3

Reconstructing a Function From Derivatives ii we see that a 1 = f (0), differentiating again f (x) = 2a 2 + 3 2a 3 x + 4 3a 4 x 2 yields a 2 = f (0)/2, and finally f (x) = 3 2a 3 + 4 3 2a 4 x yields a 3 = f (0)/3!. In general, a i = f (i) (0)/i!. 4

Reconstructing a Function From Derivatives i Found: Taylor series approximation. f (0 + x) f (0) + f (0)x + f (0) x 2 + 2 The general Taylor expansion with center = 0 is f (x) = i=0 f (i) (0) x i i! Demo: Polynomial Approximation with Derivatives (click to visit) (Part I) 5

Shifting the Expansion Center i Can you do this at points other than the origin? In this case, 0 is the center of the series expansion. We can obtain a Taylor expansion of f centered at by defining a shifted function g(x) = f (x + ) The Taylor expansion g with center 0 is as before g(x) = i=0 g (i) (0) x i i! 6

Shifting the Expansion Center ii This expansion of g yields a Taylor expansion of f with center f (x) = g(x x0) = i=0 g (i) (0) (x ) i i! It suffices to note that f (i) ( ) = g (i) (0) to get the general form f (x) = i=0 f (i) ( ) (x ) i or f ( + h) = i! i=0 f (i) ( ) h i i! 7

Errors in Taylor Approximation (I) i Can t sum infinitely many terms. Have to truncate. How big of an error does this cause? Demo: Polynomial Approximation with Derivatives (click to visit) (Part II) Suspicion, as h 0, we have n f (x f (i) ( ) 0 + h) h i C h n+1 i! i=0 }{{} Taylor error for degree n or n f (x f (i) ( ) 0 + h) h i = O(h n+1 ) i! i=0 }{{} Taylor error for degree n 8

Errors in Taylor Approximation (I) ii As we will see, there is a satisfactory bound on C for most functions f. 9

Making Predictions with Taylor Truncation Error i Suppose you expand x 10 in a Taylor polynomial of degree 3 about the center = 12. For h 1 = 0.5, you find that the Taylor truncation error is about 10 4. What is the Taylor truncation error for h 2 = 0.25? Error(h) = O(h n+1 ), where n = 3, i.e. Error(h 1 ) C h 4 1 Error(h 2 ) C h 4 2 10

Making Predictions with Taylor Truncation Error ii While not knowing C or lower order terms, we can use the ratio of h 2 /h 1 ( h2 ) 4 Error(h 1 ) ( ) 4 h2 Error(h 2 ) C h 4 2 = C h 4 1 h 1 h 1 Can make prediction of the error for one h if we know another. Demo: Polynomial Approximation with Derivatives (click to visit) (Part III) 11

Taylor Remainders: the Full Truth i Let f : R R be (n + 1)-times differentiable on the interval (, x) with f (n) continuous on [, x]. Then there exists a ξ (, x) so that f ( + h) n i=0 and since ξ h n f ( + h) i=0 f (i) ( ) i! h i = f (n+1) (ξ) (n + 1)! } {{ } C f (i) ( ) f h i (n+1) (ξ) i! (n + 1)! } {{ } C (ξ ) n+1 h n+1. 12

Intuition for Taylor Remainder Theorem i Given the value of a function and its derivative f ( ), f ( ), prove the Taylor error bound. We express f (x) as x f (x) = f ( ) + f (w 0 )dw 0 13

Intuition for Taylor Remainder Theorem ii We then express the integrand f (w 0 ) for w 0 [, x] using f in the same way f (w )dw x ( w0 ) f (x) = f ( ) + f ( ) + dw 0 1 1 x ( w0 ) = f ( ) + f ( )(x ) + }{{} f (w 1 )dw 1 dw 0 Taylor expansion t 1 (x) 14

Intuition for Taylor Remainder Theorem iii We can bound the error by finding the maximizer ξ = argmax ξ [x0,x] f (ξ) f (x) t 1 (x) x ( w0 ) f (ξ) dw 1 dw 0 = f (ξ) (x ) 2 2! f ( + h) t 1 ( + h) f (ξ) h 2 for x = + h 2! In-class activity: Taylor series 15

Proof of Taylor Remainder Theorem i We can complete the proof by induction of the Taylor degree expansion n f (x) = t n 1 (x) + x w0 wn 1 f (n) (w n )dw n dw 0 Given the formula above, it suffices to expand wn f (n) (w n ) = f (n) ( ) + f (n+1) (w n+1 )dw n+1 16

Proof of Taylor Remainder Theorem ii Inserting this back in gives us the next inductive hypothesis f (x) = t n 1 (x) + f (n) ( ) (x ) n }{{ n! } + Taylor expansion t n(x) x w0 wn f (n+1) (w n+1 )dw n+1 dw 0 17

Proof of Taylor Remainder Theorem iii The bound follows as for n = 1, with ξ = argmax ξ [x0,x] f (n+1) (ξ) f (x) t n (x) x w0 wn = f (ξ) (n + 1)! (x ) n+1 f (n+1) (ξ) dw n+1 dw 0 f ( + h) t n ( + h) f (ξ) (n + 1)! hn+1 for x = + h 18

Using Polynomial Approximation i Suppose we can approximate a function as a polynomial: f (x) a 0 + a 1 x + a 2 x 2 + a 3 x 3. How is that useful? E.g.: What if we want the integral of f? Easy: Just integrate the polynomial: t s f (x)dx t s a 0 + a 1 x + a 2 x 2 + a 3 x 3 dx t t t t = a 0 1dx + a 1 x dx + a 2 x 2 dx + a 3 x 3 dx s s s s 19

Using Polynomial Approximation ii Even if you had no idea how to integrate f using calculus, you can approximately integrate f anyway, by taking a bunch of derivatives (and forming a Taylor polynomial). Demo: Computing Pi with Taylor (click to visit) 20