Polnomial approimation and Splines 1. Weierstrass approimation theorem The basic question we ll look at toda is how to approimate a complicated function f() with a simpler function P () f() P () for eample, where P () is a polnomial of degree d. There are a variet of approaches. We have the following theorem: Weierstrass Approimation Theorem For ever continuous function f() on [a, b] and ever ɛ > 0 there is a polnomial P () such that f() P () < ɛ for each [a, b]. This is a useful idea for computer graphics and computer aided design. Instead of storing a lot of data (function values, bitmap images), just store a few numbers (polnomial coefficients), and generate the approimate function values when ou need them. In general, a polnomial of degree n has n + 1 coefficients, so we ll fit P to f using n + 1 equations in some wa. There are various was to do that. 2. Talor polnomials The idea here is to match the value of f and the values of n of its derivatives at a single point 0. That gives us our n + 1 equations. B Talor s theorem, P () = f (k) ( 0 ) ( 0 ) k. k! The problem is that this can be a great approimation near 0, but a lous one far awa. It works well for f s like polnomials or eponential functions, but not for other functions. In Figures 1-3 I ve plotted the n = 3, 5, 7 approimations to f 1 () = sin. This method is a good thing to tr if ou onl care about 0. It is often not a good choice for interpolating between widel separated values 0 and 1. These won t prove the theorem, even for infinitel differentiable f, as the following eample shows { e 1/2, > 0 Eample: Let f 2 () = 0, 0. Then f 2 is infinitel differentiable, but one can show that f2() 0 when 0. n This implies that f (k) 2 (0) = 0 for ever k. Therefore when 0 = 0, the Talor polnomials are all P () = 0. 1
2 Figure 1. Degree 3 Talor appro. to = sin, about 0 = 0 Figure 2. Degree 5 Talor appro. to = sin, about 0 = 0 3. Lagrange polnomials The idea here is that we get our n + 1 equations b matching the values of f at n + 1 points 0,..., n (called nodes) n i P () = f( k ). k i i=0,i k
3 Figure 3. Degree 7 Talor appro. to = sin, about 0 = 0 For eample, if ou substitue = j then the k j terms in the sum all = 0, and the k = j term gives f( j ). Figure 4 plots the Lagrange interpolation to the above f 2 with nodes -1.5, -1, -.5, 0,.5, 1, 1.5, 2. We re using 8 nodes, so the fit is with a degree n = 7 polnomial. It does a lot better than Talor s P () = 0, but it is still prett wiggl (ie it is eas to overfit using this method). 4. Hermite polnomials The idea here is that we get n + 1 equations b matching the values and first derivatives of f at n+1 2 points. For eample, with a cubic P we match four pieces of data at two points: f( 0 ), f ( 0 ), f( 1 ), f ( 1 ). There s a formula for this, which I ll write down but won t prove: P () = f( 0 ) ( 1 ) 2 ( 1 3 0 + 2) ( 1 0 ) 3 + f( 1 ) ( 0) 2 (3 1 0 2) ( 1 0 ) 3 + f ( 0 ) ( 0)( 1 ) 2 ( 1 0 ) 2 f ( 1 ) ( 0) 2 ( 1 ) ( 1 0 ) 2. 5. Splines The above can still be prett wiggl. To get more practical approimations, split up [a, b] into k pieces [ 0, 1 ],..., [ k 1, k ], and approimate on each using a different polnomial P j, j = 1,..., k. If we use cubics to do this, then we re tring to match 4k parameters via 4k equations. There are several was of doing this. 5.1. Hermite spline: Use Hermite interpolation to match f( k ) and f ( k ) at the beginning and end of each interval. Put another wa, we impose the constraints f( 0 ) = P 1 ( 0 ), f ( 0 ) = P 1( 0 ), f( k ) = P k ( k ), f ( k ) = P k ( k), and for j = 1
4-0.5 0.0 0.5 1.0-2 -1 0 1 2 Figure 4. A degree 7 Lagrange interpolation for = f 2 () to k 1: f( j ) = P j ( j ) = P j+1 ( j ) and f ( j ) = P j ( j) = P j+1 ( j). Because we have a nice formula for Hermite interpolation, this is eas to work out. In Figure 5 I ve plotted this for the eample f 2 with k = 3, and nodes at 0 = 2, 1 = 0.25, 2 = 1, and 3 = 2. This means we re fitting 12 parameters this time. It doesn t quite handle the sharp turn, but awa from there it is ecellent. -0.5 0.0 0.5 1.0-2 -1 0 1 2 Figure 5. A 3-window Hermite spline for = f 2 ()
5 5.2. Cubic spline: With this approach, we won t worr about matching the f ( j ), for j = 1,..., k 1. That saves us 2(k 1) constraints. Instead just make the k 1 derivatives match: P j ( j) = P j+1 ( j), and then make the k 1 second derivatives also match: P j ( j) = P j+1 ( j). The onl derivative information we use is at the ver beginning and end, but we get smoother solutions because the 2nd derivatives no longer jump. There isn t a simple formula for the coefficients an more now one needs to do some numerical linear algebra to figure them out (the solve a tridiagonal sstem ), but that is eas to do numericall, and it tpicall gives ecellent looking results. This approach is widel used in computer graphics and computer aided design. 6. Bezier curves This works fine for functions. But what about approimating curves? Now use parametric curves = P (t) and = Q(t). Using splines for each can certainl be made to work easil enough. But if ou re tring to use this to draw, ou want something geometric that is eas to adjust. In that setting, what people do is break the curve up into k arcs, and use cubic Hermite polnomials for P and Q. We need to specif values and derivatives for both and at each of k + 1 values of t. We do that in a geometric wa b giving guidepoints, ie ( j, j ) and ( j + j, j + j ). If ou use a sophisticated computer drawing program, ou ll find these somewhere in it. 7. Bernstein polnomials Let s go back to the original problem. How does one prove the Weierstrass Approimation theorem (sa on [0, 1])? The slickest proof is using the Bernstein polnomials ( ) n ( k P () = f k n) k (1 ) n k. The idea here is that if N has a Binomial(n, ) distribution, then f() = f(e[ N n ]) and P () = E[f( N n )]. So the should be close, b the law of large numbers. In fact, chasing through the standard proof of the law of large numbers, one can turn this into a rigorous proof. 8. Orthogonal polnomials There are other classes of polnomials used for approimation, for eample the Legendre polnomials or the Chebshev polnomials. Calculations for them are based on an orthogonalit propert, ver much like what happens for Fourier series. So we ll focus on those instead.