Numericl Anlysis Doron Levy Deprtment of Mthemtics Stnford University December 1, 2005
D. Levy Prefce i
D. Levy CONTENTS Contents Prefce i 1 Introduction 1 2 Interpoltion 2 2.1 Wht is Interpoltion?............................ 2 2.2 The Interpoltion Problem.......................... 3 2.3 Newton s Form of the Interpoltion Polynomil.............. 5 2.4 The Interpoltion Problem nd the Vndermonde Determinnt...... 6 2.5 The Lgrnge Form of the Interpoltion Polynomil............ 7 2.6 Divided Differences.............................. 10 2.7 The Error in Polynomil Interpoltion................... 12 2.8 Interpoltion t the Chebyshev Points................... 15 2.9 Hermite Interpoltion............................. 21 2.9.1 Divided differences with repetitions................. 23 2.9.2 The Lgrnge form of the Hermite interpolnt........... 25 2.10 Spline Interpoltion.............................. 28 2.10.1 Cubic splines............................. 30 2.10.2 Wht is nturl bout the nturl spline?............. 34 3 Approximtions 36 3.1 Bckground.................................. 36 3.2 The Minimx Approximtion Problem................... 41 3.2.1 Existence of the minimx polynomil................ 42 3.2.2 Bounds on the minimx error.................... 43 3.2.3 Chrcteriztion of the minimx polynomil............ 44 3.2.4 Uniqueness of the minimx polynomil............... 45 3.2.5 The ner-minimx polynomil.................... 46 3.2.6 Construction of the minimx polynomil.............. 46 3.3 Lest-squres Approximtions........................ 48 3.3.1 The lest-squres pproximtion problem.............. 48 3.3.2 Solving the lest-squres problem: direct method........ 48 3.3.3 Solving the lest-squres problem: with orthogonl polynomils. 50 3.3.4 The weighted lest squres problem................. 52 3.3.5 Orthogonl polynomils....................... 53 3.3.6 Another pproch to the lest-squres problem.......... 58 3.3.7 Properties of orthogonl polynomils................ 63 4 Numericl Differentition 65 4.1 Bsic Concepts................................ 65 4.2 Differentition Vi Interpoltion....................... 67 4.3 The Method of Undetermined Coefficients................. 70 4.4 Richrdson s Extrpoltion.......................... 72 iii
CONTENTS D. Levy 5 Numericl Integrtion 74 5.1 Bsic Concepts................................ 74 5.2 Integrtion vi Interpoltion......................... 77 5.3 Composite Integrtion Rules......................... 79 5.4 Additionl Integrtion Techniques...................... 82 5.4.1 The method of undetermined coefficients.............. 82 5.4.2 Chnge of n intervl......................... 83 5.4.3 Generl integrtion formuls..................... 84 5.5 Simpson s Integrtion............................. 85 5.5.1 The qudrture error......................... 85 5.5.2 Composite Simpson rule....................... 86 5.6 Gussin Qudrture............................. 87 5.6.1 Mximizing the qudrture s ccurcy............... 87 5.6.2 Convergence nd error nlysis................... 91 5.7 Romberg Integrtion............................. 93 6 Methods for Solving Nonliner Problems 95 6.1 The Bisection Method............................ 95 6.2 Newton s Method............................... 97 6.3 The Secnt Method.............................. 100 Bibliogrphy 104 iv
D. Levy 1 Introduction 1
D. Levy 2 Interpoltion 2.1 Wht is Interpoltion? Imgine tht there is n unknown function f(x) for which someone supplies you with its (exct) vlues t (n+1) distinct points x 0 < x 1 < < x n, i.e., f(x 0 ),...,f(x n ) re given. The interpoltion problem is to construct function Q(x) tht psses through these points. One esy wy of finding such function, is to connect them with stright lines. While this is legitimte solution of the interpoltion problem, usully (though not lwys) we re interested in different kind of solution, e.g., smoother function. We therefore lwys specify certin clss of functions from which we would like to find one tht solves the interpoltion problem. For exmple, we my look for polynomil, Q(x), tht psses through these points. Alterntively, we my look for trigonometric function or piecewise-smooth polynomil such tht the interpoltion requirements Q(x j ) = f(x j ), 0 j n, (2.1) re stisfied (see Figure 2.1). Q(x) f(x 2 ) f(x 1 ) f(x 0 ) f(x) x 0 x 1 x 2 Figure 2.1: The function f(x), the interpoltion points x 0,x 1,x 2, nd the interpolting polynomil Q(x) As simple exmple let s consider vlues of function tht re prescribed t two points: (x 0,f(x 0 )) nd (x 1,f(x 1 )). There re infinitely mny functions tht pss through these two points. However, if we limit ourselves to polynomils of degree less thn or equl to one, there is only one such function tht psses through these two points, which is nothing but the line tht connects them. A line, in generl, is polynomil of degree 2
D. Levy 2.2 The Interpoltion Problem one, but if we wnt to keep the discussion generl enough, it could be tht f(x 0 ) = f(x 1 ) in which cse the line tht connects the two points is the constnt Q 0 (x) f(x 0 ), which is polynomil of degree zero. This is why we sy tht there is unique polynomil of degree 1 tht connects these two points (nd not polynomil of degree 1 ). The points x 0,...,x n re clled the interpoltion points. The property of pssing through these points is referred to s interpolting the dt. The function tht interpoltes the dt is n interpolnt or n interpolting polynomil (or whtever function is being used). Sometimes the interpoltion problem hs solution. There re cses were the interpoltion problem hs no solution, hs unique solution, or hs more thn one solution. Wht we re going to study in this section is precisely how to distinguish between these cses. We re lso going to present vrious wys of ctully constructing the interpolnt. In generl, there is little hope tht the interpolnt will be identicl to the unknown function f(x). The function Q(x) tht interpoltes f(x) t the interpoltion points will be still be identicl to f(x) t these points becuse there we stisfy the interpoltion conditions (2.1). In generl, t ny other point, Q(x) nd f(x) will not hve the sme vlues. The interpoltion error mesure on how different these two functions re. We will study wys of estimting the interpoltion error. We will lso discuss strtegies on how to minimize this error. It is importnt to note tht it is possible to formulte interpoltion problem without referring to (or even ssuming the existence of) ny underlying function f(x). For exmple, you my hve list of interpoltion points x 0,...,x n, nd dt tht is given t these points, y 0,y 1,...,y n, which you would like to interpolte. The solution to this interpoltion problem is identicl to the one where the vlues re tken from n underlying function. 2.2 The Interpoltion Problem We begin our study with the problem of polynomil interpoltion: Given n + 1 distinct points x 0,...,x n, we seek polynomil Q n (x) of the lowest degree such tht the following interpoltion conditions re stisfied: Q n (x j ) = f(x j ), j = 0,...,n. (2.2) Note tht we do not ssume ny ordering between the points x 0,...,x n, s such n order will mke no difference for the present discussion. If we do not limit the degree of the interpoltion polynomil it is esy to see tht there ny infinitely mny polynomils tht interpolte the dt. However, limiting the degree to n, singles out precisely one interpolnt tht will do the job. For exmple, if n = 1, there re infinitely mny polynomils tht interpolte between (x 0,f(x 0 )) nd (x 1,f(x 1 )). There is only one polynomil of degree 1 tht does the job. This result is formlly stted in the following theorem: Theorem 2.1 If x 0,...,x n R re distinct, then for ny f(x 0 ),...f(x n ) there exists unique polynomil Q n (x) of degree n such tht the interpoltion conditions (2.2) re stisfied. 3
2.2 The Interpoltion Problem D. Levy Proof. We strt with the existence prt nd prove the result by induction. For n = 0, Q 0 = f(x 0 ). Suppose tht Q n 1 is polynomil of degree n 1, nd suppose lso tht Q n 1 (x j ) = f(x j ), 0 j n 1. Let us now construct from Q n 1 (x) new polynomil, Q n (x), in the following wy: Q n (x) = Q n 1 (x) + c(x x 0 )... (x x n 1 ). (2.3) The constnt c in (2.3) is yet to be determined. Clerly, the construction of Q n (x) implies tht deg(q n (x)) n. In ddition, the polynomil Q n (x) stisfies the interpoltion requirements Q n (x j ) = f(x j ) for 0 j n 1. All tht remins is to determine the constnt c in such wy tht the lst interpoltion condition, Q n (x n ) = f(x n ), is stisfied, i.e., Q n (x n ) = Q n 1 (x n ) + c(x n x 0 )... (x n x n 1 ). (2.4) The condition (2.4) defines c s c = f(x n) Q n 1 (x n ), (2.5) n 1 (x n x j ) nd we re done with the proof of existence. As for uniqueness, suppose tht there re two polynomils Q n (x),p n (x) of degree n tht stisfy the interpoltion conditions (2.2). Define polynomil H n (x) s the difference H n (x) = Q n (x) P n (x). The degree of H n (x) is t most n which mens tht it cn hve t most n zeros (unless it is identiclly zero). However, since both Q n (x) nd P n (x) stisfy the interpoltion requirements (2.2), we hve H n (x j ) = (Q n P n )(x j ) = 0, 0 j n, which mens tht H n (x) hs n + 1 distinct zeros. This leds to contrdiction tht cn be resolved only if H n (x) is the zero polynomil, i.e., P n (x) = Q n (x), nd uniqueness is estblished. 4
D. Levy 2.3 Newton s Form of the Interpoltion Polynomil 2.3 Newton s Form of the Interpoltion Polynomil One good thing bout the proof of Theorem 2.1 is tht it is constructive. In other words, we cn use the proof to write down formul for the interpoltion polynomil. We follow the procedure given by (2.4) for reconstructing the interpoltion polynomil. We do it in the following wy: Let Q 0 (x) = 0, where 0 = f(x 0 ). Let Q 1 (x) = 0 + 1 (x x 0 ). Following (2.5) we hve 1 = f(x 1) Q 0 (x 1 ) x 1 x 0 = f(x 1) f(x 0 ) x 1 x 0. We note tht Q 1 (x) is nothing but the stright line connecting the two points (x 0,f(x 0 )) nd (x 1,f(x 1 )). In generl, let Q n (x) = 0 + 1 (x x 0 ) +... + n (x x 0 )... (x x n 1 ) (2.6) j 1 = 0 + j (x x k ). j=1 k=0 The coefficients j in (2.6) re given by 0 = f(x 0 ), (2.7) j = f(x j) Q j 1 (x j ). j 1 (x j x k ) k=0 We refer to the interpoltion polynomil when written in the form (2.6) (2.7) s the Newton form of the interpoltion polynomil. As we shll see below, there re vrious wys of writing the interpoltion polynomil. The uniqueness of the interpoltion polynomil s gurnteed by Theorem 2.1 implies tht we will only be rewriting the sme polynomil in different wys. Exmple 2.2 The Newton form of the polynomil tht interpoltes (x 0,f(x 0 )) nd (x 1,f(x 1 )) is Q 1 (x) = f(x 0 ) + f(x 1) f(x 0 ) x 1 x 0 (x x 0 ). 5
2.4 The Interpoltion Problem nd the Vndermonde Determinnt D. Levy Exmple 2.3 The Newton form of the polynomil tht interpoltes the three points (x 0,f(x 0 )), (x 1,f(x 1 )), nd (x 2,f(x 2 )) is Q 2 (x) = f(x 0 )+ f(x 1) f(x 0 ) x 1 x 0 (x x 0 )+ f(x 2) f(x 0 ) + f(x1) f(x0) x 1 x 0 (x 2 x 0 ) (x x 0 )(x x 1 ). (x 2 x 0 )(x 2 x 1 ) 2.4 The Interpoltion Problem nd the Vndermonde Determinnt An lterntive pproch to the interpoltion problem is to consider directly polynomil of the form Q n (x) = b k x k, (2.8) k=0 nd require tht the following interpoltion conditions re stisfied Q n (x j ) = f(x j ), 0 j n. (2.9) In view of Theorem 2.1 we lredy know tht this problem hs unique solution, so we should be ble to compute directly the coefficients of the polynomil s given in (2.8). Indeed, the interpoltion conditions, (2.9), imply tht the following equtions should hold: b 0 + b 1 x j +... + b n x n j = f(x j ), j = 0,...,n. (2.10) In mtrix form, (2.10) cn be rewritten s 1 x 0... x n 0 b 0 f(x 0 ) 1 x 1... x n 1 b 1....... = f(x 1 ).. (2.11) 1 x n... x n n b n f(x n ) In order for the system (2.11) to hve unique solution, it hs to be nonsingulr. This mens, e.g., tht the determinnt of its coefficients mtrix must not vnish, i.e. 1 x 0... x n 0 1 x 1... x n 1 0. (2.12)...... 1 x n... x n n The determinnt (2.12), is known s the Vndermonde determinnt. We leve it s n exercise to verify tht 1 x 0... x n 0 1 x 1... x n 1 = i x j ). (2.13)...... i>j(x 1 x n... x n n 6
D. Levy 2.5 The Lgrnge Form of the Interpoltion Polynomil Since we ssume tht the points x 0,...,x n re distinct, the determinnt (2.13) is indeed non zero. Hence, the system (2.11) hs solution tht is lso unique, which confirms wht we lredy know ccording to Theorem 2.1. 2.5 The Lgrnge Form of the Interpoltion Polynomil The form of the interpoltion polynomil tht we used in (2.8) ssumed liner combintion of polynomils of degrees 0,...,n, in which the coefficients were unknown. In this section we tke different pproch nd ssume tht the interpoltion polynomil is given s liner combintion of n + 1 polynomils of degree n. This time, we set the coefficients s the interpolted vlues, {f(x j )} n, while the unknowns re the polynomils. We thus let Q n (x) = f(x j )lj n (x), (2.14) where l n j (x) re n+1 polynomils of degree n. We use two indices in these polynomils: the subscript j enumertes l n j (x) from 0 to n nd the superscript n is used to remind us tht the degree of l n j (x) is n. Note tht in this prticulr cse, the polynomils l n j (x) re precisely of degree n (nd not n). However, Q n (x), given by (2.14) my hve lower degree. In either cse, the degree of Q n (x) is n t the most. We now require tht Q n (x) stisfies the interpoltion conditions Q n (x i ) = f(x i ), 0 i n. (2.15) By substituting x i for x in (2.14) we hve Q n (x i ) = f(x j )lj n (x i ), 0 i n. In view of (2.15) we my conclude tht l n j (x) must stisfy l n j (x i ) = δ ij, (2.16) where δ ij is the Krönecker delt, { 1, i = j, δ ij = 0, i j. One obvious wy of constructing polynomils l n j of degree n tht stisfy (2.16) is the following: lj n (x) = (x x 0)... (x x j 1 )(x x j+1 )... (x x n ), 0 j n. (2.17) (x j x 0 )... (x j x j 1 )(x j x j+1 )... (x j x n ) Note tht the denomintor in (2.17) does not vnish since we ssume tht ll interpoltion points re distinct, nd hence the polynomils l n j (x) re well defined. The 7
2.5 The Lgrnge Form of the Interpoltion Polynomil D. Levy Lgrnge form of the interpoltion polynomil is the polynomil Q n (x) given by (2.14), where the polynomils l n j (x) of degree n re given by l n j (x) = n (x x i ) i j, j = 0,...,n. (2.18) n (x j x i ) i j Exmple 2.4 We re interested in finding the Lgrnge form of the interpoltion polynomil tht interpoltes two points: (x 0,f(x 0 )) nd (x 1,f(x 1 )). We know tht the unique interpoltion polynomil through these two points is the line tht connects the two points. Such line cn be written in mny different forms. In order to obtin the Lgrnge form we let l 1 0(x) = x x 1 x 0 x 1, l 1 1(x) = x x 0 x 1 x 0. The desired polynomil is therefore given by the fmilir formul Q 1 (x) = f(x 0 )l 1 0(x) + f(x 1 )l 1 1(x) = f(x 0 ) x x 1 x 0 x 1 + f(x 1 ) x x 0 x 1 x 0. Exmple 2.5 This time we re looking for the Lgrnge form of the interpoltion polynomil, Q 2 (x), tht interpoltes three points: (x 0,f(x 0 )), (x 1,f(x 1 )), (x 2,f(x 2 )). Unfortuntely, the Lgrnge form of the interpoltion polynomil does not let us use the interpoltion polynomil through the first two points, Q 1 (x), s building block for Q 2 (x). This mens tht we hve to compute everything from scrtch. We strt with l 2 0(x) = (x x 1)(x x 2 ) (x 0 x 1 )(x 0 x 2 ), l 2 1(x) = (x x 0)(x x 2 ) (x 1 x 0 )(x 1 x 2 ), l 2 2(x) = (x x 0)(x x 1 ) (x 2 x 0 )(x 2 x 1 ). The interpoltion polynomil is therefore given by Q 2 (x) = f(x 0 )l 2 0(x) + f(x 1 )l 2 1(x) + f(x 2 )l 2 2(x) = f(x 0 ) (x x 1)(x x 2 ) (x 0 x 1 )(x 0 x 2 ) + f(x 1) (x x 0)(x x 2 ) (x 1 x 0 )(x 1 x 2 ) + f(x 2) (x x 0)(x x 1 ) (x 2 x 0 )(x 2 x 1 ). It is esy to verify tht indeed Q 2 (x j ) = f(x j ) for j = 0, 1, 2, s desired. Remrks. 8
D. Levy 2.5 The Lgrnge Form of the Interpoltion Polynomil 1. One instnce where the Lgrnge form of the interpoltion polynomil my seem to be dvntgeous when compred with the Newton form is when you re interested in solving severl interpoltion problems, ll given t the sme interpoltion points x 0,...x n but with different vlues f(x 0 ),...,f(x n ). In this cse, the polynomils l n j (x) re identicl for ll problems since they depend only on the points but not on the vlues of the function t these points. Therefore, they hve to be constructed only once. 2. An lterntive form for l n j (x) cn be obtined in the following wy. If we define w n (x) = n (x x i ), then w n(x) = n (x x i ). (2.19) i j When we then evlute w x(x) t ny interpoltion point, x j, there is only one term in the sum in (2.19) tht does not vnish: w n(x j ) = n (x j x i ). i j Hence, l n j (x) cn be rewritten s l n j (x) = w n (x), 0 j n. (2.20) (x x j )w n(x j ) 3. For future reference we note tht the coefficient of x n in the interpoltion polynomil Q n (x) is f(x j ). (2.21) n (x j x k ) k=0 k j For exmple, the coefficient of x in Q 1 (x) in Exmple 2.4 is f(x 0 ) + f(x 1). x 0 x 1 x 1 x 0 9
2.6 Divided Differences D. Levy 2.6 Divided Differences We recll tht Newton s form of the interpoltion polynomil is Q n (x) = 0 + 1 (x x 0 ) +... + n (x x 0 )... (x x n 1 ), with 0 = f(x 0 ) nd j = f(x j) Q j 1 (x j ), 1 j n. j 1 (x j x k ) k=0 We nme the j th coefficient, j, s the j th -order divided difference. The j th -order divided difference, j, is bsed on the points x 0,...,x j nd on the vlues of the function t these points f(x 0 ),...,f(x j ). To emphsize this dependence, we use the following nottion: j = f[x 0,...,x j ], 1 j n. We lso denote the zeroth-order divided difference s where 0 = f[x 0 ], f[x 0 ] = f(x 0 ). When written in terms of the divided differences, the Newton form of the interpoltion polynomil becomes n 1 Q n (x) = f[x 0 ] + f[x 0,x 1 ](x x 0 ) +... + f[x 0,...x n ] (x x k ). (2.22) There is simple wy of computing the j th -order divided difference from lower order divided differences. This is given by the following lemm. Lemm 2.6 The divided differences stisfy: f[x 0,...x n ] = f[x 1,...x n ] f[x 0,...x n 1 ] x n x 0. (2.23) Proof. For ny k, we denote by Q k (x), polynomil of degree k, tht interpoltes f(x) t x 0,...,x k, i.e., Q k (x j ) = f(x j ), 0 j k. We now consider the unique polynomil P(x) of degree n 1 tht interpoltes f(x) t x 1,...,x n. It is esy to verify tht Q n (x) = P(x) + x x n x n x 0 [P(x) Q n 1 (x)]. (2.24) 10 k=0
D. Levy 2.6 Divided Differences The coefficient of x n on the left-hnd-side of (2.24) is f[x 0,...,x n ]. The coefficient of x n 1 in P(x) is f[x 1,...,x n ] nd the coefficient of x n 1 in Q n 1 (x) is f[x 0,...,x n 1 ]. Hence, the coefficient of x n on the right-hnd-side of (2.24) is 1 x n x 0 (f[x 1,...,x n ] f[x 0,...,x n 1 ]), which mens tht f[x 0,...x n ] = f[x 1,...x n ] f[x 0,...x n 1 ] x n x 0. Exmple 2.7 The second-order divided difference is f[x 0,x 1,x 2 ] = f[x 1,x 2 ] f[x 0,x 1 ] x 2 x 0 = f(x 2 ) f(x 1 ) x 2 x 1 f(x 1) f(x 0 ) x 1 x 0 x 2 x 0. Hence, the unique polynomil tht interpoltes (x 0,f(x 0 )), (x 1,f(x 1 )), nd (x 2,f(x 2 )) is Q 2 (x) = f[x 0 ] + f[x 0,x 1 ](x x 0 ) + f[x 0,x 1,x 2 ](x x 0 )(x x 1 ) = f(x 0 ) + f(x 1) f(x 0 ) x 1 x 0 (x x 0 ) + f(x 2 ) f(x 1 ) x 2 x 1 f(x 1) f(x 0 ) x 1 x 0 x 2 x 0 (x x 0 )(x x 1 ). For exmple, if we wnt to find the polynomil of degree 2 tht interpoltes ( 1, 9), (0, 5), nd (1, 3), we hve f( 1) = 9, f[ 1, 0] = 5 9 0 ( 1) = 4, f[0, 1] = 3 5 1 0 = 2, f[ 1, 0, 1] = f[0, 1] f[ 1, 0] 1 ( 1) = 2 + 4 2 = 1. so tht Q 2 (x) = 9 4(x + 1) + (x + 1)x = 5 3x + x 2. The reltions between the divided differences re schemticlly portryed in Tble 2.1 (up to third-order). We note tht the divided differences tht re being used s the coefficients in the interpoltion polynomil re those tht re locted in the top of every column. The recursive structure of the divided differences implies tht it is required to compute ll the low order coefficients in the tble in order to get the high-order ones. 11
2.7 The Error in Polynomil Interpoltion D. Levy x 0 f(x 0 ) f[x 0,x 1 ] x 1 f(x 1 ) f[x 0,x 1,x 2 ] f[x 1,x 2 ] f[x 0,x 1,x 2,x 3 ] x 2 f(x 2 ) f[x 1,x 2,x 3 ] f[x 2,x 3 ] x 3 f(x 3 ) Tble 2.1: Divided Differences One importnt property of ny divided difference is tht it is symmetric function of its rguments. This mens tht if we ssume tht y 0,...,y n is ny permuttion of x 0,...,x n, then f[y 0,...,y n ] = f[x 0,...,x n ]. This property cn be clerly explined by reclling tht f[x 0,...,x n ] plys the role of the coefficient of x n in the polynomil tht interpoltes f(x) t x 0,...,x n. At the sme time, f[y 0,...,y n ] is the coefficient of x n t the polynomil tht interpoltes f(x) t the sme points. Since the interpoltion polynomil is unique for ny given dt, the order of the points does not mtter, nd hence these two coefficients must be identicl. 2.7 The Error in Polynomil Interpoltion In this section we would like to provide estimtes on the error we mke when interpolting dt tht is tken from smpling n underlying function f(x). While the interpolnt nd the function gree with ech other t the interpoltion points, there is, in generl, no reson to expect them to be close to ech other elsewhere. Nevertheless, we cn estimte the difference between them, difference which we refer to s the interpoltion error. We let Π n denote the spce of polynomils of degree n. Theorem 2.8 Let f(x) C n+1 [,b]. Let Q n (x) Π n such tht it interpoltes f(x) t the n + 1 distinct points x 0,...,x n [,b]. Then x [,b], ξ n (,b) such tht f(x) Q n (x) = 1 (n + 1)! f(n+1) (ξ n ) n (x x j ). (2.25) 12
D. Levy 2.7 The Error in Polynomil Interpoltion Proof. We fix point x [,b]. If x is one of the interpoltion points x 0,...,x n, then the left-hnd-side nd the right-hnd-side of (2.25) re both zero, nd hence this result holds trivilly. We therefore ssume tht x x j 0 j n, nd let w(x) = We now let n (x x j ). F(y) = f(y) Q n (y) λw(y), where λ is chosen s to gurntee tht F(x) = 0, i.e., λ = f(x) Q n(x). w(x) Since the interpoltion points x 0,...,x n nd x re distinct, w(x) does not vnish nd λ is well defined. We now note tht since f C n+1 [,b] nd since Q n nd w re polynomils, then lso F C n+1 [,b]. In ddition, F vnishes t n + 2 points: x 0,...,x n nd x. According to Rolle s theorem, F hs t lest n + 1 distinct zeros in (,b), F hs t lest n distinct zeros in (,b), nd similrly, F (n+1) hs t lest one zero in (,b), which we denote by ξ n. We hve 0 = F (n+1) (ξ n ) = f (n+1) (ξ n ) Q (n+1) n (ξ n ) λ(x)w (n+1) (ξ n ) (2.26) = f (n+1) (ξ n ) f(x) Q n(x) (n + 1)! w(x) Here, we used the fct tht the leding term of w(x) is x n+1, which gurntees tht its (n + 1) th derivtive equls w (n+1) (x) = (n + 1)! (2.27) Reordering the terms in (2.26) we conclude with f(x) Q n (x) = 1 (n + 1)! f(n+1) (ξ n )w(x). In ddition to the interprettion of the divided difference of order n s the coefficient of x n in some interpoltion polynomil, there is nother importnt chrcteriztion which we will comment on now. Consider, e.g., the first-order divided difference f[x 0,x 1 ] = f(x 1) f(x 0 ) x 1 x 0. Since the order of the points does not chnge the vlue of the divided difference, we cn ssume, without ny loss of generlity, tht x 0 < x 1. If we ssume, in ddition, tht 13
2.7 The Error in Polynomil Interpoltion D. Levy f(x) is continuously differentible in the intervl [x 0,x 1 ], then this divided difference equls to the derivtive of f(x) t n intermedite point, i.e., f[x 0,x 1 ] = f (ξ), ξ (x 0,x 1 ). In other words, the first-order divided difference cn be viewed s n pproximtion of the first derivtive in the intervl. It is importnt to note tht while this interprettion is bsed on dditionl smoothness requirements from f(x) (i.e. its being differentible), the divided differences re well defined lso for non-differentible functions. This notion cn be extended to divided differences of higher order s stted by the following theorem. Theorem 2.9 Let x,x 0,...,x n 1 be n + 1 distinct points. Let = min(x,x 0,...,x n 1 ) nd b = mx(x,x 0,...,x n 1 ). Assume tht f(y) hs continuous derivtive of order n in the intervl (,b). Then f[x 0,...,x n 1,x] = f(n) (ξ), (2.28) n! where ξ (,b). Proof. Let Q n+1 (y) interpolte f(y) t x 0,...,x n 1,x. Then ccording to the construction of the Newton form of the interpoltion polynomil (2.22), we know tht n 1 Q n (y) = Q n 1 (y) + f[x 0,...,x n 1,x] (y x j ). Since Q n (y) interpolted f(y) t x, we hve n 1 f(x) = Q n 1 (x) + f[x 0,...,x n 1,x] (x x j ). By Theorem 2.8 we know tht the interpoltion error is given by f(x) Q n 1 (x) = 1 n 1 n! f(n) (ξ n 1 ) (x x j ), which implies the result (2.28). Remrk. In eqution (2.28), we could s well think of the interpoltion point x s ny other interpoltion point, nd nme it, e.g., x n. In this cse, the eqution (2.28) tkes the somewht more nturl form of f[x 0,...,x n ] = f(n) (ξ). n! In other words, the n th -order divided difference is n n th -derivtive of the function f(x) t n intermedite point, ssuming tht the function hs n continuous derivtives. Similrly to the first-order divided difference, we would like to emphsize tht the n th -order divided difference is lso well defined in cses where the function is not s smooth s required in the theorem, though if this is the cse, we cn no longer consider this divided difference to represent n th -order derivtive of the function. 14
D. Levy 2.8 Interpoltion t the Chebyshev Points 2.8 Interpoltion t the Chebyshev Points In the entire discussion so fr, we ssumed tht the interpoltion points re given. There my be cses where one my hve the flexibility of choosing the interpoltion points. If this is the cse, it would be resonble to use this degree of freedom to minimize the interpoltion error. We recll tht if we re interpolting vlues of function f(x) tht hs n continuous derivtives, the interpoltion error is of the form 1 n f(x) Q n (x) = (n + 1)! f(n+1) (ξ n ) (x x j ). (2.29) Here, Q n (x) is the interpolting polynomil nd ξ n is n intermedite point in the intervl of interest (see (2.25)). It is importnt to note tht the interpoltion points influence two terms on the right-hnd-side of (2.29). The obvious one is the product n (x x j ). (2.30) The second one is f (n+1) (ξ n ) s ξ n depends on x 0,...,x n. Due to the implicit dependence of ξ n on the interpoltion points, minimizing the interpoltion error is not n esy tsk. We will return to this full problem lter on in the context of the minimx pproximtion. For the time being, we re going to focus on simpler problem, nmely, how to choose the interpoltion points x 0,...,x n such tht the product (2.30) is minimized. The solution of this problem is the topic of this section. Once gin, we would like to emphsize tht solution of this problem does not (in generl) provide n optiml choice of interpoltion points tht will minimize the interpoltion error. All tht it gurntees is tht the product prt of the interpoltion error is miniml. The tool tht we re going to use is the Chebyshev polynomils. The solution of the problem will be to interpolte t Chebyshev points. We will first introduce the Chebyshev polynomils nd the Chebyshev points nd then show why interpolting t these points minimizes (2.30). We strt by defining the Chebyshev polynomils using the following recursion reltion: T 0 (x) = 1, T 1 (x) = x, T n+1 (x) = 2xT n (x) T n 1 (x), n 1. (2.31) For exmple, T 2 (x) = 2xT 1 (x) T 0 (x) = 2x 2 1, nd T 3 (x) = 4x 3 3x. The polynomils T 1 (x),t 2 (x) nd T 3 (x) re shown in Figure 2.2. Insted of writing the recursion formul, (2.31), it is possible to write n explicit formul for the Chebyshev polynomils: Lemm 2.10 For x [ 1, 1], T n (x) = cos(n cos 1 x), n 0. (2.32) 15
2.8 Interpoltion t the Chebyshev Points D. Levy 1 0.8 0.6 T 3 (x) T 1 (x) 0.4 0.2 0 0.2 0.4 0.6 0.8 T 2 (x) 1 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 x Figure 2.2: The Chebyshev polynomils T 1 (x),t 2 (x) nd T 3 (x) Proof. Stndrd trigonometric identities imply tht Hence cos(n + 1)θ = cos θ cos nθ sin θ sin nθ, cos(n 1)θ = cos θ cos nθ + sinθ sin nθ. cos(n + 1)θ = 2 cos θ cosnθ cos(n 1)θ. (2.33) We now let θ = cos 1 x, i.e., x = cos θ, nd define t n (x) = cos(n cos 1 x) = cos(nθ). Then by (2.33) t 0 (x) = 1, t 1 (x) = x, t n+1 (x) = 2xt n (x) t n 1 (x), n 1. Hence t n (x) = T n (x). Wht is so specil bout the Chebyshev polynomils, nd wht is the connection between these polynomils nd minimizing the interpoltion error? We re bout to nswer these questions, but before doing so, there is one more issue tht we must clrify. We define monic polynomil s polynomil for which the coefficient of the leding term is one, i.e., polynomil of degree n is monic, if it is of the form x n + n 1 x n 1 +... + 1 x + 0. 16
D. Levy 2.8 Interpoltion t the Chebyshev Points Note tht Chebyshev polynomils re not monic: the definition (2.31) implies tht the Chebyshev polynomil of degree n is of the form T n (x) = 2 n 1 x n +... This mens tht T n (x) divided by 2 n 1 is monic, i.e., 2 1 n T n (x) = x n +... A generl result bout monic polynomils is the following Theorem 2.11 If p n (x) is monic polynomil of degree n, then mx p n(x) 2 1 n. (2.34) 1 x 1 Proof. We prove (2.34) by contrdiction. Suppose tht Let p n (x) < 2 1 n, x 1. q n (x) = 2 1 n T n (x), nd let x j be the following n + 1 points ( ) jπ x j = cos, 0 j n. n Since we hve Hence ( T n cos jπ ) = ( 1) j, n ( 1) j q n (x j ) = 2 1 n. ( 1) j p n (x j ) p n (x j ) < 2 1 n = ( 1) j q n (x j ). This mens tht ( 1) j (q n (x j ) p n (x j )) > 0, 0 j n. Hence, the polynomil (q n p n )(x) oscilltes (n + 1) times in the intervl [ 1, 1], which mens tht (q n p n )(x) hs t lest n distinct roots in the intervl. However, p n (x) nd q n (x) re both monic polynomils which mens tht their difference is polynomil of degree n 1 t most. Such polynomil cn not hve more thn n 1 distinct roots, which leds to contrdiction. Note tht p n q n cn not be the zero polynomil becuse 17
2.8 Interpoltion t the Chebyshev Points D. Levy tht will imply tht p n (x) nd q n (x) re identicl which gin is not possible due to the ssumptions on their mximum vlues. We re now redy to use Theorem 2.11 to figure out how to reduce the interpoltion error. We know by Theorem 2.8 tht if the interpoltion points x 0,...,x n [ 1, 1], then there exists ξ n ( 1, 1) such tht the distnce between the function whose vlues we interpolte, f(x), nd the interpoltion polynomil, Q n (x), is mx f(x) Q n (x) x 1 We re interested in minimizing n mx (x x x 1 j ). 1 (n + 1)! mx x 1 f (n+1) (x) mx x 1 n (x x j ). We note tht n (x x j) is monic polynomil of degree n + 1 nd hence by Theorem 2.11 mx x 1 n (x x j ) 2 n. The miniml vlue of 2 n cn be ctully obtined if we set 2 n T n+1 (x) = n (x x j ), which is equivlent to choosing x j s the roots of the Chebyshev polynomil T n+1 (x). Here, we hve used the obvious fct tht T n (x) 1. Wht re the roots of the Chebyshev polynomil T n+1 (x)? By Lemm 2.10 T n+1 (x) = cos((n + 1) cos 1 x). The roots of T n+1 (x), x 0,...,x n, re therefore obtined if ( (n + 1) cos 1 (x j ) = j + 1 ) π, 0 j n, 2 i.e., the (n + 1) roots of T n+1 (x) re ( ) 2j + 1 x j = cos 2n + 2 π, 0 j n. (2.35) The roots of the Chebyshev polynomils re sometimes referred to s the Chebyshev points. The formul (2.35) for the roots of the Chebyshev polynomil hs the following geometricl interprettion. In order to find the roots of T n (x), define α = π/n. Divide 18
D. Levy 2.8 Interpoltion t the Chebyshev Points The unit circle 5π 8 3π 8 7π 8 π 8-1 x 0 x 1 0 x 2 x 3 1 x Figure 2.3: The roots of the Chebyshev polynomil T 4 (x), x 0,...,x 3. Note tht they become dense next to the boundry of the intervl the upper hlf of the unit circle into n + 1 prts such tht the two side ngles re α/2 nd the other ngles re α. The Chebyshev points re then obtined by projecting these points on the x-xis. This procedure is demonstrted in Figure 2.3 for T 4 (x). The following theorem summrizes the discussion on interpoltion t the Chebyshev points. It lso provides n estimte of the error for this cse. Theorem 2.12 Assume tht Q n (x) interpoltes f(x) t x 0,...,x n. Assume lso tht these (n + 1) interpoltion points re the (n + 1) roots of the Chebyshev polynomil of degree n + 1, T n+1 (x), i.e., ( ) 2j + 1 x j = cos 2n + 2 π, 0 j n. Then x 1, f(x) Q n (x) 1 2 n (n + 1)! mx f (n+1) (ξ). (2.36) ξ 1 Exmple 2.13 Problem: Let f(x) = sin(πx) in the intervl [ 1, 1]. Find Q 2 (x) which interpoltes f(x) in the Chebyshev points. Estimte the error. Solution: Since we re sked to find n interpoltion polynomil of degree 2, we need 3 interpoltion points. We re lso sked to interpolte t the Chebyshev points, nd hence we first need to compute the 3 roots of the Chebyshev polynomil of degree 3, T 3 (x) = 4x 3 3x. 19
2.8 Interpoltion t the Chebyshev Points D. Levy The roots of T 3 (x) cn be esily found from x(4x 2 3) = 0, i.e., 3 3 x 0 = 2,,x 1 = 0, x 2 = 2. The corresponding vlues of f(x) t these interpoltion points re ( ) 3 f(x 0 ) = sin 2 π 0.4086, f(x 1 ) = 0, f(x 2 ) = sin ( ) 3 2 π 0.4086. The first-order divided differences re f[x 0,x 1 ] = f(x 1) f(x 0 ) x 1 x 0 0.4718, f[x 1,x 2 ] = f(x 2) f(x 1 ) x 2 x 1 0.4718, nd the second-order divided difference is f[x 0,x 1,x 2 ] = f[x 1,x 2 ] f[x 0,x 1 ] x 2 x 0 = 0. The interpoltion polynomil is Q 2 (x) = f(x 0 ) + f[x 0,x 1 ](x x 0 ) + f[x 0,x 1,x 2 ](x x 0 )(x x 1 ) 0.4718x. The originl function f(x) nd the interpolnt t the Chebyshev points, Q 2 (x), re plotted in Figure 2.4. As of the error estimte, x 1, sin πx Q 2 (x) 1 2 2 3! mx ξ 1 (sin πt)(3) π3 2 2 3! 1.292 A brief exmintion of Figure 2.4 revels tht while this error estimte is correct, it is fr from being shrp. Remrk. In the more generl cse where the interpoltion intervl for the function f(x) is x [,b], it is still possible to use the previous results by following the following steps: Strt by converting the interpoltion intervl to y [ 1, 1]: x = (b )y + ( + b). 2 This converts the interpoltion problem for f(x) on [,b] into n interpoltion problem for f(x) = g(x(y)) in y [ 1, 1]. The Chebyshev points in the intervl y [ 1, 1] re the roots of the Chebyshev polynomil T n+1 (x), i.e., ( ) 2j + 1 y j = cos 2n + 2 π, 0 j n. 20
D. Levy 2.9 Hermite Interpoltion 1 0.8 f(x) 0.6 0.4 0.2 0 Q 2 (x) 0.2 0.4 0.6 0.8 1 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 x Figure 2.4: The function f(x) = sin(π(x)) nd the interpoltion polynomil Q 2 (x) tht interpoltes f(x) t the Chebyshev points. See Exmple 2.13. The corresponding n + 1 interpoltion points in the intervl [,b] re x j = (b )y j + ( + b), 0 j n. 2 We now hve n mx (y y j ) y [,b] = b 2 so tht the interpoltion error is f(y) Q n (y) 1 2 n (n + 1)! n+1 mx x 1 b 2 n (x x j ), n+1 mx f (n+1) (ξ). (2.37) ξ [,b] 2.9 Hermite Interpoltion We now turn to slightly different interpoltion problem in which we ssume tht in ddition to interpolting the vlues of the function t certin points, we re lso interested in interpolting its derivtives. Interpoltion tht involves the derivtives is clled Hermite interpoltion. Such n interpoltion problem is demonstrted in the following exmple: Exmple 2.14 Problem: Find polynomils p(x) such tht p(1) = 1, p (1) = 1, nd p(0) = 1. 21
2.9 Hermite Interpoltion D. Levy Solution: Since three conditions hve to be stisfied, we cn use these conditions to determine three degrees of freedom, which mens tht it is resonble to expect tht these conditions uniquely determine polynomil of degree 2. We therefore let p(x) = 0 + 1 x + 2 x 2. The conditions of the problem then imply tht 0 + 1 + 2 = 1, 1 + 2 2 = 1, 0 = 1. Hence, there is indeed unique polynomil tht stisfies the interpoltion conditions nd it is p(x) = x 2 3x + 1. In generl, we my hve to interpolte high-order derivtives nd not only firstorder derivtives. Also, we ssume tht for ny point x j in which we hve to stisfy n interpoltion condition of the form p (l) (x j ) = f(x j ), (with p (l) being the l th -order derivtive of p(x)), we re lso given ll the vlues of the lower-order derivtives up to l s prt of the interpoltion requirements, i.e., p (i) (x j ) = f (i) (x j ), 0 i l. If this is not the cse, it my not be possible to find unique interpolnt s demonstrted in the following exmple. Exmple 2.15 Problem: Find p(x) such tht p (0) = 1 nd p (1) = 1. Solution: Since we re sked to interpolte two conditions, we my expect them to uniquely determine liner function, sy p(x) = 0 + 1 x. However, both conditions specify the derivtive of p(x) t two distinct points to be of different vlues, which mounts to contrdicting informtion on the vlue of 1. Hence, liner polynomil cn not interpolte the dt nd we must consider higherorder polynomils. Unfortuntely, polynomil of order 2 will no longer be unique becuse not enough informtion is given. Note tht even if the prescribed vlues of the derivtives were identicl, we will not hve problems with the coefficient of the liner term 1, but we will still not hve enough informtion to determine the constnt 0. 22
D. Levy 2.9 Hermite Interpoltion A simple cse tht you re probbly lredy fmilir with is the Tylor series. When viewed from the point of view tht we dvocte in this section, one cn consider the Tylor series s n interpoltion problem in which one hs to interpolte the vlue of the function nd its first n derivtives t given point, sy x 0, i.e., the interpoltion conditions re: p (j) (x 0 ) = f (j) (x 0 ), 0 j n. The unique solution of this problem in terms of polynomil of degree n is p(x) = f(x 0 ) + f (x 0 )(x x 0 ) +... + f(n) (x 0 ) (x x 0 ) n = n! which is the Tylor series of f(x) expnded bout x = x 0. 2.9.1 Divided differences with repetitions f (j) (x 0 ) (x x 0 ) j, j! We re now redy to consider the Hermite interpoltion problem. The first form we study is the Newton form of the Hermite interpoltion polynomil. We strt by extending the definition of divided differences in such wy tht they cn hndle derivtives. We lredy know tht the first derivtive is connected with the first-order divided difference by f (x 0 ) = lim x x0 f(x) f(x 0 ) x x 0 = lim x x0 f[x,x 0 ]. Hence, it is nturl to extend the notion of divided differences by the following definition. Definition 2.16 The first-order divided difference with repetitions is defined s f[x 0,x 0 ] = f (x 0 ). (2.38) In similr wy, we cn extend the notion of divided differences to high-order derivtives s stted in the following lemm (which we leve without proof). Lemm 2.17 Let x 0 x 1... x n. Then the divided differences stisfy f[x 0,...x n ] = f[x 1,...,x n ] f[x 0,...,x n 1 ] x n x 0, x n x 0, f (n) (x 0 ), x n = x 0. n! (2.39) We now consider the following Hermite interpoltion problem: The interpoltion points re x 0,...,x l (which we ssume re ordered from smll to lrge). At ech interpoltion point x j, we hve to stisfy the interpoltion conditions: p (i) (x j ) = f (i) (x j ), 0 i m j. 23
2.9 Hermite Interpoltion D. Levy Here, m j denotes the number of derivtives tht we hve to interpolte for ech point x j (with the stndrd nottion tht zero derivtives refers to the vlue of the function only). In generl, the number of derivtives tht we hve to interpolte my chnge from point to point. The extended notion of divided differences llows us to write the solution to this problem in the following wy: We let n denote the totl number of points including their multiplicities (tht correspond to the number of derivtives we hve to interpolte t ech point), i.e., n = m 1 + m 2 +... + m l. We then list ll the points including their multiplicities (tht correspond to the number of derivtives we hve to interpolte). To simplify the nottions we identify these points with new ordered list of points y i : {y 0,...,y n 1 } = {x 0,...,x }{{} 0,x 1,...,x 1,...,x }{{} l,...,x l }. }{{} m 1 m l The interpoltion polynomil p n 1 (x) is given by j=1 m 2 n 1 j 1 p n 1 (x) = f[y 0 ] + f[y 0,...,y j ] (x y k ). (2.40) k=0 Whenever point repets in f[y 0,...,y j ], we interpret this divided difference in terms of the extended definition (2.39). In prctice, there is no need to shift the nottions to y s nd we work directly with the originl points. We demonstrte this interpoltion procedure in the following exmple. Exmple 2.18 Problem: Find n interpoltion polynomil p(x) tht stisfies p(x 0 ) = f(x 0 ), p(x 1 ) = f(x 1 ), p (x 1 ) = f (x 1 ). Solution: The interpoltion polynomil p(x) is p(x) = f(x 0 ) + f[x 0,x 1 ](x x 0 ) + f[x 0,x 1,x 1 ](x x 0 )(x x 1 ). The divided differences: f[x 0,x 1 ] = f(x 1) f(x 0 ) x 1 x 0. f[x 0,x 1,x 1 ] = f[x 1,x 1 ] f[x 1,x 0 ] x 1 x 0 Hence = f (x 1 ) f(x1) f(x0) x 1 x 0. x 1 x 0 p(x) = f(x 0 )+ f(x 1) f(x 0 ) x 1 x 0 (x x 0 )+ (x 1 x 0 )f (x 1 ) [f(x 1 ) f(x 0 )] (x 1 x 0 ) 2 (x x 0 )(x x 1 ). 24
D. Levy 2.9 Hermite Interpoltion 2.9.2 The Lgrnge form of the Hermite interpolnt In this section we re interested in writing the Lgrnge form of the Hermite interpolnt in the specil cse in which the nodes re x 0,...,x n nd the interpoltion conditions re p(x i ) = f(x i ), p (x i ) = f (x i ), 0 i n. (2.41) We look for n interpolnt of the form p(x) = f(x i )A i (x) + f (x i )B i (x). (2.42) In order to stisfy the interpoltion conditions (2.41), the polynomils p(x) in (2.42) must stisfy the 2n + 2 conditions: A i (x j ) = δ ij, B i (x j ) = 0, i,j = 0,...,n. (2.43) A i(x j ) = 0, B i(x j ) = δ ij, We thus expect to hve unique polynomil p(x) tht stisfies the constrints (2.43) ssuming tht we limit its degree to be 2n + 1. It is convenient to strt the construction with the functions we hve used in the Lgrnge form of the stndrd interpoltion problem (Section 2.5). We lredy know tht l i (x) = n j i x x j x i x j, stisfy l i (x j ) = δ ij. In ddition, for i j, l 2 i (x j ) = 0, (l 2 i (x j )) = 0. The degree of l i (x) is n, which mens tht the degree of l 2 i (x) is 2n. We will thus ssume tht the unknown polynomils A i (x) nd B i (x) in (2.43) cn be written s { Ai (x) = r i (x)l 2 i (x), B i (x) = s i (x)l 2 i (x). The functions r i (x) nd s i (x) re both ssumed to be liner, which implies tht deg(a i ) = deg(b i ) = 2n + 1, s desired. Now, ccording to (2.43) Hence δ ij = A i (x j ) = r i (x j )l 2 i (x j ) = r i (x j )δ ij. r i (x i ) = 1. (2.44) 25
2.9 Hermite Interpoltion D. Levy Also, 0 = A i(x j ) = r i(x j )[l i (x j )] 2 + 2r i (x j )l i (x J )l i(x j ) = r i(x j )δ ij + 2r i (x j )δ ij l i(x j ), nd thus r i(x i ) + 2l i(x i ) = 0. (2.45) Assuming tht r i (x) is liner, r i (x) = x + b, equtions (2.44),(2.45), imply tht = 2l i(x i ), b = 1 + 2l i(x i )x i. Therefore A i (x) = [1 + 2l i(x i )(x i x)]li 2 (x). As of B i (x) in (2.42), the conditions (2.43) imply tht 0 = B i (x j ) = s i (x j )li 2 (x j ) = s i (x i ) = 0, (2.46) nd δ ij = B i(x j ) = s i(x j )li 2 (x j ) + 2s i (x j )(li 2 (x j )) = s i(x i ) = 1. (2.47) Combining (2.46) nd (2.47), we obtin s i (x) = x x i, so tht B i (x) = (x x i )li 2 (x). To summrize, the Lgrnge form of the Hermite interpoltion polynomil is given by p(x) = f(x i )[1 + 2l i(x i )(x i x)]li 2 (x) + f (x i )(x x i )li 2 (x). (2.48) The error in the Hermite interpoltion (2.48) is given by the following theorem. Theorem 2.19 Let x 0,...,x n be distinct nodes in [,b] nd f C 2n+2 [,b]. If p Π 2n+1, such tht 0 i n, p(x i ) = f(x i ), p (x i ) = f (x i ), then x [,b], there exists ξ (,b) such tht f(x) p(x) = f(2n+2) (ξ) (2n + 2)! n (x x i ) 2. (2.49) 26
D. Levy 2.9 Hermite Interpoltion Proof. The proof follows the sme techniques we used in proving Theorem 2.8. If x is one of the interpoltion points, the result trivilly holds. We thus fix x s non-interpoltion point nd define w(y) = We lso hve n (y x i ) 2. φ(y) = f(y) p(y) λw(y), nd select λ such tht φ(x) = 0, i.e., λ = f(x) p(x). w(x) φ hs (t lest) n + 2 zeros in [,b]: (x,x 0,...,x n ). By Rolle s theorem, we know tht φ hs (t lest) n + 1 zeros tht re different thn (x,x 0,...,x n ). Also, φ vnishes t x 0,...,x n, which mens tht φ hs t lest 2n + 2 zeros in [,b]. Similrly, Rolle s theorem implies tht φ hs t lest 2n + 1 zeros in (,b), nd by induction, φ (2n+2) hs t lest one zero in (,b), sy ξ. Hence 0 = φ (2n+2) (ξ) = f (2n+2) (ξ) p (2n+2) (ξ) λw (2n+2) (ξ). Since the leding term in w(y) is x 2n+2, w (2n+2) (ξ) = (2n + 2)!. Also, since p(x) Π 2n+1, p (2n+2) (ξ) = 0. We recll tht x ws n rbitrry (non-interpoltion) point nd hence we hve f(x) p(x) = f(2n+2) (ξ) (2n + 2)! n (x x i ) 2. Exmple 2.20 Assume tht we would like to find the Hermite interpoltion polynomil tht stisfies: p(x 0 ) = y 0, p (x 0 ) = d 0, p(x 1 ) = y 1, p (x 1 ) = d 1. In this cse n = 1, nd l 0 (x) = x x 1 x 0 x 1, l 0(x) = 1 x 0 x 1, l 1 (x) = x x 0 x 1 x 0, l 1(x) = 1 x 1 x 0. According to (2.48), the desired polynomil is given by (check!) ] ( ) 2 [ ] ( 2 x x1 2 x x0 p(x) = y 0 [1 + (x 0 x) + y 1 1 + (x 1 x) x 0 x 1 x 0 x 1 x 1 x 0 x 1 x 0 ( ) 2 ( ) 2 x x1 x x0 +d 0 (x x 0 ) + d 1 (x x 1 ). x 0 x 1 x 1 x 0 27 ) 2
2.10 Spline Interpoltion D. Levy 2.10 Spline Interpoltion So fr, the only type of interpoltion we were deling with ws polynomil interpoltion. In this section we discuss different type of interpoltion: piecewise-polynomil interpoltion. A simple exmple of such interpolnts will be the function we get by connecting dt with stright lines (see Figure 2.5). Of course, we would like to generte functions tht re somewht smoother thn piecewise-liner functions, nd still interpolte the dt. The functions we will discuss in this section re splines. (x 1, f(x 1 )) (x 3, f(x 3 )) (x 4, f(x 4 )) (x 2, f(x 2 )) (x 0, f(x 0 )) x Figure 2.5: A piecewise-liner spline. In every subintervl the function is liner. Overll it is continuous where the regulrity is lost t the knots You my still wonder why re we interested in such functions t ll? It is esy to motivte this discussion by looking t Figure 2.6. In this figure we demonstrte wht high-order interpolnt looks like. Even though the dt tht we interpolte hs only one extrem in the domin, we hve no control over the oscilltory nture of the high-order interpolting polynomil. In generl, high-order polynomils re oscilltory, which rules them s non-prcticl for mny pplictions. Tht is why we focus our ttention in this section on splines. Splines, should be thought of s polynomils on subintervls tht re connected in smooth wy. We will be more rigorous when we define precisely wht we men by smooth. First, we pick n + 1 points which we refer to s the knots: t 0 < t 1 < < t n. A spline of degree k hving knots t 0,...,t n is function s(x) tht stisfies the following two properties: 1. On [t i 1,t i ) s(x) is polynomil of degree k, i.e., s(x) is polynomil on every subintervl tht is defined by the knots. 2. Smoothness: s(x) hs continuous (k 1) th derivtive on the intervl [t 0,t n ]. 28
D. Levy 2.10 Spline Interpoltion 2 1.5 1 Q 10 (x) 0.5 0 1 1+x 2 0.5 5 4 3 2 1 0 1 2 3 4 5 x Figure 2.6: An interpolnt goes bd. In this exmple we interpolte 11 eqully spced smples of f(x) = 1 1+x 2 with polynomil of degree 10, Q 10 (x) (t 1, f(t 1 )) (t 3, f(t 3 )) (t 4, f(t 4 )) (t 2, f(t 2 )) (t 0, f(t 0 )) x Figure 2.7: A zeroth-order (piecewise-constnt) spline. The knots re t the interpoltion points. Since the spline is of degree zero, the function is not even continuous 29
2.10 Spline Interpoltion D. Levy A spline of degree 0 is piecewise-constnt function (see Figure 2.7). A spline of degree 1 is piecewise-liner function tht cn be explicitly written s s 0 (x) = 0 x + b 0, x [t 0,t 1 ), s 1 (x) = 1 x + b 1, x [t 1,t 2 ), s(x) =.. s n 1 (x) = n 1 x + b n 1, x [t n 1,t n ], (see Figure 2.5 where the knots {t i } nd the interpoltion points {x i } re ssumed to be identicl). It is now obvious why the points t 0,...,t n re clled knots: these re the points tht connect the different polynomils with ech other. To qulify s n interpolting function, s(x) will hve to stisfy interpoltion conditions tht we will discuss below. We would like to comment lredy t this point tht knots should not be confused with the interpoltion points. Sometimes it is convenient to choose the knots to coincide with the interpoltion points but this is only optionl, nd other choices cn be mde. 2.10.1 Cubic splines A specil cse (which is the most common spline function tht is used in prctice) is the cubic spline. A cubic spline is spline for which the function is polynomil of degree 3 on every subintervl, nd function with two continuous derivtives overll (see Figure 2.8). Let s denote such function by s(x), i.e., s(x) = s 0 (x), x [t 0,t 1 ), s 1 (x), x [t 1,t 2 ),.. s n 1 (x), x [t n 1,t n ], where i, the degree of s i (x) is 3. We now ssume tht some dt (tht s(x) should interpolte) is given t the knots, i.e., s(t i ) = y i, 0 i n. (2.50) The interpoltion conditions (2.50) in ddition to requiring tht s(x) is continuous, imply tht s i 1 (t i ) = y i = s i (t i ), 1 i n 1. (2.51) We lso require the continuity of the first nd the second derivtives, i.e., s i(t i+1 ) = s i+1(t i+1 ), 0 i n 2, (2.52) s i (t i+1 ) = s i+1(t i+1 ), 0 i n 2. Before ctully computing the spline, let s check if we hve enough equtions to determine unique solution for the problem. There re n subintervls, nd in ech 30
D. Levy 2.10 Spline Interpoltion (t 1, f(t 1 )) (t 3, f(t 3 )) (t 4, f(t 4 )) (t 2, f(t 2 )) (t 0, f(t 0 )) x Figure 2.8: A cubic spline. In every subintervl [t i 1,t i, the function is polynomil of degree 2. The polynomils on the different subintervls re connected to ech other in such wy tht the spline hs second-order continuous derivtive. In this exmple we use the not--knot condition. subintervl we hve to determine polynomil of degree 3. Ech such polynomil hs 4 coefficients, which leves us with 4n coefficients to determine. The interpoltion nd continuity conditions (2.51) for s i (t i ) nd s i (t i+1 ) mount to 2n equtions. The continuity of the first nd the second derivtives (2.52) dd 2(n 1) = 2n 2 equtions. Altogether we hve 4n 2 equtions but 4n unknowns which leves us with 2 degrees of freedom. These indeed re two degrees of freedom tht cn be determined in vrious wys s we shll see below. We re now redy to compute the spline. We will use the following nottion: We lso set h i = t i+1 t i. z i = s (t i ). Since the second derivtive of cubic function is liner, we observe tht s i (x) is the line connecting (t i,z i ) nd (t i+1,z i+1 ), i.e., s i (x) = x t i h i z i+1 x t i+1 h i z i. (2.53) Integrting (2.53) once, we hve s i(x) = 1 2 (x t i) 2z i+1 h i 1 2 (x t i+1) 2 z i h i + c. 31