Draft. Chapter 2 Approximation and Interpolation. MATH 561 Numerical Analysis. Songting Luo. Department of Mathematics Iowa State University

Transcription

1 Chapter 2 Approximation and Interpolation Songting Luo Department of Mathematics Iowa State University MATH 561 Numerical Analysis Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 1 / 34

2 Approximation of Functions Approximation Given a function f and a class Φ of approximating functions φ and a norm } }. A function ˆφ P Φ is called the best approximation of f from the class Φ relative to the norm } } if For example: }f ˆφ} ď }f φ} for all φ P Φ. Least Squares Approximation: the least squares problem: min }φ f} 2,dλ. φpφ n Polynomial Interpolatioin: given tx i u n i 0 and tf i fpx i qu n i 1 of function f, find a polynomial p P P n s.t., ppx i q f i, i 0, 1,..., n. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 2 / 34

3 Approximation of Functions by Polynomials Choose polynomials as approximating functions : Polynomial Interpolation Polynomial interpolation: Vandermonde method, Lagrange formula, Barycentric formula, Newton s formula; Interpolation error; Chebyshev Nodes. Hermite interpolation: Newton s formula. Spline interpolation/approximation, piecewise Lagrange interpolation. Polynomial Approximation Weierstrass s Approximation Theorem (A Proof with Bernstein polynomial) Characterization of best approximation; Alternant set; Remez method. Least Squares Approximation; Normal equations. Orthogonal polynomials: Chebyshev polynomials, Legendre polynomials. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 3 / 34

4 Why polynomials? Weierstrass s Theorem Theorem (Weierstrass s Approximation Theorem) If fpxq P Cra, bs, then given ɛ ą 0, we can find ppxq such that sup fpxq ppxq ă ɛ. An alternative statement of it is that a continuous function is the sum of a uniformly convergent series of polynomials. For let p n1 pxq, p n2 pxq, pn 1 ď n 2 ď q be polynomials corresponding to ɛ, ɛ{2,..., ɛ{2 n,..., Then the series converges uniformly to f pxq. p n1 pxq ` tp n2 pxq p n1 pxqu ` Proof by Bernstein polynomials (Later). Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 4 / 34

5 Polynomial Interpolation Problem: given tx i u n i 0 and tf i fpx i qu n i 0 of function f, find a polynomial p P P n s.t., Existence, Uniqueness ppx i q f i, i 0, 1,..., n. Lagrange polynomials ppxq ř n i 0 f il i pxq, with Lagrange basis function (elementary Lagrange interpolation polynomial) l i pxq nź j 0;j i Existence: ppx i q f i, i 0, 1,..., n. x x j x i x j, i 0, 1, 2,..., n. Uniqueness proved by Fundamental Theorem of Algebra. Denote p as p n pf; x 0, x 1,..., x n ; xq p n pf; xq ř n i 0 fpx iql i pxq. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 5 / 34

6 Polynomial Interpolation; Interpolation Operator Consider Lagrange interpolation as a linear operator P n : Cra, bs ÞÑ P n, P n p q p n pf; q By uniqueness P n pfq P P n. Operator norm: e.g., }P n } max fpcra, bs }P n pfq} }f} }P n } 8 Λ n }λ n pxq} 8 } nÿ l i pxq } 8. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 6 / 34 i 0

7 Polynomial Interpolation; Interpolation Error Using best approximation of f on ra, bs by polynomials of degree ď n defined as E n pfq min ppp n }f p} 8 }f ˆp n } 8, we see the interpolation error: }f p n pf; q} 8 }f ˆp n p n pf ˆp n ; q} 8 ď }f ˆp n } 8 ` Λ n }f ˆp n } 8 So We know is }f p n pf; q} 8 ď p1 ` Λ n qepfq lim nñ8 Epfq 0, lim }f p npf; q} 8 0? nñ8 Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 7 / 34

8 Polynomial Interpolation; Interpolation Error Error of interpolation: fpxq p n pf; xq for any x x i in ra, bs. Interpolation Error fpxq p n pf; xq f pn`1q pξpxqq pn ` 1q! for some ξpxq P pa, bq. Proof nź px x i q, x P ra, bs, Define F ptq fptq p n pf; tq fpxq pnpf;xq ś n i 0 px x iq ś n i 0 pt x iq i 0 F P C n`1 ra, bs (assume f P C n`1 ra, bs) F px i q 0, i 0, 1,..., n; F pxq 0. By Rolle s Theorem, F pn`1q pξpxqq 0 for some ξpxq P pa, bq ñ 0 f pn`1q pξpxqq fpxq p npf; xq ś n i 0 px x pn ` 1q!. iq Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 8 / 34

9 Examples. 1. Linear interpolation (n D 1). Assume that x0 x x 1 ;thatis, [a; b]= [ x0;x1], and let h D x 1 x 0.Thenby( 2.60)and( 2.61), f.x/ p 1.f I x/ D.x x 0 /.x x 1 / f 00./ ; x0 <<x 1 ; 2 and an easy computation gives kf p 1.f I/k 1 M 82 h 2 ; M 2 Dkf 00 k 1 : (2.63) Here the 1-norm refers to the interval [x 0 ;x 1 ]. Thus, on small intervals of length h, the error for linear interpolation is O.h 2 /.

10 2. Quadratic interpolation ( n D 2)onequally spaced points x 0, x 1 D x 0 C h, x2 D x 0 C 2h. We now have, for x 2 Œx 0 ;x 2, f.x/ p 2.f I x/ D.x x 0 /.x x 1 /.x x 2 / f / ; x0 <<x 2 ; and (cf. Ex. 43(a)) giving an error of O.h 3 /. kf p 2.f I/k 1 M p h3 ; M 3 Dkf 000 k 1 ;

11 Larger the 'n', better the accuracy??? 3. nth-degree interpolation on equally spaced points i D x0 C ih, i D 0;1; :::;n. When h is small, and x0 x x n,then. x/x in ( 2.60) is constrained to a relatively small interval and f.nc1/..x// cannot vary a great deal. The behavior of the error, therefore, is mainly determined by the product Q n id0.x x i /, the graph of which, for n D 7, is shown in Fig We clearly have symmetry with respect to the midpoint.x0 C x n /= 2. It can also be shown that the relative extrema decrease monotonically in modulus as one moves from the endpoints to the center (cf. Ex. 29(c)). It is evident that the oscillations become more violent as n increases. In particular, the curve is extremely steep at the endpoints, and takes off to 1 rapidly as x moves away from the interval [x 0 ;x n ]. Although it is true that the curve representing the interpolation error is scaled by a factor of O.h nc1 /,it is also clear that one ought to interpolate near the center zone of the interval [x 0 ;x n ], if at all possible, and should avoid interpolation near the end zones, or even extrapolation outside the interval. The highly oscillatory nature of the error curve, when n is large, also casts some legitimate doubts about convergence of the interpolation process as n!1. This is studied in the next section.

12 Runge's phenomenon Fig. 2.4 Interpolation error for eight equally spaced points

13 Lagrange Interpolation; Convergence Definition (Convergence of Lagrange Interpolation) For a triangular array of interpolation nodes on ra, bs: x p0q 0 x p1q 0 x p1q 1 x p2q 0 x p2q 1 x p2q 2 x pnq 0 x pnq 1 x pnq 2 x pnq n define p n pxq p n pf; x pnq 0, xpnq 1, xpnq 2,..., xpnq n ; xq, x P ra, bs. We say Lagrange interpolation based on the triangular array of nodes converges if p n pxq Ñ fpxq as n Ñ 8, uniformly for x P ra, bs. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 9 / 34

14 Lagrange Interpolation; Convergence By the interpolation error, M n`1 fpxq p n pxq ď pb aq n`1, x P ra, bs, pn ` 1q! where f pkq pxq ď M k for a ď x ď b, k 0, 1, 2,... then convergence is obtained if lim kñ8 pb aq k k! M k 0. Theorem (Convergence) Lagrange interpolation converges (uniformly on ra, bs) for an arbitrary triangular set of nodes in ra, bs if f is analytic in the circular disk C r centered at pa ` bq{2 and having radius r s.t., r ą 3 2pb aq. Proof by Cauchy s Formula, f pkq pxq k! 2πi BC r fpzq dz, x P ra, bs. pz kqk`1 Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 10 / 34

15 Fig. 2.5 The circular disk Cr

16 Lagrange Interpolation; Convergence; A Sharp Bound Let dµptq be the limit distribution of nodes, i.e., ż b a # of nodes in ra, bs dµptq. n ` 1 A curve of constant logarithmic potential (constant γ): Let Theorem (Convergence) tz P C : upzq γ, upzq ż b a ln 1 z t dµptq. Γ sup γ. curves upzq γ containing ra, bs For the domain C Γ tz P C : upzq ě Γu, if f is analytic in any domain C containing C Γ in its interior, then fpzq p n pf; zq Ñ 0 as n Ñ 8 uniformly for z P C Γ. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 11 / 34

17 3. Runge s 7 example: f.x/ D 1 ; 5 x 5; 1 C x2 x.n/ k D 5 C k n 10 ; k D 0; 1; 2; : : : ; n: (2.75) Here the nodes are equally spaced, hence asymptotically equally distributed. Note that f.z/ has poles at z D i. These poles lie definitely inside the region C in Fig. 2.6 for the interval [ 5,5], so that f is not analytic in C. For this reason, we can no longer expect convergence on the whole interval [ 5,5]. It has been shown, indeed, that 8 lim jf.x/ p n.f I x/j D n!1 < 0 if jxj <3:633:::; : 1 if jxj >3:633:::: (2.76) We have convergence in the central zone of the interval [ 5,5], but divergence in the lateral zones. With Fig. 2.4 kept in mind, this is perhaps not all that surprising (cf. MA 7(b)).

18 Chebyshev Polynomial Choice of nodes influences the convergence. E.g., Chebyshev nodes. Assume considering interval r 1, 1s (arbitrary ra, bs by linear map). Chebyshev Polynomial of First Kind Chebyshev polynomials of first kind one can show T n pxq cospn cos 1 pxqq. T 0 pxq 1, T 1 pxq x, T k`1 pxq 2xT k pxq T k 1 pxq, k 1, 2, 3,... Leading coefficient of T n is 2 n 1 ; monic Chebyshev polynomial of degree n T n pxq 1 2 n 1 T npxq, n ě 1; T 0 T 0. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 12 / 34

19 Chebyshev Nodes From cos nθ T n pcos θq, Nodes Zeros of T n : x pnq k Extrema of T n : y pnq k cos θ pnq k cos η pnq k k, θpnq k 2k 1 π, k 1, 2,..., n. 2n, ηpnq Thus, T n pxq ś n k 1px xpnqq, and k k π, k 0, 1, 2,..., n. n T n px pnq k q 0, for xpnq k cos 2k 1 π, k 1, 2,..., n; 2n T n py pnq k q p 1qk, for y pnq k cos k π, k 0, 1, 2,..., n; n Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 13 / 34

20 Chebyshev Polynomial Theorem (Chebyshev) For any arbitrary monic polynomial p n of degree n, there holds max p npxq ě max T n pxq 1 1ďxď1 1ďxď1 2 n 1, n ě 1 Proof by contradiction. Assume that max 1ďxď1 p n pxq ă 1 2 n 1. Then the polynomial d n pxq T n pxq p n pxq has degree ď n 1 and satisfies p 1q k d n py pnq q ą 0, for k 0, 1, 2,..., n. Therefore d n pxq 0. Contradiction. From the theorem: min p n 1 PP n 1 k max 1ďxď1 xn p n 1 pxq max 1ďxď1 xn px n T n pxqq. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 14 / 34

21 Convergence with Chebyshev Nodes Convergence Recall interpolation error: fpxq p n pf; xq f pn`1q pξpxqq pn ` 1q! nź px x i q, x P r 1, 1s. By Chebyshev s Theorem, } ś n i 0 px x iq} 8 is minimized with x i being the zeros of T n`1, given as, cos 2i`1 2n`2π, i 0, 1, 2,..., n. With these nodes interpolation error: ˆx pnq i i 0 }fp q p n pf; q} 8 ď }f pn`1q } 8 pn`1q! 2. n On triangular array of Chebyshev nodes in r 1, 1s, p n pf; ˆx pnq 0, ˆxpnq 1 1,..., ˆxpnq n ; xq Ñ fpxq as n Ñ 8, uniformly on r 1, 1s provided that f P C 1 r 1, 1s. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 15 / 34

22 Fourier Expansion in Chebyshev Polynomial Orthogonality ż 1 1 Fourier Expansion where $ 0, if k l dx & T k pxqt l pxq? π, if k l 0 1 x 2 % π 2, if k l ą 0 fpxq ÿ 8 c j 2 π ż 1 1 j 0 1c j T j pxq 1 2 c 0 ` ÿ8 j 1 c jt j pxq, dx fpxqt j pxq?, j 0, 1, 2, x 2 Truncated sum τ n pxq ř n 1 j 0 c j T j pxq approximate f with error fpxq τ n pxq ÿ 8 c jt j pxq «c n`1 T n`1 pxq. j n`1 Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 16 / 34

23 Lagrange Interpolation; Barycentric Formula For efficient computation, rewrite Lagrange polynomial in a different form. Denote λ p0q 0 1, λ pnq i nź j 0,j i 1 x i x j, i 0, 1,..., n; n 1, 2, 3,.... Then elementary Lagrange interpolation polynomials λpnq i l i pxq ω n pxq, i 0, 1,..., n; ω n pxq x x i By ř n i 0 l ipxq 1, p n pf; xq nź px x j q. j 0 nÿ ř n i 0 f i l i pxq f ř n il i pxq i 0 ř n i 0 l f i λpnq i x x i ω n pxq ipxq ř. n λ pnq i i 0 x x i ω n pxq i 0 Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 17 / 34

24 Lagrange Interpolation; Barycentric Formula Barycentric formula for Lagrange interpolation polynomial: ř n i 0 f i λpnq And l i pxq Efficient computation of λ pnq n : λ p0q p n pf; xq λ pnq i x x i ř n λ pnq i i 0 i x x i ř. n λ pnq i i 0 x x i x x i, i 0, 1,..., n. 0 1, for k 1, 2,..., n do λ pkq λpk 1q i i λ pkq k, i 0, 1, 2,..., k 1, x i x k 1 ś k 1 j 0 px k x j q. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 18 / 34

25 We thus arrive at the following algorithm:.0/ 0 D 1; for k D 1;2;:::;n do k/ i.k/ k.k 1/ D x i i x k ; i D 0;1;:::;k 1; 1 D Q k 1 j D0.x k x j / :

26 Lagrange Interpolation; Barycentric Formula Efficient and stable computation of λ pnq 0, λpnq 1,..., λpnq n : 1 2npn ` 1q subtractions, 1 2 pn 1qn multiplications, 1 2npn ` 3q divisions. (compared to Opn 3 q in original form). If to incorporate next point px n`1, f n`1 q, generate λ pn`1q 0, λ pn`1q 1,..., λ pn`1q n`1, then compute p n`1pf; xq. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 19 / 34

27 Lagrange Interpolation; Newton s Formula Both function values and derivative values can be given. (Denote p n pxq p n pf; x 0, x 1,..., x n ; xq, n 0, 1, 2,....) Newton s Form For some constants a 0, a 1, a 2,...., one has p 0 pxq a 0, p n pxq p n 1 pxq ` a n px x 0 qpx x 1 q px x n 1 q, n 1, 2, 3,..., so the interpolation polynomial is p n pf; xq a 0 ` a 1 px x 0 q ` a 2 px x 0 qpx x 1 q ` ` a n px x 0 qpx x 1 q px x n 1 q. What are the constants a 0, a 1,..., a n? Determined by fpx i q p n px i q in principle. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 20 / 34

28 Newton s Formula; Divided Differences n-th divided difference of f relative to nodes x 0, x 1,..., x n : denoted as computed as a 0 rx 0 sf f 0, for k 1, 2,..., n a n rx 0, x 1,..., x n sf, n 0, 1, 2,..., a k rx 0, x 1,..., x k sf rx 1, x 2,..., x k sf rx 0, x 1,..., x k 1 sf x k x 0 A table of divided differences... each entry is the difference of the entry immediately to the left and the one above it, divided by the difference of the x-value horizontally to the left and the one opposite the f-value found by going diagonally up. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 21 / 34

29 Newton s Formula; Divided Differences Table of divided difference: x f x 0 f 0 x 1 f 1 rx 0, x 1 sf x 2 f 2 rx 1, x 2 sf rx 0, x 1, x 2 sf x 3 f 3 rx 2, x 3 sf rx 1, x 2, x 3 sf rx 0, x 1, x 2, x 3 sf Diagonal entries are the coefficients in Newton s formula. (npn ` 1q additions and 1 2npn ` 1q divisions for computing first n ` 1 diagonal entries) Adding one more term (one more node t): p n`1 pf; x 0, x 1,..., x n, t; xq p n pf; xq ` rx 0, x 1,..., x n, tsf nź px x i q. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 22 / 34 i 0

30 Example. x f (4 3)/(1 0) D (7 4)/(2 1) D 3 (3 1)/(2 0) D (19 7)/(4 2) D 6 (6 3)/(4 1) D 1 (1 1)/(4 0) D 0 The cubic interpolation polynomial is p 3.f I x/ D 3 C 1.x 0/ C 1.x 0/.x 1/ C 0.x 0/.x 1/.x 2/ D 3 C x C x.x 1/ D 3 C x 2 ; which indeed is the function tabulated. Note that the leading coefficient of p 3.f I/ is zero, which is why the last divided difference turned out to be 0.

31 Newton s Formula; Interpolation Error The interpolation error: fpxq p n pf; xq rx 0, x 1,..., x n, xsf From previous result: nź px x i q. i 0 rx 0, x 1,..., x n, xsf f pn`1q pξpxqq pn ` 1q! Letting x x n`1, and replacing n ` 1 by n, rx 0, x 1,..., x n sf 1 n! f pnq pξq. Let x 1, x 2,..., x n tend to x 0, we have rx 0, x 0,..., x 0 sf 1 n! f pnq px 0 q, i.e., the n-th divided difference at n ` 1 confluent points should be defined to be the n-th derivative at this point divided by n!. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 23 / 34

32 Hermite Interpolation Hermite Interpolation Problem Given K ` 1 distinct points x 0, x 1,..., x K P ra, bs, corresponding integers m 0, m 1,..., m K ě 1, and given a function f P C M 1 ra, bs with M max k m k, find a polynomial p of lowest degree such that for k 0, 1,..., K, p pµq px k q f pµq k f pµq px k q, µ 0, 1,..., m k 1. Solved as Lagrange interpolation, with Newton s form. E.g., if m k 3, then x f x k f k x k f k f 1 k x k f k f 1 1 k 2 f 2 k Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 24 / 34

33 Special case with only first derivatives (see Burden & Faries) If f C 1 [a, b] and x 0,..., x n [a, b] are distinct, the unique polynomial of least degree agreeing with f and f at x 0,..., x n is the Hermite polynomial of degree at most 2n + 1 given by H 2n+1 (x) = n n f(x j )H n, j (x) + f (x j )Ĥ n, j (x), j=0 j=0 where, for L n, j (x) denoting the jth Lagrange coefficient polynomial of degree n, wehave H n, j (x) =[1 2(x x j )L n, j (x j)]l n 2, j (x) and Ĥ n, j (x) = (x x j )L n 2, j (x). Moreover, if f C 2n+2 [a, b], then f(x) = H 2n+1 (x) + (x x 0) 2...(x x n ) 2 f (2n+2) (ξ(x)), (2n + 2)! for some (generally unknown) ξ(x) in the interval (a, b).

34 Proof First recall that L n, j (x i ) = { 0, if i = j, 1, if i = j. Hence when i = j, H n, j (x i ) = 0 and Ĥ n, j (x i ) = 0, whereas, for each i, H n,i (x i ) =[1 2(x i x i )L n,i (x i)] 1 = 1 and Ĥ n,i (x i ) = (x i x i ) 1 2 = 0. As a consequence H 2n+1 (x i ) = n f(x j ) 0 + f(x i ) 1 + j=0 j =i so H 2n+1 agrees with f at x 0, x 1,..., x n. n f (x j ) 0 = f(x i ), j=0

35 To show the agreement of H 2n+1 with f at the nodes, first note that L n, j (x) is a factor of H n, j (x), soh n, j (x i) = 0 when i = j. In addition, when i = j we have L n,i (x i ) = 1, so H n,i (x i) = 2L n,i (x i) L n 2,i (x i ) +[1 2(x i x i )L n,i (x i)]2l n,i (x i )L n,i (x i) = 2L n,i (x i) + 2L n,i (x i) = 0. Hence, H n, j (x i) = 0 for all i and j. Finally, Ĥ n, j (x i) = L n 2, j (x i ) + (x i x j )2L n, j (x i )L n, j (x i) = L n, j (x i )[L n, j (x i ) + 2(x i x j )L n, j (x i)], so Ĥ n, j (x i) = 0ifi = j and Ĥ n,i (x i) = 1. Combining these facts, we have H 2n+1 (x i) = n f(x j ) 0 + j=0 n f (x j ) 0 + f (x i ) 1 = f (x i ). j=0 j =i Therefore, H 2n+1 agrees with f and H 2n+1 with f at x 0, x 1,..., x n.

36 1. Find p 2 P 3 such that p.x 0 / D f 0 ;p 0.x 0 / D f 0 0 ;p00.x 0 / D f 00 0 ;p000.x 0 / D f : Here K D 0, m 0 D 4, that is, we have a single quadruple point. The table of divided differences is precisely the one in (2.120) (with k D 0); hence Newton s formula becomes p.x/ D f 0 C.x x 0 /f 0 0 C 1 2.x x 0/ 2 f 00 0 C 1 6.x x 0/ 3 f ; which is nothing but the Taylor polynomial of degree 3. Thus Taylor s polynomial is a special case of a Hermite interpolation polynomial. The error term of interpolation, furthermore, gives us f.x/ p.x/ D 1 24.x x 0/ 4 f.4/./; between x 0 and x; which is Lagrange s form of the remainder term in Taylor s formula.

37 2. Find p 2 P 3 such that p. x 0 / D f 0 ; p.x 1 / D f 1 ; p 0.x 1 / D f1 0 ; p.x 2 / D f 2 ; where x 0 <x 1 <x 2 (cf. Fig. 2.8). The table of divided differences now has the form: x f x 0 f 0 x 1 f 1 Œx 0 ;x 1 f x 1 f 1 f 0 1 Œx 0 ;x 1 ;x 1 f x 2 f 2 Œx 1 ;x 2 f Œx 1 ;x 1 ;x 2 f Œx 0 ;x 1 ;x 1 ;x 2 f. If we denote the diagonal entries, as before, by a 0, a 1, a 2, a 3, Newton s formula takes the form p.x/ D a 0 C a 1.x x 0 / C a 2.x x 0 /.x x 1 / C a 3.x x 0 /.x x 1 / 2 ; and the error formula becomes

38 f.x/ p.x/ D.x x 0 /.x x 1 / 2.x x 2 / f.4/./ ; x 0 <<x 2 : 4Š For equally spaced points, say, x 0 D x 1 h, x 2 D x 1 Ch,wehave,ifx D x 1 Cth, 1 t 1, j.x x 0 /.x x 1 / 2.x x 2 /jdj.t 2 1/t 2 h 4 j 1 4 h4 ; and so kf pk kf / 4 k 1 h h4 D kf.4/ k 1 ; with the 1-norm referring to the interval [x 0 ;x 2 ].

39 Approximation and Interpolation by Spline Functions Given a partition (subdivision) of ra, bs, with : a x 1 ă x 2 ă ă x n 1 ă x n b max x i, x i x i`1 x i. 1ďiďn 1 Recall the spline functions of degree m and smoothness class k relative to the subdivision, S k mp q ts : s P C k ra, bs, s rxi, x i`1 s P P m, i 1, 2,..., n 1u. I.e., any function in S k m is piecewise polynomial of degree ď m, and upto kth derivative is continuous everywhere including points x 2,..., x n 1 of. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 25 / 34

40 Fig. 2.9 A function s 2 S 1 m

41 Fig Piecewise linear interpolation.

42 Piecewise Linear Interpolation Interpolation by Piecewise Linear Functions Find s P S 0 1p q such that for a given function f defined on ra, bs, spx i q f i where f i fpx i q, i 1, 2,..., n. The solution is given by sp q s 1 pf; q, s 1 pf; xq f i ` px x i qrx i, x i`1 sf for x i ď x ď x i`1, i 1, 2,..., n 1. I.e., on each subinterval rx i, x i`1 s, s is a linear function. The interpolation error is (from previous results with Newton s form) fpxq s 1 pf; xq px x i qpx x i`1 qrx i, x i`1, xsf for x P rx i, x i`1 s If f P C 2 ra, bs, fpxq s 1 pf; xq ď p x iq 2 8 max f 2, x P rx i, x i`1 s. rx i,x i`1 s Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 26 / 34

43 Piecewise Linear Functions; Interpolation Error Interpolation error: }fp q s 1 pf; q} 8 ď }f 2 } 8 Furthermore, with the best approximation of f from S 0 1, dist 8 pf, S 0 1q ď }fp q s 1 pf; q} 8 ď 2dist 8 pf, S 0 1q where dist 8 pf, Sq inf }fp q s} 8 sps is the best approximation to f from S. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 27 / 34

44 Basis for S 0 1p q Dimension of S 0 1p q: n. A basis: for i 1,..., n, (denote x 0 x 1 and x n`1 x n ) Clearly, And for any s in S 0 1p q, $ x x i 1 if x i 1 ď x ď x i, & x i x i 1 B i pxq x i`1 x if x i ď x ď x i`1, x i`1 x i % 0 otherwise. B i px j q δ ij spxq # 1, if i j, 0, if i j. nÿ spx i qb i pxq. i 1 Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 28 / 34

45 Fig The functions Bi

46 Least Squares Approximation over S 0 1p q Least Squares Approximation Given f P Cra, bs, find ŝ 1 pf; q P S 0 1p q such that }f ŝ 1 } 2 min }f s} 2. sps 0 1 p q The unique solution by Normal equations Ac b (See Later) with A ra ij s rpb i, B j qs, b rb i s rpf, B i qs, denoted as ĉ A 1 b or ŝ 1 pf; xq ř n i 1 ĉib i pxq. Clearly, pb i, B j q 0 if i j ą 1, so A is tridiagonal, i.e., 1 6 x i 1ĉ i 1 ` 1 3 p x i 1 ` x i qĉ i ` 1 6 x iĉ i`1 b i, i 1, 2,..., n. The least squares error: dist 8 pf, S 0 1q ď }fp q ŝ 1 pf; q} 8 ď 4dist 8 pf, S 0 1q. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 29 / 34

47 Interpolation by Cubic Splines Cubic Splines S 1 3p q Given nodes x 1,..., x n, and numbers m 1,..., m n, find s 3 pf; q P S 1 3p q with s 3 pf; q rxi,x i`1 s p i pxq, i 1, 2,..., n 1, such that s 1 3 pf; x iq m i, i 1,..., n. We selecting each piece p i to be the solution of a Hermite interpolation problem: for i 1, 2,..., n 1, p i px i q f i, p i px i`1 q f i`1, p 1 ipx i q m i, p 1 ipx i`1 q m i`1. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 30 / 34

48 x i f i x i f i m i x ic1 f ic1 Œx i ;x ic1 f Œx i ;x ic1 f m i x i m ic1 Œx i ;x ic1 f x ic1 f ic1 m ic1 m ic1 C m i 2Œx i ;x ic1 f. x 2 i /

49 Cubic Splines S 1 3p q Cubic Splines S 1 3p q Each piece p i is given by in Newton s form p i pxq f i ` px x i qm i ` px x i q 2 rx i, x i`1 sf m i x i ` px x i q 2 px x i`1 q m i`1 ` m i 2rx i, x i`1 sf p x i q 2 in Taylor s form p i pxq c i,0 ` c i,1 px x i q ` c i,2 px x i q 2 ` c i,3 px x i q 3, with c i,0 f i ; c i,1 m i ; c i,2 rx i, x i`1 sf m i x i c i,3 x i ; c i,3 m i`1 ` m i 2rx i, x i1 sf p x i q 2 Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 31 / 34

50 Possible Choices of tm i u Piecewise cubic Hermite interpolation: m i f 1 px i q. Cubic spline interpolation: s 3 pf; q P S 2 3p q, i.e., p 2 i 1px i q p 2 i px i q, i 2, 3,..., n 1. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 32 / 34

51 (a) Piecewise cubic Hermite interpolation. Here one selects m i D f 0.x i /, assuming that these derivative values are known. This gives rise to a strictly local scheme, in that each piece p i can be determined independently from the others. Furthermore, the error of interpolation is easily estimated, since from the theory of interpolation, f.x/ p i.x/ D.x x i / 2.x x ic1 / 2 Œx i ;x i ;x ic1 ;x ic1 ;x f; x i x x ic1 I hence, if f 2 C 4 Œa; b, 1 4 jf.x/ p i.x/j xi max 2 There follows: Œx i ;x ic1 jf.4/ j ; x i x x ic1 : 4Š 1 kf. / s 3.f I/k 1 j j 4 kf.4/ k : (2.142) In the case of equally spaced points x i, one has j j D.b a/=.n 1/ and, therefore, kf. / s 3.f I/k 1 D O.n 4 / as n!1: (2.143)

52 This is quite satisfactory, but note that the derivative of f must be known at each point x i, and the interpolant is only in C 1 Œa; b. As to the derivative values, one could approximate them by the derivatives of p 2.f I x i 1 ;x i ;x ic1 I x/ at x D x i, which requires only function values of f, except at the endpoints, where again the derivatives of f are involved, the points a D x 0 D x 1 and b D x n D x nc1 being double points (cf. Ex. 78). It can be shown that this degrades the accuracy to O. j j 3 /. (b) Cubic spline interpolation. Here we require s3.f I/ 2 S 2 3. /, that is, continuity of the second derivative. In terms of the pieces ( )o fs3.f I/, this means that p i 00 1.x i / D p i 00.x i /; i D 2;3;:::;n 1; (2.144) and translates into a condition for the Taylor coefficients in (2.140), namely, 2c i 1;2 C 6c i 1;3 x i 1 D 2c i;2 ; i D 2;3;:::;n 1: Plugging in the explicit values (2.141) for these coefficients, we arrive at the linear system

53 . x i /m i 1 C 2. x i 1 C x i /m i C. x i 1 /m ic1 D b i ; i D 2;3;:::;n 1; (2.145) where b i D 3f. x i /Œx i 1 ;x i f C. x i 1 /Œx i ;x ic1 f g: (2.146) These are n 2 linear equations in the n unknowns m 1, m 2 ;:::;m n.oncem 1 and m n have been chosen in some way, the system again becomes tridiagonal in the remaining unknowns and hence is readily solved by Gauss elimination. Here are some possible choices of m 1 and m n. (b.1) Complete splines: m 1 D f 0.a/, m n D f 0.b/. It is known that for this spline kf.r/. / s.r/.f I/k 1 c r j j 4 r kf.4/ k 1 ;r D 0; 1; 2; 3; if f 2 C 4 Œa; b ; (2.147) where c 0 D 5 384, c 1 D 1 24, c 2 D 3 8,andc 3 is a constant depending on the mesh j j ratio min i x i. Rather remarkably, the bound for r D 0 is only five times larger than the bound (2.142) for the piecewise cubic Hermite interpolant, which requires derivative values of f at all interpolation nodes x i, not just at the endpoints a and b.

54 (b.2) Matching of the second derivatives at the endpoints: s3 00 f I a/ D f 00.a/, s3 00.f I b/ D f 00.b/. Each of these conditions gives ris ė to an additional equation, namely, 2m 1 C m 2 D 3Œx 1 ;x 2 f 1 2 f 00.a/ x 1 ; m n 1 C 2m n D 3Œx n 1 ;x n f C 2 1 f 00.b/ x n 1 : (2.148) One conveniently adjoins the first equation to the top of the system (2.145), and the second to the bottom, thereby preserving the tridiagonal structure of the system.

55 (b.3) Natural cubic spline: s 00.f I a/ D s 00.f I b/ D 0. This again produces two additional equations, which can be obtained from (2.148) by putting there f 00.a/ D f 00.b/ D 0. They are adjoined to the system (2.145) as described in (b.2). The nice thing about this spline is that it requires only function values of f no derivatives! but the price one pays is a degradation of the accuracy to O.j j 2 / near the endpoints (unless indeed f 00.a/ D f 00.b/ D 0). (b.4) Not-a-knot spline (C. de Boor): here we require p1.x/ p2.x/ and p n 2.x/ p n 1.x/; that is, the first two pieces of the spline should be the same polynomial, and similarly for the last two pieces. In effect, this means that the first interior knot x 2, and the last one x n 1, both are inactive (hence the name). This again gives rise to two supplementary equations expressing continuity of s3 000.f I x/ at x D x 2 and x D x n 1 (cf. Ex. 79).

56 Minimality Properties of Cubic Splines Subdivision : Subdivision 1 : : a x 1 ă x 2 ă ă x n 1 ă x n b 1 : a x 0 x 1 ă x 2 ă ă x n 1 ă x n x n`1 b This means that whenever we interpolate on 1, we interpolate not only to function values at all interior points but also to the function as well as first derivative values at the endpoints. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 33 / 34

57 Minimality Properties of Cubic Splines Theorem (Complete Cubic Spline Interpolant) For any function g P C 2 ra, bs that interpolates f on 1, there holds ż b a rg 2 pxqs 2 dx ě ż b a rs 2 compl pf; xqs2 dx with equality iff gp q s compl pf; q. Theorem (Natural Cubic Spline Interpolant) For any function g P C 2 ra, bs that interpolates f on (not 1 ), there holds ż b a rg 2 pxqs 2 dx ě with equality iff gp q s nat pf; q. ż b a rs 2 natpf; xqs 2 dx Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH561 MATH 561 Numerical Analys 34 / 34

58 Cubic Spline Interpolation (see Burden & Faries) Given a function f defined on [a, b] and a set of nodes a = x 0 < x 1 < < x n = b, acubic spline interpolant S for f is a function that satisfies the following conditions: (a) S(x) is a cubic polynomial, denoted S j (x), on the subinterval [x j, x j+1 ] for each j = 0, 1,..., n 1; (b) S j (x j ) = f(x j ) and S j (x j+1 ) = f(x j+1 ) for each j = 0, 1,..., n 1; (c) S j+1 (x j+1 ) = S j (x j+1 ) for each j = 0, 1,..., n 2; (Implied by (b).) (d) S j +1 (x j+1 ) = S j (x j+1 ) for each j = 0, 1,..., n 2; (e) S j +1 (x j+1 ) = S j (x j+1 ) for each j = 0, 1,..., n 2; (f) One of the following sets of boundary conditions is satisfied: (i) S (x 0 ) = S (x n ) = 0 (natural (or free) boundary); (ii) S (x 0 ) = f (x 0 ) and S (x n ) = f (x n ) (clamped boundary).

59 Construct a natural cubic spline that passes through the points (1, 2), (2, 3), and (3, 5). Solution This spline consists of two cubics. The first for the interval [1, 2], denoted and the other for [2, 3], denoted S 0 (x) = a 0 + b 0 (x 1) + c 0 (x 1) 2 + d 0 (x 1) 3, S 1 (x) = a 1 + b 1 (x 2) + c 1 (x 2) 2 + d 1 (x 2) 3. There are 8 constants to be determined, which requires 8 conditions. Four conditions come from the fact that the splines must agree with the data at the nodes. Hence 2 = f(1) = a 0, 3 = f(2) = a 0 + b 0 + c 0 + d 0, 3 = f(2) = a 1, and 5 = f(3) = a 1 + b 1 + c 1 + d 1.

60 Two more come from the fact that S 0 (2) = S 1 (2) and S 0 (2) = S 1 (2). These are S 0 (2) = S 1 (2) : b 0 + 2c 0 + 3d 0 = b 1 and S 0 (2) = S 1 (2) : 2c 0 + 6d 0 = 2c 1 The final two come from the natural boundary conditions: S 0 (1) = 0: 2c 0 = 0 and S 1 (3) = 0: 2c 1 + 6d 1 = 0. Solving this system of equations gives the spline S(x) = { (x 1) + 1 (x 1) 3, for x [1, 2] (x 2) + 3 (x 2) (x 2) 3, for x [2, 3]