Introduction to Numericl Anlysis Doron Levy Deprtment of Mthemtics nd Center for Scientific Computtion nd Mthemticl Modeling (CSCAMM) University of Mrylnd June 14, 2012
D. Levy CONTENTS Contents 1 Introduction 1 2 Methods for Solving Nonliner Problems 2 2.1 Preliminry Discussion............................ 2 2.1.1 Are there ny roots nywhere?.................... 3 2.1.2 Exmples of root-finding methods.................. 5 2.2 Itertive Methods............................... 6 2.3 The Bisection Method............................ 8 2.4 Newton s Method............................... 11 2.5 The Secnt Method.............................. 15 3 Interpoltion 19 3.1 Wht is Interpoltion?............................ 19 3.2 The Interpoltion Problem.......................... 20 3.3 Newton s Form of the Interpoltion Polynomil.............. 22 3.4 The Interpoltion Problem nd the Vndermonde Determinnt...... 23 3.5 The Lgrnge Form of the Interpoltion Polynomil............ 25 3.6 Divided Differences.............................. 28 3.7 The Error in Polynomil Interpoltion................... 31 3.8 Interpoltion t the Chebyshev Points................... 34 3.9 Hermite Interpoltion............................. 41 3.9.1 Divided differences with repetitions................. 43 3.9.2 The Lgrnge form of the Hermite interpolnt........... 44 3.10 Spline Interpoltion.............................. 47 3.10.1 Cubic splines............................. 50 3.10.2 Wht is nturl bout the nturl spline?............. 54 4 Approximtions 57 4.1 Bckground.................................. 57 4.2 The Minimx Approximtion Problem................... 62 4.2.1 Existence of the minimx polynomil................ 63 4.2.2 Bounds on the minimx error.................... 65 4.2.3 Chrcteriztion of the minimx polynomil............ 66 4.2.4 Uniqueness of the minimx polynomil............... 66 4.2.5 The ner-minimx polynomil.................... 67 4.2.6 Construction of the minimx polynomil.............. 68 4.3 Lest-squres Approximtions........................ 70 4.3.1 The lest-squres pproximtion problem.............. 70 4.3.2 Solving the lest-squres problem: direct method........ 70 4.3.3 Solving the lest-squres problem: with orthogonl polynomils. 72 4.3.4 The weighted lest squres problem................. 74 3
CONTENTS D. Levy 4.3.5 Orthogonl polynomils....................... 75 4.4 Exmples of orthogonl polynomils.................... 77 4.4.1 Another pproch to the lest-squres problem.......... 79 4.4.2 Properties of orthogonl polynomils................ 85 5 Numericl Differentition 88 5.1 Bsic Concepts................................ 88 5.2 Differentition Vi Interpoltion....................... 90 5.3 The Method of Undetermined Coefficients................. 93 5.4 Richrdson s Extrpoltion.......................... 95 6 Numericl Integrtion 98 6.1 Bsic Concepts................................ 98 6.2 Integrtion vi Interpoltion......................... 102 6.3 Composite Integrtion Rules......................... 105 6.4 Additionl Integrtion Techniques...................... 107 6.4.1 The method of undetermined coefficients.............. 107 6.4.2 Chnge of n intervl......................... 108 6.5 Simpson s Integrtion............................. 109 6.5.1 The qudrture error......................... 109 6.5.2 Composite Simpson rule....................... 111 6.6 Weighted Qudrtures............................ 112 6.7 Gussin Qudrture............................. 113 6.7.1 Mximizing the qudrture s ccurcy............... 113 6.7.2 Convergence nd error nlysis................... 117 6.8 Romberg Integrtion............................. 120 Bibliogrphy 122 index 123 4
D. Levy 1 Introduction 1
D. Levy 2 Methods for Solving Nonliner Problems 2.1 Preliminry Discussion In this chpter we will lern methods for pproximting solutions of nonliner lgebric equtions. We will limit our ttention to the cse of finding roots of single eqution of one vrible. Thus, given function, f(x), we will be be interested in finding points x, for which f(x ) = 0. A clssicl exmple tht we re ll fmilir with is the cse in which f(x) is qudrtic eqution. If, f(x) = x 2 + bx + c, it is well known tht the roots of f(x) re given by x 1,2 = b ± b 2 4c. 2 These roots my be complex or repet (if the discriminnt vnishes). This is simple cse in which the cn be computed using closed nlytic formul. There exist formuls for finding roots of polynomils of degree 3 nd 4, but these re rther complex. In more generl cses, when f(x) is polynomil of degree tht is 5, formuls for the roots no longer exist. Of course, there is no reson to limit ourselves to study polynomils, nd in most cses, when f(x) is n rbitrry function, there re no nlytic tools for clculting the desired roots. Insted, we must use pproximtion methods. In fct, even in cses in which exct formuls re vilble (such s with polynomils of degree 3 or 4) n exct formul might be too complex to be used in prctice, nd pproximtion methods my quickly provide n ccurte solution. An eqution f(x) = 0 my or my not hve solutions. We re not going to focus on finding methods to decide whether n eqution hs solutions or not, but we will look for pproximtion methods ssuming tht solutions ctully exist. We will lso ssume tht we re looking only for rel roots. There re extensions of some of the methods tht we will describe to the cse of complex roots but we will not del with this cse. Even with the simple exmple of the qudrtic eqution, it is cler tht nonliner eqution f(x) = 0 my hve more thn one root. We will not develop ny generl methods for clculting the number of the roots. This issue will hve to be delt with on cse by cse bsis. We will lso not del with generl methods for finding ll the solutions of given eqution. Rther, we will focus on pproximting one of the solutions. The methods tht we will describe, ll belong to the ctegory of itertive methods. Such methods will typiclly strt with n initil guess of the root (or of the neighborhood of the root) nd will grdully ttempt to pproch the root. In some cses, the sequence of itertions will converge to limit, in which cse we will then sk if the limit point is ctully solution of the eqution. If this is indeed the cse, nother question of interest is how fst does the method converge to the solution? To be more precise, this question cn be formulted in the following wy: how mny itertions of the method re required to gurntee certin ccurcy in the pproximtion of the solution of the eqution. 2
D. Levy 2.1 Preliminry Discussion 2.1.1 Are there ny roots nywhere? There relly re not tht mny generl tools to knowing up front whether the rootfinding problem cn be solved. For our purposes, there most importnt issue will be to obtin some informtion bout whether root exists or not, nd if root does exist, then it will be importnt to mke n ttempt to estimte n intervl to which such solution belongs. One of our first ttempts in solving such problem my be to try to plot the function. After ll, if the gol is to solve f(x) = 0, nd the function f(x) cn be plotted in wy tht the intersection of f(x) with the x-xis is visible, then we should hve rther good ide s of where to look for for the root. There is bsolutely nothing wrong with such method, but it is not lwys esy to plot the function. There re mny cses, in which it is rther esy to miss the root, nd the sitution lwys gets worse when moving to higher dimensions (i.e., more equtions tht should simultneously be solved). Insted, something tht is sometimes esier, is to verify tht the function f(x) is continuous (which hopefully it is) in which cse ll tht we need is to find point in which f() > 0, nd point b, in which f(b) < 0. The continuity will then gurntee (due to the intermedite vlue theorem) tht there exists point c between nd b for which f(c) = 0, nd the hunt for tht point cn then begin. How to find such points nd b? Agin, there relly is no generl recipe. A combintion of intuition, common sense, grphics, thinking, nd tril-nd-error is typiclly helpful. We would now like to consider severl exmples: Exmple 2.1 A stndrd wy of ttempting to determine if continuous function hs root in n intervl is to try to find point in which it is positive, nd second point in which it is negtive. The intermedite vlue theorem for continuous functions then gurntees the existence of t lest one point for which the function vnishes. To demonstrte this method, consider f(x) = sin(x) x + 0.5. At x = 0, f(0) = 0.5 > 0, while t x = 5, clerly f(x) must be negtive. Hence the intermedite vlue theorem gurntees the existence of t lest one point x (0, 5) for which f(x ) = 0. Exmple 2.2 Consider the problem e x = x, for which we re being sked to determine if solution exists. One possible wy to pproch this problem is to define function f(x) = e x x, rewrite the problem s f(x) = 0, nd plot f(x). This is not so bd, but lredy requires grphic clcultor or clculus-like nlysis of the function f(x) in order to plot it. Insted, it is resonble ide to strt with the originl problem, nd plot both functions e x nd x. Clerly, these functions intersect ech other, nd the intersection is the desirble root. Now, we cn return to f(x) nd use its continuity (s difference between continuous functions) to check its sign t couple of points. For exmple, t x = 0, we hve tht f(0) = 1 > 0, while t x = 1, f(1) = 1/e 1 < 0. Hence, due to the intermedite vlue theorem, there must exist point x in the intervl (0, 1) for which f(x ) = 0. At tht point x we hve e x = x. Note tht while the grphicl 3
2.1 Preliminry Discussion D. Levy rgument clerly indictes tht there exists one nd only one solution for the eqution, the rgument tht is bsed on the intermedite vlue theorem provides the existence of t lest one solution. A tool tht is relted to the intermedite vlue theorem is Brouwer s fixed point theorem: Theorem 2.3 (Brouwer s Fixed Point Theorem) Assume tht g(x) is continuous on the closed intervl [, b]. Assume tht the intervl [, b] is mpped to itself by g(x), i.e., for ny x [, b], g(x) [, b]. Then there exists point c [, b] such tht g(c) = c. The point c is fixed point of g(x). The theorem is demonstrted in Figure 2.1. Since the intervl [, b] is mpped to itself, the continuity of g(x) implies tht it must intersect the line x in the intervl [, b] t lest once. Such intersection points re the desirble fixed points of the function g(x), s gurnteed by Theorem 2.3. Figure 2.1: An illustrtion of the Brouwer fixed point theorem Proof. Let f(x) = x g(x). Since g() [, b] nd lso g(b) [, b], we know tht f() = g() 0 while f(b) = b g(b) 0. Since g(x) is continuous in [, b], so is f(x), nd hence ccording to the intermedite vlue theorem, there must exist point c [, b] t which f(c) = 0. At this point g(c) = c. How much does Theorem 2.3 dd in terms of tools for proving tht root exists in certin intervl? In prctice, the ctul contribution is rther mrginl, but there re cses where it dds something. Clerly if we re looking for roots of function f(x), we cn lwys reformulte the problem s fixed point problem for function 4
D. Levy 2.1 Preliminry Discussion g(x) by defining g(x) = f(x) + x. Usully this is not the only wy in which root finding problem cn be converted into fixed point problem. In order to be ble to use Theorem 2.3, the key point is lwys to look for fixed point problem in which the intervl of interest is mpped to itself. Exmple 2.4 To demonstrte how the fixed point theorem cn be used, consider the function f(x) = e x x 2 3 for x [1, 2]. Define g(x) = ln(x 2 + 3). Fixed points of g(x) is root of f(x). Clerly, g(1) = ln 4 > ln e = 1 nd g(2) = ln(7) < ln(e 2 ) = 2, nd since g(x) is continuous nd monotone in [1, 2], we hve tht g([1, 2]) [1, 2]. Hence the conditions of Theorem 2.3 re stisfied nd f(x) must hve root in the intervl [1, 2]. 2.1.2 Exmples of root-finding methods So fr our focus hs been on ttempting to figure out if given function hs ny roots, nd if it does hve roots, pproximtely where cn they be. However, we hve not went into ny detils in developing methods for pproximting the vlues of such roots. Before we strt with detiled study of such methods, we would like to go over couple of the methods tht will be studied lter on, emphsizing tht they re ll itertive methods. The methods tht we will briefly describe re Newton s method nd the secnt method. A more detiled study of these methods will be conducted in the following sections. 1. Newton s method. Newton s method for finding root of differentible function f(x) is given by: x n+1 = x n f(x n) f (x n ). (2.1) We note tht for the formul (2.1) to be well-defined, we must require tht f (x n ) 0 for ny x n. To provide us with list of successive pproximtion, Newton s method (2.1) should be supplemented with one initil guess, sy x 0. The eqution (2.1) will then provide the vlues of x 1, x 2,... One wy of obtining Newton s method is the following: Given point x n we re looking for the next point x n+1. A liner pproximtion of f(x) t x n+1 is f(x n+1 ) f(x n ) + (x n+1 x n )f (x n ). Since x n+1 should be n pproximtion to the root of f(x), we set f(x n+1 ) = 0, rerrnge the terms nd get (2.1). 2. The secnt method. The secnt method is obtined by replcing the derivtive in Newton s method, f (x n ), by the following finite difference pproximtion: f (x n ) f(x n) f(x n 1 ) x n x n 1. (2.2) 5
2.2 Itertive Methods D. Levy The secnt method is thus: [ x n+1 x n f(x n ) x n x n 1 f(x n ) f(x n 1 ) ]. (2.3) The secnt method (2.3) should be supplemented by two initil vlues, sy x 0, nd x 1. Using these two vlues, (2.3) will provide the vlues of x 2, x 3,.... 2.2 Itertive Methods At this point we would like to explore more tools for studying itertive methods. We strt by considering simple itertes, in which given n initil vlue x 0, the itertes re given by the following recursion: x n+1 = g(x n ), n = 0, 1,... (2.4) If the sequence {x n } in (2.4) converges, nd if the function g(x) is continuous, the limit must be fixed point of the function g(x). This is obvious, since if x n x s n, then the continuity of g(x) implies tht in the limit we hve x = g(x ). Since things seem to work well when the sequence {x n } converges, we re now interested in studying exctly how cn the convergence of this sequence be gurnteed? Intuitively, we expect tht convergence of the sequence will occur if the function g(x) is shrinking the distnce between ny two points in given intervl. Formlly, such concept is known s contrction nd is given by the following definition: Definition 2.5 Assume tht g(x) is continuous function in [, b]. Then g(x) is contrction on [, b] if there exists constnt L such tht 0 < L < 1 for which for ny x nd y in [, b]: g(x) g(y) L x y. (2.5) The eqution (2.5) is referred to s Lipschitz condition nd the constnt L is the Lipschitz constnt. Indeed, if the function g(x) is contrction, i.e., if it stisfies the Lipschitz condition (2.5), we cn expect the itertes (2.4) to converge s given by the Contrction Mpping Theorem. Theorem 2.6 (Contrction Mpping) Assume tht g(x) is continuous function on [, b]. Assume tht g(x) stisfies the Lipschitz condition (2.5), nd tht g([, b]) [, b]. Then g(x) hs unique fixed point c [, b]. Also, the sequence {x n } defined in (2.4) converges to c s n for ny x 0 [, b]. 6
D. Levy 2.2 Itertive Methods Proof. We know tht the function g(x) must hve t lest one fixed point due to Theorem 2.3. To prove the uniqueness of the fixed point, we ssume tht there re two fixed points c 1 nd c 2. We will prove tht these two points must be identicl. c 1 c 2 = g(c 1 ) g(c 2 ) L c 1 c 2, nd since 0 < L < 1, c 1 must be equl to c 2. Finlly, we prove tht the itertes in (2.4) converge to c for ny x 0 [, b]. x n+1 c = g(x n ) g(c) L x n c... L n+1 x 0 c. (2.6) Since 0 < L < 1, we hve tht s n, x n+1 c 0, nd we hve convergence of the itertes to the fixed point of g(x) independently of the strting point x 0. Remrks. 1. In order to use the Contrction Mpping Theorem, we must verify tht the function g(x) stisfies the Lipschitz condition, but wht does it men? The Lipschitz condition provides informtion bout the slope of the function. The quottion mrks re being used here, becuse we never required tht the function g(x) is differentible. Our only requirement hd to do with the continuity of g(x). The Lipschitz condition cn be rewritten s: g(x) g(y) x y L, x, y [, b], x y, with 0 < L < 1. The term on the LHS is discrete pproximtion to the slope of g(x). In fct, if the function g(x) is differentible, ccording to the Men Vlue Theorem, there exists point ξ between x nd y such tht g (ξ) = g(x) g(y). x y Hence, in prctice, if the function g(x) is differentible in the intervl (, b), nd if there exists L (0, 1), such tht g (x) < L for ny x (, b), then the ssumptions on g(x) stisfying the Lipshitz condition in Theorem 2.6 hold. Hving g(x) differentible is more thn the theorem requires but in mny prcticl cses, we nyhow del with differentible g s so it is strightforwrd to use the condition tht involves the derivtive. 2. Another typicl thing tht cn hppen is tht the function g(x) will be differentible, nd g (x) will be less thn 1, but only in neighborhood of the fixed point. In this cse, we cn still formulte locl version of the contrction mpping theorem. This theorem will gurntee convergence to fixed point, c, of g(x) if we strt the itertions sufficiently close to tht point c. 7
2.3 The Bisection Method D. Levy Strting fr from c my or my not led to convergence to c. Also, since we consider only neighborhood of the fixed point c, we cn no longer gurntee the uniqueness of the fixed point, s wy from there, we do not post ny restriction on the slope of g(x) nd therefore nything cn hppen. 3. When the contrction mpping theorem holds, nd convergence of the itertes to the unique fixed point follows, it is of interest to know how mny itertions re required in order to pproximte the fixed point with given ccurcy. If our gol is to pproximte c within distnce ε, then this mens tht we re looking for n such tht x n c ε. We know from (2.6) tht x n c L n x 0 c, n 1. (2.7) In order to get rid of c from the RHS of (2.7), we compute Hence x 0 c = x c x 1 + x 1 c x 0 x 1 + x 1 c L x 0 c + x 1 x 0. x 0 c x 1 x 0 1 L. We thus hve x n c Ln 1 L x 1 x 0, nd for x n c < ε we require tht L n ε(1 L) x 1 x 0, which implies tht the number of itertions tht will gurntee tht the pproximtion error will be under ε must exceed n 1 [ ] (1 L)ε ln(l) ln. (2.8) x 1 x 0 2.3 The Bisection Method Before returning to Newton s method, we would like to present nd study method for finding roots which is one of the most intuitive methods one cn esily come up with. The method we will consider is known s the bisection method. 8
D. Levy 2.3 The Bisection Method We re looking for root of function f(x) which we ssume is continuous on the intervl [, b]. We lso ssume tht it hs opposite signs t both edges of the intervl, i.e., f()f(b) < 0. We then know tht f(x) hs t lest one zero in [, b]. Of course f(x) my hve more thn one zero in the intervl. The bisection method is only going to converge to one of the zeros of f(x). There will lso be no indiction s of how mny zeros f(x) hs in the intervl, nd no hints regrding where cn we ctully hope to find more roots, if indeed there re dditionl roots. The first step is to divide the intervl into two equl subintervls, c = + b 2. This genertes two subintervls, [, c] nd [c, b], of equl lengths. We wnt to keep the subintervl tht is gurnteed to contin root. Of course, in the rre event where f(c) = 0 we re done. Otherwise, we check if f()f(c) < 0. If yes, we keep the left subintervl [, c]. If f()f(c) > 0, we keep the right subintervl [c, b]. This procedure repets until the stopping criterion is stisfied: we fix smll prmeter ε > 0 nd stop when f(c) < ε. To simplify the nottion, we denote the successive intervls by [ 0, b 0 ], [ 1, b 1 ],... The first two itertions in the bisection method re shown in Figure 2.2. Note tht in the cse tht is shown in the figure, the function f(x) hs multiple roots but the method converges to only one of them. 0 f( 0 ) f(c) f(b 0 ) 0 c b 0 x f(b 1 ) 0 f( 1 ) f(c) 1 c b 1 x Figure 2.2: The first two itertions in bisection root-finding method We would now like to understnd if the bisection method lwys converges to root. We would lso like to figure out how close we re to root fter iterting the lgorithm 9
2.3 The Bisection Method D. Levy severl times. We first note tht nd 0 1 2... b 0, b 0 b 1 b 2... 0. We lso know tht every itertion shrinks the length of the intervl by hlf, i.e., b n+1 n+1 = 1 2 (b n n ), n 0, which mens tht b n n = 2 n (b 0 0 ). The sequences { n } n 0 nd {b n } n 0 re monotone nd bounded, nd hence converge. Also lim b n lim n = lim 2 n (b 0 0 ) = 0, n n n so tht both sequences converge to the sme vlue. We denote tht vlue by r, i.e., r = lim n n = lim n b n. Since f( n )f(b n ) 0, we know tht (f(r)) 2 0, which mens tht f(r) = 0, i.e., r is root of f(x). We now ssume tht we stop in the intervl [ n, b n ]. This mens tht r [ n, b n ]. Given such n intervl, if we hve to guess where is the root (which we know is in the intervl), it is esy to see tht the best estimte for the loction of the root is the center of the intervl, i.e., c n = n + b n. 2 In this cse, we hve r c n 1 2 (b n n ) = 2 (n+1) (b 0 0 ). We summrize this result with the following theorem. Theorem 2.7 If [ n, b n ] is the intervl tht is obtined in the n th itertion of the bisection method, then the limits lim n n nd lim n b n exist, nd lim n = lim b n = r, n n where f(r) = 0. In ddition, if then c n = n + b n, 2 r c n 2 (n+1) (b 0 0 ). 10
D. Levy 2.4 Newton s Method 2.4 Newton s Method Newton s method is reltively simple, prcticl, nd widely-used root finding method. It is esy to see tht while in some cses the method rpidly converges to root of the function, in some other cses it my fil to converge t ll. This is one reson s of why it is so importnt not only to understnd the construction of the method, but lso to understnd its limittions. As lwys, we ssume tht f(x) hs t lest one (rel) root, nd denote it by r. We strt with n initil guess for the loction of the root, sy x 0. We then let l(x) be the tngent line to f(x) t x 0, i.e., l(x) f(x 0 ) = f (x 0 )(x x 0 ). The intersection of l(x) with the x-xis serves s the next estimte of the root. We denote this point by x 1 nd write 0 f(x 0 ) = f (x 0 )(x 1 x 0 ), which mens tht x 1 = x 0 f(x 0) f (x 0 ). (2.9) In generl, the Newton method (lso known s the Newton-Rphson method) for finding root is given by iterting (2.9) repetedly, i.e., x n+1 = x n f(x n) f (x n ). (2.10) Two smple itertions of the method re shown in Figure 2.3. Strting from point x n, we find the next pproximtion of the root x n+1, from which we find x n+2 nd so on. In this cse, we do converge to the root of f(x). It is esy to see tht Newton s method does not lwys converge. We demonstrte such cse in Figure 2.4. Here we consider the function f(x) = tn 1 (x) nd show wht hppens if we strt with point which is fixed point of Newton s method, iterted twice. In this cse, x 0 1.3917 is such point. In order to nlyze the error in Newton s method we let the error in the n th itertion be e n = x n r. We ssume tht f (x) is continuous nd tht f (r) 0, i.e., tht r is simple root of f(x). We will show tht the method hs qudrtic convergence rte, i.e., e n+1 ce 2 n. (2.11) 11
2.4 Newton s Method D. Levy f(x) 0 r x n+2 x n+1 x n x Figure 2.3: Two itertions in Newton s root-finding method. r is the root of f(x) we pproch by strting from x n, computing x n+1, then x n+2, etc. tn 1 (x) 0 x 1, x 3, x 5,... x 0, x 2, x 4,... x Figure 2.4: Newton s method does not lwys converge. In this cse, the strting point is fixed point of Newton s method iterted twice 12
D. Levy 2.4 Newton s Method A convergence rte estimte of the type (2.11) mkes sense, of course, only if the method converges. Indeed, we will prove the convergence of the method for certin functions f(x), but before we get to the convergence issue, let s derive the estimte (2.11). We rewrite e n+1 s e n+1 = x n+1 r = x n f(x n) f (x n ) r = e n f(x n) f (x n ) = e nf (x n ) f(x n ). f (x n ) Writing Tylor expnsion of f(r) bout x = x n we hve 0 = f(r) = f(x n e n ) = f(x n ) e n f (x n ) + 1 2 e2 nf (ξ n ), which mens tht e n f (x n ) f(x n ) = 1 2 f (ξ n )e 2 n. Hence, the reltion (2.11), e n+1 ce 2 n, holds with c = 1 f (ξ n ) 2 f (x n ) (2.12) Since we ssume tht the method converges, in the limit s n we cn replce (2.12) by c = 1 f (r) 2 f (r). (2.13) We now return to the issue of convergence nd prove tht for certin functions Newton s method converges regrdless of the strting point. Theorem 2.8 Assume tht f(x) hs two continuous derivtives, is monotoniclly incresing, convex, nd hs zero. Then the zero is unique nd Newton s method will converge to it from every strting point. Proof. The ssumptions on the function f(x) imply tht x, f (x) > 0 nd f (x) > 0. By (2.12), the error t the (n + 1) th itertion, e n+1, is given by e n+1 = 1 f (ξ n ) 2 f (x n ) e2 n, nd hence it is positive, i.e., e n+1 > 0. This implies tht n 1, x n > r, Since f (x) > 0, we hve f(x n ) > f(r) = 0. 13
2.4 Newton s Method D. Levy Now, subtrcting r from both sides of (2.10) we my write e n+1 = e n f(x n) f (x n ), (2.14) which mens tht e n+1 < e n (nd hence x n+1 < x n ). Hence, both {e n } n 0 nd {x n } n 0 re decresing nd bounded from below. This mens tht both series converge, i.e., there exists e such tht, e = lim n e n, nd there exists x such tht x = lim n x n. By (2.14) we hve e = e f(x ) f (x ), so tht f(x ) = 0, nd hence x = r. Theorem 2.8 gurntees globl convergence to the unique root of monotoniclly incresing, convex smooth function. If we relx some of the requirements on the function, Newton s method my still converge. The price tht we will hve to py is tht the convergence theorem will no longer be globl. Convergence to root will hppen only if we strt sufficiently close to it. Such result is formulted in the following theorem. Theorem 2.9 Assume f(x) is continuous function with continuous second derivtive, tht is defined on n intervl I = [r δ, r + δ], with δ > 0. Assume tht f(r) = 0, nd tht f (r) 0. Assume tht there exists constnt A such tht f (x) f (y) A, x, y I. If the initil guess x 0 is sufficiently close to the root r, i.e., if r x 0 min{δ, 1/A}, then the sequence {x n } defined in (2.10) converges qudrticlly to the root r. Proof. We ssume tht x n I. Since f(r) = 0, Tylor expnsion of f(x) t x = x n, evluted t x = r is: 0 = f(r) = f(x n ) + (r x n )f (x n ) + (r x n) 2 f (ξ n ), (2.15) 2 where ξ n is between r nd x n, nd hence ξ I. Eqution (2.15) implies tht r x n = 2f(x n) (r x n ) 2 f (ξ n ). 2f (x n ) 14
D. Levy 2.5 The Secnt Method Since x n+1 re the Newton itertes nd hence stisfy (2.10), we hve Hence r x n+1 = r x n + f(x n) f (x n ) = (r x n) 2 f (ξ n ). (2.16) 2f (x n ) r x n+1 (r x n) 2 2 A r x n 2... 2 (n 1) r x 0, which implies tht x n r s n. It remins to show tht the convergence rte of {x n } to r is qudrtic. Since ξ n is between the root r nd x n, it lso converges to r s n. The derivtives f nd f re continuous nd therefore we cn tke the limit of (2.16) s n nd write x n+1 r lim n x n r = 2 f (r) 2f (r), which implies the qudrtic convergence of {x n } to r. 2.5 The Secnt Method We recll tht Newton s root finding method is given by eqution (2.10), i.e., x n+1 = x n f(x n) f (x n ). We now ssume tht we do not know tht the function f(x) is differentible t x n, nd thus cn not use Newton s method s is. Insted, we cn replce the derivtive f (x n ) tht ppers in Newton s method by difference pproximtion. A prticulr choice of such n pproximtion, f (x n ) f(x n) f(x n 1 ) x n x n 1, leds to the secnt method which is given by [ x n+1 = x n f(x n ) x n x n 1 f(x n ) f(x n 1 ) ], n 1. (2.17) A geometric interprettion of the secnt method is shown in Figure 2.5. Given two points, (x n 1, f(x n 1 )) nd (x n, f(x n )), the line l(x) tht connects them stisfies l(x) f(x n ) = f(x n 1) f(x n ) x n 1 x n (x x n ). 15
2.5 The Secnt Method D. Levy f(x) 0 r x n+1 x n x n 1 x Figure 2.5: The Secnt root-finding method. The points x n 1 nd x n re used to obtin x n+1, which is the next pproximtion of the root r The next pproximtion of the root, x n+1, is defined s the intersection of l(x) nd the x-xis, i.e., 0 f(x n ) = f(x n 1) f(x n ) x n 1 x n (x n+1 x n ). (2.18) Rerrnging the terms in (2.18) we end up with the secnt method (2.17). We note tht the secnt method (2.17) requires two initil points. While this is n extr requirement compred with, e.g., Newton s method, we note tht in the secnt method there is no need to evlute ny derivtives. In ddition, if implemented properly, every stge requires only one new function evlution. We now proceed with n error nlysis for the secnt method. As usul, we denote the error t the n th itertion by e n = x n r. We clim tht the rte of convergence of the secnt method is superliner (mening, better thn liner but less thn qudrtic). More precisely, we will show tht it is given by e n+1 e n α, (2.19) with α = 1 + 5. 2 (2.20) 16
D. Levy 2.5 The Secnt Method We strt by rewriting e n+1 s e n+1 = x n+1 r = f(x n)x n 1 f(x n 1 )x n f(x n ) f(x n 1 ) r = f(x n)e n 1 f(x n 1 )e n. f(x n ) f(x n 1 ) Hence [ ] [ f(x n) x n x n 1 e e n+1 = e n e n n 1 f(x n ) f(x n 1 ) A Tylor expnsion of f(x n ) bout x = r reds nd hence f(x n 1) e n 1 x n x n 1 f(x n ) = f(r + e n ) = f(r) + e n f (r) + 1 2 e2 nf (r) + O(e 3 n), f(x n ) e n We thus hve f(x n ) e n Therefore, = f (r) + 1 2 e nf (r) + O(e 2 n). f(x n 1) e n 1 = 1 2 (e n e n 1 )f (r) + O(e 2 n 1) + O(e 2 n) = 1 2 (x n x n 1 )f (r) + O(e 2 n 1) + O(e 2 n). ]. (2.21) f(x n) e n f(x n 1) e n 1 x n x n 1 1 2 f (r), nd x n x n 1 f(x n ) f(x n 1 ) 1 f (r). The error expression (2.21) cn be now simplified to e n+1 1 f (r) 2 f (r) e ne n 1 = ce n e n 1. (2.22) Eqution (2.22) expresses the error t itertion n + 1 in terms of the errors t itertions n nd n 1. In order to turn this into reltion between the error t the (n + 1) th itertion nd the error t the n th itertion, we now ssume tht the order of convergence is α, i.e., e n+1 A e n α. (2.23) 17
2.5 The Secnt Method D. Levy Since (2.23) lso mens tht e n A e n 1 α, we hve A e n α C e n A 1 α en 1 α. This implies tht A 1+ 1 α C 1 e n 1 α+ 1 α. (2.24) The left-hnd-side of (2.24) is non-zero while the right-hnd-side of (2.24) tends to zero s n (ssuming, of course, tht the method converges). This is possible only if 1 α + 1 α = 0, which, in turn, mens tht α = 1 + 5. 2 The constnt A in (2.23) is thus given by [ ] A = C 1 1+ α 1 = C 1 f α = C α 1 α 1 (r) =. 2f (r) We summrize this result with the theorem: Theorem 2.10 Assume tht f (x) is continuous x in n intervl I. Assume tht f(r) = 0 nd tht f (r) 0. If x 0, x 1 re sufficiently close to the root r, then x n r. In this cse, the convergence is of order 1+ 5. 2 18
D. Levy 3 Interpoltion 3.1 Wht is Interpoltion? Imgine tht there is n unknown function f(x) for which someone supplies you with its (exct) vlues t (n + 1) distinct points x 0 < x 1 < < x n, i.e., f(x 0 ),..., f(x n ) re given. The interpoltion problem is to construct function Q(x) tht psses through these points, i.e., to find function Q(x) such tht the interpoltion requirements Q(x j ) = f(x j ), 0 j n, (3.1) re stisfied (see Figure 3.1). One esy wy of obtining such function, is to connect the given points with stright lines. While this is legitimte solution of the interpoltion problem, usully (though not lwys) we re interested in different kind of solution, e.g., smoother function. We therefore lwys specify certin clss of functions from which we would like to find one tht solves the interpoltion problem. For exmple, we my look for function Q(x) tht is polynomil, Q(x). Alterntively, the function Q(x) cn be trigonometric function or piecewise-smooth polynomil, nd so on. Q(x) f(x 2 ) f(x 1 ) f(x 0 ) f(x) x 0 x 1 x 2 Figure 3.1: The function f(x), the interpoltion points x 0, x 1, x 2, nd the interpolting polynomil Q(x) As simple exmple let s consider vlues of function tht re prescribed t two points: (x 0, f(x 0 )) nd (x 1, f(x 1 )). There re infinitely mny functions tht pss through these two points. However, if we limit ourselves to polynomils of degree less thn or equl to one, there is only one such function tht psses through these two points: the 19
3.2 The Interpoltion Problem D. Levy line tht connects them. A line, in generl, is polynomil of degree one, but if the two given vlues re equl, f(x 0 ) = f(x 1 ), the line tht connects them is the constnt Q 0 (x) f(x 0 ), which is polynomil of degree zero. This is why we sy tht there is unique polynomil of degree 1 tht connects these two points (nd not polynomil of degree 1 ). The points x 0,..., x n re clled the interpoltion points. The property of pssing through these points is referred to s interpolting the dt. The function tht interpoltes the dt is n interpolnt or n interpolting polynomil (or whtever function is being used). There re cses were the interpoltion problem hs no solution, e.g., if we look for liner polynomil tht interpoltes three points tht do not lie on stright line. When solution exists, it cn be unique ( liner polynomil nd two points), or the problem cn hve more thn one solution ( qudrtic polynomil nd two points). Wht we re going to study in this section is precisely how to distinguish between these cses. We re lso going to present different pproches to constructing the interpolnt. Other thn greeing t the interpoltion points, the interpolnt Q(x) nd the underlying function f(x) re generlly different. The interpoltion error is mesure on how different these two functions re. We will study wys of estimting the interpoltion error. We will lso discuss strtegies on how to minimize this error. It is importnt to note tht it is possible to formulte the interpoltion problem without referring to (or even ssuming the existence of) ny underlying function f(x). For exmple, you my hve list of interpoltion points x 0,..., x n, nd dt tht is experimentlly collected t these points, y 0, y 1,..., y n, which you would like to interpolte. The solution to this interpoltion problem is identicl to the one where the vlues re tken from n underlying function. 3.2 The Interpoltion Problem We begin our study with the problem of polynomil interpoltion: Given n + 1 distinct points x 0,..., x n, we seek polynomil Q n (x) of the lowest degree such tht the following interpoltion conditions re stisfied: Q n (x j ) = f(x j ), j = 0,..., n. (3.2) Note tht we do not ssume ny ordering between the points x 0,..., x n, s such n order will mke no difference. If we do not limit the degree of the interpoltion polynomil it is esy to see tht there ny infinitely mny polynomils tht interpolte the dt. However, limiting the degree of Q n (x) to be deg(q n (x)) n, singles out precisely one interpolnt tht will do the job. For exmple, if n = 1, there re infinitely mny polynomils tht interpolte (x 0, f(x 0 )) nd (x 1, f(x 1 )). However, there is only one polynomil Q n (x) with deg(q n (x)) 1 tht does the job. This result is formlly stted in the following theorem: 20
D. Levy 3.2 The Interpoltion Problem Theorem 3.1 If x 0,..., x n R re distinct, then for ny f(x 0 ),... f(x n ) there exists unique polynomil Q n (x) of degree n such tht the interpoltion conditions (3.2) re stisfied. Proof. We strt with the existence prt nd prove the result by induction. For n = 0, Q 0 = f(x 0 ). Suppose tht Q n 1 is polynomil of degree n 1, nd suppose lso tht Q n 1 (x j ) = f(x j ), 0 j n 1. Let us now construct from Q n 1 (x) new polynomil, Q n (x), in the following wy: Q n (x) = Q n 1 (x) + c(x x 0 )... (x x n 1 ). (3.3) The constnt c in (3.3) is yet to be determined. Clerly, the construction of Q n (x) implies tht deg(q n (x)) n. (Since we might end up with c = 0, Q n (x) could ctully be of degree tht is less thn n.) In ddition, the polynomil Q n (x) stisfies the interpoltion requirements Q n (x j ) = f(x j ) for 0 j n 1. All tht remins is to determine the constnt c in such wy tht the lst interpoltion condition, Q n (x n ) = f(x n ), is stisfied, i.e., Q n (x n ) = Q n 1 (x n ) + c(x n x 0 )... (x n x n 1 ). (3.4) The condition (3.4) implies tht c should be defined s c = f(x n) Q n 1 (x n ), (3.5) n 1 (x n x j ) nd we re done with the proof of existence. As for uniqueness, suppose tht there re two polynomils Q n (x), P n (x) of degree n tht stisfy the interpoltion conditions (3.2). Define polynomil H n (x) s the difference H n (x) = Q n (x) P n (x). The degree of H n (x) is t most n which mens tht it cn hve t most n zeros (unless it is identiclly zero). However, since both Q n (x) nd P n (x) stisfy ll the interpoltion requirements (3.2), we hve H n (x j ) = (Q n P n )(x j ) = 0, 0 j n, which mens tht H n (x) hs n + 1 distinct zeros. This contrdiction cn be resolved only if H n (x) is the zero polynomil, i.e., P n (x) Q n (x), nd uniqueness is estblished. 21
3.3 Newton s Form of the Interpoltion Polynomil D. Levy 3.3 Newton s Form of the Interpoltion Polynomil One good thing bout the proof of Theorem 3.1 is tht it is constructive. In other words, we cn use the proof to write down formul for the interpoltion polynomil. We follow the procedure given by (3.4) for reconstructing the interpoltion polynomil. We do it in the following wy: Let Q 0 (x) = 0, where 0 = f(x 0 ). Let Q 1 (x) = 0 + 1 (x x 0 ). Following (3.5) we hve 1 = f(x 1) Q 0 (x 1 ) x 1 x 0 = f(x 1) f(x 0 ) x 1 x 0. We note tht Q 1 (x) is nothing but the stright line connecting the two points (x 0, f(x 0 )) nd (x 1, f(x 1 )). In generl, let Q n (x) = 0 + 1 (x x 0 ) +... + n (x x 0 )... (x x n 1 ) (3.6) = j 1 0 + j (x x k ). j=1 k=0 The coefficients j in (3.6) re given by 0 = f(x 0 ), j = f(x j) Q j 1 (x j ) j 1 k=0 (x, 1 j n. j x k ) (3.7) We refer to the interpoltion polynomil when written in the form (3.6) (3.7) s the Newton form of the interpoltion polynomil. As we shll see below, there re vrious wys of writing the interpoltion polynomil. The uniqueness of the interpoltion polynomil s gurnteed by Theorem 3.1 implies tht we will only be rewriting the sme polynomil in different wys. Exmple 3.2 The Newton form of the polynomil tht interpoltes (x 0, f(x 0 )) nd (x 1, f(x 1 )) is Q 1 (x) = f(x 0 ) + f(x 1) f(x 0 ) x 1 x 0 (x x 0 ). 22
D. Levy 3.4 The Interpoltion Problem nd the Vndermonde Determinnt Exmple 3.3 The Newton form of the polynomil tht interpoltes the three points (x 0, f(x 0 )), (x 1, f(x 1 )), nd (x 2, f(x 2 )) is Q 2 (x) = f(x 0 )+ f(x 1) f(x 0 ) f(x 2 ) (x x 0 )+ x 1 x 0 [ ] f(x 0 ) + f(x 1) f(x 0 ) x 1 x 0 (x 2 x 0 ) (x 2 x 0 )(x 2 x 1 ) (x x 0 )(x x 1 ). 3.4 The Interpoltion Problem nd the Vndermonde Determinnt An lterntive pproch to the interpoltion problem is to consider directly polynomil of the form Q n (x) = b k x k, (3.8) k=0 nd require tht the following interpoltion conditions re stisfied Q n (x j ) = f(x j ), 0 j n. (3.9) In view of Theorem 3.1 we lredy know tht this problem hs unique solution, so we should be ble to compute the coefficients of the polynomil directly from (3.8). Indeed, the interpoltion conditions, (3.9), imply tht the following equtions should hold: b 0 + b 1 x j +... + b n x n j = f(x j ), j = 0,..., n. (3.10) In mtrix form, (3.10) cn be rewritten s 1 x 0... x n 0 b 0 f(x 0 ) 1 x 1... x n 1 b 1.... = f(x 1 ).. (3.11) 1 x n... x n n b n f(x n ) In order for the system (3.11) to hve unique solution, it hs to be nonsingulr. This mens, e.g., tht the determinnt of its coefficients mtrix must not vnish, i.e. 1 x 0... x n 0 1 x 1... x n 1 0. (3.12)... 1 x n... x n n The determinnt (3.12), is known s the Vndermonde determinnt. In Lemm 3.4 we will show tht the Vndermonde determinnt equls to the product of terms of the form x i x j for i > j. Since we ssume tht the points x 0,..., x n re distinct, the determinnt in (3.12) is indeed non zero. Hence, the system (3.11) hs solution tht is lso unique, which confirms wht we lredy know ccording to Theorem 3.1. 23
3.4 The Interpoltion Problem nd the Vndermonde Determinnt D. Levy Lemm 3.4 1 x 0... x n 0 1 x 1... x n 1 = i x j ). (3.13)... i>j(x 1 x n... x n n Proof. We will prove (3.13) by induction. First we verify tht the result holds in the 2 2 cse. Indeed, 1 x 0 1 x 1 = x 1 x 0. We now ssume tht the result holds for n 1 nd consider n. We note tht the index n corresponds to mtrix of dimensions (n + 1) (n + 1), hence our induction ssumption is tht (3.13) holds for ny Vndermonde determinnt of dimension n n. We subtrct the first row from ll other rows, nd expnd the determinnt long the first column: 1 x 0... x n 0 1 x 0... x n 1 x 1... x n 0 1 0 x 1 x 0... x n 1 x n x 1 x 0... x n 1 x n 0 0 = =........ 1 x n... x n n 0 x n x 0... x n n x n x n x 0... x n n x n 0 0 For every row k we fctor out term x k x 0 : 1 x 1 + x 0... x 1 x 0... x n 1 x n 0 n 1 x.. = (x k x 0 ) 2 + x 0... x n x 0... x n n x n k=1 0.. 1 x n + x 0... Here, we used the expnsion x n 1 x n 0 = (x 1 x 0 )(x n 1 1 + x n 2 1 x 0 + x n 3 1 x 2 0 +... + x n 1 0 ), n 1 x1 n 1 i x i 0 n 1 x2 n 1 i x i 0. n 1 xn n 1 i x i 0 for the first row, nd similr expnsions for ll other rows. For every column l, strting from the second one, subtrcting the sum of x i 0 times column i (summing only over previous columns, i.e., columns i with i < l), we end up with 1 x 1... x n 1 1 n 1 x 2... x n 1 2 (x k x 0 ). (3.14) k=1... 1 x n... x n 1 n 24
D. Levy 3.5 The Lgrnge Form of the Interpoltion Polynomil Since now we hve on the RHS of (3.14) Vndermonde determinnt of dimension n n, we cn use the induction to conclude with the desired result. 3.5 The Lgrnge Form of the Interpoltion Polynomil The form of the interpoltion polynomil tht we used in (3.8) ssumed liner combintion of polynomils of degrees 0,..., n, in which the coefficients were unknown. In this section we tke different pproch nd ssume tht the interpoltion polynomil is given s liner combintion of n + 1 polynomils of degree n. This time, we set the coefficients s the interpolted vlues, {f(x j )} n, while the unknowns re the polynomils. We thus let Q n (x) = f(x j )lj n (x), (3.15) where l n j (x) re n+1 polynomils of degree n. We use two indices in these polynomils: the subscript j enumertes l n j (x) from 0 to n nd the superscript n is used to remind us tht the degree of l n j (x) is n. Note tht in this prticulr cse, the polynomils l n j (x) re precisely of degree n (nd not n). However, Q n (x), given by (3.15) my hve lower degree. In either cse, the degree of Q n (x) is n t the most. We now require tht Q n (x) stisfies the interpoltion conditions Q n (x i ) = f(x i ), 0 i n. (3.16) By substituting x i for x in (3.15) we hve Q n (x i ) = f(x j )lj n (x i ), 0 i n. In view of (3.16) we my conclude tht l n j (x) must stisfy l n j (x i ) = δ ij, i, j = 0,..., n, (3.17) where δ ij is the Krönecker delt, defined s { 1, i = j, δ ij = 0, i j. Ech polynomil l n j (x) hs n + 1 unknown coefficients. The conditions (3.17) provide exctly n + 1 equtions tht the polynomils l n j (x) must stisfy nd these equtions cn be solved in order to determine ll l n j (x) s. Fortuntely there is shortcut. An obvious wy of constructing polynomils l n j (x) of degree n tht stisfy (3.17) is the following: lj n (x) = (x x 0)... (x x j 1 )(x x j+1 )... (x x n ), 0 j n. (3.18) (x j x 0 )... (x j x j 1 )(x j x j+1 )... (x j x n ) 25
3.5 The Lgrnge Form of the Interpoltion Polynomil D. Levy The uniqueness of the interpolting polynomil of degree n given n + 1 distinct interpoltion points implies tht the polynomils l n j (x) given by (3.17) re the only polynomils of degree n tht stisfy (3.17). Note tht the denomintor in (3.18) does not vnish since we ssume tht ll interpoltion points re distinct. The Lgrnge form of the interpoltion polynomil is the polynomil Q n (x) given by (3.15), where the polynomils l n j (x) of degree n re given by (3.18). A compct form of rewriting (3.18) using the product nottion is l n j (x) = n (x x i ) i j, j = 0,..., n. (3.19) n (x j x i ) i j Exmple 3.5 We re interested in finding the Lgrnge form of the interpoltion polynomil tht interpoltes two points: (x 0, f(x 0 )) nd (x 1, f(x 1 )). We know tht the unique interpoltion polynomil through these two points is the line tht connects the two points. Such line cn be written in mny different forms. In order to obtin the Lgrnge form we let l 1 0(x) = x x 1 x 0 x 1, l 1 1(x) = x x 0 x 1 x 0. The desired polynomil is therefore given by the fmilir formul Q 1 (x) = f(x 0 )l 1 0(x) + f(x 1 )l 1 1(x) = f(x 0 ) x x 1 x 0 x 1 + f(x 1 ) x x 0 x 1 x 0. Exmple 3.6 This time we re looking for the Lgrnge form of the interpoltion polynomil, Q 2 (x), tht interpoltes three points: (x 0, f(x 0 )), (x 1, f(x 1 )), (x 2, f(x 2 )). Unfortuntely, the Lgrnge form of the interpoltion polynomil does not let us use the interpoltion polynomil through the first two points, Q 1 (x), s building block for Q 2 (x). This mens tht we hve to compute ll the polynomils l n j (x) from scrtch. We strt with l 2 0(x) = (x x 1)(x x 2 ) (x 0 x 1 )(x 0 x 2 ), l 2 1(x) = (x x 0)(x x 2 ) (x 1 x 0 )(x 1 x 2 ), l 2 2(x) = (x x 0)(x x 1 ) (x 2 x 0 )(x 2 x 1 ). 26
D. Levy 3.5 The Lgrnge Form of the Interpoltion Polynomil The interpoltion polynomil is therefore given by Q 2 (x) = f(x 0 )l 2 0(x) + f(x 1 )l 2 1(x) + f(x 2 )l 2 2(x) = f(x 0 ) (x x 1)(x x 2 ) (x 0 x 1 )(x 0 x 2 ) + f(x 1) (x x 0)(x x 2 ) (x 1 x 0 )(x 1 x 2 ) + f(x 2) (x x 0)(x x 1 ) (x 2 x 0 )(x 2 x 1 ). It is esy to verify tht indeed Q 2 (x j ) = f(x j ) for j = 0, 1, 2, s desired. Remrks. 1. One instnce where the Lgrnge form of the interpoltion polynomil my seem to be dvntgeous when compred with the Newton form is when there is need to solve severl interpoltion problems, ll given t the sme interpoltion points x 0,... x n but with different vlues f(x 0 ),..., f(x n ). In this cse, the polynomils l n j (x) re identicl for ll problems since they depend only on the points but not on the vlues of the function t these points. Therefore, they hve to be constructed only once. 2. An lterntive form for lj n (x) cn be obtined in the following wy. Define the polynomils w n (x) of degree n + 1 by n w n (x) = (x x i ). Then it its derivtive is n w n(x) = (x x i ). (3.20) i j When w x(x) is evluted t n interpoltion point, x j, there is only one term in the sum in (3.20) tht does not vnish: n w n(x j ) = (x j x i ). i j Hence, in view of (3.19), l n j (x) cn be rewritten s l n j (x) = w n (x), 0 j n. (3.21) (x x j )w n(x j ) 3. For future reference we note tht the coefficient of x n in the interpoltion polynomil Q n (x) is f(x j ). (3.22) n (x j x k ) k=0 k j 27
3.6 Divided Differences D. Levy For exmple, the coefficient of x in Q 1 (x) in Exmple 3.5 is f(x 0 ) + f(x 1). x 0 x 1 x 1 x 0 3.6 Divided Differences We recll tht Newton s form of the interpoltion polynomil is given by (see (3.6) (3.7)) Q n (x) = 0 + 1 (x x 0 ) +... + n (x x 0 )... (x x n 1 ), with 0 = f(x 0 ) nd j = f(x j) Q j 1 (x j ), 1 j n. j 1 (x j x k ) k=0 From now on, we will refer to the coefficient, j, s the j th -order divided difference. The j th -order divided difference, j, is bsed on the points x 0,..., x j nd on the vlues of the function t these points f(x 0 ),..., f(x j ). To emphsize this dependence, we use the following nottion: j = f[x 0,..., x j ], 1 j n. (3.23) We lso denote the zeroth-order divided difference s where 0 = f[x 0 ], f[x 0 ] = f(x 0 ). Using the divided differences nottion (3.23), the Newton form of the interpoltion polynomil becomes n 1 Q n (x) = f[x 0 ] + f[x 0, x 1 ](x x 0 ) +... + f[x 0,... x n ] (x x k ). (3.24) There is simple recursive wy of computing the j th -order divided difference from divided differences of lower order, s shown by the following lemm: Lemm 3.7 The divided differences stisfy: f[x 0,... x n ] = f[x 1,... x n ] f[x 0,... x n 1 ] x n x 0. (3.25) k=0 28
D. Levy 3.6 Divided Differences Proof. For ny k, we denote by Q k (x), polynomil of degree k, tht interpoltes f(x) t x 0,..., x k, i.e., Q k (x j ) = f(x j ), 0 j k. We now consider the unique polynomil P (x) of degree n 1 tht interpoltes f(x) t x 1,..., x n, nd clim tht Q n (x) = P (x) + x x n x n x 0 [P (x) Q n 1 (x)]. (3.26) In order to verify this equlity, we note tht for i = 1,..., n 1, P (x i ) = Q n 1 (x i ) so tht RHS(x i ) = P (x i ) = f(x i ). At x n, RHS(x n ) = P (x n ) = f(x n ). Finlly, t x 0, RHS(x 0 ) = P (x 0 ) + x 0 x n x n x 0 [P (x 0 ) Q n 1 (x 0 )] = Q n 1 (x 0 ) = f(x 0 ). Hence, the RHS of (3.26) interpoltes f(x) t the n + 1 points x 0,..., x n, which is lso true for Q n (x) due to its definition. Since the RHS nd the LHS in eqution (3.26) re both polynomils of degree n, the uniqueness of the interpolting polynomil (in this cse through n + 1 points) implies the equlity in (3.26). Once we estblished the equlity in (3.26) we cn compre the coefficients of the monomils on both sides of the eqution. The coefficient of x n on the left-hnd-side of (3.26) is f[x 0,..., x n ]. The coefficient of x n 1 in P (x) is f[x 1,..., x n ] nd the coefficient of x n 1 in Q n 1 (x) is f[x 0,..., x n 1 ]. Hence, the coefficient of x n on the right-hnd-side of (3.26) is 1 x n x 0 (f[x 1,..., x n ] f[x 0,..., x n 1 ]), which mens tht f[x 0,... x n ] = f[x 1,... x n ] f[x 0,... x n 1 ] x n x 0. Remrk. In some books, insted of defining the divided difference in such wy tht they stisfy (3.25), the divided differences re defined by the formul f[x 0,... x n ] = f[x 1,... x n ] f[x 0,... x n 1 ]. If this is the cse, ll our results on divided differences should be djusted ccordingly s to ccount for the missing fctor in the denomintor. 29
3.6 Divided Differences D. Levy Exmple 3.8 The second-order divided difference is f[x 0, x 1, x 2 ] = f[x 1, x 2 ] f[x 0, x 1 ] x 2 x 0 = f(x 2 ) f(x 1 ) x 2 x 1 f(x 1) f(x 0 ) x 1 x 0 x 2 x 0. Hence, the unique polynomil tht interpoltes (x 0, f(x 0 )), (x 1, f(x 1 )), nd (x 2, f(x 2 )) is Q 2 (x) = f[x 0 ] + f[x 0, x 1 ](x x 0 ) + f[x 0, x 1, x 2 ](x x 0 )(x x 1 ) = f(x 0 ) + f(x 1) f(x 0 ) x 1 x 0 (x x 0 ) + f(x 2 ) f(x 1 ) x 2 x 1 f(x 1) f(x 0 ) x 1 x 0 x 2 x 0 (x x 0 )(x x 1 ). For exmple, if we wnt to find the polynomil of degree 2 tht interpoltes ( 1, 9), (0, 5), nd (1, 3), we hve f( 1) = 9, f[ 1, 0] = 5 9 0 ( 1) = 4, f[0, 1] = 3 5 1 0 = 2, f[ 1, 0, 1] = f[0, 1] f[ 1, 0] 1 ( 1) = 2 + 4 2 = 1. so tht Q 2 (x) = 9 4(x + 1) + (x + 1)x = 5 3x + x 2. The reltions between the divided differences re schemticlly portryed in Tble 3.1 (up to third-order). We note tht the divided differences tht re being used s the coefficients in the interpoltion polynomil re those tht re locted in the top of every column. The recursive structure of the divided differences implies tht it is required to compute ll the low order coefficients in the tble in order to get the high-order ones. One importnt property of ny divided difference is tht it is symmetric function of its rguments. This mens tht if we ssume tht y 0,..., y n is ny permuttion of x 0,..., x n, then f[y 0,..., y n ] = f[x 0,..., x n ]. This property cn be clerly explined by reclling tht f[x 0,..., x n ] plys the role of the coefficient of x n in the polynomil tht interpoltes f(x) t x 0,..., x n. At the sme time, f[y 0,..., y n ] is the coefficient of x n t the polynomil tht interpoltes f(x) t the sme points. Since the interpoltion polynomil is unique for ny given dt set, the order of the points does not mtter, nd hence these two coefficients must be identicl. 30