MAT 772: Numericl Anlysis Jmes V. Lmbers August 23, 2016
2
Contents 1 Solution of Equtions by Itertion 7 1.1 Nonliner Equtions....................... 7 1.1.1 Existence nd Uniqueness................ 7 1.1.2 Sensitivity......................... 8 1.2 Simple Itertion.......................... 9 1.3 Itertive Solution of Equtions................. 16 1.4 Relxtion............................. 17 1.5 Newton s Method......................... 18 1.6 The Secnt Method........................ 25 1.7 The Bisection Method...................... 28 1.8 Sfegurded Methods...................... 31 2 Polynomil Interpoltion 35 2.1 Lgrnge Interpoltion...................... 35 2.2 Convergence............................ 40 2.3 Hermite Interpoltion...................... 43 2.4 Differentition.......................... 45 2.4.1 Finite Difference Approximtions............ 46 3 Numericl Integrtion 53 3.1 Integrtion............................ 53 3.2 Well-Posedness.......................... 53 3.2.1 Newton-Cotes Qudrture................ 54 3.3 Error Estimtes.......................... 57 3.4 The Runge Phenomenon Revisited............... 59 3.5 Composite Formule....................... 60 3.6 Richrdson Extrpoltion.................... 63 3.7 The Euler-Mclurin Expnsion................. 65 3.8 Romberg Integrtion....................... 67 3
4 CONTENTS 4 Polynomil Approximtion in the -norm 73 4.1 Normed Liner Spces...................... 73 4.2 Best Approximtion in the -norm.............. 74 4.3 Chebyshev Polynomils..................... 77 4.4 Interpoltion........................... 78 5 Polynomil Approximtion in the 2-norm 81 5.1 Best Approximtion in the 2-norm............... 81 5.2 Inner Product Spces....................... 84 5.3 Orthogonl Polynomils..................... 86 5.4 Comprisons........................... 94 6 Numericl Integrtion - II 97 6.1 Construction of Guss Qudrture Rules............ 97 6.2 Error Estimtion for Guss Qudrture............ 101 6.3 Composite Guss Formule................... 105 6.4 Rdu nd Lobtto Qudrture................. 106 7 Piecewise Polynomil Approximtion 107 7.1 Liner Interpolting Splines................... 107 7.2 Bsis Functions for Liner Splines............... 109 7.3 Cubic Splines........................... 110 7.3.1 Cubic Spline Interpoltion................ 110 7.3.2 Constructing Cubic Splines............... 111 7.3.3 Well-Posedness nd Accurcy.............. 115 7.4 Hermite Cubic Splines...................... 117 7.5 Bsis Functions for Cubic Splines................ 118 8 Initil Vlue Problems for ODEs 121 8.1 Theory of Initil-Vlue Problems................ 121 8.2 One-Step Methods........................ 122 8.3 Consistency nd Convergence.................. 125 8.4 An Implicit One-Step Method.................. 127 8.5 Runge-Kutt Methods...................... 128 8.6 Multistep Methods........................ 132 8.7 Consistency nd Zero-Stbility................. 136 8.8 Stiff Differentil Equtions.................... 139 8.9 Dhlquist s Theorems...................... 144 8.10 Anlysis of Multistep Methods................. 145
CONTENTS 5 Index 148
6 CONTENTS
Chpter 1 Solution of Equtions by Itertion 1.1 Nonliner Equtions The solution of single liner eqution is n extremely simple tsk. We now explore the much more difficult problem of solving nonliner equtions of the form f(x) = 0, where f(x) : R n R m cn be ny known function. A solution x of such nonliner eqution is clled root of the eqution, s well s zero of the function f. 1.1.1 Existence nd Uniqueness For simplicity, we ssume tht the function f : R n R m is continuous on the domin under considertion. Then, ech eqution f i (x) = 0, i = 1,..., m, defines hypersurfce in R m. The solution of f(x) = 0 is the intersection of these hypersurfces, if the intersection is not empty. It is not hrd to see tht there cn be unique solution, infinitely mny solutions, or no solution t ll. For generl eqution f(x) = 0, it is not possible to chrcterize the conditions under which solution exists or is unique. However, in some situtions, it is possible to determine existence nlyticlly. For exmple, in one dimension, the Intermedite Vlue Theorem implies tht if continuous function f(x) stisfies f() 0 nd f(b) 0 where < b, then f(x) = 0 for some x (, b). 7
8 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION Similrly, it cn be concluded tht f(x) = 0 for some x (, b) if the function (x z)f(x) 0 for x = nd x = b, where z (, b). This condition cn be generlized to higher dimensions. If S R n is n open, bounded set, nd (x z) T f(x) 0 for ll x on the boundry of S nd for some z S, then f(x) = 0 for some x S. Unfortuntely, checking this condition cn be difficult in prctice. One useful result from clculus tht cn be used to estblish existence nd, in some sense, uniqueness of solution is the Inverse Function Theorem, which sttes tht if the Jcobin of f is nonsingulr t point x 0, then f is invertible ner x 0 nd the eqution f(x) = y hs unique solution for ll y ner f(x 0 ). If the Jcobin of f t point x 0 is singulr, then f is sid to be degenerte t x 0. Suppose tht x 0 is solution of f(x) = 0. Then, in one dimension, degenercy mens f (x 0 ) = 0, nd we sy tht x 0 is double root of f(x). Similrly, if f (j) (x 0 ) = 0 for j = 0,..., m 1, then x 0 is root of multiplicity m. We will see tht degenercy cn cuse difficulties when trying to solve nonliner equtions. 1.1.2 Sensitivity The bsolute condition number of function f(x) is mesure of how perturbtion in x, denoted by x + ɛ for some smll ɛ, is mplified by f(x). Using the Men Vlue Theorem, we hve f(x + ɛ) f(x) = f (c)(x + ɛ x) = f (c) ɛ where c is between x nd x + ɛ. With ɛ being smll, the bsolute condition number cn be pproximted by f (x), the fctor by which the perturbtion in x (ɛ) is mplified to obtin the perturbtion in f(x). In solving nonliner eqution in one dimension, we re trying to solve n inverse problem; tht is, insted of computing y = f(x) (the forwrd problem), we re computing x = f 1 (0), ssuming tht f is invertible ner the root. It follows from the differentition rule d dx [f 1 (x)] = 1 f (f 1 (x)) tht the condition number for solving f(x) = 0 is pproximtely 1/ f (x ), where x is the solution. This discussion cn be generlized to higher dimensions, where the condition number is mesured using the norm of the Jcobin.
1.2. SIMPLE ITERATION 9 Using bckwrd error nlysis, we ssume tht the pproximte solution ˆx = ˆf 1 (0), obtined by evluting n pproximtion of f 1 t the exct input y = 0, cn lso be viewed s evluting the exct function f 1 t nerby input ŷ = ɛ. Tht is, the pproximte solution ˆx = f 1 (ɛ) is the exct solution of nerby problem. From this viewpoint, it cn be seen from grph tht if f is lrge ner x, which mens tht the condition number of the problem f(x) = 0 is smll (tht is, the problem is well-conditioned), then even if ɛ is reltively lrge, ˆx = f 1 (ɛ) is close to x. On the other hnd, if f is smll ner x, so tht the problem is ill-conditioned, then even if ɛ is smll, ˆx cn be fr wy from x. 1.2 Simple Itertion A nonliner eqution of the form f(x) = 0 cn be rewritten to obtin n eqution of the form g(x) = x, in which cse the solution is fixed point of the function g. This formultion of the originl problem f(x) = 0 will leds to simple solution method known s fixed-point itertion, or simple itertion. Before we describe this method, however, we must first discuss the questions of existence nd uniqueness of solution to the modified problem g(x) = x. The following result nswers these questions. Theorem (Brouwer s Fixed Point Theorem) Let g be continuous function on the intervl [, b]. If g(x) [, b] for ech x [, b], then g hs fixed point in [, b]. Given continuous function g tht is known to hve fixed point in n intervl [, b], we cn try to find this fixed point by repetedly evluting g t points in [, b] until we find point x for which g(x) = x. This is the essence of the method of fixed-point itertion, the implementtion of which we now describe. Algorithm (Fixed-Point Itertion) Let g be continuous function defined on the intervl [, b]. The following lgorithm computes number x (, b) tht is solution to the eqution g(x) = x. Choose n initil guess x 0 in [, b]. for k = 0, 1, 2,... do x k+1 = g(x k )
10 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION if x k+1 x k is sufficiently smll then x = x k+1 return x end end Under wht circumstnces will fixed-point itertion converge to fixed point x? We sy tht function g tht is continuous on [, b] stisfies Lipschitz condition on [, b] if there exists positive constnt L such tht g(x) g(y) L x y, x, y [, b]. The constnt L is clled Lipschitz constnt. If, in ddition, L < 1, we sy tht g is contrction on [, b]. If we denote the error in x k by e k = x k x, we cn see from the fct tht g(x ) = x tht if x k [, b], then e k+1 = x k+1 x = g(x k ) g(x ) L x k x L e k < e k. Therefore, if g stisfies the conditions of the Brouwer Fixed-Point Theorem, nd g is contrction on [, b], nd x 0 [, b], then fixed-point itertion is convergent; tht is, x k converges to x. Furthermore, the fixed point x must be unique, for if there exist two distinct fixed points x nd y in [, b], then, by the Lipschitz condition, we hve 0 < x y = g(x ) g(y ) L x y < x y, which is contrdiction. Therefore, we must hve x = y. We summrize our findings with the sttement of the following result. Theorem (Contrction Mpping Theorem) Let g be continuous function on the intervl [, b]. If g(x) [, b] for ech x [, b], nd if there exists constnt 0 < L < 1 such tht g(x) g(y) L x y, x, y [, b], then g hs unique fixed point x in [, b], nd the sequence of itertes {x k } k=0 converges to x, for ny initil guess x 0 [, b]. In generl, when fixed-point itertion converges, it does so t rte tht vries inversely with the Lipschitz constnt L. If the smllest possible Lipschitz constnt on n intervl contining x ctully pproches zero s the width of the intervl converges to zero, then the itertion cn converge
1.2. SIMPLE ITERATION 11 much more rpidly. We will discuss convergence behvior of vrious methods for solving nonliner equtions in lter lecture. Often, there re mny wys to convert n eqution of the form f(x) = 0 to one of the form g(x) = x, the simplest being g(x) = x φ(x)f(x) for ny function φ. However, it is importnt to ensure tht the conversion yields function g for which fixed-point itertion will converge. Exmple We use fixed-point itertion to compute fixed point of g(x) = cos x in the intervl [0, 1]. Since cos x 1 for ll x, nd cos x 0 on [0, π/2], nd π/2 > 1, we know tht cos x mps [0, 1] into [0, 1]. Since cos x is continuous for ll x, we cn conclude tht cos x hs fixed point in [0, 1]. Becuse g (x) = sin x, nd sin x sin 1 on [0, 1], we cn pply the Men Vlue Theorem to obtin cos x cos y = sin c x y sin 1 x y, for ny x, y [0, 1], where c lies between x nd y. As sin 1 < 1, we conclude tht cos x is contrction on [0, 1], nd therefore it hs unique fixed point on [0, 1]. To use fixed-point itertion, we first choose n initil guess x 0 in [0, 1]. As discussed bove, fixed-point itertion will converge for ny initil guess, so we choose x 0 = 0.5. The tble on pge 4 shows the outcome of severl itertions, in which we compute x k+1 = cos x k for k = 0, 1, 2,.... As the tble shows, it tkes nerly 30 itertions to obtin the fixed point to five deciml plces, nd there is considerble oscilltion in the first itertions before resonble pproximte solution is obtined. This oscilltion is shown in Figure 1.1. As x k converges, it cn be seen from the tble tht the error is reduced by fctor of roughly 2/3 from itertion to itertion. This suggests tht cos x is reltively poor choice for the itertion function g(x) to solve the eqution g(x) = cos x. In generl, nonliner equtions cnnot be solved in finite sequence of steps. As liner equtions cn be solved using direct methods such s Gussin elimintion, nonliner equtions usully require itertive methods. In itertive methods, n pproximte solution is refined with ech itertion until it is determined to be sufficiently ccurte, t which time the itertion termintes. Since it is desirble for itertive methods to converge to the solution s rpidly s possible, it is necessry to be ble to mesure the speed with which n itertive method converges. To tht end, we ssume tht n itertive method genertes sequence of itertes x 0, x 1, x 2,... tht converges to the exct solution x. Idelly, we
12 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION Figure 1.1: Fixed-point itertion pplied to the eqution cos x = x, with x 0 = 0.5. would like the error in given iterte x k+1 to be much smller thn the error in the previous iterte x k. For exmple, if the error is rised to power greter thn 1 from itertion to itertion, then, becuse the error is typiclly less thn 1, it will pproch zero very rpidly. This leds to the following definition. Definition (Order nd Rte of Convergence) Let {x k } k=0 be sequence in R n tht converges to x R n nd ssume tht x k x for ech k. We sy tht the order of convergence of {x k } to x is order r, with symptotic error constnt C, if x k+1 x lim k x k x r = C, where r 1. If r = 1, then the number ρ = log 10 C is clled the symptotic rte of convergence. If r = 1, nd 0 < C < 1, we sy tht convergence is liner. If r = 1 nd C = 0, or if 1 < r < 2 for ny positive C, then we sy tht convergence is
1.2. SIMPLE ITERATION 13 superliner. If r = 2, then the method converges qudrticlly, nd if r = 3, we sy it converges cubiclly, nd so on. Note tht the vlue of C need only be bounded bove in the cse of liner convergence. When convergence is liner, the symptotic rte of convergence ρ indictes the number of correct deciml digits obtined in single itertion. In other words, 1/ρ + 1 itertions re required to obtin n dditionl correct deciml digit, where x is the floor of x, which is the lrgest integer tht is less thn or equl to x. If g stisfies the conditions of the Contrction Mpping Theorem with Lipschitz constnt L, then Fixed-point Itertion chieves t lest liner convergence, with n symptotic error constnt tht is bounded bove by L. This vlue cn be used to estimte the number of itertions needed to obtin n dditionl correct deciml digit, but it cn lso be used to estimte the totl number of itertions needed for specified degree of ccurcy. From the Lipschitz condition, we hve, for k 1, From x k x L x k 1 x L k x 0 x. x 0 x x 0 x 1 + x 1 x x 0 x 1 + x 1 x x 0 x 1 + L x 0 x we obtin x k x Lk 1 L x 1 x 0. Therefore, in order to stisfy x k x ɛ, the number of itertions, k, must stisfy k ln x 1 x 0 ln(ɛ(1 L)). ln(1/l) Tht is, we cn bound the number of itertions fter performing single itertion, s long s the Lipschitz constnt L is known. We know tht Fixed-point Itertion will converge to the unique fixed point in [, b] if g stisfies the conditions of the Contrction Mpping Theorem. However, if g is differentible on [, b], its derivtive cn be used to obtin n lterntive criterion for convergence tht cn be more prcticl thn computing the Lipschitz constnt L. If we denote the error in x k by e k = x k x, we cn see from Tylor s Theorem nd the fct tht g(x ) = x tht e k+1 = x k+1 x = g(x k ) g(x ) = g (x )(x k x ) + 1 2 g (ξ k )(x k x ) 2 = g (x )e k + O(e 2 k ),
14 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION where ξ k lies between x k nd x. Therefore, if g (x ) k, where k < 1, then Fixed-point Itertion is loclly convergent; tht is, it converges if x 0 is chosen sufficiently close to x. This leds to the following result. Theorem (Fixed-point Theorem) Let g be continuous function on the intervl [, b], nd let g be differentible on [, b]. If g(x) [, b] for ech x [, b], nd if there exists constnt k < 1 such tht g (x) k, x (, b), then the sequence of itertes {x k } k=0 converges to the unique fixed point x of g in [, b], for ny initil guess x 0 [, b]. It cn be seen from the preceding discussion why g (x) must be bounded wy from 1 on (, b), s opposed to the weker condition g (x) < 1 on (, b). If g (x) is llowed to pproch 1 s x pproches point c (, b), then it is possible tht the error e k might not pproch zero s k increses, in which cse Fixed-point Itertion would not converge. Suppose tht g stisfies the conditions of the Fixed-Point Theorem, nd tht g is lso continuously differentible on [, b]. We cn use the Men Vlue Theorem to obtin e k+1 = x k+1 x = g(x k ) g(x ) = g (ξ k )(x k x ) = g (ξ k )e k, where ξ k lies between x k nd x. It follows from the continuity of g t x tht for ny initil iterte x 0 [, b], Fixed-point Itertion converges linerly with symptotic error constnt g (x ), since, by the definition of ξ k nd the continuity of g, e k+1 lim = lim k e k k g (ξ k ) = g (x ). Recll tht the conditions we hve stted for liner convergence re nerly identicl to the conditions for g to hve unique fixed point in [, b]. The only difference is tht now, we lso require g to be continuous on [, b]. The derivtive cn lso be used to indicte why Fixed-point Itertion might not converge. Exmple The function g(x) = x 2 + 3 16 hs two fixed points, x 1 = 1/4 nd x 2 = 3/4, s cn be determined by solving the qudrtic eqution x 2 + 3 16 = x. If we consider the intervl [0, 3/8], then g stisfies the conditions of the Fixed-point Theorem, s g (x) = 2x < 1 on this intervl, nd therefore Fixed-point Itertion will converge to x 1 for ny x 0 [0, 3/8].
1.2. SIMPLE ITERATION 15 On the other hnd, g (3/4) = 2(3/4) = 3/2 > 1. Therefore, it is not possible for g to stisfy the conditions of the Fixed-point Theorem. Furthemore, if x 0 is chosen so tht 1/4 < x 0 < 3/4, then Fixed-point Itertion will converge to x 1 = 1/4, wheres if x 0 > 3/4, then Fixed-point Itertion diverges. The fixed point x 2 = 3/4 in the preceding exmple is n unstble fixed point of g, mening tht no choice of x 0 yields sequence of itertes tht converges to x 2. The fixed point x 1 = 1/4 is stble fixed point of g, mening tht ny choice of x 0 tht is sufficiently close to x 1 yields sequence of itertes tht converges to x 1. The preceding exmple shows tht Fixed-point Itertion pplied to n eqution of the form x = g(x) cn fil to converge to fixed point x if g (x ) > 1. We wish to determine whether this condition indictes nonconvergence in generl. If g (x ) > 1, nd g is continuous in neighborhood of x, then there exists n intervl x x δ such tht g (x) > 1 on the intervl. If x k lies within this intervl, it follows from the Men Vlue Theorem tht x k+1 x = g(x k ) g(x ) = g (η) x k x, where η lies between x k nd x. Becuse η is lso in this intervl, we hve x k+1 x > x k x. In other words, the error in the itertes increses whenever they fll within sufficiently smll intervl contining the fixed point. Becuse of this increse, the itertes must eventully fll outside of the intervl. Therefore, it is not possible to find k 0, for ny given δ, such tht x k x δ for ll k k 0. We hve thus proven the following result. Theorem Let g hve fixed point t x, nd let g be continuous in neighborhood of x. If g (x ) > 1, then Fixed-point Itertion does not converge to x for ny initil guess x 0 except in finite number of itertions. Now, suppose tht in ddition to the conditions of the Fixed-point Theorem, we ssume tht g (x ) = 0, nd tht g is twice continuously differentible on [, b]. Then, using Tylor s Theorem, we obtin e k+1 = g(x k ) g(x ) = g (x )(x k x ) + 1 2 g (ξ k )(x k x ) 2 = 1 2 g (ξ k )e 2 k,
16 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION where ξ k lies between x k nd x. It follows tht for ny initil iterte x 0 [, b], Fixed-point Itertion converges t lest qudrticlly, with symptotic error constnt g (x )/2. Lter, this will be exploited to obtin qudrticlly convergent method for solving nonliner equtions of the form f(x) = 0. 1.3 Itertive Solution of Equtions Now tht we understnd the convergence behvior of Fixed-point Itertion, we consider the ppliction of Fixed-point Itertion to the solution of n eqution of the form f(x) = 0. When rewriting this eqution in the form x = g(x), it is essentil to choose the function g wisely. One guideline is to choose g(x) = x φ(x)f(x), where the function φ(x) is, idelly, nonzero except possibly t solution of f(x) = 0. This cn be stisfied by choosing φ(x) to be constnt, but this cn fil, s the following exmple illustrtes. Exmple Consider the eqution x + ln x = 0. By the Intermedite Vlue Theorem, this eqution hs solution in the intervl [0.5, 0.6]. Furthermore, this solution is unique. To see this, let f(x) = x + ln x. Then f (x) = x + 1/x > 0 on the domin of f, which mens tht f is incresing on its entire domin. Therefore, it is not possible for f(x) = 0 to hve more thn one solution. We consider using Fixed-point Itertion to solve the equivlent eqution x = x (1)(x + ln x) = ln x. However, with g(x) = ln x, we hve g (x) = 1/x > 1 on the intervl [0.5, 0.6]. Therefore, by the preceding theorem, Fixed-point Itertion will fil to converge for ny initil guess in this intervl. We therefore pply g 1 (x) = e x to both sides of the eqution x = g(x) to obtin which simplifies to g 1 (x) = g 1 (g(x)) = x, x = e x. The function g(x) = e x stisfies g (x) < 1 on [0.5, 0.6], s g (x) = e x, nd e x < 1 when the rgument x is positive. By nrrowing this intervl to [0.52, 0.6], which is mpped into itself by this choice of g, we cn pply the Fixed-point Theorem to conclude tht Fixed-point Itertion will converge to the unique fixed point of g for ny choice of x 0 in the intervl.
1.4. RELAXATION 17 1.4 Relxtion As previously discussed, common choice for function g(x) to use with Fixed-point Itertion to solve the eqution f(x) = 0 is function of the form g(x) = x φ(x)f(x), where φ(x) is nonzero. Clerly, the simplest choice of φ(x) is constnt function φ(x) λ, but it is importnt to choose λ so tht Fixed-point Itertion with g(x) will converge. Suppose tht x is solution of the eqution f(x) = 0, nd tht f is continuously differentible in neighborhood of x, with f (x ) = α > 0. Then, by continuity, there exists n intervl [x δ, x + δ] contining x on which m f (x) M, for positive constnts m nd M. It follows tht for ny choice of positive constnt λ, 1 λm 1 λf (x) 1 λm. By choosing 2 λ = M + m, we obtin 1 λm = k, 1 λm = k, k = M m M + m, which stisfies 0 < k < 1. Therefore, if we define g(x) = x λf(x), we hve g (x) k < 1 on [x δ, x + δ]. Furthermore, if x x δ, then, by the Men Vlue Theorem, g(x) x = g(x) g(x ) = g (ξ) x x < δ, nd therefore g mps the intervl [x δ, x + δ] into itself. We conclude tht the Fixed-point Theorem pplies, nd Fixed-point Itertion converges linerly to x for ny choice of x 0 in [x δ, x + δ], with symptotic error constnt 1 λα k. In summry, if f is continuously differentible in neighborhood of root x of f(x) = 0, nd f(x ) is nonzero, then there exists constnt λ such tht Fixed-point Itertion with g(x) = x λf(x) converges to x for x 0 chosen sufficiently close to x. This pproch to Fixed-point Itertion, with constnt φ, is known s relxtion. Convergence cn be ccelerted by llowing λ to vry from itertion to itertion. Intuitively, n effective choice is to try to minimize g (x) ner x by setting λ = 1/f (x k ), for ech k, so tht g (x k ) = 1 λf (x k ) = 0. This results in liner convergence with n symptotic error constnt of 0, which indictes fster thn liner convergence. We will see tht convergence is ctully qudrtic.
18 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION 1.5 Newton s Method To develop more effective method for solving this problem of computing solution to f(x) = 0, we cn ddress the following questions: Are there cses in which the problem esy to solve, nd if so, how do we solve it in such cses? Is it possible to pply our method of solving the problem in these esy cses to more generl cses? In this course, we will see tht these questions re useful for solving vriety of problems. For the problem t hnd, we sk whether the eqution f(x) = 0 is esy to solve for ny prticulr choice of the function f. Certinly, if f is liner function, then it hs formul of the form f(x) = m(x )+b, where m nd b re constnts nd m 0. Setting f(x) = 0 yields the eqution m(x ) + b = 0, which cn esily be solved for x to obtin the unique solution x = b m. We now consider the cse where f is not liner function. Using Tylor s theorem, it is simple to construct liner function tht pproximtes f(x) ner given point x 0. This function is simply the first Tylor polynomil of f(x) with center x 0, P 1 (x) = f(x 0 ) + f (x 0 )(x x 0 ). This function hs useful geometric interprettion, s its grph is the tngent line of f(x) t the point (x 0, f(x 0 )). We cn obtin n pproximte solution to the eqution f(x) = 0 by determining where the liner function P 1 (x) is equl to zero. If the resulting vlue, x 1, is not solution, then we cn repet this process, pproximting f by liner function ner x 1 nd once gin determining where this pproximtion is equl to zero. The resulting lgorithm is Newton s method, which we now describe in detil. Algorithm (Newton s Method) Let f : R R be differentible function. The following lgorithm computes n pproximte solution x to the eqution f(x) = 0.
1.5. NEWTON S METHOD 19 Choose n initil guess x 0 for k = 0, 1, 2,... do if f(x k ) is sufficiently smll then x = x k return x end x k+1 = x k f(x k) f (x k ) if x k+1 x k is sufficiently smll then x = x k+1 return x end end When Newton s method converges, it does so very rpidly. However, it cn be difficult to ensure convergence, prticulrly if f(x) hs horizontl tngents ner the solution x. Typiclly, it is necessry to choose strting iterte x 0 tht is close to x. As the following result indictes, such choice, if close enough, is indeed sufficient for convergence. Theorem (Convergence of Newton s Method) Let f be twice continuously differentible on the intervl [, b], nd suppose tht f(c) = 0 nd f (c) = 0 for some c [, b]. Then there exists δ > 0 such tht Newton s Method pplied to f(x) converges to c for ny initil guess x 0 in the intervl [c δ, c + δ]. Exmple We will use of Newton s Method in computing 2. This number stisfies the eqution f(x) = 0 where f(x) = x 2 2. Since f (x) = 2x, it follows tht in Newton s Method, we cn obtin the next iterte x n+1 from the previous iterte x n by x n+1 = x n f(x n) f (x n ) = x n x2 n 2 = x n x2 n + 2 = x n 2x n 2x n 2x n 2 + 1. x n We choose our strting iterte x 0 = 1, nd compute the next severl itertes s follows: x 1 = 1 2 + 1 1 = 1.5 x 2 = 1.5 2 + 1 1.5 = 1.41666667
20 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION x 3 = 1.41421569 x 4 = 1.41421356 x 5 = 1.41421356. Since the fourth nd fifth itertes gree in to eight deciml plces, we ssume tht 1.41421356 is correct solution to f(x) = 0, to t lest eight deciml plces. The first two itertions re illustrted in Figure 1.2. Figure 1.2: Newton s Method pplied to f(x) = x 2 2. The bold curve is the grph of f. The initil iterte x 0 is chosen to be 1. The tngent line of f(x) t the point (x 0, f(x 0 )) is used to pproximte f(x), nd it crosses the x-xis t x 1 = 1.5, which is much closer to the exct solution thn x 0. Then, the tngent line t (x 1, f(x 1 )) is used to pproximte f(x), nd it crosses the x-xis t x 2 = 1.41 6, which is lredy very close to the exct solution. Exmple Newton s Method cn be used to compute the reciprocl of number without performing ny divisions. The solution, 1/, stisfies the eqution f(x) = 0, where f(x) = 1 x.
1.5. NEWTON S METHOD 21 Since f (x) = 1 x 2, it follows tht in Newton s Method, we cn obtin the next iterte x n+1 from the previous iterte x n by x n+1 = x n 1/x n 1/x 2 n = x n 2 1/x n + 1/x n 1/x 2 n = 2x n x 2 n. Note tht no divisions re necessry to obtin x n+1 from x n. This itertion ws ctully used on older IBM computers to implement division in hrdwre. We use this itertion to compute the reciprocl of = 12. Choosing our strting iterte to be 0.1, we compute the next severl itertes s follows: x 1 = 2(0.1) 12(0.1) 2 = 0.08 x 2 = 2(0.12) 12(0.12) 2 = 0.0832 x 3 = 0.0833312 x 4 = 0.08333333333279 x 5 = 0.08333333333333. We conclude tht 0.08333333333333 is n ccurte pproximtion to the correct solution. Now, suppose we repet this process, but with n initil iterte of x 0 = 1. Then, we hve x 1 = 2(1) 12(1) 2 = 10 x 2 = 2( 10) 12( 10) 2 = 1220 x 3 = 2( 1220) 12( 1220) 2 = 17863240 It is cler tht this sequence of itertes is not going to converge to the correct solution. In generl, for this itertion to converge to the reciprocl of, the initil iterte x 0 must be chosen so tht 0 < x 0 < 2/. This condition gurntees tht the next iterte x 1 will t lest be positive. The contrst between the two choices of x 0 re illustrted in Figure 1.3 for = 8. We now nlyze the convergence of Newton s Method pplied to the eqution f(x) = 0, where we ssume tht f is twice continuously differentible ner the exct solution x. Using Tylor s Theorem, we obtin e k+1 = x k+1 x
22 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION Figure 1.3: Newton s Method used to compute the reciprocl of 8 by solving the eqution f(x) = 8 1/x = 0. When x 0 = 0.1, the tngent line of f(x) t (x 0, f(x 0 )) crosses the x-xis t x 1 = 0.12, which is close to the exct solution. When x 0 = 1, the tngent line crosses the x-xis t x 1 = 6, which cuses sercing to continue on the wrong portion of the grph, so the sequence of itertes does not converge to the correct solution. = x k f(x k) f (x k ) x = e k f(x k) f (x k ) = e k 1 [f(x f ) f (x k )(x x k ) 12 ] (x k ) f (ξ k )(x k x ) 2 [f (x k )(x x k ) + 12 ] f (ξ k )(x k x ) 2 = e k + 1 f (x k ) = e k + 1 f (x k ) = e k e k + f (ξ k ) 2f (x k ) e2 k [ f (x k )e k + 1 ] 2 f (ξ k )e 2 k
1.5. NEWTON S METHOD 23 = f (ξ k ) 2f (x k ) e2 k where ξ k is between x k nd x. Suppose tht f (x ) 0. Then, by the continuity of f nd f, there exists δ > 0 such tht on the intervl I δ = [x δ, x + δ], there is constnt A such tht f (x) f (y) A, x, y I δ. If x 0 I δ is chosen so tht x 0 x 1/A, then, by the bove Tylor expnsion, Continuing by induction, we obtin x 1 x 1 2 A x 0 x 2 1 2 x 0 x. x k x 2 k min{1/a, δ}, nd therefore Newton s method converges to x, provided tht x 0 is chosen sufficiently close to x. Becuse, for ech k, ξ k lies between x k nd x, ξ k converges to x s well. By the continuity of f on I δ, we conclude tht Newton s method converges qudrticlly to x, with symptotic error constnt C = f (x ) 2 f (x ). Exmple Suppose tht Newton s Method is used to find the solution of f(x) = 0, where f(x) = x 2 2. We exmine the error e k = x k x, where x = 2 is the exct solution. We hve k x k e k 0 1 0.41421356237310 1 1.5 0.08578643762690 2 1.41666666666667 0.00245310429357 3 1.41421568627457 0.00000212390141 4 1.41421356237469 0.00000000000159 We cn determine nlyticlly tht Newton s Method converges qudrticlly, nd in this exmple, the symptotic error constnt is f ( 2)/2f ( 2) 0.35355. Exmining the numbers in the tble bove, we cn see tht the
24 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION number of correct deciml plces pproximtely doubles with ech itertion, which is typicl of qudrtic convergence. Furthermore, we hve e 4 e 3 2 0.35352, so the ctul behvior of the error is consistent with the behvior tht is predicted by theory. It is esy to see from the bove nlysis, however, tht if f (x ) is very smll, or zero, then convergence cn be very slow, or my not even occur. Exmple The function f(x) = (x 1) 2 e x hs double root t x = 1, nd therefore f (x ) = 0. Therefore, the previous convergence nlysis does not pply. Insted, we obtin e k+1 = x k+1 1 = x k f(x k) f (x k ) 1 = x k (x k 1) 2 e x k [2(x k 1) + (x k 1) 2 ]e x k 1 e 2 k = e k 2e k + e 2 k [ = 1 1 ] e k 2 + e k [ = 1 1 ] e k x k + 1 x k = x k + 1 e k. It follows tht if we choose x 0 > 0, then Newton s method converges to x = 1 linerly, with symptotic error constnt C = 1 2. Normlly, convergence of Newton s method is only ssured if x 0 is chosen sufficiently close to x. However, in some cses, it is possible to prove tht Newton s method converges on n intervl, under certin conditions on the sign of the derivtives of f on tht intervl. For exmple, suppose tht on the intervl I δ = [x, x +δ], f (x) > 0 nd f (x) > 0, so tht f is incresing nd concve up on this intervl.
1.6. THE SECANT METHOD 25 Let x k I δ. Then, from x k+1 = x k f(x k) f (x k ), we hve x k+1 < x k, becuse f, being equl to zero t x nd incresing on I δ, must be positive t x k. However, becuse x k+1 x = f (ξ k ) 2f (x k ) (x k x ) 2, nd f nd f re both positive t x k, we must lso hve x k+1 > x. It follows tht the sequence {x k } is monotonic nd bounded, nd therefore must be convergent to limit x I δ. From the convergence of the sequence nd the determintion of x k+1 from x k, it follows tht f(x ) = 0. However, f is positive on (x, x + δ], which mens tht we must hve x = x, so Newton s method converges to x. Using the previous nlysis, it cn be shown tht this convergence is qudrtic. 1.6 The Secnt Method One drwbck of Newton s method is tht it is necessry to evlute f (x) t vrious points, which my not be prcticl for some choices of f. The secnt method voids this issue by using finite difference to pproximte the derivtive. As result, f(x) is pproximted by secnt line through two points on the grph of f, rther thn tngent line through one point on the grph. Since secnt line is defined using two points on the grph of f(x), s opposed to tngent line tht requires informtion t only one point on the grph, it is necessry to choose two initil itertes x 0 nd x 1. Then, s in Newton s method, the next iterte x 2 is then obtined by computing the x-vlue t which the secnt line pssing through the points (x 0, f(x 0 )) nd (x 1, f(x 1 )) hs y-coordinte of zero. This yields the eqution which hs the solution f(x 1 ) f(x 0 ) x 1 x 0 (x 2 x 1 ) + f(x 1 ) = 0 x 2 = x 1 f(x 1)(x 1 x 0 ) f(x 1 ) f(x 0 )
26 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION which cn be rewritten s follows: x 2 = x 1 f(x 1)(x 1 x 0 ) f(x 1 ) f(x 0 ) f(x 1 ) f(x 0 ) = x 1 f(x 1 ) f(x 0 ) f(x 1)(x 1 x 0 ) f(x 1 ) f(x 0 ) = x 1(f(x 1 ) f(x 0 )) f(x 1 )(x 1 x 0 ) f(x 1 ) f(x 0 ) = x 1f(x 1 ) x 1 f(x 0 ) x 1 f(x 1 ) + x 0 f(x 1 ) f(x 1 ) f(x 0 ) = x 0f(x 1 ) x 1 f(x 0 ). f(x 1 ) f(x 0 ) This leds to the following lgorithm. Algorithm (Secnt Method) Let f : R R be continuous function. The following lgorithm computes n pproximte solution x to the eqution f(x) = 0. Choose two initil guesses x 0 nd x 1 for k = 1, 2, 3,... do if f(x k ) is sufficiently smll then x = x k return x end x k+1 = x k 1f(x k ) x k f(x k 1 ) f(x k ) f(x k 1 ) if x k+1 x k is sufficiently smll then x = x k+1 return x end end Like Newton s method, it is necessry to choose the strting iterte x 0 to be resonbly close to the solution x. Convergence is not s rpid s tht of Newton s Method, since the secnt-line pproximtion of f is not s ccurte s the tngent-line pproximtion employed by Newton s method. Exmple We will use the Secnt Method to solve the eqution f(x) = 0, where f(x) = x 2 2. This method requires tht we choose two initil itertes x 0 nd x 1, nd then compute subsequent itertes using the formul x n+1 = x n f(x n)(x n x n 1 ), n = 1, 2, 3,.... f(x n ) f(x n 1 )
1.6. THE SECANT METHOD 27 We choose x 0 = 1 nd x 1 = 1.5. Applying the bove formul, we obtin x 2 = 1.4 x 3 = 1.41379310344828 x 4 = 1.41421568627451. As we cn see, the itertes produced by the Secnt Method re converging to the exct solution x = 2, but not s rpidly s those produced by Newton s Method. We now prove tht the Secnt Method converges if x 0 is chosen sufficiently close to solution x of f(x) = 0, if f is continuously differentible ner x nd f (x ) = α 0. Without loss of generlity, we ssume α > 0. Then, by the continuity of f, there exists n intervl I δ = [x δ, x + δ] such tht 3α 4 f (x) 5α 4, x I δ. It follows from the Men Vlue Theorem tht x k+1 x = x k x x k x k 1 f(x k ) f(x k ) f(x k 1 ) = x k x f (θ k )(x k x ) f (ϕ k ) [ = 1 f ] (θ k ) f (x k x ), (ϕ k ) where θ k lies between x k nd x, nd ϕ k lies between x k nd x k 1. Therefore, if x k 1 nd x k re in I δ, then so re ϕ k nd θ k, nd x k+1 stisfies { x k+1 x mx 1 5α/4 3α/4, 1 3α/4 } 5α/4 x k x 2 3 x k x. We conclude tht if x 0, x 1 I δ, then ll subsequent itertes lie in I δ, nd the Secnt Method converges t lest linerly, with symptotic rte constnt 2/3. The order of convergence of the Secnt Method cn be determined using result, which we will not prove here, stting tht if {x k } k=0 is the sequence of itertes produced by the Secnt Method for solving f(x) = 0, nd if this sequence converges to solution x, then for k sufficiently lrge, for some constnt S. x k+1 x S x k x x k 1 x
28 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION We ssume tht {x k } converges to x of order α. Then, dividing both sides of the bove reltion by x k x α, we obtin x k+1 x x k x α S x k x 1 α x k 1 x. Becuse α is the order of convergence, the left side must converge to positive constnt C s k. It follows tht the right side must converge to positive constnt s well, s must its reciprocl. In other words, there must exist positive constnts C 1 nd C 2 x k x x k 1 x α C 1, x k x α 1 x k 1 x C 2. This cn only be the cse if there exists nonzero constnt β such tht which implies tht x k x x k 1 x α = ( xk x α 1 ) β x k 1 x, 1 = (α 1)β nd α = β. Eliminting β, we obtin the eqution which hs the solutions α 1 = 1 + 5 2 α 2 α 1 = 0, 1.618, α 2 = 1 5 2 0.618. Since we must hve α 1, the rte of convergence is 1.618. 1.7 The Bisection Method Suppose tht f(x) is continuous function tht chnges sign on the intervl [, b]. Then, by the Intermedite Vlue Theorem, f(x) = 0 for some x [, b]. How cn we find the solution, knowing tht it lies in this intervl? The method of bisection ttempts to reduce the size of the intervl in which solution is known to exist. Suppose tht we evlute f(m), where m = ( + b)/2. If f(m) = 0, then we re done. Otherwise, f must chnge sign on the intervl [, m] or [m, b], since f() nd f(b) hve different signs. Therefore, we cn cut the size of our serch spce in hlf, nd continue
1.7. THE BISECTION METHOD 29 this process until the intervl of interest is sufficiently smll, in which cse we must be close to solution. The following lgorithm implements this pproch. Algorithm (Bisection) Let f be continuous function on the intervl [, b] tht chnges sign on (, b). The following lgorithm computes n pproximtion p to number p in (, b) such tht f(p) = 0. for j = 1, 2,... do p j = ( + b)/2 if f(p j ) = 0 or b is sufficiently smll then p = p j return p end if f()f(p j ) < 0 then b = p j else = p j end end At the beginning, it is known tht (, b) contins solution. During ech itertion, this lgorithm updtes the intervl (, b) by checking whether f chnges sign in the first hlf (, p j ), or in the second hlf (p j, b). Once the correct hlf is found, the intervl (, b) is set equl to tht hlf. Therefore, t the beginning of ech itertion, it is known tht the current intervl (, b) contins solution. The test f()f(p j ) < 0 is used to determine whether f chnges sign in the intervl (, p j ) or (p j, b). This test is more efficient thn checking whether f() is positive nd f(p j ) is negtive, or vice vers, since we do not cre which vlue is positive nd which is negtive. We only cre whether they hve different signs, nd if they do, then their product must be negtive. In comprison to other methods, including some tht we will discuss, bisection tends to converge rther slowly, but it is lso gurnteed to converge. These qulities cn be seen in the following result concerning the ccurcy of bisection. Theorem Let f be continuous on [, b], nd ssume tht f()f(b) < 0. For ech positive integer n, let p n be the nth iterte tht is produced by the bisection lgorithm. Then the sequence {p n } n=1 converges to number p in
30 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION (, b) such tht f(p) = 0, nd ech iterte p n stisfies p n p b 2 n. It should be noted tht becuse the nth iterte cn lie nywhere within the intervl (, b) tht is used during the nth itertion, it is possible tht the error bound given by this theorem my be quite conservtive. Exmple We seek solution of the eqution f(x) = 0, where f(x) = x 2 x 1. Becuse f(1) = 1 nd f(2) = 1, nd f is continuous, we cn use the Intermedite Vlue Theorem to conclude tht f(x) = 0 hs solution in the intervl (1, 2), since f(x) must ssume every vlue between 1 nd 1 in this intervl. We use the method of bisection to find solution. First, we compute the midpoint of the intervl, which is (1 + 2)/2 = 1.5. Since f(1.5) = 0.25, we see tht f(x) chnges sign between x = 1.5 nd x = 2, so we cn pply the Intermedite Vlue Theorem gin to conclude tht f(x) = 0 hs solution in the intervl (1.5, 2). Continuing this process, we compute the midpoint of the intervl (1.5, 2), which is (1.5+2)/2 = 1.75. Since f(1.75) = 0.3125, we see tht f(x) chnges sign between x = 1.5 nd x = 1.75, so we conclude tht there is solution in the intervl (1.5, 1.75). The following tble shows the outcome of severl more itertions of this procedure. Ech row shows the current intervl (, b) in which we know tht solution exists, s well s the midpoint of the intervl, given by ( + b)/2, nd the vlue of f t the midpoint. Note tht from itertion to itertion, only one of or b chnges, nd the endpoint tht chnges is lwys set equl to the midpoint. b m = ( + b)/2 f(m) 1 2 1.5 0.25 1.5 2 1.75 0.3125 1.5 1.75 1.625 0.015625 1.5 1.625 1.5625 0.12109 1.5625 1.625 1.59375 0.053711 1.59375 1.625 1.609375 0.019287 1.609375 1.625 1.6171875 0.0018921 1.6171875 1.625 1.62109325 0.0068512 1.6171875 1.62109325 1.619140625 0.0024757 1.6171875 1.619140625 1.6181640625 0.00029087
1.8. SAFEGUARDED METHODS 31 The correct solution, to ten deciml plces, is 1.6180339887, which is the number known s the golden rtio. For this method, it is esier to determine the order of convergence if we use different mesure of the error in ech iterte x k. Since ech iterte is contined within n intervl [ k, b k ] where b k k = 2 k (b ), with [, b] being the originl intervl, it follows tht we cn bound the error x k x by e k = b k k. Using this mesure, we cn esily conclude tht bisection converges linerly, with symptotic error constnt 1/2. 1.8 Sfegurded Methods It is nturl to sk whether it is possible to combine the rpid convergence of methods such s Newton s method with sfe methods such s bisection tht re gurnteed to converge. This leds to the concept of sfegurded methods, which mintin n intervl within which solution is known to exist, s in bisection, but use method such s Newton s method to find solution within tht intervl. If n iterte flls outside this intervl, the sfe procedure is used to refine the intervl before trying the rpid method. An exmple of sfegurded method is the method of Regul Flsi, which is lso known s the method of flse position. It is modifiction of the secnt method in which the two initil itertes x 0 nd x 1 re chosen so tht f(x 0 ) f(x 1 ) < 0, thus gurnteeing tht solution lies between x 0 nd x 1. This condition lso gurntees tht the next iterte x 2 will lie between x 0 nd x 1, s cn be seen by pplying the Intermedite Vlue Theorem to the secnt line pssing through (x 0, f(x 0 )) nd (x 1, f(x 1 )). It follows tht if f(x 2 ) 0, then solution must lie between x 0 nd x 2, or between x 1 nd x 2. In the first scenrio, we use the secnt line pssing through (x 0, f(x 0 )) nd (x 2, f(x 2 )) to compute the next iterte x 3. Otherwise, we use the secnt line pssing through (x 1, f(x 1 )) nd (x 2, f(x 2 )). Continuing in this fshion, we obtin sequence of smller nd smller intervls tht re gurnteed to contin solution, s in bisection, but intervl is updted using superlinerly convergent method, the secnt method, rther thn simply being bisected. Algorithm (Method of Regul Flsi) Let f : R R be continuous function tht chnges sign on the intervl (, b). The following lgorithm computes n pproximte solution x to the eqution f(x) = 0. repet
32 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION c = f(b) bf() f(b) f() if f(c) = 0 or b is sufficiently smll then x = c return x end if f() f(c) < 0 then b = c else = c end end Exmple We use the Method of Regul Flsi (Flse Position) to solve f(x) = 0 where f(x) = x 2 2. First, we must choose two initil guesses x 0 nd x 1 such tht f(x) chnges sign between x 0 nd x 1. Choosing x 0 = 1 nd x 1 = 1.5, we see tht f(x 0 ) = f(1) = 1 nd f(x 1 ) = f(1.5) = 0.25, so these choices re suitble. Next, we use the Secnt Method to compute the next iterte x 2 by determining the point t which the secnt line pssing through the points (x 0, f(x 0 )) nd (x 1, f(x 1 )) intersects the line y = 0. We hve x 2 = x 0 f(x 0)(x 1 x 0 ) f(x 1 ) f(x 0 ) ( 1)(1.5 1) = 1 0.25 ( 1) = 1 + 1.5 1 0.25 + 1 = 1 + 0.5 1.25 = 1.4. Computing f(x 2 ), we obtin f(1.4) = 0.04 < 0. Since f(x 2 ) < 0 nd f(x 1 ) > 0, we cn use the Intermedite Vlue Theorem to conclude tht solution exists in the intervl (x 2, x 1 ). Therefore, we compute x 3 by determining where the secnt line through the points (x 1, f(x 1 )) nd f(x 2, f(x 2 )) intersects the line y = 0. Using the formul for the Secnt Method, we obtin x 3 = x 1 f(x 1)(x 2 x 1 ) f(x 2 ) f(x 1 )
1.8. SAFEGUARDED METHODS 33 (0.25)(1.4 1.5) = 1.5 0.04 0.25 = 1.41379. Since f(x 3 ) < 0 nd f(x 2 ) < 0, we do not know tht solution exists in the intervl (x 2, x 3 ). However, we do know tht solution exists in the intervl (x 3, x 1 ), becuse f(x 1 ) > 0. Therefore, insted of proceeding s in the Secnt Method nd using the Secnt line determined by x 2 nd x 3 to compute x 4, we use the secnt line determined by x 1 nd x 3 to compute x 4.
34 CHAPTER 1. SOLUTION OF EQUATIONS BY ITERATION k x k g(x k ) 0 0.50000000000000 0.87758256189037 1 0.87758256189037 0.63901249416526 2 0.63901249416526 0.80268510068233 3 0.80268510068233 0.69477802678801 4 0.69477802678801 0.76819583128202 5 0.76819583128202 0.71916544594242 6 0.71916544594242 0.75235575942153 7 0.75235575942153 0.73008106313782 8 0.73008106313782 0.74512034135144 9 0.74512034135144 0.73500630901484 10 0.73500630901484 0.74182652264325 11 0.74182652264325 0.73723572544223 12 0.73723572544223 0.74032965187826 13 0.74032965187826 0.73824623833223 14 0.73824623833223 0.73964996276966 15 0.73964996276966 0.73870453935698 16 0.73870453935698 0.73934145228121 17 0.73934145228121 0.73891244933210 18 0.73891244933210 0.73920144413580 19 0.73920144413580 0.73900677978081 20 0.73900677978081 0.73913791076229 21 0.73913791076229 0.73904958059521 22 0.73904958059521 0.73910908142053 23 0.73910908142053 0.73906900120401 24 0.73906900120401 0.73909599983575 25 0.73909599983575 0.73907781328518 26 0.73907781328518 0.73909006398825 27 0.73909006398825 0.73908181177811 28 0.73908181177811 0.73908737057104 29 0.73908737057104 0.73908362610348 30 0.73908362610348 0.73908614842288
Chpter 2 Polynomil Interpoltion 2.1 Lgrnge Interpoltion Clculus provides mny tools tht cn be used to understnd the behvior of functions, but in most cses it is necessry for these functions to be continuous or differentible. This presents problem in most rel pplictions, in which functions re used to model reltionships between quntities, but our only knowledge of these functions consists of set of discrete dt points, where the dt is obtined from mesurements. Therefore, we need to be ble to construct continuous functions bsed on discrete dt. The problem of constructing such continuous function is clled dt fitting. In this lecture, we discuss specil cse of dt fitting known s interpoltion, in which the gol is to find liner combintion of n known functions to fit set of dt tht imposes n constrints, thus gurnteeing unique solution tht fits the dt exctly, rther thn pproximtely. The broder term constrints is used, rther thn simply dt points, since the description of the dt my include dditionl informtion such s rtes of chnge or requirements tht the fitting function hve certin number of continuous derivtives. When it comes to the study of functions using clculus, polynomils re prticulrly simple to work with. Therefore, in this course we will focus on the problem of constructing polynomil tht, in some sense, fits given dt. We first discuss some lgorithms for computing the unique polynomil p n (x) of degree n tht stisfies p n (x i ) = y i, i = 0,..., n, where the points (x i, y i ) re given. The points x 0, x 1,..., x n re clled interpoltion points. The polynomil p n (x) is clled the interpolting polynomil of the dt (x 0, y 0 ), (x 1, y 1 ),..., (x n, y n ). At first, we will ssume tht the interpoltion points 35
36 CHAPTER 2. POLYNOMIAL INTERPOLATION re ll distinct; this ssumption will be relxed in lter lecture. If the interpoltion points x 0,..., x n re distinct, then the process of finding polynomil tht psses through the points (x i, y i ), i = 0,..., n, is equivlent to solving system of liner equtions Ax = b tht hs unique solution. However, different lgorithms for computing the interpolting polynomil use different A, since they ech use different bsis for the spce of polynomils of degree n. The most strightforwrd method of computing the interpoltion polynomil is to form the system Ax = b where b i = y i, i = 0,..., n, nd the entries of A re defined by ij = p j (x i ), i, j = 0,..., n, where x 0, x 1,..., x n re the points t which the dt y 0, y 1,..., y n re obtined, nd p j (x) = x j, j = 0, 1,..., n. The bsis {1, x,..., x n } of the spce of polynomils of degree n + 1 is clled the monomil bsis, nd the corresponding mtrix A is clled the Vndermonde mtrix for the points x 0, x 1,..., x n. Unfortuntely, this mtrix cn be ill-conditioned, especilly when interpoltion points re close together. In Lgrnge interpoltion, the mtrix A is simply the identity mtrix, by virtue of the fct tht the interpolting polynomil is written in the form p n (x) = n y j L n,j (x), j=0 where the polynomils {L n,j } n j=0 hve the property tht L n,j (x i ) = { 1 if i = j 0 if i j. The polynomils {L n,j }, j = 0,..., n, re clled the Lgrnge polynomils for the interpoltion points x 0, x 1,..., x n. They re defined by L n,j (x) = n k=0,k j x x k x j x k. As the following result indictes, the problem of polynomil interpoltion cn be solved using Lgrnge polynomils. Theorem Let x 0, x 1,..., x n be n + 1 distinct numbers, nd let f(x) be function defined on domin contining these numbers. Then the polynomil defined by n p n (x) = f(x j )L n,j j=0
2.1. LAGRANGE INTERPOLATION 37 is the unique polynomil of degree n tht stisfies p n (x j ) = f(x j ), j = 0, 1,..., n. The polynomil p n (x) is clled the interpolting polynomil of f(x). We sy tht p n (x) interpoltes f(x) t the points x 0, x 1,..., x n. Exmple We will use Lgrnge interpoltion to find the unique polynomil p 3 (x), of degree 3 or less, tht grees with the following dt: i x i y i 0 1 3 1 0 4 2 1 5 3 2 6 In other words, we must hve p 3 ( 1) = 3, p 3 (0) = 4, p 3 (1) = 5, nd p 3 (2) = 6. First, we construct the Lgrnge polynomils {L 3,j (x)} 3 j=0, using the formul 3 (x x i ) L n,j (x) = (x j x i ). This yields L 3,0 (x) = = L 3,1 (x) = i=0,i j (x x 1 )(x x 2 )(x x 3 ) (x 0 x 1 )(x 0 x 2 )(x 0 x 3 ) (x 0)(x 1)(x 2) ( 1 0)( 1 1)( 1 2) = x(x2 3x + 2) ( 1)( 2)( 3) = 1 6 (x3 3x 2 + 2x) = (x x 0 )(x x 2 )(x x 3 ) (x 1 x 0 )(x 1 x 2 )(x 1 x 3 ) (x + 1)(x 1)(x 2) (0 + 1)(0 1)(0 2) = (x2 1)(x 2) (1)( 1)( 2) = 1 2 (x3 2x 2 x + 2)