1 Ordinary Differential Equations

Transcription

1 1 Ordinry Differentil Equtions 1. Mthemticl Bckground 1..1 Smoothness Definition 1.1 A function f defined on [, b] is continuous t x [, b] if lim x x f(x) = f(x ). Remrk Note tht this implies existence of the quntities on both sides of the eqution. f is continuous on [, b], i.e., f C[, b], if f is continuous t every x [, b]. If f (n) is continuous on [, b], then f C (n) [, b]. Alterntively, one could strt with the following ε-δ definition: Definition 1.2 A function f defined on [, b] is continuous t x [, b] if for every ε > there exists δ ε > (tht depends on x ) such tht f(x) f(x ) < ε whenever x x < δ ε. FIGURE Exmple (A function tht is continuous, but not uniformly continuous) f(x) = 1 x with FIGURE. Definition 1.3 A function f is uniformly continuous on [, b] if it is continuous with uniform δ ε for ll x [, b], i.e., independent of x. Importnt for ordinry differentil equtions is Definition 1.4 A function f defined on [, b] is Lipschitz continuous on [, b] if there exists number λ such tht f(x) f(y) λ x y for ll x, y [, b]. λ is clled the Lipschitz constnt. Remrk 1. In fct, ny Lipschitz continuous function is uniformly continuous, nd therefore continuous. 2. For differentible function with bounded derivtive we cn tke λ = mx x [,b] f (x), nd we see tht Lipschitz continuity is between continuity nd differentibility. 3. If the function f is Lipschitz continuous on [, b] with Lipschitz constnt λ, then f is lmost everywhere differentible in [, b] with f (x) λ. In other words, Lipschitz continuous functions need not be differentible everywhere in [, b]. 4. See lso Assignment 1. 1

2 1..2 Polynomils A rel polynomil of degree t most n is of the form p(x) = n j x j, x R, j= with coefficients j R. Nottion: We denote the spce of (univrite) polynomils of degree t most m by p P n. Theorem 1.5 (Tylor s Theorem) If f C n [, b] nd f (n+1) exists on (, b), then for ny x [, b] f(x) = n 1 k! f (k) (x )(x x ) k 1 + (n + 1)! f (n+1) (ξ x )(x x ) n+1, k= }{{}}{{} Tylor polynomil error term where ξ x is some point between x nd x. FIGURE An lternte form commonly used is f(x + h) = n k= 1 k! f (k) (x )h k 1 + (n + 1)! f (n+1) (ξ x )h n+1. Remrk Note tht the informtion bout f is provided loclly, t x only. If the informtion is spred out, i.e., when we re given distinct points x < x 1 <... < x n [, b] nd ssocited vlues f(x ), f(x 1 ),..., f(x n ), then we will see below tht there exists unique interpoltion polynomil p n of degree t most n with l j (x) = p n (x) = n k= k j n l j (x)f(x j ) j= x x k x j x k, j =, 1,..., n, such tht p n (x i ) = f(x i ), i =, 1,..., n. The l j, j =, 1,..., n, re clled Lgrnge functions, nd the polynomil p n is sid to be in Lgrnge form. Remrk See Assignment 1 for n illustrtion of Tylor polynomils vs. interpoltion polynomils. 2

3 1.1 Polynomil Interpoltion In the following two subsections we will discuss the problem of fitting dt given in the form of discrete points (e.g., physicl mesurements, output from differentil equtions solver, design points for CAD, etc.) with n pproprite function s tken from some (finite-dimensionl) function spce S of our choice. Throughout most of our discussion the dt will be univrite, i.e., of the form (x i, y i ), i =,..., n, where the x i R re referred to s dt sites (or nodes), nd the y i R s vlues (or ordintes). Often we will ssume y i = f(x i ) for some (unknown) function f. Lter on we will lso briefly consider multivrite dt where x i R d, d > 1. The dt cn be fitted either by interpoltion, i.e., by stisfying or by pproximtion, i.e., by stisfying s(x i ) = y i, i =,..., n, (1) s y < ɛ, where s nd y hve to be considered s vectors of function or dt vlues, nd is some discrete norm on R n+1. We will focus our ttention on interpoltion. If {b,..., b n } is some bsis of the function spce S, then we cn express s s liner combintion in the form s(x) = n j b j (x). j= If we re given m + 1 dt points (x i, y i ), i =, 1,..., m, then the interpoltion conditions (1) led to n j b j (x i ) = y i, i =,..., m. j= This represents system of liner lgebric equtions for the expnsion coefficients j of the form B = y, where the m n system mtrix B hs entries b j (x i ). We re especilly interested in those function spces nd bses for which the cse m = n yields unique solution. Function spces studied in this clss include polynomils, piecewise polynomils, trigonometric polynomils, nd rdil bsis functions. We will begin by studying polynomils. There re severl motivting fctors for doing this: Everyone is fmilir with polynomils. Polynomils cn be esily nd efficiently evluted using Horner s lgorithm (i.e., nested multipliction). 3

4 We my hve herd of the Weierstrss Approximtion Theorem which sttes tht ny continuous function cn be pproximted rbitrrily closely by polynomil (of sufficiently high degree). A lot is known bout polynomil interpoltion, nd serves s strting point for other methods. Formlly, we re interested in solving the following Problem 1.6 Given dt (x i, y i ), i =,..., n, find polynomil p of miniml degree which mtches the dt in the sense of (1), i.e., for which p(x i ) = y i, i =,..., n. We illustrte this problem with some numericl exmples in the Mple worksheet PolynomilInterpoltion.mws. The numericl experiments suggest tht n + 1 dt points cn be interpolted by polynomil of degree n. Indeed, Theorem 1.7 Let n + 1 distinct rel numbers x, x 1,..., x n nd ssocited vlues y, y 1,..., y n be given. Then there exists unique polynomil p n of degree t most n such tht p n (x i ) = y i, i =, 1,..., n. Proof Assume p n (x) = + 1 x + 2 x n x n. Then the interpoltion conditions (1) led to the liner system B = y with 1 x x 2... x n 1 x 1 x x n 1 B = 1 x 2 x x n 2, = x n x 2 n... x n n 1 2. n nd y = For generl dt y this is nonhomogeneous system, nd we know tht such system hs unique solution if nd only if the ssocited homogeneous system hs only the trivil solution. The homogeneous system corresponds to p n (x i ) =, i =, 1,..., n. Thus, p n hs n + 1 zeros. Now, the Fundmentl Theorem of Algebr sttes tht ny nontrivil polynomil of degree n hs n (possibly complex) zeros. Therefore p n must be the zero polynomil, i.e., the homogeneous liner system hs only the trivil solution = 1 =... = n =, nd by the bove comment the generl nonhomogeneous problem hs unique solution. Remrk The mtrix B in the proof of Theorem 1.7 is referred to s Vndermonde mtrix. y y 1 y 2... y n. 4

5 Next, we illustrte how such (low-degree) interpolting polynomil cn be determined. Exmple Let s ssume we hve the following dt: x 1 3 y 1 4. Now, since we wnt squre liner system, we pick n pproximtion spce of dimension three, i.e., with three bsis functions. This mens we use qudrtic polynomils, nd s bsis we tke the monomils b (x) = 1, b 1 (x) = x, nd b 2 (x) = x 2. Therefore, the interpolting polynomil will be of the form p 2 (x) = + 1 x + 2 x 2. In order to determine the coefficients, 1, nd 2 we enforce the interpoltion conditions (1), i.e., p 2 (x i ) = y i, i =, 1, 2. This leds to the liner system (p 2 () =) = 1 (p 2 (1) =) = (p 2 (3) =) = 4 whose solution is esily verified to be = 1, 1 = 2, nd 2 = 1. Thus, p 2 (x) = 1 2x + x 2. Of course, this polynomil cn lso be written s p 2 (x) = (1 x) 2. So, hd we chosen the bsis of shifted monomils b (x) = 1, b 1 (x) = 1 x, nd b 2 (x) = (1 x) 2, then the coefficients (for n expnsion with respect to this bsis) would hve come out to be =, 1 =, nd 2 = 1. In generl, use of the monomil bsis leds to Vndermonde system s listed in the proof of the theorem bove. This is clssicl exmple of n ill-conditioned system, nd thus should be voided. We will look t other bses lter. We now provide second (constructive) proof of Theorem 1.7. Constructive Proof: First we estblish uniqueness. To this end we ssume tht p n nd q n both re n-th degree interpolting polynomils. Then r n (x) = p n (x) q n (x) is lso n n-th degree polynomil. Moreover, by the Fundmentl Theorem of Algebr r n hs n zeros (or is the zero polynomil). However, by the nture of p n nd q n we hve r n (x i ) = p n (x i ) q n (x i ) = y i y i =, i =, 1,..., n. Thus, r n hs n + 1 zeros, nd therefore must be identiclly equl to zero. This ensures uniqueness. 5

6 Existence is constructed by induction. For n = we tke p y. Obviously, the degree of p is less thn or equl to, nd lso p (x ) = y. Now we ssume p k 1 to be the unique polynomil of degree t most k 1 tht interpoltes the dt (x i, y i ), i =, 1,..., k 1. We will construct p k (of degree k) such tht p k (x i ) = y i, i =, 1,..., k. We let p k (x) = p k 1 (x) + c k (x x )(x x 1 )... (x x k 1 ) with c k yet to be determined. By construction, p k is polynomil of degree k which interpoltes the dt (x i, y i ), i =, 1,..., k 1. We now determine c k so tht we lso hve p k (x k ) = y k. Thus, or (p k (x k ) =) p k 1 (x k ) + c k (x k x )(x k x 1 )... (x k x k 1 ) = y k c k = y k p k 1 (x k ) (x k x )(x k x 1 )... (x k x k 1 ). This is well defined since the denomintor is nonzero due to the fct tht we ssume distinct dt sites. The construction used in this lternte proof provides the strting point for the Newton form of the interpolting polynomil. From the proof we hve p k (x) = p k 1 (x) + c k (x x )(x x 1 )... (x x k 1 ) = p k 2 (x) + c k 1 (x x )(x x 1 )... (x x k 2 ) + c k (x x )(x x 1 )... (x x k 1 ). = c + c 1 (x x ) + c 2 (x x )(x x 1 ) c k (x x )(x x 1 )... (x x k 1 ). Thus, the Newton form of the interpolting polynomil is given by p n (x) = n j= j 1 c j i= (x x i ). (2) This nottion implies tht the empty product (when j 1 < ) is equl to 1. The proof bove lso provides formul for the coefficients in the Newton form: c j = y j p j 1 (x j ) (x j x )(x j x 1 )... (x j x j 1 ), p c = y. (3) So the Newton coefficients cn be computed recursively. This leds to first lgorithm for the solution of the interpoltion problem. Algorithm Input n, x, x 1,..., x n, y, y 1,..., y n c = y for k = 1 : n 6

7 d = x k x k 1 end u = c k 1 for i = k 2 : 1 : end % build p k 1 u = u(x k x i ) + c i % Horner d = d(x k x i ) % ccumulte denomintor c k = y k u d Output c, c 1,..., c n Remrk A more detiled derivtion of this lgorithm is provided on pge 31 of the Kincid/Cheney book. However, the stndrd, more efficient, lgorithm for computing the coefficients of the Newton form is bsed on the use of divided differences nd will not be discussed here. Exmple We now compute the Newton form of the polynomil interpolting the dt According to (2) nd (3) we hve with p 2 (x) = 2 j= j 1 c j i= c = y = 1, nd c j = x 1 3 y 1 4. (x x i ) = c + c 1 (x x ) + c 2 (x x )(x x 1 ) y j p j 1 (x j ), j = 1, 2. (x j x )(x j x 1 )... (x j x j 1 ) Thus, we re representing the spce of qudrtic polynomils with the bsis b (x) = 1, b 1 (x) = x x = x, nd b 2 (x) = (x x )(x x 1 ) = x(x 1). We now determine the two remining coefficients. First, c 1 = y 1 p (x 1 ) x 1 x = y 1 y x 1 x = 1 1 = 1. This gives us Next, nd so c 2 = p 1 (x) = c + c 1 (x x ) = 1 x. y 2 p 1 (x 2 ) (x 2 x )(x 2 x 1 ) = 4 (1 x 2) = 1, 3 2 p 2 (x) = p 1 (x) + c 2 (x x )(x x 1 ) = 1 x + x(x 1). 7

8 A third representtion (fter the monomil nd Newton forms) of the sme (unique) interpolting polynomil is of the generl form n p n (x) = y j l j (x). (4) j= Note tht the coefficients here re the dt vlues, nd the polynomil bsis functions l j re so-clled crdinl (or Lgrnge) functions. We wnt these functions to depend on the dt sites x, x 1,..., x n, but not the dt vlues y, y 1,..., y n. In order to stisfy the interpoltion conditions (1) we require p n (x i ) = Clerly, this is ensured if n y j l j (x i ) = y i, i =, 1,..., n. j= l j (x i ) = δ ij, (5) which is clled crdinlity (or Lgrnge) condition. How do we determine the Lgrnge functions l j? We wnt them to be polynomils of degree n nd stisfy the crdinlity conditions (5). Let s fix j, nd ssume l j (x) = c(x x )(x x 1 )... (x x j 1 )(x x j+1 )... (x x n ) n = c (x x i ). i= i j Clerly, l j (x i ) = for j i. Also, l j depends only on the dt sites nd is polynomil of degree n. The lst requirement is l j (x j ) = 1. Thus, or 1 = c c = n (x j x i ) i= i j 1. n (x j x i ) i= i j Agin, the denomintor is nonzero since the dt sites re ssumed to be distinct. Therefore, the Lgrnge functions re given by l j (x) = n i= i j x x i x j x i, j =, 1,..., n, nd the Lgrnge form of the interpolting polynomil is n n x x i p n (x) = y j. (6) x j x i j= i= i j 8

9 Remrk 1. We mentioned bove tht the monomil bsis results in n interpoltion mtrix ( Vndermonde mtrix) tht is notoriously ill-conditioned. However, n dvntge of the monomil bsis representtion is the fct tht the interpolnt cn be evluted efficiently using Horner s method. 2. The interpoltion mtrix for the Lgrnge form is the identity mtrix nd the coefficients in the bsis expnsion re given by the dt vlues. This mkes the Lgrnge form idel for situtions in which mny experiments with the sme dt sites, but different dt vlues need to be performed. However, evlution (s well s differentition or integrtion) is more expensive. 3. A mjor dvntge of the Newton form is its efficiency in the cse of dptive interpoltion, i.e., when n existing interpolnt needs to be refined by dding more dt. Due to the recursive nture, the new interpolnt cn be determined by updting the existing one (s we did in the exmple bove). If we were to form the interpoltion mtrix for the Newton form we would get tringulr mtrix. Therefore, for the Newton form we hve blnce between stbility of computtion nd ese of evlution. Exmple Returning to our erlier exmple, we now compute the Lgrnge form of the polynomil interpolting the dt According to (4) we hve where (see (6)) Thus, nd This gives us p 2 (x) = x 1 3 y y j l j (x) = l (x) + 4l 2 (x), j= l j (x) = 2 i= i j x x i x j x i. l (x) = (x x 1)(x x 2 ) (x 1)(x 3) = = 1 (x 1)(x 3), (x x 1 )(x x 2 ) ( 1)( 3) 3 l 2 (x) = (x x )(x x 1 ) (x 2 x )(x 2 x 1 ) = x(x 1) (3 )(3 1) = 1 x(x 1). 6 p 2 (x) = 1 3 (x 1)(x 3) + 2 x(x 1). 3 Note tht here we hve represented the spce of qudrtic polynomils with the (crdinl) bsis b (x) = 1 3 (x 1)(x 3), b 2(x) = 1 6 x(x 1), nd b 1(x) = l 1 (x) = 1 2x(x 3) (which we did not need to compute for this exmple since its coefficient y 1 = ). 9

10 Summrizing our exmple, we hve found four different forms of the qudrtic polynomil interpolting our dt: monomil p 2 (x) = 1 2x + x 2 shifted monomil p 2 (x) = (1 x) 2 Newton form p 2 (x) = 1 x + x(x 1) Lgrnge form p 2 (x) = 1 3 (x 1)(x 3) + 2 3x(x 1) The bsis polynomils for these four cses re displyed in Figures 1 nd x x Figure 1: Monomil (left) nd shifted monomil (right) bses for the interpoltion exmple x x Figure 2: Bsis polynomils for the Newton (left) nd Lgrnge form (right) Error in Polynomil Interpoltion In the introduction we mentioned the Weierstrss Approximtion Theorem s one of the motivtions for the use of polynomils. Here re the detils. Theorem 1.8 Let f C[, b] nd ɛ >. Then there exists polynomil p of sufficiently high degree such tht f(x) p(x) < ɛ for ll x [, b]. 1

11 We will not prove this theorem. A proof cn, e.g., be found in the book by Kincid nd Cheney. Now we only point out tht s nice s this theorem is it does not cover interpoltion of (vlues of) f by p. For this problem we hve Theorem 1.9 Let f C n+1 [, b] nd p be polynomil of degree t most n which interpoltes f t the n + 1 distinct points x, x 1,..., x n in [, b] (i.e., p(x i ) = f(x i ), i =, 1,..., n). Then, for ech x [, b] there exists number ξ x (, b) such tht f(x) p(x) = 1 (n + 1)! f (n+1) (ξ x ) In order to prove this result we need to recll Rolle s Theorem: n (x x i ). (7) Theorem 1.1 If f C[, b] nd f exists on (, b), nd if f() = f(b) =, then there exists number ξ (, b) such tht f (ξ) =. Proof (of Theorem 1.9) If x coincides with one of the dt sites x i, i =, 1,..., n, then it is esy to see tht both sides of eqution (7) re zero. Thus we now ssume x x i to be fixed. We strt be defining i= nd with α such tht F (x) =, i.e., n w(t) = (t x i ) i= F = f p αw α = f(x) p(x). w(x) We need to show tht α = 1 (n+1)! f (n+1) (ξ x ) for some ξ x (, b). Since f C n+1 [, b] we know tht F C n+1 [, b] lso. Moreover, F (t) = for t = x, x, x 1,..., x n. The first of these equtions holds by the definition of α, the reminder by the definition of w nd the fct tht p interpoltes f t these points. Now we pply Rolle s Theorem to F on ech of the n + 1 subintervls generted by the n + 2 points x, x, x 1,..., x n. Thus, F hs (t lest) n + 1 distinct zeros in (, b). Next, by Rolle s Theorem (pplied to F on n subintervls) we know tht F hs (t lest) n zeros in (, b). Continuing this rgument we deduce tht F (n+1) hs (t lest) one zero, ξ x, in (, b). On the other hnd, since F (t) = f(t) p(t) αw(t) we hve F (n+1) (t) = f (n+1) (t) p (n+1) (t) αw (n+1) (t). 11

12 However, p is polynomil of degree t most n, so p (n+1). coefficient of the (n + 1)-degree polynomil w is 1 we hve Since the leding Therefore, w (n+1) (t) = dn+1 dt n+1 n (t x i ) = (n + 1)!. i= F (n+1) (t) = f (n+1) (t) α(n + 1)!. Combining this with the informtion bout the zero of F (n+1) we hve or = F (n+1) (ξ x ) = f (n+1) (ξ x ) α(n + 1)! = f (n+1) f(x) p(x) (ξ x ) (n + 1)! w(x) f(x) p(x) = f (n+1) (ξ x ) w(x) (n + 1)!. Remrk 1. The error formul (7) in Theorem 1.9 looks lmost like the error formul for Tylor polynomils from Theorem 1.5: f(x) T n (x) = f (n+1) (ξ x ) (x x ) n+1. (n + 1)! The difference lies in the fct tht for Tylor polynomils the informtion is concentrted t one point x, wheres for interpoltion the informtion is obtined t the points x, x 1,..., x n. 2. The error formul (7) will be used lter to derive (nd judge the ccurcy of) methods for solving differentil equtions s well s for numericl integrtion. If the dt sites re eqully spced, i.e., x i = x i x i 1 = h, i = 1, 2,..., n, it is possible to derive the bound n x x i 1 4 hn+1 n! i= in the error formul (7). If we lso ssume tht the derivtives of f re bounded, i.e., f (n+1) (x) M for ll x [, b], then we get f(x) p(x) M 4(n + 1) hn+1. Thus, the error for interpoltion with degree-n polynomils is O(h n+1 ). We illustrte this formul for fixed-degree polynomil in the Mple worksheet PolynomilInterpoltionError.mws. Formul (7) tells us tht the interpoltion error depends on the function we re interpolting s well s the dt sites (interpoltion nodes) we use. If we re interested 12

13 in minimizing the interpoltion error, then we need to choose the dt sites x i, i =, 1,..., n, such tht n w(t) = (t x i ) i= is minimized. Figure 3 shows the grph of the function w for 1 eqully spced (red,dshed) nd 1 optimlly spced (green,solid) dt sites on the intervl [ 1, 1]. We will derive the loctions of the optimlly spced points (known s Chebyshev points) below Figure 3: Grph of the function w for 1 eqully spced (red,dshed) nd 1 Chebyshev points (green,solid) on [ 1, 1]. Figure 3 shows tht the error ner the endpoints of the intervl cn be significnt if we insist on using eqully spced points. A clssicl exmple tht illustrtes this phenomenon is the Runge function f(x) = x 2. We present this exmple in the Mple worksheet Runge.mws. In order to discuss Chebyshev points we need to introduce certin fmily of orthogonl polynomils clled Chebyshev polynomils Chebyshev Polynomils The Chebyshev polynomils (of the first kind) cn be defined recursively. We hve nd T (x) = 1, T 1 (x) = x, x T n+1 (x) = 2xT n (x) T n 1 (x), n 1. (8) An explicit formul for the n-th degree Chebyshev polynomil is T n (x) = cos(n rccos x), x [ 1, 1], n =, 1, 2,.... (9) 13

14 We cn verify this explicit formul by using the trigonometric identity cos(a + B) = cos A cos B sin A sin B. This identity gives rise to the following two formuls: Addition of these two formuls yields cos(n + 1)θ = cos nθ cos θ sin nθ sin θ cos(n 1)θ = cos nθ cos θ + sin nθ sin θ. cos(n + 1)θ + cos(n 1)θ = 2 cos nθ cos θ. (1) Now, if we let x = cos θ (or θ = rccos x) then (1) becomes or cos [(n + 1) rccos x] + cos [(n 1) rccos x] = 2 cos (n rccos x) cos (rccos x) cos [(n + 1) rccos x] = 2x cos (n rccos x) cos [(n 1) rccos x], which is of the sme form s the recursion (8) for the Chebyshev polynomils provided we identify T n with the explicit formul (9). Some properties of Chebyshev polynomils re 1. T n (x) 1, x [ 1, 1]. ( 2. T n cos iπ ) = ( 1) i, n i =, 1,..., n. These re the extrem of T n. ( 3. T n cos 2i 1 ) 2n π =, i = 1,..., n. This gives the zeros of T n. 4. The leding coefficient of T n, n = 1, 2,..., is 2 n 1, i.e., T n (x) = 2 n 1 x n + lower order terms. Items 1 3 follow immeditely from (9). Item 4 is cler from the recursion formul (8). Therefore, 2 1 n T n is monic polynomil, i.e., its leding coefficient is 1. We will need the following property of monic polynomils below. Theorem 1.11 For ny monic polynomil p of degree n on [ 1, 1] we hve p = mx p(x) 1 x 1 21 n. Proof A proof cn be found on p. 317 of the Kincid nd Cheney book. We re now redy to return to the formul for the interpoltion error. From (7) we get on [ 1, 1] f p = mx f(x) p(x) 1 x 1 14

15 1 (n + 1)! mx 1 x 1 f (n+1) (x) mx 1 x 1 i= n (x x i ). } {{ } =w(x) We note tht w is monic polynomil of degree n + 1 with zeros x i, i =, 1,..., n, nd therefore mx 1 x 1 w(x) 2 n. From bove we know tht ) 2 n T n+1 is lso monic polynomil of degree n + 1 with iπ extrem T n+1 (cos = ( 1)i, i =, 1,..., n + 1. By Theorem 1.11 the miniml n + 1 2n vlue of mx w(x) is 1 x 1 2 n. We just observed tht this vlue is ttined for T n+1. Thus, the zeros x i of the optiml w should coincide with the zeros of T n+1, or ( ) 2i + 1 x i = cos 2n + 2 π, i =, 1,..., n. (11) These re the Chebyshev points used in Figure 3 nd in the Mple worksheet Runge.mws. If the dt sites re tken to be the Chebyshev points on [ 1, 1], then the interpoltion error (7) becomes f(x) p(x) 1 2 n (n + 1)! mx 1 t 1 f (n+1) (t), x 1. Remrk The Chebyshev points (11) re for interpoltion on [ 1, 1]. If different intervl is used, the Chebyshev points need to be trnsformed ccordingly. We just observed tht the Chebyshev nodes re optiml interpoltion points in the sense tht for ny given f nd fixed degree n of the interpolting polynomil, if we re free to choose the dt sites, then the Chebyshev points will yield the most ccurte pproximtion (mesured in the mximum norm). For eqully spced interpoltion points, our numericl exmples in the Mple worksheets PolynomilInterpoltion.mws nd Runge.mws hve shown tht, contrry to our intuition, the interpoltion error (mesured in the mximum norm) does not tend to zero s we increse the number of interpoltion points (or polynomil degree). The sitution is even worse. There is lso the following more generl result proved by Fber in Theorem 1.12 For ny fixed system of interpoltion nodes x (n) < x (n) 1 <... < x (n) n b there exists function f C[, b] such tht the interpolting polynomil p n does not uniformly converge to f, i.e., f p n, n. This, however, needs to be contrsted with the positive result (very much in the spirit of the Weierstrss Approximtion Theorem) for the sitution in which we re free to choose the loction of the interpoltion points. 15

16 Theorem 1.13 Let f C[, b]. Then there exists system of interpoltion nodes such tht f p n, n. Proof The proof uses the Weierstrss Approximtion Theorem s well s the Chebyshev Alterntion Theorem. (cf. Kincid nd Cheney). Finlly, if we insist on using the Chebyshev points s dt sites, then we hve the following theorem due to Fejér. Theorem 1.14 Let f C[ 1, 1], nd x, x 1,..., x n 1 be the first n Chebyshev points. Then there exists polynomil p 2n 1 of degree 2n 1 tht interpoltes f t x, x 1,..., x n 1, nd for which f p 2n 1, n. Remrk The polynomil p 2n 1 lso hs zero derivtives t x i, i =, 1,..., n Spline Interpoltion One of the min disdvntges ssocited with polynomil interpoltion were the oscilltions resulting from the use of high-degree polynomils. If we wnt to mintin such dvntges s simplicity, ese nd speed of evlution, s well s similr pproximtion properties, we re nturlly led to consider piecewise polynomil interpoltion or spline interpoltion. Definition 1.15 A spline function S of degree k is function such tht ) S is defined on n intervl [, b], b) S C k 1 [, b], c) there re points = t < t 1 <... < t n = b (clled knots) such tht S is polynomil of degree t most k on ech subintervl [t i, t i+1 ). Exmple The most commonly used spline function is the piecewise liner (k = 1) spline, i.e., given knot sequence s in Definition 1.15, S is liner function on ech subintervl with continuous joints t the knots. If we re given the dt x y 1-1 3, then we cn let the knot sequence coincide with the dt sites, i.e., t i = i, i =, 1, 2, 3. The piecewise liner spline interpolting the dt is then given by the connect-thedots pproch, or in formuls 1 x x < 1, S(x) = 1 x 1 x < 2, 4x 9 2 x < 3. This spline is displyed in Figure 4 16

17 Figure 4: Grph of the liner interpolting spline of Exmple 1. Remrk Definition 1.15 does not mke ny sttement bout the reltion between the loction of the knots nd the dt sites for interpoltion. We just observe tht for liner splines it works to let the knots coincide with the dt sites. We will not discuss the generl problem in this clss. Exmple We use the sme dt s bove, i.e., x y 1-1 3, but now we wnt to interpolte with qudrtic (C 1 ) spline. Agin, we let the knots nd dt sites coincide, i.e., t i = x i, i =, 1, 2, 3. Now it is not so obvious wht S will look like. From Definition 1.15 we know tht S will be of the form S (x) = x 2 + b x + c, x < 1, S(x) = S 1 (x) = 1 x 2 + b 1 x + c 1, 1 x < 2, S 2 (x) = 2 x 2 + b 2 x + c 2, 2 x < 3. Since we hve three qudrtic pieces there re 9 prmeters ( j, b j, c j, j =, 1, 2) to be determined. To this end, we collect the conditions we hve on S. Obviously, we hve the 4 interpoltion conditions S(x i ) = y i, i =, 1, 2, 3. Moreover, S needs to stisfy the C 1 (nd C ) smoothness conditions of Definition 1.15 t the interior knots. Tht leds to four more conditions: S (t 1 ) = S 1 (t 1 ), S 1 (t 2 ) = S 2 (t 2 ), S (t 1 ) = S 1(t 1 ), S 1(t 2 ) = S 2(t 2 ). Thus, we hve totl of 8 conditions to determine the 9 free prmeters. This leves t lest one undetermined prmeter (provided the bove conditions re linerly independent). 17

18 Hoping tht the conditions bove re indeed independent, we dd one more (rbitrry) condition S () = 1 to obtin squre liner system. Therefore, we need to solve S () = 1, S (1) =, S 1 (1) =, S 1 (2) = 1, S 2 (2) = 1, S 2 (3) = 3, S (1) S 1(1) =, S 1(2) S 2(2) =, S () = 1. Note tht we hve implemented the C conditions t the two interior knots by stting the interpoltion conditions for both djoining pieces. In mtrix form, the resulting liner system is The solution of this liner system is given by or b c 1 b 1 c 1 2 b 2 c 2 =, b = 1, c = 1, 1 =, b 1 = 1, c 1 = 1, 2 = 5, b 2 = 21, c 2 = 21, = 1 x, x < 1, S(x) = 1 x, 1 x < 2, 5x 2 21x + 21, 2 x < 3. This exmple is illustrted in the Mple worksheet SplineInterpoltion.mws nd plot of the qudrtic spline computed in Exmple 2 is lso provided in Figure 5. Remrk In order to efficiently evlute piecewise defined spline S t some point x [t, t n ] one needs to be ble to identify which polynomil piece to evlute, i.e., determine in which subintervl [t i, t i+1 ) the evlution point x lies. An lgorithm (for liner splines) is given in on pge 35 of the Kincid/Cheney book

19 x 1 Figure 5: Grph of the qudrtic interpolting spline of Exmple Cubic Splines Another very populr spline is the (C 2 ) cubic spline. Assume we re given n + 1 pieces of dt (x i, y i ), i =, 1,..., n to interpolte. Agin, we let the knot sequence {t i } coincide with the dt sites. According to Definition 1.15 the spline S will consist of n cubic polynomil pieces with totl of 4n prmeters. The conditions prescribed in Definition 1.15 re n + 1 interpoltion conditions, n 1 C continuity conditions t interior knots, n 1 C 1 continuity conditions t interior knots, n 1 C 2 continuity conditions t interior knots, for totl of 4n 2 conditions. Assuming liner independence of these conditions, we will be ble to impose two dditionl conditions on S. There re mny possible wys of doing this. We will discuss: 1. S (t ) = S (t n ) =, (so-clled nturl end conditions). 2. Other boundry conditions, such s which led to complete splines, or S (t ) = f (t ), S (t n ) = f (t n ) S (t ) = f (t ), S (t n ) = f (t n ). In either cse, f or f needs to be provided (or estimted) s dditionl dt. 3. The so-clled not--knot condition. 19

20 1.2.2 Cubic Nturl Spline Interpoltion To simplify the nottion we introduce the bbrevition z i = S (t i ), i =, 1,..., n, for the vlue of the second derivtive t the knots. We point out tht these vlues re not given s dt, but re prmeters to be determined. Since S is cubic S will be liner. If we write this liner polynomil on the subintervl [t i, t i+1 ) in its Lgrnge form we hve S i t i+1 x x t i (x) = z i + z i+1. t i+1 t i t i+1 t i With nother bbrevition h i = t i+1 t i this becomes S i (x) = z i h i (t i+1 x) + z i+1 h i (x t i ). (12) Remrk By ssigning the vlue z i to both pieces joining together t t i we will utomticlly enforce continuity of the second derivtive of S. Now, we obtin representtion for the piece S i by integrting (12) twice: S i (x) = z i 6h i (t i+1 x) 3 + z i+1 6h i (x t i ) 3 + C(x t i ) + D(t i+1 x). (13) The interpoltion conditions (for the piece S i ) S i (x) = z i 6h i (t i+1 x) 3 + z i+1 6h i (x t i ) 3 + S i (t i ) = y i, S i (t i+1 ) = y i+1 yield 2 2 liner system for the constnts C nd D. This leds to ( ) ( yi+1 yi (x t i )+ h i z i+1h i 6 z ih i h i 6 ) (t i+1 x). (14) (Note tht it is esily verified tht (14) stisfies the interpoltion conditions.) Once we hve determined the unknowns z i ech piece of the spline S cn be evluted vi (14). We hve not yet employed the C 1 continuity conditions t the interior knots, i.e., S i 1(t i ) = S i(t i ), i = 1, 2,..., n 1. (15) Thus, we hve n 1 dditionl conditions. Since there re n + 1 unknowns z i, i =, 1,..., n, we fix the second derivtive t the endpoints to be zero, i.e., z = z n =. These re the so-clled nturl end conditions. Differentition of (14) leds to ( ) yi+1 S i(x) = z i 2h i (t i+1 x) 2 + z i+1 2h i (x t i ) 2 + h i z i+1h i 6 ( yi z ih i h i 6 ). 2

21 Using this expression in (15) results in z i 1 h i 1 +2(h i 1 +h i )z i +z i+1 h i = 6 h i (y i+1 y i ) 6 h i 1 (y i y i 1 ), i = 1, 2,..., n 1. This represents symmetric tridigonl system for the unknowns z 1,..., z n 1 of the form 2(h + h 1 ) h 1... z 1 h 1 2(h 1 + h 2 ) h 2. z =. h n 3 2(h n 3 + h n 2 ) h n 2 z n 2 z... h n 2 2(h n 2 + h n 1 ) n 1 Here h i = t i+1 t i nd r i = 6 h i (y i+1 y i ) 6 h i 1 (y i y i 1 ), i = 1, 2,..., n 1. Remrk The system mtrix is even digonlly dominnt since 2(h i 1 + h i ) > h i 1 + h i since ll h i > due to the fct tht the knots t i re distinct nd form n incresing sequence. Thus, very efficient tridigonl version of Guss elimintion without pivoting cn be employed to solve for the missing z i (see the textbook on pge 353 for such n lgorithm) Optimlity of Nturl Splines Among ll smooth functions which interpolte given function f t the knots t, t 1,..., t n, the nturl spline is the smoothest, i.e., Theorem 1.16 Let f C 2 [, b] nd = t < t 1 <... < t n = b. If S is the cubic nturl spline interpolting f t t i, i =, 1,..., n, then [ S (x) ] 2 [ dx f (x) ] 2 dx. r 1 r 2. r n 2 r n 1. Remrk The integrls in Theorem 1.16 cn be interpreted s bending energies of thin rod, or s totl curvtures (since for smll deflections the curvture κ f ). This optimlity result is wht gve rise to the nme spline, since erly ship designers used piece of wood (clled spline) fixed t certin (interpoltion) points to describe the shpe of the ship s hull. Proof (of optimlity theorem) Define n uxiliry function g = f S. Then f = S +g nd f = S + g or ( f ) 2 = ( S ) 2 + ( g ) 2 + 2S g. 21

22 Therefore, Obviously, ( f (x) ) 2 ( dx = S (x) ) 2 ( dx + g (x) ) 2 dx + 2S (x)g (x)dx. so tht we re done is we cn show tht lso ( g (x) ) 2 dx, 2S (x)g (x)dx. To this end we brek the intervl [, b] into the subintervls [t i 1, t i ], nd get S (x)g (x)dx = n i=1 ti t i 1 S (x)g (x)dx. Now we cn integrte by prts (with u = S (x), dv = g (x)dx) to obtin { n [S (x)g (x) ] } ti t i S (x)g (x)dx. t i 1 t i 1 i=1 The first term is telescoping sum so tht n i=1 [ S (x)g (x) ] t i t i 1 = S (t n )g (t n ) S (t )g (t ) = due to the nturl end conditions of the spline S. This leves n ti S (x)g (x)dx = S (x)g (x)dx. t i 1 However, since S i is cubic polynomil we know tht S (x) c i on [t i 1, t i ). Thus, i=1 S (x)g (x)dx = = n ti c i i=1 n i=1 t i 1 g (x)dx c i [g(x)] t i t i 1 =. The lst eqution holds since g(t i ) = f(t i ) S(t i ), i =, 1,..., n, nd S interpoltes f t the knots t i. Remrk Cubic nturl splines should not be considered the nturl choice for cubic spline interpoltion. This is due to the fct tht one cn show tht the (rther rbitrry) choice of nturl end conditions yields n interpoltion error estimte of only O(h 2 ), where h = mx t i nd t i = t i t i 1. This needs to be compred to the estimte i=1,...,n of O(h 4 ) obtinble by cubic polynomils s well s other cubic spline methods. 22

23 1.2.4 Cubic Complete Spline Interpoltion The derivtion of cubic complete splines is similr to tht of the cubic nturl splines. However, we impose the dditionl end constrints S (t ) = f (t ), S (t n ) = f (t n ). This requires dditionl dt (f t the endpoints), but cn be shown to yields n O(h 4 ) interpoltion error estimte. Moreover, n energy minimiztion theorem nlogous to Theorem 1.16 lso holds since n i=1 [ S (x)g (x) ] t i t i 1 = S (t n )g (t n ) S (t )g (t ) by the end conditions. = S (t n ) ( f (t n ) S (t n ) ) S (t ) ( f (t ) S (t ) ) = Not--Knot Spline Interpoltion One of the most effective cubic spline interpoltion methods is obtined by choosing the knots different from the dt sites. In prticulr, if the dt is of the form then we tke the n 1 knots s x x 1 x 2... x n 1 x n y y 1 y 2... y n 1 y n, x x 2 x 3... x n 2 x n t t 1 t 2... t n 3 t n 2, i.e., the dt sites x 1 nd x n 1 re not--knot. The knots now define n 2 cubic polynomil pieces with totl of 4n 8 coefficients. On the other hnd, there re n + 1 interpoltion conditions together with three sets of (n 3) smoothness conditions t the interior knots. Thus, the number of conditions is equl to the number of unknown coefficients, nd no dditionl (rbitrry) conditions need to be imposed to solve the interpoltion problem. One cn interpret this pproch s using only one cubic polynomil piece to represent the first two (lst two) dt segments. The cubic not--knot spline hs n O(h 4 ) interpoltion error, nd requires no dditionl dt. More detils cn be found in the book A Prcticl Guide to Splines by Crl de Boor. Remrk 1. If we re given lso derivtive informtion t the dt sites x i, i =, 1,..., n, then we cn perform piecewise cubic Hermite interpoltion. The resulting function will be C 1 continuous, nd one cn show tht the interpoltion error is O(h 4 ). However, this function is not considered spline function since it does not hve the required smoothness. 23

24 2. There re lso piecewise cubic interpoltion methods tht estimte derivtive informtion t the dt sites, i.e., no derivtive informtion is provided s dt. Two such (locl) methods re nmed fter Bessel (yielding n O(h 3 ) interpoltion error) nd Akim (with n O(h 2 ) error). Agin, they re not spline functions s they re only C 1 smooth. 1.3 Numericl Integrtion Bsed on Interpoltion We now turn to pproximte integrtion (or qudrture). The simplest numericl integrtion methods re the left/right endpoint nd the midpoint rules studied in clculus. We will focus on methods bsed on polynomil interpoltion. The ide is simple one which is lso frequently used for numericl differentition: interpolte the integrnd t n + 1 points, nd then (exctly) integrte the degree n polynomil. Using the Lgrnge form of the interpolting polynomil this mens with weights f(x)dx = = p(x)dx = n f(x i ) i= n f(x i )l i (x)dx i= l i (x)dx n A i f(x i ) (16) i= A i = l i (x)dx. (17) Depending on the number of points (degree of the polynomil) used we hve different qudrture rule. Formuls of the type (16) nd (17) re collectively known s Newton- Cotes formuls. Exmple (Trpezoid Rule) In the cse n = 1 we get where nd A = A 1 = Together we get l (x)dx = l 1 (x)dx = f(x)dx A f(x ) + A 1 f(x 1 ) = A f() + A 1 f(b). x b b dx = 1 b x b dx = 1 b f(x)dx b 2 (x b) 2 2 b (x ) 2 2 [f() + f(b)]. b = b 2, = b 2. This is the well-known trpezoid rule. We cn improve the ccurcy by using piecewise liner spline interpoltion, i.e., we subdivide the intervl [, b] into subintervls 24

25 [x, x 1 ], [x 1, x 2 ],..., [x N 1, x N ], nd form liner polynomil interpolnt on ech subintervl. Then f(x)dx N 1 i= xi+1 Using the trpezoid rule on ech subintervl we hve f(x)dx N 1 i= x i x i+1 x i 2 p i (x)dx. [f(x i ) + f(x i+1 )]. To simplify this expression we ssume tht the subintervls re of equl length h, i.e., x i = + ih, i =, 1,..., N, with h = b N. This results in the composite trpezoid rule [ ] b f(x)dx T N f = b N 1 f() + 2 f( + ih) + f(b). 2N The error of the composite trpezoid rule is the subject of i=1 Theorem 1.17 If f C 2 [, b] nd h = b N re used for T N then the error where ξ (, b). E TN f = f(x)dx T N f = b 12 h2 f (ξ) = O(h 2 ), Proof We strt with the error for single subintervl (i.e., for the bsic trpezoid rule). Without loss of generlity we let [, b] = [, h]. Then, using the error formul for polynomil interpoltion (see Theorem 1.9), E T1 f = = x =,x 1 =h = = h h h 1 2 f (ξ) [f(x) p 1 (x)] dx 1 2 f (ξ x ) 1 (x x i )dx i= 1 2 f (ξ x )x(x h)dx h x(x h)dx, where we hve used the Men-Vlue Theorem for integrls of continuous functions in the lst step. The lst integrl cn of course be evluted, nd we obtin E T1 f = 1 12 h3 f (ξ), ξ (, b). For the composite rule we now hve E TN f = N E Ti f = i=1 25 N 1 12 h3 f (ξ i ). i=1

26 By rewriting one copy of h s b N Now, for some ξ (, b) we hve we get E TN f = b 12 h2 N i=1 f (ξ i ) N. N i=1 f (ξ i ) N = f (ξ), nd therefore we re done. Exmple (Simpson s Rule) The Newton-Cotes formul in the cse n = 2 (qudrtic polynomil interpolnt on single intervl [, b]) yields (the detils re omitted here) f(x)dx h 3 [ f() + 4f ( + b 2 ) ] + f(b), (18) where h = b 2. Eqution (18) is known s Simpson s rule. In order to formulte composite Simpson rule we need n even number, N, of subintervls. Then f(x)dx = = x2 x f(x)dx + N/2 x2i i=1 N/2 i=1 x4 x 2i 2 p i (x)dx x 2 f(x)dx xn h 3 [f(x 2i 2) + 4f(x 2i 1 ) + f(x 2i )], x N 2 f(x)dx where we hve used Simpson s rule (18) on ech subintervl [x 2i 2, x 2i ]. We cn rewrite the previous formul more efficiently s S N f = h N/2 N/2 f() + 2 f(x 2i 2 ) + 4 f(x 2i 1 ) + f(b) (19) 3 i=2 since N is even. This is the composite Simpson rule. For the error we hve Theorem 1.18 If f C 4 [, b] nd h = b N where ξ (, b). i=1 E SN f = b 18 h4 f (4) (ξ) = O(h 4 ), with N even re used for S N, then Remrk Note tht the ccurcy corresponds to tht expected for cubic polynomils even though only qudrtic interpolting polynomils were used. We get one order of h for free. 26

27 Proof (of Simpson error formul) Prts cn be found in the book by Kincid nd Cheney on pge 484. See lso the discussion of Peno kernels below, nd Homework Assignment 2. Remrk 1. An error estimte for the generl Newton-Cotes formuls (on single intervl) cn be found in the Kincid/Cheney book on pges The error is minimized by tking the interpoltion nodes s the zeros of the Chebyshev polynomils of the second kind. 1.4 Peno Kernels A useful (nd rther beutiful) tool for error estimtes (especilly for numericl differentition nd integrtion problems) is the use of Peno kernels nd the Peno kernel theorem. A liner functionl L on liner spce, e.g., C ν [, b], is mpping tht mps function from this spce onto sclr nd is liner, i.e., L(αf + βg) = αlf + βlg for α, β R, f, g C ν [, b]. Exmple 1. Point evlution functionl: Lf = f(x). 2. (Definite) Integrtion functionl: Lf = f(x)dx. Note tht liner combintions of liner functionls form new liner functionls. A firly generl liner functionl is n n Lf = α i (x)f (i) (x)dx + β ij f (i) (ξ ij ). (2) i= Here ξ ij [, b], f (i) denotes the ith derivtive of f, β ij re rel numbers, nd the functions α i re t lest piecewise continuous on [, b]. The function f should be in C n [, b]. Furthermore, we sy tht functionl nnihiltes polynomils P ν if j=1 Lp =, for ll p P ν. The νth Peno kernel of L s in (2) is the function where ν n nd is the truncted power function. k ν (ξ) = L [ (x ξ) ν +], ξ [, b], (x ξ) m + = { (x ξ) m, x ξ, x < ξ, 27

28 Exmple Let s compute the Peno kernel k 1 for the liner functionl defined by Lf = π By definition of the Peno kernel we hve (cos x)f (x)dx. k 1 (ξ) = L [(x ξ) + ] = π (cos x) d dx (x ξ) +dx. Using the definition of the truncted power function we cn rewrite the integrl nd then evlute π (cos x) d π (x ξ)dx = cos xdx = sin ξ. dx ξ Theorem 1.19 (Peno Kernel Theorem) If functionl L of the form (2) nnihiltes polynomils P ν, then for ll f C ν+1 [, b], Lf = 1 ν! where ν n nd k ν is the Peno kernel of L. ξ k ν (ξ)f (ν+1) (ξ)dξ Remrk The Peno kernel theorem llows estimtes of the form where we used the norms Exmple Consider the integrl Lf 1 ν! k ν 1 f (ν+1), Lf 1 ν! k ν f (ν+1) 1, Lf 1 ν! k ν 2 f (ν+1) 2, f 1 = f 2 = f(x) dx, ( 1/2 f(x) dx) 2, f = mx x [,b] f(x). 1 f(ξ)dξ nd find n pproximte integrtion formul of the form 1 f(ξ)dξ b 1 f() + b 2 f( 1 2 ) + b 3f(1) tht is exct if f is polynomil in P 3, nd find its error. 28

29 To nswer this question we consider the liner functionl Lf = 1 f(ξ)dξ b 1 f() + b 2 f( 1 2 ) + b 3f(1), nd first find b 1, b 2, nd b 3 so tht L nnihiltes P 3. If we let f(x) = 1, then we get the condition For f(x) = x we get for f(x) = x 2 we get = Lf = 1 = Lf = = Lf = nd for f(x) = x 3 we get = Lf = 1dξ (b 1 + b 2 + b 3 ) = 1 b 1 b 2 b ξdξ ( 1 2 b 2 + b 3 ) = b 2 b 3, ξ 2 dξ ( 1 4 b 2 + b 3 ) = b 2 b 3, ξ 3 dξ ( 1 8 b 2 + b 3 ) = b 2 b 3. The unique solution of this system of 4 liner equtions in 3 unknowns is nd therefore 1 f(ξ)dξ 1 6 b 1 = 1 6, b 2 = 2 3, b 3 = 1 6, [f() + 4f( 12 ) + f(1) ]. To estimte the error in this pproximtion we use the Peno kernel of L. It is given by k 3 (ξ) = L [ (x ξ) 3 ] + 1 = (x ξ) 3 +dx 1 [ ( ξ) ( 1 ] 6 2 ξ)3 + + (1 ξ) [ = 4( 1 ] 2 ξ)3 + + (1 ξ) 3 + = = ξ (x ξ) 3 dx 1 6 {[ 4( 1 2 ξ)3 + (1 ξ) 3], ξ 1 2 (1 ξ) 3, 1 2 ξ 1. (1 ξ) { 1 12 ξ3 (2 3ξ), ξ (1 ξ)3 (3ξ 1), 1 2 ξ 1. Now the Peno kernel theorem sys tht 1 f(ξ)dξ 1 [f() + 4f( 12 ] 6 ) + f(1) = Lf = 1 3! 29 1 k 3 (ξ)f (4) (ξ)dξ,

30 nd we cn explicitly clculte estimtes of the form since Lf f (4) 1, Lf k 3 1 = 1 48, k 3 2 = 1.5 ODEs nd the Lipschitz Condition We consider the system of first-order ODE IVP f (4) 2, Lf f (4) f C 4 [, 1] , k 3 = y (t) = dy(t) = f(t, y(t)), t t, (21) dt y(t ) = y. (22) Here y = y 1. y d, y = y,1. y,d, f = f 1. f d, R d. Remrk This pproch covers not only first-order ODEs, but lso higher-order ODEs, since ny d-th order ODE IVP cn be converted to system of d first-order IVPs (see Assignment 2). If f(t, y) = A(t)y + b(t) for some d d mtrix-vlued function A nd d 1 vectorvlued function b, then the ODE is liner, nd if b(t) = it is liner nd homogeneous. Otherwise it is nonliner. If f is independent of t, the ODE is clled utonomous, nd if f is independent of y, then the ODE system reduces to (vector of) indefinite integrl(s). Theorem 1.2 (Picrd-Lindelöf: Existence nd Uniqueness) Let B be the bll B = {x R d : x y b} nd let S be the cylinder S = {(t, x) : t [t, t + ], x B} where, b >. If f is continuous on S nd f lso stisfies the Lipschitz condition f(t, x) f(t, y) λ x y, x, y B, then the IVP (21), (22) hs unique solution on [t, t + α], where α is some constnt tht depends on, b nd f. In fct, { } b α = min,. sup (t,x) S f(t, x) Note tht in the system setting we need to mesure differences of vectors in some pproprite norm insted of simple bsolute vlue. 3

31 Remrk The proof of this theorem is rther involved. on S gur- Exmple For single eqution, continuity of the prtil derivtive f(t,y) y ntees Lipschitz continuity of f with λ = f(t, y) y. mx t [t,t +] y B For the initil vlue problem we hve y (t) = 2t (y(t)) 2, y() = 1, f(t, y) = 2ty 2, f(t, y) y = 4ty, which re both continuous on ll of R 2. The theorem bove gurntees existence nd uniqueness of solution for t ner t =. In fct, it is given by y(t) = 1, 1 < t < 1. 1 t2 However, we see tht just becuse f nd f(t,y) y re continuous on ll of R 2 we cnnot expect existence or uniqueness of solution y for ll t. Remrk In the system setting sufficient condition for Lipschitz continuity of f is given by continuity of the Jcobin mtrix f(t, y) y [ fi (t, y 1,..., y d ) = y j ] d. i,j=1 Remrk Recll tht liner system y = Ay, t t, y(t ) = y with d d mtrix A lwys hs unique solution. It is given by y(t) = d e λ l(t t ) α l, t t, l=1 where λ 1,..., λ d re the eigenvlues of A, nd the α 1,..., α d R d re vectors (eigenvectors if the eigenvlues re distinct). 1.6 Euler s Method The bsic Algorithm Recll tht we re interested in generl first-order IVPs (21), (22) of the form y (t) = f(t, y(t)), t t y(t ) = y. 31

32 It is our gol to derive numericl methods for the solution of this kind of problem. The first, nd probbly best known, method is clled Euler s method. Even though this is one of the originl numericl methods for the solution of IVPs, it remins importnt for both prcticl nd theoreticl purposes. The method is derived by considering the pproximtion of the first derivtive. This implies y (t) y(t + h) y(t) h y(t + h) y(t) + hy (t), which using the differentil eqution (21) becomes y(t + h) y(t) + hf(t, y(t)). (23) Introducing sequence of points t, t 1 = t + h, t 2 = t + 2h,..., t N = t + Nh, this immeditely leds to n itertive lgorithm. Algorithm Input t, y, f, h, N t = t, y = y for n = 1 to N do end y y + hf(t, y) t t + h Remrk 1. Alterntely, we cn derive the formul for Euler s method vi integrtion. Since the IVP gives us both n initil condition s well s the slope y = f(t, y) of the solution, we cn ssume tht the slope is constnt on smll intervl [t, t + h], i.e., f(t, y(t)) f(t, y(t )) for t [t, t + h]. Then we cn integrte to get t y(t) = y(t ) + f(τ, y(τ))dτ t Euler s method. t y(t ) + f(t, y(t ))dτ t = y(t ) + (t t )f(t, y(t )) 2. Note tht Euler s method yields set of discrete points (t n, y n ), n = 1,..., N, which pproximte the grph of the solution y = y(t). In order to obtin continuous solution one must use n interpoltion or pproximtion method. 3. Euler s method is illustrted in the Mple worksheet 472 Euler Tylor.mws. 4. In principle, it is esy to use Euler s method with vrible step size, i.e., y(t n+1 ) y n+1 = y n + h n f(t n, y n ), but nlysis of the method is simpler with constnt step size h n = h. 32

33 1.6.2 Tylor Series Methods An immedite generliztion of Euler s method re the so-clled generl Tylor series methods. We use Tylor expnsion y(t + h) = y(t) + hy (t) + h2 2 y (t) + h3 6 y (t) +..., nd therefore obtin the numericl pproximtion y(t + h) ν k= h k y (k) (t) k! (24) which is referred to s ν-th order Tylor series method. Remrk 1. Obviously, Euler s method is first-order Tylor method. 2. In order to progrm Tylor method we need to pre-compute ll higher-order derivtives of y required by the method since the differentil eqution only provides representtion for y. This implies tht we will end up with code tht depends on (nd chnges with) the IVP to be solved. 3. Computer softwre with symbolic mnipultion cpbilities (such s Mple or Mthemtic) llows us to write code for Tylor methods for rbitrry IVPs. We illustrte the trditionl tretment of second-order Tylor method in the following exmple. Exmple Consider the initil vlue problem (d = 1) y (t) = y(t) t y() = 1 2. The second-order Tylor pproximtion is given by y(t + h) y(t) + hy (t) + h2 2 y (t). Therefore, we need to express y (t) nd y (t) in terms of y nd t so tht n itertive lgorithm cn be formulted. From the differentil eqution Therefore, differentiting this reltion, y (t) = f(t, y(t)) = y(t) t y (t) = y (t) 2t, nd this cn be incorported into the following lgorithm. 33

34 Algorithm Input t, y, f, h, N t = t, y = y for n = 1 to N do end y = f(t, y) y = y 2t y y + hy + h2 2 y t t + h Remrk 1. Two modifictions re suggested to mke the lgorithm more efficient nd numericlly stble. () Replce the computtion of y by the nested formultion y = y + h (y + h2 ) y. (b) Advnce the time t vi t = t + nh. 2. An exmple of fourth-order Tylor method is given in the Mple worksheet Euler Tylor.mws Errors nd Convergence When considering errors introduced using the Tylor series or Euler pproximtion we need to distinguish between two different types of error: locl trunction error, nd globl trunction error. The locl trunction error is the error introduced directly by trunction of the Tylor series, i.e., t ech time step we hve n error E ν = hν+1 (ν + 1)! y(ν+1) (t + θh), < θ < 1. Thus, the ν-th order Tylor method hs n O(h ν+1 ) locl trunction error. The globl trunction error is the error tht results if we use ν-th order Tylor method hving O(h ν+1 ) locl trunction error to solve our IVP up to time t = t + t. Since we will be performing t N = h steps we see tht one order of h is lost in the globl trunction error, i.e., the globl trunction error is of the order O(h ν ). 34

35 Remrk Of course, trunction errors re independent of roundoff errors which cn dd to the overll error. As we will see lter, method with O(h ν+1 ) locl ccurcy need not be globlly ν-th order. In fct, it need not converge t ll. Stbility will be the key to convergence. A numericl method for the IVP (21), (22) is clled convergent if for every Lipschitz function f nd every t > we hve lim h + mx y n,h y(t n ) =. n=,1,...,n In other words, if the numericl solution pproches the nlytic solution for incresingly smller step sizes h. For Euler s method we cn estblish convergence (nd therefore the bove heuristics regrding trunction errors re justified). Theorem 1.21 Euler s method is convergent. Proof To simplify the proof we ssume tht f (nd therefore lso y) is nlytic. We introduce the nottion e n,h = y n,h y(t n ) fir the error t step n. We need to show lim h + mx e n,h =. n=,1,...,n Tylor s theorem for the nlytic solution y gives us (since t n+1 = t n + h) Replcing y by the ODE (21) we hve y(t n+1 ) = y(t n ) + hy (t n ) + O(h 2 ). y(t n+1 ) = y(t n ) + hf(t n, y(t n )) + O(h 2 ). From Euler s method we hve for the numericl solution y n+1,h = y n,h + hf(t n, y n,h ). The difference of these lst two expressions yields e n+1,h = y n+1,h y(t n+1 ) = [y n,h + hf(t n, y n,h )] [ y(t n ) + hf(t n, y(t n )) + O(h 2 ) ] Since y n,h = y(t n ) + e n,h we hve = e n,h + h [f(t n, y n,h ) f(t n, y(t n ))] + O(h 2 ). e n+1,h = e n,h + h [f(t n, y(t n ) + e n,h ) f(t n, y(t n ))] + O(h 2 ). 35