19.4 Spline Interpolation

Transcription

1 c9-b.qxd 9/6/5 6:4 PM Page 8 8 CHAP. 9 Numerics in General 9.4 Spline Interpolation Given data (function values, points in the xy-plane) (x, ƒ ), (x, ƒ ),, (x n, ƒ n ) can be interpolated by a polynomial P n (x) of degree n or less so that the curve of P n (x) passes through these n points (x j, ƒ j ); here ƒ ƒ(x ),, ƒ n ƒ(x n ). See Sec. 9.. Now if n is large, there may be trouble: P n (x) may tend to oscillate for x between the nodes x,, x n. Hence we must be prepared for numeric instability (Sec. 9.). Figure 4 shows a famous example by C. Runge for which the maximum error even approaches as n * (with the nodes kept equidistant and their number increased). Figure 4 illustrates the increase of the oscillation with n for some other function that is piecewise linear. Those undesirable oscillations are avoided by the method of splines initiated by I. J. Schoenberg in 946 (Quarterly of Applied Mathematics 4, pp , 4). This method is widely used in practice. It also laid the foundation for much of modern CAD (computer-aided design). Its name is borrowed from a draftman s spline, which is an elastic rod bent to pass through given points and held in place by weights. The mathematical idea of the method is as follows: Instead of using a single high-degree polynomial P n over the entire interval a x b in which the nodes lie, that is, () a x x x n b, we use n low-degree, e.g., cubic, polynomials q (x), q (x),, q n (x), one over each subinterval between adjacent nodes, hence q from x to x, then q from P (x) f(x) y 5 Fig Runge s example ƒ(x) /( x ) and interpolating polynomial P (x) x 4 f(x) 4 P (x) 4 4 P 4 (x) 4 4 P 8 (x) Fig. 4. Piecewise linear function ƒ(x) and interpolation polynomials of increasing degrees CARL RUNGE (856 97), German mathematician, also known for his work on ODEs (Sec..).

2 c9-b.qxd 9/6/5 6:4 PM Page 8 SEC. 9.4 Spline Interpolation 8 x to x, and so on. From this we compose an interpolation function g(x), called a spline, by fitting these polynomials together into a single continuous curve passing through the data points, that is, () g(x ) ƒ(x ) ƒ, g(x ) ƒ(x ) ƒ,, g(x n ) ƒ(x n ) ƒ n. Note that g(x) q (x) when x x x, then g(x) q (x) when x x x, and so on, according to our construction of g. Thus spline interpolation is piecewise polynomial interpolation. The simplest q j s would be linear polynomials. However, the curve of a piecewise linear continuous function has corners and would be of little interest in general think of designing the body of a car or a ship. We shall consider cubic splines because these are the most important ones in applications. By definition, a cubic spline g(x) interpolating given data (x, ƒ ),, (x n, ƒ n ) is a continuous function on the interval a x x x n b that has continuous first and second derivatives and satisfies the interpolation condition (); furthermore, between adjacent nodes, g(x) is given by a polynomial q j (x) of degree or less. We claim that there is such a cubic spline. And if in addition to () we also require that () g (x ) k, g (x n ) k n (given tangent directions of g(x) at the two endpoints of the interval a x b), then we have a uniquely determined cubic spline. This is the content of the following existence and uniqueness theorem, whose proof will also suggest the actual determination of splines. (Condition () will be discussed after the proof.) THEOREM Existence and Uniqueness of Cubic Splines Let (x, ƒ ), (x, ƒ ),, (x n, ƒ n ) with arbitrarily spaced given x j [see ()] and given ƒ j ƒ(x j ), j,,, n. Let k and k n be any given numbers. Then there is one and only one cubic spline g(x) corresponding to () and satisfying () and (). PROOF By definition, on every subinterval I j given by x j x x j the spline g(x) must agree with a polynomial q j (x) of degree not exceeding such that (4) q j (x j ) ƒ(x j ), q j (x j ) ƒ(x j ) (j,,, n ). For the derivatives we write (5) q j (x j ) k j, q j (x j ) k j ( j,,, n ) with k and k n given and k,, k n to be determined later. Equations (4) and (5) are four conditions for each q j (x). By direct calculation, using the notation (6*) c j ( j,,, n ) hj xj x j we can verify that the unique cubic polynomial q j (x) ( j,,, n ) satisfying

3 c9-b.qxd 9/6/5 6:4 PM Page 8 8 CHAP. 9 Numerics in General (4) and (5) is (6) q j (x) ƒ(x j )c j (x x j ) [ c j (x x j )] ƒ(x j )c j (x x j ) [ c j (x x j )] k j c j (x x j )(x x j ) Differentiating twice, we obtain k j c j (x x j ) (x x j ). (7) q j (x j ) 6c j ƒ(x j ) 6c j ƒ(x j ) 4c j k j c j k j (8) q j (x j ) 6c j ƒ(x j ) 6c j ƒ(x j ) c j k j 4c j k j. By definition, g(x) has continuous second derivatives. This gives the conditions q j (x j ) q j (x j ) (j,, n ). If we use (8) with j replaced by j, and (7), these n equations become (9) c j k j (c j c j )k j c j k j [cj ƒ j c j ƒ j ] where ƒ j ƒ(x j ) ƒ(x j ) and ƒ j ƒ(x j ) ƒ(x j ) and j,, n, as before. This linear system of n equations has a unique solution k,, k n since the coefficient matrix is strictly diagonally dominant (that is, in each row the (positive) diagonal entry is greater than the sum of the other (positive) entries). Hence the determinant of the matrix cannot be zero (as follows from Theorem in Sec..7), so that we may determine unique values k,, k n of the first derivative of g(x) at the nodes. This proves the theorem. Storage and Time Demands in solving (9) are modest, since the matrix of (9) is sparse (has few nonzero entries) and tridiagonal (may have nonzero entries only on the diagonal and on the two adjacent parallels above and below it). Pivoting (Sec. 7.) is not necessary because of that dominance. This makes splines efficient in solving large problems with thousands of nodes or more. For some literature and some critical comments, see American Mathematical Monthly 5 (998), Condition () includes the clamped conditions () g (x ) ƒ (x ), g (x n ) ƒ (x n ), in which the tangent directions ƒ (x ) and ƒ (x n ) at the ends are given. Other conditions of practical interest are the free or natural conditions () g (x ), g (x n ) (geometrically: zero curvature at the ends, as for the draftman s spline), giving a natural spline. These names are motivated by Fig. 9 in Problem Set..

4 c9-b.qxd 9/6/5 6:4 PM Page 8 SEC. 9.4 Spline Interpolation 8 Determination of Splines. Let k and k n be given. Obtain k,, k n by solving the linear system (9). Recall that the spline g(x) to be found consists of n cubic polynomials q,, q n. We write these polynomials in the form () q j (x) a j a j (x x j ) a j (x x j ) a j (x x j ) where j,, n. Using Taylor s formula, we obtain a j q j (x j ) ƒ j by (), a j q j (x j ) k j by (5), () a j q j (x j ) (ƒ j ƒ j ) (k j k j ) h h by (7), a j q j (x j ) (ƒ j ƒ j ) (k j k j ) 6 h h with a j obtained by calculating q j (x j ) from () and equating the result to (8), that is, 6 q j (x j ) a j 6a j h (ƒ j ƒ j ) (k j k j ), h h and now subtracting from this a j as given in () and simplifying. Note that for equidistant nodes of distance h j h we can write c j c /h in (6*) and have from (9) simply (4) k j 4k j k j (ƒ j ƒ j ) (j,, n ). h EXAMPLE Spline Interpolation. Equidistant Nodes Interpolate ƒ(x) x 4 on the interval x by the cubic spline g(x) corresponding to the nodes x, x, x and satisfying the clamped conditions g ( ) ƒ ( ), g () ƒ (). Solution. In our standard notation the given data are ƒ ƒ( ), ƒ ƒ(), ƒ ƒ(). We have h and n, so that our spline consists of n polynomials q (x) a a (x ) a (x ) a (x ) ( x ), q (x) a a x a x a x ( x ). We determine the k j from (4) (equidistance!) and then the coefficients of the spline from (). Since n, the system (4) is a single equation (with j and h ) k 4k k (ƒ ƒ ). Here ƒ ƒ (the value of x 4 at the ends) and k 4, k 4, the values of the derivative 4x at the ends and. Hence 4 4k 4 ( ), k. From () we can now obtain the coefficients of q, namely, a ƒ, a k 4, and

5 c9-b.qxd 9/6/5 6:4 PM Page CHAP. 9 Numerics in General a ( ) ( 8) (ƒ ƒ ) (k k ) 5. a ( ) ( 4). (ƒ ƒ ) (k k ) Similarly, for the coefficients of q we obtain from () the values a ƒ, a k, and a (ƒ ƒ ) (k k ) ( ) (4 ) a (ƒ ƒ ) (k k ) ( ) (4 ). This gives the polynomials of which the spline g(x) consists, namely, q (x) 4(x ) 5(x ) (x ) x x g(x) { q (x) x x if if x. x. Figure 4 shows ƒ(x) and this spline. Do you see that we could have saved over half of our work by using symmetry? f(x) x g(x) Fig. 4. Function ƒ(x) x 4 and cubic spline g(x) in Example EXAMPLE Natural Spline. Arbitrarily Spaced Nodes Find a spline approximation and a polynomial approximation for the curve of the cross section of the circularshaped Shrine of the Book in Jerusalem shown in Fig Fig. 44. Shrine of the Book in Jerusalem (Architects F. Kissler and A. M. Bartus)

6 c9-b.qxd 9/6/5 6:4 PM Page 85 SEC. 9.4 Spline Interpolation 85 Solution. Thirteen points, about equally distributed along the contour (not along the x-axis!), give these data: x j ƒ j The figure shows the corresponding interpolation polynomial of th degree, which is useless because of its oscillation. (Because of roundoff your software will also give you small error terms involving odd powers of x.) The polynomial is P (x).9.658x.858x 4.4x 6.4x x.7867x. The spline follows practically the contour of the roof, with a small error near the nodes.8 and.8. The spline is symmetric. Its six polynomials corresponding to positive x have the following coefficients of their representations (). (Note well that () is in terms of powers of x x j, not x!) j x-interval a j a j a j a j PROBLEM SET 9.4. WRITING PROJECT. Splines. In your own words, and using as few formulas as possible, write a short report on spline interpolation, its motivation, a comparison with polynomial interpolation, and its applications.. (Individual polynomial q j ) Show that q j (x) in (6) satisfies the interpolation condition (4) as well as the derivative condition (5).. Verify the differentiations that give (7) and (8) from (6). 4. (System for derivatives) Derive the basic linear system (9) for k,, k n as indicated in the text. 5. (Equidistant nodes) Derive (4) from (9). 6. (Coefficients) Give the details of the derivation of a j and a j in (). 7. Verify the computations in Example. 8. (Comparison) Compare the spline g in Example with the quadratic interpolation polynomial over the whole interval. Find the maximum deviations of g and p from ƒ. Comment. 9. (Natural spline condition) Using the given coefficients, verify that the spline in Example satisfies g (x) at the ends. 6 DETERMINATION OF SPLINES Find the cubic spline g(x) for the given data with k and k n as given.. ƒ( ) ƒ( ) ƒ() ƒ(), ƒ(), k k 4. If we started from the piecewise linear function in Fig. 45, we would obtain g(x) in Prob. as the spline satisfying g ( ) ƒ ( ), g () ƒ (). Find and sketch or graph the corresponding interpolation polynomial of 4th degree and compare it with the spline. Comment..5 Fig. 45. Spline and interpolation polynomial in Problems and. ƒ ƒ(), ƒ ƒ() 9, ƒ ƒ(4) 4, ƒ ƒ(6) 4, k, k x

7 c9-b.qxd 9/6/5 6:4 PM Page CHAP. 9 Numerics in General. ƒ ƒ( ), ƒ ƒ() 4, ƒ ƒ(), k, k. Is g(x) even? (Give reason.) 4. ƒ ƒ(), ƒ ƒ(), ƒ ƒ() 6, ƒ ƒ(), k, k 5. ƒ ƒ(), ƒ ƒ(), ƒ ƒ(), ƒ ƒ(), k, k 6 6. It can happen that a spline is given by the same polynomial in two adjacent subintervals. To illustrate this, find the cubic spline g(x) for ƒ(x) sin x corresponding to the partition x /, x, x / of the interval / x / and satisfying g ( /) ƒ ( /) and g ( /) ƒ ( /). 7. (Natural conditions) Explain the remark after (). 8. CAS EXPERIMENT. Spline versus Polynomial. If your CAS gives natural splines, find the natural splines when x is integer from m to m, and y() and all other y equal to. Graph each such spline along with the interpolation polynomial p m. Do this for m to (or more). What happens with increasing m? 9. If a cubic spline is three times continuously differentiable (that is, it has continuous first, second, and third derivatives), show that it must be a single polynomial.. TEAM PROJECT. Hermite Interpolation and Bezier Curves. In Hermite interpolation we are looking for a polynomial p(x) (of degree n or less) such that p(x) and its derivative p (x) have given values at n nodes. (More generally, p(x), p (x), p (x), may be required to have given values at the nodes.) (a) Curves with given endpoints and tangents. Let C be a curve in the xy-plane parametrically represented by r(t) [x(t), y(t)], t (see Sec. 9.5). Show that for given initial and terminal points of a curve and given initial and terminal tangents, say, A: r [x(), y()] [x, y ], B: r [x(), y()] [x, y ] v [x (), y ()] [x, y ], v [x (), y ()] [x, y ] we can find a curve C, namely, (5) r(t) r v t ((r r ) (v v ))t ((r r ) v v )t ; in components, x(t) x x t ((x x ) (x x ))t ((x x ) x x )t y(t) y y t ((y y ) (y y ))t ((y y ) y y )t. Note that this is a cubic Hermite interpolation polynomial, and n because we have two nodes (the endpoints of C). (This has nothing to do with the Hermite polynomials in Sec. 5.8.) The two points and G A : g r v [x x, y y ] G B : g r v [x x, y y ] are called guidepoints because the segments AG A and BG B specify the tangents graphically. A, B, G A, G B determine C, and C can be changed quickly by moving the points. A curve consisting of such Hermite interpolation polynomials is called a Bezier curve, after the French engineer P. Bezier of the Renault Automobile Company, who introduced them in the early 96s in designing car bodies. Bezier curves (and surfaces) are used in computer-aided design (CAD) and computer-aided manufacturing (CAM). (For more details, see Ref. [E] in App..) (b) Find and graph the Bezier curve and its guidepoints if A: [, ], B: [, ], v [ _, _ ], v [ _, _ ]. 4 (c) Changing guidepoints changes C. Moving guidepoints farther away makes C staying near the tangents for a longer time. Confirm this by changing v and v in (b) to v and v (see Fig. 46). (d) Make experiments of your own. What happens if you change v in (b) to v. If you rotate the tangents? If you multiply v and v by positive factors less than? y.4. A Fig. 46. G A(b) (b) G A(c) B (c) G B(b) G B(c) Team Project (b) and (c): Bezier curves x

8 9.5 Numeric Integration and Differentiation Numeric integration means the numeric evaluation of integrals J b a ƒ(x) dx where a and b are given and ƒ is a function given analytically by a formula or empirically by a table of values. Geometrically, J is the area under the curve of ƒ between a and b (Fig. 47). We know that if ƒ is such that we can find a differentiable function F whose derivative is ƒ, then we can evaluate J by applying the familiar formula J b a ƒ(x) dx F(b) F(a) [F (x) ƒ(x)]. Tables of integrals or a CAS (Mathematica, Maple, etc.) may be helpful for this purpose. However, applications often lead to integrals whose analytic evaluation would be very difficult or even impossible, or whose integrand is an empirical function given by recorded numeric values. Then we may obtain approximate numeric values of the integral by a numeric integration method. Rectangular Rule. Trapezoidal Rule Numeric integration methods are obtained by approximating the integrand ƒ by functions that can easily be integrated. The simplest formula, the rectangular rule, is obtained if we subdivide the interval of integration a x b into n subintervals of equal length h (b a)/n and in each subinterval approximate ƒ by the constant ƒ(x j *), the value of ƒ at the midpoint x j * of the jth subinterval (Fig. 48). Then ƒ is approximated by a step function (piecewise constant function), the n rectangles in Fig. 48 have the areas ƒ(x *)h,, ƒ(x n *)h, and the rectangular rule is () J b a ƒ(x) dx h[ƒ(x *) ƒ(x *) ƒ(x n *)] (h ). The trapezoidal rule is generally more accurate. We obtain it if we take the same subdivision as before and approximate ƒ by a broken line of segments (chords) with endpoint F [qual length

9 c9-b.qxd 9/6/5 6:4 PM Page CHAP. 9 Numerics in General y y = f(x) a x x x n b x Fig. 49. Trapezoidal rule By taking their sum we obtain the trapezoidal rule () J b a ƒ(x) dx h[ _ ƒ(a) ƒ(x ) ƒ(x ) ƒ(x n ) _ ƒ(b)] where h (b a)/n, as in (). The x j s and a and b are called nodes. EXAMPLE Trapezoidal Rule Evaluate J e x dx by means of () with n. Solution. J.( ).746 from Table 9.. Table 9. Computations in Example j x j x j e x j Sums Error Bounds and Estimate for the Trapezoidal Rule An error estimate for the trapezoidal rule can be derived from (5) in Sec. 9. with n by integration as follows. For a single subinterval we have ƒ(x) p (x) (x x )(x x ) ƒ (t)

10 c9-b.qxd 9/6/5 6:4 PM Page 89 SEC. 9.5 Numeric Integration and Differentiation 89 with a suitable t depending on x, between x and x. Integration over x from a x to x x h gives x h h ƒ(x) dx [ƒ(x ) ƒ(x )] x x h ƒ (t(x)) (x x )(x x h) dx. x Setting x x v and applying the mean value theorem of integral calculus, which we can use because (x x )(x x h) does not change sign, we find that the right side equals (*) h ƒ (t ) h h ƒ (t ) h v(v h) dv ( ) ƒ (t ) where t is a (suitable, unknown) value between x and x. This is the error for the trapezoidal rule with n, often called the local error. Hence the error of () with any n is the sum of such contributions from the n subintervals; since h (b a)/n, nh n(b a) /n, and (b a) n h, we obtain (b a) (b a) () ƒ (tˆ) h ƒ (tˆ) n with (suitable, unknown) tˆ between a and b. Because of () the trapezoidal rule () is also written (*) J b a ƒ(x) dx h [ _ ƒ(a) ƒ(x ) ƒ(x n ) _ ƒ(b) ] b a h ƒ (tˆ). Error Bounds are now obtained by taking the largest value for ƒ, say, M, and the smallest value, M *, in the interval of integration. Then () gives (note that K is negative) (b a) (4) KM KM * b a where K h. n Error Estimation by Halving h is advisable if h is very complicated or unknown, for instance, in the case of experimental data. Then we may apply the Error Principle of Sec. 9.. That is, we calculate by (), first with h, obtaining, say, J J h h, and then with _ h, obtaining J J h/ h/. Now if we replace h in () with ( _ h), the error is multiplied by /4. Hence h/ _ 4 h (not exactly because tˆ may differ). Together, J h/ h/ J h h J h 4 h/. Thus J h/ J h (4 ) h/. Division by gives the error formula for J h/ (5) h/ (J h/ J h ). EXAMPLE Error Estimation for the Trapezoidal Rule by (4) and (5) Estimate the error of the approximate value in Example by (4) and (5). Solution. (A) Error bounds by (4). By differentiation, ƒ (x) (x )e x. Also, ƒ (x) if x, so that the minimum and maximum occur at the ends of the interval. We compute

11 c9-b.qxd 9/6/5 6:4 PM Page 8 8 CHAP. 9 Numerics in General M ƒ () and M * ƒ (). Furthermore, K /, and (4) gives Hence the exact value of J must lie between and Actually, J , exact to 6D. (B) Error estimate by (5). J h.746 in Example. Also, J h/.5 Y 9 j e ( j/) (.67879)Z Hence h/ _ (J h/ J h ).5 and J h/ h/.74684, exact to 6D. Simpson s Rule of Integration Piecewise constant approximation of ƒ led to the rectangular rule (), piecewise linear approximation to the trapezoidal rule (), and piecewise quadratic approximation will lead to Simpson s rule, which is of great practical importance because it is sufficiently accurate for most problems, but still sufficiently simple. To derive Simpson s rule, we divide the interval of integration a x b into an even number of equal subintervals, say, into n m subintervals of length h (b a)/(m), with endpoints x ( a), x,, x m, x m ( b); see Fig. 44. We now take the first two subintervals and approximate ƒ(x) in the interval x x x x h by the Lagrange polynomial p (x) through (x, ƒ ), (x, ƒ ), (x, ƒ ), where ƒ j ƒ(x j ). From () in Sec. 9. we obtain (x x )(x x ) (x x )(x x ) (x x )(x x ) (6) p (x) ƒ ƒ ƒ. (x x )(x x ) (x x )(x x ) (x x )(x x ) The denominators in (6) are h, h, and h, respectively. Setting s (x x )/h, we have x x sh, x x x (x h) (s )h x x x (x h) (s )h and we obtain p (x) _ s(s )ƒ (s )(s )ƒ _ (s )sƒ. y First parabola Second parabola y = f(x) Last parabola a b x x x x 4 x m x m x Fig. 44. Simpson s rule

12 c9-b.qxd 9/6/5 6:4 PM Page 8 SEC. 9.5 Numeric Integration and Differentiation 8 We now integrate with respect to x from x to x. This corresponds to integrating with respect to s from to. Since dx hds, the result is (7*) x ƒ(x) dx x x 4 p (x) dx h ( ƒ ƒ ƒ ). x A similar formula holds for the next two subintervals from x to x 4, and so on. By summing all these m formulas we obtain Simpson s rule 4 (7) b h ƒ(x) dx (ƒ 4ƒ ƒ 4ƒ ƒ m 4ƒ m ƒ m ), a where h (b a)/(m) and ƒ j ƒ(x j ). Table 9.4 shows an algorithm for Simpson s rule. Table 9.4 Simpson s Rule of Integration ALGORITHM SIMPSON (a, b, m, ƒ, ƒ,, ƒ m ) This algorithm computes the integral J aƒ(x) b dx from given values ƒ j ƒ(x j ) at equidistant x a, x x h,, x m x mh b by Simpson s rule (7), where h (b a)/(m). INPUT: a, b, m, ƒ,, ƒ m OUTPUT: Approximate value J of J Compute s ƒ ƒ m s ƒ ƒ ƒ m s ƒ ƒ 4 ƒ m h (b a)/m J h (s 4s s ) End SIMPSON OUTPUT J. Stop. Error of Simpson s Rule (7). If the fourth derivative ƒ (4) exists and is continuous on a x b, the error of (7), call it S, is (b a) 5 (b a) (8) S ƒ (4) (tˆ) h 4 ƒ (4) (tˆ); 8(m) THOMAS SIMPSON (7 76), self-taught English mathematician, author of several popular textbooks. Simpson s rule was used much earlier by Torricelli, Gregory (in 668), and Newton (in 676).

13 c9-b.qxd 9/6/5 6:4 PM Page 8 8 CHAP. 9 Numerics in General here tˆ is a suitable unknown value between a and b. This is obtained similarly to (). With this we may also write Simpson s rule (7) as (7**) b h (b a) ƒ(x) dx (ƒ 4ƒ ƒ m ) h 4 ƒ (4) (tˆ). 8 a Error Bounds. By taking for ƒ (4) in (8) the maximum M 4 and minimum M 4 * on the interval of integration we obtain from (8) the error bounds (note that C is negative) (b a) (9) CM 4 S CM 4 * 5 (b a) where C h 4. 8(m) 4 8 Degree of Precision (DP) of an integration formula. This is the maximum degree of arbitrary polynomials for which the formula gives exact values of integrals over any intervals. Hence for the trapezoidal rule, DP because we approximate the curve of ƒ by portions of straight lines (linear polynomials). For Simpson s rule we might expect DP (why?). Actually, DP by (9) because ƒ (4) is identically zero for a cubic polynomial. This makes Simpson s rule sufficiently accurate for most practical problems and accounts for its popularity. Numeric Stability with respect to rounding is another important property of Simpson s rule. Indeed, for the sum of the roundoff errors j of the m values ƒ j in (7) we obtain, since h (b a)/m, h 4 m (b a) m 6mu (b a)u where u is the rounding unit (u _ 6 if we round off to 6D; see Sec. 9.). Also 6 4 is the sum of the coefficients for a pair of intervals in (7); take m in (7) to see this. The bound (b a)u is independent of m, so that it cannot increase with increasing m, that is, with decreasing h. This proves stability. Newton Cotes Formulas. We mention that the trapezoidal and Simpson rules are special closed Newton Cotes formulas, that is, integration formulas in which ƒ(x) is interpolated at equally spaced nodes by a polynomial of degree n (n for trapezoidal, n for Simpson), and closed means that a and b are nodes (a x, b x n ). n (the three-eighths rule; Review Prob. ) and a higher n are used occasionally. From n 8 on, some of the coefficients become negative, so that a positive ƒ j could make a negative contribution to an integral, which is absurd. For more on this topic see Ref. [E5] in App..

14 c9-b.qxd 9/6/5 6:4 PM Page 8 SEC. 9.5 Numeric Integration and Differentiation 8 EXAMPLE Simpson s Rule. Error Estimate Evaluate J Solution. e x dx by Simpson s rule with m and estimate the error. Since h., Table 9.5 gives. J ( ) Estimate of error. Differentiation gives ƒ (4) (x) 4(4x 4 x )e x. By considering the derivative ƒ (5) of ƒ (4) we find that the largest value of ƒ (4) in the interval of integration occurs at and the smallest value at x* (.5.5 ) /. Computation gives the values M 4 ƒ (4) () and M 4 * ƒ (4) (x*) Since m and b a, we obtain C / Therefore, from (9),. 7 S. 5. Hence J must lie between and , so that at least four digits of our approximate value are exact. Actually, the value is exact to 5D because J (exact to 6D). Thus our result is much better than that in Example obtained by the trapezoidal rule, whereas the number of operations is nearly the same in both cases. Table 9.5 Computations in Example j x j x j e x j Sums Instead of picking an n m and then estimating the error by (9), as in Example, it is better to require an accuracy (e.g., 6D) and then determine n m from (9). EXAMPLE 4 Determination of n m in Simpson s Rule from the Required Accuracy What n should we choose in Example to get 6D-accuracy? Solution. Using M 4 (which is bigger in absolute value than M 4 *), we get from (9), with b a and the required accuracy, 6 CM 4 6, thus m [ ] 8(m) / Hence we should choose n m. Do the computation, which parallels that in Example. Note that the error bounds in (4) or (9) may sometimes be loose, so that in such a case a smaller n m may already suffice.

15 c9-b.qxd 9/6/5 6:4 PM Page CHAP. 9 Numerics in General Error Estimation for Simpson s Rule by Halving h. The idea is the same as in (5) and gives () h/ (J h/ J h ). 5 J h is obtained by using h and J h/ by using _ h, and h/ is the error of J h/. Derivation. In (5) we had _ as the reciprocal of 4 and _ (_ 4 ) resulted from h in () by replacing h with h. In () we have 5 as the reciprocal of 5 6 and _ (_ 6 )4 results from h 4 in (8) by replacing h with _ h. EXAMPLE 5 Error Estimation for Simpson s Rule by Halving Integrate ƒ(x) _ 4 x4 cos _ 4 x from to with h and apply (). Solution. The exact 5D-value of the integral is J.595. Simpson s rule gives J h _ [ƒ() 4ƒ() ƒ()] _ ( ).7448, J h/ [ƒ() 4ƒ( ) ƒ() 4ƒ( ) ƒ()] 6 [ ] Hence () gives h/ _ ( ).67 and thus J 5 J h/ h/.66, with an error.8, which is less in absolute value than _ of the error.979 of J h/. Hence the use of () was well worthwhile. Adaptive Integration The idea is to adapt step h to the variability of ƒ(x). That is, where ƒ varies but little, we can proceed in large steps without causing a substantial error in the integral, but where ƒ varies rapidly, we have to take small steps in order to stay everywhere close enough to the curve of ƒ. Changing h is done systematically, usually by halving h, and automatically (not by hand ) depending on the size of the (estimated) error over a subinterval. The subinterval is halved if the corresponding error is still too large, that is, larger than a given tolerance TOL (maximum admissible absolute error), or is not halved if the error is less than or equal to TOL. Adapting is one of the techniques typical of modern software. In connection with integration it can be applied to various methods. We explain it here for Simpson s rule. In Table 9.6 a star means that for that subinterval, TOL has been reached. EXAMPLE 6 Adaptive Integration with Simpson s Rule Integrate ƒ(x) _ 4 x4 cos _ 4 x from x to by adaptive integration and with Simpson s rule and TOL[, ].. Solution. Table 9.6 shows the calculations. Figure 44 shows the integrand ƒ(x) and the adapted intervals used. The first two intervals ([,.5], [.5,.]) have length.5, hence h.5 [because we use m subintervals in Simpson s rule (7**)]. The next two intervals ([.,.5], [.5,.5]) have length.5 (hence h.5) and the last four intervals have length.5. Sample computations. For.7448 see Example 5. Formula () gives ( )/5.6. Note that.76 refers to [,.5] and [.5, ], so that we must subtract the value corresponding to [, ] in the line before. Etc. TOL[, ]. gives

16 c9-b.qxd 9/6/5 6:4 PM Page 85 SEC. 9.5 Numeric Integration and Differentiation 85. for subintervals of length,.5 for length.5, etc. The value of the integral obtained is the sum of the values marked by an asterisk (for which the error estimate has become less than TOL). This gives J The exact 5D-value is J.595. Hence the error is.7. This is about / of the absolute value of that in Example 5. Our more extensive computation has produced a much better result. Table 9.6 Computations in Example 6 Interval Integral Error () TOL Comment [, ] [, ].794 [, ].695 Sum Divide further [.,.5].478 [.5,.].894 Sum.76*.6. TOL reached [.,.5].5876 [.5,.].658 Sum..8. Divide further [.,.5].544 [.5,.5].85 Sum.58895*.48.5 TOL reached [.5,.75].885 [.75,.].8457 Sum Divide further [.5,.65].9644 [.65,.75].99 Sum.886*..5 TOL reached [.75,.875].545 [.875,.].6578 Sum.848*..5 TOL reached f(x) Fig. 44. Adaptive integration in Example 6 x

17 c9-b.qxd 9/6/5 6:4 PM Page CHAP. 9 Numerics in General Gauss Integration Formulas Maximum Degree of Precision Our integration formulas discussed so far use function values at predetermined (equidistant) x-values (nodes) and give exact results for polynomials not exceeding a certain degree [called the degree of precision; see after (9)]. But we can get much more accurate integration formulas as follows. We set () ƒ(t) dt n A j ƒ j [ƒ j ƒ(t j )] j with fixed n, and t obtained from x a, b by setting x _ [a(t ) b(t )]. Then we determine the n coefficients A,, A n and n nodes t,, t n so that () gives exact results for polynomials of degree k as high as possible. Since n n n is the number of coefficients of a polynomial of degree n, it follows that k n. Gauss has shown that exactness for polynomials of degree not exceeding n (instead of n for predetermined nodes) can be attained, and he has given the location of the t j ( the jth zero of the Legendre polynomial P n in Sec. 5.) and the coefficients A j which depend on n but not on ƒ(t), and are obtained by using Lagrange s interpolation polynomial, as shown in Ref. [E5] listed in App.. With these t j and A j, formula () is called a Gauss integration formula or Gauss quadrature formula. Its degree of precision is n, as just explained. Table 9.7 gives the values needed for n,, 5. (For larger n, see pp of Ref. [GR] in App..) Table 9.7 Gauss Integration: Nodes t j and Coefficients A j n Nodes t j Coefficients A j Degree of Precision EXAMPLE 7 Gauss Integration Formula with n Evaluate the integral in Example by the Gauss integration formula () with n. Solution. We have to convert our integral from to into an integral from to. We set x _ (t ). Then dx _ dt, and () with n and the above values of the nodes and the coefficients yields

18 c9-b.qxd 9/6/5 6:4 PM Page 87 SEC. 9.5 Numeric Integration and Differentiation 87 ( x exp ) dx exp ( 4 (t ) ) dt [ exp ( ( ) ) exp ( ) exp ( ( ) )] (exact to 6D: ), which is almost as accurate as the Simpson result obtained in Example with a much larger number of arithmetic operations. With function values (as in this example) and Simpson s rule we would get _ ( 6 4e.5 e ).747 8, with an error over times that of the Gauss integration. EXAMPLE 8 Gauss Integration Formula with n 4 and 5 Integrate ƒ(x) _ 4 x4 cos _ 4 x from x to by Gauss. Compare with the adaptive integration in Example 6 and comment. Solution. x t gives ƒ(t) _ (t 4 )4 cos ( _ 4 (t )), as needed in (). For n 4 we calculate (6S) J A ƒ A 4 ƒ 4 A (ƒ ƒ 4 ) A (ƒ ƒ ).47855(.99.57).6545( ).595. The error is. because J.595 (6S). Calculating with S and n 4 gives the same result; so the error is due to the formula, not rounding. For n 5 and S we get J , too large by the amount. 5 because J (S). The accuracy is impressive, particularly if we compare the amount of work with that in Example 6. Gauss integration is of considerable practical importance. Whenever the integrand ƒ is given by a formula (not just by a table of numbers) or when experimental measurements can be set at times t j (or whatever t represents) shown in Table 9.7 or in Ref. [GR], then the great accuracy of Gauss integration outweighs the disadvantage of the complicated t j and A j (which may have to be stored). Also, Gauss coefficients A j are positive for all n, in contrast with some of the Newton Cotes coefficients for larger n. Of course, there are frequent applications with equally spaced nodes, so that Gauss integration does not apply (or has no great advantage if one first has to get the t j in () by interpolation). Since the endpoints and of the interval of integration in () are not zeros of P n, they do not occur among t,, t n, and the Gauss formula () is called, therefore, an open formula, in contrast with a closed formula, in which the endpoints of the interval of integration are t and t n. [For example, () and (7) are closed formulas.] Numeric Differentiation Numeric differentiation is the computation of values of the derivative of a function ƒ from given values of ƒ. Numeric differentiation should be avoided whenever possible, because, whereas integration is a smoothing process and is not affected much by small inaccuracies in function values, differentiation tends to make matters rough and generally gives values of ƒ much less accurate than those of ƒ remember that the derivative is the limit of the difference quotient, and in the latter you usually have a small difference of large quantities that you then divide by a small quantity. However, the formulas to be obtained will be basic in the numeric solution of differential equations. We use the notations ƒ j ƒ (x j ), ƒ j ƒ (x j ), etc., and may obtain rough approximation formulas for derivatives by remembering that This suggests ƒ (x) lim ƒ(x h) ƒ(x) h h*.

19 c9-b.qxd 9/6/5 6:4 PM Page CHAP. 9 Numerics in General ƒ () ƒ / ƒ ƒ /. h h Similarly, for the second derivative we obtain () ƒ ƒ ƒ ƒ ƒ, etc. h More accurate approximations are obtained by differentiating suitable Lagrange polynomials. Differentiating (6) and remembering that the denominators in (6) are h, h, h, we have x x ƒ (x) p x x x x x x x (x) ƒ ƒ ƒ. h h Evaluating this at x, x, x, we obtain the three-point formulas (a) ƒ ( ƒ 4ƒ ƒ ), h (4) (b) ƒ ( ƒ ƒ ), h (c) ƒ (ƒ 4ƒ ƒ ). h Applying the same idea to the Lagrange polynomial p 4 (x), we obtain similar formulas, in particular, (5) ƒ (ƒ 8ƒ 8ƒ ƒ 4 ). h Some examples and further formulas are included in the problem set as well as in Ref. [E5] listed in App.. h h PROBLEM SET 9.5. (Rectangular rule) Evaluate the integral in Example by the rectangular rule () with a subinterval of length... Derive a formula for lower and upper bounds for the rectangular rule and apply it to Prob.. 8 TRAPEZOIDAL AND SIMPSON S RULES Evaluate the integrals numerically as indicated and determine the error by using an integration formula known from calculus. F(x) x dx*, G(x) x dx*, x* cos x* H(x) x x*e x* dx*. F() by (), n 4. F() by (7), n 5. G() by (), n 6. G() by (7), n 7. H(4) by (), n 8. H(4) by (7), n 9 HALVING Estimate the error by halving. 9. In Prob. 5. In Prob. 6. In Prob. 7. In Prob. 8

20 c9-b.qxd 9//5 :4 PM Page 89 SEC. 9.5 Numeric Integration and Differentiation 89 9 NONELEMENTARY INTEGRALS The following integrals cannot be evaluated by the usual methods of calculus. Evaluate them as indicated. Si(x) x S(x) x sin (x* ) dx*, sin x* x* dx*, C(x) x cos (x* ) dx* Si(x) is the sine integral. S(x) and C(x) are the Fresnel integrals. (See App...). Si() by (), n 5, n 4. Using the values in Prob., obtain a better value for Si(). Hint. Use (5). 5. Si() by (7), m, m 4 6. Obtain a better value in Prob. 5. Hint. Use (). 7. Si() by (7), m 8. S(.5) by (7), m 9. C(.5) by (7), m. (Stability) Prove that the trapezoidal rule is stable with respect to rounding. 4 GAUSS INTEGRATION Integrate by () with n 5:. /x from to. cos x from to _. e x from to 4. sin (x ) from to.5 5. (Given TOL) Find the smallest n in computing the integral of /x from to for which 5D-accuracy is guaranteed (a) by (4) in the use of (), (b) by (9) in the use of (7). Compare and comment. 6. TEAM PROJECT. Romberg Integration (W. Romberg, Norske Videnskab. Trondheim, Førh. 8, Nr. 7, 955). This method uses the trapezoidal rule and gains precision stepwise by halving h and adding an error estimate. Do this for the integral of ƒ(x) e x from x to x with TOL, as follows. Step. Apply the trapezoidal rule () with h (hence n ) to get an approximation J. Halve h and use () to get J and an error estimate (J J ). If TOL, stop. The result is J J. Step. Show that.66596, hence TOL and go on. Use () with h/4 to get J and add to it the error estimate _ (J J ) to get the better J J. Calculate (J J ) (J J ). 4 5 If TOL, stop. The result is J J. (Why does 4 6 come in?) Show that we obtain.66, so that we can stop. Arrange your J- and -values in a kind of difference table. J J J J J J If were greater than TOL, you would have to go on and calculate in the next step J 4 from () with h _ 4 ; then J 4 J 4 4 with 4 (J 4 J ) J 4 J 4 4 with 4 (J 4 J ) 5 J 44 J 4 4 with 4 (J 4 J ) 6 where 6 6. (How does this come in?) Apply the Romberg method to the integral of ƒ(x) _ 4 x 4 cos _ 4 x from x to with TOL 4. DIFFERENTIATION 7. Consider ƒ(x) x 4 for x, x., x.4, x.6, x 4.8. Calculate ƒ from (4a), (4b), (4c), (5). Determine the errors. Compare and comment. 8. A four-point formula for the derivative is ƒ ( ƒ ƒ 6ƒ ƒ 4 ). 6h Apply it to ƒ(x) x 4 with x,, x 4 as in Prob. 7, determine the error, and compare it with that in the case of (5). 9. The derivative ƒ (x) can also be approximated in terms of first-order and higher order differences (see Sec. 9.): ƒ (x ) ( ƒ ƒ h ƒ 4 ƒ ). 4 Compute ƒ (.4) in Prob. 7 from this formula, using differences up to and including first order, second order, third order, fourth order.. Derive the formula in Prob. 9 from (4) in Sec. 9..

21 c9-b.qxd 9/6/5 6:4 PM Page 8 8 CHAP. 9 Numerics in General CHAPTER 9 REVIEW QUESTIONS AND PROBLEMS. What is a numeric method? How has the computer influenced numeric methods?. What is floating-point representation of numbers? Overflow and underflow?. How do error and relative error behave under addition? Under multiplication? 4. Why are roundoff errors important? State the rounding rules. 5. What is an algorithm? Which of its properties are important in software implementation? 6. Why is the selection of a good method at least as important on a large computer as it is on a small one? 7. Explain methods for solving equations, in particular fixed-point iteration and its convergence. 8. Can the Newton ( Raphson) method diverge? Is it fast? Same questions for the bisection method. 9. What is the advantage of Newton s interpolation formulas over Lagrange s?. What do you remember about errors in polynomial interpolation?. What is spline interpolation? Its advantage over polynomial interpolation?. List and compare numeric integration methods. When would you apply them?. In what sense is Gauss integration optimal? Explain details. 4. What does adaptive integration mean? Why is it useful? 5. Why is numeric differentiation generally more delicate than numeric integration? 6. Write.587, ,.64, 4.948, /, 85/ 7 in floating-point form with 5S (5 significant digits, properly rounded). 7. Compute ( )/( ) as given and then rounded stepwise to S, S, S. ( Stepwise means rounding the four rounded numbers, not the given ones.) Comment on your results. 8. Compute.97/( ) with the numbers as given and then rounded stepwise (that is, rounding the rounded numbers) to 4S, S, S. Comment. 9. Solve x 5x by (6) and by (7) in Sec. 9., using 5S in the computation. Compare and comment.. Solve x x 4 by (6) and by (7) in Sec. 9., using 5S in the computation. Compare and comment.. Let 4.8 and.75 be correctly rounded to the number of digits shown. Determine the smallest interval in which the sum (using the true instead of the rounded values) must lie.. Answer the question in Prob. for the difference What is the relative error of na in terms of that of a? 4. Show that the relative error of a is about twice that of a. 5. Compute the solution of x 5 x. near x by transforming the equation into the form x g(x) and starting from x. (Use 6S.) 6. Solve cos x x by iteration (6S, x ), writing it as x (.74x cos x)/.74, obtaining x (exact to 6S!). Why does this converge so rapidly? 7. Solve x 4 x x 4 by Newton s method with x and 6S accuracy. 8. Solve cos x x by the method of false position. 9. Compute ƒ(.8) from ƒ(.). ƒ(.).987 ƒ(.4).96 ƒ(.6).854 ƒ(.8).6967 ƒ(.).54 by linear interpolation. By quadratic interpolation, using ƒ(.), ƒ(.4), ƒ(.6).. Find the cubic spline for the data ƒ( ) ƒ() ƒ() ƒ(5) 45 k k.. Compute the integral of x from to by the trapezoidal rule with n 5. What error bounds are obtained from (4) in Sec. 9.5? What is the actual error of the result? Why is this result larger than the exact value?. Compute the integral of cos (x ) from to by Simpson s rule with m and m 4 and estimate the error by () in Sec (This is the Fresnel integral (8) in App.. with x.). Compute the integral of cos x from to _ by the three-eights rule b ƒ(x) dx h(ƒ ƒ ƒ ƒ ) a 8 (b a)h 4 ƒ (iv) (tˆ) 8 and give error bounds; here a tˆ b and x j a (b a)j/, j,,.

22 c9-b.qxd 9/6/5 6:4 PM Page 8 Summary of Chapter 9 8 SUMMARY OF CHAPTER 9 Numerics in General In this chapter we discussed concepts that are relevant throughout numeric work as a whole and methods of a general nature, as opposed to methods for linear algebra (Chap. ) or differential equations (Chap. ). In scientific computations we use the floating-point representation of numbers (Sec. 9.); fixed-point representation is less suitable in most cases. Numeric methods give approximate values a of quantities. The error of a is () a a (Sec. 9.) where a is the exact value. The relative error of a is /a. Errors arise from rounding, inaccuracy of measured values, truncation (that is, replacement of integrals by sums, series by partial sums), and so on. An algorithm is called numerically stable if small changes in the initial data give only correspondingly small changes in the final results. Unstable algorithms are generally useless because errors may become so large that results will be very inaccurate. Numeric instability of algorithms must not be confused with mathematical instability of problems ( ill-conditioned problems, Sec. 9.). Fixed-point iteration is a method for solving equations ƒ(x) in which the equation is first transformed algebraically to x g(x), an initial guess x for the solution is made, and then approximations x, x,, are successively computed by iteration from (see Sec. 9.) () x n g(x n ) (n,, ). Newton s method for solving equations ƒ(x) is an iteration ƒ(x n ) () x n x n (Sec. 9.). ƒ (xn ) Here x n is the x-intercept of the tangent of the curve y ƒ(x) at the point x n. This method is of second order (Theorem, Sec. 9.). If we replace ƒ in () by a difference quotient (geometrically: we replace the tangent by a secant), we obtain the secant method; see () in Sec. 9.. For the bisection method (which converges slowly) and the method of false position, see Problem Set 9.. Polynomial interpolation means the determination of a polynomial p n (x) such that p n (x j ) ƒ j, where j,, n and (x, ƒ ),, (x n, ƒ n ) are measured or observed values, values of a function, etc. p n (x) is called an interpolation polynomial. For given data, p n (x) of degree n (or less) is unique. However, it can be written in different forms, notably in Lagrange s form (4), Sec. 9., or in Newton s divided difference form (), Sec. 9., which requires fewer operations. For regularly spaced x, x x h,, x n x nh the latter becomes Newton s forward difference formula (formula (4) in Sec. 9.)

23 c9-b.qxd 9/6/5 6:4 PM Page 8 8 CHAP. 9 Numerics in General r(r ) (r n ) (4) ƒ(x) p n (x) ƒ r ƒ n ƒ n! where r (x x )/h and the forward differences are ƒ j ƒ j ƒ j and k ƒ j k ƒ j k ƒ j (k,, ). A similar formula is Newton s backward difference interpolation formula (formula (8) in Sec. 9.). Interpolation polynomials may become numerically unstable as n increases, and instead of interpolating and approximating by a single high-degree polynomial it is preferable to use a cubic spline g(x), that is, a twice continuously differentiable interpolation function [thus, g(x j ) ƒ j], which in each subinterval x j x x j consists of a cubic polynomial q j (x); see Sec Simpson s rule of numeric integration is [see (7), Sec. 9.5] (5) b a h ƒ(x) dx (ƒ 4ƒ ƒ 4ƒ ƒ m 4ƒ m ƒ m ) with equally spaced nodes x j x jh, j,, m, h (b a)/(m), and ƒ j ƒ(x j ). It is simple but accurate enough for many applications. Its degree of precision is DP because the error (8), Sec. 9.5, involves h 4. A more practical error estimate is (), Sec. 9.5, h/ (J h/ J h ), 5 obtained by first computing with step h, then with step h/, and then taking /5 of the difference of the results. Simpson s rule is the most important of the Newton Cotes formulas, which are obtained by integrating Lagrange interpolation polynomials, linear ones for the trapezoidal rule (), Sec. 9.5, quadratic for Simpson s rule, cubic for the three-eights rule (see the Chap. 9 Review Problems), etc. Adaptive integration (Sec. 9.5, Example 6) is integration that adjusts ( adapts ) the step (automatically) to the variability of ƒ(x). Romberg integration (Team Project 6, Problem Set 9.5) starts from the trapezoidal rule (), Sec. 9.5, with h, h/, h/4, etc. and improves results by systematically adding error estimates. Gauss integration (), Sec. 9.5, is important because of its great accuracy (DP n, compared to Newton Cotes s DP n or n). This is achieved by an optimal choice of the nodes, which are not equally spaced; see Table 9.7, Sec Numeric differentiation is discussed at the end of Sec (Its main application (to differential equations) follows in Chap..)