Capter 2 Integrals as sums and derivatives as differences We now switc to te simplest metods for integrating or differentiating a function from its function samples. A careful study of Taylor expansions reveals ow accurate te constructions are. 2.1 Numerical integration Consider a function f we ll specify wic assumptions we need to make about it in a minute. Let us reformulate te integral 1 f(x) dx by breaking up te interval [a, b] into subintervals [, x j ], wit x j = j, = 1/N, and = x < x 1 <... < x N = 1. Togeter, te (x j ) are called a Cartesian grid. Still witout approximation, we can write 1 t N x j f(x) dx = f(x) dx. j=1 A quadrature is obtained wen integrals are approximated by quantities tat depend only on te samples f(x j ) of f on te grid x j. For instance, we can 1
CHAPTER 2. INTEGRALS AS SUMS AND DERIVATIVES AS DIFFERENCES approximate te integral over [x j 1, x j ] by te signed area of te rectangle of eigt f(x j 1 ) and widt : Putting te terms back togeter, we obtain te rectangle metod: (Insert picture ere) To understand te accuracy of tis approximation, we need a detour troug Taylor expansions. Given a very smoot function f(x), it sometimes makes sense to write an expansion around te point x = a, as 1 1 f """ (a)(x a) 3 f(x) = f(a) + f " (a)(x a) + f "" (a)(x a) 2 + +... 2 3! As a compact series, tis would be written as f(x) dx f( ). f(x) dx t 1 f(x) = f (n) (a)(x a) n, n! n= N f( ). were n! = n (n 1)... 2 1 is te factorial of n. Te word sometimes is important: te formula above requires te function to be infinitely differentiable, and even still, we would need to make sure tat te series converges (by means of te analysis of te previous capter). Te radius of convergence of te Taylor expansion is te largest R suc tat, for all x suc tat x a < R, te series converges. It is also possible to truncate a Taylor series after N terms, as N t 1 1 f(x) = f (n) (a)(x a) n + 1 f (N) (y)(x a) N, (2.1) n! N! n= were y is some number between x and a. Te formula does not specify wat te number y is; only tat tere exists one suc tat te remainder takes te form of te last term. In spite of te fact tat y is unspecified, tis is a muc more useful form of Taylor expansion tan te infinite series: j=1 2
2.1. NUMERICAL INTEGRATION We often only care about te size of te remainder, and write inequalities suc as ( ) 1 1 f (N) (y)(x a) N max f (N) (z) x a N, N! N! z [x,a] or, for sort, 1 f (N) (y)(x a) N = O( x a N ), N! were all te constants are idden in te O. So, te remainder is on te order of x a N. Te limit implicit in te O notation is ere x a. Te finite-n formula is valid wenever te function is N times differentiable, not infinitely differentiable! In fact, te Taylor series itself may diverge, but equation (2.1) is still valid. j In order to study te problem of approximating f(x) dx, let us expand f in a (sort, truncated) Taylor series near x = j 1 : f(x) = f( ) + f " (y(x)), were y(x) [, x]. We ve written y(x) to igligt tat it depends on x. Tis works as long as f as one derivative. Integrating on bot sides, and recognizing tat f( ) is constant wit respect to x, we get (recall x j = ) f(x) dx = f(x j 1) + f (y(x)) dx. We don t know muc about y(x), but we can certainly write te inequality x j x j y [,x j ] x j y [,x j ] f (y(x)) dx max f (y) 1 dx = 2 max f (y). 1 So, as long as f is differentiable, we ave f(x) dx = f(x j, 1) + O( 2 ) 1 x x 3
CHAPTER 2. INTEGRALS AS SUMS AND DERIVATIVES AS DIFFERENCES were te derivative of f ides in te O sign. Te integral over [, 1] as N suc terms, so wen tey are added up te error compounds to N O( 2 ) = O() (because = 1/N). Tus we ave just proved tat f(x) dx r Nt f( ) + O(). j=1 Te error is on te order of te grid spacing itself, so we say te metod is first-order accurate (because te exponent of is one.) Coosing te (signed) eigt of eac rectangle from te left endpoint does not sound like te most accurate ting to do. Evaluating te function instead at /2 = (j 1/2) is a better idea, called te midpoint metod. It is possible to sow tat Nt f(x) dx r f(/2 ) + O( 2 ), j=1 provided te function f is twice differentiable (because f "" ides in te O sign). Te accuracy is improved, since 2 gets muc smaller tan as. We say te midpoint metod is second-order accurate. Anoter solution to getting order-2 accuracy is to consider trapezoids instead of rectangles for te interval [, x j ]. Te area of te trapezoid spanned by te 4 points (, ); (, f( )); (x j, f(x j )); (x j, ) is (f( )+ f(x j ))/2. Tis gives rise to te so-called trapezoidal metod, or trapezoidal rule. (Insert picture ere) Let us compare tis quantity to te integral of f over te same interval. Consider te truncated Taylor expansions f(x) = f( ) + f " ( )(x ) + O( 2 ), f(x j ) = f( ) + f " ( ) + O( 2 ), were te second derivative of f appears in te O sign. Integrating te first relation gives 2 f(x) dx = f( ) + f " ( ) + O( 3 ). 2 4
2.1. NUMERICAL INTEGRATION Te area of te trapezoid, on te oter and, is (using te second Taylor relation) (f(xj 1 ) + f(x j )) = f( ) + 2 f " ( ) + O( 3 ). 2 2 Tose two equations agree, except for te terms O( 3 ) wic usually differ in te two expressions. Hence f(x) dx = (f( ) + f(x j )) + O( 3 ). 2 We can now sum over j (notice tat all te terms appear twice, except for te endpoints) to obtain N 1 t f(x) dx = f() + f(x j ) + f(1) + O( 2 ). 2 2 j=1 Te trapezoidal rule is second-order accurate. All it took is a modification of te end terms to obtain O( 2 ) accuracy in place of O(). Example: f(x) = x 2 in [, 1]. We ave 1 x 2 dx = 1/3. For te rectangle rule wit N = 4 and = 1/4, consider te gridpoints x =, 1/4, 1/2, 3/4, and 1. Eac rectangle as eigt f( ) were is te left endpoint. We ave 1 1 1 3 2 14 x 2 dx r + + + = =.2188... 4 4 2 2 2 4 2 64 Tis is quite far (O(), as we know) from 1/3. For te trapezoidal rule, consider te same grid. We now also consider te grid point at x = 1, but te contribution of bot x = and x = 1 is alved. 1 1 1 1 3 2 1 22 x 2 dx = + + + + 1 = =.3438... 4 2 4 2 2 2 4 2 2 64 Tis is muc closer (O( 2 ) as a matter of fact) from 1/3. We ll return to te topic of numerical integration later, after we cover polynomial interpolation. 5
CHAPTER 2. INTEGRALS AS SUMS AND DERIVATIVES AS DIFFERENCES 2.2 Numerical differentiation Te simplest idea for computing an approximation to te derivative u " (x j ) of a function u from te samples u j = u(x j ) is to form finite difference ratios. On an equispaced grid x j = j, te natural candidates are: te one-sided, forward and backward differences u j+1 u j u j u j 1 Δ + u j =, Δ =, and te two-sided centered difference u j+1 u j 1 Δ u j =. 2 Let us justify te accuracy of tese difference quotients at approximating derivatives, as. Te analysis will depend on te smootness of te underlying function u. Assume witout loss of generality tat =, x j =, and x j+1 =. For te forward difference, we can use a Taylor expansion about zero to get "" u() = u() + u " () + u (ξ), ξ [, ]. 2 (Te greek letter ξ is xi.) Wen substituted in te formula for te forward difference, we get u() u() = u " () + u "" (ξ). 2 Te error is a O() as soon as te function as two bounded derivatives. We say tat te forward difference is a metod of order one. Te analysis for te backward difference is very similar. For te centered difference, we now use two Taylor expansions about zero, for u() and u( ), tat we write up to order 3: 2 2 3 """ (ξ), u(±) = u() ± u " () + u "" () ± u 2 6 wit ξ eiter in [, ] or [, ] depending on te coice of sign. Subtract u( ) from u() to get (te 2 terms cancel out) u() u( ) 2 """ = u " () + u (ξ). 2 3 6
2.2. NUMERICAL DIFFERENTIATION Te error is now O( 2 ) provided u is tree times differentiable, ence te metod is of order 2. Te simplest coice for te second derivative is te centered second difference u j+1 2u j + u j 1 Δ 2 u j =. 2 It turns out tat Δ 2 = Δ + Δ = Δ Δ +, i.e., te tree-point formula for te second difference can be obtained by calculating te forward difference of te backward difference, or vice-versa. Δ 2 is second-order accurate: te error is O( 2 ). To see tis, write te Taylor expansions around x = to fourt order: 2 3 4 """" (ξ). u(±) = u() ± u " () + u "" () ± u """ () + u 2 6 24 wit ξ eiter in [, ] or [, ] depending on te coice of sign. Te odd terms all cancel out wen calculating te second difference, and wat is left is u 2 j+1 2u j + u j 1 """" = u "" () + u (ξ). 2 12 So te metod really is O( 2 ) only wen u is four times differentiable. Example: f(x) = x 2 again. We get f " (x) = 2x and f "" (x) = 2. (x + ) 2 x 2 2x + 2 Δ + f(x) = = = 2x +. x 2 (x ) 2 2x 2 Δ f(x) = = = 2x. (x + ) 2 (x ) 2 4x Δ f(x) = = = 2x. 2 2 (x + ) 2 2x 2 + (x ) 2 2 2 Δ 2 f(x) = = = 2. 2 2 Te error is manifestly for te forward difference, and for te backward difference (bot are O().) Coincidentally, te error is zero for Δ and Δ 2 : centered differences differentiate parabolas exactly. Tis is an exception: finite differences are never exact in general. Of course, = O( 2 ) so tere s no contradiction. Wen u as less differentiability tan is required by te remainder in te Taylor expansions, it may appen tat te difference scemes may ave lower 7
CHAPTER 2. INTEGRALS AS SUMS AND DERIVATIVES AS DIFFERENCES order tan advertised. Tis appens in numerical simulations of some differential equations were te solution is discontinuous, e.g., in fluid dynamics (interfaces between pases). Sometimes, points on te left or on te rigt of x j are not available because we are at one of te edges of te grid. In tat case one sould use a one-sided difference formula (suc as te backward or forward difference.) You can find tables of ig-order one-sided formulas for te first, second, and iger derivatives online. Te more systematic way of deriving finite difference formulas is by differentiating a polynomial interpolant. We return to te topics of integration and differentiation in te next capter. 8
MIT OpenCourseWare ttp://ocw.mit.edu 18.33 Introduction to Numerical Analysis Spring 212 For information about citing tese materials or our Terms of Use, visit: ttp://ocw.mit.edu/terms.