Chpter 5 Numericl Integrtion Numericl integrtion is the study of how the numericl vlue of n integrl cn be found. Methods of function pproximtion discussed in Chpter??, i.e., function pproximtion vi the globl interpoltion polynomil or spline interpoltion, provides bsis for numericl integrtion techniques. Let the definite integrl under considertion be I{f} = f(x)dx where [, b] is finite closed intervl. In this chpter, we primrily consider pproximtions to I{f} tht re of the form n I n+ {f} = j f(x j ) where the qudrture nodes re given by, j= x < x < x <... < x n b nd the rel coefficients j re known s the qudrture coefficients. The nodes x j re pressigned nd often eqully spced. Numericl pproximtion of definite integrls is desirble in two cses:. closed form of I{f} is not esily obtined, or. the vilble closed form of I{f} is too complicted for efficient numericl evlution, s the following exmple clerly illustrtes: [ x dt + t 4 = 4 log x + x ] + e x x + + [ ] tn x + tn x. x + x In interpoltory qudrture, the only type we will be studying in this chpter in ddition to Romberg nd dptive qudrture, we pproximte the function f(x) by the interpolting polynomil (Lgrnge form) P n (x) = n f(x j )L j (x) j= where L j (x) = (x x ) (x j x ) (x x j ) (x j x j ) (x x j+) (x j x j+ ) (x x n) (x j x n ).
Chp. 5. Numericl Integrtion CS44 Clss Notes 7 Hence, in which I n+ {f} = j = P n (x)dx = n f(x j ) j= L j (x)dx = L j (x)dx, j =,,..., n. n j f(x j ) Note tht the qudrture coefficients j re completely determined by the end points nd b nd the interpoltion nodes x j, j =,,..., n. Correspondingly, the qudrture error or trunction error is given by E n+ {f} = I{f} I n+ {f} = in which e n (x) is the interpoltion error [f(x) P n (x)]dx = j= e n (x) = (x x )(x x ) (x x n ) f (n+) (x ) (n + )! e n (x)dx where x x (x) (x, x,..., x n, x). If f(x) is polynomil of degree t most n, then f (n+) (x) = nd hence E n+ {f} =. We will use this observtion to obtin some bsic interpoltory qudrture rules. 5. Method of Exct Mtching This cn lso be clled Method of Undetermined Coefficients. 5.. The Trpezoidl Rule The trpezoidl rule uses the function vlue t two points x nd x to compute the integrl of the function in the intervl [x, x ]. For x = nd x = h, the integrl is pproximted by the trpezoidl rule I{f} = f(x)dx I {f} = f + f where the qudrture coefficients nd need to be determined. Observe tht the trpezoidl rule is exct for polynomils of degree zero nd one, i.e., for constnt nd stright line. Therefore, it is exct for the following functions:. f(x) =. f(x) = x From () we see tht nd from () we hve dx = + = h, xdx = h = h. Using these equtions, we get the vlues of the coefficients, = = h/, nd the trpezoidl rule cn be written s I {f} = h (f + f ), where h = x x. Observe tht this is lso the re of the shded trpezoid in Fig.??. Consequently, I{f} = x where E {f} is the error in trpezoidl rule. x f(x)dx = I {f} + E {f} = h [f + f ] + E {f},
Chp. 5. Numericl Integrtion CS44 Clss Notes 7 p(x) f f(x) f x = x =h Figure 5.: Trpezoidl rule. Composite Trpezoidl Rule. From the bsic trpezoidl rule we cn construct qudrture rule to compute n integrl over the intervl [, b] by dividing the intervl into N equl subintervls nd using the bsic trpezoidl rule for ech subintervl. Suppose the intervl is divided into N subintervls of equl length using the nodes x, x,..., x N, where = + ih, for i =,..., N, nd h = b N is the size of ech subintervl. Assuming f( ) = f i, the composite trpezoidl rule is given s f(x)dx = h [f + f + f + + f N + f N ] + E T, where E T is the error in the composite trpezoidl rule. h h x = x x x 3 x N- x =b N Figure 5.: Composite trpezoidl rule for equidistnt points. 5.. Simpson s Rule Simpson s rule uses 3 interpoltion nodes nd qudrtic interpolting polynomil. The generl form is I 3 {f} = f + f + f
Chp. 5. Numericl Integrtion CS44 Clss Notes 73 Since the Simpson s rule is exct for polynomils of degree zero, one, nd two, it is exct for the following functions.. f(x) =,. f(x) = x, nd 3. f(x) = x. Thus, from () we hve from (), we get nd from (3) we obtin h h h These equtions yield the qudrture coefficients Hence, the Simpson s rule is given s nd I{f} = x x dx = + + = h, xdx = h + h =, x dx = h + h = 3 h3 = = 3 h, = 4 3 h. I 3 {f} = h 3 [f + 4f + f ], f(x)dx = I 3 {f} + E 3 {f} = h 3 [f + 4f + f ] + E 3 {f}, where h = x x = x x, nd E 3 {f} is the error in Simpson s rule. f f - f f(x) p(x) x =-h x = Figure 5.3: Simpson s rule. x =h
Chp. 5. Numericl Integrtion CS44 Clss Notes 74 Composite Simpson s Rule. Now, let us derive the composite Simpson s rule for computing integrl over the intervl [, b]. Suppose we hve N equl subintervls of width h, i.e., b = hn. We lso define N + eqully spced points x j = + jh, for j =,..., N in intervl [, b]. The ith subintervl hs the endpoints nd, nd the midpoint, for i =,..., N. Applying the bsic Simpson s rule over ech subintervl, we obtin the composite rule f(x)dx = [ h 3 (f + 4f + f ) + h 3 (f + 4f 3 + f 4 ) + + h 3 (f N + 4f N + f N ) ] + E S [ ] = h N N f + 4 f i + f i + f N + E S, 3 i= where E S is the error in the composite Simpson s rule. i= h h h h x = x x x 3 x 4 x 5 x 6 x x x =b N- N- N Figure 5.4: Composite Simpson s rule. 5. The Trunction Error 5.. The Trpezoidl Rule Recll tht I{f} = f(x)dx = h [f + f ] + E {f} Let us compute the error in the trpezoidl rule for the polynomils of degree,, nd, i.e., for f(x) =, x, x. E {} = E {x} = E {x } = dx h [ + ] =, xdx h [ + h] =, x dx h [ + h ] = 6 h3.
Chp. 5. Numericl Integrtion CS44 Clss Notes 75 p(x) f f(x) f h Figure 5.5: Trunction error in trpezoidl rule. Since the error is zero for polynomils of degree or less, the trpezoidl rule is sid to hve degree of precision =. By Tylor s formul, f(x) = f + xf + x! f + x3 3! f + = (stright line) + x! f + x3 3! f + Therefore, we hve the following expression for error { } x E {f(x)} = E {f + xf } + E f + x3 6 f + Since the error for stright line is zero, the first term on the right hnd side is zero, i.e., E {f + xf } =. Thus, E {f(x)} = + f E {x } + 6 f E {x 3 } + = h3 f h4 4 f = h3 f + O(h 4 ) The bove expression of the trunction error in the trpezoidl rule is known s n symptotic error estimte. Over here, the term O(h 4 ) hs the following mening: function q(h) = O(h α ) s h, (red s q(h) is of the order h α ), mens tht there exist constnts h nd K such tht 5.. Simpson s Rule Recll tht I{f} = q(h) Kh α, < h h. h f(x)dx = h 3 [f + 4f + f ] + E 3 {f} Let us compute the error for polynomils of degree 3 or less.
Chp. 5. Numericl Integrtion CS44 Clss Notes 76 f(x) p(x) f - f f -h h Figure 5.6: Trunction error in Simpson s rule. E 3 {} = E 3 {x} = E 3 {x } = E 3 {x 3 } = h h h h dx h [ + 4 + ] =, 3 xdx h [ h + + h] =, 3 x dx h 3 [h + + h ] =, x 3 dx h 3 [ h3 + + h 3 ] =. We expected Simpson s rule to be exct for polynomils of degree or less. It turns out tht Simpson s rule is lso exct for cubics. For f(x) = x 4, E 3 {x 4 } = h x 4 dx h 3 [h4 + + h 4 ] = 4 5 h5. Therefore, Simpson s rule hs degree precision = 3. Using Tylor s formul, f(x) = f + xf + x! f + x3 3! f + x4 4! f (iv) + = (degree 3 polynomil) + x4 4 f (iv) + x5 f v + We hve shown bove tht Simpson s rule is exct for cubics. Thus, nd the symptotic error estimte is E 3 {f} = + 4 f (iv) E 3 {x 4 } + f v E 3{x 5 } + E 3 {f} = h5 9 f (iv) + O(h 7 ). We should mention here tht one cn obtin strict error estimtes for the bove two qudrture rules, rther thn just symptotic estimtes. A complete discussion of this question, however, is outside the scope of this book. We only stte the result s follows.
Chp. 5. Numericl Integrtion CS44 Clss Notes 77 Theorem 5. If n interpoltory qudrture formul hs degree of precision m, then its trunction error is given by E n+ {f} = f (m+) (z) (m + )! E n+{x m+ } where < z < b. Thus, for the trpezoidl rule m =, nd While for Simpson s rule, m = 3 nd E {f} = f (z) E {x } = h3! f (z), x < z < x. E 3 {f} = f (iv) (z) E 3 {x 4 } = h5 4! 9 f (iv) (z), x < z < x. The trunction error for the composite trpezoid rule is given by E T = h3 N f (z (b )h ) = f (z ), b = Nh, where x < z < x n. Similrly, the error for the composite Simpson s rule is given by [ ] E S = h5 f (iv) (y ) + f (iv) (y ) + + f (iv) (y N ) 9 where < y i <, for i =,,..., N. This simplifies to where < y < b. E S = h5 N 9 f (iv) (y (b )h4 ) = f (iv) (y ), 8 b = Nh 5.3 Spline Qudrture Consider the definite integrl I{f} = f(x)dx. If we pproximte f(x) vi spline interpoltion, such s the cubic interpoltory spline s(x) discussed in Chpter??, where = x < x < < x n = b, h i = +, then we obtin n interesting pproximtion of I{f} tht is given by n I sp {f} = i= xi+ s i (x)dx, where s i (x) is the cubic spline function for the ith subintervl [, + ], nd is given by s i (x) = σ i (+ x) 3 + σ ( ) i+ (x ) 3 fi+ + σ i+ h i (x ) h i h i h i ( ) fi + σ i h i (+ x) h i (see Chpter??). Let us define w = h i (x ), w = w = h i (+ x)
Chp. 5. Numericl Integrtion CS44 Clss Notes 78 Thus, s i (x) = s i (w) = h i σ i w 3 + h i σ i+ w 3 + w[f i+ σ i+ h i ] + w[f i σ i h i ] = wf i + wf i+ + h i [σ i+(w 3 w) + σ i ( w 3 w)], nd Moreover, observing tht xi+ s i (x)dx = h i w dw = s i (w)dw. w d w =, nd (w 3 w)dw = we obtin xi+ x ( w 3 w)d w = 4, s i (x)dx = h i [f i + f i+ ] h3 i 4 [σ i + σ i+ ]. Note tht this eqution cn be regrded s the trpezoidl rule plus correction term. This correction term is given by τ i = h3 4 [s ( ) + s (+ )], which indictes tht if f (x) is not too bdly behved, then In other words if f (x) is well behved, we hve τ i h3 i f (θ) E {f}, < θ < +. xi+ which cn be remrkbly close to the integrl s i (x)dx = h i [f i + f i+ ] + E {f} xi+ f(x)dx. 5.4 Richrdson s Extrpoltion In mny clcultions wht one would relly like to know is the limiting vlue of certin quntity F (h) s F. Needless to sy, the work required for computing F (h) increses shrply s h pproches zero. Furthermore, the effects of rounding errors set prcticl limit on how smll h cn be chosen. Usully, one hs some knowledge of how the trunction error [F () F (h)] behves s h. Let F (h) = F () + α h p + O(h r ) where r > p, nd α is n unknown. Compute F for two step lengths: h nd qh, (q > ) F (h) = F () + α h p + O(h r ) F (qh) = F () + α (qh) p + O(h r ). Multiplying the first eqution by q p, the second by, nd dding, we get p F (h) F (qh) = [q p ]F () + O(h r ),
Chp. 5. Numericl Integrtion CS44 Clss Notes 79 or F () = [ F (h) + ] F (h) F (qh) q p + O(h r ). This simple technique known s Richrdson s extrpoltion improves the symptotic error bound from O(h p ) to O(h r ). The repeted ppliction of Richrdson s extrpoltion to numericl integrtion is known s Romberg Integrtion. As n illustrtion, consider the composite trpezoidl rule on N pnels, ech of width h, In Appendix??, we show tht T (h) = h f + N j= f j + f N. I{f} I = T (h) + α h + α h 4 + α 3 h 6 + Using pnels of hlf the width, we get ( ) ( ) ( ) ( ) h h h 4 h 6 I = T + α + α + α 3 + 4 6 64 Multiplying this eqution by /4 nd dding to the previous eqution, we obtin [ ( ) 3 h 4 I = T 4 ] T (h) 3 6 α h 4 or I = [ ( ) h 4T 3 [ ( ) h = T + 3 ] T (h) ( T ( h 4 α h 4 + O(h 6 ) (5.) ) )] T (h) + O(h 4 ) (5.) Suppose b = Nh, nd let us denote the composite formul with N pnels by T N, nd the formul with N pnels by T N. Thus, ( ) h T N = T (h), T N = T. The formul using Richrdson s extrpoltion is given by Now, eqution?? cn be written s Similrly in which T () N = 3 (4T N T N ) = T N + 3 (T N T N ). I = T () N 4 α h 4 + O(h 6 ). (5.3) I = T () 4N 4 α ( ) 4 h + O(h 6 ), (5.4) T () 4N = 3 (4T 4N T N ) = T 4N + 3 (T 4N T N ). Eliminting α from equtions (??) nd (??), we hve I = 5 = [ 6T () 4N T () N [ T () 4N + 5 = T () 4N + O(h6 ). ] + O(h 6 ) ( T () 4N T () N ) ] + O(h 6 )
Chp. 5. Numericl Integrtion CS44 Clss Notes 8 If we systemticlly hlve the intervl, tking h, h, h 4,..., etc., we cn construct the tble of which section is shown below. T N T () N T N T () 4N T 4N T () 4N T () 8N T () 8N T (3) 8N T 8N Error : O(h ) O(h 4 ) O(h 6 ) O(h 8 ) In generl, I = T (j) + O(h r ) in which r = (j + ), nd T (j) i = 4j T (j ) i (4 j ) T (j ) i = T (j ) i + T (j ) i T (j ) i (4 j ). Exmple 5. Compute Solution.8 sin x dx using Romberg s integrtion. x h.8 T =.75868 T () =.77.4 T =.76876,. T 4 =.776, T () 4 =.7795 T () 4 =.7796, T () 8 =.7795,. T 8 =.77887, T () 8 =.7795 5.5 Adptive Qudrture An dptive qudrture lgorithm uses one or two bsic qudrture rules, nmely, the trpezoidl rule nd Simpson s rule, nd determines the subintervl sizes so tht the computed result meets some prescribed ccurcy requirement. In this wy, n ttempt is mde to provide result with the prescribed ccurcy t the lowest cost possible. By cost we men computer time, which in turn is directly proportionl to the number of function evlutions necessry to obtin the result. The user of n dptive qudrture routine specifies the intervl [, b], provides subroutine which computes the function f(x) for ny x [, b], nd chooses tolernce ɛ. The dptive routine ttempts to compute quntity Q such tht Q f(x)dx ɛ.
Chp. 5. Numericl Integrtion CS44 Clss Notes 8 b Smll intervl size used in these regions Figure 5.7: Adptive qudrture. Exmple 5. Consider the problem of pproximtion I = prescribed tolernce ɛ. H f(x)dx using n dptive Simpson s rule with H/ H/4 H/8 Figure 5.8: Division of intervl [, b] for dptive qudrture. We first pproximte I using the bsic Simpson s rule over the intervl [, b], i.e., using one pnel, thus P = H [ ( f() + 4f + H ) ] + f(b). 6 Divide [, b] into two equl subintervls, nd pply Simpson s rule to ech. For the left hlf we get P = H [ ( f() + 4f + H ) ( + f + H )], 4 nd for the right hlf we obtin P = H Note tht we need only compute f ( + H 4 the previous level. Now, compute [ ( f + H ) + 4f ( + 34 ) ] H + f(b). ) nd f ( + 3 4 H), since the other vlues of f re vilble from Q = P + P b
Chp. 5. Numericl Integrtion CS44 Clss Notes 8 nd compre with P. If P Q ɛ for prescribed tolernce ɛ, report Q s the desired pproximtion to I. If not, set the right hlf [ + H, b] side for the moment nd proceed in the sme wy with the left hlf. In other words, compute P = H [ ( f() + 4f + H ) ( + f + H )], 4 8 4 P = H [ ( f + H ) + 4f ( + 38 ) ( 4 4 H + f + H )]. We ccept (P + P ) s n pproximtion to +H/ f(x)df P (P + P ) ɛ, nd repet the process for the right hlf [ + H, b]. Thus, we hve constructed the following tree Level P : [, b] Level P : [, + H ] P : [ + H, b] Level 3 P : [, + H 4 ] P : [ + H 4, + H ] P : [ + H, + 3H 4 ] P : [ + 3H 4, b] If the bove test fils, however, set the right hlf of [, + H ] side nd repet the process for the left hlf [, + H 4 ]. In prctice, we hve to limit the number of levels used since the number of function evlutions increses rpidly. Hence, when we rech the mximum level permitted the lst pproximtion is ccepted nd we move to the right. In generl, let us ssume tht on n intervl [, + ] the bsic Simpson s rule yields n pproximtion S i to the true vlue I i = + f(x)dx. Then where < z i < + nd h i = +. Let S (L) i I i S i = (h i/) 5 f (iv) (z i ) (5.5) 9 nd S (R) i [, + hi ] nd [ + hi, +], respectively. Also, let Q i = S (L) i be the results of the bsic Simpson s rules on + S (R) i, then I i Q i = 4 (h i/) 5 f (iv) (y i ) (5.6) 9 where < y i < +. Assuming tht h i is smll enough so tht f (iv) x) is essentilly constnt, i.e., f (iv) (η) = f (iv) (y i ) = f (iv) (z i ), equtions (??) nd (??) yield or Q i S i = (h i /) 5 9 f (iv) (η) [ ] 4 I i = Qi + 4 [Q i S i ]. (5.7) Note tht (??) is one step of Richrdson s extrpoltion. In fct, similr to Appendix??, we cn show tht I i = S i + β h 4 i + β h 6 i ( ) + + 4 ( ) 6 hi hi I i = Q i + β + β + nd I i = Q i + 4 [Q i S i ] + O(h 6 i ).
Chp. 5. Numericl Integrtion CS44 Clss Notes 83 Therefore, the bsic tsk of typicl routine is to bisect ech subintervl until the following inequlity is stisfied S i Q i 4 h i b ɛ where ɛ is the user specified tolernce. To see this, let Thus, Q Q = I = N i= N i= f(x)dx Q i, I i = = N i= xi+ f(x)dx. N (Q i I i ) i= N i= N i= Q i I i 4 Q i S i N ɛ h i = ɛ, b which is the desired gol! In this exmple we hve ssumed tht the routine is using n bsolute error criterion Q f(x)dx ɛ. It is usully preferble to use the reltive error tolernce Q b f(x)dx i= f(x) dx σ. Finlly, we would like to note tht it is not difficult to construct function f(x) for which given dptive qudrture routine fils. For exmple, the bove dptive Simpson s rule fils for the integrl I = b f(x)dx, f(x) = x (x ) (x ) (x 3) (x 4). In this cse, P =, nd both P = nd P = becuse f(x) hs roots t x =,,, 3, nd 4. And since P = P + P =, the qudrture rule will ssume the nswer to be ccurte! 5.6 Some Difficulties in Numericl Integrtion Discontinuous functions. When f(x) is hs discontinuity t the point c lying within the intervl [, b], we cn split the integrl s shown below f(x)dx = f(x)dx + c f(x)dx.
Chp. 5. Numericl Integrtion CS44 Clss Notes 84 f(x) c b Figure 5.9: Discontinuous function. Exmple 5.3 Compute the integrl I = f(x)dx for the pulse function { x /3 x < /3 f(x) = /3 - Figure 5.: Pulse function. Without splitting, the dptive Simpson s procedure converges slowly: P = /3, P = /3, P = /, P P + P, nd P = /4, P = /6, P P + P,... nd so on. On the other hnd, we get immedite convergence by splitting the integrl I = /3 f(x)dx + f(x)dx. /3
Chp. 5. Numericl Integrtion CS44 Clss Notes 85 Integrtion over infinite intervl. When the rnge of integrtion is infinite, i.e., I = we cn resort to chnge of vribles. Let y = x, then Exmple 5.4 Compute the infinite integrl Let y = x I = I = f(x)dx, >. / f e x ( ) dy y y. x dx. dy ; then x = y =, nd x = y =. Moreover, dx = y. Hence, I = e /y y dy = g(y)dy, where we cn esily evlute g(y) for ny y >. For y = we see tht e /y lim y y = lim y y( + y +! y + ) =. Singulrity. Singulrity t point c [, b] is tckled s follows: We split the function into two prts I = f(x)dx = f (x)dx + f (x)dx, where f (x) is smooth nd cn be integrted using n dptive procedure, nd f (x) contins the singulrity but cn be integrted nlyticlly. Exmple 5.5 Compute the integrl which hs singulrity t x =. Clerly, the function cn be split s follows I = cos x x dx cos x x = cos x x + x = f (x) + f (x). Even though f (x) hs singulrity t x =, its integrl cn be obtined nlyticlly, f (x)dx = x dx =. Integrl of f (x) cn now be hndled by n dptive qudrture routine provided we observe tht )] cos x lim = lim x [ + ( x x x x! + x4 4! x6 6! +... ] = lim [ x3/ + x7/ x/ + x! 4! 6! =, nd in the neighborhood of x =, cos x x.
Chp. 5. Numericl Integrtion CS44 Clss Notes 86 f(x) c b Figure 5.: Singulr function. 5.7 Appendix: Numericl Integrtion We wish to show tht the trunction error in trpezoidl rule is given by the following I{f} = f(x)dx = T (h) + α h + α h 4 + α 3 h 6 + where T (h) = h [ f + N i= f i + f N ], h = b N. f(x) x Figure 5.: Grph of f(x) = cos x x.
Chp. 5. Numericl Integrtion CS44 Clss Notes 87 Consider the pnel [, + ] with the midpoint y i = ( + + )/. Then Observing tht where h = +, then f(x) = f(y i ) + (x y i )f (y i ) + (x y i) xi+! (x y i ) m dx = f (y i ) + (x y i) 3 f (y i ) + 3! h m = m = h 3 m = m = 3 h 5 8 m = 4 However, Hence, xi+ f(x)dx = hf(y i ) + h3 4 f (y i ) + h5 9 f (iv) (y i ) + (5.8) f( ) = f(y i ) h f (y i ) + h 8 f (y i ) h3 48 f (y i ) + h4 384 f (iv) (y i )... f(+ ) = f(y i ) + h f (y i ) + h 8 f (y i ) + h3 48 f (y i ) + h4 384 f (iv) (y i ) + T i = h [f() + f(+ )] = hf(y i ) + h3 8 f (y i ) + h5 384 f (iv) (y i ) + Substituting in (??), we obtin Consequently, xi+ f(x)dx = f(x)dx = T i h3 f (y i ) h5 48 f (iv) (y i ) N in which < η, η < b. In other words, i= xi+ f(x)dx = T (h) h (b )f (η ) h4 48 (b )f (iv) (η ) I{f} = T (h) + α h + α h 4 +