Lecture notes for Numericl Anlysis Integrtion Topics:. Problem sttement nd motivtion 2. First pproches: Riemnn sums 3. A slightly more dvnced pproch: the Trpezoid rule 4. Tylor series (the most importnt technique in numericl nlysis) 5. Error nlysis of Trpezoid rule 6. Rtes of convergence 7. A look further field: Newton-Cotes formuls 8. Notes on reding nd further topics. Problem sttement nd motivtion The gol of this section is to lern how to use computer to pproximte definite integrls, i.e. expressions of the form f(x)dx () for finite rel numbers nd b nd smooth rel-vlued functions f. It is worth tking moment to reflect on the question of why we might wnt to use computer to compute definite integrls. After ll, in first yer clculus, we lern how to solve definite integrl using nti-derivtives. Wht is wrong with the nti-derivtive pproch? And in rel life, why would we ctully wnt to solve definite integrls nywy? One nswer to the question of wht s wrong with the nti-derivtive pproch is tht not ll functions hve nti-derivtives. For exmple, the clssic Bell curve hs the functionl form f(x) = p 2 2 e (x µ)2 /2 2 (2) for constnts µ nd. Bell curves cn be used to describe mny kinds of dt, nd show up ll the time in the nturl sciences, but the ntiderivtives of these functions hve no nlyticlly trctble form. Even when integrnds do hve nice nti-derivtes, finding them cn be lborious nd time consuming, wheres numericl methods tend to be esy, quick, nd ccurte. Thus nother nswer to the wht s wrong with the nti-derivtive pproch question is tht it cn ctully be much more cumbersome thn numericl pproch. The question of why we would wnt to solve definite integrls nywy hs myrid nswers. Here re few exmples. Exmple. (Probbility). A density function is positive functions f(x) : R! [0, ) such tht Z f(x)dx =. A rndom vrible X is quntity whose vlue is determined by some rndom process (e.g. X could be the mount of money gined or lost in tomorrow s stock mrket.) A rndom vrible X is sid to be described by density function f if, for ny intervl [, b], the probbility tht X lies in [, b] is given by P ( pple X pple b) = f(x)dx. In other words, probbilities re computed by solving definite integrls. Applictions re ubiquitous, nd include engineering (e.g. the probbility of system filure for geosttionry stellites), biology (e.g. the probbility of genetic survivl), chemistry (the probbility of bond forming), nd mny other fields.
.2 First pproches: Riemnn sums Lecture notes for Numericl Anlysis Exmple.2 (Bridge design). Bridges often employ cbles hung in rcs from tll supports. Clculting the length of the cble is importnt for cost estimtes. While function f(x) might be produced tht describes the position of the cble, the length of the cble needs to be ttined by computing n rc-length integrl..2 First pproches: Riemnn sums There re mny wys to evlute definite integrls. Before lerning bout these techniques, it is useful to see wht you cn come up with yourself. Clssroom Activity: Together with prtner, brinstorm bout how you might go bout pproximting definite integrl. Assume tht the tools t your disposl re computer the cpcity to evlute f(x) t ny x 2 [, b]. It s likely tht you stumbled cross the ide of using Riemnn sum to estimte the integrl. Recll tht one wy to pproximte the definite integrl is vi the right hnd endpoint rule. Definition.. The right hnd endpoint pproximtion of R b f(x)dx is f(x)dx h f(x i ), x i = + ih, h =(b )/n. (3) i= For fixed n, the right hnd side is n pproximtion to the left. When n =, the pproximtion hs the form (b )f(b). This pproximtion should get better s n gets lrger, nd indeed converge s n!. A few nturl questions emerge from this nive pproch, however: how lrge should n be in order to chieve given ccurcy? how should tht ccurcy be mesured? how long will the process tke? cn n be mde rbitrrily lrge, or is there threshold beyond which my mchine will blk? is there better pproch? We will ddress these nd other questions in wht follows..3 The Trpezoid Rule One thing to note bout formul (3) is tht its clcultionl requirements re s follows: n function evlutions n dditions multipliction Clssroom Activity: Are there other wys to implement this lgorithm? If so, re they more or less e cient? Discuss. Inccurcies in this formul might rise from severl plces. 2
.4 Tylor Series Lecture notes for Numericl Anlysis Clssroom Activity: Where might these inccurcies come from? Discuss. Ides: roundo error in representing numbers on the computer mismtch between f(x) nd the pproximtion f(x i ). One of the problems with the right hnd endpoint rule is tht it tries to cpture the verge vlue of f(x) on the intervl [x i,x i+ ] vi smple tht is chosen to lie t n extreme endpoint of this intervl. A more resonble pproch is to figure out wht f(x) is t the left hnd endpoint nd the right hnd endpoint, nd form the verge. This is the content of the Trpezoid rule. The single trpezoid pproximtion to R b f(x)dx is (b ) f()+f(b), 2 which essentilly tries to pproximte f(x) on [, b] vitheverge of the endpoints. If we divide [, b] into n subintervls nd pply the single trpezoid rule to ech, we get the following: Definition.2. The trpezoid rule pproximtes I = R b f(x)dx vi the formul Z " # b f(x)dx b f()+2 f(x i )+f(b), x i = + ih, h =(b )/n. (4) 2 i= Clssroom Activity: Prove this formul. The trpezoid rule seems more promising thn the right-hnd endpoint rule, in tht we ve mde more intelligent choice for our pproximtion of f(x) in the intervl [x i,x i+ ]. The computtionl requirements re bout the sme: n + function evlutions n + dditions 3 multiplictions In order to figure out if the trpezoid rule relly bets the right hnd endpoint rule, however, we need to do some hrd mthemticl nlysis. Our tool for this nlysis is clled Tylor series..4 Tylor Series The bsic ide behind Tylor series is tht it is possible to pproximte n rbitrry function f(x) with polynomil, s long s f(x) is smooth (see below.) The pproximtion tends to get better s the degree of the pproximting polynomil gets lrger, nd there re forml bounds on how lrge the pointwise error cn be. To stte Tylor s Theorem, let s first define wht we men by smooth function. Definition.3. A function is sid to be of clss C f(x) :! R if f(x) nd ll of its derivtives re defined on the domin. Note: functions in C re relly nice you cn di erentite them s much s you wnt, nd you never run into problems. In pplictions, is generlly tken to be some finite intervl [, b], but it doesn t hve to be. In stting Tylor s theorem (below), I tke = R, just to simplify things. Tylor s theorem cn be extended to other clsses of functions, but this one is good plce to strt. 3
.5 Error nlysis of Trpezoid Rule Lecture notes for Numericl Anlysis Theorem.. Suppose f(x) 2CR. Then for ny point 2 R, the vlue of f t the point x cn be expressed s f(x) =f()+f 0 ()(x )+f 00 (x )2 () + f 000 (x )3 () +. 2! 3! If T n (x) represents the first n + terms of the right hnd side nd R n+ (x) represents the reminder, then we hve f(x) =T n (x)+r n+ (x), where the reminder is of the form R n+ (x) =f (n+) ( ) (x )n+ n! for some between nd x. Nottion: Sometimes will write T n (f,,x) nd R n+ (f,,x) if we wish to emphsize the dependence of these terms on f nd. Clssroom Activity: Find T 2 (x) iff(x) =sin(x) nd =. Then find n expression for R 3 (x). Finlly, suppose you use T 2 (x) to estimte sin(5 /4). Clculte bound on the error..5 Error nlysis of Trpezoid Rule At first blush, it s not totlly obvious why Tylor s theorem is so importnt. One wy to think bout it is tht it llows us to use simple polynomil pproximtions to clculte how big errors will be. Those pproximtions will hve their own errors, of course, but the explicit form of the reminder term in the Tylor Theorem llows us to get bounds on these errors. As n exmple of the Tylor Series in ction, we cn clculte bound on the errors involved in the Trpezoid method. Theorem.2. Suppose f(x) 2 C 2 [,b]. Suppose we use the single intervl trpezoid pproximtion to pproximte Then the error of the pproximtion stisfies where Proof. See clss notes for detils. Ĩ =(b I = ) f()+f(b) 2 f(x)dx. I Ĩ pple(b ) 3 M, M = sup f 00 (x) x2[,b] Corollry.. Suppose f(x) 2 C 2 [,b], nd we use the trpezoid pproximtion with n intervls " # n (b ) X Ĩ = f()+2 f(x i )+f(b) n to pproximte I = i f(x)dx. Then the error of the pproximtion stisfies 3 b I Ĩ pple Mn = h 2 M(b n where M is s bove nd h =(b ), )/n is the spcing between smple points. 4
.6 Convergence rtes Lecture notes for Numericl Anlysis Proof. Apply Theorem (.2) to ech subintervl, sum the errors, nd simplify. In other words, we hve used Tylor series to show tht if we use the Trpezoid rule on smooth enough function, the error decreses in proportion to the squre of the spce between smple points. This is good news: it mens, mong other things, tht if we decrese tht smple spcing, we decrese the error. Not only tht, the fct tht the error decreses in proportion to the squre of this spcing mens we don t hve to decrese it very much to see big improvement. For exmple, if we double the number of smple points (nd thus decrese h by /2), we will decrese the error by fctor of four (i.e. (/2) 2.) It will turn out tht other lgorithms yield other error formuls tht involve di erent powers of h. To tlk cogently bout how these vrious lgorithms stck up to one nother, we need to define wht we men by convergence rte..6 Convergence rtes Definition.4. Suppose n is sequence converging to, nd if there exists positive k nd n integer N such tht n > 0 is sequence converging to 0. Then n pplek n 8n>N, we sy n converges to with rte of convergence O( n ) (sy big-oh n ), nd write n = + O( n ). If it hppens tht we cn choose sequence K n > 0 such tht K n! 0 nd n pplek n n, we sy n! little-oh n nd write n = + o( n ). Note: The definition is frmed for generl n, but in relity the cse of interest is when n =/n p for some p>0. In generl, we re interested in knowing wht the lrgest p is for which the convergence is O(/n p ). Exmple.3. Let n =(n + )/n 2. Set It follows tht n converges to 0 with O(/n). n =/n. Then n 0 = n + n 2 pple 2n n 2 = 2 n. The sequence definition of convergence rte cn esily be extended to functions: Definition.5. Suppose If there exists K>0 nd lim G(h) =0, lim F (h) =L. h!0 h!0 >0 such tht F (h) L pplek G(h) 8h <, then we write F (h) =L + O(G(h)). Once gin, the cse of interest is when G(h) =h p,p>0. Exmple.4. Let F (h) =( cos h)/h. The second order Tylor pproximtion of cos h ner =0is h 2 /2+R 3 (h), nd by the Tylor reminder theorem, R 3 is O(h 3 ).Itfollowstht cos h h = ( h2 /2+O(h 3 )) h = h + O(h 2 )=h( + O(h)) Since nything O(h) will eventully decrese below s h! 0, we cn set K =2nd tke G(h) =h. Then F (h) pple KG(h) for ll su ciently smll h, sof (h)! 0 O(h). 5
.7 A look further field: Newton-Cotes formuls Lecture notes for Numericl Anlysis.7 A look further field: Newton-Cotes formuls As we hve seen, the right hnd endpoint rule tries to pproximte f(x) in n intervl vi horizontl line tht hppens to mtch f(x) t one point in the intervl, while the trpezoid rule tries to mtch f(x) in n intervl with stright line tht mtches f(x) t two points in the intervl. This suggests n ide: if we choose n + points in n intervl, we cn fit n n-degree polynomil to them, nd use tht polynomil s our pproximtion to f(x) over the intervl. Bsed on the Tylor Reminder theorem, we expect the pproximtion to improve s the degree of the pproximting polynomil increses. Formuls derived this wy re clled Newton-Cotes formuls when the n + smple points re uniformly spced. There re two kinds of Newton-Cotes, closed nd open. The closed formuls include the endpoints of the intervl s smple points, the open formuls do not. But in essence, the wy to develop Newton-Cotes formul is s follows: Algorithm How to develop Newton-Cotes formul. Choose n + smple points in [, b]. For closed formuls, these points should be x i, i =0,,n, x i = + ih, h =(b )/n nd for open formuls they should be x i, i =0,,n, x i = +(i + )h, h =(b )/(n + 2) 2. Find the unique n degree polynomil p n (x) tht interpoltes f t the x i, i.e. for which p n (x i )=f(x i ) i =0,,n. 3. Define the (open or closed) n + point Newton-Cotes pproximtion s f(x)dx p n (x)dx It turns out there is clever wy to find the interpolting polynomils p n (x). Definition.6. Given n + distinct smple points x i,theith Lgrnge interpolting polynomil L i,n is the unique nth order polynomil tht stisfies xk = x L i,n (x k )= i, i =0,,n 0 x k 6= x i for ny member x k of the smple points. Theorem.3. Given n + distinct smple points x i nd function f(x), thepolynomilp n (x) tht interpoltes the points (x i,f(x i )) is given by p n (x) = f(x i )L i,n (x). (5) i=0 Proof. Since the right hnd side of (5) is the sum of degree n polynomils, it itself must be degree n polynomil. Moreover, t ech smple point x k, ll terms in the sum dispper except for the kth one, which evlutes to f(x k ). QED. 6
.7 A look further field: Newton-Cotes formuls Lecture notes for Numericl Anlysis It turns out tht is ctully relly esy to come up with the Lgrnge polynomils L i,n (x). Here s formul: L i (x) = ny k=0,k6=i (x x k ) (x i x k ). (6) Clssroom Activity: Discuss why this formul works. Clssroom Activity: Find L 0,2 (x), L,2 (x), nd L 2,2 (x) for smple points, 0, nd. Finlly, we cn put ll this together to get ctul, implementble formuls for Newton-Cotes integrtion. Specificlly, substituting formul (6) into (5) nd the result into (3), we get where f(x)dx p n (x)dx c i = = = = f(x i )L i,n (x)dx i=0 f(x i ) i=0 c i f(x i ) i=0 L i,n (x)dx. L i,n (x)dx Note tht the generl Newton-Cotes formul is thus just weighted sum of smples of f(x). To get the weight coe cients, we solve for the L i,n (x) nd integrte them from to b. Clssroom Activity: Using your results from the lst clss ctivity, find the closed Newton-Cotes formul for n = 2 on the intervl [, ]. It is worth noting tht if we get the Newton-Cotes formuls for one intervl, we get them for ll intervls. Clssroom Activity: Use the fct tht the mp (x) x b mps n intervl [, b] into[0, ] to rgue tht if c i re the Newton-Cotes coe cients on [0, ], then (b )c i re the Newton-Cotes coe cients on [, b]. Hint: you ll need u subsitution. Of course, in prctice, we don t spend lot of time solving for Newton Cotes formuls becuse someone else hs lredy done it. Here is list of some common ones: Closed Newton-Cotes formuls with error terms (Burden nd Fires, pg. 99.) n =, Trpezoid: Z x x 0 f(x)dx = h 2 [f(x 0)+f(x )] h 3 2 f 00 ( ). 7
.8 Notes on reding nd further topics Lecture notes for Numericl Anlysis n = 2, Simpson: Z x2 x 0 f(x)dx = h 3 [f(x 0)+4f(x )+f(x 2 )] h 5 90 f (4) ( ). n = 3, Simpson s 3/8ths Rule: Z x3 x 0 f(x)dx = 3h 8 [f(x 0)+3f(x )+3f(x 2 )+f(x 3 )] 3h 5 80 f (4) ( ). n = 4: Z x4 x 0 f(x)dx = 2h 45 [7f(x 0) + 32f(x ) + 2f(x 2 ) + 32f(x 3 )+7f(x 4 )] 8h 7 945 f (6) ( ). Open Newton-Cotes formuls with error terms (Burden nd Fires, pg. 99.) n = 0, Midpoint Rule: n = : n = 3: Z x3 Z x2 x Z x x f(x)dx =2hf(x 0 )+ h3 3 f 00 ( ). f(x)dx = 3h 2 [f(x 0)+f(x )] + 3h3 4 f 00 ( ). x f(x)dx = 4h 3 [2f(x 0) f(x )+2f(x 2 )] + 45h5 45 f (4) ( ). n = 4: Z x4 x f(x)dx = 5h 24 [f(x 0)+f(x )+f(x 2 ) + f(x 3 )] + 95h5 44 f (4) ( )..8 Notes on reding nd further topics The mteril in this section ws tken primrily from three sources:. our textbook 2. Numericl Anlysis, by Burden nd Fires 3. Numericl Anlysis, by Cheney nd Kincid Numericl integrtion is big topic, nd there re lots of innovtions tht we hven t covered. Here is very prtil list of things you might explore for finl project:. Gussin qudrture 2. Adptive qudrture 3. Romber integrtion 4. Extensions to multiple dimensions 5. How to hndle improper integrls 8