Numericl Integrtion Newton Cotes Formuls Given function f : R R nd two rel numbers, b R, < b, we clculte (pproximtely) the integrl I(f,, b) = f (x) dx K. Frischmuth (IfM UR) Numerics for CSE 08/09 8 / 35 Newton Cotes Formuls To pproximte the integrl f (x)dx we choose n integer n N. Now we divide the intervl [, b] into n equl prts. For the step size h n nd the gridpoints x k we get h n := b nd n x k = + kh n with k = 0,..., n. The ide of the Newton Cotes Formuls consists in replcing the function f (x) by its Lngrnge interpoltion polynomil p n (x) n with p n (x) = f (x k )l k (x) nd l k (x) = k=0 n j=0 j k x x j x k x j through the gridpoints (, f ( )),..., (x n, f (x n )). K. Frischmuth (IfM UR) Numerics for CSE 08/09 8 / 35 Qudrture With f (x) = p n (x) r n (x) }{{} interpoltion error for the integrl it holds f (x)dx } {{ } I(f ) exct integrl = p n (x)dx } {{ } Q n (f ) pproximtion of I(f ) r n (x) dx. } {{ } R n (f ) integrtion error The qudrture formul of the integrl I(f ) n Q n (f ) = p n (x)dx = f (x k ) l k (x)dx k=0 }{{} q k R weights is weighted sum of the function vlues f (x k ). K. Frischmuth (IfM UR) Numerics for CSE 08/09 83 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 84 / 35
Degree Degree of exctness The Lgrnge interpoltion polynomil through the gridpoints (, f ( )), (x, f (x )) hs the form p (x) = f ( )l 0 (x) + f (x )l (x) p (x) = f ( ) x x x + f (x ) x x. The degree of exctness of n integrtion formul Q is the mximl integer r 0, for which holds I(p r (x)) = Q(p r (x)) for ll polynomils p r (x) of degree r. K. Frischmuth (IfM UR) Numerics for CSE 08/09 85 / 35 Midpoint rule The pproximtion of the integrl f (x)dx by the midpoint forumul ( ) + b Q 0 (f ) = (b )f is equivlent to constructing tngent line in ( ( )) + b + b, f nd computing the re under this tngent line ( ) ( ) ( + b + b t(x) = f + f x + b ). K. Frischmuth (IfM UR) Numerics for CSE 08/09 87 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 86 / 35 Estimte For the integrl of the tngent line on the intervl [, b] it holds: [ ( ) ( ) ( + b + b f + f x + b )] dx ( ) ( ) + b + b b ( = f dx + f x + b ) dx [ ( )] + b b ( ) [ + b x = f + f + b ] b x ( ) ( ) [ + b + b b = (b ) f + f + b ] (b ) ( ) ( ) [ + b + b + b = (b ) f + f (b ) + b ] }{{} 0 ( ) + b = (b )f := Q 0 (f ) K. Frischmuth (IfM UR) Numerics for CSE 08/09 88 / 35
Remrk Interpoltion polynomil of degree n= Let f (x) be convex, then Q 0 (f ) < I(f ) < Q (f ). (Q 0 (f ) ˆ= tngent trpezoidl rule) The Lgrnge interpoltion polynomil through the gridpoints (, f ( )), (x, f (x )), (x, f (x )) hs the form p (x) = f ( )l 0 (x) + f (x )l (x) + f (x )l (x). It holds h = b. K. Frischmuth (IfM UR) Numerics for CSE 08/09 89 / 35 Weights We get the weights q 0, q, q for the pproximtion of the interpoltion polynommil Q (f ) = f ( )q 0 + f (x )q + f (x )q by solving the three integrls q 0 = q = q = x l 0 (x)dx = x l (x)dx = x l (x)dx = x (x x )(x x ) ( x )( x ) dx = b = 6 3 h (x )(x x ) dx = 4b = 4 (x )(x x ) 6 3 h x x (x )(x x ) (x )(x x ) dx = b 6 = 3 h K. Frischmuth (IfM UR) Numerics for CSE 08/09 90 / 35 thus we get Q (f ) = b [ ] f ( ) + 4f (x ) + f (x ) 6 Q (f ) = h [ ] f ( ) + 4f (x ) + f (x ) Simpson s Rule 3 Alredy Kepler used this method in 65 to compute the volume of brrel: For the pproximtion of [ dx, V = π f (x)] he used the formul V π h ( ) r + 4rm + rb. 6 K. Frischmuth (IfM UR) Numerics for CSE 08/09 9 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 9 / 35
Error estimtes for Simpson s rule Newton Cotes Formuls of higher degree n the error of Simpson s rule cn be estimted by R (f ) = h5 90 f (4) (ζ) for ζ [, b] hence, we cn integrte polynomils of degree n = 3 exctly with h n = b n, x k = + kh n, xn l (x)dx nd q k = k n s = k=0 q k we get the following pproximtion of the integrl Q n (f ) = b n f (x k )q k s k=0 K. Frischmuth (IfM UR) Numerics for CSE 08/09 93 / 35 Error estimtes K. Frischmuth (IfM UR) Numerics for CSE 08/09 94 / 35 Wrning The following tble holds for the first four degrees: n q k s error,, 4, 6 3, 3, 3, 8 h 3 f (ζ) chord trpezoidl rule h 5 90 f (4) (ζ) Simson s Rule 3h3 5 80 f (4) (ζ) Simpson s Rule - don t use Newton Cotes Formuls of higher order, becuse some weights will become negtive, which will cuse cncelltion of significnt digits - in generl, the sequence Q n (f ), n =,, 3,..., does not converge to f (x)dx 4 7, 3,, 3, 7 90 8h 7 4 945 f (6) (ζ) Boole s Rule K. Frischmuth (IfM UR) Numerics for CSE 08/09 95 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 96 / 35
Multiple stripes Estimtes im bse methods conditions pproximtion of the determined b integrl I = I(f,, b) := f (x) dx trpezoidl rule, Simpson s rule trpezoidl rule: f twice continuously differentible Simpson s rule: f four times continuously differentible formuls h = (b )/n, x i = + i h, y i = f (x i ) for i = 0,,,... n ( y0 T n = T (f,, b, n) = h + y + + y n + y ) n S n = S(f,, b, n) = h 3 (y 0 + 4y + y + 4y 3 + + 4y n + y n ) error terms trpezoidl rule: I = T n + R T n, R T n = Simpson s rule: I = S n + R S n with R S n = (b )h f (ξ) (b )h4 f (iv) (ξ) 80 where ξ rbitrry intermedite point from (, b) (in generl different for R T n, respectively R S n ) (n even!) K. Frischmuth (IfM UR) Numerics for CSE 08/09 97 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 98 / 35 Algorithm (doubling) Trick Romberg multiple evlution of T n with doubling the number of strips T n = T n + h n f ( + i h) i= T i k = T i k + T i k T i k ( ) ni n i k+ h = b ; f() + f(b) T = h ; for k = : mx M = 0; for j = 0 : n M = M + f ( + (j + )h) ; end M = Mh; T = T + M ; h = h ; n = n; K. Frischmuth (IfM UR) Numerics for CSE 08/09 99 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 300 / 35
Exmple (doubling) Exmple (Romberg) Consider the definite integrl 4 0 + x. The nlytic solution is [4 rctn x] 0 = 4π 4 = π. Using the Trpezoidl Rules with intervl bisection we cn evlute the integrl: h M(x) T (x) 3.00000000 3.000000000 3.63594 3.00000000 4 3.4680050 3.376470 8 3.489473 3.38988495 6 3.49874 3.409464 3 3.4674034 3.449894 64 3.46300 3.455964 8 3.4597744 3.458483 56 3.459395 3.45904 5 3.459963 3.45909 K. Frischmuth (IfM UR) Numerics for CSE 08/09 30 / 35 Algorithm (Romberg) The evlution of the integrl (8x 7 6x 5 + 6x 3)dx yields to the exct solution 9.0. The evlution by Trpezoidl Rule followed by Romberg Method yields: 45.0000000 6.35000 44.7500000 36.0375976 7.79968 4.78499 7.433987 94.5386657 9.3559570 9.0000000 K. Frischmuth (IfM UR) Numerics for CSE 08/09 30 / 35 In order to sve memory we use vector insted of the mtrix A. (0) = (b ) for k = : mx n = n + n; h = b n ; f() + f(b) ; n=;. step: Applying the Trpezoidl Rule on the integrl T = 0; for j = : : n ; T = T + f( + jh); end T = h T;. step: Applying the Romberg Method (k) = (k ) + T ; for i = k : : 0 end (i) = (i + ) + (i + ) (i) 4 k ; Ordinry Differentil Equtions K. Frischmuth (IfM UR) Numerics for CSE 08/09 303 / 35
Ordinry Differentil Equtions Nottion Given function f : R R n R n, two rel numbers, x f R, < x f, nd vector y 0 R n, we solve (pproximtely) the Cuchy problem for x [, x f ]. Remrk y (x) = f (x, y(x)) with y( ) = y 0 For strters, we set n = nd spek of n ordinry differentil eqution. Otherwise, we cll it system of ODEs. We consider equtions of the type y (x) = f (x, y(x)). Often x cn be interpreted s time, then we write ẏ(t) = f (t, y(t)). Specil cses: ẏ = f (y) (utonomous eqution), or ẏ = f (t). The ltter cse is obvious: ll prime functions of f re solutions (indefinite integrl), they form the generl solution. To pick prticulr solution we need to prescribe the constnt of integrtion. This is usully done by ssigning n initil vlue. K. Frischmuth (IfM UR) Numerics for CSE 08/09 305 / 35 Initil vlue problems K. Frischmuth (IfM UR) Numerics for CSE 08/09 306 / 35 Existence nd uniqueness Theorem: (Picrd-Lindelof) If f fulfills Lipschitz condition The two equtions ẏ = f (t, y(t)), y(t 0 ) = y 0 together form n initil vlue or Cuchy problem. We hve t y(t) = y 0 + f (τ, y(τ))dτ. t 0 f (t, y ) f (t, y ) < L y y (t, y ), (t, y ) Dom(f ) with ny rel number L (the Lipschitz constnt), then there is unique solution with mximl domin to ech Cuchy problem with initil dt in the domin of f. This mximl solution reches the boundry of Dom(f ) the domin of f. For importnt specil cses nlyticl solutions cn be found. (e.g. seprble, projective, liner, Bernoulli, exct, nd logistic equtions... ) K. Frischmuth (IfM UR) Numerics for CSE 08/09 307 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 308 / 35
Numericl pproches Runge-Kutt methods The best known numericl procedures re the Runge-Kutt methods. Given y(x), we find n pproximtion to y(x + h) in the form ỹ(x + h) = y(x) + hd where h is the stepsize nd d is the men derivtive of the numericl solution on the intervl [x, x + h]. We clculte this vlue by tking the weighted men of vlues of the right-hnd side f (, ) of the ode y (x) = f (x, y). The trick is to choose suitble points nd weights in wy to get the smllest possible difference between the exct solution t x + h. There re mny pproprite choices, depending on ccurcy requirements nd the effort we re going to ccept. Other methods mke use of pst steps, interpoltion techniques re pplied (multistep methods). The generl form of the method is k = f (x, y) i k i = f (x + c i h, y + h i j k j ), i =... r r d = b j k j j= The simplest choice is known s Euler s (explicit) method, d = k = f (x, y) y(x + h) y(x) + hf (x, y(x)). j= K. Frischmuth (IfM UR) Numerics for CSE 08/09 309 / 35 Heun s method K. Frischmuth (IfM UR) Numerics for CSE 08/09 30 / 35 Fourth order RK method On cn show tht the bove method is very rough, nd the only wy to get better results is to reduce the stepsize h. In order to void excessive numbers of steps nd thus cut-off errors we increse the order of the method. A very simple improvement of Euler s method is Heun s method k = f (x, y) k = f (x + h, y + hk ) d = / k + / k The clssicl Runge-Kutt method is of order 4. In this cse, the coefficients re / / 0 0 c = / = 0 / 0 0 0 b = ( /6 /3 /3 /6 ) We will verify tht this method is lredy much better thn the originl one. K. Frischmuth (IfM UR) Numerics for CSE 08/09 3 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 3 / 35
Fifth order RK method The bove method gives the exct vlue of y(x + h) up to term proportionl to h 4. This cn be improved populr 5th order method is known s Dormnd Prince method, or simply dopri. ( It needs r = ) 7 stges. c = b 0. 0. 0.3 3/40 9/40 0.8 44/45 56/5 3/9 8/9.0 937 656 907 368 5360 87 355/33 64448 656 4673 547 /79 49/76.0 35/384 0.0 500/3 5/9 7 57600 0.0 7 6695 7 90 753 33900 503 8656 87 6784 /84 /55 /40 K. Frischmuth (IfM UR) Numerics for CSE 08/09 33 / 35 progrm DormndPrince(input, output); {Integrtion of y (x) = 4*Exp(0.8x) - 0.5y(x) by Dormnd-Prince-Method} vr x0, y0: double; {initil vlues} xf: double; {finl rgument} h: double; {step} eps: double; {ccurcy} x, y: double; uround: double; dy0, dy: double; y, hneu: double; nenner, fehler, fktor: double; fil, ccept: boolen; mxit, i, j: integer; b, c, k: rry[..7] of rel; : rry[..7,..7] of rel; K. Frischmuth (IfM UR) Numerics for CSE 08/09 33 / 35 function fxy(x, y: double): double; begin fxy := 4.0*Exp(0.8*x) - 0.5*y end; {fxy} procedure init; begin {initilize} x0:=0.0; y0:=.0; {set end of intervl of integrtion} xf:=0.0; {step size} h:=0.5; {Tolernce on locl cut off error} eps:=0.00000; uround :=.0E-6; while (+uround) > do uround := uround*0.; c[]:=0.0; c[]:=0.; c[3]:=3/0; c[4]:=0.8; c[5]:=8/9; c[6]:=.0; c[7]:=.0; b[]:=7/57600; b[]:=0.0; b[3]:=-7/6695; b[4]:=7/90; b[5]:=-753/33900; b[6]:=/55; b[7]:=-/40; [,]:=0.; [3,]:=3/40; [3,]:=9/40; [4,]:=44/45; [4,]:=-56/5; [4,3]:=3/9; [5,]:=937/656; [5,]:=-5360/87; [5,3]:=64448/656; [5,4]:=-/79; [6,]:=907/368; [6,]:=-355/33; [6,3]:=4673/547; [6,4]:=49/76; [6,5]:=-503/8656; [7,]:=35/384; [7,]:=0.0; [7,3]:=500/3; [7,4]:=5/9; [7,5]:=-87/6784; [7,6]:=/84; K. Frischmuth (IfM UR) Numerics for CSE 08/09 33 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 33 / 35
x := x0; y := y0; fil := flse; ccept:= true; mxit :=0; writeln( x end; {init} y ) procedure result; begin writeln( x::5,, y::5) end; {result} K. Frischmuth (IfM UR) Numerics for CSE 08/09 33 / 35 if (*uround/eps) > nenner then begin nenner := *uround/eps fehler := bs(h*dy/nenner); if 5.0 < (.*exp((/5)*ln(fehler/eps))) then begin fktor := 5.0 end else begin {if} fktor :=.*exp((/5)*ln(fehler/eps)) if.0 > fktor then begin fktor :=.0; hneu := h/fktor; if fehler < eps then begin y := y; x := x + h; result; if (not ccept) then begin if h < hneu then begin hneu := h end {if} K. Frischmuth (IfM UR) Numerics for CSE 08/09 33 / 35 begin {Dormnd-Prince} init; repet dy := 0.0; for j:= to 7 do begin dy0 := 0.0; for i:= to j- do begin dy0 := dy0 + [j,i]*k[i] end; {for} k[j] := fxy(x + c[j]*h, y + h*dy0); dy := dy + b[j]*k[j] end; {for} y := y + h*dy0; if.0e-5 > bs(y) then begin nenner :=.0E-5 end else begin {if} nenner := bs(y) if bs(y) > nenner then begin nenner := bs(y) K. Frischmuth (IfM UR) Numerics for CSE 08/09 33 / 35 ccept:=true; mxit :=0; end else begin {if} ccept := flse; mxit := mxit + ; fil := (mxit = 0) or (x + 0.*h = x) h := hneu; until (x > xf) or fil; end. {Dormnd-Prince} K. Frischmuth (IfM UR) Numerics for CSE 08/09 34 / 35
Stbility issues Multistep methods (explicit) All methods considered so fr were explicit methods. For certin so-clled stiff differentil equtions (nd systems), numericl solutions tend to oscillte, unless very smll step sizes re pplied. Implicit methods perform better t stiff problems. The most successful implicit methods re multistep methods. We introduce shortly the ide of multistep methods, first on the exmple of explicit schemes (Adm-Bshfort). Finlly, the implicit Adm-Moulton-Methods re presented. K. Frischmuth (IfM UR) Numerics for CSE 08/09 34 / 35 Multistep methods (implicit) The new pproximte vlue of y t x i+ = h(i + ) with h = const is clculted on the bsis of the right-hnd side f, evluted t the most recent n points (x j, y j ), j = i, i,... i n +. n = : n = 3 : y i+ = y i + h 3 y i+ = y i + h ( 3f (ti, y i ) f (t i, y i ) ) ( 3f (ti, y i ) 6f (t i, y i ) + 5f (t i, y i ) ) For the higher vlues of n, we give just the coefficients: h ( ) n = 4 : 55, 59, 37, 9 4 h ( ) n = 5 : 90, 774, 66, 74, 5 70 K. Frischmuth (IfM UR) Numerics for CSE 08/09 35 / 35 BDF-Methods Here we use lso the unknown point (x i+, y i+ ), i.e., we hve to solve n eqution for y i+, which is only implicitly determined. n = : y i+ = y i + h (5f (t i+, y i+ ) + 8f (t i, y i ) f (t i, y i )) n = 3 : y i+ = y i + h 4( 9f (ti+, y i+ ) + 9f (t i, y i ) 5f (ti, y i ) + f (t i, y i )) ide: interpolte lst m nodes nd the unknown next step fulfill ODE in new unknown node in order to interpolte, the Lgrngen bse polynomils re used to find the unknown vlue y i+, system of equtions hs to be solved in generl, some version of Newton s method hs to be pplied n = 4 : h ( ) 5, 646, 64, +06, 9 70 K. Frischmuth (IfM UR) Numerics for CSE 08/09 36 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 37 / 35
Generl Form for constnt stepsize Mtlb Code α i+ y i+ + α i y i + α i y i +... + α i+ m y i+ m = hf (x i+, y i+ ) The vlues of the α s re the following: m =, α i+ =, α i = m =, α i+ = 3/, α i =, α i = / m = 3, α i+ = /6, α i = 3, α i = 3/, α i = /3 m = 4, α i+ = 5/, α i = 4, α i = 3, α i = 4/3, α i 3 = /4 m = 5, α = (37/60, 5, 5, 0/3, 5/4, /5) m = 6, α = (47/60, 6, 5/, 0/3, 5/4, 6/5, /6) K. Frischmuth (IfM UR) Numerics for CSE 08/09 38 / 35 Exmple K. Frischmuth (IfM UR) Numerics for CSE 08/09 39 / 35 Dhlquist s Test y (t) = f (t, y(t)) = λ(y(t) ); y(0) = y 0 = ; y (t) = f (t, y(t)) = Ay(t); y(0) = y 0 ; A digonizble A method is clled A-stble if it works for ll problems with the spectrum of A in the left complex hlfplne. solution for λ = 00, h = /40 K. Frischmuth (IfM UR) Numerics for CSE 08/09 30 / 35 K. Frischmuth (IfM UR) Numerics for CSE 08/09 3 / 35