2

1 Notes for Numericl Anlysis Mth 5466 by S. Adjerid Virgini Polytechnic Institute nd Stte University (A Rough Drft)

2

Contents 1 Differentition nd Integrtion 5 1.1 Introduction............................ 5 1.2 Numericl Differentition..................... 5 1.2.1 Approximting first-order derivtives.......... 5 1.2.2 Approximting second-order derivtives......... 9 1.2.3 Forwrd nd bckwrd difference formuls....... 10 1.2.4 Effects of round-off errors................ 12 1.3 Numericl Integrtion...................... 12 1.3.1 Newton-Cotes formuls.................. 13 1.3.2 Composite qudrtures:................. 16 1.3.3 Error nlysis....................... 20 1.3.4 Romberg method..................... 25 1.3.5 Guss Qudrture.................... 27 1.3.6 Multiple integrls..................... 37 3

4 CONTENTS

Chpter 1 Differentition nd Integrtion 1.1 Introduction In this chpter we will develop nd study wys to pproximte derivtives nd integrls. Approximting derivtives is very importnt in constructing finite differences methods for solving differentil equtions. Finding pproximte vlues of integrls is importnt in its own ske other res such s defining multistep methods for ordinry differentils equtions nd finite elements methods for solving prtil differentil equtions. 1.2 Numericl Differentition 1.2.1 Approximting first-order derivtives We seek to pproximte the derivtive of function f using vlues of f t severl points. First-order formuls We first note tht the derivtive is defined s the limit of the rtio f 0 f(x + h) f(x) (x) =lim : (1.1.1) h!0 h Then, for smll h, we will pproximte the derivtive s f 0 (x) ß f(x + h) f(x) h 5 (1.1.2)

6 CHAPTER 1. DIFFERENTIATION AND INTEGRATION if h>0 we obtin the forwrd difference formul if h<0 we obtin the bckwrd difference formul The centered difference formul is f 0 (x) ß f(x + h) f(x h) 2h (1.1.3) Applying the Tylor series we obtin f(x + h) =f(x)+hf 0 (x)+ h2 2 f 00 (ο); x<ο<x+ h This leds to f 0 (x) = f(x + h) f(x) h h 2 f 00 (ο) The trunction error is E 1 = f 0 (x) nd my be bounded s f(x + h) f(x) h je 1 j <M 2 h=2 = h 2 f 00 (ο) This formul is exct for ll polynomils of degree 1 or less. Second-order formuls f(x 0 + h) =f(x 0 )+hf 0 (x 0 )+ h2 2 f 00 (x 0 )+ h3 6 f (3) (ο 1 ) (1.1.4) f(x 0 h) =f(x 0 ) hf 0 (x 0 )+ h2 2 f 00 (x 0 ) h3 6 f (3) (ο 2 ) (1.1.5) Subtrcting (1.1.5) from (1.1.4) we obtin

1.2. NUMERICAL DIFFERENTIATION 7 f 0 (x 0 )= f(x 0 + h) f(x 0 h) 2h h2 6 f (3) (ο 1 )+f (3) (ο 2 ) : 2 Applying the intermedite vlue theorem we obtin the centered difference formul f 0 (x 0 )= f(x 0 + h) f(x 0 h) 2h The trunction error cn be written s h2 6 f (3) (ο); x 0 h<ο<x 0 + h: (1.1.6) nd bounded by E 2 = f 0 (x 0 ) f(x 0 + h) f(x 0 h) 2h je 2 j»m 3 h 2 =6 = h2 6 f (3) (ο) The centered difference formul is exct for ll polynomils of degree 2 or less. Fourth-order centered difference formul: f(x 0 + h) =f(x 0 )+hf 0 (x 0 )+ h2 2 f 00 (x 0 )+ h3 6 f (3) (x 0 )+ h4 4! f (4) (x 0 )+ h 5 5! f (5) (ο 1 ) (1.1.7) f(x 0 h) =f(x 0 ) hf 0 (x 0 )+ h2 2 f 00 (x 0 ) h3 6 f (3) (x 0 )+ h4 4! f (4) (x 0 ) h 5 5! f (5) (ο 2 ) (1.1.8) f(x 0 +2h) =f(x 0 )+2hf 0 (x 0 )+ 4h2 2 f 00 (x 0 )+ 8h3 6 f (3) (x 0 )+

8 CHAPTER 1. DIFFERENTIATION AND INTEGRATION 16h 4 4! f (4) (x 0 )+ (2h)5 f (5) (ο 3 ) (1.1.9) 5! f(x 0 2h) =f(x 0 ) 2hf 0 (x 0 )+ h2 2 f 00 (x 0 )+ 8h3 6 f (3) (x 0 )+ 16h 4 4! f (4) (x 0 ) (2h)5 5 f (5) (ο 4 ) (1.1.10) Subtrcting (1.1.8) from (1.1.7) nd using the intermedite vlue theorem we obtin f(x 0 + h) f(x 0 h) =2hf 0 (x 0 )+ h3 3 f (3) (x 0 )+ 2h5 5! f (5) (c 1 ) (1.1.11) f(x 0 +2h) f(x 0 2h) =4hf 0 (x 0 )+ (2h)3 3 f (3) (x 0 )+ 2(2h)5 f (5) (c 2 ) 5! (1.1.12) Multiplying (1.1.11) by eight nd subtrcting (1.1.12) we obtin 8(f(x 0 + h) f(x 0 h)) (f(x 0 +2h) f(x 0 2h)) = 12hf 0 (x 0 )+ 16h 5 (f (5) (c 1 ) 4f (5) (c 2 )) (1.1.13) 5! If f (5) does not chnge sign nd f (5) does not chnge rpidly, then there exists x 0 2h <c<x 0 +2h such tht 4f (5) (c 2 ) f (5) (c 1 )=3f (5) (c) Solving for f 0 (x 0 )we obtin f 0 (x 0 )= f(x 0 +2h)+8f(x 0 + h) 8f(x 0 h)+f(x 0 2h) 12h + h4 30 f (5) (c) (1.1.14)

1.2. NUMERICAL DIFFERENTIATION 9 The trunction error is nd my be bounded s E 4 = h4 30 f (5) (c) je 4 j <h 4 M 5 =30 The centered difference formul (1.1.14) is exct for ll polynomils of degree 4orless. Exmple: x i f(x i ) f 0 (x i ) formul 0.4-0.1 2 forwrd 0.6 0.3 1.5 centered 0.8 0.5 0.25 centered 1.0 0.4 0.5 bckwrd 1.2.2 Approximting second-order derivtives Summing (1.1.7) nd (1.1.8) to obtin f(x 0 + h)+f(x 0 h) =2f(x 0 )+h 2 f 0 (x 0 )+ 2h4 4! f (4) (c) Solving for f 00 (x 0 ) we obtin f 00 (x 0 )= f(x 0 + h) 2f(x 0 )+f(x 0 h) h 2 h2 12 f (4) (c); x 0 h<c<x 0 + h We cn construct fourth-order centered difference formul for f 00 (x 0 ) s f 00 (x 0 )= f(x 0 +2h)+16f(x 0 + h) 30f(x 0 )+16f(x 0 h) f(x 0 2h) 12h 2 +O(h 4 ) (1.1.15)

10 CHAPTER 1. DIFFERENTIATION AND INTEGRATION 1.2.3 Forwrd nd bckwrd difference formuls We obtin O(h 2 ) formuls by considering the three points x 0 ; x 1 = x 0 + h; x 2 = x 0 +2h nd p 2 (x) tht interpoltes f t the three points p 2 (x) = (x x 1)(x x 2 ) (x 0 x 1 )(x 0 x 2 ) f(x 0)+ (x x 0)(x x 2 ) (x 1 x 0 )(x 1 x 2 ) f(x 1) Tking the derivtive we obtin + (x x 0)(x x 1 ) (x 2 x 0 )(x 2 x 1 ) f(x 2) p 0 2(x) = (2x (x 1 + x 2 )) 2h 2 f(x 0 ) (2x (x 0 + x 2 )) h 2 f(x 1 ) A forwrd difference formul: + (2x (x 0 + x 1 )) 2h 2 f(x 2 ) f 0 (x 0 ) ß p 2 (x 0 )= ( 3f(x 0)+4f(x 1 ) f(x 2 )) 2h (1.1.16) f 0 (x 0 )= ( 3f(x 0)+4f(x 1 ) f(x 2 )) 2h + h2 3 f (3) (c) (1.1.17) A centered difference formul: f 0 (x 2 ) ß p 0 2(x 1 )= f(x 2) f(x 0 ) 2h (1.1.18) f 0 (x 1 )= f(x 2) f(x 0 ) 2h h2 6 f (3) (c) (1.1.19) A bckwrd difference formul: f 0 (x 0 ) ß p 0 2 (x 0)= (f(x 0) 4f(x 1 )+3f(x 2 )) 2h (1.1.20)

1.2. NUMERICAL DIFFERENTIATION 11 f 0 (x 2 )= (f(x 0) 4f(x 1 )+3f(x 2 )) 2h + h2 3 f (3) (c) (1.1.21) The centered difference formul (1.1.19) requires two function evlutions nd hs smller trunction error. Similr formuls cn be constructed to pproximte prtil derivtives u x nd u y. Generl formul for numericl differentition Let p n interpolte f t x 0 ;x 1 ; ;x n p n (x) =f(x 0 )+ where the interpoltion error is Differentiting to obtin which in turn yields nd E n (x) = ny k=0 p 0 n(x 0 )=f[x 0 ;x 1 ]+ E 0 n(x 0 )= i Y k=0 (x x k )f[x 0 ; ;x i ] (x x k ) f (n+1) (ο(x)) : (n + 1)! f(x) =p n (x)+e n (x) f 0 (x 0 )=p 0 n(x 0 )+E 0 n(x 0 ) ny k=1 i Y i=2 k=1 (x 0 x k )f[x 0 ; ;x i ] (x 0 x k ) f (n+1) (ο(x 0 )) (n + 1)! If H = mx ; ;n jx 0 x i j,we my bound the error s je 0 n(x 0 )j = O(H n ):

12 CHAPTER 1. DIFFERENTIATION AND INTEGRATION 1.2.4 Effects of round-off errors We ssume perturbed dt nd investigte its effects of smll on the ccurcy of numericl differentition. Let us consider the first order forwrd difference formul d(x 0 )= ~ f(x 0 + h) ~ f(x 0 ) h = f(x 0 + h)+ffi 1 (f(x 0 )+ffi 0 ) h = f(x 0 + h) f(x 0 ) h + (ffi 1 ffi 0 ) h = f 0 (x 0 )+ h 2 f 00 (c)+ (ffi 1 ffi 0 ) ; x 0 <c<x 0 + h; h>0: h Thus the totl error is d(x 0 ) f 0 (x 0 )= h 2 f 00 (c)+ (ffi 1 ffi 0 ) h Here we hve the two errors one is the trunction error nd the round-off error. For h > ffi the trunction error domintes, otherwise, the round-off error becomes more dominnt. If ffi i» ffi, we my bound the totl error jd(x) f 0 (x)j» hm 2 2 + 2ffi h = E(h) which my beused to find the optiml vlue for h tht minimizes E(h) Thus h opt =2 q ffi M 2. de dh = M 2 2 2 ffi h 2 =0 1.3 Numericl Integrtion Numericl qudrtures re used to pproximte integrls nd to obtin numericl methods for ordinry nd prtil differentil nd integrl equtions.

1.3. NUMERICAL INTEGRATION 13 The generl form of numericl qudrture hs the form f(x)dx ß Q(f) = i=0 w i f(x i ); (1.1.22) where such tht» x 0» x 1 ; ;x n = b; f(x)dx = Q(f)+E(f): (1.1.23) 1. x 0 ; ;x n re the qudrture nodes 2. w 0 ; ;w n re the weights 3. E(f) is the trunction error Definition 1. The numericl qudrture is of precision k, if nd only if, E(p) = 0 for ll polynomils p of degree t most k nd there exists p of degree k +1such tht E(p) 6= 0. 1.3.1 Newton-Cotes formuls The generl procedure to construct Newton-Cotes qudrture consists of the following steps: 1. Select set of nodes x 0 ;x 1 ; ;x n 2 [; b] 2. Construct p n (x) such tht p n (x i )=f(x i ) p n (x) = i=0 f(x i )L i (x) 3. Let Q(f) = R b p n(x)dx = P n i=0 R b L i(x)dxf(x i ) The weights re w i = R b L i(x)dx

14 CHAPTER 1. DIFFERENTIATION AND INTEGRATION The trpezoidl rule: Theorem 1.3.1. Let x 0 = nd x 1 = x 0 + h = b with h = b. If f 2 C 2 [; b], then Proof. let nd f(x)dx = h h3 (f()+f(b)) 2 12 f (2) (ο); ο 2 (; b) (1.1.24) p 1 (x) = b x h f()+x h f(b) f(x) =p 1 (x)+ f (2) (c(x)) (x )(x b) 2 Integrting on [; b] to obtin f(x)dx = p 1 (x)dx + f (2) (c(x)) (x )(x b)dx 2 Applying the weighted men-vlue theorem with w(x) =(x )(x b) which does not chnge sign in (; b) we obtin f(x)dx = This leds to (1.1.24). p 1 (x)dx + f 2 (ο(x)) 2 (x )(x b)dx The trpezoidl rule is exct for ll polynomils of degree less or equl to 1. Degree of precision is 1. Simpson's Rule: Theorem 1.3.2. Let us consider three points x 0, x 1 = x 0 + h; x 2 = x 0 +2h nd if f 2 C 4, then we cn write f(x)dx = h 3 [f()+4f(( + b)=2) + f(b)] h5 90 f (4) (ο); <ο<b: (1.1.25)

1.3. NUMERICAL INTEGRATION 15 Proof. We first construct p 2 (x) tht interpoltes f t x i ; i =0; 1; 2 p 2 (x) = (x x 1)(x x 2 ) x 0 x 1 )(x 0 x 2 ) f(x 0)+ (x x 0)(x x 2 ) x 1 x 0 )(x 1 x 2 ) f(x 1) Integrting p 2 (x) we obtin Z x2 + (x x 0)(x x 1 ) x 2 x 0 )(x 2 x 1 ) f(x 2): x 0 p 2 (x)dx = h 3 (f(x 0)+4f(x 1 )+f(x 2 )): Since we cnnot use the weighted men-vlue theorem like in the cse of the trpezoidl rule, we will dely the proof to the next section. Simpson rule is exct for ll polynomils of degree less or equl 3, i.e., its degree of precision is 3. Exmple: Let us consider the integrl I(f) = R 2 0 x3 + e x dx Trpezoidl rule: h = 2 I(f) ß Q(f) =f(0) + f(2)=9+e 2 ß 16:3890: The true vlue is I(f) =3+e 2 ß 10:3890561 Simpson's rule: x 0 =0,x 1 =1,x 2 =2,h =1 I(f) ß Q(f) = 1 3 (f(0)+4f(1) + f(2)) = 12 + e2 +4e 3 ß 10:421: We cn found the errors in bioth methods Trpezoidl error: f (2) (x) =6x + e x > 0 f (3) (x) =6+e x > 0, thus M 2 = mx jf (2) (x)j = jf (2) (2)j =12+e 2 0<x<2

16 CHAPTER 1. DIFFERENTIATION AND INTEGRATION je(f)j» 2 12 mx jf (2) j = 2(12 + e2 ) ß 12:92: 0<x<2 3 Simpson's error my be bounded s f (4) (x) =e x, M 4 = mx jf (4) j = e 2, or 0<x<2 je(f)j» h5 M 4 90 = e2 90 ß 0:082: We note tht one dditionl function evlution results in two orders of mgnitude reduction in the error bounds. 1.3.2 Composite qudrtures: Composite qudrtures re useful to (1) integrte discontinuous functions (2) integrte to prescribed ccurcy The composite trpezoidl rule consists of (i) subdividing the intervl into N subintervls using h = (b )=N nd setting x i = + ih; i =0; 1; 2; ;N (ii) Splitting the integrl s f(x)dx = N X i=0 Z xi+1 x i f(x)dx: (1.1.26) (iii) Applying the trpezoidl rule to obtin Z xi+1 x i f(x)dx = h 2 (f(x i)+f(x i+1 )) h3 12 f (2) (ο i ); ο 2 (x i ;x i+1 ):

1.3. NUMERICAL INTEGRATION 17 Now, let us substitute the previous eqution in (1.1.26) to obtin f(x)dx = h N X 2 (f()+f(b)+2 X N f(x i )) h3 12 i=0 f (2) (ο i ): (1.1.27) Since f is smooth we hve m 2» f 2 (x)» M 2 which leds to Nm 2» N X i=0 f (2) (ο i )» NM 2 (1.1.28) Dividing by N we hve m 2» N P i=0 f (2) (ο i ) N» M 2 (1.1.29) Applying the intermedite vlue theorem, we show tht there exists ο 2 [; b] such tht Since N =(b )=h we hve N P i=0 f (2) (ο i ) N = f (2) (ο) (1.1.30) f(x)dx = h N X 2 (f()+f(b)+2 f(x i )) (b )h2 f (2) (ο): (1.1.31) 12 Exmple: using h = 1=4 we write 0 sin(x 2 )dx ß 1 (sin(0) + sin(1) + 2(sin(1=16)+ 8 sin(1=4) + sin(9=16))) ß 0:63195072:

18 CHAPTER 1. DIFFERENTIATION AND INTEGRATION Let us bound the trunction error: je(f;h)j» (b )h2 M 2 : 12 Since f (2) (x) =2cos(x 2 )+4x 2 sin(x 2 ), the tringle inequlity yields jf (2) (x)j =2jcos(x 2 )j +4x 2 jsin(x 2 )j»2+4=6: Thus, je(f;h)j» (1=4)2 6 12 =1=32: In order to find the smllest number of subintervls N such tht je(f;h)j» 10 4,we set (b )h 2 M 2» 10 4 12 nd solves for h nd then for N to obtin h 2» 210 4 ; which leds to h» p 210 2, or N» 102 p 2 :. Thus, the smllest number of subdivisions required to chieve the prescribed ccurcy is estimted to N ß 71. Composite Simpson's rule: We prtition the intervl [; b] sx i = + i Λ h; i =0; 1; 2n, with h = b 2n. nd split the integrl s f(x)dx = n Z x2k+2 X k=0 x 2k f(x)dx (1.1.32) On ech intervl Simpson rule yields Z x2k+2 x 2k f(x)dx = h 3 [f(x 2k)+4f(x 2k+1 )+f(x 2k+2 )]

1.3. NUMERICAL INTEGRATION 19 1 90 h5 f 4 (ο k ); x 2k <ο k <x 2k+2 : (1.1.33) Substituting (1.1.33) into (1.1.32) we obtin X f(x)dx = h n 3 [f()+f(b)+4 f(x 2k+1 )+2 k=0 n X k=1 X f(x 2k )] 1 n 90 h5 k=0 f 4 (ο k ): (1.1.34) By the intermedite vlue theorem there exists ο 2 (; b) such tht n P k=0 f 4 (ο k ) n = f 4 (ο) (1.1.35) combining (1.1.34) nd (1.1.35) with h =(b )=2n we estblish (1.1.32). Generl Closed Newton-Cotes Qudrtures We subdivide [,b] into n 1 subintervls [x i ;x i+1 ]; i =0; 1; n 1 where x i = + i Λ h; i =0; 1; n, with h =(b )=n Theorem 1.3.3. If n is even nd f 2 C n+2 [; b], then f(x)dx = i=0 w i f(x i )+ hn+3 f (n+2) (ο) (n + 2)! if n is odd nd f 2 C n+1 [; b], then where w i = f(x)dx = i=0 w i f(x i )+ hn+2 f (n+1) (ο) (n + 1)! Z n 0 Z n 0 t 2 (t 1)(t 2) (t n)dt (1.1.36) t(t 1)(t 2) (t n)dt (1.1.37) L i (x)dx; L i (x) = (x x 0) (x x i )(x x i+1 ) (x x n ) (x i x 0 ) (x i x i )(x i x i+1 ) (x i x n )

20 CHAPTER 1. DIFFERENTIATION AND INTEGRATION Proof. For proof see Iscson nd Keller. Open Newton-Cotes Qudrtures One cn lso construct open Newton-Cotes formuls by using the points x 0 = +h; x 1 = +2h; ;x n = b h, where h =(b )=(n+2) nd construct p n (x) tht interpoltes f t the interior points x i = +(i+1)h; i =0; 1; ::; n. Theorem 1.3.4. If n is even nd f 2 C n+2 [; b], then f(x)dx = i=0 w i f(x i )+ hn+3 f (n+2) (ο) (n + 2)! Z n+1 t 2 (t 1)(t 2) (t n)dt (1.1.38) where if n is odd nd f 2 C n+1 [; b], then w i = f(x)dx = i=0 w i f(x i )+ hn+2 f (n+1) (ο) (n + 1)! Z n+1 t(t 1)(t 2) (t n)dt (1.1.39) L i (x)dx; L i (x) = (x x 0) (x x i )(x x i+1 ) (x x n ) (x i x 0 ) (x i x i )(x i x i+1 ) (x i x n ) Proof. Iscson nd Keller. 1.3.3 Error nlysis In this section we introduce the Peno kernel nd show how it is used to estblish integrtion errors. First we let E(f) =Q(f) I(f) =Q(f) f(x)dx (1.1.40) such tht E(f) = 0 for f 2P n, the spce of polynomils of degree n or less.

1.3. NUMERICAL INTEGRATION 21 Theorem 1.3.5. If E(f) =0; 8f 2P n nd f 2 C n+1 [; b] then where with E(f) = f (n+1) (t)k(t)dt K(t) = 1 n! E x((x t) n +) (x t) n + = K(t) is clled the Peno Kernel. ( (x t) n ; x t 0; x<t Proof. Let us consider the Tylor series of f t x = where f(x) =f()+(x )f 0 ()+ + f (n) () (x ) n + r n (1.1.41) n! r n = 1 n! Z x f (n+1) (t)(x t) n dt = 1 n! To prove this reltion we observe tht for n =0 f (n+1) (t)(x t) n +dt (1.1.41b) f(x) f() = Integrtion by prts we obtin for n =1 Z x f(x) f() = (x t)f 0 (t)j t=x t= + By induction we ssume f(x) f() =(x )f 0 ()+ f(x) =T n (x)+ 1 (n 1)! Z x f 0 (t)dt Z x Z x f 00 (t)(x t)dt f 00 (t)(x t)dt f (n) (t)(x t) n dt

22 CHAPTER 1. DIFFERENTIATION AND INTEGRATION Using integrtion by prts we obtin the Tylor formul. Now pply the qudrture with E(T n ) = 0 to obtin Z E(f) =E(T n )+E(r n )=E(T n )+ 1 b n! E( f (n+1) (t)(x t) n +dt) = Q(r n ) r n (x)dx: Next, we will rgue tht since (x t) n + 2 C n [; b], we my interchnge the integrtion nd E. If Q involves derivtives of f, we need to show tht derivtives lso permutes with integrtion. First, we note since (x t) n + 2 C n [; b], from elementry clculus, we hve d k r n dx k = 1 n! In prticulr, for k = n 1we hve f (n+1) (t) dk (x t) n + dx k dt; 0» k» n 1: (1.1.42) d n r n dx n = 1 n! f (n+1) (t) dn (x t) n + dx n dt = f (n+1) (t)(x t) + dt = Z x f (n+1) (t)(x t)dt Next, we estblish (1.1.42) using Leibnitz formul s d n r n dx n = d dx ( Z x f (n+1) (t)(x t)dt) (1.1.42b) = f (n+1) (x)(x x)+ Z x (f (n+1) (t) d (x t))dt dx = 1 n! Z x f (n+1) (t) dn (x t) n dx n (1.1.42c)

1.3. NUMERICAL INTEGRATION 23 Thus, integrtion permutes with d k =dx k ;k =0; 1; 2; ;n which yields Q(r n )= 1 n! f (n+1) (t)q x ((x t) n +)dt: (1.1.43) Now, pplying Fubini theorem we invert the order of integrtion to obtin 1 n! ( f (n+1) (t)(x t) n +dt)dx = 1 n! f (n+1) (t) (x t) n +dxdt (1.1.44) Combining (1.1.43) nd (1.1.44) leds to E(f) = 1 n! f (n+1) (t)e((x t) n +)dt: This completes the proof of the Theorem with K(t) = 1 n! E((x t)n +) We note tht Peno kernel is positive for most qudrtures. This llows us to use the men-vlue theorem to obtin our qudrture formuls. Exmple: R Now, we will construct the Peno kernel for Simpson's rule for b I = f(x)dx with h =(b )=2 nd b = +2h, Q(f) = h (f()+4f( + h)+f(b)): 3 One cn esily check tht Simpson rule is exct for ll polynomils of degree three or less. Thus, E(p) = 0 for ll polynomils of degree three or less. If we pply the previous theorem with n =3we obtin Peno kernel K(t) = 1 3! E((x t)3 +);

24 CHAPTER 1. DIFFERENTIATION AND INTEGRATION where integrtion is done with respect to x. K(t) = 1 6 [h 3 (( t)3 + +4( + h t) 3 + +( +2h t) 3 +) Z +2h (x t) 3 +dx] Using the definition (x t) 3 + = ( (x t) n ; t» x 0; t>x ; we write K(t) = ( 1 [ h(4( + h 6 3 t)3 +( R R +2h t) 3 +2h ) (x t) 3 dx; <t<+ h t 1 [ h( +2h +2h 6 3 t)3 (x t) 3 dx]; + h<t<+2h t This cn be further simplified s K(t) = ( 1 [ h(4( + h 6 3 t)3 +( +2h t) 3 ) (+2h t)4 ] 4 <t<+ h 1 6 3 t)3 (+2h t)4 ] 4 + h<t<+2h We note tht K(t) does not chnge sign on [; b] nd thus we cn pply the men vlue theorem to write the qudrture error s E(f) = Z +2h Z +2h f (4) (t)k(t)dt = f (4) (ο) K(t)dt; <ο<+2h We note tht the integrl must be split into two prts s follows Z +h K(t)dt + Z +2h +h K(t)dt = h5 90 Thus we hve estblished the Simpson formul (1.1.25). The Peno kernel my be used to derive generl Newton-Cotes qudrtures.

1.3. NUMERICAL INTEGRATION 25 1.3.4 Romberg method We first consider the expnsion of the error in the trpezoidl rule T (f;h) for f 2 C 2m+2 s f(x)dx = T (f;h)+c 1 h 2 + c 2 h 4 + c m h 2m + O(h 2m+2 ): (1.1.45) c k = B 2k+2 (2k)! (f (2k) (b) f (2k) (); k =1; 2; where B 2k re the clssicl Bernoulli numbers. B 2 = 1=6, B 4 = =30, B 6 =1=42. See Kincid nd Cheney for definition of Bernoulli polynomils nd numbers. We will use the Euler expnsion (1.1.45) in terms of h with Richrdson extrpoltion to construct the Romberg method. For instnce to eliminte the c 1 h 2 term we consider the trpezoidl rule with h=2 f(x)dx = T (f;h=2) + c 1 (h=2) 2 + c 2 (h=2) 4 + c m (h=2) 2m + O(h 2m+2 ): (1.1.46) Multiplying (1.1.46) by four nd subtrcting (1.1.45) we obtin f(x)dx = 4T (f;h=2) T (f;h) 3 + d 1 h 2 + d 2 h 4 + d m h 2m + O(h 2m+2 ): (1.1.47) We note tht the O(h 4 4T (f;h=2) T (f;h) ) pproximtion is S(f;h=2). Similrly, 3 we my use Richrdson extrpoltion second time to eliminte the O(h 4 ) term nd obtin n O(h 6 ) qudrture.

26 CHAPTER 1. DIFFERENTIATION AND INTEGRATION The Romberg integrtion scheme is then obtined by using the following tble O(h 2 ) O(h 4 ) O(h 6 ) O(h 8 ) O(h 2m+2 R(0,0) R(1,0) R(1,1) R(2,0) R(2,1) R(2,2) R(3,0) R(3,1) R(3,2) R(3,3) ::: ::: ::: ::: R(m-1,0) R(m-1,1) R(m-1,2) R(m-1,3) R(m, 0) R(m,1) R(m,2) R(m,3) R(m,m) Where R(i; 0) = T (f;h i ); i =0; 1; 2; ;M nd the other entries re generted using the recurrence formul R(i; j) = 4j R(i; j 1) R(i 1;j 1) 4 j 1 = R(i; j)+ R(i; j 1) R(i 1;j 1) ; 4 j 1 i =1; 2; ;M; j» i: (1.1.48) Remrks: 1. R(1;i)=S(f;h i ); i 1: Simpson's method 2. R(2;i)=B(f;h i ); i 2: Boole's method 3. better pproximtions re obtined by dding new rows to the bottom of the tble 4. Ech row contins methods of different order with sme h 5. Ech column corresponds to composite method with different h

1.3. NUMERICAL INTEGRATION 27 1.3.5 Guss Qudrture Our gol is to construct numericl qudrtures of the form f(x)dx ß w i f(x i ) with the highest degree of precision possible, i.e., x k = w i x k i ; k =0; 1; 2; :::; 2n 1: (1.1.49) Now we will derive such formul on [-1,1]. n =1 dο =2=w 1 οdο =0=w 1 ο 1 solving this system we obtin the one-point Guss qudrture where w 1 =2; ο 1 =0 n =2 dο =2=w 1 + w 2 οdο =0=w 1 ο 1 + w 2 ο 2 ο 2 dο =2=3 =w 1 ο 2 1 + w 2 ο 2 2 ο 3 dο =0=w 1 ο 3 1 + w 2 ο 3 2

28 CHAPTER 1. DIFFERENTIATION AND INTEGRATION #points ο w i Deg of prec 1 0.0 2.0 1 2 1= p 3 1 3 p= p 3 1 3 3:0=5:0 5.0/9.0 5 p 0.0 8.0/9.0 3:0=5:0 5.0/9.0 4 0.861136311594053 0.347854845137454 7 0.339981043584856 0.652145154862546-0.861136311594053 0.347854845137454-0.339981043584856 0.652145154862546 5 0.90617984593866 0.236926885056189 9 0.538469310105683 0.478628670499366 0.0 0.568888888888889-0.90617984593866 0.236926885056189-0.538469310105683 0.478628670499366 Tble 1.1: Guss points nd weights Solving this system we obtin the two-point Guss-qudrture w 1 = w 2 =1; ο 1 = ο 2 =1= p 3 Guss Integrtion with shifted intervls: We will use the mpping from [; 1] to [; b] defined by nd trnsform the integrl ο! x(ο) = + (b ) (1 + ο) 2 f(x)dx = b 2 f(x(ο))dο We denote ^f(ο) =f(x(ο)) nd pply the Guss qudrture to the new integrl

1.3. NUMERICAL INTEGRATION 29 ^f(ο)dο = w i ^f(οi ) combining the two previous equtions to obtin where f(x)dx ß ^w i = b 2 w i; x i = + ^w i f(x i ) (b ) (1 + ο i ): 2 Lter, we will show tht ο i ; i = 1; 2; ;n; re the roots of the Legendre polynomil of degree n defined by the Rodrigues formul p k (ο) = 1 d k 2 k (k)! dο k (ο2 1) k ; k =0; 1; 2; (1.1.50) Legendre polynomils cn lso be defined by the recursive formul p 0 (ο) =1; p 1 (ο) =ο; (1.1.51) p k (ο) = 2k 1 οp k (ο) k 1 k k p k 2(ο); k =2; 3; (1.1.51b) By induction we cn show tht Legendre polynomils given by (1.1.50) nd (1.1.51b) my be written s p k (ο) = (2k)! 2 k (k!) 2 οk + (1.1.51c) Remrks: 1. p k (±1) = (±1) k 2. p k ( ο) =() k p k (ο)

30 CHAPTER 1. DIFFERENTIATION AND INTEGRATION Grm-Schmidt Method: A third wy to construct orthogonl polynomils is by using Grm-Schmidt method strting from the cnonicl bsis q i (x) =x i ; i =0; 1; 2; ;p; with the inner product (p; q) = where w(x) 0, for ll» x» b. w(x)p(x)q(x)dx (1.1.52) Grm-Schmidt process on [; 1]: Alterntively, we my construct orthogonl polynomils with respect to the inner product (1.1.52) by setting p 0 (ο) =1 (1.1.53) p k (ο) =ο k k X j=0 (ο k ;p j ) (p j ;p j ) p j(ο); k =1; 2; (1.1.53b) These Legendre polynomils my be generted by the following recursion formul where p n (ο) =(ο n )p n (ο) b n p n 2 (ο); n 2 (1.1.54) p 0 (ο) =1; p 1 (ο) =ο (1.1.54b) Remrks: n = (οp n;p n ) (p n ;p n ) nd b n = (οp n;p n 2 ) (p n 2 ;p n 2 ) : (1.1.54c) 1. p k given by nd (1.1.53) nd (1.1.54) re monic polynomils 2. If w = 1, we obtin Legendre polynomils 3. If w = p1 ο 1, we obtin Chebyshev polynomils 2

1.3. NUMERICAL INTEGRATION 31 4. If w = e ο2 on (; 1), we obtin Hermite polynomils 5. if w = e ο on [0; 1), we obtin Lguerre polynomils 6. These polynomils my bescled by setting either (p n ;p n )=1. Theorem 1.3.6. Legendre polynomils (1.1.50) stisfy the following properties ( 0 ; if k 6= j (p k ;p j )= p k (ο)p j (ο)dο = 2 ; if k = j ; (1.1.55) 2k+1 (p k ;q)=0; for ll polynomils of degree <k; (1.1.55b) p n (ο) hs n simple roots ο i 2 (; 1); i =1; ;n, i.e., p n (ο i )=0; i =1; ;n: (1.1.55c) Proof. First we use the fct tht p n (ο)dο =0 nd since p n (x) is not identiclly zero it must chnge sign in (; 1) t ο 0. Now we cn write p n (ο) =q n j (ο)(ο ο 0 ) j, where j is odd. Using the fct p n (ο)(ο ο 0 )dο = q n j (ο)(ο ο 0 ) j+1 dο =0 Thus, q n j (ο); n j > 0must lso chnge sign in (; 1). Now ssume tht ο 1 ;ο 2 ; :::ο l ; re the roots of p n (ο) of odd multiplicity, i.e., where p n (x) chnges sign, nd consider the polynomil r l (ο) = ly (ο ο i ); l» n: Thus, p n (ο)r l (ο) doesnot chnge sign on (; 1) which leds to

32 CHAPTER 1. DIFFERENTIATION AND INTEGRATION p n (ο)r l (ο)dο > 0; l» n: Q This is only true when l = p, i.e., p n (ο) =c n (ο ο i ) which completes the proof. Remrk: The roots of Legendre polynomils re symmetric with respect to the center of the intervl. In the next theorem we will show nother property of Legendre polynomils Theorem 1.3.7. Let p i ; i = 1; 2; be the Legendre polynomils (1.1.50) nd t 1 ;t 2 ; :::; t n ; be n distinct points in (; 1). then the mtrix A defined s is nonsingulr. i;j = p i (t j ); i =0; 1; 2; ;n 1; j =1; 2; ;n; (1.1.56) Proof. Assume A to be singulr, i.e., there exists c = [c 0 ;c 1 ; ;c n ] 6= 0 such tht ca = 0. This leds to the (n 1) degree polynomil q(ο) = n X i=0 c i p i (ο) with n roots, t i ; i =1; 2; ;n. Thus, q must be identiclly zero which leds to the contrdiction c = 0. Now we re redy to stte the min theorem of this section. Theorem 1.3.8. Let ο 1 ; ;ο n ; be the roots of p n (ο) nd w i ; i =1; 2; ;n; be the weights stisfying the system

1.3. NUMERICAL INTEGRATION 33 p k (ο i )w i = ( (p 0 ;p 0 ) ; if k =0 0 ; k =1; 2; ;n 1: (1.1.57) Then, p(ο)dο = w i p(ο i ) (1.1.58) for ll polynomils of degree less or equl to 2n 1. Moreover, w i > 0; for i =1; 2; ;n: Conversely, if w i nd ο i re such tht (1.1.58) holds, then ο i re the roots of p n (ο) nd w i ; i =1; 2; n re solution of (1.1.57). Proof. By Theorem 1.3.6, ο i ; i =1; 2; ;n re rel nd distinct nd thus, by Theorem 1.3.7 the system (1.1.57) hs unique solution. All polynomils p of degree 2n 1 cn be written s p = p n (ο)q(ο)+r(ο) where q(ο) nd r(ο) re polynomils of degree less or equl to n 1. p(x)dx = p n (ο)q(ο)dο + r(ο)dο = r(ο)dο: (1.1.59) Apply the qudrture to obtin w i p(ο i )= w i r(ο i ): (1.1.60) Writing r(ο) s liner combintion of p k we hve r(ο) = n X k=0 c k p k (ο):

34 CHAPTER 1. DIFFERENTIATION AND INTEGRATION which leds to Using (1.1.57) we show tht w i r(ο i )= r(ο)dο = c 0 (p 0 ;p 0 ): (1.1.61) n X k=0 X n c k [ w i p k (ο i )] = c 0 (p 0 ;p 0 ): (1.1.62) Combining (1.1.59-1.1.62) we estblish (1.1.58). In order to show tht w i re positivewe pply (1.1.57) to the (2n 2) degree polynomil nd note tht 0 < r j (x) = r j (ο)dο = ny ;i6=j (ο ο i ) 2 ; w i r j (ο i )=w j n Y ;i6=j (ο i ο j ) 2 This estblishes w j > 0. In order to show tht our result is optiml we ssume (1.1.58) to hold for ll polynomils of degree 2n nd pply it to the polynomil to obtin 0 < which leds to contrdiction. r(ο) = ny r(ο)dο = (ο ο i ) 2 w i r(ο i )=0: Next, we will prove the converse. Let us suppose tht ο i ; i = 1; ;n; re distinct nd w i ;; ;n; re such tht (1.1.58) holds for ll polynomils

1.3. NUMERICAL INTEGRATION 35 of degree 2n 1 nd show tht ο i ; i =1; 2; ;n; re the roots of p n (ο) nd w i re solution of (1.1.57). To estblish (1.1.57) we write w i p k (ο i )= Thus, w j > 0. p k (ο)dο =(p k ;p 0 )= Applying (1.1.58) to p n (ο)p k (ο); k<n;we obtin 0=(p k ;p n )= This cn be written s liner system ( (p 0 ;p 0 ) ; k =0 0 ; k =1; 2; ;n 1 w i p n (ο i )p k (ο i ); k =0; 1; 2; ;n 1: Ac = 0 where A is the mtrix i;j = p i (ο j ) nd c j = w j p n (ο j ). We recll tht the mtrix A is nonsingulr which leds to c j = w j p n (ο j ) = 0; j = 1; 2; ;n. Since w j > 0, p n (ο j )=0. This completes the proof of the theorem The error formul for the Guss-Legendre qudrture on [-1,1] is stted in the following theorem. Theorem 1.3.9. Under the sme condition s the previous theorem the Guss- Legendre qudrture cn be written s f(ο)dο = Q n (f)+ 22n+1 (n!) 4 (2n + 1)[(2n)!] 3 f (2n) (c); <c<1; (1.1.63) where Q n (f) = w i f(ο i ):

36 CHAPTER 1. DIFFERENTIATION AND INTEGRATION Proof. Let H 2n (x) be the Hermite interpolnt of f t the roots of p n on [-1,1]. Since Q n (p) =I(p) is exct for ll polynomils of degree less or equl to 2n 1we hve H 2n (ο)dο = Q n (H 2n )=Q n (f): Thus, I(f) Q n (f) = Using the interpoltion error result (f(ο) H 2n (ο))dο f(ο) H 2n (ο) = f (2n) (c) (ο ο 1 ) 2 (ο ο n ) 2 (2n)! Applying the men vlue theorem leds to I(f) Q n (f) = f (2n) (c) (2n)! (ο ο 1 ) 2 (ο ο n ) 2 dο Using the Rodrigues (1.1.50) formul one cn show tht nd to obtin (1.1.63). p n (ο) = (2n)! 2 n (n!) 2 (ο ο 1) (ο ο n ) (p n ;p n )= 2 2n +1 One cn show similr theorem for the shifted Guss-Legendre qudrture. Theorem 1.3.10. Let f 2 C 2n [; b] then the shifted Guss-Legendre qudrture cn be written s f(x)dx = μq n (f)+ (b )2n+1 (n!) 4 (2n + 1)[(2n)!] 3 f (2n) (c); <c<b: (1.1.64)

1.3. NUMERICAL INTEGRATION 37 where μq n (f) = (b ) 2 w i f(x i ) x i = +(b ) (1 + ο i) : 2 with ο i re the roots of p n (ο) nd w i re the weights for Guss qudrture. Proof. We use the chnge of vrible x = +(b )(1 + ο) towrite nd use the previous theorem. Remrks: f(x)dx = b 2 f(x(ο))dο ffl Composite Guss qudrtures my be constructed. ffl Efficient dptive qudrtures re described in section 7.5 of Kincid nd Cheney. ffl Sub-optiml convergence rtes re obtined for singulr functions f(x). 1.3.6 Multiple integrls We consider the rectngulr region D =[; b] [c; d] nd the double integrl I(f) = Z d c f(x; y)dydx Applying Fubini theorem for multiple integrls we write the iterted integrl I(f) = ( Z d c f(x; y)dy)dx Applying the trpezoidl rule with respect to the y vrible we obtin Z d c f(x; y)dy = d c (f(x; d)+f(x; c)) + Ey 2

38 CHAPTER 1. DIFFERENTIATION AND INTEGRATION I(f) = d c 2 ( f(x; d)dx + Using trpezoidl in x vrible we obtin I(f) = f(x; c)dx + Eydx (b )(d c) (f(; c)+f(; d)+f(b; c)+f(b; d)) + E(f) 4 This is exct for 1;x;y;xy. Other Newton-Cotes qudrtures re generlized in the sme mnner. The n-point Guss qudrture is I(f) = f(ο; )dοd = j=1 w i w j f(ο i ;ο j )+E(f;n) (1.1.65) Shifted qudrtures re obtined using the mpping from [; 1] 2! [; b] [c; d] defined by x(ο) = +(b )(1 + ο)=2 (1.1.66) y( ) = c +(d c)(1 + )=2 (1.1.67) If x i = x(ο i ); y i = y(ο i ); i =1; 2; ;n, then the Qudrture is Z d c f(x; y)dxdy = (b )(d c) 4 j=1 w i w j f(x i ;y j )+E(f;n): (1.1.68) Remrks: ffl This qudrture is exct for x l y m ; 0» n; m» 2n 1. ffl Composite qudrtures cn be generted in two nd three dimensions by prtitioning the domin into smll rectngles or tringles nd pply the qudrture on ech subdomin.