Numerical Analysis. Chassidy Bozeman. May 30, 2016

Size: px
Start display at page:

Download "Numerical Analysis. Chassidy Bozeman. May 30, 2016"

Transcription

1 Numericl Anlysis Chssidy Bozemn My 30, 016 Contents 1 Theorems Theorems Theorems Root Finding.1 Bisection Method Newton s Method Secnt Method Fixed Point Itertion Approximtion Approximtion Theorem of Weierstrss Difference Approximtions to the first derivtive: Approximting f by Interpoltion Lest Squres Approximtion Orthogonl Polynomils 16 5 Interpoltion Liner Interpoltion Lgrnge Interpoltion Newton s Interpoltion Interpoltion Error Hermite Interpoltion Spline Interpoltion Interpoltion t the Chebyshev Nodes

2 6 Numericl Integrtion Trpezoid Rule Simpson Rule Midpoint Rule Gussin Qudrture Peno Kernel Formul Solutions of Systems of Equtions Gussin Elimintion Perturbtion, Conditioning, Stbility Cholesky Decomposition of Positive Definite Mtrices QR Decomposition Splitting Methods Steepest Descent Conjugte Directions/ Conjugte Grdient Approximte Solution of the Eigenvlue Problems for Squre Mtrices Gerschgorin s Theorem for Eigenvlues Reduction to Hessenberg & Tridigonl Form Power Method Inverse Power Method Shifted Inverse Power Method Ryleigh Quotient Itertion QR Itertion Shifted QR Itertion Jcobi Method Numericl Methods for Ordinry Differentil Equtions Euler s Method Vrints of Euler s Method Residul, Trunction Error, Order of Accurcy Implicit Methods nd Predictor-Corrector Schemes Runge-Kutt Multistep Methods Stbility

3 1 Theorems Theorems Theorem 1. Tylor s Theorem Let f C n+1 [, b] for some n 0, nd let x, x 0 [, b]. Then f(x) p n (x) + R n (x) where p n (x) R n (x) 1 n! (x x 0 ) k f k (x 0 ) k! k0 x x 0 (x t) n f (n+1) (t)dt (1) R n (x) (x x 0) (n+1) f (n+1) (ξ x )dt () (n + 1)! for some point ξ x between x nd x 0. By using the men vlue theorem, it cn be shown tht 1 nd re equivlent. Theorem. Men Vlue Theorem Let f be continuous on [, b] nd differentible on (, b). Then there exists point c [, b] such tht f f(b) f() (c). b (The men vlue theorem llows us to express differences of function vlues in terms of the differences of rguments.) Theorem 3. Rolle s Theorem Let f C[, b] nd differentible on (, b) such tht f() f(b), then there exists c (, b) such tht f (c) 0.(Note tht is follows directly from the intermedite vlue theorem when f(b) f().) Theorem 4. Intermedite Vlue Theorem Let f C([, b]) be given nd ssume tht W is vlue between f() nd f(b). Then there exists point c [, b] such tht f(c) W. (The intermedite Vlue Theorem is the bsis for the Bisection root finding method). Theorem 5. Extreme Vlue Theorem Let f C([, b]) given. Then there exists point m [, b] such tht f(m) f(x) for ll x [, b] nd point M [, b] such tht f(m) f(x) for ll x [, b]. Moreover, f chieves its mximum nd minimum vlues on [, b] either t the endpoints or b or t criticl point. Theorem 6. Integrl Men Vlue Theorem Let f nd g both be in C([, b]) nd ssume tht g does not chnge signs on [, b]. Then there exists point ξ [, b] such tht g(t)f(t)dt f(ξ) g(t)dt Theorem 7. Discrete Averge Vlue Theorem Let f C([, b]) nd consider the sum S k f(x k ) k1 where ech point x k [, b] nd the coefficients stisfy Then there exists point η [, b] such tht k 0, f(η) k 1. k1 k f(x k ). k1 3

4 1. 56 Theorems Root Finding The three bsic methods for finding α such tht f(α) 0 re: Bisection method, Newton s method, nd Secnt method. The most common of these three is the Newton s method, but we begin the discussion with the Bisection Method..1 Bisection Method The bisection method is very simple root finding method tht is bsed on the continuity of function. Suppose we know tht f()f(b) < 0. By the Intermedite Vlue theorem, there exists point α (, b), such tht f(α) 0. To pproximte (or find α in some nice cses), we use the following itertive process: Set x 1 to the the midpoint of the intervl [ 0, b 0 ] [, b]. i.e x 1 1 (b + ). Obviously, if f(x 1) 0, we hve found root. If f()f(x 1 ) < 0, then root α lies in (, x 0 ), nd we repet this process on the intervl [ 1, b 1 ] [, x 0 ] by letting x 1 ( + x 0). Similrly, if f(x 1 )f(b) < 0, then root α lies in (x 1, b), nd we repet this process on the intervl [ 1, b 1 ] [x 1, b] by letting x 1 (b + x 0). Continuing this process, we eventully hve root α loclized wthin n intervl of length s smll s we wnt. We now turn to the bisection convergene error: Let [ 0, b 0 ] [, b] be the initl intervl with f()f(b) < 0. For n 1,,... define the pproximte root s x n 1 (b n 1 + n 1 ). Then there exists root α in [, b] such tht ( ) n 1 α x n (b ). To see this, note tht since the intervl is split in hlf during ech itertion, it follows tht so b n n 1 (b n 1 n 1 ), b n n ( ) n 1 (b 0 0 ). Since x n is the midpoint of the intervl [ n 1, b n 1 ], nd α fll in the intervl [ n 1, b n 1 ], then α x n 1 (b n 1 n 1 ). It follows tht α x n 1 (b n 1 n 1 ) 1 ( ) n 1 1 (b 0 0 ) ( ) n 1 (b 0 0 ). So, if we wnt x n such tht α x n ɛ for some ɛ, then we set ( ) n 1 (b ) ɛ nd solve for n.. Newton s Method Newton s Method is the clssic method used to find root of function. To use Newton s method, we must first hve some initil pproximtion, x 0, to the root α. We improve this initil pproximtion to 4

5 obtin better pproximtion, x 1, where x 1 is the root of the tngent line pproximtion of the function t (x 0, f(x 0 )). This line is given by y f(x 0 ) f (x 0 )(x x 0 ). So, y f (x 0 )(x x 0 ) + f(x 0 ). Setting y 0 nd solving for x we hve tht x x 0 f(x 0) f (x 0 ) nd x 1 x is our new pproximtion to α. We continue this itertive process to find pproximtions x, x 3,... This gives the Newton s Method x n+1 x n f(x n) f (x n ). We cn lso derive Newton s Method using Tylor s Theorem. Given n pproximtion x n, we expnd the function f bout the point x n : f(x) f(x n ) + (x x n )f (x n ) + (x x n) f (ξ n ), where ξ n is some point between x nd x n. To derive Newton s Method, set f(x) 0, nd solve for x. This gives x x n f(x n) f (x n ) 1 (x x n) f (ξ n ) f (x n ). By dropping the reminder term nd ltering nottion, we hve the newton s method x n+1 x n f(x n) f (x n ). Newton s Methods is not globl method. There re exmples in which convergenve will be poor or even for which convergence does not occur. However, if f, f nd f re ll continuous ner the root, nd if f does not equl zero t the root, then Newton s method will converge whenever the initil guess is sufficiently close to the root. Furthermore, Newton s Method converges qudrticlly (the number of correct digits roughly doubles ech itertion). We now turn to the convergence of Newton s method more precisely: Newton s Error Formul Let f C (I) be given, for some intervl I R with f(α) 0 for some α I. For given x n I, define x n+1 x n f(x n) f (x n ). Then there exists point ξ n between α nd x n such tht Proof. We first expnd f bout the point x n : α x n+1 1 (α x n) f (ξ n ) f (x n ). f(x) f(x n ) + (x x n )f (x n ) + (x x n) f (ξ n ) 5

6 for some ξ n between α nd x n. Set x α to get 0 (x n ) + (α x n )f (x n ) + (α x n) f (ξ n ) (x n α) f(x n) f 1 (x n ) (α x n) f (ξ n ) f (x n ) ( x n f(x ) n) f α 1 (x n ) (α x n) f (ξ n ) f (x n ) x n+1 α 1 (α x n) f (ξ n ) f (x n ) α x n+1 1 (α x n) f (ξ n ) f (x n ) s desired. We now show tht Newton s method converges qudrticlly. Theorem 8. Let f C (I), where α I R is root nd I is n open intervl. Assume tht f (α) 0 nd let the vlues x n be defined by pplying Newton s method to f. Then for x 0 sufficiently close to α we hve tht lim n x n α nd α x n+1 lim n (α x n ) f (α) f (α). Proof. Since f (α) 0 nd f is continuous we cn find closed intervl J I round α such tht f(x) 0 for ll x J. Since J is closed nd f, f re continuous on J, by the extreme vlue theorem, M mx x J f (x) min x J f (x) exists. In order to lter use M in bounding α x n+1 we need to ensure tht the entire sequence of Newton s itertes is in J. Recll from bove tht α x n+1 1 (α x n) f (ξ n ) f (x n ). If we choose x 0 J such tht M α x 0 < 1, then it follows tht α x 1 1 (α x 0) f (ξ 0 ) f (x 0 ) (α x 0 ) mx x J f (x) min x J f (x) (α x 0 ) M α x 0 α x 0 M α x 0. (3) (4) This forces x 1 to be in J since x 0 ws chosen to be in J nd x 1 is closer to α thn is x 0. Note tht from 3 to 4 follows becuse ξ 0 J (since ξ 0 is between α nd x 0 which re both in J), nd so f (ξ 0 ) mx x J f (x). The sme rgument cn now recursively be done to show tht the entire sequence of itertes is in J. It follows tht α x n+1 1 (α x n) f (ξ n ) f (x n ) (α x n) M. 6

7 For convenience, define e n α x n. Then e 1 e 0M e e 1M (e 0M) M e 4 0M 3 (e 0 M) 4 M 1. e n (e 0 M) n M 1 so for e 0 M α x 0 M < 1, convergence occurs since lim n x n α. To see tht holds, we hve α x n+1 lim n (α x n ) f (α) f (α) α x n+1 1 (α x n) f (ξ n ) f (x n ) α x n+1 (α x n ) 1 f (ξ n ) f (x n ) α x n+1 lim n (α x n ) 1 lim f (ξ n ) n f (x n ) α x n+1 lim n (α x n ) 1 f ( lim ξ n) n f ( lim x n) n α x n+1 lim n (α x n ) 1 f (α) f (α). This shows tht Newton s method is qudrtic..3 Secnt Method In Newton s Method, we itertively use the zeros of lines tht re tngent to f to pproximte the root, α. In the secnt method, we use secnt lines. Let x 0 nd x 1 be two initil guesses to the root α. We construct the line tht psses through (x 0, f(x 0 )) nd (x 1, f(x 1 )) nd use its root to define the next iterte, x. This line is given by y y 1 y 1 y 0 (x x 1 ). x 1 x 0 Setting y 0 nd solving for x gives the next itertion x : ( x x 1 f(x 1 ) More generlly, we hve the itertive secnt method ( x n+1 x n f(x n ) x 1 x 0 f(x 1 ) f(x 0 ) ). x n x n 1 f(x n ) f(x n 1 ) Note tht the secnt method is just vrint of Newton s method by replcing f (x n ) with the pproximtion f(xn) f(xn 1) x n x n 1. An dvntge of using secnt method over Newton s method is tht secnt method does not require the derivtive. Error nd Convergence of the Secnt Method ). 7

8 Theorem 9. Let f be two times continuously differentible in neightborhood of root α nd ssume tht f (α) 0 for ll x in this neighborhood. Then for x 0 nd x 1 sufficiently close to α the secnt itertion converges to α with nd α x n+1 lim 0 (5) n α x n α x n+1 lim n (α x n ) p ( 1 f ) p 1 (α) f (6) (α) Proof. Just s in the proof of Newton s convergence, we begin by estblishing n error the error term. Let s be the liner polynomil tht interpolte f t x n 1, x n. Then s(x) From the liner interpoltion error we hve tht x x n x n 1 x n f(x n 1 ) + x x n 1 x n x n 1 f(x n ). (7) f(x) s(x) 1 (x x n 1)(x x n )f (ξ n ), where ξ n is in the intervl defined by x n 1, x n, α. So the error term is given by Since f(α) 0, it follows tht f(α) s(α) 1 (α x n 1)(α x n )f (ξ n ). (8) s(α) 1 (α x n 1)(α x n )f (ξ n ). Futhermore, since s(x n+1 ) 0 by the definition of x n+1, it lso follows tht From the men vlue theorem, From 9 nd 10, we hve which gives Tking the derivtive of s(x) given in 7, we find tht s (ν n ) s(x n+1 ) s(α) 1 (α x n 1)(α x n )f (ξ n ). (9) s(x n+1 ) s(α) s (ν n )(x n+1 α). (10) s (ν n )(x n+1 α) 1 (α x n 1)(α x n )f (ξ n ) (x n+1 α) 1 (α x n 1)(α x n ) f (ξ n ) s (ν n ). (11) f(x n 1) x n 1 x n + f(x n) x n x n 1 f(x n) f(x n 1 ) x n x n 1 f (µ n ). (1) for some µ n. (The lst equlity follows from the men vlue theorem). Using 11 nd 1, the secnt method error term is given by (x n+1 α) 1 (α x n 1)(α x n ) f (ξ n ) f (µ n ). (13) Using the bove error term, 5 quickly follows, given tht the method convergences. 8

9 We now show tht if the initil guesses x 0, x 1 re chosen sufficiently close to α, then the method will converge. Similr to the proof of the newton s method, we cn choose closed intervl J I such tht f is twice continuously differentible on J nd f (α) 0 for ll x J. By the extreme vlue theorem, we know tht f, f both chieve mximum nd minimum vlues on J, so M mx f (x) x J min f (x) x J is well defined. Choose x 0, x 1 J such tht α x i δ for i 0, 1 nd for some δ with Mδ < 1. (Note tht is is possible by choosing x 0, x 1 close enough to α. We now show tht ech iterte x n is still in J. To see this we hve α x 1 α x 1 α x 0 f (ξ n ) f (µ n ) M α x 1 α x 0 α x 0. So, x J. Recursively, it cn be shown in the sme wy tht x n J for ll itertes. Letting e n α x n, it follows tht e e 1 e 0 M < δ M (δm) M 1 e 3 e e 1 M < (δ M)e 1 M < δ 3 M (δm) 3 M 1 e 4 e 3 e M < (δ M)(δ 3 M )M δ 5 M 4 (δm) 5 M 1. e n M 1 (δm) qn where q n is the Fiboncci numbers defined be q n+1 q n 1 + q n. Futhermore, since Mδ < 1 nd q n s n, then lim converges to α. n We now show the order of convergence. From the error term given in 13, we hve so lim n α x n+1 (α x n )(α x n 1 ) 1 f (ξ n ) f (µ n ), Suppose tht the method converges with order p. Then α x n+1 (α x n )(α x n 1 ) 1 lim f (ξ n ) n f (µ n ) 1 f (α) f (α). (14) For some some nonzero, rel number C. By shifting, we lso hve tht α x n+1 lim n (α x n ) p C (15) lim n α x n C. (16) (α x n 1 ) p Rising 16 to the p power, we hve lim n (α x n ) p (α x n 1 ) p C p. (17) 9

10 By multiplying 15 nd 17 together, we get lim n Furthermore, by multiplying 14 nd 15 together, we get lim n α x n+1 (α x n 1 ) p C p+1. (18) α x n+1 (α x n 1 ) p+1 C 1 f (α) f (α). (19) Since the order of convergence is unique, 18 nd 19 implies tht p p + 1 nd C p+1 C 1 f (α) f (α). Solving this gives p 1 (1 + 5) nd C ( 1 f ) 1/p (α) f. (α).4 Fixed Point Itertion Given function g nd point α such tht g(α) α x 0, we sy tht α is fixed point of g. We define the fixed point itertion of g by x n+1 g(x n ) which gives rise to the sequence x 0, x 1, x,... which is hoped to converge to point α. If g is continuous, nd it s fixed point itertion converges to α, then one cn show tht α must be fixed point of g. Although root finding should not be confused with finding fixed points, in mny cses, point α is root of function f if nd only if α is fixed point of different function g. So, in this section we study fixed point itertions. We first present theorem tht summrizes the conditions under which given function will hve fixed point, the conditions under which fixed point itertion will converge for n given x 0 on given intervl, nd lso provides us with the error estimtes of this convergence. Theorem 10. Let g C([, b]) with g(x) b for ll x [, b], then 1. g hs t lest one fixed point in [, b].. If there exists vlue γ < 1 such tht for ll x nd y in [, b], then: () there is unique fixed point, α g(x) g(y) γ x y (b) the itertion x n+1 g(x n ) converges to α for ny initil guess x 0 [, b]. (c) we hve the error estimte 3. If g is continuously differentible on [, b] with then α x n γn 1 γ x 1 x 0. mx x [,b] g (x) γ < 1, 10

11 () there is unique fixed point, α (b) the itertion x n+1 g(x n ) converges to α for ny initil guess x 0 [, b]. (c) we hve the error estimte (d) The limit holds. Proof. (1). Define h(x) g(x) x. Then, nd α x n γn 1 γ x 1 x 0. α x n+1 lim g (α) n α x n h() g() 0 h(b) g(b) b 0. This follows from the ssumption tht g(x) b for ll x. By the intermedite vlue theorem, h hs root α in [, b]. i.e there exists α [, b] such tht h(α) 0, which gives tht g(α) α. () Suppose tht there exist α, β such tht g(α) α, g(β) β. By hypothesis This implies tht g(α) g(β) α β γ α β. (1 γ) α β 0. Since γ 1, it must be the cse tht α β 0. So, α β. This shows tht g hs unique fixed point, α. (b) We hve Recursively we find tht So s n, α x n 0. This shows convergence. (c) First note tht α x n g(α) g(x n 1 ) γ α x n 1. α x n γ n α x 0. α x 0 g(α) g(x 0 ) + x 1 x 0 g(α) g(x 0 ) + x 1 x 0 γ α x 0 + x 1 x 0. So, It recusively follows tht α x γ x 1 x 0. α x n g(α) g(x n 1 ) γ α x n 1 γ α x n. γ n α x 0 γ n 1 γ x 1 x 0. 11

12 This proves (c). (3) From the men vlue theorem, g(x) g(y) g (c)(x y) for some point c [, b]. Let γ mx x [,b] g (x). Then, From () we know tht (), (b), nd (c) holds. (d) We hve g(x) g(y) g (c) x y γ x y. α x n+1 α x n g(α) g(x n) α x n g (c)(α x n ) α x n g (c) for some c between α nd x n. Since x n α s n, it follows tht α x n+1 lim lim n α x n n g (c) g ( lim c) n g (α). (Note tht this show tht fixed point itertions converge linerly). Theorem 11. Consider the fixed point itertion x n+1 g(x n ) where g is p times continuously differentible nd α g(α). If but g (α) g (α)... g (p 1) (α) 0 g p (α), 0 then the fixed point itertion converges with order p for x 0 sufficiently close to α. 3 Approximtion 3.1 Approximtion Theorem of Weierstrss In this section, we show tht every continuous function f on closed intervl [, b] cn be pproximted rbitrrily well by polynomil. Theorem 1. Let < < b < +, nd suppose tht f C[, b] is n rbitrry continuous function. Then for every ɛ > 0, there exists n n N nd polynomil p P n such tht f p < ɛ. Proof. Since every intervl [, b] cb be mpped to [0, 1], WLOG we choose [, b] [0, 1].Consider the sequence of (B n f)(x) f k0 ( k n 1 ) ( ) n x k (1 x) n k. (0) k

13 We show tht this sequence converges uniformly to f. Let q nk (x) : ( n k) x k (1 x) n k. By the binomil theorem, we get tht ( ) n 1 [x + (1 x)] n x k (1 x) n k q nk (x). k So, k0 f(x) (B n f)(x) f(x) f(x) k0 q nk (x) (B n f)(x) k0 q nk (x) k0 f( k n )q nk(x) k0 [f(x) f( n k )]q nk(x). k0 By the continuity of f, for ny ɛ > 0, there exists δ > 0 such tht f(x) f( n k ) < ɛ if x n k < δ. With this in mind, we split the sum in the following wy: f(x) (B n f)(x) N [f(x) f( n k )]q nk(x) + N [f(x) f( n k )]q nk(x) where k N if x n k < δ nd k N if x n k δ. This gives f(x) (B n f)(x) N f(x) f( n k ) q nk(x) + N f(x) f( n k ) q nk(x). For the first sum, we hve f(x) f( n k ) q nk(x) ɛ q nk (x) ɛ N N Let M mx x [0,1]. For the second sum we get q nk (x) ɛ. k0 N f(x) f( n k ) q nk(x) f(x) f( n k ) (x k δ N M n δ (x k n ) q nk (x) k0 n ) q nk (x) M n δ (x x k n + k n )q nk(x). k0 We show tht thus summtion is less thn ɛ by considering ech of the three terms seprtely. We hve x k0 k n q nk(x) x x k0 x n x, k1 n k0 q nk (x) x, ( ) k n x k (1 x) n k n k (n 1)! (n k)!(k 1)! xk 1 (1 x) (n 1) (k 1) 13

14 nd So, k0 k n q nk(x) x x n k0 k1 k n ( n k ) x k (1 x) n k ( ) k n 1 x (k 1) (1 x) (n 1) (k 1) n k 1 ( n ( ) n 1 [(k 1) + 1] )x k 1 (1 x) (n 1) (k 1) k 1 k1 n 1 n x ( ) n x k (1 x) (n ) (k ) + x k n k n 1 n x + x n x + x (1 x). n f(x) f( n k ) q nk(x) M δ (x x + x x n (1 x)) M δ ( x (1 x)). n N Since x n (1 x) reches its mximum vlue t x 1, it follows tht N f(x) f( k n ) q nk(x) M δ ( 1 4n ). Thus, if we choose n such tht n > M δ ɛ we get tht f(x) (B n f)(x) < ɛ. 3. Difference Approximtions to the first derivtive: From Clculus we hve the one sided difference pproximtion f (x) Using Tylors series expnsion of f(x + h), we hve tht f(x + h) f(x). h f (x) f(x + h) f(x) h f (x) f(x) + hf (x) + h f (ξ x, h) h h f (ξ x, h). 14

15 So, this one-sided pproximtion is of O(h). For better pproximtion, we use the Tylor expnsions of f(x + h) nd f(x h) s follows: f(x + h) f(x) + hf (x) + h f (x) + h3 6 f (ξ x1, h) f(x h) f(x) hf (x) + h f (x) h3 6 f (ξ x, h) f(x + h) f(x h) h f (x) + h 6 (f (ξ x1, h) + f (ξ x, h)) f (x) f(x + h) + f(x h) h h 6 (f (ξ x1, h) + f (ξ x, h) ) f(x + h) f(x h) h h 6 f (η) where the lst line follows from the Discrete Averge Vlue Theorem. pproximtion to f (x) nd it is of O(h ). This is the centrl difference 3.3 Approximting f by Interpoltion We cn lso use interpolting polynomils to pproximte the derivtive of function. The interpoltion error theorem gives tht 1 f(x) p n (x) (n + 1)! w n(x)f n+1 (ξ x ). Reclling tht ξ x is function of x it follows tht ( f (x) p 1 n(x) w (n + 1)! n(x)f n+1 (ξ x ) + w n (x) d ) dx f n+1 (ξ x ), so, f (x i ) p n(x i ) 1 (n + 1)! w n(x i )f n+1 (ξ xi ). (1) Let s use p (x), the qudrtic pproximtion of f t eqully spced nodes x 0, x 1, x, to pproximte the derivtive of f t these nodes. By Lgrnge s interpoltion, p (x) is given by The pproximtion of f (x i ) is given by with error where the lst line follows from 1. Using p (x) L 0 (x)f(x 0 ) + L 1 (x)f(x 1 ) + L (x)f(x ). f (x i ) p (x) L 0(x)f(x 0 ) + L 1(x)f(x 1 ) + L (x)f(x ), f (x i ) p (x i ) 1 6 w (x i )f 3 (ξ xi ), L 0(x) (x x 1) + (x x ) (x 0 x 1 )(x 0 x ), L 1(x) (x x 0) + (x x ) (x 1 x 0 )(x 1 x ), nd L (x) (x x 0) + (x x 1 ) (x x 0 )(x x 1 ), we find tht ( 3h f (x 0 ) f(x 0 ) h 1 h ) + f(x 1 ) ( ) h h ( 3 f(x 0) + f(x 1 ) 1 f(x ) 1 h ( f(x ) + 4f(x 1 ) 3f(x 0 )), 15 + f(x ) ) ( ) h h

16 with error Similrly, we find tht 1 6 w (x 0 )f 3 (ξ x0 ) h 3 f 3 (ξ x0 ). f (x 1 ) 1 h (f(x ) f(x 0 )) nd f (x ) 1 h (3f(x ) 4f(x 1 ) + f(x 0 )) with errors h 6 f 3 (ξ x1 ) nd h 3 f 3 (ξ x ) respectfully. 3.4 Lest Squres Approximtion Suppose, for given set of dt points, we wish to construct function f tht best represents the dt. The method of lest squres tkes the pproch of defining f to be the stright line tht minimizes the sum of the squre of the distnces between the dt points nd the line. Exmple 13. Lest Squres Liner Approximtion Let the experimentl dt be defined s (x k, y k ), 1 k n. The lest squres pproch is to find y c 1 x+c 0 such tht is minimized. Note tht F (m, b) ((c 1 x k + c 0 ) y k ) () k1 F (m, b) ((c 0 + c 1 x 1 ) y 1 ) + ((c 0 + c 1 x ) y ) ((c 0 + c 1 x n ) y n ) c 0 + c 1 x 1 y 1.. c 0 + c n x n y n 1 x 1 ( ) y 1 c0.. c 1. 1 x n y n Ax b. Since minimizing Ax b is equivlent to minimizing Ax b, we focus on the ltter. In generl, s seen in the bove exmple, lest squres problem cn be formulted in the form min Ax b (3) x Cn where A C m n nd b C m re given, nd m n. The key in solving the bove minimiztion problem is the notion of othogonl projections. The gol is to find the point, Ax, in the rnge(a) tht is closest to b. We show tht the point Ax P b, where P is the orthogonl projection tht mps C m onto the rnge(a), is the desired point. In other words, r b Ax (the vector connecting Ax nd b) must be orthogonl to rnge(a). Theorem 14. Let A C m n (m n) nd b C m be given. A vector x C n minimizes the residul norm r b Ax, thus solving the lest squres problem 3 if nd only if r is orthogonl to rnge(a). i.e A r 0, (4) or equivlently, A Ax A b, (5) 16

17 or equivlently P b Ax, (6) where P C m m is the orthogonl projection onto rnge(a). The n n system of equtions 5, known s the norml equtions, is nonsingulr if nd only if A hs full rnk. Thus, the solution x is unique if nd only if A hs full rnk. Proof. We first show tht 4 nd 5 re equivlent, nd tht 4 nd 6 re equivlent. We then prove tht vector x minimizes the residul norm if nd only if 6 holds. We finlly show tht the solution is unique if nd only if A hs full rnk. To see tht 4 nd 5 re equivlent, we hve A r 0 A (b Ax) 0 A b A Ax 0 A b A Ax. To see tht 4 nd 6 re equivlent recll tht by the properties of orthogonl projections, b P b is orthogonl to rnge(a). i.e. b P b ker(a ). If P b Ax, then 0 A (b P b) A (b Ax) A r. Conversely, if A r 0, since A (b P b) 0, it follows tht A (b Ax) A (b P b) 0. Thus, A (P b Ax) 0,. i.e. (P b Ax) is orthogonl to rnge(a). Since P b Ax is in rnge(a), then P b AX is orthogonl to itself. Hence P b Ax 0. This proves the equivlence of 4 nd 6. To see tht vector x minimizes the residul norm if nd only if 6 holds, let z P b be in rnge(a). Since P b is lso in rnge(a), P b z is in rnge(a). So, b P b is orthogonl to P b z. It follows tht b z (b P b) + (P b z) (b P b) + (P b z) > (b P b). This shows tht P b is the vector in rnge(a) tht minimizes the residul norm. So, x is solution to 3 if nd only if x is such tht Ax P b. We now show tht the solution is unique if nd only if A hs full rnk. If A hs full rnk, then A A hs full rnk, so 5 is uniquely solvble. Conversely, suppose Ax 0 for some nonzero x. Then A Ax 0, so A Ax is singulr nd 5 is not uniquely solvble. Lest Squres Approximtion to Function The method of lest squres is not restricted to dt fitting. We cn lso use this concept to find n pproximtion to given function f in terms of set of bsis functions {φ k, 1 k n}. We seek constnts c k, 1 k n such tht q n f when n k1 c k φ k best pproximtes f. In the lest squre sense, q n best pproximtes R n f q n (7) is minimzed. To solve this problem, we use the concept of inner products on rel vector spce. Note tht for ny norm, it follows tht 17

18 R n f q n (f q n, f q n ) (f q n ) (f q n ) f f f q n q nf + q nq n f f f q n + qnq n f f (c 1 φ c n φ n ) + (c 1 φ c n φ n ) (c 1 φ c n φ n ) f c k (f, φ k ) + c i c j (φ i, φ j ). k1 i1 j1 Thus, R n is function of n vribles c 1,..., c n, nd it is minimzed when the grdient is zero. Note tht so R n c i (f, φ i ) + R n c i 0 c j (φ i, φ j ), j1 c j (φ i, φ j ) (f, φ i ). This gives the liner system (φ 1, φ 1 ) (φ 1, φ )... (φ 1, φ n ) c 1 (φ, φ 1 ) (φ, φ )... (φ, φ n ) c (φ n, φ 1 ) (φ n, φ )... (φ n, φ n ) j1 c n (f, φ 1 ) (f, φ ). (f, φ n ). Suppose our bsis functon {φ n } is n orthogonl set. Then the bove system becomes (φ 1, φ 1 ) c 1 (f, φ 1 ) 0 (φ, φ )... 0 c (f, φ )., (φ n, φ n ) (f, φ n ). c n nd it follows tht c k (f, φ k) (φ k, φ k ). 4 Orthogonl Polynomils In this section, we briefly discuss the existence of orthogonl polynomils (given norm defined by weight function), properties of orthogonl polynomils, nd common fmilies of orthogonl polynomils. Theorem 15. Let w be integrble in [, b] nd non-negtive, i.e., x [, b]. For given f nd g in C([, b]), define (f, g) w s (f, g) w Then (, ) w defines n inner product on C([, b]). Proof. This follows immeditely from the properties of inner products. w(x)dx is defined nd w(x) 0 for ll w(x)f(x)g(x)dx, (8) 18

19 Theorem 16. Let w be given non-negtive weight function on n intervl [, b], nd (, ) w the ssocited iner product, defined by 8. Then there exist fmily of polynomils {φ k }, φ k P k, 0 k N, tht re orthogonl with respect to (, ) w such tht 1. The set {φ 0, φ 1,..., φ N } is bsis for P N.. If p k is n rbitrry element of P k for k < N, then (p k, φ N ) 0 for ll N > k. i.e orthogonl polynomils re orthogonl to ll polynomils of strickly lower degree. 3. For j 1, the roots of ech φ j re ll in [, b] nd re ll distinct. Proof. The proof is lengthy (due to the length of the sttement), but it is not difficult. We first show tht the orthogonl fmily exists. We strt with the set {q k }, q k x k, 0 k N, nd use Grm-Schmidt to form the orthogonl fmily {φ k }. Let φ 0 1. Then φ k, for 1 k N, is found by the formul k 1 (φ j, q k ) w φ k (x) q k (x) φ j (x), (9) (φ j, φ j ) w j0 nd {φ k } is orthogonl, with respect to w, by construction. Since {φ k } is n orthogonl set, then it is lso linerly independnet. Since P N hs dimension N + 1 nd {φ k } is set of N + 1 linerly independent vectors, it follows tht {φ k } is bsis for P N. This proves 1. For., write p k s k p k j φ j (x). Then since {φ k } is orthogonl. (p k, φ N ) ( j0 k j φ j (x), φ N ) j0 To see 3., first suppose tht φ j hs no root in [, b]. Then (φ 0, φ j ) k j (φ j (x), φ N ) 0 j0 w(x)φ 0 (x)φ j (x)dx 0 since w(x)φ 0 (x)φ j (x) does not chnge signs in [, b]. This is contrdition, so φ j must hve root in [, b]. Now, suppose x 1 [, b] is ny root of φ j such tht it is multiple root. We cn write φ j (x) s φ j (x) (x x 1 ) r(x) for some polynomil r(x). Then r(x) φ j (x)(x x 1 ) is polynomil of degree j, so (φ j, r(x)) 0. However, (φ j (x), r(x)) w(x)(φ j (x)) (x x 1 ) dx > 0 since ech fctor is positive. This is contrdiction. So ny root of φ j (x) must be simple. Suppose now tht only some of the roots of φ j re in [, b]. Lbel these roots x 1,..., x i where i < j.then φ j (x) r(x)(x x 1 )...(x x i ), nd since r(x) hs no root in [, b], then r(x) does not chnge sign in [, b]. Thus, w(x)φ j (x)(x x 1 )...(x x i ) w(x)r(x)(x x 1 )...(x x i ) 19

20 does not chnge sign in [, b]. So, w(x)r(x)(x x 1 )...(x x i ) dx > 0. However, since (x x 1 )...(x x i ) hs degree i < j, it follows tht 0 (φ j, (x x 1 )...(x x i )) w(x)φ j (x)(x x 1 )...(x x i )dx w(x)r(x)(x x 1 )...(x x i ) dx. This is contrdiction, so i j. i.e ll of the roots of φ j lie in [, b]. Fmilies of Orthogonl Polynomils The four most common fmilies of orthogonl polynomils re Legendre Polynomils, Chebyshev Polynomils, Hermite Polynomils, nd Lguerre Polynomils. Before introducing these fmilies, note tht scling orthogonl polynomils by non zero constnt still results in orthogonl polynomils. Thus in some cses, specific scling contrint is dded so tht the resulting fmily of orthogonl polynomils is specific set. 1. Legendre Polynomils: The Legendre polynomils re the orthogonl polynomils on [ 1, 1] with weight function w(x) 1. i.e., for i j The usul scling is to tke P n (1) P i (x)p j (x)dx 0.. Chebyshev Polynomils: The Chebyshev polynomils re the orthogonl polynomils on [ 1, 1] with weight function w(x) 1 1 x. i.e., for i j 1 1 T i (x)t j (x) dx 0. 1 x The usul scling is to set the leding coefficient to n Hermite Polynomils: The Hermite polynomils re the orthogonl polynomils on the entire rel line with weight function w(x) e x. i.e., for i j e x H i (x)h j (x)dx 0. The usul scling is to set the leding coefficient to n. 4. Lguerre Polynomils: The Lguerre polynomils re the orthogonl polynomils on the positive rel line with weight function w(x) e x. i.e., for i j 0 e x L i (x)l j (x)dx 0. The usul scling is to set the leding coefficient to ( 1)n n!. 0

21 5 Interpoltion 5.1 Liner Interpoltion In Liner Intepoltion, we use stright line to pproximte function f. Given x 0, x 1 we pproximte f by interpolting t the points (x 0, f(x 0 )), (x 1, f(x 1 )) Tht is, we find the stright line tht psses through these points. Using Lgrnge interpoltion formul, we hve p 1 (x) x x 1 x 0 x 1 f(x 0 ) + x x 0 x 1 x 0 f(x 1 ) is the polynomil tht interpoltes f t x 1, x. We use Rolle s theorem to determine the ccurcy of the liner interpoltion: Define E(x) f(x) p(x), w(x) (x x 0 )(x x 1 ), G(x) E(x) w(x) w(t) E(t), for t (x 0, x 1 ). Then G(x i ) G(t) 0, nd be Rolle s Theorem, there exists c 1 (x 0, t) nd c (t, x 1 ) such tht G (c 1 ) G (c ) 0.Thus, by pplying Rolle s theorem to G we hve tht there exists c 3 (x 0, x 1 ) such tht G (c 3 ) 0. This gives tht, So, Thus, 0 G (c 3 ) f (c 3 ) w(t) E(t). E(t) 1 (t x 0)(t x 1 )f (c 3 ). E(t) 1 (t x 0)(t x 1 ) f (c 3 ) E(t) 1 mx x 0 t x 1 (t x 0 )(t x 1 ) f (c 3 ). By tking the derivtive, we find tht t (x0+x1) gives the mx for (t x 0 )(t x 1 ), so it follows tht E(t) 1 8 (x 1 x 0 ) ( mx f (x) ). x 0 x x 1 5. Lgrnge Interpoltion The liner interpoltion given bove is cse of Lgrnge interpoltion. Theorem 17. Let the nodes x i I, 0 i n nd y i f(x i ). If ech x i is distinct, then there exists unique polynomil p n of degree less thn or equl to n, such tht Proof. We first prove existence. For ech i define L ni (x) p n (x i ) y i. n k i,k0 x x k x i x k. Note tht Define L ni (x) p n (x) { 1 : i j 0 : i j L ni (x)y i. i0 1

22 It follows tht p n (x i ) y i. Since ech L ni is of degree t most n, p n is of degree t most n. This proves existence. To prove uniqueness, suppose there exists nother polynominl q of degree t most n such tht q(x i ) y i for ll i. Then, r(x) p n (x) q(x) is polynomil of degree t most n, nd r(x i ) p n (x i ) q(x i ) 0, so r hs n + 1 roots. This forces r 0. i.e. p n (x) q(x) which proves uniqueness. 5.3 Newton s Interpoltion A drwbck of the Lgrnge interpolton formul is tht if new dt point x n+1 is dded, we must reclulte the functions L ni completely. In other words, we cnnot esily write p n+1 in terms of the previously found p n. An lternte form of the polynomil, know s the the Newton s form, voids this problem. Theorem 18. Newton s Interpoltion Construction Let p n be the polynomil of degree t most n tht interpoltes f t the nodes x i, 0 i n. Let p n+1 be the polynomil of degree t most n + 1 tht interpoltes f t the nodes x i, 0 i n + 1. Then p n+1 is given by p n+1 (x) p n (x) + n+1 w n (x), where nd w n (x) n (x x i ), i0 n+1 f(x n+1) p n (x n+1 ), w n (x n+1 ) 0 f(x 0 ). Proof. Since the interpoltion polynomil is unqiue, we need only to show tht p n+1 stisfies the interpoltion conditions. For x i, 0 i n, For x n+1, p n+1 (x i ) p n (x i ) + n+1 w n (x i ) p n (x i ) f(x i ). p n+1 (x n+1 ) p n (x n+1 ) + n+1 w n (x n+1 ) p n (x n+1 ) + f(x n+1) p n (x n+1 ) w n (x n+1 ) w n (x n+1 ) f(x n+1 ). Thus, p n+1 interpoltes f t the points x i, 0 i n + 1 nd it hs degree t most n + 1, so we re done. Corollry 19. For { k } nd w n s defined in theorem 18, we hve Proof. With p 0 (x) f(x 0 ), we hve n 1 p n (x) (x x 0 ) + (x x 0 )(x x 1 ) n (x x k ). p 1 (x) p 0 (x) + 1 w 0 (x) f(x 0 ) + 1 (x x 0 ) (x x 0 ). k0

23 n Suppose p n 1 (x) (x x 0 ) + (x x 0 )(x x 1 ) n 1 (x x k ) holds. Then k0 s desired. The form p n (x) p n 1 (x) + n w n 1 (x) n (x x 0 ) + (x x 0 )(x x 1 ) n 1 (x x k ) + n w n 1 (x) k0 n (x x 0 ) + (x x 0 )(x x 1 ) n (x x k ) n 1 p n (x) (x x 0 ) + (x x 0 )(x x 1 ) n (x x k ) is clled Newton form of the interpoltion polynomil, nd the coefficients k re clled divided differences. We use divided differences formul to form the coeffiecients k. We first introduce the terminology j-th divided difference t node x k denoted by f j (x k ). These vlues re recursively found, where f 0 (x k ) f(x k ) nd f j (x k ) f j 1(x k+1 ) f j 1 (x k ) x k+j x k. In prctice, tble is used to keep trck of the vlues. Once ll vlues re found, it follows tht i f i (x 0 ), nd the newton form of the interpoltion polynomil is given by p n (x) f(x 0 ) + f 1 (x 0 )(x x 0 ) + f (x 0 )(x x 0 )(x x 1 ) f n (x 0 )(x x 0 )(x x 1 )... (x x n 1 ). k0 k0 5.4 Interpoltion Error Theorem 0. Let f C n+1 [, b] nd let the nodes x k [, b] for 0 k n. Then for ech x [, b] there exists ξ x [, b] such tht f(x) p(x) w n(x) (n + 1)! f n+1 (ξ x ) Proof. Define w(x) n (x x i ), E(x) f(x) p n (x), nd for t [, b], t x i, G(x) E(x) w(x) w(t) E(t). i0 Then G(x i ) 0 for 0 i n nd G(t) 0. Thus G is identicl on n + points. By repeted ppliction of Rolle s theorem, it follows tht there exist ξ t such tht G n+1 (ξ t ) 0. i.e. Solving for E(t) gives 0 G n+1 (ξ t ) f n+1 (ξ n ) f(t) p n (t) (n + 1)!E(t). w(t) w(t) (n + 1)! f n+1 (ξ t ) for ll t [, b] with t x i. Futhermore, since f(x i ) p n (x i ) 0 by definition, it holds tht for ll x [, b]. f(x) p n (x) w(x) (n + 1)! f n+1 (ξ n ) 3

24 5.5 Hermite Interpoltion In Hermite interpoltion, we find polynomil H n (x) tht interpoltes f nd f t the the n nodes x 1,..., x n. Theorem 1. Given the n nodes x i 1 i n nd differentible function f(x), if the nodes re distinct then there exists unique polynomil H n of degree less thn or equl to n 1, such tht H n (x i ) f(x i ), H n(x i ) f (x i ). Proof. This is bscilly the sme s existence nd uniqueness of the interpoltion polynomil of degree t most n. We define two fmilies of polynomils nd Then note tht nd It follows tht h k (x) [1 L nk(x k )(x x k )](L nk (x)) H n (x) ĥ k (x x k )(L nk (x)). h k (x j ) δ kj, h k(x j ) 0 ĥ k (x j ) 0, ĥ k(x j ) δ kj. (f(x k )h k (x) + f (x k )ĥk(x)) k1 interpoltes f nd f t the nodes. Uniquness is shown by the sme unqiuness rgument for regulr interpoltion. Theorem. Let f C n ([, b]) nd let the nodes x k [, b] for ll k, 1 k n. Then, for ech x [, b], there is ξ x [, b] such tht f(x) H n (x) ψ n(x) (n)! f (n) (ξ x ). Proof. This proof is essentilly the sme s (but slightly different from) the proof given in the interpoltion error. Define ψ(x) n (x x i ), E(x) f(x) H n (x), nd for t [, b], t x i, G(x) E(x) ψ(x) ψ(t) E(t). i0 Then G(x i ) 0 for 1 i n nd G(t) 0. Thus G is identicl on tlest n + 1 points. By rolles s theorem, there exist points c i in between ech consecutive zero of G such tht G (c i ) 0. i.e. G hs t lest n zeros, non of which re locted t the nodes. Also note tht G (x i ) 0 by construction. So, G hs t lest n zeros in [, b]. By repeted ppliction of Rolle s theorem, it follows tht there exist ξ t such tht G n (ξ t ) 0. i.e. 0 G n (ξ t ) f n (ξ n ) (n)!e(t). ψ(t) Solving for E(t) gives f(t) H n (t) ψ(t) (n)! f n (ξ t ) for ll t [, b] with t x i. Futhermore, since f(x i ) H n (x i ) 0 by definition, it holds tht for ll x [, b]. f(x) H n (x) ψ(x) (n)! f n (ξ x ) 4

25 5.6 Spline Interpoltion The bsic ide is to construct piecewise polynomil pproximtion tht not only interpoltes given dt or function vlues, but which lso is smooth mening continuously differentible to some degree. Suppose we re given n intervl [, b] tht is prtitioned into subintervls by the nodes x 0,..., x n. Let l be nonnegtive integer. A function s : [, b] R is clled polynomil spline of degree l over [, b] provided tht it possesses the following properties: () s is l 1-times continuously differentible. (b) s is polynomil of degree t most l on ech subintervl. Define Ω n : {x 0, x 1,..., x n }. Let S l (Ω n ) denote the set of ll polynomil splines of degree l with the prtition Ω n. Theorem 3. The set S l (Ω n ) is liner spce of dimension (n + l) nd bsis is given by the functions {p 0, p 1,..., p l, q l1,..., q l,n 1 } where p k (x) x k for 0 k l nd { q lν (x) (x x ν ) l (x xν ) + l : x x ν 0 : x < x ν Proof. Define I k [x 0, x k ]. Gol: Show for ech 1 k n tht s(x) l λ0 k 1 λ x λ + b ν (x x ν ) l +. For k 1, i.e, I k [x 0, x 1 ], we hve tht s is polynomil of degree t most l by definition. So where we define ν1 s(x) x l x l + 0 b ν (x x ν ) l +, ν1 0 b ν (x x ν ) l + 0 ν1 so the clim holds true for k 1. The proof proceeds by induction. Suppose tht the clim holds true for I k for some k. Define l k 1 ρ(x) : s(x) λ x λ b ν (x x ν ) l +. λ0 Then ρ 0 for ll x I k, ρ(x) C l 1 (I k+1 ), nd ρ P l. Thus ρ is solution to the differentil eqution with initil conditions y l+1 (x) 0 ν1 y(x k ) y (x k )... y (l 1) (x k ) 0. Since initil vlue problems re unique up constnt multiple, nd (x x k ) l + is lso solution, then ρ(x) b k (x x k ) l + 5

26 for some b k. Thus on I k+1, which gives So, in prticulr, it follows tht on I n [, b]. b k (x x k ) l + s(x) s(x) l λ0 l λ0 l λ0 k 1 λ x λ b ν (x x ν ) l +, ν1 k 1 λ x λ + b ν (x x ν ) l +. ν1 n 1 λ x λ + b ν (x x ν ) l + Interpolting Splines In mny cses splines re used to interpolte function t given set of nodes Ω {x 0,.., x n } tht prtitions the intervl [, b] into n intervls with x 0, x n b. Most commonly, splines of odd degree re used (see below for rough explntion of why this is the cse). The simplest exmple of spline of odd degree is liner spline interpolte which is uniquely defined in ech subintervl s the liner polynomil tht interpoltes t the endpoints. More generlly, we now consider splines of order m 1 for m. Suppose we wnt to find spline s S m 1 (Ω n ) where s interpoltes function f. From bove, dim(s m 1 (Ω n )) n + m 1. The fct tht s must interpolte f t the (n + 1) nodes x 1,..., x n, gives (n+1) constrints, but there remins (m ) free prmeters which cn be used in vrious wys. The following three uses of these free prmeters re the most common: Hermite End Condition, Ntul End Conditions, Periodic End Conditions. In ech of the bove, the dditonl m constrints re imposed on the endpoints nd b. This is why odd degree is commonly used; since the degree is odd, there is n even number of free prmeter which cn be split evenly mong the two endpoints. 1. Hermite End Conditions (k Clmped) Given f C m [, b] find s S m 1 (Ω n ) such tht ) s(x ν ) f(x ν ) for ν 0, 1,..., n ν1 b) s (µ) () f (µ) () nd s (µ) (b) f (µ) (b) for µ 0, 1,..., m 1. Nturl End Conditions Given f C m [, b] with m n + 1, find s S m 1 (Ω n ) such tht ) s(x ν ) f(x ν ) for ν 0, 1,..., n b) s (µ) () 0 nd s (µ) (b) 0 for µ m,.., m 3. Periodic End Conditions Let f C m [, b] be such tht f k () f k (b) for k 0,..., m 1. Find s S m 1 (Ω n ) such tht ) s(x ν ) f(x ν ) for ν 0, 1,..., n b) s (µ) () s (µ) (b) for µ 1,..., m The miniml property of spline interpoltion is probbly one of the most importnt properties of spline interpoltion. Before we stte the miniml property, we note tht it is just consequence of the folowing theorem. 6

27 Theorem 4. Integrl Reltion Theorem: Let f C m [, b]m nd let s S m 1 (Ω n ) be n interpolting spline of f. Let g C m [, b] be function tht lso interpoltes f on Ω n such tht m ( 1) µ s m+µ (x)[g m µ 1 (x) s m µ 1 (x)] b 0, then µ0 holds. [g m (x)] dx [g m (x) s m (x)] dx + [s m (x)] dx (30) Proof. Note tht 30 holds if nd only if holds. We show tht 31 holds by using integrtion by prts: s m (x)[g m (x) s m (x)] s m (x)[g m 1 (x) s m 1 (x)] By repeted integrtion by prts, it follows tht s m (x)[g m (x) s m (x)]dx 0 (31) s m+1 (x)[g m 1 (x) s m 1 (x)]dx. m 3 s m (x)[g m (x) s m (x)] ( 1) µ s m+µ (x)[g m µ 1 (x) s m µ 1 (x)] b +( 1)m s m (x)[g (x) s (x)]dx. µ0 Since s is only in c m [, b] by definition, we must compute the lst integrl over the individul subintervls. Futhermore, since s is polynomil of degree t most m 1, it follows tht s m 1 (x) c for some constnt c. So we hve s m (x)[g m (x) s m (x)] It follows tht s desired. i0 i0 i0 i0 xi+1 x i s m (x)[g (x) s (x)]dx ( [s m (x)(g (x) s (x)) xi+1 x i ( [s m (x)(g (x) s (x)) xi+1 c x i xi+1 x i xi+1 x i ) s m 1 (x)(g (x) s (x))dx ) (g (x) s (x))dx ([s m (x)(g (x) s (x)) ) xi+1 c[(g(x x i+1 ) s(x i+1 )) (g(x i ) s(x i ))] i [s m (x)(g (x) s (x))] xi+1 x i i0 s m (x)(g (x) s (x)) b. m s m (x)[g m (x) s m (x)] ( 1) µ s m+µ (x)[g m µ 1 (x) s m µ 1 (x)] b 0 µ0 7

28 Theorem 5. Miniml Property Let f C m [, b]m nd let s S m 1 (Ω n ) be the interpolting spline with respect to one of the interpoltion conditions 1,, or 3 given bove. Let g be n rbitry function in C m [, b] which stisfies the sme interpoltion conditions s s. Then s m g m. (3) Proof. This follows immeditely from Theorem 4. Cubic Splines Cubic splines (m ) re by fr the most hevily used of the spline spces! By the miniml property, we hve tht [s (x)] [g (x)] dx 5.7 Interpoltion t the Chebyshev Nodes We hve lredy seen the Chebyshev polynomils bove s fmily of orthogonl polynomils on the intervl [ 1, 1] with the weight function w(x) 1, where the leding coefficient of T 1 x n is n 1. We now see them in terms of cosine. Define the fmily of functions {T n }, n 0 by the formul T n (x) cos(ncos 1 (x)), x [ 1, 1]. (33) It turns out tht this fmily of function is precisely the fmily of Chebyshev polynomils mentioned previously. Theorem 6. The functions T n (x) stisfy 1. Ech T n is polynomil of degree n;. For n 1, T n+1 (x) xt n (x) T n 1 (x); 3. T n (x) hs leding coefficient n 1, n 1. Proof. We first prove since 1 nd 3 follow. The trick is to set x cos(θ) nd use the trig identity cos(nθ)cos(θ) 1 (cos(n + 1)θ cos(n 1)θ). (34) With x cos(θ), by substitution, T n (x) cos(nθ). Using 34, it follows tht T n+1 (x) cos((n + 1)θ) cos(nθ)cos(θ) cos(nθ) xt n (x) T n 1 (x) which proves. For 1, note tht T 0 (x) 1 hs degree 0, nd T 1 (x) x hs degree 1. By induction, suppose T n hs degree n nd T n 1 hs degree n 1. Then it follows from tht T n+1 hs degree n + 1. Note tht 3 holds for T 1 (x) 1 nd T (x) x 1. Suppose true for T n nd T n+1. From, the clim lso holds for n + 1. We now consider the monic Chebyshev polynomil of degree n, ˆT n, given by ˆ T 0 (x) 1 nd ˆT n (x) 1 n 1 T n(x), n 1. (35) The roots of ˆT n (i.e the roots of T n ) re found by solving cos(ncos 1 (x)) 0. We hve 8

29 Thus, we cn write cos(ncos 1 (x)) 0 ncos 1 (x) cos 1 (0) ncos 1 (x) (k 1) π, k 1,..., n cos 1 (x) (k 1) π k 1,..., n ( n ) (k 1)π x k cos k 1,..., n. n ˆT (x) n (x x k ). (36) Chebyshev polynomils re of gret importnce becuse of the following theorem: Theorem 7. For n rbitrry monic polynomil, p n, of degree n, it holds tht mx p n(x) 1 x 1 k1 where ˆT is the monic Chebyshev polynomil of degree n. Proof. Quickly note tht Next, ssume by contrdiction tht mx ˆT (x) 1, n 1, (37) 1 x 1 n 1 mx ˆT n (x) 1 1 x 1 n 1 mx 1 x 1 cos(ncos 1 x) 1 n 1. mx p n(x) < 1 1 x 1 n 1. By clculus, we know tht ˆT n chieves its extrem on its criticl points. By setting ˆT n(x) 0 (nd using the endpoint -1,1,we find tht the criticl points re given by y k cos( kπ n ) for k 0,..., n nd ˆT n (y k ) ( 1) k. Define d(x) ˆT n (x) p n (x). Since ˆT n (x) nd p n (x) re both monic, then d(x) P n 1. Furthermore, since 1 1 < p(x) < n 1 n 1 it follows tht d(y 0 ) > 0, d(y 1 ) < 0,..., ( 1) n d(y n ) > 0. So d chnges signs t lest n times which mens tht it hs t lest n distinct roots. But d hs degree t most n 1, so d must be the zero polynomil. This is contrdiction, nd the theorem is proved. Interpoltion t Chebyshev nodes Wht is the significnce of the Cheyshev polynomils in interpoltion? Recll tht the interpoltion error bound is given by mx f n+1 (x) x [,b] n f(x) p n (x) (x x i ). (n + 1)! So, if we choose the norm to the the infinity norm, it follows from 7 nd 36 tht n (x x i ) is minimized when the nodes re chosen to be the roots of ˆT n+1. i0 i0 9

30 6 Numericl Integrtion 6.1 Trpezoid Rule One of the most importnt pplictions of Liner Interpoltion is the construction of the trpezoid rule for pproximting definite integrls. Suppose we wnt to pproximte interpoltion of f(x). Then p 1 (x) For the bsic trpezoid rule, we pproximte (x b) (x ) f() + ( b) (b ) f(b). f(x)dx by using f(x)dx. Let p 1 (x) be the liner p 1 (x)dx. This gives f(x)dx p 1 (x)dx 1 (x b) b f() + 1 (x ) b f(b) b 1 (b ) b f(b) 1 ( b) ( b) f() Consider the error I(f) T 1 (f) where I(f) 1 (b )f(b) 1 ( b)f() 1 (b )(f(b) + f()). I(f) T 1 (f) f(x)dx, T 1 (f) From the liner interpoltion error nlysis,we know tht for some c x [, b]. So, f(x)dx p 1 (x)dx. We hve p 1 (x)dx (f(x) p 1 (x))dx. f(x) p 1 (x) 1 (x )(x b)f (c x ) I(f) T 1 (f) 1 (x )(x b)f (c x ). Since (x )(x b) does not chnge signs on [, b], by the integrl men vlue theorem, there exist η [, b] such tht 1 (x )(x b)f (c x ) 1 f (η) (x )(x b)dx 1 ) f (b )3 (η) ( (b )3 f (η). 30

31 We hve shown tht the error of the bcic trpezoid method is 1 1 (b )3 f (η). This error is smll only if we choose very smll intervl [, b]. So, rther thn using the liner interpoltion on the intervl [, b], we subdivide the intervl into n subintervls nd pply the bsic trpezoid rule to ech intervl [x i, x i+1 ] for 0 i n 1. This is clled the composite trpezoid rule, lso known s the n subintervl rule This gives T n (f) n 1 i0 x i+1 x i f(x)dx when the nodes re eqully spced. h (f(x 0) + f(x 1 ) f(x n 1 ) + f(x n )) We now show tht for the composite trpezoid rule on eqully spced intervls, there exists η [, b] such tht I(n) T n (f) b 1 h f (η). We hve I(f) T n (f) n 1 1 f(x)dx (x i x i+1 )[f(x i ) + f(x i+1 )] n 1 xi+1 i0 x i n 1 ( xi+1 i0 i0 i0 n 1 1 f(x)dx (x i x i+1 )[f(x i ) + f(x i+1 )] i0 f(x)dx 1 ) x i (x i x i+1 )[f(x i ) + f(x i+1 )] n 1 (x i+1 x i ) 3 f (ξ i,h ) 1 n 1 h3 1 f (ξ i,h ) i0 h b 1 n b 1 h n 1 f (ξ i,h ) i0 ( n 1 1 n b 1 h f (η) i0 f (ξ i,h ) where the lst equlity follows from the Discrete Averge Vlue Theorem. Note tht this grees with the bsic trpezoid rule, since h b in tht cse. ) 6. Simpson Rule In the trpezoid rule, we used liner interpoltion on ech intervl. Similrly, Simpson rule uses qudrtic interpoltion to pproximte I(f) f(x)dx.let p (x) be the qudrtic polynomil tht interpoltes f(x) 31

32 t x 0, x 1 c b, x b. The bsic Simpson rule is given by where i.e where L 0 (x) S (f) I(p ) (L 0 (x)f() + L 1 (x)f(b) + L (x)f(c))dx (38) (x b)(x c) ( b)( c), L (x )(x c) 1(x) (b )(b c), L (x )(x b) (x) (c )(c b). A S (f) Af() + bf(b) + Cf(c), L 0 (x)dx, B L 1 (x)dx, C To clculte these vlues, we use h b c c b, so Integrting gives A +h L 0 (x)dx, B A h 3 +h +h 1 h +h L 0 (x)dx L 1 (x)dx, C (x b)(x c) ( b)( c) dx +h 1 h 3 h 3 B h 3, C 4h 3. Thus, the bsic Simpson rule is given by (x b)(x c)dx S (f) h (f() + 4f(c) + f(b)) 3 L (x)dx. +h L (x)dx. where h b. Similr to trpezoid rule, we cn form the composite Simpson rule. In the trpezoid rule, the bsic rule is pplied to single subintervl. However, since Simpson rule require three points we pply the bsic rule using two consecutive sub-intervls. Note tht this mens tht the composite Simpson rule requires n odd number of nodes (i.e n even number of subintervls). When pplying the composite rule to the prtition x 0,..., x n it becomes S n (f) n 1 where h i xi+ xi. If we ssume uniform spcing, this becomes i0 h i 3 (f(x i) + 4f(x i+1 ) + f(x i+ )), (39) S n (f) h 3 (f(x 0) + 4f(x 1 ) + f(x ) + 4f(x 3 ) f(x n ) + 4f(x n 1 ) + f(x n )). (40) 3

33 We now tht tht Simpson s rule is exctly for ll polynomils of degree t most 3 (i.e hs degree of precision 3). This is ctully surprising result since Simpson s rule is constructed using only the qudrtic interpolting polynomil. To see this, let q 3 (x) Ax 3 + q (x) be n rbitrry polynomil of degree t most 3. Then S (q 3 ) on n intervl [, b] is given by So the error is given by S (q 3 ) h 3 (q 3() + 4q 3 (c) + q 3 (b)) h 3 A(3 + 4c 3 + b 3 ) + h 3 (q () + 4q (c) + q (b)) S (Ax 3 ) + S (q (x)). I(q 3 ) S 3 (q 3 ) [I(Ax 3 ) + I(q (x))] [S (Ax 3 ) + S (q (x))] A[I(x 3 ) S (x 3 )] + [I(q (x)) S (q (x))]. Since we use qudrtic interpolting polynomil to construct Simpson s rule nd the polynomil is exct for ll polynomils of degree t most, then it follows tht the second term bove is equl to zero, nd we need only show tht I(x 3 ) S (x 3 ) 0. i.e we show tht S (x 3 ) 1 4 (b4 4 ). By direct computtion,we hve S (x 3 ) b (f() + 4(f(c)) + f(b) 6 b 6 (3 + 4c 3 + b 3 ) b 6 (3 + (b + )3 + b 3 ) b 1 (33 + 3b( + b) + 3b 3 ) b 4 (3 + b( + b) + b 3 ) 1 4 (b4 4 ) I(x 3 ). x 3 dx We now use the fct tht Simpson s Rule is exct for ll polynomils of degree t most 3 to show tht for th bsic composite rule, we hve I(f) S (f) 1 ( ) 5 b f (4) (ξ) (41) 90 for some ξ [, b]. To see this, for ɛ > 0, let x 0, x 1 c ɛ, x c + ɛ, x 3 b, nd let p 3,ɛ (x) be the polynomil tht interpoltes f t these points. By the interpoltion error, we know tht f(x) p 3,ɛ (x) + R n,ɛ (x) where R n,ɛ (x) 1 4! (x x 0)(x x 1 )(x x )(x x 3 )f 4 (ξ x ). Thus, I(f) S (f) p 3,ɛ (x)dx + R n,ɛ (x)dx S (p 3,ɛ (x)) S (R n,ɛ (x)) R n,ɛ (x)dx S (R n,ɛ (x)). (4) 33

1 The Lagrange interpolation formula

1 The Lagrange interpolation formula Notes on Qudrture 1 The Lgrnge interpoltion formul We briefly recll the Lgrnge interpoltion formul. The strting point is collection of N + 1 rel points (x 0, y 0 ), (x 1, y 1 ),..., (x N, y N ), with x

More information

Orthogonal Polynomials

Orthogonal Polynomials Mth 4401 Gussin Qudrture Pge 1 Orthogonl Polynomils Orthogonl polynomils rise from series solutions to differentil equtions, lthough they cn be rrived t in vriety of different mnners. Orthogonl polynomils

More information

Numerical Integration

Numerical Integration Chpter 5 Numericl Integrtion Numericl integrtion is the study of how the numericl vlue of n integrl cn be found. Methods of function pproximtion discussed in Chpter??, i.e., function pproximtion vi the

More information

CMDA 4604: Intermediate Topics in Mathematical Modeling Lecture 19: Interpolation and Quadrature

CMDA 4604: Intermediate Topics in Mathematical Modeling Lecture 19: Interpolation and Quadrature CMDA 4604: Intermedite Topics in Mthemticl Modeling Lecture 19: Interpoltion nd Qudrture In this lecture we mke brief diversion into the res of interpoltion nd qudrture. Given function f C[, b], we sy

More information

Discrete Least-squares Approximations

Discrete Least-squares Approximations Discrete Lest-squres Approximtions Given set of dt points (x, y ), (x, y ),, (x m, y m ), norml nd useful prctice in mny pplictions in sttistics, engineering nd other pplied sciences is to construct curve

More information

Orthogonal Polynomials and Least-Squares Approximations to Functions

Orthogonal Polynomials and Least-Squares Approximations to Functions Chpter Orthogonl Polynomils nd Lest-Squres Approximtions to Functions **4/5/3 ET. Discrete Lest-Squres Approximtions Given set of dt points (x,y ), (x,y ),..., (x m,y m ), norml nd useful prctice in mny

More information

The Regulated and Riemann Integrals

The Regulated and Riemann Integrals Chpter 1 The Regulted nd Riemnn Integrls 1.1 Introduction We will consider severl different pproches to defining the definite integrl f(x) dx of function f(x). These definitions will ll ssign the sme vlue

More information

Theoretical foundations of Gaussian quadrature

Theoretical foundations of Gaussian quadrature Theoreticl foundtions of Gussin qudrture 1 Inner product vector spce Definition 1. A vector spce (or liner spce) is set V = {u, v, w,...} in which the following two opertions re defined: (A) Addition of

More information

Review of Calculus, cont d

Review of Calculus, cont d Jim Lmbers MAT 460 Fll Semester 2009-10 Lecture 3 Notes These notes correspond to Section 1.1 in the text. Review of Clculus, cont d Riemnn Sums nd the Definite Integrl There re mny cses in which some

More information

Lecture 6: Singular Integrals, Open Quadrature rules, and Gauss Quadrature

Lecture 6: Singular Integrals, Open Quadrature rules, and Gauss Quadrature Lecture notes on Vritionl nd Approximte Methods in Applied Mthemtics - A Peirce UBC Lecture 6: Singulr Integrls, Open Qudrture rules, nd Guss Qudrture (Compiled 6 August 7) In this lecture we discuss the

More information

Numerical Methods I Orthogonal Polynomials

Numerical Methods I Orthogonal Polynomials Numericl Methods I Orthogonl Polynomils Aleksndr Donev Cournt Institute, NYU 1 donev@cournt.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fll 2014 Nov 6th, 2014 A. Donev (Cournt Institute) Lecture IX

More information

NUMERICAL INTEGRATION. The inverse process to differentiation in calculus is integration. Mathematically, integration is represented by.

NUMERICAL INTEGRATION. The inverse process to differentiation in calculus is integration. Mathematically, integration is represented by. NUMERICAL INTEGRATION 1 Introduction The inverse process to differentition in clculus is integrtion. Mthemticlly, integrtion is represented by f(x) dx which stnds for the integrl of the function f(x) with

More information

Best Approximation. Chapter The General Case

Best Approximation. Chapter The General Case Chpter 4 Best Approximtion 4.1 The Generl Cse In the previous chpter, we hve seen how n interpolting polynomil cn be used s n pproximtion to given function. We now wnt to find the best pproximtion to given

More information

Lecture 14: Quadrature

Lecture 14: Quadrature Lecture 14: Qudrture This lecture is concerned with the evlution of integrls fx)dx 1) over finite intervl [, b] The integrnd fx) is ssumed to be rel-vlues nd smooth The pproximtion of n integrl by numericl

More information

1. Gauss-Jacobi quadrature and Legendre polynomials. p(t)w(t)dt, p {p(x 0 ),...p(x n )} p(t)w(t)dt = w k p(x k ),

1. Gauss-Jacobi quadrature and Legendre polynomials. p(t)w(t)dt, p {p(x 0 ),...p(x n )} p(t)w(t)dt = w k p(x k ), 1. Guss-Jcobi qudrture nd Legendre polynomils Simpson s rule for evluting n integrl f(t)dt gives the correct nswer with error of bout O(n 4 ) (with constnt tht depends on f, in prticulr, it depends on

More information

Advanced Computational Fluid Dynamics AA215A Lecture 3 Polynomial Interpolation: Numerical Differentiation and Integration.

Advanced Computational Fluid Dynamics AA215A Lecture 3 Polynomial Interpolation: Numerical Differentiation and Integration. Advnced Computtionl Fluid Dynmics AA215A Lecture 3 Polynomil Interpoltion: Numericl Differentition nd Integrtion Antony Jmeson Winter Qurter, 2016, Stnford, CA Lst revised on Jnury 7, 2016 Contents 3 Polynomil

More information

1 The Riemann Integral

1 The Riemann Integral The Riemnn Integrl. An exmple leding to the notion of integrl (res) We know how to find (i.e. define) the re of rectngle (bse height), tringle ( (sum of res of tringles). But how do we find/define n re

More information

CAAM 453 NUMERICAL ANALYSIS I Examination There are four questions, plus a bonus. Do not look at them until you begin the exam.

CAAM 453 NUMERICAL ANALYSIS I Examination There are four questions, plus a bonus. Do not look at them until you begin the exam. Exmintion 1 Posted 23 October 2002. Due no lter thn 5pm on Mondy, 28 October 2002. Instructions: 1. Time limit: 3 uninterrupted hours. 2. There re four questions, plus bonus. Do not look t them until you

More information

Numerical Analysis: Trapezoidal and Simpson s Rule

Numerical Analysis: Trapezoidal and Simpson s Rule nd Simpson s Mthemticl question we re interested in numericlly nswering How to we evlute I = f (x) dx? Clculus tells us tht if F(x) is the ntiderivtive of function f (x) on the intervl [, b], then I =

More information

Math 360: A primitive integral and elementary functions

Math 360: A primitive integral and elementary functions Mth 360: A primitive integrl nd elementry functions D. DeTurck University of Pennsylvni October 16, 2017 D. DeTurck Mth 360 001 2017C: Integrl/functions 1 / 32 Setup for the integrl prtitions Definition:

More information

Review of basic calculus

Review of basic calculus Review of bsic clculus This brief review reclls some of the most importnt concepts, definitions, nd theorems from bsic clculus. It is not intended to tech bsic clculus from scrtch. If ny of the items below

More information

Chapter 3 Polynomials

Chapter 3 Polynomials Dr M DRAIEF As described in the introduction of Chpter 1, pplictions of solving liner equtions rise in number of different settings In prticulr, we will in this chpter focus on the problem of modelling

More information

Part IB Numerical Analysis

Part IB Numerical Analysis Prt IB Numericl Anlysis Theorems with proof Bsed on lectures by G. Moore Notes tken by Dexter Chu Lent 2016 These notes re not endorsed by the lecturers, nd I hve modified them (often significntly) fter

More information

Abstract inner product spaces

Abstract inner product spaces WEEK 4 Abstrct inner product spces Definition An inner product spce is vector spce V over the rel field R equipped with rule for multiplying vectors, such tht the product of two vectors is sclr, nd the

More information

Math 270A: Numerical Linear Algebra

Math 270A: Numerical Linear Algebra Mth 70A: Numericl Liner Algebr Instructor: Michel Holst Fll Qurter 014 Homework Assignment #3 Due Give to TA t lest few dys before finl if you wnt feedbck. Exercise 3.1. (The Bsic Liner Method for Liner

More information

Math 1B, lecture 4: Error bounds for numerical methods

Math 1B, lecture 4: Error bounds for numerical methods Mth B, lecture 4: Error bounds for numericl methods Nthn Pflueger 4 September 0 Introduction The five numericl methods descried in the previous lecture ll operte by the sme principle: they pproximte the

More information

Lecture Note 4: Numerical differentiation and integration. Xiaoqun Zhang Shanghai Jiao Tong University

Lecture Note 4: Numerical differentiation and integration. Xiaoqun Zhang Shanghai Jiao Tong University Lecture Note 4: Numericl differentition nd integrtion Xioqun Zng Sngi Jio Tong University Lst updted: November, 0 Numericl Anlysis. Numericl differentition.. Introduction Find n pproximtion of f (x 0 ),

More information

f(x)dx . Show that there 1, 0 < x 1 does not exist a differentiable function g : [ 1, 1] R such that g (x) = f(x) for all

f(x)dx . Show that there 1, 0 < x 1 does not exist a differentiable function g : [ 1, 1] R such that g (x) = f(x) for all 3 Definite Integrl 3.1 Introduction In school one comes cross the definition of the integrl of rel vlued function defined on closed nd bounded intervl [, b] between the limits nd b, i.e., f(x)dx s the

More information

III. Lecture on Numerical Integration. File faclib/dattab/lecture-notes/numerical-inter03.tex /by EC, 3/14/2008 at 15:11, version 9

III. Lecture on Numerical Integration. File faclib/dattab/lecture-notes/numerical-inter03.tex /by EC, 3/14/2008 at 15:11, version 9 III Lecture on Numericl Integrtion File fclib/dttb/lecture-notes/numerical-inter03.tex /by EC, 3/14/008 t 15:11, version 9 1 Sttement of the Numericl Integrtion Problem In this lecture we consider the

More information

Best Approximation in the 2-norm

Best Approximation in the 2-norm Jim Lmbers MAT 77 Fll Semester 1-11 Lecture 1 Notes These notes correspond to Sections 9. nd 9.3 in the text. Best Approximtion in the -norm Suppose tht we wish to obtin function f n (x) tht is liner combintion

More information

Euler, Ioachimescu and the trapezium rule. G.J.O. Jameson (Math. Gazette 96 (2012), )

Euler, Ioachimescu and the trapezium rule. G.J.O. Jameson (Math. Gazette 96 (2012), ) Euler, Iochimescu nd the trpezium rule G.J.O. Jmeson (Mth. Gzette 96 (0), 36 4) The following results were estblished in recent Gzette rticle [, Theorems, 3, 4]. Given > 0 nd 0 < s

More information

Lecture 20: Numerical Integration III

Lecture 20: Numerical Integration III cs4: introduction to numericl nlysis /8/0 Lecture 0: Numericl Integrtion III Instructor: Professor Amos Ron Scribes: Mrk Cowlishw, Yunpeng Li, Nthnel Fillmore For the lst few lectures we hve discussed

More information

Lecture 17. Integration: Gauss Quadrature. David Semeraro. University of Illinois at Urbana-Champaign. March 20, 2014

Lecture 17. Integration: Gauss Quadrature. David Semeraro. University of Illinois at Urbana-Champaign. March 20, 2014 Lecture 17 Integrtion: Guss Qudrture Dvid Semerro University of Illinois t Urbn-Chmpign Mrch 0, 014 Dvid Semerro (NCSA) CS 57 Mrch 0, 014 1 / 9 Tody: Objectives identify the most widely used qudrture method

More information

Chapter 3 Solving Nonlinear Equations

Chapter 3 Solving Nonlinear Equations Chpter 3 Solving Nonliner Equtions 3.1 Introduction The nonliner function of unknown vrible x is in the form of where n could be non-integer. Root is the numericl vlue of x tht stisfies f ( x) 0. Grphiclly,

More information

Main topics for the First Midterm

Main topics for the First Midterm Min topics for the First Midterm The Midterm will cover Section 1.8, Chpters 2-3, Sections 4.1-4.8, nd Sections 5.1-5.3 (essentilly ll of the mteril covered in clss). Be sure to know the results of the

More information

Numerical Analysis. Doron Levy. Department of Mathematics Stanford University

Numerical Analysis. Doron Levy. Department of Mathematics Stanford University Numericl Anlysis Doron Levy Deprtment of Mthemtics Stnford University December 1, 2005 D. Levy Prefce i D. Levy CONTENTS Contents Prefce i 1 Introduction 1 2 Interpoltion 2 2.1 Wht is Interpoltion?............................

More information

MATH34032: Green s Functions, Integral Equations and the Calculus of Variations 1

MATH34032: Green s Functions, Integral Equations and the Calculus of Variations 1 MATH34032: Green s Functions, Integrl Equtions nd the Clculus of Vritions 1 Section 1 Function spces nd opertors Here we gives some brief detils nd definitions, prticulrly relting to opertors. For further

More information

THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS.

THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS. THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS RADON ROSBOROUGH https://intuitiveexplntionscom/picrd-lindelof-theorem/ This document is proof of the existence-uniqueness theorem

More information

Math& 152 Section Integration by Parts

Math& 152 Section Integration by Parts Mth& 5 Section 7. - Integrtion by Prts Integrtion by prts is rule tht trnsforms the integrl of the product of two functions into other (idelly simpler) integrls. Recll from Clculus I tht given two differentible

More information

Z b. f(x)dx. Yet in the above two cases we know what f(x) is. Sometimes, engineers want to calculate an area by computing I, but...

Z b. f(x)dx. Yet in the above two cases we know what f(x) is. Sometimes, engineers want to calculate an area by computing I, but... Chpter 7 Numericl Methods 7. Introduction In mny cses the integrl f(x)dx cn be found by finding function F (x) such tht F 0 (x) =f(x), nd using f(x)dx = F (b) F () which is known s the nlyticl (exct) solution.

More information

Numerical Analysis. 10th ed. R L Burden, J D Faires, and A M Burden

Numerical Analysis. 10th ed. R L Burden, J D Faires, and A M Burden Numericl Anlysis 10th ed R L Burden, J D Fires, nd A M Burden Bemer Presenttion Slides Prepred by Dr. Annette M. Burden Youngstown Stte University July 9, 2015 Chpter 4.1: Numericl Differentition 1 Three-Point

More information

Convex Sets and Functions

Convex Sets and Functions B Convex Sets nd Functions Definition B1 Let L, +, ) be rel liner spce nd let C be subset of L The set C is convex if, for ll x,y C nd ll [, 1], we hve 1 )x+y C In other words, every point on the line

More information

Numerical Integration. 1 Introduction. 2 Midpoint Rule, Trapezoid Rule, Simpson Rule. AMSC/CMSC 460/466 T. von Petersdorff 1

Numerical Integration. 1 Introduction. 2 Midpoint Rule, Trapezoid Rule, Simpson Rule. AMSC/CMSC 460/466 T. von Petersdorff 1 AMSC/CMSC 46/466 T. von Petersdorff 1 umericl Integrtion 1 Introduction We wnt to pproximte the integrl I := f xdx where we re given, b nd the function f s subroutine. We evlute f t points x 1,...,x n

More information

Matrices, Moments and Quadrature, cont d

Matrices, Moments and Quadrature, cont d Jim Lmbers MAT 285 Summer Session 2015-16 Lecture 2 Notes Mtrices, Moments nd Qudrture, cont d We hve described how Jcobi mtrices cn be used to compute nodes nd weights for Gussin qudrture rules for generl

More information

Lecture 1. Functional series. Pointwise and uniform convergence.

Lecture 1. Functional series. Pointwise and uniform convergence. 1 Introduction. Lecture 1. Functionl series. Pointwise nd uniform convergence. In this course we study mongst other things Fourier series. The Fourier series for periodic function f(x) with period 2π is

More information

3.4 Numerical integration

3.4 Numerical integration 3.4. Numericl integrtion 63 3.4 Numericl integrtion In mny economic pplictions it is necessry to compute the definite integrl of relvlued function f with respect to "weight" function w over n intervl [,

More information

Lecture 19: Continuous Least Squares Approximation

Lecture 19: Continuous Least Squares Approximation Lecture 19: Continuous Lest Squres Approximtion 33 Continuous lest squres pproximtion We begn 31 with the problem of pproximting some f C[, b] with polynomil p P n t the discrete points x, x 1,, x m for

More information

Numerical quadrature based on interpolating functions: A MATLAB implementation

Numerical quadrature based on interpolating functions: A MATLAB implementation SEMINAR REPORT Numericl qudrture bsed on interpolting functions: A MATLAB implementtion by Venkt Ayylsomyjul A seminr report submitted in prtil fulfillment for the degree of Mster of Science (M.Sc) in

More information

Math 554 Integration

Math 554 Integration Mth 554 Integrtion Hndout #9 4/12/96 Defn. A collection of n + 1 distinct points of the intervl [, b] P := {x 0 = < x 1 < < x i 1 < x i < < b =: x n } is clled prtition of the intervl. In this cse, we

More information

The Riemann Integral

The Riemann Integral Deprtment of Mthemtics King Sud University 2017-2018 Tble of contents 1 Anti-derivtive Function nd Indefinite Integrls 2 3 4 5 Indefinite Integrls & Anti-derivtive Function Definition Let f : I R be function

More information

Exam 2, Mathematics 4701, Section ETY6 6:05 pm 7:40 pm, March 31, 2016, IH-1105 Instructor: Attila Máté 1

Exam 2, Mathematics 4701, Section ETY6 6:05 pm 7:40 pm, March 31, 2016, IH-1105 Instructor: Attila Máté 1 Exm, Mthemtics 471, Section ETY6 6:5 pm 7:4 pm, Mrch 1, 16, IH-115 Instructor: Attil Máté 1 17 copies 1. ) Stte the usul sufficient condition for the fixed-point itertion to converge when solving the eqution

More information

Chapter 1. Basic Concepts

Chapter 1. Basic Concepts Socrtes Dilecticl Process: The Þrst step is the seprtion of subject into its elements. After this, by deþning nd discovering more bout its prts, one better comprehends the entire subject Socrtes (469-399)

More information

Advanced Calculus: MATH 410 Notes on Integrals and Integrability Professor David Levermore 17 October 2004

Advanced Calculus: MATH 410 Notes on Integrals and Integrability Professor David Levermore 17 October 2004 Advnced Clculus: MATH 410 Notes on Integrls nd Integrbility Professor Dvid Levermore 17 October 2004 1. Definite Integrls In this section we revisit the definite integrl tht you were introduced to when

More information

COSC 3361 Numerical Analysis I Numerical Integration and Differentiation (III) - Gauss Quadrature and Adaptive Quadrature

COSC 3361 Numerical Analysis I Numerical Integration and Differentiation (III) - Gauss Quadrature and Adaptive Quadrature COSC 336 Numericl Anlysis I Numericl Integrtion nd Dierentition III - Guss Qudrture nd Adptive Qudrture Edgr Griel Fll 5 COSC 336 Numericl Anlysis I Edgr Griel Summry o the lst lecture I For pproximting

More information

Advanced Calculus: MATH 410 Uniform Convergence of Functions Professor David Levermore 11 December 2015

Advanced Calculus: MATH 410 Uniform Convergence of Functions Professor David Levermore 11 December 2015 Advnced Clculus: MATH 410 Uniform Convergence of Functions Professor Dvid Levermore 11 December 2015 12. Sequences of Functions We now explore two notions of wht it mens for sequence of functions {f n

More information

Lecture 4: Piecewise Cubic Interpolation

Lecture 4: Piecewise Cubic Interpolation Lecture notes on Vritionl nd Approximte Methods in Applied Mthemtics - A Peirce UBC Lecture 4: Piecewise Cubic Interpoltion Compiled 5 September In this lecture we consider piecewise cubic interpoltion

More information

Introduction to Numerical Analysis

Introduction to Numerical Analysis Introduction to Numericl Anlysis Doron Levy Deprtment of Mthemtics nd Center for Scientific Computtion nd Mthemticl Modeling (CSCAMM) University of Mrylnd June 14, 2012 D. Levy CONTENTS Contents 1 Introduction

More information

MORE FUNCTION GRAPHING; OPTIMIZATION. (Last edited October 28, 2013 at 11:09pm.)

MORE FUNCTION GRAPHING; OPTIMIZATION. (Last edited October 28, 2013 at 11:09pm.) MORE FUNCTION GRAPHING; OPTIMIZATION FRI, OCT 25, 203 (Lst edited October 28, 203 t :09pm.) Exercise. Let n be n rbitrry positive integer. Give n exmple of function with exctly n verticl symptotes. Give

More information

Lecture 14 Numerical integration: advanced topics

Lecture 14 Numerical integration: advanced topics Lecture 14 Numericl integrtion: dvnced topics Weinn E 1,2 nd Tiejun Li 2 1 Deprtment of Mthemtics, Princeton University, weinn@princeton.edu 2 School of Mthemticl Sciences, Peking University, tieli@pku.edu.cn

More information

ODE: Existence and Uniqueness of a Solution

ODE: Existence and Uniqueness of a Solution Mth 22 Fll 213 Jerry Kzdn ODE: Existence nd Uniqueness of Solution The Fundmentl Theorem of Clculus tells us how to solve the ordinry differentil eqution (ODE) du = f(t) dt with initil condition u() =

More information

Line Integrals. Partitioning the Curve. Estimating the Mass

Line Integrals. Partitioning the Curve. Estimating the Mass Line Integrls Suppose we hve curve in the xy plne nd ssocite density δ(p ) = δ(x, y) t ech point on the curve. urves, of course, do not hve density or mss, but it my sometimes be convenient or useful to

More information

Construction of Gauss Quadrature Rules

Construction of Gauss Quadrature Rules Jim Lmbers MAT 772 Fll Semester 2010-11 Lecture 15 Notes These notes correspond to Sections 10.2 nd 10.3 in the text. Construction of Guss Qudrture Rules Previously, we lerned tht Newton-Cotes qudrture

More information

Anti-derivatives/Indefinite Integrals of Basic Functions

Anti-derivatives/Indefinite Integrals of Basic Functions Anti-derivtives/Indefinite Integrls of Bsic Functions Power Rule: In prticulr, this mens tht x n+ x n n + + C, dx = ln x + C, if n if n = x 0 dx = dx = dx = x + C nd x (lthough you won t use the second

More information

NUMERICAL INTEGRATION

NUMERICAL INTEGRATION NUMERICAL INTEGRATION How do we evlute I = f (x) dx By the fundmentl theorem of clculus, if F (x) is n ntiderivtive of f (x), then I = f (x) dx = F (x) b = F (b) F () However, in prctice most integrls

More information

Physics 116C Solution of inhomogeneous ordinary differential equations using Green s functions

Physics 116C Solution of inhomogeneous ordinary differential equations using Green s functions Physics 6C Solution of inhomogeneous ordinry differentil equtions using Green s functions Peter Young November 5, 29 Homogeneous Equtions We hve studied, especilly in long HW problem, second order liner

More information

Calculus I-II Review Sheet

Calculus I-II Review Sheet Clculus I-II Review Sheet 1 Definitions 1.1 Functions A function is f is incresing on n intervl if x y implies f(x) f(y), nd decresing if x y implies f(x) f(y). It is clled monotonic if it is either incresing

More information

A REVIEW OF CALCULUS CONCEPTS FOR JDEP 384H. Thomas Shores Department of Mathematics University of Nebraska Spring 2007

A REVIEW OF CALCULUS CONCEPTS FOR JDEP 384H. Thomas Shores Department of Mathematics University of Nebraska Spring 2007 A REVIEW OF CALCULUS CONCEPTS FOR JDEP 384H Thoms Shores Deprtment of Mthemtics University of Nebrsk Spring 2007 Contents Rtes of Chnge nd Derivtives 1 Dierentils 4 Are nd Integrls 5 Multivrite Clculus

More information

Continuous Random Variables

Continuous Random Variables STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 217 Néhémy Lim Continuous Rndom Vribles Nottion. The indictor function of set S is rel-vlued function defined by : { 1 if x S 1 S (x) if x S Suppose tht

More information

B.Sc. in Mathematics (Ordinary)

B.Sc. in Mathematics (Ordinary) R48/0 DUBLIN INSTITUTE OF TECHNOLOGY KEVIN STREET, DUBLIN 8 B.Sc. in Mthemtics (Ordinry) SUPPLEMENTAL EXAMINATIONS 01 Numericl Methods Dr. D. Mckey Dr. C. Hills Dr. E.A. Cox Full mrks for complete nswers

More information

1 Linear Least Squares

1 Linear Least Squares Lest Squres Pge 1 1 Liner Lest Squres I will try to be consistent in nottion, with n being the number of dt points, nd m < n being the number of prmeters in model function. We re interested in solving

More information

Definite integral. Mathematics FRDIS MENDELU. Simona Fišnarová (Mendel University) Definite integral MENDELU 1 / 30

Definite integral. Mathematics FRDIS MENDELU. Simona Fišnarová (Mendel University) Definite integral MENDELU 1 / 30 Definite integrl Mthemtics FRDIS MENDELU Simon Fišnrová (Mendel University) Definite integrl MENDELU / Motivtion - re under curve Suppose, for simplicity, tht y = f(x) is nonnegtive nd continuous function

More information

1 Error Analysis of Simple Rules for Numerical Integration

1 Error Analysis of Simple Rules for Numerical Integration cs41: introduction to numericl nlysis 11/16/10 Lecture 19: Numericl Integrtion II Instructor: Professor Amos Ron Scries: Mrk Cowlishw, Nthnel Fillmore 1 Error Anlysis of Simple Rules for Numericl Integrtion

More information

Definite integral. Mathematics FRDIS MENDELU

Definite integral. Mathematics FRDIS MENDELU Definite integrl Mthemtics FRDIS MENDELU Simon Fišnrová Brno 1 Motivtion - re under curve Suppose, for simplicity, tht y = f(x) is nonnegtive nd continuous function defined on [, b]. Wht is the re of the

More information

Chapters 4 & 5 Integrals & Applications

Chapters 4 & 5 Integrals & Applications Contents Chpters 4 & 5 Integrls & Applictions Motivtion to Chpters 4 & 5 2 Chpter 4 3 Ares nd Distnces 3. VIDEO - Ares Under Functions............................................ 3.2 VIDEO - Applictions

More information

Definition of Continuity: The function f(x) is continuous at x = a if f(a) exists and lim

Definition of Continuity: The function f(x) is continuous at x = a if f(a) exists and lim Mth 9 Course Summry/Study Guide Fll, 2005 [1] Limits Definition of Limit: We sy tht L is the limit of f(x) s x pproches if f(x) gets closer nd closer to L s x gets closer nd closer to. We write lim f(x)

More information

STURM-LIOUVILLE BOUNDARY VALUE PROBLEMS

STURM-LIOUVILLE BOUNDARY VALUE PROBLEMS STURM-LIOUVILLE BOUNDARY VALUE PROBLEMS Throughout, we let [, b] be bounded intervl in R. C 2 ([, b]) denotes the spce of functions with derivtives of second order continuous up to the endpoints. Cc 2

More information

Riemann Sums and Riemann Integrals

Riemann Sums and Riemann Integrals Riemnn Sums nd Riemnn Integrls Jmes K. Peterson Deprtment of Biologicl Sciences nd Deprtment of Mthemticl Sciences Clemson University August 26, 2013 Outline 1 Riemnn Sums 2 Riemnn Integrls 3 Properties

More information

APPROXIMATE INTEGRATION

APPROXIMATE INTEGRATION APPROXIMATE INTEGRATION. Introduction We hve seen tht there re functions whose nti-derivtives cnnot be expressed in closed form. For these resons ny definite integrl involving these integrnds cnnot be

More information

Lecture 12: Numerical Quadrature

Lecture 12: Numerical Quadrature Lecture 12: Numericl Qudrture J.K. Ryn@tudelft.nl WI3097TU Delft Institute of Applied Mthemtics Delft University of Technology 5 December 2012 () Numericl Qudrture 5 December 2012 1 / 46 Outline 1 Review

More information

1 Ordinary Differential Equations

1 Ordinary Differential Equations 1 Ordinry Differentil Equtions 1. Mthemticl Bckground 1..1 Smoothness Definition 1.1 A function f defined on [, b] is continuous t x [, b] if lim x x f(x) = f(x ). Remrk Note tht this implies existence

More information

The Fundamental Theorem of Calculus. The Total Change Theorem and the Area Under a Curve.

The Fundamental Theorem of Calculus. The Total Change Theorem and the Area Under a Curve. Clculus Li Vs The Fundmentl Theorem of Clculus. The Totl Chnge Theorem nd the Are Under Curve. Recll the following fct from Clculus course. If continuous function f(x) represents the rte of chnge of F

More information

Overview of Calculus I

Overview of Calculus I Overview of Clculus I Prof. Jim Swift Northern Arizon University There re three key concepts in clculus: The limit, the derivtive, nd the integrl. You need to understnd the definitions of these three things,

More information

Riemann Sums and Riemann Integrals

Riemann Sums and Riemann Integrals Riemnn Sums nd Riemnn Integrls Jmes K. Peterson Deprtment of Biologicl Sciences nd Deprtment of Mthemticl Sciences Clemson University August 26, 203 Outline Riemnn Sums Riemnn Integrls Properties Abstrct

More information

20 MATHEMATICS POLYNOMIALS

20 MATHEMATICS POLYNOMIALS 0 MATHEMATICS POLYNOMIALS.1 Introduction In Clss IX, you hve studied polynomils in one vrible nd their degrees. Recll tht if p(x) is polynomil in x, the highest power of x in p(x) is clled the degree of

More information

Numerical Integration. Newton Cotes Formulas. Quadrature. Newton Cotes Formulas. To approximate the integral b

Numerical Integration. Newton Cotes Formulas. Quadrature. Newton Cotes Formulas. To approximate the integral b Numericl Integrtion Newton Cotes Formuls Given function f : R R nd two rel numbers, b R, < b, we clculte (pproximtely) the integrl I(f,, b) = f (x) dx K. Frischmuth (IfM UR) Numerics for CSE 08/09 8 /

More information

MAT 772: Numerical Analysis. James V. Lambers

MAT 772: Numerical Analysis. James V. Lambers MAT 772: Numericl Anlysis Jmes V. Lmbers August 23, 2016 2 Contents 1 Solution of Equtions by Itertion 7 1.1 Nonliner Equtions....................... 7 1.1.1 Existence nd Uniqueness................ 7 1.1.2

More information

Integration Techniques

Integration Techniques Integrtion Techniques. Integrtion of Trigonometric Functions Exmple. Evlute cos x. Recll tht cos x = cos x. Hence, cos x Exmple. Evlute = ( + cos x) = (x + sin x) + C = x + 4 sin x + C. cos 3 x. Let u

More information

g i fφdx dx = x i i=1 is a Hilbert space. We shall, henceforth, abuse notation and write g i f(x) = f

g i fφdx dx = x i i=1 is a Hilbert space. We shall, henceforth, abuse notation and write g i f(x) = f 1. Appliction of functionl nlysis to PEs 1.1. Introduction. In this section we give little introduction to prtil differentil equtions. In prticulr we consider the problem u(x) = f(x) x, u(x) = x (1) where

More information

W. We shall do so one by one, starting with I 1, and we shall do it greedily, trying

W. We shall do so one by one, starting with I 1, and we shall do it greedily, trying Vitli covers 1 Definition. A Vitli cover of set E R is set V of closed intervls with positive length so tht, for every δ > 0 nd every x E, there is some I V with λ(i ) < δ nd x I. 2 Lemm (Vitli covering)

More information

1.9 C 2 inner variations

1.9 C 2 inner variations 46 CHAPTER 1. INDIRECT METHODS 1.9 C 2 inner vritions So fr, we hve restricted ttention to liner vritions. These re vritions of the form vx; ǫ = ux + ǫφx where φ is in some liner perturbtion clss P, for

More information

INTRODUCTION TO INTEGRATION

INTRODUCTION TO INTEGRATION INTRODUCTION TO INTEGRATION 5.1 Ares nd Distnces Assume f(x) 0 on the intervl [, b]. Let A be the re under the grph of f(x). b We will obtin n pproximtion of A in the following three steps. STEP 1: Divide

More information

Math 8 Winter 2015 Applications of Integration

Math 8 Winter 2015 Applications of Integration Mth 8 Winter 205 Applictions of Integrtion Here re few importnt pplictions of integrtion. The pplictions you my see on n exm in this course include only the Net Chnge Theorem (which is relly just the Fundmentl

More information

P 3 (x) = f(0) + f (0)x + f (0) 2. x 2 + f (0) . In the problem set, you are asked to show, in general, the n th order term is a n = f (n) (0)

P 3 (x) = f(0) + f (0)x + f (0) 2. x 2 + f (0) . In the problem set, you are asked to show, in general, the n th order term is a n = f (n) (0) 1 Tylor polynomils In Section 3.5, we discussed how to pproximte function f(x) round point in terms of its first derivtive f (x) evluted t, tht is using the liner pproximtion f() + f ()(x ). We clled this

More information

Unit #9 : Definite Integral Properties; Fundamental Theorem of Calculus

Unit #9 : Definite Integral Properties; Fundamental Theorem of Calculus Unit #9 : Definite Integrl Properties; Fundmentl Theorem of Clculus Gols: Identify properties of definite integrls Define odd nd even functions, nd reltionship to integrl vlues Introduce the Fundmentl

More information

How can we approximate the area of a region in the plane? What is an interpretation of the area under the graph of a velocity function?

How can we approximate the area of a region in the plane? What is an interpretation of the area under the graph of a velocity function? Mth 125 Summry Here re some thoughts I ws hving while considering wht to put on the first midterm. The core of your studying should be the ssigned homework problems: mke sure you relly understnd those

More information

The final exam will take place on Friday May 11th from 8am 11am in Evans room 60.

The final exam will take place on Friday May 11th from 8am 11am in Evans room 60. Mth 104: finl informtion The finl exm will tke plce on Fridy My 11th from 8m 11m in Evns room 60. The exm will cover ll prts of the course with equl weighting. It will cover Chpters 1 5, 7 15, 17 21, 23

More information

1. On some properties of definite integrals. We prove

1. On some properties of definite integrals. We prove This short collection of notes is intended to complement the textbook Anlisi Mtemtic 2 by Crl Mdern, published by Città Studi Editore, [M]. We refer to [M] for nottion nd the logicl stremline of the rguments.

More information

II. Integration and Cauchy s Theorem

II. Integration and Cauchy s Theorem MTH6111 Complex Anlysis 2009-10 Lecture Notes c Shun Bullett QMUL 2009 II. Integrtion nd Cuchy s Theorem 1. Pths nd integrtion Wrning Different uthors hve different definitions for terms like pth nd curve.

More information

Math Advanced Calculus II

Math Advanced Calculus II Mth 452 - Advnced Clculus II Line Integrls nd Green s Theorem The min gol of this chpter is to prove Stoke s theorem, which is the multivrible version of the fundmentl theorem of clculus. We will be focused

More information

6.5 Numerical Approximations of Definite Integrals

6.5 Numerical Approximations of Definite Integrals Arknss Tech University MATH 94: Clculus II Dr. Mrcel B. Finn 6.5 Numericl Approximtions of Definite Integrls Sometimes the integrl of function cnnot be expressed with elementry functions, i.e., polynomil,

More information