# Discrete Least-squares Approximations

Size: px
Start display at page:

Transcription

1

2 Discrete Lest-squres Approximtions Given set of dt points (x, y ), (x, y ),, (x m, y m ), norml nd useful prctice in mny pplictions in sttistics, engineering nd other pplied sciences is to construct curve tht is considered to be the fit best for the dt, in some sense Severl types of fits cn be considered But the one tht is used most in pplictions is the lest-squres fit Mthemticlly, the problem is the following: Discrete Lest-Squres Approximtion Problem Given set of dt points (x k, y i ), i =,, m, find n lgebric polynomil P n (x) = + x + + n x n (n < m) such tht the error in the lest-squres m sense in minimized; tht is, E = (y i x i n x n i ) is minimum For E to be minimum, we must hve i= E j =, j =,, n Now, E = E = m (y i x i n x n i ) i= m x i (y i x i n x n i ) i= E n = m x n i (y i x i n x n i ) i= Setting these equtions to be zero we hve + m i= m i= x i + x i + m i= m i= x i + + n x i + + n m i= m i= x n i = x n+ i = m i= y i m x i y i i= m i= x n i + m i= x n+ i + + n m i= x n i = m x n i y i i=

3 Set now m x k i = s k, k =,,, n, nd denoting the right hnd side entries s b,, b n, i= the bove eqution cn be written s: s + s + + s n n = b (Note tht m i= x i = s = m) s + s + + s n+ n = b s n + s n+ + s n n = b n This is system of (n + ) equtions in (n + ) unknowns,,, n These equtions re clled Norml Equtions This system now cn be solved to obtin these (n + ) unknowns, provided solution to the system exists We will not show tht this system hs unique solution if x i s re distinct The system cn be written in the following mtrix form: or where Define s = s s s n s s s n+ s n s n+ s n s s s n s s s n+ s n s n+ s n Then the bove system hs the form: s = b n, = = n x x x n x x x n V = x 3 x 3 x n 3 x m x m x n m V T V = b b b b n, b = The mtrix V is known s the Vndermonde mtrix, nd it cn be shown [Exercise] tht it hs full rnk if x i s re distinct In this cse the mtrix S = V T V is symmetric nd positive definite [Exercise] nd is therefore nonsingulr Thus, if x i s re distinct, the eqution S = b hs unique solution b b b n

4 Theorem (Existence nd uniqueness of Discrete Lest-Squres Solutions) Let (x, y ), (x, y ),, (x n, y n ) be n distinct points Then the discrete lest-squre pproximtion problem hs unique solution Lest-Squres Approximtion of Function We hve described lest-squres pproximtion to fit set of discrete dt Here we describe continuous lest-squre pproximtions of function f(x) by using polynomils The problem cn be stted s follows: Lest-Squre Approximtions of Function Using Stndrd Polynomils Given function f(x), continuous on [, b], find polynomil P n (x) of degree t most n: P n (x) = + x + x + + n x n such tht the integrl of the squre of the error is minimized Tht is, is minimized E = [f(x) P n (x)] dx The polynomil P n (x) is clled the Lest-Squres Polynomil,,, n, we denote this by E(,,, n ) For minimiztion, we must hve E i =, i =,,, n Since E is function of As before, these conditions will give rise to norml system of (n + ) equtions in (n + ) unknowns:,,, n Solution of these equtions will yield the unknowns Setting up the Norml Equtions Since We hve E = [f(x) ( + x + x + + n x n )] dx 3

5 E = E = E n = so, E = [f(x) x x n x n ]dx x[f(x) x x n x n ]dx x n [f(x) x x n x n ]dx Similrly, E = i,, 3,, n dx + x dx + x dx + n x n dx = So, (n + ) norml equtions in this cse re: x i dx+ x i+ dx+ x i+ dx + n x i+n dx = f(x)dx x i dx, i = i = : i = : dx + x dx + x dx + + n x n dx = x dx + x dx + 3 x 3 dx + + n x n dx = f(x) xf(x)dx i = n : Denote x n dx + x n+ dx + x n+ dx + + n x n dx = x n f(x)dx x i dx = s i, i =,,, 3,, n, nd b i = Then the bove (n + ) equtions cn be written s x i f(x)dx, i =,,, n s + s + s + + n s n = b s + s + s n+ s n+ = b s n + s n+ + s n+ + + n s n = b n 4

6 or in mtrix nottion s s s s n s s s 3 s n+ s n s n+ s n n = b b b n Denote S = (s ii ), = n, b = b b b n Then we hve the liner system: S = b The solution of these equtions will yield the coefficients,,, n of the lest-squres polynomil P n (x) A Specil Cse: Let the intervl be [, ] Then s i = x i dx = i +, Thus, in this cse the mtrix of the norml equtions i =,,, n n S = 3 n + n n+ n + n + n which is Hilbert Mtrix It is well-known to be ill-conditioned Algorithm: Lest-Squres Approximtion using Polynomils Inputs: (i) f(x) - A continuous function on [, b] (ii) n - The degree of the desired lest-squre polynomil 5

7 Output: The coefficients,,, n of the desired lest-squres polynomil: P n (x) = + x + + n x n Step Compute s, s,, s n For i =,, n do End s i = x i dx Step Compute b, b,, b n : For i =,,, n do b i = End x i f(x)dx Step 3 Form the mtrix S nd the vector b s s s n s s s n+ S = s n s n+ s n b = b b b n Step 4 Solve the (n + ) (n + ) system of equtions: S = b, where = n Exmple Find Liner nd Qudrtic lest-squres pproximtions to f(x) = e x on [, ] 6

8 Liner Approximtion: n = ; P (x) = + x s = dx = [ ] x s = xdx = = ( ) = [ ] x s = x 3 dx = = ( ) 3 3 = 3 3 ( ) ( ) s s Thus, S = = s s 3 b = b = e x dx = e e 354 e x xdx = e 7358 The norml system of equtions is: ( 3 ) ( ) = ( b b ) This gives = 75, = 37 The liner lest-squres polynomil P (x) = x Check Accurcy: P (5) = 77 e 5 = 6487 Reltive Error = 453 7

9 Qudrtic Fitting: n = P (x) = + x + x s =, s =, s = 3 s 3 = [ ] x 4 x3 dx = = 4 s 4 = [ ] x 5 x4 dx = = 5 5 b = b = b = The system of norml equtions is: e x dx = e e 354 xe x dx = e 7358 x e x dx = e 5 e 8789 = The solution is: = 9963, = 37, = The qudrtic lest-squres polynomil P (x) = x x 8

10 Check the ccurcy: P (5) = 6889 e 5 = 6487 Reltive error P (5) e 5 e 5 = = 4 Exmple Find liner nd Qudrtic lest-squres polynomil pproximtion to f(x) = x + 5x + 6 in [, ] Liner Fit: b = b = The norml equtions re: 3 P (x) = + x s = s = s = dx = xdx = x dx = 3 (x + 5x + 6)dx = = 53 6 x(x + 5x + 6)dx = ( = = 59 ) = (x 3 + 5x + 6x)dx ] = = 6 The liner lest squres polynomil P (x) = x Check Accurcy: f(5) = 875; P (5) =

11 Reltive error: = 95 Qudrtic Lest-Squre Approximtion: P (x) = + x + x b = 53 6, b = 59 b = x (x + 5x + 6)dx = S = (x 4 + 5x 3 + 6x )dx = = 69 The solution of the liner system is: = 6, = 5, = P (x) = 6 + 5x + x (Exct) Use of Orthogonl Polynomils in Lest-squres Approximtions The lest-squres pproximtion using polynomils, s described bove, is not numericlly effective; since the system mtrix S of norml equtions is very often ill-conditioned For exmple, when the intervl is [,], we hve seen tht S is Hilbert mtrix, which is notoriously ill-conditioned for even modest vlues of n When n = 5, the condition number of this mtrix = cond(s) = O( 5 ) Such computtions cn, however, be mde computtionlly effective by using specil type of polynomils, clled orthogonl polynomils Definition The set of functions φ, φ,, φ n is clled set of orthogonl functions, with respect to weight function w(x), if { if i j w(x)φ j (x)φ i (x)dx = if i = j where C j is rel positive number Furthermore, if C j =, j =,,, n, then the orthogonl set is clled n orthonorml set Using this interesting property, lest-squres computtions cn be more numericlly effective, s shown below Without ny loss of generlity, let s ssume tht w(x) = Ide: The ide is to find n pproximtion of f(x) on [, b] by mens of polynomil of the form P n (x) = φ (x) + φ (x) + + n φ n (x), where {φ n } n k= is set of orthogonl polynomils Tht is, the bsis for generting P n(x) in this cse is set of orthogonl polynomils C j

12 Lest-squres Approximtion of Function Using Orthogonl Polynomils Given f(x), continuous on [, b], find,,, n using polynomil of the form: P n (x) = φ (x) + φ (x) + + n φ n (x), where {φ k (x)} n k= is given set of orthogonl polynomils on [, b], such tht the error function: is minimized As before, we set E(,,, n ) = [f(x) ( φ (x) + n φ n (x))] dx Now E i =, i =,,, n E = Setting this equl to zero, we get φ (x)[f(x) φ (x) φ (x) n φ n (x)]dx φ (x)f(x)dx = Since, {φ k (x)} n k= is n orthogonl set, we hve, nd ( φ (x) + + n φ n (x))φ (x)dx φ (x) dx = C, φ (x)φ i (x) dx =, i Applying the bove orthogonl property, we see from bove tht Tht is, φ (x)f(x)dx = C

13 = φ (x)f(x)dx C Similrly, E = φ (x)[f(x) φ (x) φ (x) n φ n (x)]dx The orthogonl property of {φ j (x)} n j= implies tht so, setting E =, we get φ (x) = C nd φ (x)φ i (x) =, i, = φ (x)f(x)dx C In generl, k = φ k (x)f(x)dx, k =,,, n, C k where C k = φ k(x)dx Expresions for k with Weight Function w(x) If the weight function w(x) is included, we obtin k = w(x)f(x)φ k (x)dx, k =,, n C k

14 Algorithm: Lest-Squres Approximtion Using Orthogonl Polynomils Inputs: f(x) - A continuous function on [, b] w(x) - A weight function (n integrble function on [, b]) {φ k (x)} n k= - A set of n orthogonl functions on [, b] Output: The coefficients,,, n such tht is minimized w(x)[f(x) φ (x) φ (x) n φ n (x)] dx Step Compute C k, k =,,, n s follows: For k =,,,, n do End C k = w(x)φ k(x)dx Step Compute k, k =,, n s follows: For k =,,,, n do k = w(x)f(x)φ k (x)dx C k End Lest-Squres Approximtion Using Legendre s Polynomils Recll tht the Legendre Polynomils {φ k (x)} re given by φ (x) = φ (x) = x φ (x) = x 3 φ 3 (x) = x x etc re orthogonl polynomils on [, ], with respect to the weight function w(x) = If these polynomils re used for lest-squres pproximtion, then it is esy to see tht 3

15 C = C = C = φ (x)dx = φ (x)dx = φ (x)dx = dx = x dx = ( 3 x ) dx = nd so on Exmple: Find liner nd qudrtic lest-squres pproximtion to f(x) = e x using Legendre polynomils Liner Approximtion: P (x) = φ (x) + φ (x) φ (x) =, φ (x) = x C = φ (x)dx = = φ (x)e x dx C The liner lest-squres polynomil dx = [x] = So, = e x dx = [ex ] = [ x C = φ (x)dx = x 3 dx = 3 = 3 xe x dx = 3 [ ] = 3 e e ( e ) e ] P (x) = φ (x) + φ (x) = [ e ] + 3 e e x = = 3 Accurcy Check: P (5) = e 5 = 6487 [ e ] + 3 e e 5 = 77 Reltive error: = 475 4

16 Qudrtic Approximtion: P (x) = φ (x) + φ (x) + φ (x) = ( e ), = 3 e e C = φ (x)dx = = ( x x3 3 + x = e x φ (x)dx C = 45 8 (x 3 ) dx ) ( e x x ) dx 3 = 8 45 = e 7 e Qudrtic lest-squres polynomil: P (x) = ( e ) + 3e ( e x + e 7 ) ( x ) e 3 Accurcy check: Reltive error P n (5) = 5868 e 5 = = 375 Compre this reltive error with tht obtined erlier with n non-orthogonl polynomil of degree 5

17 Chebyshev polynomils: Another wonderful fmily of orthogonl polynomils Definition: The set of polynomils defined by T n (x) = cos[n rccos x], n on [, ] re clled the Chebyshev polynomils To see tht T n (x) is polynomil of degree n in our fmilir form, we derive recursive reltion by noting tht T (x) = (A polynomil of degree zero) T (x) = x (A polynomil of degree ) A Recursive Reltion for Generting Chebyshev Polynomils: Substitute θ = rc cos x Then, T n (x) = cos(nθ), θ π T n+ (x) = cos(n + )θ = cos nθ cos θ sin nθ sin θ T n (x) = cos(n )θ = cos nθ cos θ + sin nθ sin θ Adding the lst two equtions, we obtin T n+ (x) + T n (x) = cos nθ cos θ The right hnd side still does not look like polynomil in x But note tht cos θ = x So, or T n+ (x) = cos nθ cos θ T n (x) = x cos(n cos rc x) T n (x) = xt n (x) T n (x) T n+ (x) = xt n (x) T n (x), n Using this recursive reltion, the Chebyshev polynomils of the succesive degrees cn be generted n = : T (x) = xt (x) T (x) = x n = : T 3 (x) = xt (x) T (x) = x(x ) x = 4x 3 3x nd so on 6

18 The orthogonl property of the Chebyshev polynomils We now show tht Chebyshev polynomils re orthogonl with respect to the weight function w(x) =, in the intervl [, ] x To demonstrte the orthogonl property of these polynomils, consider = = T m (x)t n (x)dx, m n x π = = = cos(rccos x) cos(n rccos x) x dx cos mθ cos nθdθ ( By chnging the vrible from x to θ with substitution of rccosx = θ) π [ cos(m + n)θdθ + sin(m + n)θ (m + n) ] π π + cos(m n)θdθ [ sin(m n)θ (m n) ] π Similrly, it cn be shown [Exercise] tht Summrizing: T n(x)dx x = π for n Orthogonl Property of the Chebyshev Polynomils T m (x)t n (x) dx = x if m n π if m = n The Lest-Squre Approximtion using Chebyshev Polynomils 7

19 As before, the Chebyshev polynomils cn be used to find lest-squres pproximtions to function f(x) s stted below The lest-squres pproximting polynomil P n (x) of f(x) using Chebyshev polynomils is given by: P n (x) = C T (x) + C T (x) + + C n T n where nd C i = π f(x)t i (x)dx, i =,, n x C = π f(x)dx x Find liner lest-squres pproximtion of f(x) = e x using Chebyshev poly- Exmple: nomils Here P (x) = φ (x) + φ i (x) = T (x) + T (x) = + x, where Thus, P (x) = x Check the ccurcy: = π = π e x dx x 66 xe x dx 33 x P (5) = 975; e 5 = 6487 Reltive error = 4 Monic Chebyshev Polynomils Note tht T k (x) is Chebyshev polynomil of degree k with the leding coefficient k, k Thus we cn generte set of monic Chebyshev polynomils from the polynomils T k (x) s follows: 8

20 The Monic Chebyshev Polynomils, T k (x), re then given by T (x) =, T k (x) = k T k(x), k The k zeros of T k (x) re esily clculted [Exercise]: ( ) j x j = cos k π, j =,,, k The mximum or minimum vlues of T k (x) occur t x j = cos T k ( x j ) = ()j, j =,,, k k ( ) jπ, nd k Polynomil Approximtions with Chebyshev the polynomils: As seen bove the Chebyshev polynomils cn, of course, be used to find lest-squres polynomil pproximtions However, these polynomils hve severl other wonderful polynomil pproximtion properties Some of them re stted below The mximum bsolute vlue of ny monic polynomil of degree n over [, ] is lwys greter thn or equl to tht of T n (x) over the sme intervl; which is, by the lst property, n Minimx Property of the Chebyshev Polynomils If P n (x) is ny monic polynomil of degree n, then = mx T n n (x) mx P n (x) x [,] x [,] Moreover, this hppens when P n (x) = T n (x) Proof: By contrdiction [Exercise] 9

21 Choosing the interpolting nodes with the Chebyshev Zeros Recll tht error in polynomil interpoltion by polynomil P n (x) of degree t most n is given by where Ψ(x) = (x x )(x x ) (x x n ) E = f(x) P (x) = f n+ (ξ) (n + )! Ψ(x), The question is: How to choose these (n + ) nodes x, x,, x n so tht Ψ(x) is minimized in [-, ]? The nswer cn be given from the lst-mentioned property of the monic Chebyshev polynomils Note tht Ψ(x) is monic polynomil of degree (n + ) So, by the minimx property mx T n+ (x) mx Ψ(x) x [,] x [,] Tht is, the mximum vlue of ψ(x) is smllest when x, x,, x n re chosen s the (n + ) zeros of T n+ (x) nd this mximum vlue is n Choosing the Nodes for Minimizing Polynomil Interpoltion error To minimize the polynomil interpoltion error, choose the nodes x, x,, x n s the (n+) zeros of the (n + )th degree monic Chebyshev polynomil Note (Working with n rbitrry intervl) If the intervl is [, b], different from [, ], then, the zeros of T n+ (x) need to be shifted by using the trnsformtion: x = [(b )x + ( + b)] Exmple Let the interpolting polynomil be of degree t most nd the intervl be [5, ] The three zeros of T 3 (x) in [, ] re given by x = cos π 6, x = cos π, nd x 3 = cos 5 6 π These zeros re to be shifted using trnsformtion: x new = [( 5) x i + ( + 5)]

22 Use of Chebyshev Polynomils to Economize Power Series Power Series Economiztion Let P n (x) = + x+ + n x n be polynomil of degree n obtined by truncting power series expnsion of continuous function on [, b] The problem is to find polynomil P r (x) of degree r (< n) such tht P n (x) P r (x) < ɛ, where ɛ is tolernce supplied by users The problem is esily solved by using the Minimx Property of the Chebyshev polynomils First note tht n P n (x) P n (x) is monic polynomil So, by the minimx property, we hve Thus, if we choose mx P n (x) P n (x) n mx T n (x) = n n P n (x) = P n (x) n Tn (x), then the minimum vlue of mx P n (x) P n (x) = n n n If this quntity,, plus error due to the trunction of the power series is within the n permissible tolernce ɛ, we cn then repet the process by constructing P n (x) from P n (x) s bove The process cn be continued until the ccumulted error exceeds the tolernce ɛ So, the process cn be summrized s follows: Power Series Economiztion Process by Chebyshev Polynomils Obtin P n (x) = + x n + + n x n by truncting the power series expnsion of f(x) Find the error of trunction E P T Compute P n (x): P n (x) = P n (x) n Tn (x) Check if the totl error ( E P T + n n ) is less thn ɛ If so, continue the process by decresing the degree of the polynomils successively until the ccumulted error becomes greter thn ɛ

### Orthogonal Polynomials and Least-Squares Approximations to Functions

Chpter Orthogonl Polynomils nd Lest-Squres Approximtions to Functions **4/5/3 ET. Discrete Lest-Squres Approximtions Given set of dt points (x,y ), (x,y ),..., (x m,y m ), norml nd useful prctice in mny

### Best Approximation. Chapter The General Case

Chpter 4 Best Approximtion 4.1 The Generl Cse In the previous chpter, we hve seen how n interpolting polynomil cn be used s n pproximtion to given function. We now wnt to find the best pproximtion to given

### Numerical Methods I Orthogonal Polynomials

Numericl Methods I Orthogonl Polynomils Aleksndr Donev Cournt Institute, NYU 1 donev@cournt.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fll 2014 Nov 6th, 2014 A. Donev (Cournt Institute) Lecture IX

### Chapter 3 Polynomials

Dr M DRAIEF As described in the introduction of Chpter 1, pplictions of solving liner equtions rise in number of different settings In prticulr, we will in this chpter focus on the problem of modelling

Lecture notes on Vritionl nd Approximte Methods in Applied Mthemtics - A Peirce UBC Lecture 6: Singulr Integrls, Open Qudrture rules, nd Guss Qudrture (Compiled 6 August 7) In this lecture we discuss the

### Lecture 19: Continuous Least Squares Approximation

Lecture 19: Continuous Lest Squres Approximtion 33 Continuous lest squres pproximtion We begn 31 with the problem of pproximting some f C[, b] with polynomil p P n t the discrete points x, x 1,, x m for

### Best Approximation in the 2-norm

Jim Lmbers MAT 77 Fll Semester 1-11 Lecture 1 Notes These notes correspond to Sections 9. nd 9.3 in the text. Best Approximtion in the -norm Suppose tht we wish to obtin function f n (x) tht is liner combintion

### 1 The Lagrange interpolation formula

Notes on Qudrture 1 The Lgrnge interpoltion formul We briefly recll the Lgrnge interpoltion formul. The strting point is collection of N + 1 rel points (x 0, y 0 ), (x 1, y 1 ),..., (x N, y N ), with x

### Theoretical foundations of Gaussian quadrature

Theoreticl foundtions of Gussin qudrture 1 Inner product vector spce Definition 1. A vector spce (or liner spce) is set V = {u, v, w,...} in which the following two opertions re defined: (A) Addition of

### Orthogonal Polynomials

Mth 4401 Gussin Qudrture Pge 1 Orthogonl Polynomils Orthogonl polynomils rise from series solutions to differentil equtions, lthough they cn be rrived t in vriety of different mnners. Orthogonl polynomils

### Numerical Analysis: Trapezoidal and Simpson s Rule

nd Simpson s Mthemticl question we re interested in numericlly nswering How to we evlute I = f (x) dx? Clculus tells us tht if F(x) is the ntiderivtive of function f (x) on the intervl [, b], then I =

Lecture 14: Qudrture This lecture is concerned with the evlution of integrls fx)dx 1) over finite intervl [, b] The integrnd fx) is ssumed to be rel-vlues nd smooth The pproximtion of n integrl by numericl

### Numerical Integration

Chpter 5 Numericl Integrtion Numericl integrtion is the study of how the numericl vlue of n integrl cn be found. Methods of function pproximtion discussed in Chpter??, i.e., function pproximtion vi the

### Numerical Analysis. Doron Levy. Department of Mathematics Stanford University

Numericl Anlysis Doron Levy Deprtment of Mthemtics Stnford University December 1, 2005 D. Levy Prefce i D. Levy CONTENTS Contents Prefce i 1 Introduction 1 2 Interpoltion 2 2.1 Wht is Interpoltion?............................

### III. Lecture on Numerical Integration. File faclib/dattab/lecture-notes/numerical-inter03.tex /by EC, 3/14/2008 at 15:11, version 9

III Lecture on Numericl Integrtion File fclib/dttb/lecture-notes/numerical-inter03.tex /by EC, 3/14/008 t 15:11, version 9 1 Sttement of the Numericl Integrtion Problem In this lecture we consider the

### Math 1B, lecture 4: Error bounds for numerical methods

Mth B, lecture 4: Error bounds for numericl methods Nthn Pflueger 4 September 0 Introduction The five numericl methods descried in the previous lecture ll operte by the sme principle: they pproximte the

### 1. Gauss-Jacobi quadrature and Legendre polynomials. p(t)w(t)dt, p {p(x 0 ),...p(x n )} p(t)w(t)dt = w k p(x k ),

1. Guss-Jcobi qudrture nd Legendre polynomils Simpson s rule for evluting n integrl f(t)dt gives the correct nswer with error of bout O(n 4 ) (with constnt tht depends on f, in prticulr, it depends on

### Numerical quadrature based on interpolating functions: A MATLAB implementation

SEMINAR REPORT Numericl qudrture bsed on interpolting functions: A MATLAB implementtion by Venkt Ayylsomyjul A seminr report submitted in prtil fulfillment for the degree of Mster of Science (M.Sc) in

### CAAM 453 NUMERICAL ANALYSIS I Examination There are four questions, plus a bonus. Do not look at them until you begin the exam.

Exmintion 1 Posted 23 October 2002. Due no lter thn 5pm on Mondy, 28 October 2002. Instructions: 1. Time limit: 3 uninterrupted hours. 2. There re four questions, plus bonus. Do not look t them until you

### Numerical integration

2 Numericl integrtion This is pge i Printer: Opque this 2. Introduction Numericl integrtion is problem tht is prt of mny problems in the economics nd econometrics literture. The orgniztion of this chpter

### MORE FUNCTION GRAPHING; OPTIMIZATION. (Last edited October 28, 2013 at 11:09pm.)

MORE FUNCTION GRAPHING; OPTIMIZATION FRI, OCT 25, 203 (Lst edited October 28, 203 t :09pm.) Exercise. Let n be n rbitrry positive integer. Give n exmple of function with exctly n verticl symptotes. Give

### Abstract inner product spaces

WEEK 4 Abstrct inner product spces Definition An inner product spce is vector spce V over the rel field R equipped with rule for multiplying vectors, such tht the product of two vectors is sclr, nd the

### Math 270A: Numerical Linear Algebra

Mth 70A: Numericl Liner Algebr Instructor: Michel Holst Fll Qurter 014 Homework Assignment #3 Due Give to TA t lest few dys before finl if you wnt feedbck. Exercise 3.1. (The Bsic Liner Method for Liner

### NUMERICAL INTEGRATION. The inverse process to differentiation in calculus is integration. Mathematically, integration is represented by.

NUMERICAL INTEGRATION 1 Introduction The inverse process to differentition in clculus is integrtion. Mthemticlly, integrtion is represented by f(x) dx which stnds for the integrl of the function f(x) with

### Lecture 4: Piecewise Cubic Interpolation

Lecture notes on Vritionl nd Approximte Methods in Applied Mthemtics - A Peirce UBC Lecture 4: Piecewise Cubic Interpoltion Compiled 5 September In this lecture we consider piecewise cubic interpoltion

### CMDA 4604: Intermediate Topics in Mathematical Modeling Lecture 19: Interpolation and Quadrature

CMDA 4604: Intermedite Topics in Mthemticl Modeling Lecture 19: Interpoltion nd Qudrture In this lecture we mke brief diversion into the res of interpoltion nd qudrture. Given function f C[, b], we sy

### Review of Calculus, cont d

Jim Lmbers MAT 460 Fll Semester 2009-10 Lecture 3 Notes These notes correspond to Section 1.1 in the text. Review of Clculus, cont d Riemnn Sums nd the Definite Integrl There re mny cses in which some

### P 3 (x) = f(0) + f (0)x + f (0) 2. x 2 + f (0) . In the problem set, you are asked to show, in general, the n th order term is a n = f (n) (0)

1 Tylor polynomils In Section 3.5, we discussed how to pproximte function f(x) round point in terms of its first derivtive f (x) evluted t, tht is using the liner pproximtion f() + f ()(x ). We clled this

### Construction of Gauss Quadrature Rules

Jim Lmbers MAT 772 Fll Semester 2010-11 Lecture 15 Notes These notes correspond to Sections 10.2 nd 10.3 in the text. Construction of Guss Qudrture Rules Previously, we lerned tht Newton-Cotes qudrture

### Sturm-Liouville Eigenvalue problem: Let p(x) > 0, q(x) 0, r(x) 0 in I = (a, b). Here we assume b > a. Let X C 2 1

Ch.4. INTEGRAL EQUATIONS AND GREEN S FUNCTIONS Ronld B Guenther nd John W Lee, Prtil Differentil Equtions of Mthemticl Physics nd Integrl Equtions. Hildebrnd, Methods of Applied Mthemtics, second edition

### Lecture 17. Integration: Gauss Quadrature. David Semeraro. University of Illinois at Urbana-Champaign. March 20, 2014

Lecture 17 Integrtion: Guss Qudrture Dvid Semerro University of Illinois t Urbn-Chmpign Mrch 0, 014 Dvid Semerro (NCSA) CS 57 Mrch 0, 014 1 / 9 Tody: Objectives identify the most widely used qudrture method

### Numerical Analysis. Chassidy Bozeman. May 30, 2016

Numericl Anlysis Chssidy Bozemn My 30, 016 Contents 1 Theorems 1 1.1 561 Theorems............................................. 1 1. 56 Theorems............................................. Root Finding.1

### The Regulated and Riemann Integrals

Chpter 1 The Regulted nd Riemnn Integrls 1.1 Introduction We will consider severl different pproches to defining the definite integrl f(x) dx of function f(x). These definitions will ll ssign the sme vlue

### Properties of Integrals, Indefinite Integrals. Goals: Definition of the Definite Integral Integral Calculations using Antiderivatives

Block #6: Properties of Integrls, Indefinite Integrls Gols: Definition of the Definite Integrl Integrl Clcultions using Antiderivtives Properties of Integrls The Indefinite Integrl 1 Riemnn Sums - 1 Riemnn

### Week 10: Riemann integral and its properties

Clculus nd Liner Algebr for Biomedicl Engineering Week 10: Riemnn integrl nd its properties H. Führ, Lehrstuhl A für Mthemtik, RWTH Achen, WS 07 Motivtion: Computing flow from flow rtes 1 We observe the

### DEFINITION The inner product of two functions f 1 and f 2 on an interval [a, b] is the number. ( f 1, f 2 ) b DEFINITION 11.1.

398 CHAPTER 11 ORTHOGONAL FUNCTIONS AND FOURIER SERIES 11.1 ORTHOGONAL FUNCTIONS REVIEW MATERIAL The notions of generlized vectors nd vector spces cn e found in ny liner lger text. INTRODUCTION The concepts

### Section 7.1 Integration by Substitution

Section 7. Integrtion by Substitution Evlute ech of the following integrls. Keep in mind tht using substitution my not work on some problems. For one of the definite integrls, it is not possible to find

### UNIFORM CONVERGENCE MA 403: REAL ANALYSIS, INSTRUCTOR: B. V. LIMAYE

UNIFORM CONVERGENCE MA 403: REAL ANALYSIS, INSTRUCTOR: B. V. LIMAYE 1. Pointwise Convergence of Sequence Let E be set nd Y be metric spce. Consider functions f n : E Y for n = 1, 2,.... We sy tht the sequence

### The Fundamental Theorem of Calculus. The Total Change Theorem and the Area Under a Curve.

Clculus Li Vs The Fundmentl Theorem of Clculus. The Totl Chnge Theorem nd the Are Under Curve. Recll the following fct from Clculus course. If continuous function f(x) represents the rte of chnge of F

### Lecture 1. Functional series. Pointwise and uniform convergence.

1 Introduction. Lecture 1. Functionl series. Pointwise nd uniform convergence. In this course we study mongst other things Fourier series. The Fourier series for periodic function f(x) with period 2π is

### Partial Derivatives. Limits. For a single variable function f (x), the limit lim

Limits Prtil Derivtives For single vrible function f (x), the limit lim x f (x) exists only if the right-hnd side limit equls to the left-hnd side limit, i.e., lim f (x) = lim f (x). x x + For two vribles

### New Expansion and Infinite Series

Interntionl Mthemticl Forum, Vol. 9, 204, no. 22, 06-073 HIKARI Ltd, www.m-hikri.com http://dx.doi.org/0.2988/imf.204.4502 New Expnsion nd Infinite Series Diyun Zhng College of Computer Nnjing University

### Numerical Integration. 1 Introduction. 2 Midpoint Rule, Trapezoid Rule, Simpson Rule. AMSC/CMSC 460/466 T. von Petersdorff 1

AMSC/CMSC 46/466 T. von Petersdorff 1 umericl Integrtion 1 Introduction We wnt to pproximte the integrl I := f xdx where we re given, b nd the function f s subroutine. We evlute f t points x 1,...,x n

### Numerical Integration

Chpter 1 Numericl Integrtion Numericl differentition methods compute pproximtions to the derivtive of function from known vlues of the function. Numericl integrtion uses the sme informtion to compute numericl

### 1 The Riemann Integral

The Riemnn Integrl. An exmple leding to the notion of integrl (res) We know how to find (i.e. define) the re of rectngle (bse height), tringle ( (sum of res of tringles). But how do we find/define n re

### Chapter 3 Solving Nonlinear Equations

Chpter 3 Solving Nonliner Equtions 3.1 Introduction The nonliner function of unknown vrible x is in the form of where n could be non-integer. Root is the numericl vlue of x tht stisfies f ( x) 0. Grphiclly,

### NUMERICAL INTEGRATION

NUMERICAL INTEGRATION How do we evlute I = f (x) dx By the fundmentl theorem of clculus, if F (x) is n ntiderivtive of f (x), then I = f (x) dx = F (x) b = F (b) F () However, in prctice most integrls

### 1 Linear Least Squares

Lest Squres Pge 1 1 Liner Lest Squres I will try to be consistent in nottion, with n being the number of dt points, nd m < n being the number of prmeters in model function. We re interested in solving

### Euler-Maclaurin Summation Formula 1

Jnury 9, Euler-Mclurin Summtion Formul Suppose tht f nd its derivtive re continuous functions on the closed intervl [, b]. Let ψ(x) {x}, where {x} x [x] is the frctionl prt of x. Lemm : If < b nd, b Z,

### Lecture Note 4: Numerical differentiation and integration. Xiaoqun Zhang Shanghai Jiao Tong University

Lecture Note 4: Numericl differentition nd integrtion Xioqun Zng Sngi Jio Tong University Lst updted: November, 0 Numericl Anlysis. Numericl differentition.. Introduction Find n pproximtion of f (x 0 ),

### Review of Riemann Integral

1 Review of Riemnn Integrl In this chpter we review the definition of Riemnn integrl of bounded function f : [, b] R, nd point out its limittions so s to be convinced of the necessity of more generl integrl.

### B.Sc. in Mathematics (Ordinary)

R48/0 DUBLIN INSTITUTE OF TECHNOLOGY KEVIN STREET, DUBLIN 8 B.Sc. in Mthemtics (Ordinry) SUPPLEMENTAL EXAMINATIONS 01 Numericl Methods Dr. D. Mckey Dr. C. Hills Dr. E.A. Cox Full mrks for complete nswers

### 63. Representation of functions as power series Consider a power series. ( 1) n x 2n for all 1 < x < 1

3 9. SEQUENCES AND SERIES 63. Representtion of functions s power series Consider power series x 2 + x 4 x 6 + x 8 + = ( ) n x 2n It is geometric series with q = x 2 nd therefore it converges for ll q =

### Matrices, Moments and Quadrature, cont d

Jim Lmbers MAT 285 Summer Session 2015-16 Lecture 2 Notes Mtrices, Moments nd Qudrture, cont d We hve described how Jcobi mtrices cn be used to compute nodes nd weights for Gussin qudrture rules for generl

### 3.4 Numerical integration

3.4. Numericl integrtion 63 3.4 Numericl integrtion In mny economic pplictions it is necessry to compute the definite integrl of relvlued function f with respect to "weight" function w over n intervl [,

### Part IB Numerical Analysis

Prt IB Numericl Anlysis Theorems with proof Bsed on lectures by G. Moore Notes tken by Dexter Chu Lent 2016 These notes re not endorsed by the lecturers, nd I hve modified them (often significntly) fter

### Calculus and linear algebra for biomedical engineering Week 11: The Riemann integral and its properties

Clculus nd liner lgebr for biomedicl engineering Week 11: The Riemnn integrl nd its properties Hrtmut Führ fuehr@mth.rwth-chen.de Lehrstuhl A für Mthemtik, RWTH Achen Jnury 9, 2009 Overview 1 Motivtion:

### Unit #9 : Definite Integral Properties; Fundamental Theorem of Calculus

Unit #9 : Definite Integrl Properties; Fundmentl Theorem of Clculus Gols: Identify properties of definite integrls Define odd nd even functions, nd reltionship to integrl vlues Introduce the Fundmentl

### AM1 Mathematical Analysis 1 Oct Feb Exercises Lecture 3. sin(x + h) sin x h cos(x + h) cos x h

AM Mthemticl Anlysis Oct. Feb. Dte: October Exercises Lecture Exercise.. If h, prove the following identities hold for ll x: sin(x + h) sin x h cos(x + h) cos x h = sin γ γ = sin γ γ cos(x + γ) (.) sin(x

### Continuous Random Variables

STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 217 Néhémy Lim Continuous Rndom Vribles Nottion. The indictor function of set S is rel-vlued function defined by : { 1 if x S 1 S (x) if x S Suppose tht

### DOING PHYSICS WITH MATLAB MATHEMATICAL ROUTINES

DOIG PHYSICS WITH MATLAB MATHEMATICAL ROUTIES COMPUTATIO OF OE-DIMESIOAL ITEGRALS In Cooper School of Physics, University of Sydney in.cooper@sydney.edu.u DOWLOAD DIRECTORY FOR MATLAB SCRIPTS mth_integrtion_1d.m

### Review of Gaussian Quadrature method

Review of Gussin Qudrture method Nsser M. Asi Spring 006 compiled on Sundy Decemer 1, 017 t 09:1 PM 1 The prolem To find numericl vlue for the integrl of rel vlued function of rel vrile over specific rnge

### Integrals - Motivation

Integrls - Motivtion When we looked t function s rte of chnge If f(x) is liner, the nswer is esy slope If f(x) is non-liner, we hd to work hrd limits derivtive A relted question is the re under f(x) (but

### LECTURE 19. Numerical Integration. Z b. is generally thought of as representing the area under the graph of fèxè between the points x = a and

LECTURE 9 Numericl Integrtion Recll from Clculus I tht denite integrl is generlly thought of s representing the re under the grph of fèxè between the points x = nd x = b, even though this is ctully only

### Introduction to Numerical Analysis. Hector D. Ceniceros

Introduction to Numericl Anlysis Hector D. Ceniceros c Drft dte November 27, 2018 Contents Contents i Prefce 1 1 Introduction 3 1.1 Wht is Numericl Anlysis?.................. 3 1.2 An Illustrtive Exmple.....................

Undergrdute Reserch A Trigonometric Simpson s Rule By Ctherine Cusimno Kirby nd Sony Stnley Biogrphicl Sketch Ctherine Cusimno Kirby is the dughter of Donn nd Sm Cusimno. Originlly from Vestvi Hills, Albm,

### Cambridge International Examinations Cambridge International Advanced Subsidiary and Advanced Level. Published

Cmbridge Interntionl Exmintions Cmbridge Interntionl Advnced Subsidiry nd Advnced Level MATHEMATICS 9709/ Pper October/November 06 MARK SCHEME Mximum Mrk: 75 Published This mrk scheme is published s n

### Euler, Ioachimescu and the trapezium rule. G.J.O. Jameson (Math. Gazette 96 (2012), )

Euler, Iochimescu nd the trpezium rule G.J.O. Jmeson (Mth. Gzette 96 (0), 36 4) The following results were estblished in recent Gzette rticle [, Theorems, 3, 4]. Given > 0 nd 0 < s

### COSC 3361 Numerical Analysis I Numerical Integration and Differentiation (III) - Gauss Quadrature and Adaptive Quadrature

COSC 336 Numericl Anlysis I Numericl Integrtion nd Dierentition III - Guss Qudrture nd Adptive Qudrture Edgr Griel Fll 5 COSC 336 Numericl Anlysis I Edgr Griel Summry o the lst lecture I For pproximting

### Physics 116C Solution of inhomogeneous ordinary differential equations using Green s functions

Physics 6C Solution of inhomogeneous ordinry differentil equtions using Green s functions Peter Young November 5, 29 Homogeneous Equtions We hve studied, especilly in long HW problem, second order liner

### Polynomials and Division Theory

Higher Checklist (Unit ) Higher Checklist (Unit ) Polynomils nd Division Theory Skill Achieved? Know tht polynomil (expression) is of the form: n x + n x n + n x n + + n x + x + 0 where the i R re the

### Math 473: Practice Problems for the Material after Test 2, Fall 2011

Mth 473: Prctice Problems for the Mteril fter Test, Fll SOLUTION. Consider the following modified het eqution u t = u xx + u x. () Use the relevnt 3 point pproximtion formuls for u xx nd u x to derive

### Numerical Integration

Numericl Integrtion Wouter J. Den Hn London School of Economics c 2011 by Wouter J. Den Hn June 3, 2011 Qudrture techniques I = f (x)dx n n w i f (x i ) = w i f i i=1 i=1 Nodes: x i Weights: w i Qudrture

### Advanced Computational Fluid Dynamics AA215A Lecture 3 Polynomial Interpolation: Numerical Differentiation and Integration.

Advnced Computtionl Fluid Dynmics AA215A Lecture 3 Polynomil Interpoltion: Numericl Differentition nd Integrtion Antony Jmeson Winter Qurter, 2016, Stnford, CA Lst revised on Jnury 7, 2016 Contents 3 Polynomil

### Math& 152 Section Integration by Parts

Mth& 5 Section 7. - Integrtion by Prts Integrtion by prts is rule tht trnsforms the integrl of the product of two functions into other (idelly simpler) integrls. Recll from Clculus I tht given two differentible

### Chapter 6 Notes, Larson/Hostetler 3e

Contents 6. Antiderivtives nd the Rules of Integrtion.......................... 6. Are nd the Definite Integrl.................................. 6.. Are............................................ 6. Reimnn

### MATH34032: Green s Functions, Integral Equations and the Calculus of Variations 1

MATH34032: Green s Functions, Integrl Equtions nd the Clculus of Vritions 1 Section 1 Function spces nd opertors Here we gives some brief detils nd definitions, prticulrly relting to opertors. For further

### Lecture Notes: Orthogonal Polynomials, Gaussian Quadrature, and Integral Equations

18330 Lecture Notes: Orthogonl Polynomils, Gussin Qudrture, nd Integrl Equtions Homer Reid My 1, 2014 In the previous set of notes we rrived t the definition of Chebyshev polynomils T n (x) vi the following

### Overview of Calculus I

Overview of Clculus I Prof. Jim Swift Northern Arizon University There re three key concepts in clculus: The limit, the derivtive, nd the integrl. You need to understnd the definitions of these three things,

### Chapter 4 Contravariance, Covariance, and Spacetime Diagrams

Chpter 4 Contrvrince, Covrince, nd Spcetime Digrms 4. The Components of Vector in Skewed Coordintes We hve seen in Chpter 3; figure 3.9, tht in order to show inertil motion tht is consistent with the Lorentz

### We partition C into n small arcs by forming a partition of [a, b] by picking s i as follows: a = s 0 < s 1 < < s n = b.

Mth 255 - Vector lculus II Notes 4.2 Pth nd Line Integrls We begin with discussion of pth integrls (the book clls them sclr line integrls). We will do this for function of two vribles, but these ides cn

### Introduction to Numerical Analysis

Introduction to Numericl Anlysis Doron Levy Deprtment of Mthemtics nd Center for Scientific Computtion nd Mthemticl Modeling (CSCAMM) University of Mrylnd June 14, 2012 D. Levy CONTENTS Contents 1 Introduction

### Chapter 3 MATRIX. In this chapter: 3.1 MATRIX NOTATION AND TERMINOLOGY

Chpter 3 MTRIX In this chpter: Definition nd terms Specil Mtrices Mtrix Opertion: Trnspose, Equlity, Sum, Difference, Sclr Multipliction, Mtrix Multipliction, Determinnt, Inverse ppliction of Mtrix in

### Chapter 5 : Continuous Random Variables

STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 216 Néhémy Lim Chpter 5 : Continuous Rndom Vribles Nottions. N {, 1, 2,...}, set of nturl numbers (i.e. ll nonnegtive integers); N {1, 2,...}, set of ll

### Section 4.8. D v(t j 1 ) t. (4.8.1) j=1

Difference Equtions to Differentil Equtions Section.8 Distnce, Position, nd the Length of Curves Although we motivted the definition of the definite integrl with the notion of re, there re mny pplictions

### How can we approximate the area of a region in the plane? What is an interpretation of the area under the graph of a velocity function?

Mth 125 Summry Here re some thoughts I ws hving while considering wht to put on the first midterm. The core of your studying should be the ssigned homework problems: mke sure you relly understnd those

### x = b a N. (13-1) The set of points used to subdivide the range [a, b] (see Fig. 13.1) is

Jnury 28, 2002 13. The Integrl The concept of integrtion, nd the motivtion for developing this concept, were described in the previous chpter. Now we must define the integrl, crefully nd completely. According

### The Riemann Integral

Deprtment of Mthemtics King Sud University 2017-2018 Tble of contents 1 Anti-derivtive Function nd Indefinite Integrls 2 3 4 5 Indefinite Integrls & Anti-derivtive Function Definition Let f : I R be function

### A REVIEW OF CALCULUS CONCEPTS FOR JDEP 384H. Thomas Shores Department of Mathematics University of Nebraska Spring 2007

A REVIEW OF CALCULUS CONCEPTS FOR JDEP 384H Thoms Shores Deprtment of Mthemtics University of Nebrsk Spring 2007 Contents Rtes of Chnge nd Derivtives 1 Dierentils 4 Are nd Integrls 5 Multivrite Clculus

### Math 520 Final Exam Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008

Mth 520 Finl Exm Topic Outline Sections 1 3 (Xio/Dums/Liw) Spring 2008 The finl exm will be held on Tuesdy, My 13, 2-5pm in 117 McMilln Wht will be covered The finl exm will cover the mteril from ll of

### Interpolation. Gaussian Quadrature. September 25, 2011

Gussin Qudrture September 25, 2011 Approximtion of integrls Approximtion of integrls by qudrture Mny definite integrls cnnot be computed in closed form, nd must be pproximted numericlly. Bsic building

### THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS.

THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS RADON ROSBOROUGH https://intuitiveexplntionscom/picrd-lindelof-theorem/ This document is proof of the existence-uniqueness theorem

### Numerical Linear Algebra Assignment 008

Numericl Liner Algebr Assignment 008 Nguyen Qun B Hong Students t Fculty of Mth nd Computer Science, Ho Chi Minh University of Science, Vietnm emil. nguyenqunbhong@gmil.com blog. http://hongnguyenqunb.wordpress.com

Lecture 12: Numericl Qudrture J.K. Ryn@tudelft.nl WI3097TU Delft Institute of Applied Mthemtics Delft University of Technology 5 December 2012 () Numericl Qudrture 5 December 2012 1 / 46 Outline 1 Review

### Convex Sets and Functions

B Convex Sets nd Functions Definition B1 Let L, +, ) be rel liner spce nd let C be subset of L The set C is convex if, for ll x,y C nd ll [, 1], we hve 1 )x+y C In other words, every point on the line

### c n φ n (x), 0 < x < L, (1) n=1

SECTION : Fourier Series. MATH4. In section 4, we will study method clled Seprtion of Vribles for finding exct solutions to certin clss of prtil differentil equtions (PDEs. To do this, it will be necessry

### Quantum Physics II (8.05) Fall 2013 Assignment 2

Quntum Physics II (8.05) Fll 2013 Assignment 2 Msschusetts Institute of Technology Physics Deprtment Due Fridy September 20, 2013 September 13, 2013 3:00 pm Suggested Reding Continued from lst week: 1.

### Sections 5.2: The Definite Integral

Sections 5.2: The Definite Integrl In this section we shll formlize the ides from the lst section to functions in generl. We strt with forml definition.. The Definite Integrl Definition.. Suppose f(x)