MATH745 Fall 5 MATH745 Fall 5 INTRODUCTION WELCOME TO MATH 745 TOPICS IN NUMERICAL ANALYSIS Instructor: Dr Bartosz Protas Department of Matematics & Statistics Email: bprotas@mcmasterca Office HH 36, Ext 46 Course Webpage: ttp://wwwmatmcmasterca/ bprotas/math745 Wat is NUMERICAL ANALYSIS? Development of COMPUTATIONAL ALGORITHMS for solutions of problems in algebra and analysis Use of metods of MATHEMATICAL ANALYSIS to determine a priori properties of tese algoritms suc as: ACCURACY, STABILITY, CONVERGENCE Application of tese algoritms to solve actual problems arising in practice MATH745 Fall 5 3 MATH745 Fall 5 4 BASICS a I PART I Finite Differences A Review ASSUMPTIONS : f : Ω R is a smoot function, ie is continuously differentiable sufficiently many times, te domain Ω = [a,b] is discretized wit a uniform grid {x = a,,x N = b}, suc tat x + x = = extensions to nonuniform grids are straigtforward PROBLEM given te nodal values of te function f, ie, f = f x, =,, N approximate te nodal values of te function derivative d f dx x = f x, =,,N a Details can be found in any standard textbook on elementary numerical analysis, eg, K Atkinson and W Han, Elementary Numerical Analysis, Wiley, 4
MATH745 Fall 5 5 MATH745 Fall 5 6 BASICS II Te simplest approac Derivation of finite difference formulae via TAYLOR SERIES EXPANSIONS f + = f + x + x f + x + x Rearrange te expansion! = f + f + + 3 + f = f + f f + x + x 3 f + 3! f + = f + f +O, were O α denotes te contribution from all terms wit powers of greater or equal α ere α = Neglecting O, we obtain a FIRST ORDER FORWARD DIFFERENCE FORMULA : = f + f BASICS III Backward difference formula is obtained by expanding f about x and proceeding as before: f = f f f + = = f f Neglected term wit te lowest power of is te LEADING ORDER APPROXIMATION ERROR, ie, Err = f x C α Te exponent α of in te leading order error represents te ORDER OF ACCURACY OF THE METHOD it tells ow quickly te approximation error vanises wen te resolution is refined Te actual value of te approximation error depends on te constant C caracterizing te function f In te examples above Err = f, ence te metods are FIRST ORDER ACCURATE MATH745 Fall 5 7 MATH745 Fall 5 8 HIGHER ORDER FORMULA I Consider two expansions: Subtracting te second from te first: Central Difference Formula f = f + f Te leading order error is ACCURATE f + = f + f + + 3 + f = f f + 3 + f + f = f + 3 3 f + + = = f + f, tus te metod is SECOND ORDER Manipulating four different Taylor series expansions one can obtain a fourt order central difference formula : = f + + 8 f + 8 f + f, Err = 4 3 f v APPROXIMATION OF THE SECOND DERIVATIVE Consider two expansions: Adding te two expansions f + = f + f + + 3 + f = f f + 3 + f + + f = f + + 4 f iv + Central difference formula for te second derivative: f = f + f + f f iv + = Te leading order error is ACCURATE f iv δ f = f + f + f, tus te metod is SECOND ORDER
MATH745 Fall 5 9 MATH745 Fall 5 AN ALTERNATIVE APPROACH I An alternative derivation of a finite difference sceme: Find an N t order accurate interpolating function px wic interpolates te function f x at te nodes x, =,,N, ie, suc tat px = f x, =,,N Differentiate te interpolating function px and evaluate at te nodes to obtain an approximation of te derivative p x f x, =,,N Example: for =,,N, let te interpolant ave te form of a quadratic polynomial p x on [x,x + ] Lagrange interpolating polynomial p x = x x x x + f + x x x x + f + x x x x f + p x = x x x + f + x x x + f + x x x f + Evaluating at x = x we obtain f x p x = f + f ie, second order accurate center difference formula AN ALTERNATIVE APPROACH II Generalization to iger orders straigtforward Example: for = 3,,N, one can use a fourt order polynomial as interpolant p x on [x,x + ] Differentiating wit respect to x and evaluating at x = x we arrive at te fourt order accurate finite difference formula = f + + 8 f + 8 f + f, Err = 4 3 f v Order of accuracy of te finite difference formula is one less tan te order of te interpolating polynomial Te set of grid points needed to evaluate a finite difference formula is called STENCIL In general, iger order formulas ave larger stencils MATH745 Fall 5 MATH745 Fall 5 TAYLOR TABLE I A general metod for coosing te coefficients of a finite difference formula to ensure te igest possible order of accuracy Example: consider a one sided finite difference formula p= α p f +p, were te coefficients α p, p =,, are to be determined Form an expression for te approximation error f p= and expand it about x in te powers of α p f +p = ε TAYLOR TABLE II Expansions can be collected in a Taylor table f f f f f a f a a f + a a a a 3 6 a f + a a a a 3 6 te leftmost column contains te terms present in te expression for te approximation error te corresponding rows multiplied by te top row represent te terms obtained from expansions about x columns represent terms wit te same order in sums of columns are te contributions to te approximation error wit te given order in Te coefficients α p, p =,, can now be cosen to cancel te contributions to te approximation error wit te lowest powers of
MATH745 Fall 5 3 MATH745 Fall 5 4 TAYLOR TABLE III Setting te coefficients of te first tree terms to zero: a a a = a a = = a = 3 a a, a =, a = = Te resulting formula: = f + + 4 f + 3 f Te approximation error determined te evaluating te first column wit non zero coefficient: a 3 6 a 3 f = 6 3 f Te formula is tus SECOND ORDER ACCURATE AN OPERATOR PERSPECTIVE I Quick review of FUNCTIONAL ANALYSIS background NORMED SPACES X: : X R suc tat x,y X Banac spaces x, x + y x + y, x = x vector spaces: finite dimensional R N vs infinite dimensional l p function spaces on Ω R N : Lebesgue spaces L p Ω, Sobolev spaces W p,q Ω Hilbert spaces: inner products, ortogonality & proections, bases, etc Linear Operators: operator norms, functionals, Riesz Teorem MATH745 Fall 5 5 MATH745 Fall 5 6 AN OPERATOR PERSPECTIVE II Assume tat f and f belong to a function space X; DIFFERENTIATION d dx : f f can ten be regarded as a LINEAR OPERATOR dx d : X X Wen f and f are approximated by teir nodal values as f = [ f f f N ] T and f = [ f f f N ]T, ten te differential operator dx d can be approximated by a DIFFERENTIATION MATRIX A R N N suc tat f = Af ; How can we determine tis matrix? Assume for simplicity tat te domain Ω is periodic, ie, f = f N and f = f N+ ; ten differentiation wit te second order center difference formula can be represented as te following matrix vector product f f = f N f N AN OPERATOR PERSPECTIVE III Using te fourt order center difference formula we would obtain a pentadiagonal system increased order of accuracy entails increased bandwidt of te differentiation matrix A A is a TOEPLITZ MATRIX, since is as constant entries along te te diagonals; in fact, it a also a CIRCULANT MATRIX wit entries a i depending only on i mod N Note tat te matrix A defined above is SINGULAR as a zero eigenvalue λ = Wy? Tis property is in fact inerited from te original continuous operator d dx wic is also singular and as a zero eigenvalue A singular matrix A does not ave an inverse at least, now in te classical sense; wat can we do to get around tis difficulty?
MATH745 Fall 5 7 MATH745 Fall 5 8 AN OPERATOR PERSPECTIVE IV Matrix singularity linearly dependent rows te LHS vector does not contain enoug information to determine UNIQUELY te RHS vector MATRIX DESINGULARIZATION incorporating additional information into te matrix, so tat its argument te RHS vector can be determined uniquely Example desingularization of te second order center difference differentiation matrix: in a center difference formula, even and odd nodes are decoupled knowing f, =,,N and f, one can recover f, = 3,5, ie, te odd nodes only f must also be provided ence, te zero eigenvalue as multiplicity two wen desingularizing te differentiation matrix one must modify at least two rows see, eg, sing_diff_mat_m AN OPERATOR PERSPECTIVE V Wat is WRONG wit te differentiation operator? Te differentiation operator dx d is UNBOUNDED! One usually cannot find a constant C R independent of f, suc tat d dx f x X C f X, f X For instance, f x = e ikx, so tat C = k for k Unfortunately, finite dimensional emulations of te differentiation operator te DIFFERENTIATION MATRICES inerit tis property OPERATOR NORM for matrices A = max x = Ax = max x Ax, Ax x,x x,a T Ax = max = λ max A T A = σ x x,x maxa Tus, te norm of a matrix is given by te square root of its largest SINGULAR VALUE σ max A MATH745 Fall 5 9 MATH745 Fall 5 AN OPERATOR PERSPECTIVE VI As can be rigorously proved in many specific cases, A grows witout bound as N or, tis is a reflection of te unbounded nature of te underlying dim operator Te loss of precision wen solving te system Ax = b is caracterized by te CONDITION NUMBER wit respect to inversion κ p A = A p A p for p =, κ A = σ maxa σ min A wen te condition number is large, te matrix is said to be ILL CONDITIONED solution of te system Ax = b is prone to round off errors if A is singular, κ p A = + SUBTRACTIVE CANCELLATION ERRORS SUBTRACTIVE CANCELLATION ERRORS wen comparing two numbers wic are almost te same using finite precision aritmetic, te relative round off error is proportional to te inverse of te difference between te two numbers Tus, if te difference between te two numbers is decreased by an order of magnitude, te relative accuracy wit wic tis difference may be calculated using finite precision aritmetic is also decreased by an order of magnitude Problems wit finite difference formulae wen loss of precision due to finite precision aritmetic SUBTRACTIVE CANCELLATION, eg, for double precision: 345 e 8% error 34 e 3 9% error
MATH745 Fall 5 MATH745 Fall 5 COMPLEX STEP DERIVATIVE a PADÉ APPROXIMATION I Consider te complex extension f z, were z = x + iy, of f x and compute te complex Taylor series expansion Take imaginary part and divide by f x + i = f + i f i 3 +O 4 f = I f x + i + 6 f +O 3 = = I f x + i Note tat te sceme is second order accurate were is conservation of complexity? Te metod doesn t suffer from cancellation errors, is easy to implement and quite useful a J N Lyness and C BMoler, Numerical differentiation of analytical functions, SIAM J Numer Anal 4, -, 967 GENERAL IDEA include in te finite difference formula not only te function values, but also te values of te FUNCTION DERIVATIVE at te adacent nodes, eg: b f + f + b f + p= α p f +p = ε Construct te Taylor table using te following expansions: f + = f + f + f + 3 6 f f + = f + f + f + 3 + 4 4 f iv + 5 f v + 6 f iv + 4 4 f v + NOTE need an expansion for te derivative and a iger order expansion for te function more coefficient to determine MATH745 Fall 5 3 MATH745 Fall 5 4 PADÉ APPROXIMATION II Te Taylor table f f f f b f b b b b 3 6 b 4 4 f b f + b b b b 3 6 b 4 4 a f a a a a 3 6 a 4 4 a 5 a f a a f + a a a a 3 6 a 4 4 a 5 Te algebraic system: / / / / 3 /6 3 /6 3 /6 3 /6 4 /4 4 /4 b b a a a f iv = = b b a a a f v /4 /4 = 3/4 3/4 PADÉ APPROXIMATION III Te Padé approximation: 4 + + + 4 = 3 f + f 4 Leading order error 4 3 f v FOURTH ORDER ACCURATE Te approximation is NONLOCAL, in tat it requires derivatives at te adacent nodes wic are also unknowns; Tus all derivatives must be determined at once via te solution of te following algebraic system /4 /4 + = 3 4 f + f
MATH745 Fall 5 5 MATH745 Fall 5 6 PADÉ APPROXIMATION IV Closing te system at ENDPOINTS were neigbors are not available use a lower order one sided ie, forward or backward finite difference formula Te vector of derivatives can tus be obtained via solution of te following algebraic system were Bf = 3 Af = f = 3 B Af B is a tri diagonal matrix wit b i,i = and b i,i = b i,i+ = 4, i =,,N A is a second order accurate differentiation matrix MODIFIED WAVENUMBER ANALYSIS I How do finite differences perform at different WAVELENGTHS? Finite Difference formulae applied to THE FOURIER MODE f x = e ikx wit te exact derivative f x = ike ikx Central Difference formula: = f + f = eikx + e ikx were te modified wavenumber k sink = eik e ik e ikx = i sink f = ik f, Comparison of te modified wavenumber k wit te actual wavenumber k sows ow numerical differentiation errors affect different Fourier components of a given function MATH745 Fall 5 7 MODIFIED WAVENUMBER ANALYSIS II Fourt-order central difference formula = f + + 8 f + 8 f + f ] [ 4 = i 3 sink 6 sink = e ik e ik f e ik e ik f 3 f = ik f were te modified wavenumber k [ 4 3 sink 6 sink] Fourt order Padé sceme: were Tus: 4 + + + 4 + = ik e ikx + = ik e ik f and = 3 4 f + f, = ik e ikx = ik e ik f ik 4 eik + + 4 e ik f = 3 e ik e ik f 4 ik + cosk f = i 3 sink f = k 3sink + cosk