Finite Differences for Differential Equations 28 PART II. Finite Difference Methods for Differential Equations

Finite Differences for Differential Equations 28 PART II Finite Difference Methods for Differential Equations

Finite Differences for Differential Equations 29 BOUNDARY VALUE PROBLEMS (I) Solving a TWO POINT BOUNDARY VALUE PROBLEM with DIRICHLET BOUNDARY CONDITIONS : Finite difference approximation: d 2 y dx 2 = g for x (0,2π) y(0) = y(2π) = 0 Second order center difference formula for the interior nodes: y j+1 2y j + y j 1 h 2 = g j for j = 1,...,N where h = 2π N+1 and x j = jh Endpoint nodes: y 0 = 0 = y 2 2y 1 = h 2 g 1 y N+1 = 0 = 2y N + y N 1 = h 2 g N Tridiagonal algebraic system solved very efficiently with the THOMAS ALGORITHM (a version of the Gaussian elimination)

Finite Differences for Differential Equations 30 BOUNDARY VALUE PROBLEMS (II) Solving a TWO POINT BOUNDARY VALUE PROBLEM with NEUMANN BOUNDARY CONDITIONS : Finite difference approximation: d 2 y dx 2 = g for x (0,2π) dy dy (0) = dx dx (2π) = 0 Second order center difference formula for the interior nodes: y j+1 2y j + y j 1 h 2 = g j for j = 1,...,N First order Forward/Backward Difference formulae to re express endpoint values: y 1 y 0 h y N+1 y N h = 0 = y 0 = y 1 = 0 = y N+1 = y N First order only DEGRADED ACCURACY! Tridiagonal algebraic system Is there any problem? Where?

Finite Differences for Differential Equations 31 BOUNDARY VALUE PROBLEMS (III) In order to retain the SECOND ORDER ACCURACY in the approximation of the Neumann problem need to use higher-order formulae at endpoints, e.g. y 0 = y 2 + 4y 1 3y 0 2h = 0 = y 0 = 1 3 ( y 2 + 4y 1 ) The first row thus becomes 2 3 y 2 2 3 y 1 = h 2 g 1 SECOND ORDER ACCURACY RECOVERED!

Finite Differences for Differential Equations 32 BOUNDARY VALUE PROBLEMS (IV) COMPACT STENCILS stencils based on three grid points (in every direction) only: {x j+1,x j,x j 1 } at the j th node Is is possible to obtain higher (then second) order of accuracy on compact stencils? YES! Consider the central difference approximation to the equation d2 y dy 2 = g y j+1 2y j + y j 1 h 2 h2 12 y(iv) j +O(h 4 ) = g j Re-express the error term h2 h 2 12 y(iv) j 12 y(iv) j = h2 12 g j = h2 12 using the equation in question: [ g j+1 2g j + g j 1 Inserting into the original finite difference equation: y j+1 2y j + y j 1 h 2 Slight modification of the RHS = h 2 = g j + g j+1 2g j + g j 1 12 ] h2 12 g(iv) j +O(h 4 ) +O(h 4 ) FOURTH ORDER ACCURACY!!!

Finite Differences for Differential Equations 33 BOUNDARY VALUE PROBLEMS (V) COMPACT FINITE DIFFERENCE SCHEMES ADVANTAGES: Increased accuracy on compact grids DRAWBACKS: need to be tailored to the specific equation solved can get fairly complicated for more complex equations

Finite Differences for Differential Equations 34 GENERAL REMARKS Consider the following CAUCHY PROBLEM : dy dt = f(y,t) with y(t 0) = y 0 The independent variable t is usually referred to as TIME. Equations with higher order derivatives can be reduced to systems of first order equations Generalizations to systems of ODEs straightforward When the RHS function does not depend on y, i.e., f(y,t) = f(t), solution obtained via a QUADRATURE Assume uniform time steps ( h is constant )

Finite Differences for Differential Equations 35 CHARACTERIZATION OF INTEGRATION METHODS ACCURACY unlike in the Boundary Value Problems, there is no terminal condition and approximation errors may accumulate in time; consequently, a relevant characterization of accuracy is provided by the GLOBAL ERROR rather than the LOCAL ERROR. (global error) = (local error) (# of time steps), STABILITY unlike in the Boundary Value Problems, where boundedness of the solution at final time is enforced via a suitable terminal condition, in Initial Value Problems there is a priori no guarantee that the solution will remain bounded.

Finite Differences for Differential Equations 36 MODEL PROBLEM STABILITY of various numerical schemes is usually analyzed by applying these schemes to the following LINEAR MODEL : which is stable when λ r <= 0. EXACT SOLUTION: y(t) = y 0 e λt = dy dt = λy = (λ r + iλ i )y with y(t 0 ) = y 0, (1+λh+ λ2 h 2 2 + λ3 h 3 6 ) +... y 0 MOTIVATION consider the following ADVECTION DIFFUSION PDE : u t + c u x a 2 u x 2 = 0 Taking Fourier transform yields (k is the wavenumber): dû k dt + cik û k + ak 2 û k = 0 where the real term ak 2 û k represents DIFFUSION the imaginary term cik û k represents ADVECTION

Finite Differences for Differential Equations 37 EXPLICIT EULER SCHEME (I) Consider a Taylor series expansion Using the ODE we obtain y(t n+1 ) = y(t n )+hy (t n )+ h2 2 y (t n )+... y = dy dt y = dy dt = f = d f dt = f t + f f y Neglecting terms proportional to second and higher powers of h yields the EXPLICIT EULER METHOD y n+1 = y n + h f(y n,t n ) Retaining higher order terms is inconvenient, as it requires differentiation of f and does not lead to schemes with desirable stability properties.

Finite Differences for Differential Equations 38 EXPLICIT EULER SCHEME (II) LOCAL ERROR analysis: GLOBAL ERROR analysis: Thus, the scheme is locally second order accurate y n+1 = (1+λh)y n +[O(h 2 )] (global error) = Ch 2 N = Ch 2 T h = C h globally (over the interval [t 0,t 0 + Nh]) first order accurate

Finite Differences for Differential Equations 39 EXPLICIT EULER SCHEME (III) Stability (for the model problem) Thus, the solution after n time steps y n+1 = y n + λhy n = (1+λh)y n y n = (1+λh) n y 0 σ n y 0 = σ = 1+λh For large n, the numerical solution remains stable iff σ 1 = (1+λ r h) 2 +(λ i h) 2 1 CONDITIONALLY STABLE for real λ UNSTABLE for imaginary λ

Finite Differences for Differential Equations 40 IMPLICIT EULER SCHEME (I) IMPLICIT SCHEMES based on approximation of the RHS that involve f(y n+1,t), where y n+1 is the unknown to be determined IMPLICIT EULER SCHEME obtained by neglecting second and higher order terms in the expansion: Upon substitution dy dt The scheme is y(t n ) = y(t n+1 ) hy (t n+1 )+ h2 2 y (t n+1 )... tn+1 = f(y n+1,t n+1 ) we obtain locally SECOND ORDER accurate y n+1 = y n + h f(y n+1,t n+1 ) globally (over the interval [t 0,t 0 + Nh]) FIRST ORDER accurate

Finite Differences for Differential Equations 41 IMPLICIT EULER SCHEME (II) Stability (for the model problem): y n+1 = y n+1 = y n + λhy n+1 ( 1 1 λh Implicit Euler scheme is thus stable for all stable model problems most unstable model problems = y n+1 = (1 λh) 1 y n ) n y 0 σ n y 0 = σ = 1 1 λh σ 1 = (1 λ r h) 2 +(λ i h) 2 1 REMARK: When solving systems of ODEs of the form y =A(t)y, each implicit step requires solution of an algebraic system: y n+1 = (I ha) 1 y n Implicit schemes are generally hard to implement for nonlinear problems

Finite Differences for Differential Equations 42 CRANK NICHOLSON SCHEME (I) Obtained by approximating the formal solution of the ODE y n+1 = y n + R t n+1 t n f(y,t)dt using the TRAPEZOIDAL QUADRATURE : The scheme is locally THIRD ORDER accurate y n+1 = y n + h 2 [ f(y n,t n )+ f(y n+1,t n+1 )] globally (over the interval [t 0,t 0 + Nh]) SECOND ORDER accurate Stability (for the model problem): y n+1 = y n + λh 2 (y n+1 + y n ) = y n+1 = ( ) 1+ λh n 2 1 λh y 0 σ n y 0 = σ = 2 y n+1 = 1+ λh 2 1 λh 2 σ 1 = R(λh) 0 STABLE for all model ODEs with stable solutions ( 1+ λh 2 1 λh 2 ) y n

Finite Differences for Differential Equations 43 LEAPFROG SCHEME (I) LEAPFROG as an example of a TWO STEP METHOD : y n+1 = y n 1 + 2hλy n CHARACTERISTIC EQUATION for the AMPLIFICATION FACTOR (y n = σ n y 0 ) σ 2 2hλσ 1 = 0 where roots give the amplification factors: σ 1 = λh+ 1+λ 2 h 2 1+λh+ λ2 h 2 2 σ 2 = λh 1+λ 2 h 2 (1 λh+ λ2 h 2 2 Thus, the scheme is locally THIRD ORDER accurate +... = e λh +O(h 3 )...) = e λh +O(h 3 ) globally (over the interval [t 0,t 0 + Nh]) SECOND ORDER accurate

Finite Differences for Differential Equations 44 LEAPFROG SCHEME (II) Stability for diffusion problems ( λ = λ r ): σ 1 = λh+ 1+λ 2 r h 2 > 1 for all h > 0 Thus the scheme is UNCONDITIONALLY UNSTABLE for diffusion problems! Stability for advection problems ( λ = iλ i ): σ 2 1/2 = 1 (!!!) for h < 1 λ i Thus, the scheme is CONDITIONALLY UNSTABLE and NON DIFFUSIVE for advection problems! QUESTION analyze dispersive (i.e., related to arg(σ)) errors of the leapfrog scheme.

Finite Differences for Differential Equations 45 MULTISTEP PROCEDURES General form of a MULTISTEP PROCEDURE : p j=1 α j y n+ j = h q j=1 β j f(y n+ j,t n+ j ) with characteristic polynomials ξ p (z) = α p z p + α p 1 z p 1 + +α 0 if p > q EXPLICIT SCHEME if p q IMPLICIT SCHEME ζ q (z) = β q z q + β q 1 z q 1 + +β 0 A (ξ,ζ) procedure converges uniformly in [a,b], i.e., lim h 0 max tn [a,b] y n y(t n ) = 0 if: the following consistency conditions are verified: ξ(1) = 0 and ξ (1) = ζ(1) ( CONSISTENCY CONDITION ) all roots of the polynomial ξ(z) are such that z i 1 and the roots with z k = 1 are simple ( STABILITY CONDITION )

Finite Differences for Differential Equations 46 RUNGE KUTTA METHODS (I) General form of a FRACTIONAL STEP METHOD : y n+1 = y n + γ 1 hk 1 + γ 2 hk 2 + γ 3 hk 3 +... where k 1 = f(y n,t n ) k 2 = f(y n + β 1 hk 1,t n + α 1 h) k 3 = f(y n + β 2 hk 1 + β 3 hk 2,t n + α 2 h). Choose γ i, β i and α i to match as many expansion coefficients as possible in y(t n+1 ) = y(t n )+hy (t n )+ h2 2 y (t n )+ h3 6 y (t n )... y = f y = f t + f f y y = f tt + f t f y 2 f f yt + f 2 f yt + f 2 f yy Runge Kutta methods are SELF STARTING with fairly good stability and accuracy properties.

Finite Differences for Differential Equations 47 RUNGE KUTTA METHODS (II) RK4 an ODE workhorse : The amplification factor: Thus, stability iff σ 1 ACCURACY: Thus, the scheme is y n+1 = y n + h 6 k 1 + h 3 (k 2 + k 3 )+ h 6 k 4 k 1 = f(y n,t n ) k 2 = f(y n + h 2 k 1,t n+1/2 ) k 3 = f(y n + h 2 k 2,t n+1/2 ) k 4 = f(y n + hk 3,t n+1 ) σ = 1+λh+ λ2 h 2 locally FIFTH ORDER accurate 2 + λ3 h 3 6 e λh = σ+o(h 5 ) + λ4 h 4 24 globally (over the interval [t 0,t 0 + Nh]) FOURTH ORDER accurate

Finite Differences for Differential Equations 48 RUNGE S PRINCIPLE Let (k+1) be the order of the local truncation error; denote Y(t,h) an approximation of the exact solution y(t) computed with the step size h; then at t = t 0 + 2nh: y(t) Y(t,h) C 2nh k+1 = C(t t 0 )h k y(t) Y(t,2h) C n(2h) k+1 = C(t t 0 )2 k h k Subtracting: Y(t,2h) Y(t,h) C(t t 0 )(1 2 k )h k Thus, we can obtain an estimate of the ABSOLUTE ERROR based on solution with two step sizes only: y(t) Y(t,h) Y(t,h) Y(t,2h) 2 k 1 Runge s principle is very useful for ADAPTIVE STEP SIZE REFINEMENT

Finite Differences for Differential Equations 49 LAX EQUIVALENCE THEOREM a Consider an INITIAL VALUE PROBLEM du dt =Lu with u(t 0) = u 0 and assume that it is well posed, i.e., it admits solutions which are unique and stable Consider a numerical method defined by a finite difference operator C(h) such that the approximate solution is given by u h (nh) =C(h) n u 0, n = 1,2,... The above method is CONSISTENT iff C(h) I h of the operatorl is a convergent approximation LAX THEOREM For a CONSISTENT difference method STABILITY is equivalent to CONVERGENCE a For a more technical discussion, see $ 5.2 in Atkinson & Han (2001)

Finite Differences for Differential Equations 50 CONSERVATION PROPERTIES (I) Is ACCURACY and STABILITY all that matters? CONSERVATION PROPERTIES conservation by the numerical method (i.e., in the discrete sense) of various invariants the original equation may possess REMARK conservation properties are particularly relevant for solution of Hamiltonian / hyperbolic systems Example conservation of the solution norm: In the continuous setting (assume u = u e iϕ ) du dt = iλ iu d u = 0 = u(t) = u 0, dt dϕ = λ i, dt In the discrete setting: u h (nh) = u h ((n 1)h) = = u h (0) Necessary and sufficient condition for discrete conservation: h, σ(h) = 1

Finite Differences for Differential Equations 51 Implicit Euler CONSERVATION PROPERTIES (II) 1 σ = = 1 iλ i h 1 1+λ 2 i h2 = 1 1 2 λ2 i h 2 + < 1 for all h The scheme is thus DISSIPATIVE (i.e., not conservative) Fourth Order Runge Kutta σ = 1+iλ ih λ2 i h2 2 i λ3 i h3 6 + λ4 i h4 24 = 1 24 = 1 1 144 λ6 i h 6 + < 1 for small h The scheme is thus DISSIPATIVE (i.e., not conservative) 576 8λ 6 i h6 + λ 8 i h8 Leapfrog σ 1/2 1 for all h < 1 λ i The scheme is thus CONSERVATIVE for all time steps for which it is stable!!! Leapfrog is an example of a SYMPLECTIC INTEGRATOR which are designed to have good conservation properties.

Finite Differences for Differential Equations 52 FINITE DIFFERENCES FOR PDES REVIEW Classification of linear PDEs in 2D: consider u : Ω 2 R and A,B,C R such that ELLIPTIC PROBLEMS : B 2 4AC < 0 Poisson equation: A 2 u x 2 + B 2 u x y +C 2 u y 2 + f(x,y,u) = 0 2 u x 2 + 2 u y 2 = g(x,y) PARABOLIC PROBLEMS : B 2 4AC = 0 Heat equation: ( u 2 ) t = a u x 2 + 2 u y 2 + g(x,y) HYPERBOLIC PROBLEMS : B 2 4AC > 0 Wave equation: 2 ( u 2 ) t 2 = a u x 2 + 2 u y 2 + g(x,y)

Finite Differences for Differential Equations 53 FINITE DIFFERENCES FOR PDES ELLIPTIC PROBLEMS See Homework Assignment # 1...

Finite Differences for Differential Equations 54 FINITE DIFFERENCES FOR PDES PARABOLIC PROBLEMS u t = 2 u x 2 (I) CRANK NICHOLSON METHOD (x j = j x, j = 1,...,M, t = n t, n = 1,...,N): spatial derivative: time derivative: ( u t ) n+1 j = un+1 j t u n j ( 2 u x 2 ) n j = un j+1 2un j +un j 1 ( x) 2 +O(( x) 2 ) +O( t) = 1 2 [ ( 2 ) n+1 u x 2 + j ( 2 u x 2 u n+1 j u n j = t 2( x) 2 ( u n+1 j+1 2un+1 j + u n+1 j 1 + un j+1 2un j + u n j 1 ) n j ] +O(( t) 2 ) )+O ( ( x) 2 +( t) 2) thus, defining r = t ( x) 2,we have at every time step n ru n+1 j+1 + 2(1+r)un+1 j ru n+1 j 1 = run j+1 + 2(1 r)un j + ru n j 1 which for U n = [u n 1,...,un M ]T can be written as an algebraic system (2I A)U n+1 = (2I+A)U n, where A is a tridiagonal matrix

Finite Differences for Differential Equations 55 θ METHOD FINITE DIFFERENCES FOR PDES PARABOLIC PROBLEMS u t = 2 u x 2 (II) allow for a more general approximation in time of the RHS (θ [0,1]) ( ) u n+1 t j special cases Stability: = un+1 j t u n j +O( t) = 1 2 [ θ ( 2 ) n+1 ( u 2 u x 2 +(1 θ) j x 2 θ = 0 = EXPLICIT METHOD: U n+1 = A 0 U n θ = 1 2 = CRANK NICHOLSON METHOD (see previous slide) θ = 1 = IMPLICIT METHOD: A 1 U n+1 = U n ) n j ] +O( t) The EXPLICIT SCHEME is STABLE for r = t 2( x) 2 < 1 2 The CRANK NICHOLSON and IMPLICIT SCHEME are STABLE for all r

Finite Differences for Differential Equations 56 FINITE DIFFERENCES FOR PDES HYPERBOLIC PROBLEMS 2 u = 2 u t 2 Spatial derivative: Time derivative: ( 2 u t 2 u n+1 j ) n j = un+1 j ( 2 u x 2 ) n j = un j+1 2un j +un j 1 ( x) 2 +O(( x) 2 ) 2u n j + un 1 j ( t) 2 +O(( t) 2 ) = = ( t)2 ( u n ( x) 2 j+1 + u n ) j 1 u n 1 j + 2 Stability for ( t)2 ( x) 2 1 REMARK: need two initial conditions! ( 2 ) n u x 2 (1 ( t)2 ( x) 2 j x 2 (I) +O(( t) 3 ) ) u n j +O ( ( x) 2 +( t) 4)