Introduction to multiscale modeling and simulation Lecture 5 Numerical methods for ODEs, SDEs and PDEs The need for multiscale methods Two generic frameworks for multiscale computation Explicit methods for ODEs : forward Euler We consider an initial value problem dy dt = f(y), y(0) = y 0 An analytical solution y(t) cannot be obtained, so we resort to numerical methods. y n y(t n ) t n = n t The simplest method of this type is forward Euler y n+ = y n + tf(y n )
Consistency of the forward Euler method Forward Euler method is given by y n+ = y n + tf(y n ) Let us fill in the exact solution and Taylor expand y(t n+ ) y(t n )+ tf(y(t n )) + t 2 y (t n ) 2 Thus, in one step, we make an error of O( t 2 ) n = t / t To get at a given time t, we must take steps The global error is then O( t), i.e. first order Higher-order methods can be developed (see a dedicated course) Stability of the forward Euler method Consistency shows that the method converges for t 0 What happens for finite t? Look at stability of the method! Consider the test equation y = λy Re(λ) < 0 Solutions of the forward Euler scheme need to tend to 0 for y n+ = y n ( + λ t) t Im(λ t) λ R : t<2/ λ -2 - Re(λ t) -
Possibly unphysical behavior of forward Euler Just because a numerical method is stable, does not imply desired behavior! y n+ = y n ( + λ t) =ρy n λ R : t<2/ λ Exact solution : y(t n )=exp( λn t) y 0 Numerical approximation : y n =(+λ t) n y 0 = ρ n y 0 amplification factor 0.75 ( + λ t) > 0 ( + λ t) < 0 0.5 y(t) 0.5 y(t) 0.25 0 0 0 2 3 4 t 0.5 0 2 4 6 8 0 t Forward Euler for a two-scale test equation Consider a linear system with two time scales dx dt = λx dy dt = y λ / λ R This system has eigenvalues and. Consider Then, forward Euler is stable if both λ t <2 t/ < 2 So, for 0, we have the condition t <2
Forward Euler does not always represent fast modes well y(t) 0.75 0.5 0.25 x n+ = x n ( + λ t) =ρ λ x n y n+ = y n ( t/) =ρ y n t = /2 When t = O(), time-scales of fast equation are changed! - Im(ρ) Re(ρ) 0 0.5 0 0.5.5 2 t t =.8 Fast decaying modes may become slowly decaying, but oscillatory! - Im(ρ) y(t) 0 0.5 - Re(ρ) 0 0.5.5 2 t - Slow modes are unaffected by the time step Im(ρ) x n+ = x n ( + λ t) =ρ λ x n y n+ = y n ( t/) =ρ y n Im(ρ) - Re(ρ) - Re(ρ) - - 2.5 t = /2 2.5 t =.8 x(t) x(t) 0.5 0.5 0 0 0.5.5 2 t 0 0 0.5.5 2 t
Stiffness Consider a slow-fast system du = u u 2 +2 dt du 2 = dt (u3 u 2 ) Quickly relaxes to du dt = u u 3 +2 u 2 = u 3 A discrepancy between t - the (large) time step that can be taken to obtain required accuracy u u 2 - the (small) time step for that is needed for stability δt = O() Alternative approach : Implicit methods for ODEs dy dt = f(y), y(0) = y 0 An alternative to forward Euler is backward Euler y n+ = y n + tf(y n+ ) Now, the solution y n+ is found as the solution of a nonlinear system (more work per time-step!) Method is still first order accurate y(t n+ ) y(t n ) t = f(y(t n+ )) + O( t)
Stability of the backward Euler method Again consider the test equation y = λy Then, backward Euler gives y n+ = y n + tλy n+ ( λ t) y n+ = y n y n+ = λ t yn When Re(λ) < 0 ρ = λ t yn < Im(λ t) 2 - Re(λ t) When Re(λ) ρ 0 Explicit methods for SDEs : Euler-Maruyama We consider an initial value problem dy = g(y)dt + β(y)dw, y(0) = y 0 Discretizing in time gives y n+ = y n + g(y n ) t + β(y n ) W The Brownian increments W have mean zero and variance The scheme then can be written as t y n+ = y n + g(y n ) t + β(y n ) tξ n, ξ n N(0, )
Consistency of the Euler-Maruyama method We will only state some results without proof. Order of consistency depends on how the error is defined. Strong convergence... -... tells something about the expectation of the error y n y(t n ) = O( t /2 ) -... measures convergence of individual trajectories Weak convergence... -... tells something about the error in expectations y n y(t n ) = O( t) -... measures convergence in a distribution sense Stability of the Euler-Maruyama method () As for consistency, stability can be defined in different ways. Consider a linear test equation dy = λy dt + µy dw Method is stable if solutions tends to 0 for t Mean-square stability lim y(t) 2 =0 t Asymptotic stability lim y(t) =0, a.s. t
Stability of the Euler-Maruyama method (2) Stability can crucially depend on the type of test equation considered. Consider for instance an Ornstein-Uhlenbeck process dy = λy dt + µdw, Re(λ) < 0 The previous definitions are useless here, since the process itself approximates a normal distribution for t. We define the numerical method to be stable if its solution also approaches a normal distribution for t. One can show that the Euler-Maruyama scheme for the SDE is stable for a given time step t, if and only if the forward Euler method is stable for the deterministic part. Euler-Maruyama for a two-scale test equation Consider a fast Ornstein-Uhlenbeck process dy = ydt+ 2 dw At the macroscopic level, equivalent to (see lecture 4) dµ dt = µ dσ dt = 2 Σ+2 µ = y Σ= (y µ) 2 It is easy to show that (exercise!) Euler-Maruyama gives us µ n+ = t µ n Σ n+ = 2 t + t2 2 Σ n
Euler-Maruyama may affect time-scale separation Effect of Euler-Maruyama on mean and variance µ n+ = t µ n As for the deterministic case, choosing behavior : t Σ n+ = 2 t + t2 2 Σ n + 2 t t = O() significantly affects the - : very quick relaxation to equilibrium distribution t 2 - : very slow (oscillatory) relaxation to equilibrium distribution Implicit Euler doesn t capture invariant distribution Consider a stochastic version of implicit Euler y n+ = y n + g(y n+ ) t + β(y n ) W Apply this method to the fast Ornstein-Uhlenbeck process dy = 2 ydt+ dw This gives + 2 t t y n+ = y n + ξn Then (do as an exercise! ) lim n yn =0 lim n (y n ) 2 = + t/(2)
Explicit methods for PDEs Method of lines semi-discretization + forward Euler Consider an advection-diffusion equation t u(x, t)+c x u(x, t) =D xx u(x, t) Discretize the spatial derivatives with finite differences du(x i,t) dt + c u(x i,t) u(x i,t) x Results in a coupled system of ODEs du dt = Au, A = tridiag Apply forward Euler to this system of ODEs u n+ =(I + A t)u n = D u(x i+,t) 2u(x i,t)+u(x i,t) x 2 c x + D x 2, c x 2 D x 2, D x 2 Stability of forward Euler for PDE discretization Consider the method-of-lines system, and define the spectrum of A du dt = Au, σ(a) ={λ i} N i= Forward Euler is stable if the stability criterion is satisfied for all eigenvalues +λ i t < For pure diffusion D>0,c=0, this leads to For pure advection D =0,c>0, this leads to t x2 2D t x c
Overview of the lecture Numerical methods for ODEs, SDEs, and PDEs Multiscale challenges Computational frameworks : - equation-free methods - heterogeneous multiscale methods Challenge for multiscale ODEs We have a multiscale ODE dx dt dy dt = f(x, y) = g(x, y) lim t ϕt ξ(y) =η(ξ) g(ξ,η(ξ)) = 0 Explicit methods have time-step limitation δt = O() We know that a macroscopic model exists when dx dt = F 0(X) =f(x, η(x)) 0 For this macroscopic model, we have t = O()
Challenge for multiscale SDEs We have a multiscale SDE dx = f(x, y)dt dy = g(x, y)dt + β(x, y)dw lim ρ ξ(y, t) =ρ ξ (y) t Implicit methods don t work; explicit methods have a time-step restriction We know that a macroscopic model exists when dx dt = F (X) = δt = O() For this macroscopic model, we have t = O() 0 f(x, y)dµ X (y) = f(x, y)ρ X (y)dy Challenge for PDE homogenization problems We have a PDE with heterogeneous coefficient t u (x, t) x A x x u (x, t)) = f(x) Explicit methods have a time-step restriction δt = O(δx 2 )=O( 2 ) We know that a macroscopic model exists for 0 t U(x, t) x Ā x U(x, t) = f(x) For this macroscopic model, we have t = O( x 2 )=O()
Challenge for kinetic equations We have a kinetic equation for the phase-space density t f (x, v, t)+ v xf (x, v, t) = f f 2 Explicit discretization has a time-step restriction δt = O( 2 ) We know that a macroscopic model exists when 0 ρ ρ : t ρ = D xx ρ For this macroscopic model, we have t = O( x 2 ) 25 Challenge for Monte Carlo simulation of SDEs This is a slightly different setting : we do not impose an infinite time-scale separation here. We have an SDE for behavior of an individual particle (here: FENE model) dx = ux 2ζ F (X) dt + σ 2 dw t, F(X) = ζ X X 2 /b Direct Monte Carlo simulation requires an ensemble of M particles, with time step δt. We assume an approximate macroscopic model exists in terms of some macroscopic state variables U i = f i (X) du dt = H(U, u) τ p = T (U)
Overview of the lecture Numerical methods for ODEs, SDEs, and PDEs Multiscale challenges Computational frameworks : - equation-free methods - heterogeneous multiscale methods Equation-free multiscale framework Generic strategy for the development of multiscale methods Coarse time-stepper as a building block for efficient algorithms Microscopic level model known simulation code available Macroscopic level only state variables corresponding evolution law unknown Simulatie Lifting Restrictie t* t* +!t Kevrekidis et al., 2000 -... / Kevrekidis & S, Annual Review on Physical Chemistry 60:32-344, 2009 28
Coarse projective integration t* t* + "t t* +!t Extrapolate macroscopic state forward in time using an estimated time derivative Resembles a forward Euler method for just the macroscopic state variables 29 Related strategy: heterogeneous multiscale methods Postulate a general form for the unknown macroscopic equation Supplement this equation with an estimation of missing macroscopic quantities from a microscopic simulation - Initialization of the microscopic model from a given macroscopic state - Estimation of a macroscopic quantity from microscopic data This formulation has advantages from a numerical analysis viewpoint E, Engquist, Vanden-Eijnden, et al., 2003 -... 30
Bifurcation analysis Consider a system that depends on a number of physical parameters dx dt = F (X; λ) Find asymptotic states (equilibria, periodic solutions,...) F (X; λ) =0 Solutions are located on branches (X(s),λ(s)) Multiple solutions possible for the same parameter value Stability can change along a branch 3 Equation-free bifurcation analysis U(t) U(t + τ) =Φ τ (U(t)) Time-stepper is a black box Compute directly macroscopic steady states and their stability U Φ τ (U )=0 Use iterative methods (like Newton-Krylov) Matrix-vector products Φ τ (Ū) DΦ τ Ū (Ū) v Φ τ (Ū + v) Ū + v 32
Goals for the remaining lectures The equation-free and heterogeneous multiscale frameworks provide a common algorithmic structure to design and analyze multiscale methods. For each concrete problem, specific methods can be constructed : - Lifting and restriction operators need to be developed - Accuracy, stability, efficiency need to be examined We will discuss numerical techniques for each of the model problems. We will also look into alternative techniques and relate them to the generic frameworks of this course. 33