Introduction to multiscale modeling and simulation Giovanni Samaey, Scientific Computing Dept. of Computer Science, K.U.Leuven Lecture 1: Course introduction Almost every system is multiscale. We are interested in behavior on large space and long time scales. The phenomenon is influenced by effects taking place on smaller spatial scales and shorter time scales.
We don t always notice this multiscale behavior. Microscopically, molecules are subject to Newton s equations of motion. Macroscopically, partial differential equations often describe evolution of density, momentum, energy to sufficient accurately. The microscopic description is then unnecessary! These phenomenological models often predate a microscopic understanding, and were only later derived from first principles. Multiscale problems cannot always be reduced. Bacterial chemotaxis : biased motion towards chemoattractant ( food ) Motion often steered by internal chemical networks source: Wikipedia taken from S. Lubkin microscopic level modeling level detailed individual-based models macroscopic level observation/simulation simplified advection-diffusion PDE
Why are multiscale problems difficult to simulate? Effects of different time scales - Micro: small time-steps necessary - Macro: large time intervals Effects of different spatial scales - Micro: fine grid necessary - Macro: large spatial domains y(t) 3.5 3 2.5 2 1.5 1 0 2 4 6 8 10 t Effects of different modeling scales - Micro: many degrees of freedom - Macro: averaged effect Often all these effects simultaneously! What will we do in this course? Multiscale model problems (3 lectures) - Relation between microscopic and macroscopic levels of description - Analytic reduction to macroscopic model and reconstruction of microscopic state Multiscale numerical techniques (6 lectures) - Exploit scale separation to reduce computational cost when analytical reduction is not possible - Accurately transfer information from micro to macro and back again Evaluation : read, summarize and evaluate a research paper in the field
Disclaimer The literature on analytical and numerical multiscale techniques is - enormous - spread out over many disconnected disciplines - in continuous evolution In this course, you will learn - a particular way of thinking about multiscale problems - that you can use as a starting point to dive into the literature In particular, this course is very much biased by my own research expertise! What will we do in this lecture? Present a number of applications in which multiple scales arise Isolate a single multiscale feature that is relevant in that application The main goals of this lecture are to : - illustrate that multiscale problems are ubiquitous in science and engineering - identify prior knowledge (or gaps therein) in this diverse audience (to adjust the exposition in the remaining lectures) - give an overview of the type of problems that will be tackled (first analytically, then numerically) throughout the remainder of the course
Time-scale separation Application 1 : Chemical kinetics Application 2 : A mechanical system (Dahlquist s clock) Application 1 : Chemical kinetics Chemical reactions and mass balance laws Kinetics of enzyme-substrate reaction (Michaelis-Menten reaction) k1 k 3 E+P E + S ES k2 Mass action laws describe evolution of concentration via set of ordinary differential equations (ODEs) d[e]/dt = d[s]/dt = d[es]/dt = d[p ]/dt = k1 [E][S] + k2 [ES] + k3 [ES] k1 [E][S] + k2 [ES] k1 [E][S] k2 [ES] k3[ ES] k3 [ES]
Reduction and nondimensionalization Due to mass conservation, we have So that the system can be reduced to a system of 2 ODEs Now we nondimensionalize... t = t/(k 1 E tot ) x =[S]/S tot y =[ES]/E tot... and introduce the nondimensional parameters...... to obtain [E]+[ES] E tot [S]+[ES]+[P ] S tot d[s]/dt = k 1 E tot [S]+(k 1 [S]+k 2 )[ES] d[es]/dt = k 1 E tot [S] (k 1 [S]+k 2 + k 3 )[ES] E + S k 1 ES k 3 E + P k2 = E tot /S tot λ = k 3 /(k 1 S tot ) κ(k 2 + k 3 )/(k 1 S tot ) x = x +(x + κ λ)y y = x (x + κ)y Time evolution with separated time scales E + S k 1 ES k 3 E + P k2 1 x = x +(x + κ λ)y y = x (x + κ)y 1 := E tot S tot 1 0.8 0.75 x(t) 0.6 y(t) 0.5 0.4 0.25 0.2 0 0.5 1 1.5 2 t Equation for y evolves on a much faster time scale than equation for x Small time-steps are needed to simulate the y-dynamics After a fast initial transient, the system has forgotten its initial condition in y (all behavior in y is very close to each other) 0 0 0.5 1 1.5 2 t
Phase portrait and slaving An informal derivation of a macroscopic model y 1 0.75 0.5 x = x +(x + κ λ)y y = x (x + κ)y κ =1 λ =1/2 0.25 0 0 0.2 0.4 0.6 0.8 1 x We see that y is uniquely determined by x after a transient. y(x) x x +1 This phenomenon is called slaving, and implies that an approximate macroscopic model exists X X = 2(X + 1) Because the small parameter disappeared, larger time-steps can be taken. Application 2 : Dahlquist s clock A very informal averaging argument Very fast vibrations Slow movement (towards the edge of the table...) Equations of motion x = v v = g(x, v) Instead of approaching a fixed point for each x, v oscillates quickly. The averaged effect of this velocity is felt on long time scales. We would again like to obtain a macroscopic model of the form X = V (X)
Time-scale separation : two types of problems Model problem 1 : ODE with fast slaving/invariant manifold structure - Fast variables slaved to (uniquely determined by) slow variables - Slaving relation is used to obtain a model for the slow variables alone Model problem 2 : averaging/homogenization for ODEs - Fast variables behave oscillatory for every value of the slow variables - Appropriately averaged fast behavior drives slow evolution Later on in the course : - derivation of approximate macroscopic models in idealized situations (lecture 2) - numerical methods that exploit scale separation when deriving the macroscopic model analytically is not possible (second part of the course) Space-scale separation Application 1 : Flow through porous media (groundwater, oil reservoir, nuclear waste, CO2 storage,...) Application 2 : Deformation of solid materials with local defects (cracks) The examples are equilibrium problems (no time dependence). If time dependent, the problems of time-scale separation are added.
Application 1 : Flow through porous media Darcy s law and groundwater flow equation Darcy s law : flow velocity is proportional to the pressure gradient q(x) = κ p(x) The proportionality constant is called the permeability... κ κ(x; )... and it depends on space and a small-scale parameter. We also have conservation of mass (fluid is incompressible) div q(x) =f(x) Combined, this gives the groundwater flow equation div κ(x; ) p(x) =f(x) source term Heterogeneity of the permeability div κ(x; ) p(x) =f(x) The heterogeneous permeability cannot be measured everywhere. It is therefore modeled as a random field : denotes an average over all possible realizations - Decompose into (space-independent) average + fluctuations κ(x; ) =κ + δκ(x; ) - The fluctuations are correlated with some fine-scale correlation length δκ(x; ) δκ(x ; ) = w(x x ; ) - Generalization to anisotropic fields is also possible One needs to simulate on a very fine grid to sample the heterogeneity.
div κ(x; ) p(x) =f(x) Homogenized equation with effective permeability To reduce computational cost, we want to solve on a coarser mesh. First idea : Underresolve (evaluate the heterogeneous permeability on the coarse mesh). This clearly does not converge. Second idea : Homogenize the heterogeneous coefficient. Two options : div κ p(x) =f(x) Replace the permeability coefficient by its mean. This does not converge either (but it is -at this point- not so clear why). Perform a mathematical homogenization to obtain an effective permeability. Application 2 : Solid materials Continuum modeling of deformation Deformed position is original position plus a displacement ϕ(x) =x + u(x) x ϕ(x) Equilibrium of stresses (ignoring boundary conditions) div σ(x) =f(x) System is closed by a constitutive law σ(x) σ(x, u(x),...) For elastic materials, stress only depends on deformation gradient F (x) = ϕ(x) σ σ(f ) If we have hyperelasticity, the stress is derived from an energy σ(f )= W(F )/ F The problem then becomes an energy minimization problem inf W ( ϕ)+...
Localized breakdown of continuum description Intendation experiment (large forces locally) Fracture propagation Some problems require an incredibly fine mesh locally. The mesh becomes commensurate with the size of individual atoms. In that region, the continuum description breaks down (cannot be used). Locally, an atomistic description is necessary. Atomistic modeling of deformation Microscopically, a solid material consists of atoms, aligned on a grid. Each atom is displaced Y i = ϕ(x i ), i I There is an interaction potential between every two atoms V (r ij )=V ( Y i Y j ) The energy of the sample is then 1 V (r ij ) 2 i,j, i=j
Space-scale separation : two types of problems Model problem 1 : homogenization of PDEs - Partial differential equation with highly heterogeneous coefficients - Goal : obtain/simulate an effective equation with homogenized coefficient Model problem 2 : localized breakdown of continuum descriptions - Macroscopic description: partial differential equation - Microscopic description: particle (atomistic) model - Goal : relate the two level of description/simulate using domain decomposition Later on in the course : - derivation of approximate macroscopic models in idealized situations (lecture 3) - numerical methods that exploit scale separation to simulate when deriving the macroscopic model analytically is not possible (second part of the course) Model-scale separation Application 1 : Complex fluids (blood, liquid crystals, polymers,...) Application 2 : Kinetic models (plasma physics, fluid mechanics, chemotaxis)
Before proceeding to application 1... A brief introduction to stochastic differential equations Consider the movement of a heavy particle in a solvent. 0.75 0.5 The heavy particle is bombarded with solvent molecules. y 0.25 0 0.25 The effect is an excitation into random fluctuations + dissipation due to friction. The resulting motion is very irregular (nowhere differentiable). We need a mathematical construction to model such motion. x(t) 0.5 0.4 0.2 0 0.2 0.4 x 0.4 0.2 0 0.2 0.4 0 2 4 6 8 10 t Definition of Brownian motion Aim: model random movement of a particle (drunk man s walk). Consider a process W(t) with the following properties : - W(t) is continuous as a function of time - Increments on a time interval [s,t] are Gaussian variables with variance t-s W (t) W (s) t sn(0, 1) - Non-overlapping increments are independent cov (W (t) W (s),w(u) W (v)) = 0 0 t<s u<v T The resulting Brownian motion is nowhere differentiable, because W (t + t) W (t) t is a random variable with mean 0 and variance 1/ t
A different viewpoint: probability distributions Consider starting a Brownian motion at 0 : W (0) = 0 Then, at each point in time, the process evaluates to a Gaussian random variable with variance t and mean 0. W (t) = tn(0, 1) The probability of finding the particle at position x at time t is then 4 P(x W (t) <x+ dx) =ρ(x, t) = exp( x2 /2t) 2πt 2 x(t) 0 2 4 0 0.2 0.4 0.6 0.8 1 t Expectation and Monte Carlo simulation Given a random variable X with a density ρ(x, t) We are interested in the expected value of a function of X (the average value over all possible Brownian paths). f(t) =f(x(t)) Numerically, this value can be approximated by taking many samples f(t) 1 M M f(x m (t)) m=1 Analytically, this corresponds to taking f(x(t)) = f(x)ρ(x, t)dx
Definition of stochastic differential equations (SDEs) A particle subject to pure Brownian motion, satisfies formally an SDE... dx(t) =dw (t), X(0) = X 0... which corresponds to saying that t X(t) =X 0 + dw (s) =X(0) + W (t) 0 We can add a drift term, which will make the expectation of X nonzero dx(t) =a(x) dt + bdw(t) This is a lot more to say about this type of equations, especially when the coefficients a and b depend on X. We will say some of these things in lecture 2... The Fokker-Planck equation A relation with a partial differential equation (PDE) Consider a stochastic differential equation dx(t) =a(x) dt + bdw(t) The probability distribution satisfies the Fokker-Planck equation ρ t + (a(x) ρ) x = b2 2 2 ρ x 2 This equation clearly shows that - the drift term in the SDE shows up as an advection term for the density - the Brownian motion in the SDE shows up as a diffusion term for the density
Example SDE and corresponding Fokker-Planck Heavy particle in a fluid The SDE for the heavy particle is (Langevin equation) dv = av dt + dw - Friction causes particle to slow down - Collisions with solvent molecules modeled as a Brownian motion of velocity The corresponding Fokker-Planck equation is ρ(v, t) t [avρ(v, t)] v = 1 2 2 ρ(v, t) v 2 - Density is Gaussian at all times if initial condition is Gaussian v(t) ρ(v, t) 2 1 0 1 2 3 1 0.75 0.5 0.25 0 0 2 4 6 8 10 t 5 2.5 0 2.5 5 v Application 1 : Complex fluids Micro-macro simulation of dilute polymer solutions Macroscopic part : Navier-Stokes equation for the solvent ρ u t + u u = η s u p +div(τ p ) div(u) =0 Coupling : non-newtonian stress (Kramers expression) τ p = nx F (X) nk B T Id Microscopic part : SDE for dissolved polymers dx + u Xdt = ux 2ζ F (X) dt + σ 2 ζ dw t,
Concrete model problem: FENE dumbbells One-dimensional case / Known velocity / Lagrangian framework / FENE force dx = ux 2ζ F (X) dt + σ 2 X dw t, F(X) = ζ 1 X 2 /b Consider a time-stepper for an ensemble of such polymers X k+1 = s X (X k, u(t),δt), X R N The ensemble represents a distribution of polymer configurations 0.4 0.3 δ = 0.50 δ = 1.00 Example: u(t) =2 ϕ( X ) 0.2 0.1 0 0 1 2 3 4 5 6 7 X - 50000 realizations - Polymer configuration distributions at two different moments in time Closure approximations Aim : Avoid computationally expensive micro/macro simulation Means : Derive a closed equation for a few macroscopic state variables Questions : U t U i = f i (X) + u U = H(U, u) τ p = T (U) - How to choose the macroscopic state variables? - How to determine the closure relations? Many closures have been proposed: FENE-L, FENE-S, FENE-P, FENE- LS, FENE-DS, { FENE-D }, quasi-equilibrium,...
Application 2 : Kinetic models Velocity-jump process for bacterial chemotaxis Each bacteria moves with a constant speed in a given direction. dx t dt = V t Direction changes at jump times, generated via a Poisson process. Tn+1 λ(t)dt = log(u n ), U n = U(0, 1) T n The modeling effort goes into the definition of the intensity of the Poisson process (turning rate). Turning rate for velocity jump processes Constant turning rate (leads to advection-diffusion) λ(t) λ 0 Turning rate biased by chemoattractant concentration λ(t) λ 0 g ( x S(X(t))) T V (t) - Direct sensing of chemoattractant gradient - Reduced turning rate if moving in favorable direction Turning rate biased by internal state (small bacteria) λ(t) =λ(y(t)), dy t dt = F (Y t,s(x t )) - Intensity of Poisson process depends on internal state - Evolution of internal state depends on the environment (memory! )
Example: cartoon internal dynamics Excitation/adaptation mechanism y Alignment with desired direction t λ(y) :=λ 0 by 2 λ(y) > λ 0 for y 2 < 0 λ Bacteria has a higher jump rate if it moves in an unfavorable direction. y 2 The nondimensionalized model Non-dimensionalize: t x : time scale associated with environmental changes t λ : typical time between two jumps t y : time scale associated with evolution of internal state := t λ /t x τ := t y /t λ dx t = V t, dt V t =1 V t = V n t [T n,t n+1 ) Tn+1 T n λ(y t )dt = θ n dy t dt = F (Y t,s(x t )) τ 1 <K
Kinetic description Equivalent continuum description of the same model in terms of the distribution p(x, v, y, t) Several ways to simulate this kinetic equation deterministically - direct discretization - computationally expensive due to high number of dimensions (x,v,y) stochastically t p + v x p + y (F (y, S(x))p) = Q(p) advection internal dynamics collisions - via an ensemble of individual velocity-jump processes - introduction of numerical noise (variance) due to finite number of particles reduction to an approximate macroscopic model Advection-diffusion limit for internal state model Define the density n(x, t) = Y In the limit 0, and scaling to long time scales t t/ 2, we get 1 t n = x 2 xn T (x)n The advection term depends on the details of the internal dynamics T (x) =b T V p(x, v, y, t)dvdy τ λ 0 τ + I xs(x) For nonzero, the difference of the density with respect to the limiting density is of order. Convergence is pathwise: trajectories converge to the corresponding SDE.
Model-scale separation : two types of problems Model problem 1 : approximation of Monte Carlo SDE simulation by moments - Microscopic description : Stochastic differential equation with fast evolution - Macroscopic level : Slowly evolving function of interest (expectation) Model problem 2 : approximation of kinetic equations on long time scales - Microscopic description: kinetic (phase-space) model - Macroscopic description: partial differential equation in space-time domain Later on in the course : - derivation of approximate macroscopic models in idealized situations (lecture 4) - numerical methods that exploit scale separation to simulate when deriving the macroscopic model analytically is not possible (second part of the course) Summary / Take-home message for today Problems with scale separation are everywhere. The multiscale nature can arise by having - multiple time scales - multiple spatial scales - multiple levels of description We have seen a number of applications and model problems for each. In the remainder of the course, we will look into - analytical techniques to obtain macroscopic models - conditions under which these can be derived - numerical techniques for when these conditions are not met