Gillespie s Algorithm and its Approximations. Des Higham Department of Mathematics and Statistics University of Strathclyde

Transcription

1 Gillespie s Algorithm and its Approximations Des Higham Department of Mathematics and Statistics University of Strathclyde djh@maths.strath.ac.uk

2 The Three Lectures 1 Gillespie s algorithm and its relation to SDEs 2 Tau-leaping and multilevel approximations to Gillespie 3 Monte Carlo/SDEs in mathematical finance Vienna Des Higham Gillespie 2 / 3

3 Overview of this lecture Gillespie Chemical Master Equation Chemical Langevin Equation Thermodynamic limit Mean hitting times Eric Renshaw (Strathclyde): Watch out for the boundaries! Vienna Des Higham Gillespie 3 / 3

4 Chemical Kinetics We have some chemical species undergoing some chemical reactions Not discussing molecular dynamics: not willing or able to measure initial location and velocity of every molecule, and/or not able to compute interactions between every pair of molecules Question of interest: How does the level of each species change over time? Vienna Des Higham Gillespie 4 / 3

5 Random? Inherently stochastic? Vienna Des Higham Gillespie 5 / 3

6 Random? Inherently stochastic? NO Vienna Des Higham Gillespie 5 / 3

7 Random? Inherently stochastic? Effectively stochastic? NO Vienna Des Higham Gillespie 5 / 3

8 Random? Inherently stochastic? Effectively stochastic? NO YES Vienna Des Higham Gillespie 5 / 3

9 Random? Inherently stochastic? Effectively stochastic? NO YES Justification for stochastic modelling... Vienna Des Higham Gillespie 5 / 3

10 Protein Toggle Model is from Gene regulatory networks: A coarse-grained equation-free approach to multiscale computation, by Erban, Kevrekidis, Adalsteinsson and Elston, Journal of Chemical Physics, 124, 200. Mutually repressive proteins: mass action (ODE) dz 1 dt = κz 1 ( γ 1 + ωz 2 2 δz 1 ) For certain realistic parameter values this ODE has two linearly stable steady states: one has z 1 (t) z a 481 and the other z 1 (t) z b Unrealistic that the cell s fate is completely specified by the initial condition, and a cell cannot switch dynamically between states. Stochastic simulation (Gillespie) gives... Vienna Des Higham Gillespie / 3

11 Protein Toggle Vienna Des Higham Gillespie / 3

12 Pure decay: S c=1, initially 10 molecules CME CLE RRE Vienna Des Higham Gillespie 8 / 3

13 Pure decay: S c=1, initially 100 molecules CME CLE RRE Vienna Des Higham Gillespie 9 / 3

14 S c=1, initially K molecules CME: p i (t) is prob. of having i molecules at time t d dt p i(t) = (i + 1)p i+1 (t) ip i (t), 0 i K. Process X(t) has E[X(t)] = Ke t and Var[X(t)] = Ke t (1 e t ) For the CLE, process is an SDE dy(t) = Y(t)dt Y(t) dw(t). This has same mean and variance as X(t) For the RRE, process is an ODE with solution Ke t Vienna Des Higham Gillespie 10 / 3

15 Chemical Kinetics (Gillespie 19, 2000) Well-stirred, thermal equilibrium, fixed volume Chemical Master Equation (CME) Species have integer values ODE for prob. of every possible state at time t Chemical Langevin Equation (CLE) Species have real values SDE for concentration of each species at time t Reaction Rate Equations (RRE) Species have real values ODE for concentration of each species at time t Vienna Des Higham Gillespie 11 / 3

16 Chemical Kinetics (Gillespie 19) N chemical species, M types of reaction (e.g. A + B C) State vector X 1 (t) X 2 (t) X(t) =., X(0) = X 0 X N (t) Each reaction, 1 j M, is described by a stoichiometric vector ν j R N such that X(t) X(t) + ν j, a propensity function a j (X(t)) such that the prob. of this reaction taking place over time [t, t + dt) is a j (X(t)) dt Vienna Des Higham Gillespie 12 / 3

17 Chemical Master Equation Discrete state space, continuous time Markov chain. Let P(x, t) be the prob. that X(t) = x dp(x, t) dt = M (a j (x ν j )P(x ν j, t) a j (x)p(x, t)) j=1 (Forward Kolmogorov equation) Gillespie s stochastic simulation algorithm gives a way to compute realisations of (t, X(t)) Also called Bortz-Kalos-Lebowitz algorithm, discrete event simulation, dynamic Monte Carlo, kinetic Monte Carlo,... Takes account of every reaction expensive Vienna Des Higham Gillespie 13 / 3

18 Chemical Langevin Equation Poisson with large mean normal SDE in R N dy(t) = M ν j a j (Y(t))dt + j=1 M ν j a j (Y(t)) dw j (t) Euler Maruyama computes approximate realisations of (t, Y(t)). j=1 (Switching off the noise gives the Reaction Rate Equation) Vienna Des Higham Gillespie 14 / 3

19 Remarks: the good The CLE is a rare example where an SDE arises naturally from fundamental modelling arguments The CLE captures the fluctuations of Poisson processes about their mean accurate when the means are large The CLE gets the first and second moments exactly right for first order reaction networks We can develop hybrid/multi-scale models that combine discrete and continuous regimes In the thermodynamic limit, over a finite time [0, T], the CLE and CME converge to the deterministic RRE; but the CLE and CME are much closer to each other than to the RRE (Kurtz, 190s) Vienna Des Higham Gillespie 15 / 3

20 Remarks: the bad Looking at the modelling/approximation arguments, we see that the CLE does not make sense for small population sizes Consequently, in general the CLE solution only exists up to a stopping time In particular, we can t generally talk about the steady state of the CLE Vienna Des Higham Gillespie 1 / 3

21 Reversible Isometry c 1 X 1 X 1 X 2 c 2 X 2 X 1 (0) + X 2 (0) = V Gillespie, The chemical Langevin and Fokker Planck equations for the reversible isomerization reaction, J. Phys. Chem. A, 2002 For c 1 = c 2, CME gives binomial steady state for X 1 CLE for X 1 : dy = ( c 1 Y + c 2 (V Y)) dt c 1 YdW 1 + c 2 (V Y)dW 2 Gillespie solves the steady Fokker-Planck equation and argues that Y has a Gaussian steady state with the correct mean and variance Vienna Des Higham Gillespie 1 / 3

22 c 1 = c 2 steady state distributions? Vienna Des Higham Gillespie 18 / 3

23 Reflect on this? Recall that V denotes the number of molecules If we add reflecting boundary conditions to the CLE then we obtain a Gaussian shaped steady state, restricted to the interval [0, V] This reflected CLE steady state has the correct mean, and the variance matches the CME up to O(V 2 e V ) Vienna Des Higham Gillespie 19 / 3

24 Reflected CLE Let q( ) be the steady state probability for the CME Let p( ) be the steady state probability for the reflected CLE We can show that Also p(0) 2 q(0) = p(v/2) q(v/2) = 1 + O(1/V) 1 Vπ e( 2 + log e 2)V ( 1 + O For V = 50, we have log Vπ e( 2 e 2)V ( )) e V which agrees with Gillespie s figure (similarly for V = 500) Vienna Des Higham Gillespie 20 / 3

25 Modifying the CLE? Wilkie & Wong Positivity preserving chemical Langevin equations, Chem. Phys., 2008 Idea: delete the offending diffusion coefficients in dy 1 = ( c 1 Y 1 + c 2 Y 2 ) dt c 1 Y 1 dw 1 + c 2 Y 2 dw 2 dy 2 = (c 1 Y 1 c 2 Y 2 ) dt + c 1 Y 1 dw 1 c 2 Y 2 dw 2 But this removes all the approximation power... Vienna Des Higham Gillespie 21 / 3

26 Modifying the CLE? 1 15 V = 1 Master Equation Modified Langevin Mass Action ODE 1 15 V = 10 Master Equation Modified Langevin Mass Action ODE <X 1 2 > 11 <X 1 2 > t t Vienna Des Higham Gillespie 22 / 3

27 Focus now on mean hitting time b x a T(x) := inf (t : Z(t) = a or Z(t) = b, given Z(0) = x) Vienna Des Higham Gillespie 23 / 3

28 Mean hitting time E[T(x)] a x b Vienna Des Higham Gillespie 24 / 3

29 Markov jump/birth & death process, Z(t) Discrete states {0, 1, 2,...,M}, with 0 and M absorbing: P (Z(t + h) = i + 1 Z(t) = i) = B i h + o(h) P (Z(t + h) = i 1 Z(t) = i) = D i h + o(h) P (Z(t + h) = i Z(t) = i) = 1 (B i + D i )h + o(h) Here, B 0 = D 0 = B M = D M = 0 Starting at state Z(0) = j, the expected time to be absorbed into state 0 or M is given by U j, where 2 4 (B 1 + D 1 ) B 1 D 2 (B 2 + D 2 ) B BM 2 D M 1 (B M 1 + D M 1 ) U 1 U 2 U U M = Vienna Des Higham Gillespie 25 / 3

30 Numerical Analysis Viewpoint Linear system can be written, for 1 i M 1, B i + D i 2 (U i+1 2U i + U i 1 ) + (B i D i ) U i+1 U i 1 2 Standard finite differences on the 2 point BVP ODE B(x) + D(x) u (x) + (B(x) D(x)) u (x) = 1 2 with u(a) = u(b) = 0 Here b a = M and we have x = 1 Interesting regime is M = 1 Vienna Des Higham Gillespie 2 / 3

31 Diffusion Approximation SDE: dy(t) = (B(y(t)) D(y(t))) dt + B(y(t)) dw 1 (t) D(y(t)) dw 2 (t) Let w(x) := E[T(x)] be the expected first time to hit a or b, given that y(0) = x Then w(x) satisfies the same 2 point BVP ODE Want to show that this ODE converges to the finite difference scheme Focus on specific examples... Vienna Des Higham Gillespie 2 / 3

32 Production from a source k X B i = k and D i = 0 Mean hitting times: Jump process b x k Diffusion process [ ( 1 e 2x + e 2a e 2b ( e 2b e 2x) ) + b x k e 2b + e 2a 2 ) + (1 e 2x + e (a 2a x + e 2a ( e 2x e 2a))] e 2b + e 2a 2 Vienna Des Higham Gillespie 28 / 3

33 Convergence: fix a = 0 and let b With x = αb for fixed α (0, 1), we have b min{2(1 α),α} Jump Diffusion C e where C is independent of b Example: k = 5, a = 0, α = 1 2 : Absolute Difference in Exit Time b Vienna Des Higham Gillespie 29 / 3

34 Production c X X B i = i and D i = 0 Mean hitting times: Jump process Diffusion process 1 c b 1 s=x ( 1 e 2x e 2a b ( e 2b c e 2b e 2a x + [1 e 2x e 2a e 2b e 2a 1 s e 2l ) dl + ln b ln x l x ](ln a ln x + e 2a a e 2l l )) dl Vienna Des Higham Gillespie 30 / 3

35 Convergence With x = αb for fixed α (0, 1), we have Jump (with a = 0) lim aց0 Diffusion Cb 2 where C is independent of b Proof Uses the expansions n 1 s = ln n + γ + 1 2n + O(n 2 ), as n, s=1 x e t t dt = ln x + γ + o(1), as x 0, where the Euler-Mascheroni constant γ = , and x ( e t dt = ex ) t x x + O(x 2 ), as x Vienna Des Higham Gillespie 31 / 3

36 Example, c = 5, a = 10 3, α = Absolute Difference in Exit Time b Vienna Des Higham Gillespie 32 / 3

37 Degradation X c X B i = 0 and D i = i Mean hitting times: Jump process Diffusion process ( e 2x e 2a 1 b (e 2b e 2b e 2a c + [1 e2x e 2a e 2b e 2a] x ( 1 c 1 c x s=a+1 1 s )) e 2l dl ln b + ln x l x (ln x ln a e 2a a e 2l l )) dl Vienna Des Higham Gillespie 33 / 3

38 Convergence With x = αb for fixed α (0, 1), we have lim lim ln 2 (Jump Diffusion) = aց0 b c Proof Uses asymptotic expansions for E 1 (x) = e t x t dt, x > 0 Note: the actual hitting times grow like ln(b), so we have relative convergence like O(1/ ln b) Vienna Des Higham Gillespie 34 / 3

39 Example, c = 5, α = 1 2, a = 10 2, 10 4, Absolute Difference in Exit Time a=1e 2 a=1e 4 a=1e b ln 2 5 = Vienna Des Higham Gillespie 35 / 3

40 Key Points Three modeling regimes: CME, CLE, RRE CME is most accurate, but expensive Thermodynamic limit can help to justify the use of CLE and RRE, but must be used with care Finding cheaper alternatives to full Gillespie/CME simulations is a big area We will look at tau-leaping (Euler approximation) multi-level Monte Carlo Vienna Des Higham Gillespie 3 / 3