Simulation methods for stochastic models in chemistry

Transcription

1 Simulation methods for stochastic models in chemistry David F. Anderson Department of Mathematics University of Wisconsin - Madison SIAM: Barcelona June 4th, 21

2 Overview 1. Notation and terminology: biochemical reaction networks. 2. Discuss class of numerical approximation methods: leaping algorithms. 3. Question: how well do different methods approximate exact process? Scaling issues. 4. Discuss diffusion approximations Accuracy in approximating the jump process. Methods of approximating the diffusion approximation. Structural issues.

3 Stochastic models of (bio)chemical reactions Standard notation for chemical reactions: S 1 + S 2 S 3 is interpreted as a molecule of S 1 combines with a molecule of S 2 to give a molecule of S 3. We consider a network of reactions involving d chemical species, S 1,..., S d : dx dx ν ik S i i=1 i=1 ν iks i Each instance of the reaction S 1 + S 2 S 3 changes the state of the system by the vector: ν k ν k = =

4 Markov chain models The state of the system, X(t) Z d, gives the number of molecules of each species in the system at time t. ν k : number of molecules of each chemical species consumed in the kth reaction. ν k: number of molecules of each chemical species created in the kth reaction. If kth reaction occurs at time t, the new state becomes X(t) = X(t ) + ν k ν k. The waiting times for the reactions are exponentially distributed with rate (intensity/propensity) functions λ k : R d R.

5 Some examples Gene transcription & translation:. G G + M M M + P M P G + P BoundGene. transcription translation degradation degradation Cartoon representation: 1 1 J. Paulsson, Physics of Life Reviews, 2, Hye Won Kang, presentation at SPA in 27.

6 Some examples Gene transcription & translation:. G G + M M M + P M P G + P BoundGene. transcription translation degradation degradation Cartoon representation: 1 E. coli Heat Shock Response Model. 9 species, 18 reactions. 2 1 J. Paulsson, Physics of Life Reviews, 2, Hye Won Kang, presentation at SPA in 27.

7 Markov chain models This model is a continuous time Markov chain in Z d with generator (Af )(x) = X k λ k (x)(f (x + ν k ν k ) f (x)). Kolmogorov s forward equation ( Chemical Master Equation ) describes the evolution of the distribution of the state of the system d dt P(x, t µ) = X k λ k (x ν k + ν k )P(x ν k + ν k, t µ) X k λ k (x)p(x, t µ), where P(x, t µ) is probability X(t) = x conditioned on initial distribution µ.

8 Pathwise Representations Random time changes The number of times the kth reaction occurs by time t is a counting process with intensity λ k (X(t)) and can be written as Z t «R k (t) = Y k λ k (X(s))ds, where the Y k are independent, unit-rate Poisson process. Recall, Y k is a unit-rate Poisson process if Y k () =, Y k ( ) has independent increments, and Y k (u + v) Y k (v) has a Poisson distribution with parameter u for all u, v

9 This representation is equivalent to the chemical master equation (Kolmogorov s forward equation) representation found in biology/chemistry literature. Pathwise Representations Random time changes The number of times the kth reaction occurs by time t is a counting process with intensity λ k (X(t)) and can be written as Z t «R k (t) = Y k λ k (X(s))ds, where the Y k are independent, unit-rate Poisson process. Recall, Y k is a unit-rate Poisson process if Y k () =, Y k ( ) has independent increments, and Y k (u + v) Y k (v) has a Poisson distribution with parameter u for all u, v Therefore, a path-wise representation is given by random time-changes of Poisson processes X(t) = X() + X Z t «Y k λ k (X(s))ds (ν k ν k ). k

10 Mass-action kinetics The standard intensity function chosen is mass-action kinetics: λ k (x) = κ k ( Y! x Y x i! ν ik!) = κ k ν k (x i ν ik )!. i i Rate is proportional to the number of distinct subsets of the molecules present that can form inputs for the reaction. (this assumes vessel is well-stirred.) Example: If S 1 anything, then λ k (x) = κ k x 1. Example: If S 1 + S 2 anything, then λ k (x) = κ k x 1 x 2. Example: If S 1 + 2S 2 anything, then λ k (x) = κ k x 1 x 2 (x 2 1).

11 Methods of investigation: numerical simulation X(t) = X() + X k Y k Z t «λ k (X(s))ds (ν k ν k ), (GOOD NEWS) There are a number of numerical methods that produce statistically exact sample paths: 1. Gillespie s algorithm. 2. The first reaction method. 3. The next reaction method. For each step of these methods must find : (i) the amount of time that passes until the next reaction takes place: n. (the minimum of exponential RVs). (ii) which reaction takes place at that time.

12 Methods of investigation: numerical simulation X(t) = X() + X k Y k Z t «λ k (X(s))ds (ν k ν k ), (GOOD NEWS) There are a number of numerical methods that produce statistically exact sample paths: 1. Gillespie s algorithm. 2. The first reaction method. 3. The next reaction method. For each step of these methods must find : (i) the amount of time that passes until the next reaction takes place: n. (the minimum of exponential RVs). (ii) which reaction takes place at that time. (BAD NEWS) If P k λ k(x(t)) 1, then n 1 Pk λ k(x(t)) 1 and the time needed to produce a single exact sample path over an interval [, T ] can be prohibitive.

13 Tau-leaping Standard τ-leaping 3 was developed by Dan Gillespie in an effort to overcome the problem that n may be prohibitively small. Tau-leaping is essentially an Euler approximation of Z (τ) = Z () + X k Z () + X k d = Z () + X k Z t λ k (X(s))ds: Z τ «Y k λ k (Z (s)) ds (ν k ν k ) Y k λ k (Z ()) τ «(ν k ν k ) «Poisson λ k (Z ()) τ (ν k ν k ). 3 D. T. Gillespie, J. Chem. Phys., 115,

14 Tau-leaping Algorithm: For each time step do the following: 1. For each reaction, calculate P k,n = Poisson(λ k (Z (t n)) τ). 2. Update, Z (t n+1 ) = Z (t n) + P k P k,n (ν k ν k ). A pathwise representation for Z (t) generated by τ-leaping is Z (t) = X() + X k Y k Z t «λ k (Z η(s))ds (ν k ν k ), where η(s) = t n, if t n s < t n+1 = t n + τ is a step function giving left endpoints of time discretization.

15 An approximate midpoint method For a time discretization = t < t 1 < < t N = T, with τ = t n t n 1, let ρ(z) = z τ X k λ k (z)(ν k ν k ), be a deterministic midpoint approximation Algorithm: For each time step do the following: 1. Let yn = ρ(z(t n)). 2. For each reaction, calculate P k,n = Poisson(λ k (yn ) τ). 3. Update, Z(t n+1 ) = Z(t n) + P k P k,n (ν k ν k ). Note: still just require one random variable per reaction per step.

16 An approximate midpoint method For a time discretization = t < t 1 < < t N = T, with τ = t n t n 1, let ρ(z) = z τ X k λ k (z)(ν k ν k ), be a deterministic midpoint approximation Algorithm: For each time step do the following: 1. Let yn = ρ(z(t n)). 2. For each reaction, calculate P k,n = Poisson(λ k (yn ) τ). 3. Update, Z(t n+1 ) = Z(t n) + P k P k,n (ν k ν k ). Note: still just require one random variable per reaction per step. Then Z(t) solves: Z(t) = X() + X k Y k Z t «λ k ( ρ(z η(s)) )ds (ν k ν k ), where η(s) = t n, if t n s < t n+1 = t n + τ.

17 Previous error analysis Under the scaling τ : 1. Rathinam, Petzold, Cao, and Gillespie 4 showed, among other things (implicit methods, etc.), that tau-leaping is first order accurate in a weak sense if the intensity functions λ k are linear. 2. Li 5 extended this result by showing that standard tau-leaping has a strong error (in the L 2 norm) of order 1/2 and a weak error of order one: q E Z (t n) X(t n) 2 Cτ 1/2 sup n N Ef (Z (T )) Ef (X(T )) Cτ, where = t < t 1 < < t N = T is a partition of [, T ]. 3. The midpoint method is no more accurate than standard tau-leaping 4 M. Rathinam et al., SIAM Multi. Model. Simul., 4, 25, T. Li, SIAM Multi. Model. Simul., 6, 27,

18 Example Consider A as a prey and B as a predator. A κ 1 2A, A + B κ 2 2B, B κ 3, with A() = B() = 1 and κ 1 = 2, κ 2 =.2, κ 3 = 2. Letting τ = 1/2 and simulating 3, sample paths with each method yields the following approximate distributions for B(1): Gillespie Algorithm Standard Tau Leaping Midpoint Tau Leaping

19 Problem with the τ scaling Recall, tau-leaping methods are only useful if τ n, for otherwise an exact method would be performed. Therefore, we should require that 1 τ P k λ k(x(t)) n while X λ k (X(t)) 1. k So τ does not seem to be an appropriate scaling to describe the accuracy of these methods.

20 What is an appropriate scaling? I can not answer in generality. It will depend upon how the system satisfies X λ k (X(t)) 1 k Multiple possibilities: 1. Could be that X i = O(V α i ) for V 1 with κ k = O(1). 2. Could be that κ k = O(V β k ) for V 1 with X i = O(1). 3. Could be multiple scales. 4. Infinite number of possibilities.

21 The classical scaling We will convert from numbers of molecules, X V, to concentrations: (i) Let V = volume Avogadro s number. (ii) Set X V = X V /V.

22 The classical scaling We will convert from numbers of molecules, X V, to concentrations: (i) Let V = volume Avogadro s number. (ii) Set X V = X V /V. (iii) Assume X V () = x V. (iv) Units of X V are moles per liter; rate constants must be scaled to write sensible equations: Reaction type S 1 S 1 + S 2, S 1 + S 2 + S 3 Rate constant κ k = c k V κ k = c k κ k = c k /V κ k = c k /V 2

23 The classical scaling For each k and x R d : λ k (X V ) = λ k (V X V ) = V λ k (X V ), where λ k is deterministic mass action kinetics with rate constants c k. For example, S 1 + S 2 S 3 : λ k (X V ) = κ k X V 1 X V 2 = c k V V X V 1 V X V 2 = Vc k X V 1 X V 2.

24 The classical scaling For each k and x R d : λ k (X V ) = λ k (V X V ) = V λ k (X V ), where λ k is deterministic mass action kinetics with rate constants c k. For example, S 1 + S 2 S 3 : λ k (X V ) = κ k X V 1 X V 2 = c k V V X V 1 V X V 2 = Vc k X V 1 X V 2. We had X V (t) = X V () + X k Y k Z t «λ k (X V (s))ds (ν k ν k ) Converting now gives us X V (t) = X V () + X k Z 1 t «V Y k V λ k (X V (s))ds (ν k ν k )

25 The classical scaling For each k and x R d : λ k (X V ) = λ k (V X V ) = V λ k (X V ), where λ k is deterministic mass action kinetics with rate constants c k. For example, S 1 + S 2 S 3 : λ k (X V ) = κ k X V 1 X V 2 = c k V V X V 1 V X V 2 = Vc k X V 1 X V 2. We had X V (t) = X V () + X k Y k Z t «λ k (X V (s))ds (ν k ν k ) Converting now gives us X V (t) = X V () + X k Z 1 t «V Y k V λ k (X V (s))ds (ν k ν k ) X V ( ) V c( ), which satisfies deterministic mass action kinetics.

26 Important to recognize 1. It is clear that the choice of scaling laid out above will not explicitly cover all cases of interest: i. Multiple scales ii. Low copy numbers of constituent molecules. 2. However, the purpose is to give a more accurate picture of how tau-leaping methods approximate the exact solution, both strongly and weakly, in at least one plausible setting.

27 Time-step Q: Given a system satisfying our scaling, what size τ would a user choose?

28 Time-step Q: Given a system satisfying our scaling, what size τ would a user choose? Partial answer: something small, τ < 1.

29 Time-step Q: Given a system satisfying our scaling, what size τ would a user choose? Partial answer: something small, τ < 1. Complete answer: something not too small «1 1 τ Pk λ k(x(t)) n = O. V

30 Time-step Q: Given a system satisfying our scaling, what size τ would a user choose? Partial answer: something small, τ < 1. Complete answer: something not too small «1 1 τ Pk λ k(x(t)) n = O. V That is, for some β (, 1), τ = 1 V β.

31 Error analysis on scaled processes We consider the scaled processes X V (t) = def X V (t) V, Z V (t) = def Z V (t) V, ZV (t) = def ZV (t) V. which satisfy X V (t) = X V () + 1 X Z t «Y k V λ k (X V (s))ds (ν k ν k ) V k Z V (t) = X V () + 1 X Z t «Y k V λ k (Z V η(s))ds (ν k ν k ) V k Z V (t) = X V () + 1 X Z t «Y k V λ k ρ V (Z V η(s))ds (ν k ν k ), V k where ρ V (z) = def z τ X k λ k (z)(ν k ν k ), and consider the relationship of the normalized approximate processes, Z V (t) and Z V (t), to the original process X V (t), normalized similarly.

32 Exact asymptotics for Euler tau-leaping Theorem (Anderson, Ganguly, Kurtz 21) 1 Let X V (t) and Z V (t) denote the exact and Euler tau-leap solutions for t [, T ]. Then as V V β (X V Z V ) E, where E(t) is deterministic and independent of V and β. And where f V f means strong path-wise convergence: lim P{sup f V (t) f (t) > ɛ} =, for all ɛ > V t T lim E sup f V (t) f (t) =. V t T Interpret as Z V = X V + E V β + H.O.T. 1 To appear in Annals of Applied Probability

33 Idea of proof Can show: X V (t) Z V (t) M V (t) + + V β 1 2 Z t Z t DF V (Z V η(s))(x V (s) Z V (s))ds DF V (Z V η(s))f V (Z V η(s))ds where M V (t) is a martingale and V (1+β)/2 M V M, where M is a mean-zero Gaussian process. (1 + β)/2 > β, so done.

34 Weak error of Euler tau-leaping Theorem (Anderson, Ganguly, Kurtz 21) 1 Let X V (t) and Z V (t) be as before. Then, for any continuously differentiable function f lim V β Ef (X V (t)) Ef (Z V (t)) = E(t) f (x(t)). V Interpret as Ef (Z V (t)) = Ef (X V (t)) + 1 E(t) f (x(t)) + H.O.T. V β 1 To appear in Annals of Applied Probability

35 Exact asymptotics for midpoint tau-leaping Theorem (Anderson, Ganguly, Kurtz 21) 1 Let X V (t) and Z V (t) denote the exact and midpoint tau-leap solutions for t [, T ]. Then as V V 2β (X V Z V ) E 1, if β < 1/3. V 2β (X V Z V ) E 2, if β = 1/3. V (1+β)/2 (X V Z V ) E 3, if β > 1/3. Where E 1 is deterministic and E 2, E 3 are stochastic. 1.8 β min(2β, (β+1) / 2) β

36 Idea of proof Can show: X V (t) Z V (t) M V (t) + Z t DF V (Z V η(s))(x V (s) Z V (s))ds + V 2β H V (t) where V (1+β)/2 M V M(t) and H V H.

37 Weak error of midpoint tau-leaping Theorem (Anderson, Ganguly, Kurtz 21) 1 Let X V (t) and Z V (t) be as before. Then, for any C 3 function f, there exists a constant C = C(f, T ) > such that V 2β sup Ef (X V (t)) Ef (Z V (t)) C. t T 1 To appear in Annals of Applied Probability

38 Example Consider A as a prey and B as a predator. A κ 1 2A, A + B κ 2 2B, B κ 3, with A() = B() = 1 and κ 1 = 2, κ 2 =.2, κ 3 = 2. V = 1, and τ = 1/2 = 1/V.434 (so β =.434). Simulating 3, sample paths yields the following approximate distributions for B(1): Gillespie Algorithm Standard Tau Leaping Midpoint Tau Leaping

39 Diffusion Approximation Under the classical scaling, can also use a diffusion approximation. Let Ỹ k (u) = def Y k (u) u, denote the centered Poisson process. Then, X V (t) satisfies Z t X V (t) = X V () + X λ k (X V (s))ds(ν k ν k ) k + X Z 1 t V Ỹk(V λ k (X V (s))ds)(ν k ν k ). k

40 Diffusion Approximation Under the classical scaling, can also use a diffusion approximation. Let Ỹ k (u) = def Y k (u) u, denote the centered Poisson process. Then, X V (t) satisfies Z t X V (t) = X V () + X λ k (X V (s))ds(ν k ν k ) k + X Z 1 t V Ỹk(V λ k (X V (s))ds)(ν k ν k ). k Using functional central limit theorem: Ỹk(V )/ V W k ( ), yields Z t X V (t) X V () + X λ k (X V (s))ds(ν k ν k ) k + X Z 1 t W k ( λ k (X V (s))ds)(ν k ν k ). k V

41 Diffusion Approximation This suggests a good approximation to X V will be the solution to the SDE Z t Z V (t) = X V () + X λ k (Z V (s))ds (ν k ν k ) k + 1 X Z t W k ( λ k (Z V (s))ds)(ν k ν k ), V k where the W k are independent Brownian motions.

42 Diffusion Approximation This suggests a good approximation to X V will be the solution to the SDE Z t Z V (t) = X V () + X λ k (Z V (s))ds (ν k ν k ) k + 1 X Z t W k ( λ k (Z V (s))ds)(ν k ν k ), V k where the W k are independent Brownian motions. An equivalent formulation is given by the Itô equations dz V (t) = X k λ k (Z V (t)) (ν k ν k ) dt + 1 X q (ν k ν k ) λ k (Z V (t)) dw k (t), V k (1) System (1) is termed the chemical Langevin equation in the biology/chemistry literature.

43 Error of Diffusion Approximation Accuracy: «log(v ) Pathwise 1, the error is O. V Pathwise, tau-leaping methods can be more or less accurate, depending upon choice of β and V. V Euler τ-leap Step-size Midpoint τ leap Step-size more accurate if (Euler τ) more accurate if (mdpt τ) 1 β > β >.3368 τ <.212 1, β > β >.444 τ <.477 1, β > β >.5178 τ <.85 1, β > β >.5756 τ <.13 1 T. Kurtz, SPA, 6, 1978,

44 Approximating the diffusion approximation If A is the generator for the jump process B is the generator for the diffusion. «1 Af = Bf + O. V 2 Suggests that for fixed t, weak error is O(1/V 2 ). For fixed t, even midpoint τ leap will never be as accurate. However, now you must approximate the diffusion approximation.

45 Approximating SDES with special structure Consider SDEs of the form dx(t) = b(x(t))dt + X k σ k (X(t)) ν k dw k (t), where b : R d R d, σ k : R d R, ν k R d, and {W k (t) : k = 1,..., m} are independent, one-dimensional Wiener processes. Thus, randomness is entering the system in fixed directions ν k, but at variable rates σ k (X(t)). 2 T. Kurtz, Annals of Probability, 8(4), 198,

46 Approximating SDES with special structure Consider SDEs of the form dx(t) = b(x(t))dt + X k σ k (X(t)) ν k dw k (t), where b : R d R d, σ k : R d R, ν k R d, and {W k (t) : k = 1,..., m} are independent, one-dimensional Wiener processes. Thus, randomness is entering the system in fixed directions ν k, but at variable rates σ k (X(t)). In fact, all uniformly elliptic SDEs can be represented in such a way 2. 2 T. Kurtz, Annals of Probability, 8(4), 198,

47 Approximating SDES with special structure Let approximate path be Y i = Y (t i ). Algorithm 1 Fixing a θ (, 1), we define α 1 def = θ(1 θ) and def α 2 = 1 (1 θ) 2 + θ 2. (2) 2 θ(1 θ) Fixing h, we first compute a θ-midpoint y and then the new value Y i : (η (i) jk ind. normals) are Step 1. Set Step 2. Set y = Y i 1 + b(y i 1 )θh + X k σ k (Y i 1 ) ν k η (i) 1k θh Y i = y + (α 1 b(y ) α 2 b(y i 1 ))(1 θ)h + X k qˆα1 σ 2 k (y ) α 2 σ 2 k (Y i 1) + ν k η (i) 2k p (1 θ)h. 1 D. F. Anderson and J. C. Mattingly, to appear in Comm. Math. Sci., 21

48 Order of approximation Theorem (One-step approximation 1 ) If b, σ k C 6 with inf xσ k (x) >, then for h sufficiently small and f C 6 : E xf (X(h)) E xf (Y 1 ) K f 6 h 3. 1 D. F. Anderson and J. C. Mattingly, to appear in Comm. Math. Sci., 21

49 Order of approximation Theorem (One-step approximation 1 ) If b, σ k C 6 with inf xσ k (x) >, then for h sufficiently small and f C 6 : E xf (X(h)) E xf (Y 1 ) K f 6 h 3. Theorem (Global approximation 1 ) Under the same assumptions for any T > : sup E xf (X(nh)) E xf (Y n) C(T ) f 6 h 2. n T /h 1 D. F. Anderson and J. C. Mattingly, to appear in Comm. Math. Sci., 21

50 Why this works System dx(t) = b(x(t))dt + X k σ k (X(t)) ν k dw k (t), is equivalent to X(t) = X() + Z t b(x(s))ds + X k=1 ν k Z Z t 1 [,σ 2 k (X(s))) (u) W k (ds du), where the W k are independent, space-time white noise processes: Random measure such that: 1. for A R 2, W k (A) Normal(, Area(A)). 2. A, B R 2, with A B = W k (A) and W k (B) are independent. Challenge is in approximating diffusion term.

51 Why this works: θ = 1/2 σ 2 k (X()) σ 2 k (X(t)) σ 2 k (y ) Region 1

52 Why this works: θ = 1/2 σ 2 k (X()) σ 2 k (X(t)) Region 3 σ 2 k (X(t)) σ 2 k (y ) Region 1 Region 2

53 Why this works: θ = 1/2 σ 2 k (X()) σ 2 k (X(t)) Region 3 σ 2 k (X(t)) σ 2 k (y ) Region 1 Region 2 σ 2 k (X(t)) Region 3 V 2 V Region 4 Region 5

54 Example Consider the following system» dx1 (t) dx 2 (t) =» X2 (t) X 1 (t) dt + + s sin 2 (X 1 (t) + X 2 (t)) + 6 t + 1 r cos2 (X 1 (t) + X 2 (t)) + 6 t + 1 where W 1 (t) and W 2 (t) are independent Wiener processes.» 1» 1 dw 1 (t) dw 2 (t). It is simple to show that E X(t) 2 = EX() log(1 + t). (3)

55 Example 2 1 log error Weak Trapezoidal Euler Midpoint drift log h Figure: Log-log plots. Approximating E X(1) 2. The best fit lines have slopes 2.223,.952, and 1.98, for the Weak Trapezoidal Algorithm, Euler s method, and the midpoint drift method, respectively.

56 % of degenerate steps θ log errror θ =.5 θ =.25 θ =.5 θ = log h Figure: (a) # degenerate steps, first example, h = 1/1, (b) slopes 1.865, 1.996, 2.29, and 2.33 for θ =.5,.25,.5,.75, respectively. For the model, dx(t) = p X(t) 2 + 1dW (t), X() = 1

57 Outlook 1. These methods will rarely be used in isolation. 2. Need to begin to combine them into hybrid strategies 2.1 Some pieces of the network (species and/or reactions) are treated discretely. 2.2 Some treated diffusively. 2.3 some treated as absolutely continuous (ODE approximation depending on stochastic parts). 3. And, want to be able to dynamically change the treatment.

58 Outlook 1. These methods will rarely be used in isolation. 2. Need to begin to combine them into hybrid strategies 2.1 Some pieces of the network (species and/or reactions) are treated discretely. 2.2 Some treated diffusively. 2.3 some treated as absolutely continuous (ODE approximation depending on stochastic parts). 3. And, want to be able to dynamically change the treatment. Work has begun on algorithms: 1. E. Haseltine and J. Rawlings or Salis & Kaznessis: hybrid discrete/langevin. 2. L. Harris, K. Iyengar, P. Clancy, M. Picirilli, E. Majusiak: Partitioned leaping algorithm.