Stochastic Models for Chemically Reacting Systems Using Polynomial Stochastic Hybrid Systems

Transcription

1 Stohasti Models for Chemially Reating Systems Using Polynomial Stohasti Hybrid Systems João Pedro Hespanha and Abhyudai Singh Center for Control Engineering and Computation University of California, Santa Barbara, CA 93 Abstrat. A stohasti model for hemial reations is presented, whih represents the population of various speies involved in a hemial reation as the ontinuous state of a polynomial Stohasti Hybrid System (pshs). pshss orrespond to stohasti hybrid systems with polynomial ontinuous vetor fields, reset maps, and transition intensities. We show that for pshss, the dynamis of the statistial moments of its ontinuous states, evolves aording to infinite-dimensional linear ordinary differential equations (ODEs), whih an be approximated by finite-dimensional nonlinear ODEs with arbitrary preision. Based on this result, a proedure to build this types of approximation is provided. This proedure is used to onstrut approximate stohasti models for a variety of hemial reations that have appeared in literature. These reations inlude a simple bimoleular reation, for whih one an solve the master equation; a deaying-dimerizing reation set whih exhibits two distint time sales; a reation for whih the hemial rate equations have a ontinuum of equilibrium points; and the bistable Shögl reation. The auray of the approximate models is investigated by omparing with Monte Carlo simulations or the solution to the Master equation, when available. Introdution The time evolution of a spatially homogeneous mixture of hemially reating moleules is often modeled using a stohasti formulation, whih takes into aount the inherent randomness of thermal moleular motion. This formulation is superior to the traditional deterministi formulation of hemial kinetis and is motivated by omplex reations inside living ells, where small populations of key reatants an set the stage for signifiant stohasti effets [ 3. Although most of the mathematial modeling of geneti networks represents gene expression and regulation as deterministi proesses, reent observations of gene expression in individual ells illustrate the stohasti nature of transription [4, 5. Furthermore, studies of engineered geneti iruits designed to at as toggle swithes or osillators have revealed large stohasti effets [, 6. Stohastially is therefore an inherent feature of biologial dynamis and developing stohasti models whih apture this stohastially have beome inreasingly important. In the stohasti formulation, the time evolution of the system is desribed by a single equation for a grand probability funtion, where time and speies populations appear as independent variables, alled the Master equation [7. However, this equation an only be solved for relatively few, highly idealized ases and generally Monte Carlo simulation tehniques are used whih are also known as the Stohasti Simulation Algorithm (SSA) [8. Sine one is often interested in only the first and seond order moments for the number of moleules of the different speies involved, muh effort an be saved by applying approximate methods to produe these low-order moments, without atually having to solve for the probability density funtion. Various suh approximate methods have been developed, for example, using the Fokker-Plank approximation, expanding the Master equation, et. [7. In this paper, an alternative approximate method for This version of the paper differs from the one that appeared in print in that we have orreted a few typos in Example 3. This material is based upon work supported by the National Siene Foundation under Grants No. CCR-384, ECS-4798.

2 estimating lower-order moments is introdued, by representing the dynamis of a hemial reation as a Stohasti Hybrid System (SHS). Hybrid systems are haraterized by a state-spae that an be partitioned into a ontinuous domain (typially R n ) and a disrete set (typially finite). In Stohasti Hybrid systems (SHS), both omponents of the state are stohasti proesses, with the evolution of the ontinuous-state determined by a stohasti differential equation (SDE) and a transition or reset map. Eah time a transition is ativated, the SHS s state is updated aording to the reset map. The rate at whih these ativations our, may depend on the overall state. A formal definition for this SHS model, whih was taken from [, an be found in Se.. Polynomial stohasti hybrid systems (pshss) arise when the ontinuous vetor fields in the SDE, the reset maps, and the transition intensities are all polynomial funtions of the ontinuous state. In this paper we show that the evolution of the populations of several speies involved in a set of hemial reations an be modelled by a SHS. Eah reation orresponds to a reset map defined by its stoihiometry, whih is ativated at a rate determined by the reation rate. In fat, we show in Se. that moleular reations an atually be modelled by pshss. An important property of pshss is that, if one reates an infinite vetor ontaining the probabilities of all the statistial moments of the ontinuous state, the dynamis of this vetor are governed by an infinite-dimensional linear ordinary differential equation (ODE), whih we all the infinite-dimensional moment dynamis and define formally in Se. 3. In general, the infinite-dimensional linear ODEs that desribe the moment dynamis for pshss are still not easy to solve analytially, and solving them would be essentially equivalent to solving the Master equation. However, they an be aurately approximated by a finite-dimensional nonlinear ODE, whih we all the trunated moment dynamis. The state of this trunated moment dynamis typially ontains the lowerorder moments of interest. We show in Se. 3 that, under suitable stability assumptions, it is in priniple possible for a finite-dimensional nonlinear ODE to approximate the infinite-dimensional moment dynamis, up to an error that an be made arbitrarily small. A proedure to atually onstrut these finite-dimensional approximations is outlined in Se. 4. These finite-dimensional ODE s provide time evolution of lower order moments for populations of speies involved in a hemial reation. Apart from providing fast simulation times and lesser omputation burden ompared to Monte Carlo simulations these approximate models also open the doors to other types of analysis tools, for example, sensitivity analysis of hemial master equation [3. However, they provide lesser information about the probability distribution as ompared to Monte Carlo simulations, for example, these approximate models do not provide information about time orrelations. To illustrate the appliability of the results we onsider several systems that have appeared in the literature. For eah example, we onstrut in Se. 5 trunated moment dynamis and evaluate how they ompare with estimates for the moments obtained from a large number of Monte Carlo simulations whih are arried using the SSA. The examples onsidered here are as follows:. A simple bimoleular reation, for whih one an atually ompute the first and seond order moments diretly from the Master equation [4. The exat results are ompared to their estimates from the trunated model.. A deaying-dimerizing reation set [9, 5. This reation is diffiult to simulate due to the existene of two very distint time sales and methods that do not require Monte Carlo simulations are of speial interest. 3. A reation for whih the deterministi hemial rate equations have a ontinuum of equilibrium points. This reation was seleted as an example beause the approximate methods developed in [7, are not appliable. 4. The Shögl reation [6. This reation is partiularly interesting as the deterministi hemial rate equations are bistable, i.e., have two stable equilibrium points for partiularly hosen reation rates.

3 Polynomial Stohasti Hybrid Systems A SHS is defined by a stohasti differential equation (SDE) ẋ = f(q,x,t)+g(q,x,t)ẇ, f : Q R n [, ) R n, g : Q R n [, ) R n k, () a family of m disrete transition/reset maps (q,x) = φ l (q,x,t), φ l : Q R n [, ) Q R n, l {,...,m}, () and a family of m transition intensities λ l (q,x,t), λ l : Q R n [, ) [, ), l {,...,m}, (3) where w denotes a k-vetor of independent Brownian motion proesses and Q a (typially finite) disrete set. A SHS haraterizes a jump proess q : [, ) Q alled the disrete state; a stohasti proess x : [, ) R n with pieewise ontinuous sample paths alled the ontinuous state; and m stohasti ounters N l : [, ) N alled the transition ounters. In essene, between transition ounter inrements the disrete state remains onstant whereas the ontinuous state flows aording to (). At transition times, the ontinuous and disrete states are reset aording to (). Eah transition ounter N l ounts the number of times that the orresponding disrete transition/reset map φ l is ativated. The frequeny at whih this ours is determined by the transition intensities (3). In partiular, the probability that the ounter N l will inrement in an elementary interval (t, t + dt, and therefore that the orresponding transition takes plae, is given by λ l (q(t),x(t),t)dt. In pratie, one an think of the intensity of a transition as the instantaneous rate at whih that transition ours. Expetations of the state of a SHS an be omputed using the following result from [. Theorem. For every funtion ψ : Q R n [, ) R that is ontinuously differentiable with respet to its seond and third arguments, we have that where (q,x,t) Q R n [, ) (Lψ)(q,x,t) := ψ(q,x,t) x de[ψ(q(t),x(t),t) dt f(q,x,t)+ ψ(q,x,t) + t + trae ( ψ(q,x) x g(q,x,t)g(q,x,t) ) + = E[(Lψ)(q(t),x(t),t), (4) m l= ( ψ ( φ l (q,x,t),t ) ) ψ(q,x,t) λ l (q,x,t), (5) and ψ(q,x,t) t, ψ(q,x,t) x, and ψ(q,x) x denote the partial derivative of ψ(q,x,t) with respet to t, the gradient of ψ(q,x,t) with respet to x, and the Hessian matrix of ψ with respet to x, respetively. The operator ψ Lψ defined by (5) is alled the extended generator of the SHS. We say that a SHS is polynomial (pshs) if its extended generator L is losed on the set of finitepolynomials in x, i.e., (Lψ)(q,x,t) is a finite-polynomial in x for every finite-polynomial ψ(q,x,t) in x. By a finite-polynomials in x we mean a funtion ψ(q,x,t) suh that x ψ(q,x,t) is a (multi-variable) polynomial of finite degree for eah fixed q Q, t [, ). A pshs is obtained, e.g., when the vetor fields f and g, the reset maps φ l, and the transition intensities λ l are all finite-polynomials in x. It turns out that sets of hemial reations an be represented by suh a pshs. To understand why this is so, onsider n speies X, X,..., X n inside a fixed volume V involved in a system of k reations R, R,..., R k of the form R : u X +u X +...+u n X n v X +v X +...+v n X n 3

4 Table. h i and i for different reation types. Reation R i h i i X i reation produts x i k i X i +X j reation produts, (i j) x ix j k i V X i reation produts X i +X j reation produts, (i j) 3X i reation produts xi(xi ) k i V xixj(xj ) k i V 6 xi(xi )(xi ) 6k i V R : u X +u X +...+u n X n v X +v X +...+v n X n. R k : u k X +u k X +...+u kn X k n vk X +v k X +...+v kn X n, where u ij is the stoihiometry assoiated with the j th reatant of the i th reation and v ij is the stoihiometry assoiated with the j th produt of the i th reation. The reation parameter i haraterizes the reation R i and, together with the stoihiometry, defines the probability that a partiular reation takes plae in an infinitesimal time interval (t, t + dt. This probability is given by the produt of the following two terms:. the number h i of distint moleular reatant ombinations present in V at time t for the reation R i,. the probability i dt that apartiularombinationofr i reatantmoleuleswill atuallyreaton(t,t+dt. In general, h i depends both on the reatantsstoihiometry u ij in R i and on the number of reatant moleules in V. Table shows the value of h i for different reation types [8. In this table and in the sequel, we denote by x i the number of moleules of the speies X i in the volume V. The reation parameter i is related to the reation rate k i in the deterministi formulation of hemial kinetis by the formulas shown in the right-most olumn of Table. The evolution of the number of moleules x, x,..., x n an be generated by a pshs. Sine the number of moleules take values in the disrete set on integers, they an be regarded then as either part of the disrete or the ontinuous state of the pshs. However, sine we are interested in omputing the statistial moments of x i, we hose to view them as part of a ontinuous state. In this ase the SHS has a single disrete mode, whih we omit for simpliity. The ontinuous state onsists of a vetor x := [ x,x,...,x n with trivial ontinuous dynamis ẋ =. Eah one of the k reations is assoiated with a reset map defined by the stoihiometry x φ (x) := x u +v x u +v. x n u n+v n with transition intensities given by x φ (x) := x u +v x u +v. x n u n+v n... x φ k (x) := λ (x) := h λ (x) := h... λ k (x) := k h k, x u k +v k x u k +v k. x n u kn +v kn respetively. This results in a pshs beause the reset maps and the transition intensities are finite polynomials in x i. 3 Moment Dynamis To fully haraterize the dynamis of a hemial reation one would like to determine the evolution of the probability distribution for x(t), i.e. solve the Master equation. In general, this is diffiult and a more reasonable goal is to determine the evolution of a few low-order moments., 4

5 Given a vetor of n integers m = (m,m,...,m n ) N n, we define the test-funtion assoiated with m to be ψ (m) (x) := x (m), x R n and the (unentered) moment assoiated with m to be µ (m) (t) := E [ ψ (m)( x(t) ), t. (6) Hereandinthesequel,givenavetorx = (x,x,...,x n ),weusex (m) todenotethemonomialx m xm x mn n. pshss have the property that if one staks all moments in an infinite vetor µ, its dynamis an be written as µ = A (t)µ, t, (7) forsomeappropriatelydefinedinfinitematrixa (t).thisisbeause m = (m,...,m n ) N n,(lψ(m) )(x,t) is a finite-polynomial in x and therefore an be written as a finite linear ombination of test-funtions (possibly with time-varying oeffiients). Equation (7) then follows diretly from (4), (5), and (6). In the sequel, we refer to (7) as the infinite-dimensional moment dynamis. Sine we are only interested in omputing a few low-order moments, we rewrite (7) as µ = I k A (t)µ = A(t)µ+B(t) µ, µ = Cµ, (8) where µ R k ontains to the top k elements of µ, whih orrespond to the lower-order moments of interest. I k denotes a matrix omposed of the first k rows of the infinite identity matrix, µ R r ontains all the moments that appear in the first k elements of A (t)µ but that do not appear in µ, and C is the projetion matrix that extrats µ from µ. Our goal is to approximate the infinite dimensional system (7) by a finite-dimensional nonlinear ODE of the form ν = A(t)ν +B(t) ν(t), ν = ϕ(ν,t), (9) where the map ϕ : R k [, ) R r should be hosen so as to keep ν(t) lose to µ(t). We all (9) the trunated moment dynamis and ϕ the trunation funtion. We make the following two stability assumptions to establish suffiient onditions on ϕ, for the approximation to be valid. Assumption (Boundedness). There exist sets Ω µ R and Ω ν R k suh that all solutions to (7) and (9) starting at some time t in Ω µ and Ω ν, respetively, exist and are smooth on [t, ) with all derivatives of their first k elements uniformly bounded by the same onstant. The set Ω ν is assumed to be forward invariant under the dynamis of (9). Assumption (Inremental Asymptoti Stability). There exists a funtion β KL suh that, for every solution µ to (7) starting in Ω µ at some time t, and every t t, ν Ω ν there exists some ˆµ (t ) Ω µ whose first k elements math ν and µ(t) ˆµ(t) β( µ(t ) ˆµ(t ),t t ), t t, where µ(t) and ˆµ(t) denote the first k elements of the solutions to (7) starting at µ (t ) and ˆµ (t ), respetively. Let di µ(t) dt i and di ν(t) dt i represent the i th time derivative of µ(t) and ν(t) along the trajetories of system (7) and (9) respetively. The following result from [7 shows that if a suffiiently large but finite number of these derivatives math point-wise, then, the differene between solutions to (8) and (9), i.e. µ(t) and ν(t) onverges to an arbitrarily small ball. A funtion β : [, ) [, ) [, ) is said to be of lass KL if β(,t) =, t ; β(s,t) is ontinuous and stritly inreasing on s for eah fixed t ; and lim t β(s,t) =, s. 5

6 Theorem. Suppose Assumptions and hold. Then, for every δ >, there exists an integer N, suffiiently large, for whih the following result holds: Assuming that for every t, µ (t ) Ω µ, ν(t ) Ω ν where µ is the first k elements of µ. Then, along solutions of (7) and (9). µ(t ) = ν(t ) di µ(t ) dt i = di ν(t ) dt i, i {,...,N} () µ(t) ν(t) δ, t [t, ) () For the problems onsidered in this paper, Assumptions and are hard to verify. Moreover, given a onstant δ > and sets Ω µ, Ω ν, it is generally diffiult to determine the integer N for whih the bound () holds. Nevertheless, Theorem is still useful beause it provides the expliit onditions () that the trunation funtion ϕ should satisfy for the solution to the trunated system to approximate the one of the original system. In the subsequent setion we use () for N = to expliitly ompute ϕ. 4 Constrution of Approximate Trunations In this setion, we ompute the trunation funtion ϕ by requiring () to hold for N =. Using (8) and (9), equality () redues to µ(t ) = ν(t ) µ(t ) = ϕ(µ(t ),t ) (i = ) () µ(t ) = ν(t ) d µ(t) t=t = ϕ(µ(t ),t ) I k A (t )µ (t )+ ϕ(µ(t),t) t=t (i = ). (3) dt µ t We look for solutions to the PDE (3) having the following separable form ϕ(ν,t) = ν (Γ) := ν γ ν γ ν γ k k ν γ ν γ ν γ k k.. ν γ r ν γ r ν γ rk k for an appropriately hosen onstant matrix Γ := [γ ij R r k. In order to ompute Γ, we take Ω µ to be the set that orresponds to the family of deterministi distributions. For this set, µ (m) := x (m) and a neessary ondition determined in [7 for the existene of a matrix Γ that satisfies () and (3) is that, for every moment µ (ml) in µ, the polynomial i= a l,ix (mi) must belong to the linear subspae generated by the polynomials { where a l,i denotes the entry in the l th row and i th olumn of A. i=, } a j,i x (m l m j+m i) : j k, (4) Although the family of deterministi distributions may seem very restritive, it provide us with trunations that are aurate even when the system evolves towards nondeterministi distributions, as will be demonstrated in the next setion. When the ondition (4) does not hold, one an often drop lower order terms from the right-hand-side of (5) to form a new matrix Ā that satisfies this ondition. This will be demonstrated in some of the examples in Setion 5. When the solution ϕ to () and (3) is not unique, a speifi ϕ an be seleted by requiring () to hold for additional families of distribution, e.g., the family of lognormal distributions. The polynomials in (4) an have both positive and negative powers. 6

7 Remark. For the lass of elementary reations, i.e, uni- or bi-moleular reations, suffiient onditions for the existene and uniqueness for the solution ϕ to () and (3) will appear in [8. More speifially, it has been shown in [8, that given a vetor µ ontaining all the first and seond order moments of x, (4) holds, if one drops some seond and all first order moments from the seond time derivative of µ. This will be valid, as long as these moments are dominated by the fourth order moments of x, whih also appear in the seond time derivative of µ. Furthermore, expliit formulas for ϕ also appear in [8. The striking features of these formulas is that the dependene of higher order moments on lower order as given by ϕ is independent of the reation parameters, moreover, this dependane is onsistent with x being lognormally distributed. 5 Examples We now develop trunated models for the moment dynamis of several hemial reations and disuss how they ompare with estimates of moments obtained from averaging a large number of Monte Carlo simulations or the exat solution of the Master equation, when available. All Monte Carlo simulations were arried out using the algorithm desribed in [, whih is equivalent to SSA [8. Sine the quantities of interest in the stohasti approah are often the first and seond order moments, we onsider trunation models whose state ontains only the first and seond order moments for the number of moleules in the speies involved in the reation. Example. Consider the following bi-moleular irreversible reation [4: X X. For the sake of simpliity, we omit moments for the population of X from the trunation. The number of moleules x of X, an be generated by a SHS with ontinuous dynamis ẋ = and a reset map x φ(x) := x with intensity λ(x) := x(x ). For ψ(m) (x) = x m, m N we have (Lψ (m) )(x) = x(x )[(x )m x m = and (8) an be written as follows: where µ := µ (3) evolves aording to [ µ () µ () = [ 4 [ µ () µ () m ( m i )(x m i+ x m i+ )( ) i (5) i= µ (3) = 3µ (4) +9µ (3) µ () +4µ (). [ + µ, (6) It an be easily verified that for this system, ondition (4) is not satisfied, as the polynomial a 3,i x (mi) = 3x 4 +9x 3 x +4x does not belong to the subspae generated by the polynomials i= a,i x (m3 m+mi) = x 3 x 4, i= a,i x (m3 m+mi) = x +4x 3 x 4, i= due to the first-order term 4x. However, if we retain only the two highest powers of x in the summation in the right-hand side of (5), we have the following simplified version of (6) [ [ µ () [ [ = µ () + µ, µ () 4 7 µ ()

8 Table. E[x for the exat and approximate trunated models at different times t for Example. E[x refers to the exat solution, µ () refers to the trunated model (6) and (7) and µ () refers to the trunated model developed in [4. t E[x µ () µ () where now µ := µ (3) evolves aording to µ (3) = 3µ (4) +9µ (3) for whih ondition (4) does hold, allowing us to find a unique solution ϕ to () and (3), resulting in a trunated system given by (6) and ( ) µ () 3 µ = ϕ(µ) =. (7) Owing to the simpliity of this reation, one an obtain the exat solution for µ () = E[x and µ () = E[x from the Master equation, as has been done in [4. An alternate approximate trunated model, was also developed in [4, where µ (3) is approximated as µ () µ (). Table and Figure ompare our estimates for the mean and the oeffiient of variation respetively, with that of the exat solution and approximated model developed in [4, for = and initial ondition x() =. Our estimates of E[x are almost idential to those of the exat solution, while, those of CV[x are lose. One an see that the trunation model yields good results for fairly small populations, even though to onstrut it we dropped lower-order powers of x, whih impliitly assumes that the population of the speie X is large. It should be noted that if we retain only the highest powers of x in the summation in the right-hand-side of (5), the unique solution ϕ to () and (3) would have been ϕ(µ) = µ () µ () as in [4. As this approximation is obtained by dropping more terms, it is not surprising to disover that it does not perform as well as (7). Example. Consider the following deaying-dimerizing reation set [9, 5: X, X X, X 3 X, X 4 X3. The number of partiles x := (x,x,x 3 ) of three speies involved in the following set of deaying-dimerizing reation an be generated by a SHS with ontinuous dynamis ẋ = and four reset maps [ x [ x [ x+ [ x x φ (x) := x x φ (x) := x + x φ 3 (x) := x x φ 4 (x) := x x 3 x 3 x 3 x 3+ with intensities λ (x) := x, λ (x) := x (x ), λ 3 (x) := 3 x, and λ 4 (x) := 4 x, respetively. The generator for this SHS is given by (Lψ)(x,x,x 3 ) = x ( ψ(x,x,x 3 ) ψ(x) ) + x (x ) ( ψ(x,x +,x 3 ) ψ(x) ) µ () + 3 x ( ψ(x +,x,x 3 ) ψ(x) ) + 4 x ( ψ(x,x,x 3 +) ψ(x) ). Taking ψ (m,m,m) (x) = x m xm xm3 3, m,m,m 3 N we have (Lψ (m,m,m ) ( )(x) = x (x ) m x m ) x m xm x(x )( (x ) m (x +) m x m ) xm x m 3 3 ( + 3x (x +) m (x ) m x m x m ) x m 3 ( 3 + 4x (x ) m (x 3 +) m 3 x m x m 3 3 ) x m 8

9 E[x Fig.. CV[x = E[x for the exat and approximate trunated models at different times t for Example. E[x + refers to the exat solution, to the trunated model (6) and (7) and to the trunated model developed in [4. m = i= m,m + 3 ( m i )( ) m i ψ (i+,m,m 3 ) (x)+ i,j= (i,j) (m,m ) m,m i,j= (i,j) (m,m ) ( m i ) ( m j ) m i ( ) m j ψ (i,j+,m 3) (x)+ 4 ( m i ) ( m j ) ( ) m i ( ψ (i+,j,m 3) (x) ψ (i+,j,m 3) (x) ) m,m 3 i,j= (i,j) (m,m 3 ) ( m i ) ( m 3 j ) ( ) m i ψ (m,i+,j) (x), where the summations result from the power expansions of the terms (x i ) mi. To keep the formulas short, we omit from the trunation the seond moments of x 3, whih does not appear as a reatant in any reation and therefore its higher order statistis do not affet the first two. In this ase, (8) an be written as follows: µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) = where µ := [µ (3,,) µ (,,) evolves aording to µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) + (8) µ, (9) µ (3,,) = ( +4 )µ (,,) +8 3 µ (,,) +(3 )µ (,,) + 3 µ (,,) +( 3 +9 )µ (3,,) +6 3 µ (,,) 3 µ (4,,) µ (,,) = µ (,,) 4 3 µ (,,) +4 µ (,,) +4 3 µ (,,) +( 4 3 )µ (,,) 5 µ (3,,) +( )µ (,,) +4 3 µ (,,) + µ (4,,) µ (3,,). (a) (b) This system also does not satisfy ondition (4) beause the µ (,,), µ (,,) terms in the right-hand-sides of () lead to monomials in x and x in { i= a l,ix (mi) that do not exist in any of the polynomials i= a j,ix (m l m j+m i) : j 6 }. These terms an be easily traed bak to the lowest-order terms in power expansions in (8) and disappear if we only keep the three highest powers of x in the expansion of (x ) m that orresponds to the first reation and the two highest powers of x and x in the expansions of (x ± ) m and (x ±) m that orrespond to the seond and third reations. In pratie, this leads to 9

10 Table 3. Comparison between estimates obtained from Monte Carlo simulations and the trunated model for Example. Soure for the estimates E[x (.) E[x (.) StdDev[x (.) StdDev[x (.), MC. simulations (data from [5) trunated model (9), () the following simplified version of (9) µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) = where now µ := [µ (3,,) µ (,,) evolves aording to µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) µ (,,) µ (3,,) = 3 µ (,,) +( 3 +3 )µ (3,,) +6 3 µ (,,) 3 µ (4,,) µ (,,) = µ (,,) +( 4 3 )µ (,,) 5 µ (3,,) + +( )µ (,,) +4 3 µ (,,) + µ (4,,) µ (3,,), µ, () for whih ondition (4) does hold, allowing us to find a unique solution ϕ to () and (3), resulting in a trunated system given by (9) and [ (µ ) (,,) 3 µ = ϕ(µ) =, µ (,,) µ (,,) µ (,,) ( ) µ (,,). () µ (,,) We perform simulations for both large and small initial populations. Figures and 3 show a omparison between Monte Carlo simulations and the trunated model (9), (). The oeffiients used were taken from [5, Example : =, =, 3 =, 4 =. In Fig. (a) we used the same initial onditions as in [5, Example : x () = 4, x () = 798, x 3 () =. (3) The math is very aurate, as an be onfirmed from Table 3. The values of the parameters hosen result in a pshs with two distint time sales, whih makes this pshs omputationally diffiult to simulate ( stiff in the terminology of [5). The initial onditions (3) start in the slow manifold x = 5 x (x ) and Fig. (a) essentially shows the evolution of the system on this manifold. Figure (b) zooms in on the interval [,5 4 and shows the evolution of the system towards the manifold when it starts away from it at x () = 8, x () =, x 3 () =. (4) Figure 3 shows another simulation of the same reations but for muh smaller initial populations: x () =, x () =, x 3 () = 5. (5) The trunated model still provides an extremely good approximation, with signifiant error only in the ovariane between x and x when the averages and standard deviation of these variables get below one. This happens in spite of having used () instead of () to ompute ϕ.

11 5 E[x E[x population means E[x STD[x STD[x population standard deviations populations orrelation oeffiient E[x E[x population means STD[x STD[x population standard deviations E[x 3 x Corr[x,x populations orrelation oeffiient x 4 Corr[x,x (a) Long time sale with initial ondition (3) x 4 (b) Short time sale with initial ondition (4) Fig.. Comparison between Monte Carlo simulations (solid lines) and the trunated model (9), () (dashed lines) for Example with large populations. 5 5 E[x population means E[x 3 5 E[x STD[x STD[x population standard deviations Corr[x,x populations orrelation oeffiient (a) Long time sale with initial ondition (5) 3 E[x E[x population means STD[x STD[x population standard deviations E[x 3 x Corr[x,x populations orrelation oeffiient x (b) Short time sale with initial ondition (5) Fig. 3. Comparison between Monte Carlo simulations (solid lines) and the trunated model (9), () (dashed lines) for Example with small populations. x 3 Example 3. Consider the following system of hemial reations: X +X X, X +X X, and let x := (x,x ) be the number of moleules for X and X, respetively. The deterministi hemial rate equations for this system are given by ẋ = x x, ẋ = x x. (6) Equation (6) does not have a unique equilibrium point, in fat, every vetor of the form (x,) or (,x ), x R is an equilibrium point for (6). Beause of this, general approximate methods based on loal

12 expansions of the Master equation [7, do not apply. The stohasti dynamis of the number of moleules x := (x,x ) an be desribed by a SHS with ontinuous dynamis ẋ = and reset maps x φ (x) := [ x x, x φ (x) := [ x x, with intensities λ (x) := x x and λ (x) := x x respetively. For ψ (m,m) m (x) = x m x, m,m N, it an be verified that (8) an be written as follows: µ (,) µ (,) µ (,) µ (,) µ (,) = where µ := [µ (,) µ (,) evolves aording to µ (,) µ (,) µ (,) µ (,) µ (,) + µ (,) = µ (,) +µ (,) µ (3,) µ (,) = µ (,) +µ (,) µ (,3). (7) µ, This system satisfies ondition (4) and any funtion ϕ of the form [ µ (,) ( ) µ (,) ν µ = ϕ(µ) = ( ) µ (,) ( ) µ (,) ν µ (,) ν, ( ) µ (,) ν (8) µ (,) satisfies () and (3), where ν, ν R. Expliit onditions on the stoihiometry of the reation for whih () and (3) yield a unique solution for ϕ will appear in [8. In order to find a unique funtion ϕ we now require () to hold for the family of lognormal distribution also. It is straightforward to hek that for bivariate lognormal variables y and z, E[y z = E[y E[z µ (,) ( ) E[yz, (9) E[y whih orresponds to taking ν = ν = in (8), and hene, a unique funtion ϕ an be found as [ µ (,) ( ) µ (,) ( ) µ = ϕ(µ) =, µ(,) µ (,). (3) µ (,) µ (,) Figure 4 shows a omparison between Monte Carlo simulations and the trunated model (7), (3) for = 5 and initial onditions, x () = 55 and x () = 5. The math is very aurate espeially for the mean and standard deviation, as an be seen from the zoomed inserts. Example 4. Consider the Shögl reation set [6: µ (,) µ (,) A+X 3X, 3X A+X, X 3 B, B 4 X, where the number of moleules n A and n B of speies A and B, respetively, are assumed onstants. The deterministi hemial rate equation for this system is ẋ = n Ax 6 x3 3 x+ 4 n B, (3) where x denotes the number of moleules of X. This equation has a unique equilibrium point for some values of the oeffiients i ; while it is bistable, for other values (i.e., it has two stable equilibrium points). The evolution of x an be desribed by a SHS with ontinuous dynamis ẋ = and reset maps x φ (x) := x+, x φ (x) := x, x φ 3 (x) := x, x φ 4 (x) := x+,

13 population means E[x E[x x population standard deviations STD[x STD[x x population orrelation oeffiients Corr[x,x Fig. 4. Comparison between Monte Carlo simulations (solid lines) and the trunated model (7), (3) (dashed lines) for Example 3. with intensities λ (x) := n Ax(x ), λ (x) := 6 x(x )(x ), λ 3(x) := 3 x, λ 4 (x) := 4 n B, respetively. For ψ (m) (x) = x m, m N we have (Lψ (m) )(x) = n Ax(x )[(x+) m x m + 6 x(x )(x )[(x )m x m and (8) an be written as follows: [ [ µ () = µ () 3 na 3 + na 3+n B 4 na+ 3 where µ := [µ (3) µ (4) evolves aording to na x[(x ) m x m + 4 n B [(x+) m x m (3) [µ () µ () + [ 6 3 na+7 6 µ+ [ nb 4 µ (3) = µ (5) +( 3 n A + )µ (4) +( )µ (3) +( n A +3n B )µ () n B 4, (33) +( 3 +3n B 4 n A 3 )µ () +n B 4 µ (4) = 3 µ (6) +(n a +3 )µ (5) +( 5 +n A 4 3 )µ (4) +( n A +4n B )µ (3) 3

14 +( 3 n A +6n b )µ () +( 3 n A +4n B + 3 )µ () +n B 4. Condition (4) is not satisfied, however, if we retain only the two highest powers of x in the right-hand-side of (3), we have the following simplified version of (33) [ [ [ [ µ () = µ () 6 + µ, µ () µ () 3 na na where µ := [µ (3) µ (4) evolves aording to µ (3) = µ (5) +( 3 n A + )µ (4) µ (4) = 3 µ (6) +(n a +3 )µ (5), for whih ondition (4) does hold, allowing us to find a unique solution ϕ to () and (3), resulting in a trunated system given by (33) and µ = ϕ(µ) = [ (µ () µ () )3, ( µ (,) ) 6 ( µ (,) ) 8. (35) Weperformsimulationsfortwosetsofvaluesofoeffiients i.first,theoeffiientsaretakenas =., =.6, 3 =, 4 =, for whih (3) has a unique equilibrium point. Figure 5 shows a omparison between Monte Carlo simulations and the trunated model (33), (35) for x() = 3 and n A = n B =. The math is very aurate as an be onfirmed from the figure E[x STD[x Fig. 5. Comparison between Monte Carlo simulations (solid lines) and the trunated model (33), (35) (dashed lines) for population mean (top) and population standard deviation (bottom) for Example 4, when (3) has a unique equilibrium point. Next, the oeffiients are hanged to =.4, =.6, 3 =.4, 4 =.8, and it an be verified that for these partiular values, (3) is bistable. Figure 6 shows a omparison between Monte Carlo 4

15 simulations and the trunated model for x() = 55 and n A = n B =. The trunated model provides aurate estimates for times less than one seond, with some error after that. We onjeture that the larger errors observed with the trunated model for this reation are related to the stoihiometry of this equation with three moleules of X needed for the reation to take plae. This leads to the fat that the dynamis of the m th order moment depend on the (m +) th and the (m +) th order moments and therefore the trunation funtion must provide an estimate of the third and forth order moments based on the first and seond. This seems signifiantly more hallenging than just estimating third moments from the first two. Fortunately, stoihiometri oeffiients larger than two(as in the Shögl reation) are not very ommon E[x STD[x Fig. 6. Comparison between Monte Carlo simulations (solid lines) and the trunated model (33), (35) (dashed lines) for population mean (top) and population standard deviation (bottom) for Example 4, when (3) is bistable. 6 Conlusions and Future Work An approximate stohasti model for hemially reating systems was presented in this paper. This was done by representing the population of various speies involved in a set of hemial reations as the ontinuous state of a pshs. With suh a representation, the dynamis of the infinite vetor ontaining all the statistial moments of the ontinuous state are governed by an infinite-dimensional linear system of ODEs. We showed that these infinite-dimensional ODEs an be approximated by finite-dimensional nonlinear ordinary differential equations with arbitrary preision and provided a proedure to build this type of approximation. The methodology was illustrate using a variety of examples of hemial reations that have appeared in literature. By omparison with Monte Carlo algorithms we showed that these approximate stohasti models indeed provided very aurate results. Several observations arise from these examples, whih point to diretions for future researh: 5

16 . For all the examples onsidered, it an be seen from equations (7), (), (8) and (35) that the dependene of ϕ(µ) on lower-order moments, is as if x i, is lognormally distributed. As of now we are unable to understand this dependane and further investigation is required.. As seen from the examples, the approximate trunated models provide aurate time evolution of lower order moments for the population of speies even when the variane is non-zero, i.e., the system has evolved to a nondeterministi distribution, although the approximate trunated model was onstruted assuming a deterministi distribution. As part of our future work we are trying to understand this and develop expliit error bounds when the system evolves to their nondeterministi distributions. 3. In [9,, for infinite-dimensional systems that an be partitioned into a finite-dimensional slow subsystems and an infinite-dimensional fast one, a finite dimensional system ( whose dimension is equal to the slow sub-system) an be obtained, that approximates the solution to the infinite-dimensional system, upto a desired auray, for almost all times. Although, the proof of Theorem does not need this kind of struture for the infinite-dimensional system it is possible that in some of the examples something similar might be happening whih requires further investigation. 4. Thetrunated modelswereonstrutedbyrequiring()toholdforn =.Adiretionoffutureresearh onsists of onstruting approximations based on () but for N >. Many proesses in moleular biology involve a large number of moleular speies with very low populations, for whih stohasti effet are ruial. Hene, an additional diretion for future researh is to model stohasti reations in moleular biology using the tehniques presented in this paper. 7 Aknowledgments We would like to thank Mustafa Khammash and Hana El-Samad for several disussions that led us to onsider pshss as a modeling tool for hemial reations. Bibliography [ MAdams, H.H., Arkin, A.P.: Stohasti mehanisms in gene expression. Proeedings of the National Aademy of Sienes U.S.A 94 (997) [ Arkin, A., Ross, J., MAdams, H.H.: Stohasti kineti analysis of developmental pathway bifuration in phage λ-infeted Esherihia oli ells. Genetis 49 (998) [3 Endy, D., Brent, R.: Modelling ellular behaviour. Nature (London) 49 () [4 Elowitz, M., Leibler, S.: A syntheti osillatory netwrok of transriptional regulators. Nature (London) 43 () [5 Gardner, T.S., Cantor, C.R., Collings, J.J.: Constrution of a geneti toggle swith in Esherihia oli. Nature (London) 43 () [6 Walters, M.C., Fiering, S., Eidemiller, J., Magis, W., Groudine, M., Martin, D.I.K.: Enhaners inrease the probability but not the level of gene expression. Proeedings of the National Aademy of Sienes U.S.A 9 (995) [7 Kampen, N.G.V.: Stohasti Proesses in Physis and Chemistry. Elsevier Siene, Amsterdam, The Netherlands () [8 Gillespie, D.T.: A general method for numerially simulating the stohasti time evolution of oupled hemial reations. J. of Computational Physis (976) [9 Gillespie, D.T., Petzold, L.R.: Improved leap-size seletion for aelerated stohasti simulation. J. of Chemial Physis 9 (3) [ Gibson, M.A., Bruk, J.: Effiient exat stohasti simulation of hemial systems with many speies and many hannels. J. of Physial Chemistry A 4 () [ Gillespie, D.T.: Approximate aelerated stohasti simulation of hemially reating systems. J. of Chemial Physis 5 ()

17 [ Hespanha, J.P.: Stohasti hybrid systems: Appliations to ommuniation networks. In Alur, R., Pappas, G.J., eds.: Hybrid Systems: Computation and Control. Number 993 in Let. Notes in Comput. Siene. Springer-Verlag, Berlin (4) [3 Gunawan, R., Cao, Y., Petzold, L., Doyle, F.J.: Sensitivity analysis of disrete stohasti systems. Biophysial Journal 88 (5) [4 MQuarrie, D.A.: Stohasti approah to hemial kinetis. J. of Applied Probability 4 (967) [5 Rathinam, M., Petzold, L.R., Cao, Y., Gillespie, D.T.: Stiffness in stohasti hemially reating systems: The impliit tau-leaping method. J. of Chemial Physis 9 (3) [6 Matheson, I., Walls, D.F., Gardiner, C.W.: Stohasti models of first order nonequilibrium phase transitions in hemial reations. J. of Statistial Physis (975) 34 [7 Hespanha, J.P.: Polynomial stohasti hybrid systems. In: Hybrid Systems : Computation and Control (HSCC) 5, Zurih, Switzerland (5) [8 Singh, A., Hespanha, J.P.: Models for gene regulatory networks using polynomial stohasti hybrid systems. Submitted (5) [9 Chiu, T., D.Christofides, P.: Nonliear ontrol of partiulate proesses. AIChE J. 45 (999) [ Christofides, P.D., Daoutidis, P.: Finite-dimensional ontrol of paraboli pde systems using approximate inertial manifolds. J. of Math. Anal. Appl. 6 (997)