6.2.2 Point processes and counting processes

Transcription

1 56 CHAPTER 6. MASTER EQUATIONS which yield a system of differential equations The solution to the system of equations is d p (t) λp, (6.3) d p k(t) λ (p k p k 1 (t)), (6.4) k 1. (6.5) p n (t) (λt)n e λt (6.6) n! which is called Poisson process. We see that the key assumption underlying a Poisson process is the uniform rate, i.e., constant λ; non-bunching Pr{N(τ) 2} o(τ); and independence of the events Point processes and counting processes More precisely, Eq. (6.6) is called Poisson counting process. There is also another way to look at the events: the Poisson point process T k, the stochastic time for the kth event to occur. It can be shown that Pr{T k t} Pr{N(t) k} (6.7) Then we can show that f Tk (x) λk x k 1 (k 1)! e λx (6.8) which is known as the Gamma distribution of kth order. Note that T 1 is exponentially distributed. It has the memoryless property Pr{T 1 >t+ τ T 1 >t} Pr{T 1 >τ}. (6.9) It can be shown that exponential distribution is the only one with the memoryless property. T 2 can be shown to be T (1) 1 + T (2) 1, the sum of two iid T 1. In fact, T k is the sum of k exponential distributed iid. Both N(t) and T k are Markov processes; Neither are statioanry processes but both have stationary independent increments Relation to two simple ordinary differential equations We now ask the expected value and variance of the Poisson counting process: E[N(t)] λt. (6.1) Hence, d E[N(t)] λ, (6.11)

2 6.3. BIRTH-DEATH PROCESSES IN ONE-DIMENSION 57 representing a process with constant growth. On the other hand, if we consider m number of identially independent radioactive nuclei, then the probabiltiy of an individual nucleus survives to time t is e λt. Therefore, the distribution of k( m ) number of nuclei survive to time t is binomially distributed, with the mean being m(t) which satisfies the ODE dm λm(t), m() m. (6.12) More importantly, the variance of the binomial distribution shows that for large m the pribability distribution has a very small relative variance. Hence it is justified to be modeled by a deterministic ODE (6.12). 6.3 Birth-Death Processes in One-Dimension It should not escape our notice that Eq. (6.3) is in fact a master equation. Poisson process is a discrete-state, continuous time Markov process with countable number of states in N. It represents a simple birth process with a uniform growth rate. A more general class of master equations is the one-dimensional birth-death process, N(t), with transition probability rate (i.e., Eq. 4.55) W (n m, t) μ(m)δ n,m+1 + λ(m)δ n,m 1. (6.13) A birth process and a death process are represented in the Eq. (6.13): Therefore, the general master equation has the form n n +1: μ(n) growth rate, (6.14) n n 1: λ(n) death rate. (6.15) dp n (t) μ(n 1)p n 1 [λ(n)+μ(n)]p n + λ(n +1)p n+1. (6.16) As for one-dimensional Kolmogorov forward equations, there is no general solution to this time-dependent equation. However, its stationary solution can be solved. In particular, if there is no probability flux in the system, then we have the stationary distribution p ss n p ss n k Rate equation for expected value μ(k 1). (6.17) λ(k) Even though one can not solve the time-dependent master equation (6.16) in general, one can obtain a differential equation for its expected value E[N(t)] m(t) np n (t). (6.18) n

3 58 CHAPTER 6. MASTER EQUATIONS We have dm(t) nμ(n 1)p n 1 n[λ(n)+μ(n)]p n + nλ(n +1)p n+1 n μ(n 1)p n 1 λ(n +1)p n+1 n1 n E [μ(n)] E [λ(n)]. (6.19) If the variance of N(t) is very small, then one can approximate E [μ(n)] μ (E[N]) μ(m(t)), E[λ(N)] λ (E[N]) λ(m(t)). (6.2) Then, dm μ(m) λ(m). (6.21) We see that the fixed points of ODE (6.21) are at μ(m) λ(m), which correspond to the modal values of stationary dsitribution in Eq. (6.17). Here is also the place to emphasize the difference between the fixed points of the ODE (6.21) and the stationary distribution (6.17): they are clearly different things, even though they are intimately related. The latter is a generalization of the former. It contains more informations about the relative importance of different fixed points. Note in literature, a fixed point is often refered to as a steady state ; and a stationary distribution is also often refered to as a steady-state distribution. The terminology here can be confusing, and one needs to be more careful and precise. 6.4 Examples A simple example Let us consider the chemical reaction A k1 X (6.22) in which the species A has a fixed concentration of a, while the number of molecules of species X, N, changes with time. Ordinary Differential Equation Based on the Law of Mass Action. First, if we denote the concentration of X by x, then the traditional differential equation model based on the Law of Mass Action gives dx k 1a x. (6.23) The time-dependent solution to the ODE is ( x(t) x() k ) 1a e k 1t + k 1a. (6.24)

4 6.4. EXAMPLES 59 Birth-Death Process Formulation of Chemical Reaction Systems. In terms of a birth-death process, we have Hence, the master equation for the birth-death process is dp n (t) μ(n) k 1 a, λ(n) n. (6.25) k 1 ap n 1 (k 1 a + n)p n +(n +1)p n+1. (6.26) We use the method of generating function to solve the time-dependent problem. Let G(s, t) E [ s N(t)], therefore, G(s, t) t k 1 a(s 1)G(s, t) (s 1) G(s, t). (6.27) s By the method of characteristics, we have the general solution to the first-order partial differential equation: G(s, t) F [ (s 1)e k 1t] { } (s 1)k1 a exp. (6.28) in which F () is an arbitrary function to be determined by the initial condition. Note F () G(1,t)1. Stationary Poisson Distribution. We see that for t, { } (s 1)k1 a G(s, ) exp, (6.29) which is the generating function of Poisson distribution with mean k 1 a/. This is precisely the same result as the x( ) in Eq. (6.24). Time-dependent Solution to the Stochastic Model. Furthermore, let us assume the initial distribution for N(t) is N() N. Then Therefore, { } (s 1)k1 a G(s, ) s N F (s 1) exp. (6.3) G(s, t) [ 1+(s 1)e k 1t] { N (s 1)k1 a ( exp 1 e k ) } 1t. (6.31) The generating function in Eq. (6.31) gives the complete dynamics of the birth-

5 6 CHAPTER 6. MASTER EQUATIONS death process, including the expected value and variance: [ ] G(s, t) E[N(t)] s k ( 1a + N k ) 1a e k 1t, (6.32) [ 2 ] G(s, t) E [N(t)(N(t) 1)] s 2 Var[N(t)] s1 s1 (E[N(t)]) 2 N e 2k 1t, (6.33) ( N e k 1t + k ) 1a (1 e k ) 1t. (6.34) Eq. (6.32) agrees with Eq. (6.24). Stationary Process. The stationary process has a Poisson distribution according to Eq. (6.29): p ss n And the autocovariance function A system with bistability (k 1a/ ) n e k1a/k 1. (6.35) n! E[N(t)N()] (E[N]) 2 k 1a e k 1t. (6.36) We note that the chemical reactions in system (6.22) are first-order, i.e., the differential equation (6.23) based on the Law of Mass Action is linear. The linearity is responsible for the solvability of the model in the previous example. We now consider a system of nonlinear chemcal reactions: A +2X FGGG GGGB k1 3X, X FGGG GGGB k2 B, (6.37) k 2 in which the species A and B have again fixed concentrations of a and b. Ordinary Differential Equation Based on the Law of Mass Action. The dynamics of the centration of the X, denoted by x(t), follows the ordinary differential equation based on the Law of Mass Action: dx bk 2 k 2 x + k 1 ax 2 x 3. (6.38) This equation can be solved by the method of separation of variables, which yields: ( ) x3 x x 2 ( ) x1 x x1 x 3 ( ) x2 x x2 x 1 x3 x x 1 x x 2 x x 3

6 6.5. APPROXIMATING BIRTH-DEATH PROCESSES 61 exp[ (x 1 x 2 )(x 2 x 3 )(x 3 x 1 )t], (6.39) in which x is the initial condition x() x, and x 1 x 2 x 3 are the three roots of the cubic equation bk 2 k 2 x + k 1 ax 2 x 3 (x x 1 )(x x 2 )(x x 3 ). (6.4) Birth-Death Process Formulation of Chemical Reaction Systems. In terms of a birth-death process, we have μ(n) k 1an(n 1) V + k 2 bv, λ(n) n(n 1)(n 2) V 2 + k 2 n, (6.41) in which V is the volume of the chemical reaction system: n/v x is the concentration of the species X. Therefore, the stationary distribution for the master equation, according to Eq. (6.17), is where p ss l p ss α k 1a V, l n1 [ ] α(n 1)(n 2) + β, (6.42) n(n 1)(n 2) + γn β k 2b V 2, γ k 2 V 2. (6.43) The master equation for the birth-death process is p n (t) μ(n 1)p n 1 (μ(n)+λ(n))p n + λ(n +1)p n+1. (6.44) Chemical Equilibrium and Stationary Poisson Distribution. The chemical reactions in (6.37) have the net transformation of A B, via the intermediate steps with intermediate species X. Hence, if the concentrations of A and B satisfy a b k 2, (6.45) k 1 k 2 then the system will eventually reach a chemical equilibrium. The condition (6.45) means α βv/γ, which in turn means that the distribution in (6.42) is Poissonian p eq l (αv )l e αv, (6.46) l! where αv k 1 av/ x eq V is the number of X molecule in chemical equilibrium; x eq is the equilibrium concentration of X. 6.5 Approximating Birth-Death Processes by Fokker- Planck Equations We note that the μ(n) and λ(n) in Eq. (6.41) are functions of the parameter V, the volume of the chemical reaction system. When V tends to infinity while the concentrations of A and B, a and b, are kept at constant, the number of X molecules in the system is expected to be n Vxwhere x is the concentration.

7 62 CHAPTER 6. MASTER EQUATIONS In mathematical terms, if we identify 1 V dx, p μ(n) n(t) f(x, t)dx, v(x), and λ(n) w(x), (6.47) V V then we have the master equation for a birth-death process f(x, t) μ(x dx)f(x dx, t) (μ(x)+λ(x)) f(x, t)+λ(x +dx)f(x +dx, t) t (wf)(x +dx/2) (vf)(x dx/2) x ( x ) 2 v(x)+w(x) x 2 f(x, t) (v(x) w(x))f(x, t), (6.48) 2V x in which x. Eq. (6.48) is a Fokker-Planck equation with a(x) v(x)+w(x), and b(x) v(x) w(x). (6.49) V One notices that when V, the diffusion term vanishes, and the Eq. (6.48) is a first-order partial differential equation whose characteristics is simply Eq. (6.38), the traditional deterministic dynamics for the reaction in Eq. (6.37). This relation shows that the ODEs are the rigorous limit of the master equation. The stochastic model is not an alternative to the deterministic kinetics, it is a more complete kinetic description which is capable of modeling chemical reaction systems with and without fluctuations The Chemical Master Equation One can look at the problem from a different angle. We shall now focus on the CME. One of the most important features of the CME is the natural parameter V which tends to infinity in the thermodynamic limit, giving rise to a system of ordinary differential equations. For example, for the birth and death processes with μ n and λ n, in the thermodynamic limit, we have μ n V { μ(x)+o ( V 1)}, λ n V { λ(x)+o ( V 1)}. (6.5) Now let us consider the standard Fokker-Planck equation: f t 1 2 x 2 (a(x)f) (b(x)f). (6.51) x If we descretize the x in terms of uniform interval δ, wehave [ ] df (x, t) a(x δ) b(x δ) 2δ 2 + f(x δ) 2δ [( a(x) + 2δ 2 + b(x) ) ( a(x) + 2δ 2δ 2 b(x) )] f(x) 2δ [ ] a(x + δ) b(x + δ) + 2δ 2 f(x + δ). (6.52) 2δ 2

8 6.5. APPROXIMATING BIRTH-DEATH PROCESSES 63 Therefore, if μ n and λ n are in the forms of μ n V ( ( a(x)v + b(x)+o V 1 )), λ n V ( ( a(x)v b(x)+o V 1 )), 2 2 (6.53) then the master equation will have a legitimate limit, with a proper Fokker-Planck equation. In fact, μ n + λ n a(x) lim V V 2, (6.54) μ n λ n b(x) lim, (6.55) V V Now let us take a look of the CME. We realize that for the CME, the a(x) 1/V! Hence, there is only first-order PDE in the limit of V. The diffusion is a higher order term. So will this asymptotic expansion be useful as an approximation to the CME? The answer is yes for finite time t, but not for stationary behavior Keizer s Paradox for System with Multi-stability One can obtain the stationary distribution from Eq. (6.48): f ss (x) 2V { v(x) w(x) v(x)+w(x) exp 2V x } v(z) w(z) v(z)+w(z) dz (6.56) On the other hand, from the exact expression for the stationary distribution given in Eq. (6.17), we have in the limit of V, ln p ss k k ln j1 k j1 k/v V μ(j 1) λ(j) + C ln v(j/v ) ( 1 w(j/v ) + o V ln ) + C ( ) v(z) dz + C (6.57) w(z) in which C lnp ss. Noting that probability density function f ss (x) Vp ss Vx : 1 2V ln f ss (x) 1 2V ln pss Vx+ ln V 2V + C 1 x ( ) v(z) ln dz + ln V + C. (6.58) 2 w(z) 2V Compare Eq. (6.58) with Eq. (6.56), both can be written in compact forms { x } f ss q(z) 1 (x) C 1 V exp 2V q(z)+1 dz (6.59a) { x } f ss (x) C 2 V exp V ln q(z)dz (6.59b)

9 64 CHAPTER 6. MASTER EQUATIONS where q(u) w(u)/v(u). Both functions have identical local extrema since d dx f ss (x) and d dx f ss (x) lead to the same equation for the roots of q(x) 1. Furthermore, the corresponding local maxima of f ss (x) and f ss (x) are asymptotically equivalent in the limit of V. To show this, let x be a root of q(x) 1. Then we have { f ss (x) C 1 V exp 2V x } q(z) 1 q(z)+1 dz Vq (x )(x x ) 2, (6.6a) 2 and { f ss (x) C 2 V exp V x } ln q(z)dz Vq (x )(x x ) 2. (6.6b) 2 Stationary distributions f ss (x) and f ss (x) have their minima and maxima at same place with same curvature. However, something is very different between f ss (x) and f ss (x). To see this, let x and x be two roots of q(x) 1with q (x) >, each represent an local maximum for functions f ss (x) and f ss (x). The important question, then, is which maximum is the larger one. This is determined by the signs of x x ln q(z)dz and 2 x q(z) 1 dz. (6.61) x q(z)+1 A simple counter example that shows the terms in (6.61) having different signs can be constructed by using q(z) z 3 + k 2 z k 1 az 2 + k 2 b. Therefore, in the limit of V, f ss (x) and f ss (x) could converge to different local maxima. In the limit of V, f ss (x) and f ss converging to different fixed points of the deterministic dynamics is called Keizer s paradox. The reason of this has to do with the non-uniform convergence of f ss (x) lim lim Vp t Vx(t) lim lim Vp Vx(t) f ss (x). (6.62) V V t

10 6.6. KRAMERS-MOYAL EXPANSION AND VAN KAMPEN S CONDITIONAL DIFFUSION Kramers-Moyal Expansion and van Kampen s Conditional Diffusion f(x) t where W (x y)f(y)dy W (x + z z x z)f(x z)dz W (x + z x)f(x)dz ( ) zw(x + z x)f(x)dz x ( ) 2 x 2 z 2 1 n ( ) W (x + z x)f(x)dz + n! x n ( z) n W (x + z x)f(x)dz 2 n3 x (b(x)f(x)) + 1 (a(x)f(x)) + (6.63) 2 x2 a(x) z 2 W (x + z x)dz, b(x) zw(x + z x)dz. (6.64) This seemingly elegant derivation, however, has a problem. Note that W (y x) is for jump process; it contains many Dirac-delta functions. So it is not a smooth function of x and y! Hence, it is not legitimate to expand the function in Taylor expansion. Van Kampen developed a more appropriate expansion. Following the ideas from the chemical master equation, he considerd the natural parameter V as the system s size, which tends to infinity. Furthermore, the variable in the continuous limit should be density φ n/v, where n is the discrete random variable in the birth-death processes. The limit is such that V, n,but n(t) φ(t)v + z(t)v 1/2 in whch φ(t) satisfies ordinary differential equation dφ/ μ(φ) λ(φ), and stochastic process z(t) has a continuous path and it satisfies a conditional diffusion equation f z (z,t) t where a(x) and b(x) are given in Eq. (6.64). a(φ(t)) fz 2 (z,t) 2 z 2 b (φ(t)) z (zf z(z,t)), (6.65)