6 Continuous-Time Birth and Death Chains

Transcription

1 6 Continuous-Time Birth and Death Chains Angela Peace Biomathematics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press, A. Peace Continuous-Time Birth and Death Chains 1/32

2 General Birth and Death Processes Let X ( t) denote the change in the process from t to t: X (t) = X (t + t) X (t) Standard notation for birth and death rates for X (t) = i λ i = birth rate & µ i = death rate The CTMC {X (t) : t [0, )} may have a finite or infinite state space: {0, 1,..., N}, {0, 1, 2...} A. Peace Continuous-Time Birth and Death Chains 2/32

3 General Birth and Death Processes The transition probabilities: p i+j,i ( t) = Prob{ x(t) = j X (t) = i} λ i t + o( t), j = 1 µ i t + o( t), j = 1 = 1 (λ i + µ i ) t + o( t), j = 0 o( t), j 1, 0, 1 where λ i, µ i 0 for i = 0, 1, 2,... and µ 0 = 0 If λ 0 > 0 then there is immigration. t is sufficiently small so at most one change in state can occur in the time interval a birth i i + 1 a death i i = 1 A. Peace Continuous-Time Birth and Death Chains 3/32

4 General Birth and Death Processes The forward Kolmogorov differential equations can be derived directly from the transition probabilities. Consider the transition probability p ji (t + t) p ji (t + t) = p j 1,i (t)[λ j 1 t + o( t)] + p j+1,i (t)[µ j+1 t + o( t)] + P ji (t)[1 (λ j + µ j ) t + o( t)] + p j+k,i (t)o( t) k 1,0,1 = p j 1,i (t)λ j 1 t +p j+1,i (t)µ j+1 t +P ji (t)[1 (λ j +µ j ) t]+o( t) If j = 0 then p 0i (t + t) = p 1i (t)µ 1 t + p 0i (t)[1 λ 0 t] + o( t) If j = N and λ N = 0, p kn (t) = 0 for K > N then p Ni (t + t) = p N 1,i (t)λ N 1 t + p Ni (t)[1 µ N t] + o( t) A. Peace Continuous-Time Birth and Death Chains 4/32

5 General Birth and Death Processes Question Given the transition probabilities of the finite general birth and death process p ji (t + t) = p j 1,i (t)λ j 1 t + p j+1,i (t)µ j+1 t + P ji (t)[1 (λ j + µ j ) t] + p 0i (t + t) = p 1i (t)µ 1 t + p 0i (t)[1 λ 0 t] + o( t) p Ni (t + t) = p N 1,i (t)λ N 1 t + p Ni (t)[1 µ N t] + o( t) 1. Derive the forward Kolmogorov differential equations for dp ji (t), dt 2. What is the generator matrix Q? dp 0i (t), & dt dp Ni (t) dt 3. What is the transition matrix T of the embedded DTMC? A. Peace Continuous-Time Birth and Death Chains 5/32

6 Stationary Probability Distribution π for general CTMC Stationary probability distribution π = (π 0, π 1,...) T of a CTMC with generator matrix Q satisfies Qπ = 0, π i = 1, & π i 0 i=0 A. Peace Continuous-Time Birth and Death Chains 6/32

7 Stationary Probability Distribution Theorem 6.1: π for birth and death processes Let {X (t) : t [0, )}, be a general birth and death chain. If the state space is infinite, {0, 1, 2,...}, a unique positive stationary probability distribution exists iff µ i > 0 and λ i 1 > 0 for i = 1, 2,..., and i=1 λ 0 λ 1 λ i 1 µ 1 µ 2 µ i < For i = 1, 2,..., the stationary probability distribution equals π i = λ 0λ 1 λ i 1 µ 1 µ 2 µ i π 0 & π 0 = i=1 λ 0 λ 1 λ i 1 µ 1 µ 2 µ i If the state space is finite {0, 1, 2,..., N} a unique positive stationary probability distribution exists iff µ i > 0 and λ i 1 > 0 for i = 1, 2,..., N. Here the above summations extend from i = 1, 2,..., N If λ 0 = 0 and µ i > 0 for i 1, then π = (1, 0, 0,...) T A. Peace Continuous-Time Birth and Death Chains 7/32

8 Stationary Probability Distribution Example Suppose a continuous-time birth and death Markov chain had birth and death rates λ i = b and µ i = id for i = 0, 1, 2, Show that the unique stationary probability distribution is π i = (b/d)i e b/d i! A. Peace Continuous-Time Birth and Death Chains 8/32

9 Simple Birth Process Let X (t) represent the population size at time t Assume X (0) = N so that p i (0) = δ in. The only event is a birth The population can only increase in size For t sufficiently small the transition probabilities are p i+j,i ( t) = Prob{ x(t) = j X (t) = i} λi t + o( t), j = 1 1 λi t + o( t), j = 0 = o( t), j 2 0, j < 0 A. Peace Continuous-Time Birth and Death Chains 9/32

10 Simple Birth Process The probabilities p i (t) = Prob{X (t) = i} are solutions of the forward Kolmogorov differential equations dp dt = Qp where dp i (t) = λ(i 1)p i 1 (t) λip i (t), i = N, N + 1,..., dt dp i (t) = 0, i = 0, 1,..., N 1 dt with initial conditions p i (0) = δ in. The state space is {N, N + 1,...} A. Peace Continuous-Time Birth and Death Chains 10/32

11 Simple Birth Process The generating functions for the simple birth process correspond to a negative binomial distribution. We can see this by deriving a PDE for the p.g.f. and the m.g.f. and solving them using the method of characteristics. Working through the details 1. Multiply the forward equations by z i and sum over i to get the p.g.f. P(z, t) t = λz(z 1) P z with initial conditions P(z, 0) = Z N. 2. Derive the PDE for the m.g.f. from the above PDE using the change of variables: z = e θ. 3. Use the method of characteristics to find the solutions M(θ, t) 4. Write the solution for the p.g.f by changing the variables back: θ = ln z A. Peace Continuous-Time Birth and Death Chains 11/32

12 Simple Birth Process The m.g.f. for the simple birth process is M(θ, t) = [1 e λt (1 e θ )] N The p.g.f. for the simple birth process is P(z, t) = (pz)n (1 zq) N where p = e λt and q = 1 e λt. The probabilities can be written as ( ) n 1 p n (t) = e λnt (1 e λt ) n N, n = N, N + 1,... n N The mean and variance are m(t) = Ne λt & σ 2 (t) = Ne 2λt (1 e λt ) The mean corresponds to exponential growth The variance increases exponentially with time A. Peace Continuous-Time Birth and Death Chains 12/32

13 Simple Death Process Let X (t) represent the population size at time t Assume X (0) = N so that p i (0) = δ in. The only event is a death The population can only decrease in size For t sufficiently small the transition probabilities are p i+j,i ( t) = Prob{ x(t) = j X (t) = i} µi t + o( t), j = 1 1 µi t + o( t), j = 0 = o( t), j 2 0, j < 0 The state space is {0, 1, 2,..., N} A. Peace Continuous-Time Birth and Death Chains 13/32

14 Simple Death Process The forward Kolmogorov equations are dp i (t) dt dp N (t) dt = µ(i + 1)p i+1 (t) µip i (t), i = 0, 1,...N 1 = µnp N (t) Initial conditions: p i0 = δ in Here 0 is an absorbing state The stationary probability distribution is π i = (1, 0,...0) T. A. Peace Continuous-Time Birth and Death Chains 14/32

15 Simple Death Process The generating function technique and method of characteristics are used to find the probability distribution The PDE for the p.g.f is P(z, t) t = µ(1 z) P z, P(z, 0) = zn The PDE for the m.g.f is M t = µ(e θ 1) M θ, M(θ, 0) = enθ Applying the method of characteristics yields the solutions: P(z, t) = (1 e µt + e µt z) N & M(θ, t) = (1 e µt (1 e θ )) N A. Peace Continuous-Time Birth and Death Chains 15/32

16 Simple Death Process The p.g.f had the form P(z, t) = (q + pz) N where p = e µt and q = 1 e µt This corresponds to a binomial distribution b(n, p) For i = 0, 1,..., N the probabilities satisfy ( ) N p i (t) = p i q N i i The mean and variance for a binomial distribution b(n, p) are m(t) = Np and σ 2 (t) = Npq: m(t) = Ne µt & σ 2 (t) = Ne µt (1 e µt ) The mean corresponds to exponential decay The variance decreases exponentially with time A. Peace Continuous-Time Birth and Death Chains 16/32

17 Simple Birth and Death Process Let X (t) represent the population size at time t Assume X (0) = N so that p i (0) = δ in. An event can be a birth or a death The population can increase or decrease in size For t sufficiently small the transition probabilities are p i+j,i ( t) = Prob{ x(t) = j X (t) = i} µi t + o( t), j = 1 λi t + o( t), j = 1 = 1 (λ + µ)i t + o( t), j = 0 o( t), j 1, 0, 1 The state space is {0, 1, 2,..., N} λ 0 = 0 and π = (1, 0,..., ) T A. Peace Continuous-Time Birth and Death Chains 17/32

18 Simple Birth and Death Process The forward Kolmogorov equations are dp i (t) = λ(i 1)P i 1 (t) + µ(i + 1)p i+1 (t) (λ + µ)ip i (t) dt dp 0 (t) = µp 1 (t) dt for i = 0, 1,... with initial conditions p i (0) = δ in. An explicit solution to P(t) is not possible but we can determine the moments of this distribution by using the generating function technique First-order partial differential equations can be derived for the p.g.f and m.g.f. A. Peace Continuous-Time Birth and Death Chains 18/32

19 Simple Birth and Death Process Generating function technique P(z, t) = i=0 p i(t)z i is the p.g.f. and M(θ, t) = P(e θ, t) the m.g.f. Multiply P(t) = QP(t) by z i and sum over i i=0 z i dp i(t) dt = z i [λ(i 1)p i 1 (t)+µ(i +1)p i+1 (t) (λ+µ)ip i (t)] i=0 Interchange differentiation and integration, evaluate at z = 1 to obtain the PDE for the p.g.f. P(z, t) t = [µ(1 z) + λz(z 1)] P z, P(z, 0) = zn The change of variables z = e θ leads to the m.g.f. equation M(θ, t) t = [µ(e θ 1) + λ(e θ 1)] M θ, M(θ, 0) = eθn A. Peace Continuous-Time Birth and Death Chains 19/32

20 Simple Birth and Death Process Generating function technique Application of the method of characteristics to the m.g.f yields the ODES dt dτ = 1, dθ dm µ(e θ 1) + λ(e θ = dτ, & 1) dτ = 0 with initial conditions t(s, 0) = 0, θ(s, 0) = s, & M(s, 0) = e sn A. Peace Continuous-Time Birth and Death Chains 20/32

21 Simple Birth and Death Process Generating function technique The m.g.f. is M(θ, t) = ( e t(µ λ) (λe θ µ) µ(e θ 1) e t(µ λ) (λe θ µ) λ(e θ 1) ( 1 (λt 1)(e θ 1) 1 λt(e θ 1) ) N, if λ µ ) N, if λ = µ Making the change of variables θ = ln z yields the p.g.f ( ) e t(µ λ) N (λz µ) µ(z 1) e P(z, t) = t(µ λ) (λz µ) λ(z 1), if λ µ ( ) N 1 (λt 1)(z 1), if λ = µ 1 λt(z 1) A. Peace Continuous-Time Birth and Death Chains 21/32

22 Simple Birth and Death Process Generating function technique Unlike, the simple birth process and the simple death process, these generating functions can not be associated with a well known probability distribution. But, probabilities and moments associated with the process can be obtained directly from the generating functions. Recall that P(z, t) = p i (t)z i, & p i (t) = 1 i P i! z i i=0 z=0 The first term in the series expansion of P(z, t) are p 0 (t) = P(0, t): ( ) µ µe (µ λ)t N λ µe, if λ µ p 0 (t) = ( ) (µ λ)t N λt 1+λt, if λ = µ A. Peace Continuous-Time Birth and Death Chains 22/32

23 Simple Birth and Death Process Generating function technique The probability of extinction has a simple expression when t { 1, if λ µ p 0 ( ) = lim p 0 (t) = ( t µ ) N λ, if λ > µ The mean and variance for λ µ are ( ) λ + µ m(t) = Ne (λ µ)t & σ 2 (t) = N e (λ µ)t (e (λ µ)t 1) λ µ The mean corresponds to exponential growth when λ > µ and exponential decay when λ < µ The mean and variance for λ = µ are m(t) = N & σ 2 (t) = 2Nλt A. Peace Continuous-Time Birth and Death Chains 23/32

24 Simple Birth and Death Process Population Extinction The probability of extinction has a simple expression with t : { 1 if λ µ p 0 (t) = ( µ ) N λ if λ > µ This is similar to the gambler s ruin problem, a semi-infinite random walk with an absorbing barrier at x = 0. µ is probability of losing a game λ is the probability of wining a game If probability of losing (death) is greater than or equal to the probability of winning (birth), then, in the long run the probability of losing all of the initial capital N (probability of absorption) approaches 1. If the probability of winning is greater than the probability of losing, then, in the long run, the probability of losing all of the initial capital is (µ/λ) N A. Peace Continuous-Time Birth and Death Chains 24/32

25 Population Extinction Theorem Let µ 0, λ 0 = 0 in a general birth and death chain with X (0) = m 1 1. Suppose µ i > 0 and λ i > 0 for i = 12,..., If i=1 µ 1 µ 2 µ i λ 1 λ 2 λ i = then lim p 0 (t) = 1 t If i=1 µ 1 µ 2 µ i λ 1 λ 2 λ i < and p 0 (t) 0 as m then lim p 0 (t) = t µ 1 µ 2 µ i i=m λ 1 λ 2 λ i 1 + i=1 µ 1 µ 2 µ i λ 1 λ 2 λ i 2. Suppose µ i > 0 for i = 1, 2,... and λ i > 0 for i = 1, 2,..., N 1 and λ i = 0 for i = N, N + 1,... Then lim p 0(t) = 1 t A. Peace Continuous-Time Birth and Death Chains 25/32

26 First Passage Time The time until the process reachers a certain stage for the first time is the first passage time. Suppose a population size is a and we want to find the time it takes until it reaches a size of b (b < a or b > a). The first passage time problems are related to the backward Kolmogorov equations. We will see how to compute the expected time to extinction A. Peace Continuous-Time Birth and Death Chains 26/32

27 First Passage Time Time to Extinction Suppose λ 0, µ 0 = 0 and lim t p 0 (t) = 1 in a finite MC. Define τ = (τ 1, τ 2,..., τ n ) as the expected time to extinction τ is the solution of the following system τ T Q = 1 T where Q is the n n generator matrix with the first row and column deleted. These elements are deleted to correspond with τ 0 = 0. A. Peace Continuous-Time Birth and Death Chains 27/32

28 Logistic Growth Process Recall deterministic logistic model: dn (1 dt = rn n ), n(0) = n o > 0 K where r is the intrinsic growth rate and K is the carrying capacity lim t n(t) = K The right hand side equals the birth minus the death rate λ n µ n = rn r k n2 Similar to the case for DTMC there are ways to describe a stochastic logistic growth process A. Peace Continuous-Time Birth and Death Chains 28/32

29 Logistic Growth Process Recall the forumlation for DTMC DTMC Stochastic Logistic Growth Model Case a: b i = r (i i 2 ) and d i = r i 2 2K 2K for i = 0, 1, 2,..., 2K Case b: b i = for i = 0, 1, 2,..., N { ri, i = 0, 1, 2,..., N 0, i N and d i = r i 2 K A. Peace Continuous-Time Birth and Death Chains 29/32

30 Logistic Growth Process Analogous forumlation for CTMC CTMC Stochastic Logistic Growth Model Case a: for i = 0, 1, 2,..., 2K λ i = r (i i 2 ) and µ i = r i 2 2K 2K Case b: λ i = for i = 0, 1, 2,..., N { ri, i = 0, 1, 2,..., N 0, i N and d i = r i 2 K Both cases correspond to the deterministic system dn (1 dt = rn n ) K A. Peace Continuous-Time Birth and Death Chains 30/32

31 Logistic Growth Process A. Peace Continuous-Time Birth and Death Chains 31/32

32 Logistic Growth Process Unlike the deterministic Logistic growth model, In the limit the stochastic logistic growth process does not approach K. Extinction is an absorbing state For large population size, the time to extinction is very large (but finite) A. Peace Continuous-Time Birth and Death Chains 32/32