Probability Distributions

Transcription

1 Lecture 1: Background in Probability Theory Probability Distributions The probability mass function (pmf) or probability density functions (pdf), mean, µ, variance, σ 2, and moment generating function (mgf) M(t) for some well-known discrete and continuous probability distributions Discrete Distributions Discrete Uniform: f(x) = { 1, n x = 1, 2,, n, 0, otherwise µ = 1 + n, σ 2 = n , M(t) = e(n+1)t e t n(e t 1), t 0 Geometric: f(x) = { p(1 p) x, x = 0, 1, 2,, 0, otherwise, where 0 < p < 1 µ = 1 p p, σ2 = 1 p p 2, M(t) = Binomial b(n, p): f(x) = ( ) n 0 < p < 1 The notation = x p 1 (1 p)e t ( ) n p x (1 p) n x, x = 0, 1, 2,, n, 0, otherwise, n! x!(n x)! where n is a positive integer and µ = np, σ 2 = np(1 p), M(t) = (1 p + pe t ) n where n is a positive inte- ( ) x + n 1 p Negative Binomial: f(x) = n 1 n (1 p) x, x = 0, 1, 2,, 0, otherwise, ger and 0 < p < 1 µ = n(1 p), σ 2 n(1 p) = p p 2, M(t) = p n [1 (1 p)e t ] n λ x e λ Poisson, P o(λ): f(x) =, x = 0, 1, 2,, x! 0, otherwise, µ = λ, σ 2 = λ, M(t) = e λ(et 1) where λ is a positive constant Continuous Distributions Uniform U(a, b): f(x) = 1 b a, a x b, 0, otherwise, where a < b are constants µ = a + b 2, σ2 = (b a)2, M(t) = ebt e at 12 t(b a), t 0 1

2 Gamma: f(x) = 0 1 Γ(α)β α xα 1 e x/β, x 0, 0, x < 0, 1 β α xα 1 e x/β dx For a positive integer n, Γ(n) = (n 1)! µ = αβ, σ 2 = αβ 2, M(t) = 1 (1 βt) α, t < 1/β where α and β are positive constants and Γ(α) = Exponential: f(x) = { λe λx, x 0, 0, x < 0, where λ is a positive constant µ = 1 λ, σ2 = 1 λ 2, M(t) = λ λ t, t < λ Normal, N(µ, σ 2 ): f(x) = 1 ( ) σ 2π exp (x µ)2, < x <, where µ and σ are constants 2σ 2 E(X) = µ, V ar(x) = σ 2, M(t) = e µt+σ2 t 2 /2 The probability generating function (pgf) is P X (t) = E(t X ), moment generating function (mgf) is M X (t) = E(e tx ) and cumulant generating function (cgf) is K X (t) = ln(m X (t)) The mean µ X = E(X) and variance σ 2 X = E(X2 ) E 2 (X) computed from the pgf, P X (t), mgf, M X (t) and cgf, K X (t): µ X = P X(1) = M X(0) = K X(0) and P X (1) + P X (1) [P X (1)]2 σx 2 = M X (0) [M X (0)]2 K X (0) Continuous-time Markov Chain Example, Simple Birth Process: The per capita rate of birth is λ = Population size Time Figure 1: Three sample paths (stochastic realizations) of a simple birth process, X(0) = 1; n(t) = e t is the dashed curve See Table 1 2

3 Table 1: For two stochastic realizations, the times at which a birth occurs for a simple birth process Realization 1 Realization 2 Size X(t) Event Time t Size X(t) Event Time t

4 Lecture 2: Discrete-Time Markov Chains (DTMCs) Transition probabilities: p ji (n) = Prob{X n+1 = j X n = i} If p ji (n) does not depend on n, then the process is said to be time homogeneous The transition matrix of a DTMC {X n } n=0 with state space {1, 2, } and one-step transition probabilities, {p ij} i,j=1, is denoted as P = (p ij ), where p 11 p 12 p 13 p 21 p 22 p 23 P = p 31 p 32 p 33 The column sums equal one, a stochastic matrix, j=1 p ji = 1 The n-step transition matrix P n = (p (n) The probabilities p i (n) = Prob{X n = i}, p i (n + 1) = j=1 p ijp j (n), i = 1, 2 p(n + 1) = P p(n) Random walk model with absorbing barriers See the directed graph in Figure 2 and the corresponding (N + 1) (N + 1) transition matrix: 1 q q p p P = q p p 1 The Markov chain, graphed in Figure 2, has three communication classes: {0}, {1, 2,, N 1}, and {N} The Markov chain is reducible States 0 and N are absorbing; the remaining states are transient ij ) N Figure 2: Probability of moving to right is p and to the left is q, p + q = 1 Boundaries 0 and N are absorbing, p 00 = 1 = p NN (random walk with absorbing barriers or gambler s ruin problem) A DTMC is irreducible if its digraph is strongly connected Otherwise it is called reducible An irreducible DTMC can be positive recurrent or null recurrent or transient It may also be classified as periodic or aperiodic Recurrence is defined for each state i in the chain Recurrence means for each state i if the process leaves state i it will return to state i at some future time If not, the state is transient A state i is positive recurrent if the mean recurrence time (µ ii mean return time) is finite A state i with an infinite mean recurrence time is called null recurrent be a recurrent, ir- Theorem 1 (Basic Limit Theorem for aperiodic Markov chains) Let {X n } n=0 reducible, and aperiodic DTMC with transition matrix P = (p ij ) Then lim n p(n) ij = 1, µ ii where µ ii is the mean recurrence time for state i and i and j are any states of the chain [If µ ii =, then lim n p (n) ij = 0] 4

5 Theorem 2 (Basic Limit Theorem for periodic Markov chains) Let {X n } n=0 be a recurrent, irreducible, and d-periodic DTMC, d > 1, with transition matrix P = (p ij ) Then lim n p(nd) ii = d µ ii and p (m) ii = 0 if m is not a multiple of d, where µ ii is the mean recurrence time for state i [If µ ii =, then lim n p (nd) ii = 0] Random Walk with Absorbing Boundaries or Gambler s Ruin Let s return to the random walk model with absorbing boundaries This DTMC does not satisfy either of the Basic Limit Theorems because it is a reducible Markov chain with three communicating classes, two absorbing states {0} and {N} and the transient class {1, 2,, N 1} Interesting questions about problems with absorbing states such as an extinction state in population dynamics are (1) what is the probability of absorption? and (2) what is the expected time until absorption? We will investigate this question with the random walk model and with genetics of inbreeding for a finite DTMC Later, we will investigate the probability of extinction in a population model via a branching process Consider the random walk model with absorbing boundaries at 0 and N In the gambler s ruin problem, state 0 is ruin and state N is winning the maximum amount of money In either case, after hitting 0 or N, you stop playing the game Let a k be the probability of ruin (absorption at k = 0) beginning with a capital of k: a k = pa k+1 + qa k 1 (1) for 1 k N 1 Equation (1) is a second-order difference equation in a k The difference equation can be written as pa k+1 a k + qa k 1 = 0, k = 1,, N 1 (2) This method of deriving equation (1) is referred to as a first-step analysis (Taylor and Karlin, 1998) This difference equation (2) can be solved with the following boundary conditions Alternately, it will be shown that a 0 = 1 and a N = 0 (a 1,, a N 1 ) = (p 01, p 02,, p 0,N 1 )(I T ) 1 In general, the difference equation for probability of absorption has the form: a k = N p ik a i To find the probability of absorption into state N, the boundary conditions are a 0 = 0 and a N = 1 k=0 Expected Time until Absorption: Let τ k be the expected time until absorption at either 0 or N in the gambler s ruin problem The difference equation for τ k has the following form: for k = 1, 2,, N 1 or τ k = p(1 + τ k+1 ) + q(1 + τ k 1 ), pτ k+1 τ k + qτ k 1 = 1, (3) The boundary conditions are τ 0 = 0 = τ N In the gambler s ruin problem, this equation can be solved explicitly But in general, N N τ k = p ik (1 + τ i ) or τ k p ik τ i = 1 i=0 5 i=0

6 It will be shown that (τ 1, τ 2,, τ N 1 ) = 1(I T ) 1, 1 = (1, 1,, 1) Table 2: Gambler s ruin problem with a beginning capital of k = 50 and a total capital of N = 100 Prob a 50 b 50 τ 50 A 50 (1) B 50 (1) q = q = q = q = Capital Games Figure 3: Three sample paths for the gambler s ruin problem when N = 100, k = 50, and q = 055 Example of Genetics of Inbreeding: Two alleles A and a There are six possible breeding pairs which denote the six states of the DTMC, 1: AA AA, 2 aa aa, 3 Aa Aa, 4 Aa aa, 5 AA aa, 6 AA Aa Inbreeding of the first two types results in offspring of the same genotypes and inbreeding in the next generation will be of the same type; they are absorbing states The remaining states, 3,4,5,6 are transient The transition matrix has the following form: 1 f 0 1/ / /16 1/4 0 0 ( ) P = 0 0 1/4 1/4 1 1/4 I A 0 0 1/4 1/2 0 0 = O T 0 0 1/ / /2 Probability of absorption into states 1 or 2 from states 3, 4, 5, 6 is computed from the fundamental matrix, (I T ) 1, ( lim P n I (A + AT + AT = 2 ) ( ) + ) I A(I T ) 1 = n O O O O Thus, A(I T ) 1 is the probability of absorption into states 1 or 2 (fixation) from states 3, 4, 5 or 6 Let 1 = (1, 1,, 1), then 1(I T ) 1 is the mean time until absorption from states 3, 4, 5, or 6 6

7 Lecture 3: Discrete-Time Branching Processes Figure 4: Sample path of a branching process {X n } n=0 In the first generation, four individuals are born, X 1 = 4 The four individuals in generation one give birth to three, zero, four, and one individuals, respectively, making a total of eight individuals in generation two, X 2 = 8 Generating functions rather than transition matrices are useful in analysis of branching processes Offspring pgf: f(t) = p k t k k=0 Recall f(1) = 1 and m = f (1) is the mean number of offspring The branching process is called subcritical if m < 1, critical if m = 1, and supercritical if m > 1 Theorem 3 (Branching Process Theorem) Let X 0 = 1 Assume f(0) = p 0 > 0 and p 0 + p 1 < 1 (i) If m 1, then lim n Prob{X n = 0} = 1 (ii) If m > 1, then lim n Prob{X n = 0} = q, where q = f(q) is the unique fixed point in the interval (0, 1) To verify this theorem, we show that given X 0 = 1, then P Xn (t) = f n 1 (f(t)) = f n (t), the pgf of X n is the n-fold composition of the offspring pgf Then p 0 (n) = P Xn (0) = f n (0) and p 0 (n) = f(p 0 (n 1)) The sequence {p 0 (n)} is an increasing function, bounded above by one and therefore, has a limit q This limit is a fixed point of f, f(q) = q If X 0 = N and m > 1, then lim n p 0 (n) = lim n Prob{X n = 0} = q N The conditional expectation E(X n+1 X n ) = mx n, E(X n+1 ) = me(x n ) If the growth rate varies with n, with a mean m n, then E(X n+1 ) = m n E(X n ) Multitype Branching Process, n different types: (X 1,, X n ) The offspring random variable of type i is Y i Offspring pgfs: f i (t 1,, n) = Expectation matrix M = (m ij ), where s n s 1 P i (s 1,, s n )t s1 1 tsn n P i (s 1,, s n ) = Prob{Y 1 = s 1,, Y n = s n } m ji = f t j t1 =1,,t n=1 Assume M is irreducible Denote the spectral radius of M as ρ(m), the maximum modulus of the eigenvalues of MThe multitype branching process is called subcritical if ρ(m) < 1, critical if ρ(m) = 1 and supercritical if ρ(m) > 1 See Chapter on Branching Processes 7

8 Lecture 4: Continuous-Time Markov Chains (CTMCs), Introduction Discrete random variable X(t), t [0, ) Probabilities p i (t) = Prob{X(t) = i} Transition probability: p ji (t, s) = Prob{X(t) = j X(s) = i}, s < t We will assume time-homogenous transition probabilities p ji (t, s) = p ji (t s) The transition matrix is a stochastic matrix: where p 00 (t) p 01 (t) P (t) = (p ij (t)) = p 10 (t) p 11 (t), p ji ( t) = δ ji + q ji t + o( t) is an infinitesimal transition probability The infinitesimal generator matrix: q 00 q 01 Q = (q ij ) = q 10 q 11 P ( t) I, Q = lim t 0 t The column sums of Q equal zero dp (t) Forward Kolmogorov differential equations: = QP (t) dt dp (t) Backward Kolmogorov differential equations: = P (t)q dt In physics and chemistry, the forward Kolmogorov differential equations are often referred to as the Master equation The embedded DTMC is used to define irreducible, recurrent, and transient states or chains for the associated CTMC Let Y n denote the random variable for the state of a CTMC {X(t) : t [0, )} at the time of the nth jump, Y n = X(W n ) (See Figure 5) The set of discrete random variables {Y n } 0 is the embedded Markov chain T0 T1 T2 T3 0 W1 W2 W3 W4 Figure 5: Sample path of a CTMC, illustrating waiting times {W i } and interevent times, {T i } A CTMC is irreducible, recurrent or transient if the corresponding embedded Markov chain has these properties Some differences in the dynamics of a CTMC as opposed to a DTMC are the possibility of a finite-time blow up in a CTMC (explosive process) and the fact that CTMC are not periodic See Figure 6 The embedded MC cannot be used to classify chains as positive recurrent or null recurrent This latter classification depends on the mean recurrence time µ ii 8

9 0 W1 W2 W3 W4 W Figure 6: One sample path of a continuous time Markov chain that is explosive Theorem 4 (Basic Limit Theorem for CTMCs) If the CTMC {X(t) : t [0, )} is nonexplosive and irreducible, then for all i and j, lim p ij(t) = 1, (4) t q ii µ ii where µ ii is the mean recurrence time, 0 < µ ii In particular, a finite, irreducible CTMC is nonexplosive and the limit (4) exists and is positive If the DTMC is nonexplosive and positive recurrent, it has a limiting positive stationary distribution π satisfying Qπ = 0, where π i = 1 q ii µ ii Poisson process with X(0) = 0, p i+1,i ( t) = λ t + o( t) and p i (t) = e λt (λt) i /i! has generator matrix λ 0 0 λ λ 0 Q = 0 λ λ The associated embedded Markov chain has a transition matrix T = The Poisson process is transient A finite CTMC with two states {1, 2} The generator matrix ( ) a b Q =, a, b > 0 a b The CTMC is irreducible and positive recurrent The limiting stationary distribution can be found from the forward Kolmogorov differential equations by solving for the stationary distribution Qπ = 0 In this case π = (b/(a + b), a/(a + b)) tr The mean recurrence times are µ ii = a + b, i = 1, 2 ab 9

10 Lecture 5: Continuous-Time Markov Chains (CTMCs), Interevent Time To generate sample paths, we must know the time between jumps and the state to which the process jumps The Markov assumption implies the interevent time is exponentially distributed because the exponential distribution has the memoryless property Let T i be the continuous random variable for the time until the i + 1st event See Figure 5 Theorem 5 (Interevent Time) Assume j n p jn( t) = α(n) t + o( t) Then the cumulative distribution function for the interevent time T i is F i (t) = 1 exp( α(n)t) with mean and variance µ Ti = 1 α(n) and σ 2 T i = 1 [α(n)] 2 Theorem 6 (Interevent Time Simulation) Let U U[0, 1] be the uniform distribution on [0,1] and T i the continuous random variable for interevent time with state space [0, ) Then T i = Fi 1 (U) = ln(u) α(n) Simple Birth and Death Markov Chain: In the simple birth and death process, an event can be a birth or a death Let X(0) = N The infinitesimal transition probabilities are p i+j,i ( t) = Prob{ X(t) = j X(t) = i} µi t + o( t), j = 1 λi t + o( t), j = 1 = 1 (λ + µ)i t + o( t), j = 0 o( t), j 1, 0, 1 Use two random numbers, u 1 and u 2, from the uniform distribution U(0, 1) to determine the interevent time and the state to which the process jumps In MATLAB, indices begin from 1, so instead of writing t(0), we use t(1) Consider the simple birth and death chain, in a MATLAB program, t(1) = 0, and the time to the next event is t(2) = t(1)+ln(u 1 )/(α(n)), where α(n) = λn+µn, given the process is in state n Since there are two events, to determine whether there is a birth or a death, the unit interval is divided into two subintervals, one subinterval has probability λ/(λ + µ) and the other has probability µ/(λ + µ) Generate a uniform random number u 2 If u 2 < λ/(λ + µ), then this random number lies in the first subinterval and there is a birth, otherwise if u 2 > λ/(λ + µ), the random number lies in the second subinterval and there is a death This concept can be easily extended to k > 2 events In a MATLAB program with k events the unit interval must be divided into k subintervals, each with predetermined probability for i = 1,, k that depends on the current state and the transition probabilities For example, suppose there are four events with the following rates a i (n), i = 1, 2, 3, 4 which depend on the current state n The probabilities of these four events are a i (n)/a(n), a(n) = i a i(n), i = 1, 2, 3, 4 The subinterval [0, 1] is subdivided into four subintervals with the following endpoints: Therefore, in a MATLAB program: 0, a 1 a, a 1 + a 2, a a 1 + a 2 + a 3, 1 a if u2<a1/a, then event 1 occurs elseif u2>=a1/a & u2<(a1+a2)/a, then event 2 occurs elseif u2>=(a1+a2)/2 & u2<(a1+a2+a3)/a, then event 3 occurs else u2>=(a1+a2+a3)/a, then event 4 occurs The subintervals change each time the process changes state n If the number of events are large, deciding which event occurs can become quite lengthy and there are ways to speed up the process of selecting a particular event 10

11 %MatLab program: simple birth and death process clear all x0=5; b=1; d=05; % initial and parameter values for j=1:3 % Three sample paths clear x t n=1; t(1,j)=0; x(n)=x0; % starting values while x(n)>0 & x(n)<50; % continue until the process hits zero or reaches 50 u1=rand; u2=rand; % two uniform random numbers t(n+1,j)=-log(u1)/(b*x(n)+d*x(n))+t(n,j); if u2< b/(b+d); x(n+1)=x(n)+1; else x(n+1)=x(n)-1; end n=n+1; end s=stairs(t(:,j),x, r-, Linewidth,2); hold on end xlabel( Time ); ylabel( Population size ); hold off Figure 7: Three sample paths of the simple birth and death process, X(0) = 5, λ = b = 1, µ = d = Population size Time 11

12 Lecture 6: Continuous-Time Birth and Death Processes: The generator matrix Q for a general birth and death process is either λ 0 µ λ 0 λ 1 µ 1 µ 2 0 Q = 0 λ 1 λ 2 µ 2 µ 3, 0 0 λ 2 λ 3 µ 3 or for a finite state space, λ 0 µ λ 0 λ 1 µ 1 µ λ 1 λ 2 µ 2 0 Q = µ N µ N The simple birth, simple death, simple birth and death, and simple birth, death, and immigration processes are linear in the rates, λ i = λi + ν and µ i = µi Applying the forward Kolmogorov differential equations, dp = Qp, first-order partial differential equations for the pgf and mgf can be derived Applying the method of characteristics to the first-order dt partial differential equations, explicit expressions can be found for the pgf and mgf of these processes Denote the pgf for these simple process as follows (note change in notation z and θ because t is time): M(θ, t) = P(z, t) = p i (t)z i i=0 p i (t)e iθ = P(e θ, t) i=0 Simple Birth, Death, and Immigration Process: Let X(0) = N The infinitesimal transition probabilities are p i+j,i ( t) = Prob{ X(t) = j X(t) = i} µi t + o( t), j = 1 (ν + λi) t + o( t), j = 1 = 1 [ν + (λ + µ)i] t + o( t), j = 0 o( t), j 1, 0, 1 The forward Kolmogorov differential equations are dp i dt dp 0 dt = [λ(i 1) + ν]p i 1 + µ(i + 1)p i+1 (λi + µi + ν)p i = νp 0 + µp 1 for i = 1, 2, with initial condition X(0) = N Applying the generating function technique, it follows that the mgf M(θ, t) is a solution of the following first-order partial differential equation M t = [ ] M λ(e θ 1) + µ(e θ 1) θ + ν(eθ 1)M 12

13 with initial condition M(θ, 0) = e Nθ The preceding differential equation is first-order because the rates are linear The mgf is the solution of this first-order partial differential equation, M(θ, t) = (λ µ)ν/λ [ µ(e (λ µ)t 1) e θ (µe (λ µ)t λ) ] N [ (λe (λ µ)t µ) λ(e (λ µ)t 1)e θ] N+ν/λ The moments E(X n (t)) of the probability distribution X(t) can be found by differentiating the mgf with respect to θ and evaluating at θ = 0: E(X n (t)) = n M(θ, t) θ n θ=0 Table 3: Mean, variance, and pgf for the simple birth, simple death, and simple birth and death processes, where X(0) = N and ρ = e (λ µ)t, λ µ Simple Simple Simple Birth Death Birth and Death m(t) Ne λt Ne µt Ne (λ µ)t σ 2 (t) Ne 2λt (1 e λt ) Ne µt (1 e µt ) N λ + µ ρ(ρ 1) λ µ (pz) N (1 z(1 p)) N (1 p + pz) N P(z, t) Negative binomial p = e λt Binomial b(n,p) p = e µt ( ρ 1 ) N (λz µ) µ(z 1) ρ 1 (λz µ) λ(z 1) Table 4: Mean, variance, and pgf X(0) = N and ρ = e (λ µ)t, λ µ for the simple birth and death with immigration process, where m(t) Simple Birth and Death with Immigration ρ[n(λ µ) + ν] ν λ µ σ 2 (t) N (λ2 µ 2 )ρ[ρ 1] µ + ρ(λρ µ λ) (λ µ) 2 + ν (λ µ) 2 P(z, t) (λ µ) ν/λ [µ(ρ 1) z(µρ λ)] N [λρ µ λ(ρ 1)z] N+ν/λ For each of the preceding simple birth and death processes, the probability of extinction p 0 (t) can be found by evaluating the pgf at z = 0, P(0, t), and the probability of ultimate extinction by taking the limit: lim t p 0 (t) Except for the stochastic process with immigration, the other birth and death 13

14 processes either hit zero or approach infinity For example, given X(0) = N, for the simple birth process, p 0 (t) = 0, simple death process p 0 (t) = (1 e µt ) N and for the simple birth and death process: lim p 0(t) = t { ( µ ) N, λ λ > µ 1, λ µ dn Logistic Growth Model: (1 dt = rn n ) What is the birth rate and what is the death rate? There K are an infinite number of choices for stochastic birth and death rates that yield the same deterministic logistic growth model The birth and death rates should have the form so that λ n µ n = rn(1 n/k) leads to λ i = b 1 i + b 2 i 2 > 0 and µ i = d 1 i + d 2 i 2 > 0 dn dt = (b 1 d 1 )n + (b 2 d 2 )n 2 = rn r K n2, b 1 d 1 = r > 0 and b 1 d 1 d 2 b 2 = K > 0 Define the birth and death rates, λ i and µ i, as follows: (a) λ i = i and µ i = i2, i = 0, 1, 2, 10 (b) λ i = i i2, i = 0, 1,, and µ i = i2, i = 0, 1, 2, 0, i > In both cases the deterministic model is dn (1 dt = n n ), 10 where r = 1 and K = 10 In the deterministic model, solutions approach the carrying capacity K = 10 Three sample paths for models (a) and (b) are graphed in Figure 8 20 (a) 20 (b) Population size Population size Time Time Figure 8: Three sample paths of the stochastic logistic model for cases (a) and (b) with X(0) = 10 For these stochastic logistic models, lim p 0(t) = 1 t 14

15 For a population of finite size N, we will show that the mean time to extinction is a solution of the following matrix equation, τ Q = 1 where Q is the truncated generator matrix without state zero, τ = (τ 1,, τ N ), τ k is the mean time until extinction from state k, τ = 1 Q 1 Let λ i = i i2 N, i = 0, 1,, N and µ i = i2, i = 0, 1, 2,, N 0, i > N N The intrinsic growth rate r = 1 and the carrying capacity K = N/2 The expected time to extinction can be calculated for X(0) = m, m = 1, 2,, N by solving τ Q = 1 When N = 10, the carrying capacity is K = 5, and when N = 20, the carrying capacity is K = 10 The time units depend on the particular problem If the time is measured in days, then for the first example, extinction occurs, on the average, in less than one year but for the second example, the mean time to extinction is over 300 years Hence, for even larger values of K, the mean time to extinction will take much longer; the convergence of p 0 (t) to 1 is slow x 105 Expected time to extinction K = 5 Expected time to extinction K = Initial population size Initial population size Figure 9: Expected time until extinction in the stochastic logistic model with K = 5 and K = 10 15

16 Lecture 7 Biological Applications of CTMC Models: Let P(z, t) be the pgf for X(t), and f(t) be the offspring pgf, P(z, t) = p 0 (t) + p 1 (t)z + p 2 (t)z 2 + f(z) = p 0 + p 1 z + p 2 z 2 + As in discrete-time branching processes, the pgf P(z, t + t) is a composition of pgfs At time t = 0, P(z, 0) = z or X(0) = 1 The same theorem applies to the continuous-time Markov branching process as in discrete-time Theorem 7 Suppose {X(t) : t [0, )} is a nonexplosive, continuous-time Markov branching process with X(0) = 1 Assume f is the pgf of the offspring distribution, where m = f (1) and P(z, t) is the pgf of X(t) [f(0) = p 0 > 0 and p 0 + p 1 < 1] If m 1, then lim Prob{X(t) = 0} = lim p 0(t) = 1 t t and if m > 1, then there exists a q satisfying f(q) = q such that lim Prob{X(t) = 0} = lim p 0(t) = q < 1 t t Simple Birth and Death Process: Let λ > 0 be the per capita birth rate and µ > 0 be the per capita death rate of an individual, p i+1,i ( t) = λi t + o( t) and p i 1,i ( t) = µi t + o( t) The offspring pgf is f(z) = µ λ + µ + λ λ + µ z2 An individual either dies or survives and gives birth There is an important difference in this offspring pgf and the one for the discrete-time model For this pgf, a single offspring does not replace the parent but adds to the population size Also, m = f (1) = 2λ/(µ + λ) > 1 if λ > µ In this case, the fixed point q of f(q) = q is µ/λ The pgf of X(t) for the simple birth and death process is given in Table 3 SIS Epidemic Model ds dt = β SI + γi N di dt = β SI γi = N (β SN ) ( γ I = β γ β I ) I N because S + I = N The value R 0 = β is called the basic reproduction number This system has two γ equilibria, ( S, Ī) = (N, 0) and (N/R 0, N(1 1/R 0 ) if R 0 > 1 If R 0 > 1, then there is an outbreak and solutions approach the endemic equilibrium and if R 0 1, there is no outbreak For the CTMC SIS epidemic model, let Prob{ I(t) = j I(t) = i} β i(n i) t + o( t), j = 1 N γi t [ + o( t), ] j = 1 = β 1 i(n i) + γi t N +o( t), j = 0 o( t), j 1, 0, 1, 16

17 where i {0, 1,, N} The SIS epidemic model is a birth and death process with λ i = β N i(n i) and µ i = γi, for i = 0, 1,, N There is a single absorbing state at zero, lim t p 0 (t) = 1 However, before complete extinction, which may take a long time, I(t) behaves like a simple birth and death process Consider only the state I when S N, then di = βi γi = (β γ)i dt Thus, if N is large, I behaves like a simple birth and death process, where β = λ and γ = µ Thus, using branching process theory or the simple birth and death process we can approximate the dynamics at the beginning of an epidemic What is the probability that I hits zero (no outbreak)? The probability of absorption from branching process theory tells us that the probability of absorption as t (no outbreak) given I(0) = 1 is µ/λ = γ/β = 1/R 0 if R 0 > 1 If I(0) = k, then the probability of no outbreak is (1/R 0 ) k The process either hits zero rapidly or grows Even though this is an asymptotic approximation, it is a good for prediction of probability of no outbreak See Exercise 15 for the SIR epidemic model Lectures 8 & 9 Stochastic Epidemic Models, DTMC, CTMC and SDE Models See Chapter on Stochastic Epidemic Models 17