Probability Distributions

Transcription

1 Lecture : Background in Probability Theory Probability Distributions The probability mass function (pmf) or probability density functions (pdf), mean, µ, variance, σ 2, and moment generating function (mgf) M(t) for some well-known discrete and continuous probability distributions Discrete Distributions Discrete Uniform: f(x) {, n x, 2,, n, 0, otherwise µ + n, σ 2 n2 2 2, M(t) e(n+)t e t n(e t ), t 0 Geometric: f(x) { p( p) x, x 0,, 2,, 0, otherwise, where 0 < p < µ p p, σ2 p p 2, M(t) Binomial b(n, p): f(x) ( ) n 0 < p < The notation x p ( p)e t ( ) n p x ( p) n x, x 0,, 2,, n, 0, otherwise, n! x!(n x)! where n is a positive integer and µ np, σ 2 np( p), M(t) ( p + pe t ) n where n is a positive inte- ( ) x + n p Negative Binomial: f(x) n n ( p) x, x 0,, 2,, 0, otherwise, ger and 0 < p < µ n( p), σ 2 n( p) p p 2, M(t) p n [ ( p)e t ] n λ x e λ Poisson, P o(λ): f(x), x 0,, 2,, x! 0, otherwise, µ λ, σ 2 λ, M(t) e λ(et ) where λ is a positive constant Continuous Distributions Uniform U(a, b): f(x) b a, a x b, 0, otherwise, where a < b are constants µ a + b 2, σ2 (b a)2, M(t) ebt e at 2 t(b a), t 0

2 Gamma: f(x) 0 Γ(α)β α xα e x/β, x 0, 0, x < 0, β α xα e x/β dx For a positive integer n, Γ(n) (n )! µ αβ, σ 2 αβ 2, M(t) ( βt) α, t < /β where α and β are positive constants and Γ(α) Exponential: f(x) { λe λx, x 0, 0, x < 0, where λ is a positive constant µ λ, σ2 λ 2, M(t) λ λ t, t < λ Normal, N(µ, σ 2 ): f(x) ( ) σ 2π exp (x µ)2, < x <, where µ and σ are constants 2σ 2 E(X) µ, V ar(x) σ 2, M(t) e µt+σ2 t 2 /2 The probability generating function (pgf) is P X (t) E(t X ), moment generating function (mgf) is M X (t) E(e tx ) and cumulant generating function (cgf) is K X (t) ln(m X (t)) The mean µ X E(X) and variance σ 2 X E(X2 ) E 2 (X) computed from the pgf, P X (t), mgf, M X (t) and cgf, K X (t): µ X P X() M X(0) K X(0) and P X () + P X () [P X ()]2 σx 2 M X (0) [M X (0)]2 K X (0) Continuous-time Markov Chain Example, Simple Birth Process: The per capita rate of birth is λ Population size Time Figure : Three sample paths (stochastic realizations) of a simple birth process, X(0) ; n(t) e t is the dashed curve See Table 2

3 Table : For two stochastic realizations, the times at which a birth occurs for a simple birth process Realization Realization 2 Size X(t) Event Time t Size X(t) Event Time t Lecture 2: Discrete-Time Markov Chains (DTMCs) Transition probabilities: p ji (n) Prob{X n+ j X n i} If p ji (n) does not depend on n, then the process is said to be time homogeneous The transition matrix of a DTMC {X n } n0 with state space {, 2, } and one-step transition probabilities, {p ij} i,j, is denoted as P (p ij ), where p p 2 p 3 p 2 p 22 p 23 P p 3 p 32 p 33 The column sums equal one, a stochastic matrix, j p ji The n-step transition matrix P n (p (n) The probabilities p i (n) Prob{X n i}, p i (n + ) j p ijp j (n), i, 2 p(n + ) P p(n) See the directed graph in Figure 2 and the corre- Random walk model with absorbing barriers sponding (N + ) (N + ) transition matrix: q q p p P q p p ij ) The Markov chain, graphed in Figure 2, has three communication classes: {0}, {, 2,, N }, and {N} The Markov chain is reducible States 0 and N are absorbing; the remaining states are transient 3

4 0 2 N Figure 2: Probability of moving to right is p and to the left is q, p + q Boundaries 0 and N are absorbing, p 00 p NN (random walk with absorbing barriers or gambler s ruin problem) A DTMC is irreducible if its digraph is strongly connected Otherwise it is called reducible An irreducible DTMC can be positive recurrent or null recurrent or transient It may also be classified as periodic or aperiodic Recurrence is defined for each state i in the chain Recurrence means for each state i if the process leaves state i it will return to state i at some future time If not, the state is transient A state i is positive recurrent if the mean recurrence time (µ ii mean return time) is finite A state i with an infinite mean recurrence time is called null recurrent be a recurrent, ir- Theorem (Basic Limit Theorem for aperiodic Markov chains) Let {X n } n0 reducible, and aperiodic DTMC with transition matrix P (p ij ) Then lim n p(n) ij, µ ii where µ ii is the mean recurrence time for state i and i and j are any states of the chain [If µ ii, then lim n p (n) ij 0] be a recurrent, irre- Theorem 2 (Basic Limit Theorem for periodic Markov chains) Let {X n } n0 ducible, and d-periodic DTMC, d >, with transition matrix P (p ij ) Then lim n p(nd) ii d µ ii and p (m) ii 0 if m is not a multiple of d, where µ ii is the mean recurrence time for state i [If µ ii, then lim n p (nd) ii 0] Example of Genetics of Inbreeding: Two alleles A and a There are six possible breeding pairs which denote the six states of the DTMC, : AA AA, 2 aa aa, 3 Aa Aa, 4 Aa aa, 5 AA aa, 6 AA Aa Inbreeding of the first two types results in offspring of the same genotypes and inbreeding in the next generation will be of the same type; they are absorbing states The remaining states, 3,4,5,6 are transient The transition matrix has the following form: 0 /6 0 0 /4 0 /6 /4 0 0 ( ) P 0 0 /4 /4 /4 I A 0 0 /4 /2 0 0 O T 0 0 / /4 0 0 /2 Probability of absorption into ( states or 2 from states 3, 4, 5, 6 is computed from the fundamental matrix, I (A + AT + AT (I T ), lim n P n 2 ) ( ) + ) I A(I T ) Thus, A(I T ) O O O O is the probability of absorption into states or 2 (fixation) from states 3, 4, 5 or 6 Let the matrix E be a 2 2 matrix of ones, then E(I T ) is the mean time until absorption from states 3, 4, 5, or 6 4

5 Lecture 3: Discrete-Time Branching Processes Figure 3: Sample path of a branching process {X n } n0 In the first generation, four individuals are born, X 4 The four individuals in generation one give birth to three, zero, four, and one individuals, respectively, making a total of eight individuals in generation two, X 2 8 Generating functions rather than transition matrices are useful in analysis of branching processes Offspring pgf: f(t) p k t k k0 Recall f() and m f () is the mean number of offspring The branching process is called subcritical if m <, critical if m, and supercritical if m > Theorem 3 (Branching Process Theorem) Let X 0 Assume f(0) p 0 > 0 and p 0 + p < (i) If m, then lim n Prob{X n 0} (ii) If m >, then lim n Prob{X n 0} q, where q f(q) is the unique fixed point in the interval (0, ) If X 0 N and m >, then lim n p 0 (n) lim n Prob{X n 0} q N The conditional expectation E(X n+ X n ) mx n, E(X n+ ) me(x n ) Multitype Branching Process, n different types: (X,, X n ) The offspring random variable of type i is Y i Offspring pgfs: f i (t,, n) s n s P i (s,, s n )t s tsn n P i (s,, s n ) Prob{Y s,, Y n s n } Expectation matrix M (m ij ), where m ji f t j t,,t n Assume M is irreducible Denote the spectral radius of M as ρ(m), the maximum modulus of the eigenvalues of MThe multitype branching process is called subcritical if ρ(m) <, critical if ρ(m) and supercritical if ρ(m) > 5

6 Lecture 4: Continuous-Time Markov Chains (CTMCs), Introduction Discrete random variable X(t), t [0, ) Probabilities p i (t) Prob{X(t) i} Transition probability: p ji (t, s) Prob{X(t) j X(s) i}, s < t We will assume time-homogenous transition probabilities p ji (t, s) p ji (t s) The transition matrix is a stochastic matrix: where p 00 (t) p 0 (t) P (t) (p ij (t)) p 0 (t) p (t), p ji ( t) δ ji + q ji t + o( t) is an infinitesimal transition probability The infinitesimal generator matrix: q 00 q 0 Q (q ij ) q 0 q P (t) I, Q lim t 0 t The column sums of Q equal zero dp (t) Forward Kolmogorov differential equations: QP (t) dt dp (t) Backward Kolmogorov differential equations: P (t)q dt The embedded DTMC is used to define irreducible, recurrent, and transient states or chains for the associated CTMC Let Y n denote the random variable for the state of a CTMC {X(t) : t [0, )} at the time of the nth jump, Y n X(W n ) (See Figure 4) The set of discrete random variables {Y n } 0 is the embedded Markov chain T0 T T2 T3 0 W W2 W3 W4 Figure 4: Sample path of a CTMC, illustrating waiting times {W i } and interevent times, {T i } A CTMC is irreducible, recurrent or transient if the corresponding embedded Markov chain has these properties Some differences in the dynamics of a CTMC as opposed to a DTMC are the possibility of a finite-time blow up in a CTMC (explosive process) and the fact that CTMC are not periodic See Figure 5 The embedded MC cannot be used to classify chains as positive recurrent or null recurrent This latter classification depends on the mean recurrence time µ ii 6

7 0 W W2 W3 W4 W Figure 5: One sample path of a continuous time Markov chain that is explosive Theorem 4 (Basic Limit Theorem for CTMCs) If the CTMC {X(t) : t [0, )} is nonexplosive and irreducible, then for all i and j, lim p ij(t), () t q ii µ ii where µ ii is the mean recurrence time, 0 < µ ii In particular, a finite, irreducible CTMC is nonexplosive and the limit () exists and is positive If the DTMC is nonexplosive and positive recurrent, it has a limiting positive stationary distribution π satisfying Qπ 0 Poisson process with X(0) 0, p i+,i ( t) λ t + o( t) and p i (t) e λt (λt) i /i! has generator matrix λ 0 0 λ λ 0 Q 0 λ λ The associated embedded Markov chain has a transition matrix T 0 0 The Poisson process is transient A finite CTMC with two states {, 2} The generator matrix ( ) a b Q, a, b > 0 a b The CTMC is irreducible and positive recurrent The limiting stationary distribution can be found from the forward Kolmogorov differential equations by solving for the stationary distribution Qπ 0 In this case π (b/(a + b), a/(a + b)) tr The mean recurrence times are µ ii a + b, i, 2 ab 7

8 Lecture 5: Continuous-Time Markov Chains (CTMCs), Interevent Time To generate sample paths, we must know the time between jumps and the state to which the process jumps The Markov assumption implies the interevent time is exponentially distributed because the exponential distribution has the memoryless property Let T i be the continuous random variable for the time until the i + st event See Figure 4 Theorem 5 (Interevent Time) Assume j n p jn( t) α(n) t + o( t) Then the cumulative distribution function for the interevent time T i is F i (t) exp( α(n)t) with mean and variance µ Ti α(n) and σ 2 T i [α(n)] 2 Theorem 6 (Interevent Time Simulation) Let U U[0, ] be the uniform distribution on [0,] and T i the continuous random variable for interevent time with state space [0, ) Then T i Fi (U) ln(u) α(n) Simple Birth and Death Markov Chain: In the simple birth and death process, an event can be a birth or a death Let X(0) N The infinitesimal transition probabilities are p i+j,i ( t) Prob{ X(t) j X(t) i} µi t + o( t), j λi t + o( t), j (λ + µ)i t + o( t), j 0 o( t), j, 0, Use two random numbers, u and u 2, from the uniform distribution U(0, ) to determine the interevent time and the state to which the process jumps In MATLAB, indices begin from, so instead of writing t(0), we use t() Consider the simple birth and death chain, in a MATLAB program, t() 0, and the time to the next event is t(2) t()+ln(u )/(α(n)), where α(n) λn+µn, given the process is in state n Since there are two events, to determine whether there is a birth or a death, the unit interval is divided into two subintervals, one subinterval has probability λ/(λ + µ) and the other has probability µ/(λ + µ) Generate a uniform random number u 2 If u 2 < λ/(λ + µ), then this random number lies in the first subinterval and there is a birth, otherwise if u 2 > λ/(λ + µ), the random number lies in the second subinterval and there is a death This concept can be easily extended to k > 2 events In a MATLAB program with k events the unit interval must be divided into k subintervals, each with predetermined probability for i,, k that depends on the current state and the transition probabilities For example, suppose there are four events with the following rates a i (n)m, i, 2, 3, 4 which depend on the current state n The probabilities of these four events are a i (n)/a(n), a(n) i a i(n), i, 2, 3, 4 The subinterval [0, ] is subdivided into four subintervals with the following endpoints: Therefore, in a MATLAB program: 0, a a, a + a 2, a a + a 2 + a 3, a if u2<a/a, then event occurs elseif u2>a/a & u2<(a+a2)/a, then event 2 occurs elseif u2>(a+a2)/2 & u2<(a+a2+a3)/a, then event 3 occurs else u2>(a+a2+a3)/a, then event 4 occurs The subintervals change each time the process changes state n If the number of events are large, deciding which event occurs can become quite lengthy and there are ways to speed up the process of selecting a particular event 8

9 %MatLab program: simple birth and death process clear all x05; b; d05; % initial and parameter values for j:3 % Three sample paths clear x t n; t(,j)0; x(n)x0; % starting values while x(n)>0 & x(n)<50; % continue until the process hits zero or reaches 0 urand; u2rand; % two uniform random numbers t(n+,j)-log(u)/(b*x(n)+d*x(n))+t(n,j); if u2< b/(b+d); x(n+)x(n)+; else x(n+)x(n)-; end nn+; end sstairs(t(:,j),x, r-, Linewidth,2); hold on end xlabel( Time ); ylabel( Population size ); hold off Figure 6: Three sample paths of the simple birth and death process, X(0) 5, λ b, µ d Population size Time Lecture 6: Continuous-Time Birth and Death Processes: The simple birth, simple death, simple birth and death, and simple birth, death, and immigration processes are linear in the rates, λi + ν, µi From the forward Kolmogorov differential equations, first order partial differential equations for the pgf and mgf can be derived Applying the method of characteristics to the first-order partial differential equations, explicit expressions can be found for the pgf and mgf of these processes Denote the pgf for these simple process as M(θ, t) P(z, t) p i (t)z i i0 p i (t)e iθ P(e θ, t) i0 9

10 Simple Birth, Death, and Immigration Process: Let X(0) N The infinitesimal transition probabilities are p i+j,i ( t) Prob{ X(t) j X(t) i} µi t + o( t), j (ν + λi) t + o( t), j [ν + (λ + µ)i] t + o( t), j 0 o( t), j, 0, The forward Kolmogorov differential equations withx(0) N are dp i dt dp 0 dt [λ(i ) + ν]p i + µ(i + )p i+ (λi + µi + ν)p i νp 0 + µp for i, 2, with initial conditions p i (0) δ in Applying the generating function technique, it follows that the mgf M(θ, t) is a solution of the following first-order partial differential equation M t [ ] M λ(e θ ) + µ(e θ ) θ + ν(eθ )M with initial condition M(θ, 0) e Nθ The preceding differential equation is first-order because the rates are linear The mgf is the solution of this first-order partial differential equation, M(θ, t) (λ µ)ν/λ [ µ(e (λ µ)t ) e θ (µe (λ µ)t λ) ] N [ (λe (λ µ)t µ) λ(e (λ µ)t )e θ] N+ν/λ The moments E(X n (t)) of the probability distribution X(t) can be found by differentiating the mgf with respect to θ and evaluating at θ 0: E(X n (t)) n M(θ, t) θ n θ0 Table 2: Mean, variance, and pgf for the simple birth, simple death, and simple birth and death processes, where X(0) N and ρ e (λ µ)t, λ µ Simple Simple Simple Birth Death Birth and Death m(t) Ne λt Ne µt Ne (λ µ)t σ 2 (t) Ne 2λt ( e λt ) Ne µt ( e µt ) N λ + µ ρ(ρ ) λ µ (pz) N ( z( p)) N ( p + pz) N P(z, t) Negative binomial p e λt Binomial b(n,p) p e µt ( ρ ) N (λz µ) µ(z ) ρ (λz µ) λ(z ) 0

11 Table 3: Mean, variance, and pgf X(0) N and ρ e (λ µ)t, λ µ for the simple birth and death with immigration process, where m(t) Simple Birth and Death with Immigration ρ[n(λ µ) + ν] ν λ µ σ 2 (t) N (λ2 µ 2 )ρ[ρ ] µ + ρ(λρ µ λ) (λ µ) 2 + ν (λ µ) 2 P(z, t) (λ µ) ν/λ [µ(ρ ) z(µρ λ)] N [λρ µ λ(ρ )z] N+ν/λ