Birth-Death Processes

Birth-Death Processes Birth-Death Processes: Transient Solution Poisson Process: State Distribution Poisson Process: Inter-arrival Times Dr Conor McArdle EE414 - Birth-Death Processes 1/17

Birth-Death Processes Birth-Death Processes are Markov chains where transitions are allowed only between neighbouring states of the process. They may be discrete- or continuous-time. Our primary interest is in the continuous-time case. Birth-Death processes are used to simplify the analysis of systems that see random arrivals and departures of customers (packets, requests, or calls etc.) over time. We will develop results that are useful for performance evaluation of servers (or transmission lines) with associated input queues. The results also apply to circuit switches with limited numbers of outgoing channels. We start in this section by seeking the transient solution to the Birth-Death system. Recall that the general transient solution π(t) of a continuous-time Markov chain is given implicitly by π(t) = π(t)q where Q is the matrix of state transition-rates between each pair of states in the process. The Birth-Death assumption imposes a certain structure on Q which gives the solution π(t) a simplified form. Dr Conor McArdle EE414 - Birth-Death Processes 2/17

Birth-Death Processes: Transition-rates Consider the state transition-rate diagram of the general Birth-Death process below. ë 0 ë 1 ë k-1 ë k 0 1 2 k-1 k k+1 ì 1 ì 2 ì k ì k+1 The states of the process are represented by numbered circles, the number indicating the state. For a birth-death process this number represents the number of individuals in the population, which for our purposes we interpret as the number of packets (or requests or calls) in the system. Arrows represent the possible transitions between states and the labels on the arrows are the state transition-rates between states. Transitions which relate to births (an increase in the population by one) are labeled with the birth rate (λ k = birth rate when process is in state k). Transitions which relate to deaths (a decrease in the population by one) are labeled with the death rate (µ k = death rate when process is in state k). Dr Conor McArdle EE414 - Birth-Death Processes 3/17

Birth-Death Processes: Transient Solution Let us construct the matrix of transition rates Q from the Birth-Death state transition-rate diagram. The component q ij of Q is the transition rate from state i to state j (i j). For j = i + 1 the rate is given by the birth-rate λ i and for j = i 1 the rate is given by the death-rate µ i. Given the stipulation that transitions are only allowed between neighbouring states, all rates for larger transition steps in the process (e.g. q ij where j = i + 2) are zero. The rates at each state i must balance, that is j q ij = 0, so q ii = j,i j q ij. q ii may be interpreted as the total rate at which state i is exited to other states. Forming the matrix Q from the above rates gives: λ 0 λ 0 0 0... µ 1 (λ 1 + µ 1 ) λ 1 0... Q = 0 µ 2 (λ 2 + µ 2 ) λ 2... 0 0 µ 3 (λ 3 + µ 3 ).......... Dr Conor McArdle EE414 - Birth-Death Processes 4/17

Birth-Death Processes: Transient Solution With this Q matrix, the transient solution equation π(t) = π(t)q, can be written as the set of differential equations: dπ 0 (t) dt dπ k (t) dt = λ 0 π 0 (t) + µ 1 π 1 (t) k = 0 (1) = λ k 1 π k 1 (t) (λ k + µ k )π k (t) + µ k+1 π k+1 (t) k 1 (2) The solution to these equations is not trivial and, for general sets of {λ k } and {µ k }, it is often the case that numerical methods are required to find a solution. However, we will consider an important Birth-Death process, the Poisson process, for which we can easily find an exact (analytic) solution. The Poisson process is used extensively to model the arrival times of packets (or calls) to a system. Dr Conor McArdle EE414 - Birth-Death Processes 5/17

Birth-Death Processes Birth-Death Processes: Transient Solution Poisson Process: State Distribution Poisson Process: Inter-arrival Times Dr Conor McArdle EE414 - Birth-Death Processes 6/17

The Poisson Process Consider a pure birth system where µ k = 0 for all k and λ k = λ for all k. Thus our transition rate matrix Q is: λ λ 0 0... 0 λ λ 0... 0 0 λ λ... 0 0 0 λ.......... and the system of equations π(t) = π(t)q is: dπ 0 (t) = λπ 0 (t) dt k = 0 (3) dπ k (t) = λπ k 1 (t) λπ k (t) dt k 1 (4) We assume the initial condition (initial state distribution) for the system is: { 1 k = 0 π k (0) = 0 k 0 Dr Conor McArdle EE414 - Birth-Death Processes 7/17

The Poisson Process Thus (from the usual differenetial equation solution methods) the solution for π 0 (t) is π 0 (t) = e λt. Using this solution in equation (4) for k = 1 gives: dπ 1 (t) dt The solution to this differential equation is: = λe λt λπ 1 (t) π 1 (t) = λte λt Continuing in this manner we have an expression for the transient solution of the pure birth process. π k (t) = (λt)k e λt k 0, t 0 (5) k! This pure-birth process is called the Poisson Process and equation (5) is the pmf of the state probabilities at time t (often called the Poisson distribution). Dr Conor McArdle EE414 - Birth-Death Processes 8/17

The Poisson Process At some fixed time τ, equation (5) gives the state distribution of the process at that time (a pmf). For some chosen state n, equation (5) gives the probability of being in that state as time progresses from 0 to (a pdf). ¼ 0 1 2 3 4 Figure: The Family of Curves π k (t) as a function of k and λt Dr Conor McArdle EE414 - Birth-Death Processes 9/17

The Poisson Process: Interpretation/Uses The Poisson process is widely used for modelling the arrival process of packets (or calls) to a system. If we assume arrivals of packets to a system are independent events and that each arrival occurs at the same average rate λ, then we have a Poisson process, where the state of the process corresponds to the number of arrivals since time 0. Thus equation (5) gives a complete probabilistic description of the number of arrivals during a given time period and also a probabilistic description of how long the process stays in any one state, or alternatively, how long we expect between arrivals or until the next arrival. We can note that a Poisson process arises from combining an infinite number of independent stationary (renewal) processes, each generating arrivals (births) at some rate. In the limit, the overall arrival process has a constant arrival rate (birth rate) λ. When we have any large population of such independent processes (e.g. a large number of people making phone calls through a switch), the Poisson process provides a very useful approximation to the overall arrival process. We will now examine the previously mentioned probabilistic descriptions. Dr Conor McArdle EE414 - Birth-Death Processes 10/17

The Poisson Process: Moments Consider the random variable N(t), associated with the Poisson process, describing the number of arrivals (births) in the interval [0, t) The pmf of N(t) is the state distribution of the Poisson process at time t, that is, the pmf of N(t) is p k (t) = π k (t) = (λt)k e λt k 0, t 0 k! We would expect that the average number of arrivals in this interval would be λt, given that λ is the arrival rate. So the expected value of N(t) should be λt. We can check this: E[N(t)] = kp k (t) k=0 = e λt k=0 k (λt)k k! = e λt k=1 (λt) k (k 1)! = e λt λt Noting that the Taylor series expansion of e x is: e x = 1 + x + x2 2! +... E[N(t)] = λt (λt) k k=0 k! Dr Conor McArdle EE414 - Birth-Death Processes 11/17

The Poisson Process: Moments We may also calculate the variance of the number of arrivals in [0, t). We first calculate: E[N(t) (N(t) 1)] = k(k 1)p k (t) k=0 = e λt k=0 = e λt (λt) 2 k(k 1) (λt)k k! k=2 (λt) k 2 (k 2)! = e λt (λt) 2 (λt) k = (λt) 2 k=0 Using this result and the expression for E[N(t)] we can calculate: Var(N(t)) = E[N 2 (t)] E 2 [N(t)] = E[N(t) (N(t) 1)] + E[N(t)] (E[N(t)]) 2 = (λt) 2 + λt (λt) 2 Var(N(t)) = λt k! Dr Conor McArdle EE414 - Birth-Death Processes 12/17

Birth-Death Processes Birth-Death Processes: Transient Solution Poisson Process: State Distribution Poisson Process: Inter-arrival Times Dr Conor McArdle EE414 - Birth-Death Processes 13/17

The Poisson Process: Inter-arrival Times We may also calculate the distribution of the times between arrivals (births) for a Poisson process. Let τ be the random variable describing the time between adjacent arrivals (the inter-arrival time) for a Poisson process. We may set t = 0 when an arrival occurs and so, equivalently, τ is the random variable describing the time until the next arrival. Arrivals ¼ time We know the probability of having no arrivals in [0, t), that is Pr(τ > t), from the Poisson state distribution: π 0 (t) = e λt = Pr(τ > t) so the probability of the complementary event (τ t) is 1 e λt. This gives the cdf of τ, the inter-arrival time: F τ (t) = Pr(τ t) = 1 e λt t 0. The pdf of τ is given by the derivative of F τ (t): f τ (t) = λe λt t 0 Dr Conor McArdle EE414 - Birth-Death Processes 14/17

The Poisson Process: Inter-arrival Times We may calculate the expected value of τ as: E[τ] = 0 tf τ (t) dt = 0 tλe λt dt = 1 λ This is the average time between arrivals, which is as expected since the arrival rate is equal to λ. To summarise, a pure-birth process with constant birth rate λ is described by a Poisson process whose mean state is λt. The times between births (arrivals) are exponentially distributed with mean 1/λ. The Poisson process approximates the combined arrival process of a large group of independent arrival processes. Dr Conor McArdle EE414 - Birth-Death Processes 15/17

The Poisson Process : The Memoryless Property Consider the Poisson arrival process where the inter-arrival time is the random variable τ. Suppose that an arrival has occurred at time zero. Let time t 0 pass with no arrivals. What is the probability that the next arrival will occur within t seconds from now? This is the conditional probability: Pr [τ t + t 0 τ > t 0 ] = Pr [t 0 < τ t + t 0 ] Pr [τ > t 0 ] = Pr [τ t + t 0] Pr [τ t 0 ] Pr [τ > t 0 ] = 1 e λ(t+t 0) (1 e λt 0 ) 1 (1 e λt 0 ) = 1 e λt So the distribution of remaining time until the next arrival, given that t 0 seconds has elapsed since the last arrival, is identically equal to the unconditional distribution of the inter-arrival time. This fact is called the memoryless property of the exponential distribution. The exponential distribution is the only continuous distribution with this property. Dr Conor McArdle EE414 - Birth-Death Processes 16/17

Birth-Death Process: Conclusion We have found the form of the general transient solution of the Birth-Death process and found a closed-form solution for the pure birth (Poisson) process. The Poisson process is useful for describing arrival processes. We will next turn to Birth-Death processes which describe queueing systems. The processes we will study have stationary state distributions and we are most interested in performance measures relating to mean values of their stationary distributions. Dr Conor McArdle EE414 - Birth-Death Processes 17/17