Stochastic2010 Page 1 Extinction Probability for Branching Processes Friday, April 02, 2010 2:03 PM Long-time properties for branching processes Clearly state 0 is an absorbing state, forming its own recurrent class. The other states in the state space will generically be in a communication class except for unusual choices of offspring distribution (i.e., only even numbers of offspring). In any case, the states 1,2, are all transient since they have one-way communication with the absorbing state 0. This implies in particular that it isn't sensible to talk about stationary distributions -- what will happen, in any realization, is either the branching process will go extinct or will increase indefinitely. This suggests the following questions are relevant for branching processes, for each choice of offspring probability distribution What is the probability to go extinct (rather than to grow indefinitely)? How long does it take to go extinct when the branching process does go extinct? If the branching process goes extinct, how much cost/reward did the branching process incur until it went extinct? When the branching process grows indefinitely, how fast does it grow? These questions find particular application in, for example epidemiology or technology markets. We'll address the first question in detail; the second and third can be attacked by similar methods. The fourth is trickier. Consider the extinction probability, which is same notion as an absorption probability: The special structure of the branching process, namely the independence of each agent, means that:
Stochastic2010 Page 2 But each agent has equivalent stochastic rules for its descendants so each of these factors is just the probability for a given single agent to go extinct, namely a(1). Use first-step analysis just as we did for general absorption probability calculation but now exploit special branching process structure. (partition on the state of the Markov chain at epoch 1)
Stochastic2010 Page 3 So the extinction probability must satisfy the equation: Does this equation have unique solution, and if not, which solution is the right one? Let's take care of some special cases first: But on the other hand, if This implies that one of the two following graphical possibilities apply:
Stochastic2010 Page 4 We see that the possible solutions to are: a=1 (always a solution) nontrivial solution 0<a<1 when One can show (see the references) that when the nontrivial solution exists, it is the correct value for the extinction probability. Notice that this all says the following: When the mean number of offspring per agent is less than one, the population is guaranteed to go extinct. When the mean number of offspring per agent is greater than or equal to one, the population has some nonnegative probability to go extinct, which can be computed by the solution to a nonlinear equation Continuous-time Markov chains References: Lawler Ch. 3, Karlin & Taylor Ch. 4 We now consider how to modify our modeling and analysis when the transitions in state are allowed to happen at moments along a continuous time (parameter) domain rather than at discrete epochs. But state space is still discrete (possibly countably infinite).
Stochastic2010 Page 5 state space is still discrete (possibly countably infinite). The realizations of a CTMC will be piecewise constant, with the transition times being random and continuously distributed, and the transitions also being random. By convention, the realizations of a CTMC are taken to be right-continuous (cadlag process). Markov property in continuous time: Notions are easier to work with these finite-time representations of the past; if process is nice (cadlag is enough) then can approximate a complete history with a sufficiently fine finite mesh of points -- passing to the limit involves some measure-theoretic technicalities (see filtrations). Just as for discrete-time Markov chains, many powerful formulas become available if we consider (as we will) time-homogenous CTMC In what situations are CTMC used more modeling? First of all, why would one use a CTMC rather than a DTMC? The reason one should ask this is that for any given CTMC X(t), one can construct some associated DTMC: X n = X(n t) Also we can construct the embedded DTMC : (need strong Markov property which says that Markov property (formulated for deterministic times) carries over to randomly chosen times, provided those random times are Markov times, meaning that they are determined only by information available and up to including the actual time.
Stochastic2010 Page 6 So why would a CTMC model be any better than these associated DTMC? one may have a large number of states,some of which are visited sometimes very briefly, and the DTMC obtained from regular time observations may miss it. The embedded DTMC misses timing information; if you augment the embedded DTMC with time spent in each state,then equivalent to a CTMC. CTMC are sometimes good spatial discretizations of continuous-space processes. Simulating a CTMC can actually be easier than simulating a DTMC with regular time steps. Often times in physics and engineering contexts, CTMC models are easier to formulate because it describes instantaneous changes, which can generally be linearly superposed. But finite-time effects generally do not linearly superpose. This explains the kinds of contexts in which one sees CTMCs often used: statistical modeling in atomic and quantum physics chemical reaction networks and biomolecular reaction networks dynamics on more general networks (disease spread on sexual or social networks)
Stochastic2010 Page 7