Stochastic modelling of epidemic spread
|
|
- Mildred Woods
- 5 years ago
- Views:
Transcription
1 Stochastic modelling of epidemic spread Julien Arino Centre for Research on Inner City Health St Michael s Hospital Toronto On leave from Department of Mathematics University of Manitoba Julien Arino@umanitoba.ca 5 March 2014
2 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC 6 ODE to CTMC
3 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC 6 ODE to CTMC
4 Deterministic SIR Use the standard KMK S = βsi I = βsi αi R = αi Basic reproduction number R 0 = β α S 0 If R 0 < 1, no outbreak If R 0 > 1, outbreak Use α = 1/4, N = S + I + R = 100, I (0) = 1, R 0 = {0.5, 1.5} and β so that this is true
5 R 0 = 0.5 R 0 = S S 90 I 90 I R R Time (days) Time (days) Number infectious Number infectious Time (days) Time (days)
6 Stochastic DTMC SIR Consider S(t) and I (t) (R(t) = N S(t) I (t)). Define transition probabilities p (s,i ),(s,i)( t) = P{(S(t + t), I (t + t)) = (s + s, i + i ) (S(t), I (t)) = (s, i)} by βsi t (s, i ) = ( 1, 1) αi t (s, i ) = (0, 1) p (s,i ),(s,i)( t) = 1 [βsi + αi] t (s, i ) = (0, 0) 0 otherwise Parameters as previously and t = 0.01
7 7 R 0 = 0.5 R 0 = Number infectious 4 3 Number infectious Time (days) Time (days) Number infectious Number infectious Time (days) Time (days)
8 1000 Number of outbreaks 900 Number of simulations with an outbreak (/1000) R 0 =0.5 R 0 = Value of I 0 outbreak: t s.t. I (t) > I (0)
9 Why stochastic models? Important to choose right type of model for given application Deterministic models Mathematically easier Perfect reproducibility For given parameter set, one sim suffices Stochastic models Often harder Each realization different Needs many simulations In early stage of epidemic, very few infective individuals: Deterministic systems: behaviour entirely governed by parameters Stochastic systems: allows chance to play a role
10 In the context of the SIR model Deterministic R 0 strict threshold R 0 < 1 I (t) 0 monotonically R 0 > 1 I (t) 0 after a bump (outbreak) Stochastic R 0 threshold for mean R 0 < 1 I (t) 0 monotonically (roughly) on average but some realizations have outbreaks R 0 > 1 I (t) 0 after a bump on average (outbreak) but some realizations have no outbreaks
11 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC 6 ODE to CTMC
12 Theoretical setting of stochastic processes Stochastic processes form a well defined area of probability, a subset of measure theory, itself a well established area of mathematics Strong theoretical content Very briefly presented here for completeness, will not be used in what follows Used here: A. Klemke, Probability Theory, Springer 2008 DON T PANIC!!!
13 σ-algebras Definition 1 (σ-algebra) Let Ω be a set and A 2 Ω, the set of all subsets of Ω, be a class of subsets of Ω. A is called a σ-algebra if: 1 Ω A 2 A is closed under complements, i.e., A c := Ω \ A A for any A A 3 A is closed under countable unions, i.e., n=1 A n A for any choice of countably many sets A 1, A 2,... A Ω is the space of all elementary events and A the system of observable events
14 Probability space Definition 2 1 A pair (Ω, A), with Ω a nonempty set and A 2 Ω a σ-algebra is a measurable space, with sets A A measurable sets. If Ω is at most countably infinite and if A = 2 Ω, then the measurable space (Ω, 2 Ω ) is discrete 2 A triple (Ω, A, µ) is a measure space if (Ω, A) is a measurable space and µ is a measure on A 3 If in addition, µ(ω) = 1, then (Ω, A, µ) is a probability space and the sets A A are events. µ is then usually denoted P (or Prob)
15 Topological spaces and Borel σ-algebras Definition 3 (Topology) Let Ω be an arbitrary set. A class of sets τ Ω is a topology on Ω if: 1, Ω τ 2 A B τ for any A, B τ 3 ( A F A) τ for any F τ (Ω, τ) is a topological space, sets A τ are open and A Ω with A c τ are closed Definition 4 (Borel σ-algebra) Let (Ω, τ) be a topological space. The σ-algebra B(Ω, τ) generated by the open sets is the Borel σ-algebra on Ω, with elements A B(Ω, τ) the Borel sets
16 Polish spaces Definition 5 A separable topological space whose topology is induced by a complete metric is a Polish space R d, Z d, R N, (C([0, 1]), ) are Polish spaces
17 Stochastic process Definition 6 (Stochastic process) Let (Ω, F, P) be a probability space, (E, τ) be a Polish space with Borel σ-algebra E and T R be arbitrary (typically, T = N, T = Z, T = R + or T an interval). A family of random variables X = (X t, t T ) on (Ω, F, P) with values in (E, E) is called a stochastic process with index set (or time set) T and range E.
18 Types of processes So a stochastic process is a collection of random variables (r.v.) {X (t, ω), t T, ω Ω} The index set T and r.v. can be discrete or continuous Discrete time Markov chain (DTMC) has T and X discrete (e.g., T = N and X (t) N) Continuous time Markov chain (CTMC) has T continuous and X discrete (e.g., T = R + and X (t) N) In this course, we focus on these two. Also exist Stochastic differential equations (SDE), with both T and X (t) continuous
19 Definition 7 (Markov property) Markov property Let E be a Polish space with Borel σ-algebra B(E), T R and (X t ) t T an E-valued stochastic process. Assume that (F t ) t T = σ(x ) is the filtration generated by X. X has the Markov property if for every A B(E) and all s, t T with s t, P[X t A F s ] = P[X t A X s ] Theorem 8 (The usable one) If E is countable then X has the Markov property if for all n N, all s 1 < < s n < t and all i 1,..., i n, i E with P[X s1 = i 1,..., X sn = i n ] > 0, there holds P[X t = i X s1 = i 1,..., X sn = i n ] = P[X t = i X sn = i n ]
20 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC 6 ODE to CTMC
21 SIS model We consider an SIS model with demography. In ODE, S = b ds βsi + γi I = βsi (γ + d)i Total population N = b dn asymptotically constant (N(t) b/d =: N as t ). Can work with asymptotically autonomous system where N = N (and S = N I )
22 S = b βsi ds + γi I = βsi (γ + d)i DFE: (S, I ) = (b/d, 0). EEP: (S, I ) = ( γ+d β ), b d γ+d β VdD&W: F = D I [βs] DFE = βn, V = D I [(γ + d)i ] DFE = γ + d R 0 = ρ(fv 1 ) = β γ + d N If R 0 < 1, lim t (S(t), I (t)) = (N, 0) ( ) If R 0 > 1, lim t (S(t), I (t)) = N R 0, N N R 0
23 Reduction to one equation As N is asymptotically constant, we can write S = N I and so only need I = β(n I )I (γ + d)i We can then reconstruct dynamics of S from that of I
24 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC 6 ODE to CTMC
25 Discrete time Markov chain Definition 9 (DTMC Simple definition) An experiment with finite number of possible outcomes S 1,..., S N is repeated. The sequence of outcomes is a discrete time Markov chain if there is a set of N 2 numbers {p ij } such that the conditional probability of outcome S j on any experiment given outcome S i on the previous experiment is p ij, i.e., for 1 i, j N, t = 1,..., p ji = Pr(S j on experiment t + 1 S i on experiment t). The outcomes S 1,..., S N are the states, and the p ij are the transition probabilities. The matrix P = [p ij ] is the transition matrix.
26 Homogeneity One major distinction: If p ij does not depend on t, the chain is homogeneous If p ij (t), i.e, transition probabilities depend on t, the chain is nonhomogeneous
27 Markov chains operate on two level: Description of probability that the system is in a given state Description of individual realizations of the process Because we assume the Markov property, we need only describe how system switches (transitions) from one state to the next We say that the system has no memory, the next state depends only on the current one
28 States The states are here the different values of I. I can take integer values from 0 to N The chain links these states Infection Infection Infection I=0 I=1 I=2 I=3 I=N-1 I=N Recovery or Death Recovery or Death Recovery or Death Recovery or Death
29 Probability vector Let p i (t) = P(I (t) = i) is the probability distribution We must have i p i(t) = 1 p 0 (t) p 1 (t) p(t) =. p N (t) Initial condition at time t = 0: p(0)
30 Transition probabilities We then need description of probability of transition In general, suppose that S i is the current state, then one of S 1,..., S N must be the next state. If then we must have p ji = P{X (t + 1) = S j X (t) = S i } p i1 + p i2 + + p in = 1, 1 i N Some of the p ij can be zero
31 Transition matrix The matrix p 11 p 12 p 1N P = p 21 p 22 p 2N p N1 p N2 p NN has nonnegative entries, p ij 0 entries less than 1, p ij 1 row sum 1, which we write N p ij = 1, j=1 i = 1,..., N or, using the notation 1l = (1,..., 1) T, P1l = 1l
32 In matrix form p(t + 1) = Pp(t), t = 1, 2, 3,... where p(t) = (p 1 (t), p 2 (t),..., p N (t)) T is a (column) probability vector and P = (p ij ) is a N N transition matrix, p 11 p 12 p 1N P = p 21 p 22 p 2N p N1 p N2 p NN
33 Stochastic matrices Definition 10 (Stochastic matrix) The nonnegative N N matrix M is stochastic if N i=1 a ij = 1 for all j = 1, 2,..., N. In other words, 1l T M = 1l T Theorem 11 Let M be a stochastic matrix M. Then the spectral radius ρ(m) 1, i.e., all eigenvalues λ of M are such that λ 1. Furthermore, λ = 1 is an eigenvalue of M M has all column sums 1 1l T M = 1l T λ e.value of M with associated left e.vector v v T M = λv T 1l T M = 1l T : left e.vector 1l T and e.value 1..
34 Long time behavior Let p(0) be the initial distribution (column) vector. Then p(1) = Pp(0) p(2) = Pp(1) = P(Pp(0)) = P 2 p(0) Iterating, we get that for any t N, Therefore, lim p(t) = t + p(t) = P t p(0) ( lim t + Pt p(0) = lim t + Pt ) p(0) if this limit exists (it does if P is constant and chain is regular)
35 Regular Markov chain Regular Markov chains not covered here, as most chains in the demographic context are absorbing. For information, though.. Definition 12 (Regular Markov chain) A regular Markov chain is one in which P k is positive for some integer k > 0, i.e., P k has only positive entries, no zero entries. Theorem 13 If P is the transition matrix of a regular Markov chain, then 1 the powers P t approach a stochastic matrix W, 2 each column of W is the same (column) vector w = (w 1,..., w N ) T, 3 the components of w are positive. So if the Markov chain is regular, lim p(t) = Wp(0) t +
36 Interesting properties of matrices Definition 14 (Irreducibility) The square nonnegative matrix M is reducible if there exists a permutation matrix P such that P T MP is block triangular. If M is not reducible, then it is irreducible Definition 15 (Primitivity) A nonnegative matrix M is primitive if and only if there is an integer k > 0 such that M k is positive
37 Definition 16 (Digraph) Directed graphs (digraphs) A directed graph (or digraph) G = (V, A) consists in a set V of vertices and a set A of arcs linking these vertices
38 Connecting graphs and matrices Adjacency matrix M has m ij = 1 if arc from vertex j to vertex i, 0 otherwise M = Column of zeros (except maybe diagonal entry): vertex is a sink (nothing leaves)
39 Paths of length exactly 2 Given adjacency matrix M, M 2 has number of paths of length exactly two between vertices M = Write m k ij the (i, j) entry in M k m 2 32 = 2 since and
40 Connecting matrices, graphs and regular chains Theorem 17 A matrix M is irreducible iff the associated connection graph is strongly connected, i.e., there is a path between any pair (i, j) of vertices Irreducibility: i, j, k, m k ij > 0 Theorem 18 A matrix M is primitive if it is irreducible and there is at least one positive entry on the diagonal of M Primitivity: k, i, j, m k ij > 0 Theorem 19 Markov chain regular transition matrix P primitive
41 Strong connectedness (1) Not strongly connected
42 Strong connectedness (2) Strongly connected
43 Strong components Strong component 1 Strong component 2
44 Absorbing states, absorbing chains Definition 20 A state S i in a Markov chain is absorbing if whenever it occurs on the n th generation of the experiment, it then occurs on every subsequent step. In other words, S i is absorbing if p ii = 1 and p ij = 0 for i j Definition 21 A Markov chain is said to be absorbing if it has at least one absorbing state, and if from every state it is possible to go to an absorbing state In an absorbing Markov chain, a state that is not absorbing is called transient
45 Some questions on absorbing chains 1 Does the process eventually reach an absorbing state? 2 Average number of times spent in a transient state, if starting in a transient state? 3 Average number of steps before entering an absorbing state? 4 Probability of being absorbed by a given absorbing state, when there are more than one, when starting in a given transient state?
46 Reaching an absorbing state Answer to question 1: Theorem 22 In an absorbing Markov chain, the probability of reaching an absorbing state is 1
47 Standard form of the transition matrix For an absorbing chain with k absorbing states and N k transient states, the transition matrix can be written as ( ) Ik 0 P = R Q with following meaning, Absorbing states Transient states Absorbing states I k 0 Transient states R Q with I k the k k identity matrix, 0 an k (N k) matrix of zeros, R an (N k) k matrix and Q an (N k) (N k) matrix.
48 The matrix I N k Q is invertible. Let F = (I N k Q) 1 be the fundamental matrix of the Markov chain T i be the sum of the entries on row i of F B = FR Answers to our remaining questions: 2 F ij is the average number of times the process is in the jth transient state if it starts in the ith transient state 3 T i is the average number of steps before the process enters an absorbing state if it starts in the ith transient state 4 B ij is the probability of eventually entering the jth absorbing state if the process starts in the ith transient state
49 Back to the SIS model ODE: I = β(n I )I (γ + d)i This equation has 3 components: 1 β(n I )I new infection 2 γi recovery 3 di death In the DTMC, want to know what transitions mean and what their probabilities are Infection Infection Infection I=0 I=1 I=2 I=3 I=N-1 I=N Recovery or Death Recovery or Death Recovery or Death Recovery or Death
50 Consider small amount of time t, small enough that only one event occurs: new infection, loss of infectious individual, nothing.. Let B(i) := β(n i)i D(i) := (γ + d)i be rates of birth and death of infectious. Then p ji ( t) = P {I (t + ) = j I (t) = i} B(i) t j = i + 1 D(i) t j = i 1 = 1 [B(i) + D(i)] t j = i 0 otherwise β( N 1)Δ t β( N 2)2 Δ t β( N 1)Δ t Infection Infection Infection I=0 I=1 I=2 I=3 I=N-1 I=N Recovery Recovery Recovery Recovery or Death or Death or Death or Death (γ+d )Δ t (γ+d )2 Δt (γ+d )3 Δ t (γ+d ) N Δ t
51 Transition matrix 1 D(1) t 0 1 (B(1) + D(1)) t 0 B(1) t.... D(N) t B(N 1) t 1 D(N) t Note that indices in the matrix range from 0 to N here
52 Properties p 0 is an absorbing state For any initial distribution p(0) = (p 0 (t), p 1 (0),..., p N (0)), lim p(t) = (1, 0,..., 0)T lim t p 0 (t) = 1 t Exists quasi-stationary distribution conditioned on non-extinction q i (t) = p i(t) 1 p 0 (t), i = 1,..., N
53 Consequence of absorption We have lim p(t) = (1, 0,..., 0)T lim t p 0 (t) = 1 t but time to absorption can be exponential
54 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC 6 ODE to CTMC
55 Several ways to formulate CTMC s A continuous time Markov chain can be formulated in terms of infinitesimal transition probabilities branching process time to next event Here, time is in R +
56 For small t, p ji ( t) = P {I (t + ) = j I (t) = i} B(i) t + o( t) j = i + 1 D(i) t + o( t) j = i 1 = 1 [B(i) + D(i)] t + o( t) j = i o( t) otherwise with o( t) 0 as t 0
57 Forward Kolmogorov equations Assume we know I (0) = k. Then p i (t + t) = p i 1 (t)b(i 1) t + p i+1 (t)d(i + 1) t + p i (t)[1 (B(i) + D(i)) t] + o( t) Compute (p i (t + t) p i (t))/ t and take lim t 0, giving d dt p 0 = p 1 D(1) d dt p i = p i 1 B(i 1) + p i+1 D(i + 1) p i [B(i) + D(i)] i = 1,..., N Forward Kolmogorov equations associated to the CTMC
58 Write previous system as In vector form p = Qp with 0 D(1) (B(1) + D(1)) D(2) 0 Q = 0 B(1) (B(2) + D(2)) 0 D(N) D(N) Q generator matrix. Of course, p(t) = e Qt p(0)
59 Linking DTMC and CTMC for small t DTMC: p(t + t) = P( t)p(t) for transition matrix P( t). Let t 0, obtain Kolmogorov equations for CTMC d dt p = Qp where P( t) I Q = lim = P (0) t 0 t
60 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC 6 ODE to CTMC
61 Converting ODE to CTMC Extremely easy to convert compartmental ODE model to CTMC!!
62 The general setting: flow diagram b S βsi γ I I ds di
63 ODE approach b S βsi γ I I ds di Focus on compartments b S βsi γ I βsi γ I I ds di
64 CTMC approach b S βsi γ I I ds di b Focus on arrows (transitions) S ds βsi γ I I di
65 Focus on transitions b S βsi γ I I ds di b: S S + 1 ds: S S 1 βsi : S S 1, I I + 1 γi : S S + 1, I I 1 di : I I 1 birth death of susceptible new infection recovery death of infectious Will use S = N I and omit S dynamics
66 Gillespie s algorithm (SIS model with only I equation) 1: Set t t 0 and I (t) I (t 0 ) 2: while t t f do 3: ξ t β(n i)i + γi + di 4: Draw τ t from T E(ξ t ) 5: v [β(n i)i/ξ t, β(n i)i/ξ t + γi/ξ t, 1] 6: Draw ζ t from U([0, 1]) 7: Find pos such that ζ t v(pos) 8: switch (pos) 9: case 1: New infection, I (t + τ t ) = I (t) : case 2: End of infectious period, I (t + τ t ) = I (t) 1 11: case 3: Death, I (t + τ t ) = I (t) 1 12: end switch 13: t t + τ t 14: end while
67 IMPORTANT: in Gillespie s algorithm, we do not consider p ii, the event nothing happens
68 Drawing at random from an exponential distribution Want τ t from T E(ξ t ), i.e., T has probability density function f (x, ξ t ) = ξ t e ξtx 1 x 0 Use cumulative distribution function F (x, ξ t ) = x f (s, ξ t) ds F (x, ξ t ) = (1 e ξtx )1 x 0 which has values in [0, 1]. So draw ζ from U([0, 1]) and solve F (x, ξ t ) = ζ for x: F (x, ξ t ) = ζ 1 e ξtx = ζ e ξtx = 1 ζ ξ t x = ln(1 ζ) x = ln(1 ζ) ξ t
Stochastic modelling of epidemic spread
Stochastic modelling of epidemic spread Julien Arino Department of Mathematics University of Manitoba Winnipeg Julien Arino@umanitoba.ca 19 May 2012 1 Introduction 2 Stochastic processes 3 The SIS model
More informationStochastic process. X, a series of random variables indexed by t
Stochastic process X, a series of random variables indexed by t X={X(t), t 0} is a continuous time stochastic process X={X(t), t=0,1, } is a discrete time stochastic process X(t) is the state at time t,
More informationAN INTRODUCTION TO STOCHASTIC EPIDEMIC MODELS-PART I
AN INTRODUCTION TO STOCHASTIC EPIDEMIC MODELS-PART I Linda J. S. Allen Department of Mathematics and Statistics Texas Tech University Lubbock, Texas U.S.A. 2008 Summer School on Mathematical Modeling of
More informationAn Introduction to Stochastic Epidemic Models
An Introduction to Stochastic Epidemic Models Linda J. S. Allen Department of Mathematics and Statistics Texas Tech University Lubbock, Texas 79409-1042, U.S.A. linda.j.allen@ttu.edu 1 Introduction The
More informationIrreducibility. Irreducible. every state can be reached from every other state For any i,j, exist an m 0, such that. Absorbing state: p jj =1
Irreducibility Irreducible every state can be reached from every other state For any i,j, exist an m 0, such that i,j are communicate, if the above condition is valid Irreducible: all states are communicate
More informationTransmission in finite populations
Transmission in finite populations Juliet Pulliam, PhD Department of Biology and Emerging Pathogens Institute University of Florida and RAPIDD Program, DIEPS Fogarty International Center US National Institutes
More informationThursday. Threshold and Sensitivity Analysis
Thursday Threshold and Sensitivity Analysis SIR Model without Demography ds dt di dt dr dt = βsi (2.1) = βsi γi (2.2) = γi (2.3) With initial conditions S(0) > 0, I(0) > 0, and R(0) = 0. This model can
More informationMARKOV PROCESSES. Valerio Di Valerio
MARKOV PROCESSES Valerio Di Valerio Stochastic Process Definition: a stochastic process is a collection of random variables {X(t)} indexed by time t T Each X(t) X is a random variable that satisfy some
More informationStatistics 992 Continuous-time Markov Chains Spring 2004
Summary Continuous-time finite-state-space Markov chains are stochastic processes that are widely used to model the process of nucleotide substitution. This chapter aims to present much of the mathematics
More informationAARMS Homework Exercises
1 For the gamma distribution, AARMS Homework Exercises (a) Show that the mgf is M(t) = (1 βt) α for t < 1/β (b) Use the mgf to find the mean and variance of the gamma distribution 2 A well-known inequality
More informationOutlines. Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC)
Markov Chains (2) Outlines Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC) 2 pj ( n) denotes the pmf of the random variable p ( n) P( X j) j We will only be concerned with homogenous
More informationThe SIS and SIR stochastic epidemic models revisited
The SIS and SIR stochastic epidemic models revisited Jesús Artalejo Faculty of Mathematics, University Complutense of Madrid Madrid, Spain jesus_artalejomat.ucm.es BCAM Talk, June 16, 2011 Talk Schedule
More informationLIMITING PROBABILITY TRANSITION MATRIX OF A CONDENSED FIBONACCI TREE
International Journal of Applied Mathematics Volume 31 No. 18, 41-49 ISSN: 1311-178 (printed version); ISSN: 1314-86 (on-line version) doi: http://dx.doi.org/1.173/ijam.v31i.6 LIMITING PROBABILITY TRANSITION
More informationMarkov Chains, Stochastic Processes, and Matrix Decompositions
Markov Chains, Stochastic Processes, and Matrix Decompositions 5 May 2014 Outline 1 Markov Chains Outline 1 Markov Chains 2 Introduction Perron-Frobenius Matrix Decompositions and Markov Chains Spectral
More information2. Transience and Recurrence
Virtual Laboratories > 15. Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12 2. Transience and Recurrence The study of Markov chains, particularly the limiting behavior, depends critically on the random times
More informationDerivation of Itô SDE and Relationship to ODE and CTMC Models
Derivation of Itô SDE and Relationship to ODE and CTMC Models Biomathematics II April 23, 2015 Linda J. S. Allen Texas Tech University TTU 1 Euler-Maruyama Method for Numerical Solution of an Itô SDE dx(t)
More informationStochastic processes. MAS275 Probability Modelling. Introduction and Markov chains. Continuous time. Markov property
Chapter 1: and Markov chains Stochastic processes We study stochastic processes, which are families of random variables describing the evolution of a quantity with time. In some situations, we can treat
More information6 Continuous-Time Birth and Death Chains
6 Continuous-Time Birth and Death Chains Angela Peace Biomathematics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology.
More informationhttp://www.math.uah.edu/stat/markov/.xhtml 1 of 9 7/16/2009 7:20 AM Virtual Laboratories > 16. Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12 1. A Markov process is a random process in which the future is
More informationMarkov Chains CK eqns Classes Hitting times Rec./trans. Strong Markov Stat. distr. Reversibility * Markov Chains
Markov Chains A random process X is a family {X t : t T } of random variables indexed by some set T. When T = {0, 1, 2,... } one speaks about a discrete-time process, for T = R or T = [0, ) one has a continuous-time
More informationAPM 541: Stochastic Modelling in Biology Discrete-time Markov Chains. Jay Taylor Fall Jay Taylor (ASU) APM 541 Fall / 92
APM 541: Stochastic Modelling in Biology Discrete-time Markov Chains Jay Taylor Fall 2013 Jay Taylor (ASU) APM 541 Fall 2013 1 / 92 Outline 1 Motivation 2 Markov Processes 3 Markov Chains: Basic Properties
More informationMean-field dual of cooperative reproduction
The mean-field dual of systems with cooperative reproduction joint with Tibor Mach (Prague) A. Sturm (Göttingen) Friday, July 6th, 2018 Poisson construction of Markov processes Let (X t ) t 0 be a continuous-time
More informationMath Homework 5 Solutions
Math 45 - Homework 5 Solutions. Exercise.3., textbook. The stochastic matrix for the gambler problem has the following form, where the states are ordered as (,, 4, 6, 8, ): P = The corresponding diagram
More informationStochastic2010 Page 1
Stochastic2010 Page 1 Extinction Probability for Branching Processes Friday, April 02, 2010 2:03 PM Long-time properties for branching processes Clearly state 0 is an absorbing state, forming its own recurrent
More informationProbability Distributions
Lecture 1: Background in Probability Theory Probability Distributions The probability mass function (pmf) or probability density functions (pdf), mean, µ, variance, σ 2, and moment generating function
More informationThree Disguises of 1 x = e λx
Three Disguises of 1 x = e λx Chathuri Karunarathna Mudiyanselage Rabi K.C. Winfried Just Department of Mathematics, Ohio University Mathematical Biology and Dynamical Systems Seminar Ohio University November
More informationSTOCHASTIC PROCESSES Basic notions
J. Virtamo 38.3143 Queueing Theory / Stochastic processes 1 STOCHASTIC PROCESSES Basic notions Often the systems we consider evolve in time and we are interested in their dynamic behaviour, usually involving
More informationMarkov Processes Hamid R. Rabiee
Markov Processes Hamid R. Rabiee Overview Markov Property Markov Chains Definition Stationary Property Paths in Markov Chains Classification of States Steady States in MCs. 2 Markov Property A discrete
More informationIntroduction to SEIR Models
Department of Epidemiology and Public Health Health Systems Research and Dynamical Modelling Unit Introduction to SEIR Models Nakul Chitnis Workshop on Mathematical Models of Climate Variability, Environmental
More informationEpidemics in Networks Part 2 Compartmental Disease Models
Epidemics in Networks Part 2 Compartmental Disease Models Joel C. Miller & Tom Hladish 18 20 July 2018 1 / 35 Introduction to Compartmental Models Dynamics R 0 Epidemic Probability Epidemic size Review
More information8. Statistical Equilibrium and Classification of States: Discrete Time Markov Chains
8. Statistical Equilibrium and Classification of States: Discrete Time Markov Chains 8.1 Review 8.2 Statistical Equilibrium 8.3 Two-State Markov Chain 8.4 Existence of P ( ) 8.5 Classification of States
More informationMATH 56A: STOCHASTIC PROCESSES CHAPTER 1
MATH 56A: STOCHASTIC PROCESSES CHAPTER. Finite Markov chains For the sake of completeness of these notes I decided to write a summary of the basic concepts of finite Markov chains. The topics in this chapter
More informationMetapopulation models of disease spread
Metapopulation models of disease spread Julien Arino Department of Mathematics University of Manitoba Winnipeg, Manitoba, Canada Centre for Inner City Health Saint Michael s Hospital Toronto, Ontario,
More informationLecture 5. If we interpret the index n 0 as time, then a Markov chain simply requires that the future depends only on the present and not on the past.
1 Markov chain: definition Lecture 5 Definition 1.1 Markov chain] A sequence of random variables (X n ) n 0 taking values in a measurable state space (S, S) is called a (discrete time) Markov chain, if
More informationPart I Stochastic variables and Markov chains
Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)
More informationOn the Spread of Epidemics in a Closed Heterogeneous Population
On the Spread of Epidemics in a Closed Heterogeneous Population Artem Novozhilov Applied Mathematics 1 Moscow State University of Railway Engineering (MIIT) the 3d Workshop on Mathematical Models and Numerical
More informationExample: physical systems. If the state space. Example: speech recognition. Context can be. Example: epidemics. Suppose each infected
4. Markov Chains A discrete time process {X n,n = 0,1,2,...} with discrete state space X n {0,1,2,...} is a Markov chain if it has the Markov property: P[X n+1 =j X n =i,x n 1 =i n 1,...,X 0 =i 0 ] = P[X
More informationLecture 20 : Markov Chains
CSCI 3560 Probability and Computing Instructor: Bogdan Chlebus Lecture 0 : Markov Chains We consider stochastic processes. A process represents a system that evolves through incremental changes called
More informationMarkov Chains. X(t) is a Markov Process if, for arbitrary times t 1 < t 2 <... < t k < t k+1. If X(t) is discrete-valued. If X(t) is continuous-valued
Markov Chains X(t) is a Markov Process if, for arbitrary times t 1 < t 2
More informationCopyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and
Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and private study only. The thesis may not be reproduced elsewhere
More informationLet (Ω, F) be a measureable space. A filtration in discrete time is a sequence of. F s F t
2.2 Filtrations Let (Ω, F) be a measureable space. A filtration in discrete time is a sequence of σ algebras {F t } such that F t F and F t F t+1 for all t = 0, 1,.... In continuous time, the second condition
More informationSection 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns.
Section 9.2: Matrices Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. That is, a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn A
More informationThe Theory behind PageRank
The Theory behind PageRank Mauro Sozio Telecom ParisTech May 21, 2014 Mauro Sozio (LTCI TPT) The Theory behind PageRank May 21, 2014 1 / 19 A Crash Course on Discrete Probability Events and Probability
More informationLecture 11: Introduction to Markov Chains. Copyright G. Caire (Sample Lectures) 321
Lecture 11: Introduction to Markov Chains Copyright G. Caire (Sample Lectures) 321 Discrete-time random processes A sequence of RVs indexed by a variable n 2 {0, 1, 2,...} forms a discretetime random process
More informationKernels of Directed Graph Laplacians. J. S. Caughman and J.J.P. Veerman
Kernels of Directed Graph Laplacians J. S. Caughman and J.J.P. Veerman Department of Mathematics and Statistics Portland State University PO Box 751, Portland, OR 97207. caughman@pdx.edu, veerman@pdx.edu
More informationSMSTC (2007/08) Probability.
SMSTC (27/8) Probability www.smstc.ac.uk Contents 12 Markov chains in continuous time 12 1 12.1 Markov property and the Kolmogorov equations.................... 12 2 12.1.1 Finite state space.................................
More informationPerron Frobenius Theory
Perron Frobenius Theory Oskar Perron Georg Frobenius (1880 1975) (1849 1917) Stefan Güttel Perron Frobenius Theory 1 / 10 Positive and Nonnegative Matrices Let A, B R m n. A B if a ij b ij i, j, A > B
More information18.175: Lecture 30 Markov chains
18.175: Lecture 30 Markov chains Scott Sheffield MIT Outline Review what you know about finite state Markov chains Finite state ergodicity and stationarity More general setup Outline Review what you know
More informationBirth and Death Processes. Birth and Death Processes. Linear Growth with Immigration. Limiting Behaviour for Birth and Death Processes
DTU Informatics 247 Stochastic Processes 6, October 27 Today: Limiting behaviour of birth and death processes Birth and death processes with absorbing states Finite state continuous time Markov chains
More informationIEOR 6711, HMWK 5, Professor Sigman
IEOR 6711, HMWK 5, Professor Sigman 1. Semi-Markov processes: Consider an irreducible positive recurrent discrete-time Markov chain {X n } with transition matrix P (P i,j ), i, j S, and finite state space.
More informationHomework 4 due on Thursday, December 15 at 5 PM (hard deadline).
Large-Time Behavior for Continuous-Time Markov Chains Friday, December 02, 2011 10:58 AM Homework 4 due on Thursday, December 15 at 5 PM (hard deadline). How are formulas for large-time behavior of discrete-time
More informationProbability Distributions
Lecture : Background in Probability Theory Probability Distributions The probability mass function (pmf) or probability density functions (pdf), mean, µ, variance, σ 2, and moment generating function (mgf)
More information1.3 Convergence of Regular Markov Chains
Markov Chains and Random Walks on Graphs 3 Applying the same argument to A T, which has the same λ 0 as A, yields the row sum bounds Corollary 0 Let P 0 be the transition matrix of a regular Markov chain
More informationAvailability. M(t) = 1 - e -mt
Availability Availability - A(t) the probability that the system is operating correctly and is available to perform its functions at the instant of time t More general concept than reliability: failure
More information1 Stochastic Dynamic Programming
1 Stochastic Dynamic Programming Formally, a stochastic dynamic program has the same components as a deterministic one; the only modification is to the state transition equation. When events in the future
More informationELEMENTS OF PROBABILITY THEORY
ELEMENTS OF PROBABILITY THEORY Elements of Probability Theory A collection of subsets of a set Ω is called a σ algebra if it contains Ω and is closed under the operations of taking complements and countable
More informationIEOR 6711: Professor Whitt. Introduction to Markov Chains
IEOR 6711: Professor Whitt Introduction to Markov Chains 1. Markov Mouse: The Closed Maze We start by considering how to model a mouse moving around in a maze. The maze is a closed space containing nine
More informationLecture 15 Perron-Frobenius Theory
EE363 Winter 2005-06 Lecture 15 Perron-Frobenius Theory Positive and nonnegative matrices and vectors Perron-Frobenius theorems Markov chains Economic growth Population dynamics Max-min and min-max characterization
More informationCDA6530: Performance Models of Computers and Networks. Chapter 3: Review of Practical Stochastic Processes
CDA6530: Performance Models of Computers and Networks Chapter 3: Review of Practical Stochastic Processes Definition Stochastic process X = {X(t), t2 T} is a collection of random variables (rvs); one rv
More informationDefinition A finite Markov chain is a memoryless homogeneous discrete stochastic process with a finite number of states.
Chapter 8 Finite Markov Chains A discrete system is characterized by a set V of states and transitions between the states. V is referred to as the state space. We think of the transitions as occurring
More informationAn Introduction to Entropy and Subshifts of. Finite Type
An Introduction to Entropy and Subshifts of Finite Type Abby Pekoske Department of Mathematics Oregon State University pekoskea@math.oregonstate.edu August 4, 2015 Abstract This work gives an overview
More informationSocial network analysis: social learning
Social network analysis: social learning Donglei Du (ddu@unb.edu) Faculty of Business Administration, University of New Brunswick, NB Canada Fredericton E3B 9Y2 October 20, 2016 Donglei Du (UNB) AlgoTrading
More informationLecture 20: Reversible Processes and Queues
Lecture 20: Reversible Processes and Queues 1 Examples of reversible processes 11 Birth-death processes We define two non-negative sequences birth and death rates denoted by {λ n : n N 0 } and {µ n : n
More informationProbability, Random Processes and Inference
INSTITUTO POLITÉCNICO NACIONAL CENTRO DE INVESTIGACION EN COMPUTACION Laboratorio de Ciberseguridad Probability, Random Processes and Inference Dr. Ponciano Jorge Escamilla Ambrosio pescamilla@cic.ipn.mx
More informationReinforcement Learning
Reinforcement Learning March May, 2013 Schedule Update Introduction 03/13/2015 (10:15-12:15) Sala conferenze MDPs 03/18/2015 (10:15-12:15) Sala conferenze Solving MDPs 03/20/2015 (10:15-12:15) Aula Alpha
More informationStatistics 253/317 Introduction to Probability Models. Winter Midterm Exam Monday, Feb 10, 2014
Statistics 253/317 Introduction to Probability Models Winter 2014 - Midterm Exam Monday, Feb 10, 2014 Student Name (print): (a) Do not sit directly next to another student. (b) This is a closed-book, closed-note
More informationIntroduction. Genetic Algorithm Theory. Overview of tutorial. The Simple Genetic Algorithm. Jonathan E. Rowe
Introduction Genetic Algorithm Theory Jonathan E. Rowe University of Birmingham, UK GECCO 2012 The theory of genetic algorithms is beginning to come together into a coherent framework. However, there are
More informationMathematical Modeling and Analysis of Infectious Disease Dynamics
Mathematical Modeling and Analysis of Infectious Disease Dynamics V. A. Bokil Department of Mathematics Oregon State University Corvallis, OR MTH 323: Mathematical Modeling May 22, 2017 V. A. Bokil (OSU-Math)
More informationMarkov chains. 1 Discrete time Markov chains. c A. J. Ganesh, University of Bristol, 2015
Markov chains c A. J. Ganesh, University of Bristol, 2015 1 Discrete time Markov chains Example: A drunkard is walking home from the pub. There are n lampposts between the pub and his home, at each of
More informationDiscrete Time Markov Chain of a Dynamical System with a Rest Phase
Discrete Time Markov Chain of a Dynamical System with a Rest Phase Abstract A stochastic model, in the form of a discrete time Markov chain, is constructed to describe the dynamics of a population that
More informationIEOR 6711: Stochastic Models I Professor Whitt, Thursday, November 29, Weirdness in CTMC s
IEOR 6711: Stochastic Models I Professor Whitt, Thursday, November 29, 2012 Weirdness in CTMC s Where s your will to be weird? Jim Morrison, The Doors We are all a little weird. And life is a little weird.
More informationStochastic Modelling Unit 1: Markov chain models
Stochastic Modelling Unit 1: Markov chain models Russell Gerrard and Douglas Wright Cass Business School, City University, London June 2004 Contents of Unit 1 1 Stochastic Processes 2 Markov Chains 3 Poisson
More informationLanguage Acquisition and Parameters: Part II
Language Acquisition and Parameters: Part II Matilde Marcolli CS0: Mathematical and Computational Linguistics Winter 205 Transition Matrices in the Markov Chain Model absorbing states correspond to local
More informationLectures on Markov Chains
Lectures on Markov Chains David M. McClendon Department of Mathematics Ferris State University 2016 edition 1 Contents Contents 2 1 Markov chains 4 1.1 The definition of a Markov chain.....................
More informationReaching a Consensus in a Dynamically Changing Environment A Graphical Approach
Reaching a Consensus in a Dynamically Changing Environment A Graphical Approach M. Cao Yale Univesity A. S. Morse Yale University B. D. O. Anderson Australia National University and National ICT Australia
More informationNonnegative Matrices I
Nonnegative Matrices I Daisuke Oyama Topics in Economic Theory September 26, 2017 References J. L. Stuart, Digraphs and Matrices, in Handbook of Linear Algebra, Chapter 29, 2006. R. A. Brualdi and H. J.
More informationStochastic Processes
Stochastic Processes 8.445 MIT, fall 20 Mid Term Exam Solutions October 27, 20 Your Name: Alberto De Sole Exercise Max Grade Grade 5 5 2 5 5 3 5 5 4 5 5 5 5 5 6 5 5 Total 30 30 Problem :. True / False
More informationData analysis and stochastic modeling
Data analysis and stochastic modeling Lecture 7 An introduction to queueing theory Guillaume Gravier guillaume.gravier@irisa.fr with a lot of help from Paul Jensen s course http://www.me.utexas.edu/ jensen/ormm/instruction/powerpoint/or_models_09/14_queuing.ppt
More informationMarkov Chains and Stochastic Sampling
Part I Markov Chains and Stochastic Sampling 1 Markov Chains and Random Walks on Graphs 1.1 Structure of Finite Markov Chains We shall only consider Markov chains with a finite, but usually very large,
More informationRecap. Probability, stochastic processes, Markov chains. ELEC-C7210 Modeling and analysis of communication networks
Recap Probability, stochastic processes, Markov chains ELEC-C7210 Modeling and analysis of communication networks 1 Recap: Probability theory important distributions Discrete distributions Geometric distribution
More informationSection 9.2: Matrices.. a m1 a m2 a mn
Section 9.2: Matrices Definition: A matrix is a rectangular array of numbers: a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn In general, a ij denotes the (i, j) entry of A. That is, the entry in
More informationStochastic Models. John M. Drake & Pejman Rohani
Epidemiological data are noisy Two types of noise: Observation error: the data are probabilistically related to the true state of the system Process noise: the system progresses probabilistically Environmental
More informationP(X 0 = j 0,... X nk = j k )
Introduction to Probability Example Sheet 3 - Michaelmas 2006 Michael Tehranchi Problem. Let (X n ) n 0 be a homogeneous Markov chain on S with transition matrix P. Given a k N, let Z n = X kn. Prove that
More informationMath Ordinary Differential Equations
Math 411 - Ordinary Differential Equations Review Notes - 1 1 - Basic Theory A first order ordinary differential equation has the form x = f(t, x) (11) Here x = dx/dt Given an initial data x(t 0 ) = x
More informationEPIDEMIC MODELS I REPRODUCTION NUMBERS AND FINAL SIZE RELATIONS FRED BRAUER
EPIDEMIC MODELS I REPRODUCTION NUMBERS AND FINAL SIZE RELATIONS FRED BRAUER 1 THE SIR MODEL Start with simple SIR epidemic model with initial conditions S = βsi I = (βs α)i, S() = S, I() = I, S + I = N
More informationMarkov Chains and Pandemics
Markov Chains and Pandemics Caleb Dedmore and Brad Smith December 8, 2016 Page 1 of 16 Abstract Markov Chain Theory is a powerful tool used in statistical analysis to make predictions about future events
More informationOn asymptotic behavior of a finite Markov chain
1 On asymptotic behavior of a finite Markov chain Alina Nicolae Department of Mathematical Analysis Probability. University Transilvania of Braşov. Romania. Keywords: convergence, weak ergodicity, strong
More informationNo class on Thursday, October 1. No office hours on Tuesday, September 29 and Thursday, October 1.
Stationary Distributions Monday, September 28, 2015 2:02 PM No class on Thursday, October 1. No office hours on Tuesday, September 29 and Thursday, October 1. Homework 1 due Friday, October 2 at 5 PM strongly
More informationQuivers of Period 2. Mariya Sardarli Max Wimberley Heyi Zhu. November 26, 2014
Quivers of Period 2 Mariya Sardarli Max Wimberley Heyi Zhu ovember 26, 2014 Abstract A quiver with vertices labeled from 1,..., n is said to have period 2 if the quiver obtained by mutating at 1 and then
More informationSystem theory and system identification of compartmental systems Hof, Jacoba Marchiena van den
University of Groningen System theory and system identification of compartmental systems Hof, Jacoba Marchiena van den IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF)
More information2 Discrete-Time Markov Chains
2 Discrete-Time Markov Chains Angela Peace Biomathematics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,
More informationStochastic Petri Net
Stochastic Petri Net Serge Haddad LSV ENS Cachan & CNRS & INRIA haddad@lsv.ens-cachan.fr Petri Nets 2013, June 24th 2013 1 Stochastic Petri Net 2 Markov Chain 3 Markovian Stochastic Petri Net 4 Generalized
More informationMarkov chains. Randomness and Computation. Markov chains. Markov processes
Markov chains Randomness and Computation or, Randomized Algorithms Mary Cryan School of Informatics University of Edinburgh Definition (Definition 7) A discrete-time stochastic process on the state space
More informationRelations Graphical View
Introduction Relations Computer Science & Engineering 235: Discrete Mathematics Christopher M. Bourke cbourke@cse.unl.edu Recall that a relation between elements of two sets is a subset of their Cartesian
More informationFinite-Horizon Statistics for Markov chains
Analyzing FSDT Markov chains Friday, September 30, 2011 2:03 PM Simulating FSDT Markov chains, as we have said is very straightforward, either by using probability transition matrix or stochastic update
More informationConvex Optimization CMU-10725
Convex Optimization CMU-10725 Simulated Annealing Barnabás Póczos & Ryan Tibshirani Andrey Markov Markov Chains 2 Markov Chains Markov chain: Homogen Markov chain: 3 Markov Chains Assume that the state
More informationThe Transition Probability Function P ij (t)
The Transition Probability Function P ij (t) Consider a continuous time Markov chain {X(t), t 0}. We are interested in the probability that in t time units the process will be in state j, given that it
More informationRegular finite Markov chains with interval probabilities
5th International Symposium on Imprecise Probability: Theories and Applications, Prague, Czech Republic, 2007 Regular finite Markov chains with interval probabilities Damjan Škulj Faculty of Social Sciences
More information10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections )
c Dr. Igor Zelenko, Fall 2017 1 10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections 7.2-7.4) 1. When each of the functions F 1, F 2,..., F n in right-hand side
More informationModelling in Systems Biology
Modelling in Systems Biology Maria Grazia Vigliotti thanks to my students Anton Stefanek, Ahmed Guecioueur Imperial College Formal representation of chemical reactions precise qualitative and quantitative
More information