Stochastic modelling of epidemic spread

Size: px
Start display at page:

Download "Stochastic modelling of epidemic spread"

Transcription

1 Stochastic modelling of epidemic spread Julien Arino Department of Mathematics University of Manitoba Winnipeg Julien 19 May 2012

2 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

3 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

4 Deterministic SIR Use the standard KMK as in Fred s lectures: S 0 = SI I 0 = SI I R 0 = I Basic reproduction number R 0 = S 0 If R 0 < 1, no outbreak If R 0 > 1, outbreak Use =1/4, N = S + I + R = 100, I (0) = 1, R 0 = {0.5, 1.5} and so that this is true

5 R 0 =0.5 R 0 = S S 90 I 90 I R R Time (days) Time (days) Number infectious Number infectious Time (days) Time (days)

6 Stochastic DTMC SIR Consider S(t) andi (t) (R(t) = N S(t) I (t)). Define transition probabilities p (s 0,i 0 ),(s,i)( t) = P{(S(t + t), I (t + t)) = (s + s 0, i + i 0 ) (S(t), I (t)) = (s, i)} by 8 si t (s 0, i 0 )=( 1, 1) >< i t (s 0, i 0 )=(0, 1) p (s 0,i 0 ),(s,i)( t) = 1 [ si + i] t (s 0, i 0 )=(0, 0) >: 0 otherwise Parameters as previously and t = 0.01

7 7 R 0 =0.5 R 0 = Number infectious 4 3 Number infectious Time (days) Time (days) Number infectious Number infectious Time (days) Time (days)

8 1000 Number of outbreaks 900 Number of simulations with an outbreak (/1000) R 0 =0.5 R 0 = Value of I 0 outbreak: 9t s.t. I (t) > I (0)

9 Why stochastic models? Important to choose right type of model for given application Deterministic models Mathematically easier Perfect reproducibility For given parameter set, one sim su ces Stochastic models Often harder Each realization di erent Needs many simulations In early stage of epidemic, very few infective individuals: Deterministic systems: behaviour entirely governed by parameters Stochastic systems: allows chance to play a role

10 In the context of the SIR model Deterministic R 0 strict threshold R 0 < 1 ) I (t)! 0 monotonically R 0 > 1 ) I (t)! 0 after a bump (outbreak) Stochastic R 0 threshold for mean R 0 < 1 ) I (t)! 0 monotonically (roughly) on average but some realizations have outbreaks R 0 > 1 ) I (t)! 0 after a bump on average (outbreak) but some realizations have no outbreaks

11 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

12 Theoretical setting of stochastic processes Stochastic processes form a well defined area of probability, a subset of measure theory, itself a well established area of mathematics Strong theoretical content Very briefly presented here for completeness, will not be used in what follows Used here: A. Klemke, Probability Theory, Springer 2008 DON T PANIC!!!

13 -algebras Definition 1 ( -algebra) Let 6= ; be a set and A 2, the set of all subsets of, bea class of subsets of. A is called a -algebra if: 1 2 A 2 A is closed under complements, i.e., A c := \ A 2 A for any A 2 A 3 A is closed under countable unions, i.e., [ 1 n=1 A n 2 A for any choice of countably many sets A 1, A 2,...2 A is the space of all elementary events and A the system of observable events

14 Probability space Definition 2 1 Apair(, A), with a nonempty set and A 2 a -algebra is a measurable space, withsetsa 2 A measurable sets. If is at most countably infinite and if A =2,thenthemeasurablespace(, 2 )isdiscrete 2 Atriple(, A,µ)isameasure space if (, A) isa measurable space and µ is a measure on A 3 If in addition, µ( ) = 1, then (, A,µ)isaprobability space and the sets A 2 A are events. µ is then usually denoted P (or Prob)

15 Topological spaces and Borel Definition 3 (Topology) -algebras Let 6= ; be an arbitrary set. A class of sets is a topology on if: 1 ;, 2 2 A \ B 2 for any A, B 2 3 ([ A2F A) 2 for any F 2 (, ) isatopological space, setsa 2 are open and A 2 with A c 2 are closed Definition 4 (Borel -algebra) Let (, ) be a topological space. The -algebra B(, ) generated by the open sets is the Borel -algebra on, with elements A 2 B(, ) theborel sets

16 Polish spaces Definition 5 A separable topological space whose topology is induced by a complete metric is a Polish space R d, Z d, R N,(C([0, 1]), k k 1 ) are Polish spaces

17 Stochastic process Definition 6 (Stochastic process) Let (, F, P) be a probability space, (E, ) be a Polish space with Borel -algebra E and T R be arbitrary (typically, T = N, T = Z, T = R + or T an interval). A family of random variables X =(X t, t 2 T ) on (, F, P) with values in (E, E) iscalledastochastic process with index set (or time set) T and range E.

18 OKAY, DONE WITH HARD THEORY!!!

19 Types of processes So a stochastic process is a collection of random variables (r.v.) {X (t,!), t 2 T,! 2 } The index set T and r.v. can be discrete or continuous Discrete time Markov chain (DTMC) hast and X discrete (e.g., T = N and X (t) 2 N) Continuous time Markov chain (CTMC) hast continuous and X discrete (e.g., T = R + and X (t) 2 N) In this course, we focus on these two. Also exist Stochastic di erential equations (SDE), with both T and X (t) continuous

20 Definition 7 (Markov property) Markov property Let E be a Polish space with Borel -algebra B(E), T R and (X t ) t2t an E-valued stochastic process. Assume that (F t ) t2t = (X ) is the filtration generated by X. X has the Markov property if for every A 2 B(E) andalls, t 2 T with s apple t, P[X t 2 A F s ]=P[X t 2 A X s ] Theorem 8 (The usable one) If E is countable then X has the Markov property if for all n 2 N, all s 1 < < s n < tandalli 1,...,i n, i 2 Ewith P[X s1 = i 1,...,X sn = i n ] > 0, there holds P[X t = i X s1 = i 1,...,X sn = i n ]=P[X t = i X sn = i n ]

21 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

22 SIS model We consider an SIS model with demography. In ODE, S 0 = b ds SI + I I 0 = SI ( + d)i Total population N 0 = b dn asymptotically constant (N(t)! b/d =: N as t!1). Can work with asymptotically autonomous system where N = N (and S = N I )

23 S 0 = b SI ds + I I 0 = SI ( + d)i DFE: (S, I )=(b/d, 0). EEP: (S, I )= +d, b d +d VdD&W: F = D I [ S] DFE = N, V = D I [( + d)i ] DFE = + d ) R 0 = (FV 1 )= + d N If R 0 < 1, lim t!1 (S(t), I (t)) = (N, 0) If R 0 > 1, lim t!1 (S(t), I (t)) = N R 0, N N R 0

24 Reduction to one equation As N is asymptotically constant, we can write S = N only need I 0 = (N I )I ( + d)i I and so We can then reconstruct dynamics of S from that of I

25 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

26 Discrete time Markov chain Definition 9 (DTMC Simple definition) An experiment with finite number of possible outcomes S 1,...,S N is repeated. The sequence of outcomes is a discrete time Markov chain if there is a set of N 2 numbers {p ij } such that the conditional probability of outcome S j on any experiment given outcome S i on the previous experiment is p ij,i.e.,for1apple i, j apple N, t =1,..., p ji =Pr(S j on experiment t +1 S i on experiment t). The outcomes S 1,...,S N are the states, andthep ij are the transition probabilities. ThematrixP =[p ij ]isthetransition matrix.

27 Homogeneity One major distinction: If p ij does not depend on t, thechainishomogeneous If p ij (t), i.e, transition probabilities depend on t, thechainis nonhomogeneous

28 Markov chains operate on two level: Description of probability that the system is in a given state Description of individual realizations of the process Because we assume the Markov property, we need only describe how system switches (transitions) from one state to the next We say that the system has no memory, thenextstatedepends only on the current one

29 States The states are here the di erent values of I. I can take integer values from 0 to N The chain links these states Infection Infection Infection I=0 I=1 I=2 I=3 I=N-1 I=N Recovery or Death Recovery or Death Recovery or Death Recovery or Death

30 Probability vector Let p i (t) =P(I (t) =i) is the probability distribution We must have P i p i(t) =1 0 1 p 0 (t) p 1 (t) p(t) = B A p N (t) Initial condition at time t = 0: p(0)

31 Transition probabilities We then need description of probability of transition In general, suppose that S i is the current state, then one of S 1,...,S N must be the next state. If then we must have p ji = P{X (t + 1) = S j X (t) =S i } p i1 + p i2 + + p in =1, 1 apple i apple N Some of the p ij can be zero

32 Transition matrix The matrix 0 1 p 11 p 12 p 1N P = Bp 21 p 22 p 2N A p N1 p N2 p NN has nonnegative entries, p ij 0 entries less than 1, p ij apple 1 column sum 1, which we write NX p ij =1, i=1 j =1,...,N or, using the notation 1l =(1,...,1) T, 1l T P = 1l T

33 In matrix form p(t + 1) = Pp(t), t =1, 2, 3,... where p(t) =(p 1 (t), p 2 (t),...,p N (t)) T is a (column) probability vector and P =(p ij )isan N transition matrix, 0 1 p 11 p 12 p 1N P = Bp 21 p 22 p 2N A p N1 p N2 p NN

34 Stochastic matrices Definition 10 (Stochastic matrix) The nonnegative N N matrix M is stochastic if P N i=1 a ij =1for all j =1, 2,...,N. In other words, 1l T M = 1l T Theorem 11 Let M be a stochastic matrix M. Then the spectral radius (M) apple 1, i.e., all eigenvalues of M are such that apple 1. Furthermore, =1is an eigenvalue of M M has all column sums 1, 1l T M = 1l T e.value of M with associated left e.vector v, v T M = v T ) 1l T M = 1l T :lefte.vector1l T and e.value 1..

35 Long time behavior Let p(0) be the initial distribution (column) vector. Then p(1) = Pp(0) p(2) = Pp(1) = P(Pp(0)) = P 2 p(0) Iterating, we get that for any t 2 N, Therefore, lim p(t) = t!+1 p(t) =P t p(0) lim t!+1 Pt p(0) = lim t!+1 Pt p(0) if this limit exists (it does if P is constant and chain is regular)

36 Regular Markov chain Regular Markov chains not covered here, as most chains in the demographic context are absorbing. For information, though.. Definition 12 (Regular Markov chain) A regular Markov chain is one in which P k is positive for some integer k > 0, i.e., P k has only positive entries, no zero entries. Theorem 13 If P is the transition matrix of a regular Markov chain, then 1 the powers P t approach a stochastic matrix W, 2 each column of W is the same (column) vector w =(w 1,...,w N ) T, 3 the components of w are positive. So if the Markov chain is regular, lim p(t) =Wp(0) t!+1

37 Interesting properties of matrices Definition 14 (Irreducibility) The square nonnegative matrix M is reducible if there exists a permutation matrix P such that P T MP is block triangular. If M is not reducible, then it is irreducible Definition 15 (Primitivity) A nonnegative matrix M is primitive if and only if there is an integer k > 0 such that M k is positive

38 Definition 16 (Digraph) Directed graphs (digraphs) A directed graph (or digraph) G =(V, A) consists in a set V of vertices and a set A of arcs linking these vertices

39 Connecting graphs and matrices Adjacency matrix M has m ij = 1 if arc from vertex j to vertex i, 0 otherwise ) M = B A Column of zeros (except maybe diagonal entry): vertex is a sink (nothing leaves)

40 Paths of length exactly 2 Given adjacency matrix M, M 2 has number of paths of length exactly two between vertices ) M = B A Write m k ij the (i, j) entryinm k m 2 32 =2since2! 1! 3and2! 4! 3..

41 Connecting matrices, graphs and regular chains Theorem 17 A matrix M is irreducible i the associated connection graph is strongly connected, i.e.,thereisapathbetweenanypair(i, j) of vertices Irreducibility: 8i, j, 9k, m k ij > 0 Theorem 18 A matrix M is primitive if it is irreducible and there is at least one positive entry on the diagonal of M Primitivity: 9k, 8i, j, m k ij > 0 Theorem 19 Markov chain regular, transition matrix P primitive

42 Strong connectedness (1) Not strongly connected

43 Strong connectedness (2) Strongly connected

44 Strong components Strong component 1 Strong component 2

45 Absorbing states, absorbing chains Definition 20 AstateS i in a Markov chain is absorbing if whenever it occurs on the n th generation of the experiment, it then occurs on every subsequent step. In other words, S i is absorbing if p ii =1and p ij =0fori 6= j Definition 21 A Markov chain is said to be absorbing if it has at least one absorbing state, and if from every state it is possible to go to an absorbing state In an absorbing Markov chain, a state that is not absorbing is called transient

46 Here, we focus on absorbing strong components containing only one vertex Absorbing strong components Absorbing strong component Transient states

47 Some questions on absorbing chains 1 Does the process eventually reach an absorbing state? 2 Average number of times spent in a transient state, if starting in a transient state? 3 Average number of steps before entering an absorbing state? 4 Probability of being absorbed by a given absorbing state, when there are more than one, when starting in a given transient state?

48 Reaching an absorbing state Answer to question 1: Theorem 22 In an absorbing Markov chain, the probability of reaching an absorbing state is 1

49 Standard form of the transition matrix For an absorbing chain with k absorbing states and N transient states, the transition matrix can be written as Ik 0 P = R Q k with following meaning, Absorbing states Transient states Absorbing states I k 0 Transient states R Q with I k the k k identity matrix, 0 an k (N k) matrix of zeros, R an (N k) k matrix and Q an (N k) (N k) matrix.

50 The matrix I N k Q is invertible. Let F =(I N k Q) 1 be the fundamental matrix of the Markov chain T i be the sum of the entries on row i of F B = FR Answers to our remaining questions: 2 F ij is the average number of times the process is in the jth transient state if it starts in the ith transient state 3 T i is the average number of steps before the process enters an absorbing state if it starts in the ith transient state 4 B ij is the probability of eventually entering the jth absorbing state if the process starts in the ith transient state

51 Back to the SIS model ODE: I 0 = (N I )I ( + d)i This equation has 3 components: 1 (N I )I new infection 2 I recovery 3 di death In the DTMC, want to know what transitions mean and what their probabilities are Infection Infection Infection I=0 I=1 I=2 I=3 I=N-1 I=N Recovery or Death Recovery or Death Recovery or Death Recovery or Death

52 Consider small amount of time t, su ciently small that only one event occurs: new infection, loss of infectious individual, nothing.. Let B(i) := (N i)i D(i) :=( + d)i be rates of birth and death of infectious. Then p ji ( t) =P {I (t + )=j I(t) =i} 8 B(i) t j = i +1 >< D(i) t j = i 1 = 1 [B(i)+D(i)] t j = i >: 0 otherwise β( N 1)Δ t β( N 2)2 Δ t β( N 1)Δ t Infection Infection Infection I=0 I=1 I=2 I=3 I=N-1 I=N Recovery Recovery Recovery Recovery or Death or Death or Death or Death (γ+d )Δ t (γ+d )2 Δt (γ+d )3 Δ t (γ+d ) N Δ t

53 Transition matrix D(1) t 0 1 (B(1) + D(1)) t 0 B(1) t B.... A D(N) t B(N 1) t 1 D(N) t Note that indices in the matrix range from 0 to N here

54 Properties p 0 is an absorbing state For any initial distribution p(0) = (p 0 (t), p 1 (0),...,p N (0)), lim p(t) =(1, t!1 0,...,0)T lim p 0 (t) =1 t!1 Exists quasi-stationary distribution conditioned on non-extinction q i (t) = p i(t) 1 p 0 (t), i =1,...,N

55 Consequence of absorption We have lim p(t) =(1, t!1 0,...,0)T lim p 0 (t) =1 t!1 but time to absorption can be exponential

56 1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

57 Several ways to formulate CTMC s A continuous time Markov chain can be formulated in terms of infinitesimal transition probabilities branching process time to next event Here, time is in R +

58 For small t, p ji ( t) =P {I (t + )=j I(t) =i} 8 B(i) t + o( t) j = i +1 >< D(i) t + o( t) j = i 1 = 1 [B(i)+D(i)] t + o( t) j = i >: o( t) otherwise with o( t)! 0as t! 0

59 Forward Kolmogorov equations Assume we know I (0) = k. Then p i (t + t) =p i 1 (t)b(i 1) t + p i+1 (t)d(i + 1) t + p i (t)[1 (B(i)+D(i)) t]+o( t) Compute (p i (t + t) p i (t))/ t and take lim t!0, giving d dt p 0 = p 1 D(1) d dt p i = p i 1 B(i 1) + p i+1 D(i + 1) p i [B(i)+D(i)] i =1,...,N Forward Kolmogorov equations associated to the CTMC

60 Write previous system as In vector form p 0 = Qp with D(1) (B(1) + D(1)) D(2) 0 Q = 0 B(1) (B(2) + D(2)) 0 B D(N) A D(N) Q generator matrix. Of course, p(t) =e Qt p(0)

61 Linking DTMC and CTMC for small t DTMC: p(t + t) =P( t)p(t) for transition matrix P( t). Let t! 0, obtain Kolmogorov equations for CTMC d dt p = Qp where P( t) Q = lim t!0 t I = P 0 (0)

62 Going ODE to CTMC Easy to convert compartmental ODE model to CTMC!!

63 The general setting: flow diagram b S βsi γ I I ds di

64 ODE approach b S βsi γ I I ds di Focus on compartments b S βsi γ I βsi γ I I ds di

65 CTMC approach b S βsi γ I I ds di b Focus on arrows (transitions) S ds βsi γ I I di

66 Focus on transitions b S βsi γ I I ds di b: S! S +1 birth ds: S! S 1 death of susceptible SI : S! S 1, I! I + 1 new infection I : S! S + 1, I! I 1 recovery di : I! I 1 death of infectious Will use S = N I and omit S dynamics

67 Gillespie s algorithm (SIS model) 1: Set t t 0 and I (t) I (t 0 ) 2: while t apple t f do 3: t (N i)i + i + di 4: Draw t from T s E( t ) 5: v [ (N i)i/ t, (N i)i/ t + i/ t, 1] 6: Draw t from U([0, 1]) 7: Find pos such that t v(pos) 8: switch (pos) 9: case 1: New infection, I (t + t )=I(t)+1 10: case 2: End of infectious period, I (t + t )=I(t) 1 11: case 3: Death, I (t + t )=I(t) 1 12: end switch 13: t t + t 14: end while

68 IMPORTANT: in Gillespie s algorithm, we do not consider p ii,the event nothing happens

69 Drawing at random from an exponential distribution Want t from T s E( t ), i.e., T has probability density function f (x, t )= t e tx 1 x 0 Use cumulative distribution function F (x, t )= R x1 f (s, t) ds F (x, t )=(1 e tx )1 x 0 which has values in [0, 1]. So draw from U([0, 1]) and solve F (x, t )= for x: F (x, t )=, 1 e tx =, e tx =1, t x = ln(1 ), x = ln(1 ) t

Stochastic modelling of epidemic spread

Stochastic modelling of epidemic spread Stochastic modelling of epidemic spread Julien Arino Centre for Research on Inner City Health St Michael s Hospital Toronto On leave from Department of Mathematics University of Manitoba Julien Arino@umanitoba.ca

More information

An Introduction to Stochastic Epidemic Models

An Introduction to Stochastic Epidemic Models An Introduction to Stochastic Epidemic Models Linda J. S. Allen Department of Mathematics and Statistics Texas Tech University Lubbock, Texas 79409-1042, U.S.A. linda.j.allen@ttu.edu 1 Introduction The

More information

AN INTRODUCTION TO STOCHASTIC EPIDEMIC MODELS-PART I

AN INTRODUCTION TO STOCHASTIC EPIDEMIC MODELS-PART I AN INTRODUCTION TO STOCHASTIC EPIDEMIC MODELS-PART I Linda J. S. Allen Department of Mathematics and Statistics Texas Tech University Lubbock, Texas U.S.A. 2008 Summer School on Mathematical Modeling of

More information

Stochastic process. X, a series of random variables indexed by t

Stochastic process. X, a series of random variables indexed by t Stochastic process X, a series of random variables indexed by t X={X(t), t 0} is a continuous time stochastic process X={X(t), t=0,1, } is a discrete time stochastic process X(t) is the state at time t,

More information

Irreducibility. Irreducible. every state can be reached from every other state For any i,j, exist an m 0, such that. Absorbing state: p jj =1

Irreducibility. Irreducible. every state can be reached from every other state For any i,j, exist an m 0, such that. Absorbing state: p jj =1 Irreducibility Irreducible every state can be reached from every other state For any i,j, exist an m 0, such that i,j are communicate, if the above condition is valid Irreducible: all states are communicate

More information

MARKOV PROCESSES. Valerio Di Valerio

MARKOV PROCESSES. Valerio Di Valerio MARKOV PROCESSES Valerio Di Valerio Stochastic Process Definition: a stochastic process is a collection of random variables {X(t)} indexed by time t T Each X(t) X is a random variable that satisfy some

More information

AARMS Homework Exercises

AARMS Homework Exercises 1 For the gamma distribution, AARMS Homework Exercises (a) Show that the mgf is M(t) = (1 βt) α for t < 1/β (b) Use the mgf to find the mean and variance of the gamma distribution 2 A well-known inequality

More information

2. Transience and Recurrence

2. Transience and Recurrence Virtual Laboratories > 15. Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12 2. Transience and Recurrence The study of Markov chains, particularly the limiting behavior, depends critically on the random times

More information

The SIS and SIR stochastic epidemic models revisited

The SIS and SIR stochastic epidemic models revisited The SIS and SIR stochastic epidemic models revisited Jesús Artalejo Faculty of Mathematics, University Complutense of Madrid Madrid, Spain jesus_artalejomat.ucm.es BCAM Talk, June 16, 2011 Talk Schedule

More information

Markov Chains, Stochastic Processes, and Matrix Decompositions

Markov Chains, Stochastic Processes, and Matrix Decompositions Markov Chains, Stochastic Processes, and Matrix Decompositions 5 May 2014 Outline 1 Markov Chains Outline 1 Markov Chains 2 Introduction Perron-Frobenius Matrix Decompositions and Markov Chains Spectral

More information

Thursday. Threshold and Sensitivity Analysis

Thursday. Threshold and Sensitivity Analysis Thursday Threshold and Sensitivity Analysis SIR Model without Demography ds dt di dt dr dt = βsi (2.1) = βsi γi (2.2) = γi (2.3) With initial conditions S(0) > 0, I(0) > 0, and R(0) = 0. This model can

More information

LIMITING PROBABILITY TRANSITION MATRIX OF A CONDENSED FIBONACCI TREE

LIMITING PROBABILITY TRANSITION MATRIX OF A CONDENSED FIBONACCI TREE International Journal of Applied Mathematics Volume 31 No. 18, 41-49 ISSN: 1311-178 (printed version); ISSN: 1314-86 (on-line version) doi: http://dx.doi.org/1.173/ijam.v31i.6 LIMITING PROBABILITY TRANSITION

More information

Outlines. Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC)

Outlines. Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC) Markov Chains (2) Outlines Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC) 2 pj ( n) denotes the pmf of the random variable p ( n) P( X j) j We will only be concerned with homogenous

More information

Derivation of Itô SDE and Relationship to ODE and CTMC Models

Derivation of Itô SDE and Relationship to ODE and CTMC Models Derivation of Itô SDE and Relationship to ODE and CTMC Models Biomathematics II April 23, 2015 Linda J. S. Allen Texas Tech University TTU 1 Euler-Maruyama Method for Numerical Solution of an Itô SDE dx(t)

More information

Statistics 992 Continuous-time Markov Chains Spring 2004

Statistics 992 Continuous-time Markov Chains Spring 2004 Summary Continuous-time finite-state-space Markov chains are stochastic processes that are widely used to model the process of nucleotide substitution. This chapter aims to present much of the mathematics

More information

Stochastic processes. MAS275 Probability Modelling. Introduction and Markov chains. Continuous time. Markov property

Stochastic processes. MAS275 Probability Modelling. Introduction and Markov chains. Continuous time. Markov property Chapter 1: and Markov chains Stochastic processes We study stochastic processes, which are families of random variables describing the evolution of a quantity with time. In some situations, we can treat

More information

Transmission in finite populations

Transmission in finite populations Transmission in finite populations Juliet Pulliam, PhD Department of Biology and Emerging Pathogens Institute University of Florida and RAPIDD Program, DIEPS Fogarty International Center US National Institutes

More information

Lecture 20 : Markov Chains

Lecture 20 : Markov Chains CSCI 3560 Probability and Computing Instructor: Bogdan Chlebus Lecture 0 : Markov Chains We consider stochastic processes. A process represents a system that evolves through incremental changes called

More information

6 Continuous-Time Birth and Death Chains

6 Continuous-Time Birth and Death Chains 6 Continuous-Time Birth and Death Chains Angela Peace Biomathematics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology.

More information

Stochastic2010 Page 1

Stochastic2010 Page 1 Stochastic2010 Page 1 Extinction Probability for Branching Processes Friday, April 02, 2010 2:03 PM Long-time properties for branching processes Clearly state 0 is an absorbing state, forming its own recurrent

More information

Lecture 11: Introduction to Markov Chains. Copyright G. Caire (Sample Lectures) 321

Lecture 11: Introduction to Markov Chains. Copyright G. Caire (Sample Lectures) 321 Lecture 11: Introduction to Markov Chains Copyright G. Caire (Sample Lectures) 321 Discrete-time random processes A sequence of RVs indexed by a variable n 2 {0, 1, 2,...} forms a discretetime random process

More information

Example: physical systems. If the state space. Example: speech recognition. Context can be. Example: epidemics. Suppose each infected

Example: physical systems. If the state space. Example: speech recognition. Context can be. Example: epidemics. Suppose each infected 4. Markov Chains A discrete time process {X n,n = 0,1,2,...} with discrete state space X n {0,1,2,...} is a Markov chain if it has the Markov property: P[X n+1 =j X n =i,x n 1 =i n 1,...,X 0 =i 0 ] = P[X

More information

Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and

Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and private study only. The thesis may not be reproduced elsewhere

More information

http://www.math.uah.edu/stat/markov/.xhtml 1 of 9 7/16/2009 7:20 AM Virtual Laboratories > 16. Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12 1. A Markov process is a random process in which the future is

More information

APM 541: Stochastic Modelling in Biology Discrete-time Markov Chains. Jay Taylor Fall Jay Taylor (ASU) APM 541 Fall / 92

APM 541: Stochastic Modelling in Biology Discrete-time Markov Chains. Jay Taylor Fall Jay Taylor (ASU) APM 541 Fall / 92 APM 541: Stochastic Modelling in Biology Discrete-time Markov Chains Jay Taylor Fall 2013 Jay Taylor (ASU) APM 541 Fall 2013 1 / 92 Outline 1 Motivation 2 Markov Processes 3 Markov Chains: Basic Properties

More information

Markov Processes Hamid R. Rabiee

Markov Processes Hamid R. Rabiee Markov Processes Hamid R. Rabiee Overview Markov Property Markov Chains Definition Stationary Property Paths in Markov Chains Classification of States Steady States in MCs. 2 Markov Property A discrete

More information

Convex Optimization CMU-10725

Convex Optimization CMU-10725 Convex Optimization CMU-10725 Simulated Annealing Barnabás Póczos & Ryan Tibshirani Andrey Markov Markov Chains 2 Markov Chains Markov chain: Homogen Markov chain: 3 Markov Chains Assume that the state

More information

Markov Chains CK eqns Classes Hitting times Rec./trans. Strong Markov Stat. distr. Reversibility * Markov Chains

Markov Chains CK eqns Classes Hitting times Rec./trans. Strong Markov Stat. distr. Reversibility * Markov Chains Markov Chains A random process X is a family {X t : t T } of random variables indexed by some set T. When T = {0, 1, 2,... } one speaks about a discrete-time process, for T = R or T = [0, ) one has a continuous-time

More information

Section 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns.

Section 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. Section 9.2: Matrices Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. That is, a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn A

More information

CDA6530: Performance Models of Computers and Networks. Chapter 3: Review of Practical Stochastic Processes

CDA6530: Performance Models of Computers and Networks. Chapter 3: Review of Practical Stochastic Processes CDA6530: Performance Models of Computers and Networks Chapter 3: Review of Practical Stochastic Processes Definition Stochastic process X = {X(t), t2 T} is a collection of random variables (rvs); one rv

More information

Part I Stochastic variables and Markov chains

Part I Stochastic variables and Markov chains Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)

More information

Lecture 20: Reversible Processes and Queues

Lecture 20: Reversible Processes and Queues Lecture 20: Reversible Processes and Queues 1 Examples of reversible processes 11 Birth-death processes We define two non-negative sequences birth and death rates denoted by {λ n : n N 0 } and {µ n : n

More information

STOCHASTIC PROCESSES Basic notions

STOCHASTIC PROCESSES Basic notions J. Virtamo 38.3143 Queueing Theory / Stochastic processes 1 STOCHASTIC PROCESSES Basic notions Often the systems we consider evolve in time and we are interested in their dynamic behaviour, usually involving

More information

Perron Frobenius Theory

Perron Frobenius Theory Perron Frobenius Theory Oskar Perron Georg Frobenius (1880 1975) (1849 1917) Stefan Güttel Perron Frobenius Theory 1 / 10 Positive and Nonnegative Matrices Let A, B R m n. A B if a ij b ij i, j, A > B

More information

Probability Distributions

Probability Distributions Lecture : Background in Probability Theory Probability Distributions The probability mass function (pmf) or probability density functions (pdf), mean, µ, variance, σ 2, and moment generating function (mgf)

More information

SMSTC (2007/08) Probability.

SMSTC (2007/08) Probability. SMSTC (27/8) Probability www.smstc.ac.uk Contents 12 Markov chains in continuous time 12 1 12.1 Markov property and the Kolmogorov equations.................... 12 2 12.1.1 Finite state space.................................

More information

Lecture 21. David Aldous. 16 October David Aldous Lecture 21

Lecture 21. David Aldous. 16 October David Aldous Lecture 21 Lecture 21 David Aldous 16 October 2015 In continuous time 0 t < we specify transition rates or informally P(X (t+δ)=j X (t)=i, past ) q ij = lim δ 0 δ P(X (t + dt) = j X (t) = i) = q ij dt but note these

More information

6.207/14.15: Networks Lectures 4, 5 & 6: Linear Dynamics, Markov Chains, Centralities

6.207/14.15: Networks Lectures 4, 5 & 6: Linear Dynamics, Markov Chains, Centralities 6.207/14.15: Networks Lectures 4, 5 & 6: Linear Dynamics, Markov Chains, Centralities 1 Outline Outline Dynamical systems. Linear and Non-linear. Convergence. Linear algebra and Lyapunov functions. Markov

More information

MATH 56A: STOCHASTIC PROCESSES CHAPTER 1

MATH 56A: STOCHASTIC PROCESSES CHAPTER 1 MATH 56A: STOCHASTIC PROCESSES CHAPTER. Finite Markov chains For the sake of completeness of these notes I decided to write a summary of the basic concepts of finite Markov chains. The topics in this chapter

More information

Discrete Time Markov Chain of a Dynamical System with a Rest Phase

Discrete Time Markov Chain of a Dynamical System with a Rest Phase Discrete Time Markov Chain of a Dynamical System with a Rest Phase Abstract A stochastic model, in the form of a discrete time Markov chain, is constructed to describe the dynamics of a population that

More information

Three Disguises of 1 x = e λx

Three Disguises of 1 x = e λx Three Disguises of 1 x = e λx Chathuri Karunarathna Mudiyanselage Rabi K.C. Winfried Just Department of Mathematics, Ohio University Mathematical Biology and Dynamical Systems Seminar Ohio University November

More information

Homework 4 due on Thursday, December 15 at 5 PM (hard deadline).

Homework 4 due on Thursday, December 15 at 5 PM (hard deadline). Large-Time Behavior for Continuous-Time Markov Chains Friday, December 02, 2011 10:58 AM Homework 4 due on Thursday, December 15 at 5 PM (hard deadline). How are formulas for large-time behavior of discrete-time

More information

Probability Distributions

Probability Distributions Lecture 1: Background in Probability Theory Probability Distributions The probability mass function (pmf) or probability density functions (pdf), mean, µ, variance, σ 2, and moment generating function

More information

IEOR 6711, HMWK 5, Professor Sigman

IEOR 6711, HMWK 5, Professor Sigman IEOR 6711, HMWK 5, Professor Sigman 1. Semi-Markov processes: Consider an irreducible positive recurrent discrete-time Markov chain {X n } with transition matrix P (P i,j ), i, j S, and finite state space.

More information

8. Statistical Equilibrium and Classification of States: Discrete Time Markov Chains

8. Statistical Equilibrium and Classification of States: Discrete Time Markov Chains 8. Statistical Equilibrium and Classification of States: Discrete Time Markov Chains 8.1 Review 8.2 Statistical Equilibrium 8.3 Two-State Markov Chain 8.4 Existence of P ( ) 8.5 Classification of States

More information

18.175: Lecture 30 Markov chains

18.175: Lecture 30 Markov chains 18.175: Lecture 30 Markov chains Scott Sheffield MIT Outline Review what you know about finite state Markov chains Finite state ergodicity and stationarity More general setup Outline Review what you know

More information

Mean-field dual of cooperative reproduction

Mean-field dual of cooperative reproduction The mean-field dual of systems with cooperative reproduction joint with Tibor Mach (Prague) A. Sturm (Göttingen) Friday, July 6th, 2018 Poisson construction of Markov processes Let (X t ) t 0 be a continuous-time

More information

Introduction to SEIR Models

Introduction to SEIR Models Department of Epidemiology and Public Health Health Systems Research and Dynamical Modelling Unit Introduction to SEIR Models Nakul Chitnis Workshop on Mathematical Models of Climate Variability, Environmental

More information

Math Homework 5 Solutions

Math Homework 5 Solutions Math 45 - Homework 5 Solutions. Exercise.3., textbook. The stochastic matrix for the gambler problem has the following form, where the states are ordered as (,, 4, 6, 8, ): P = The corresponding diagram

More information

Availability. M(t) = 1 - e -mt

Availability. M(t) = 1 - e -mt Availability Availability - A(t) the probability that the system is operating correctly and is available to perform its functions at the instant of time t More general concept than reliability: failure

More information

IEOR 6711: Professor Whitt. Introduction to Markov Chains

IEOR 6711: Professor Whitt. Introduction to Markov Chains IEOR 6711: Professor Whitt Introduction to Markov Chains 1. Markov Mouse: The Closed Maze We start by considering how to model a mouse moving around in a maze. The maze is a closed space containing nine

More information

Definition A finite Markov chain is a memoryless homogeneous discrete stochastic process with a finite number of states.

Definition A finite Markov chain is a memoryless homogeneous discrete stochastic process with a finite number of states. Chapter 8 Finite Markov Chains A discrete system is characterized by a set V of states and transitions between the states. V is referred to as the state space. We think of the transitions as occurring

More information

Recap. Probability, stochastic processes, Markov chains. ELEC-C7210 Modeling and analysis of communication networks

Recap. Probability, stochastic processes, Markov chains. ELEC-C7210 Modeling and analysis of communication networks Recap Probability, stochastic processes, Markov chains ELEC-C7210 Modeling and analysis of communication networks 1 Recap: Probability theory important distributions Discrete distributions Geometric distribution

More information

The Transition Probability Function P ij (t)

The Transition Probability Function P ij (t) The Transition Probability Function P ij (t) Consider a continuous time Markov chain {X(t), t 0}. We are interested in the probability that in t time units the process will be in state j, given that it

More information

No class on Thursday, October 1. No office hours on Tuesday, September 29 and Thursday, October 1.

No class on Thursday, October 1. No office hours on Tuesday, September 29 and Thursday, October 1. Stationary Distributions Monday, September 28, 2015 2:02 PM No class on Thursday, October 1. No office hours on Tuesday, September 29 and Thursday, October 1. Homework 1 due Friday, October 2 at 5 PM strongly

More information

Probability, Random Processes and Inference

Probability, Random Processes and Inference INSTITUTO POLITÉCNICO NACIONAL CENTRO DE INVESTIGACION EN COMPUTACION Laboratorio de Ciberseguridad Probability, Random Processes and Inference Dr. Ponciano Jorge Escamilla Ambrosio pescamilla@cic.ipn.mx

More information

2 Discrete-Time Markov Chains

2 Discrete-Time Markov Chains 2 Discrete-Time Markov Chains Angela Peace Biomathematics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,

More information

Epidemics in Networks Part 2 Compartmental Disease Models

Epidemics in Networks Part 2 Compartmental Disease Models Epidemics in Networks Part 2 Compartmental Disease Models Joel C. Miller & Tom Hladish 18 20 July 2018 1 / 35 Introduction to Compartmental Models Dynamics R 0 Epidemic Probability Epidemic size Review

More information

1.3 Convergence of Regular Markov Chains

1.3 Convergence of Regular Markov Chains Markov Chains and Random Walks on Graphs 3 Applying the same argument to A T, which has the same λ 0 as A, yields the row sum bounds Corollary 0 Let P 0 be the transition matrix of a regular Markov chain

More information

Markov Chains. As part of Interdisciplinary Mathematical Modeling, By Warren Weckesser Copyright c 2006.

Markov Chains. As part of Interdisciplinary Mathematical Modeling, By Warren Weckesser Copyright c 2006. Markov Chains As part of Interdisciplinary Mathematical Modeling, By Warren Weckesser Copyright c 2006 1 Introduction A (finite) Markov chain is a process with a finite number of states (or outcomes, or

More information

Matrix analytic methods. Lecture 1: Structured Markov chains and their stationary distribution

Matrix analytic methods. Lecture 1: Structured Markov chains and their stationary distribution 1/29 Matrix analytic methods Lecture 1: Structured Markov chains and their stationary distribution Sophie Hautphenne and David Stanford (with thanks to Guy Latouche, U. Brussels and Peter Taylor, U. Melbourne

More information

IEOR 6711: Stochastic Models I Professor Whitt, Thursday, November 29, Weirdness in CTMC s

IEOR 6711: Stochastic Models I Professor Whitt, Thursday, November 29, Weirdness in CTMC s IEOR 6711: Stochastic Models I Professor Whitt, Thursday, November 29, 2012 Weirdness in CTMC s Where s your will to be weird? Jim Morrison, The Doors We are all a little weird. And life is a little weird.

More information

Birth and Death Processes. Birth and Death Processes. Linear Growth with Immigration. Limiting Behaviour for Birth and Death Processes

Birth and Death Processes. Birth and Death Processes. Linear Growth with Immigration. Limiting Behaviour for Birth and Death Processes DTU Informatics 247 Stochastic Processes 6, October 27 Today: Limiting behaviour of birth and death processes Birth and death processes with absorbing states Finite state continuous time Markov chains

More information

Nonnegative Matrices I

Nonnegative Matrices I Nonnegative Matrices I Daisuke Oyama Topics in Economic Theory September 26, 2017 References J. L. Stuart, Digraphs and Matrices, in Handbook of Linear Algebra, Chapter 29, 2006. R. A. Brualdi and H. J.

More information

Lecture 15 Perron-Frobenius Theory

Lecture 15 Perron-Frobenius Theory EE363 Winter 2005-06 Lecture 15 Perron-Frobenius Theory Positive and nonnegative matrices and vectors Perron-Frobenius theorems Markov chains Economic growth Population dynamics Max-min and min-max characterization

More information

Stochastic Models. John M. Drake & Pejman Rohani

Stochastic Models. John M. Drake & Pejman Rohani Epidemiological data are noisy Two types of noise: Observation error: the data are probabilistically related to the true state of the system Process noise: the system progresses probabilistically Environmental

More information

Markov Chains and Stochastic Sampling

Markov Chains and Stochastic Sampling Part I Markov Chains and Stochastic Sampling 1 Markov Chains and Random Walks on Graphs 1.1 Structure of Finite Markov Chains We shall only consider Markov chains with a finite, but usually very large,

More information

Section 9.2: Matrices.. a m1 a m2 a mn

Section 9.2: Matrices.. a m1 a m2 a mn Section 9.2: Matrices Definition: A matrix is a rectangular array of numbers: a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn In general, a ij denotes the (i, j) entry of A. That is, the entry in

More information

Data analysis and stochastic modeling

Data analysis and stochastic modeling Data analysis and stochastic modeling Lecture 7 An introduction to queueing theory Guillaume Gravier guillaume.gravier@irisa.fr with a lot of help from Paul Jensen s course http://www.me.utexas.edu/ jensen/ormm/instruction/powerpoint/or_models_09/14_queuing.ppt

More information

Markov Chains and Pandemics

Markov Chains and Pandemics Markov Chains and Pandemics Caleb Dedmore and Brad Smith December 8, 2016 Page 1 of 16 Abstract Markov Chain Theory is a powerful tool used in statistical analysis to make predictions about future events

More information

Introduction to Machine Learning CMU-10701

Introduction to Machine Learning CMU-10701 Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov

More information

Social network analysis: social learning

Social network analysis: social learning Social network analysis: social learning Donglei Du (ddu@unb.edu) Faculty of Business Administration, University of New Brunswick, NB Canada Fredericton E3B 9Y2 October 20, 2016 Donglei Du (UNB) AlgoTrading

More information

System theory and system identification of compartmental systems Hof, Jacoba Marchiena van den

System theory and system identification of compartmental systems Hof, Jacoba Marchiena van den University of Groningen System theory and system identification of compartmental systems Hof, Jacoba Marchiena van den IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF)

More information

Let (Ω, F) be a measureable space. A filtration in discrete time is a sequence of. F s F t

Let (Ω, F) be a measureable space. A filtration in discrete time is a sequence of. F s F t 2.2 Filtrations Let (Ω, F) be a measureable space. A filtration in discrete time is a sequence of σ algebras {F t } such that F t F and F t F t+1 for all t = 0, 1,.... In continuous time, the second condition

More information

Markov Chains, Random Walks on Graphs, and the Laplacian

Markov Chains, Random Walks on Graphs, and the Laplacian Markov Chains, Random Walks on Graphs, and the Laplacian CMPSCI 791BB: Advanced ML Sridhar Mahadevan Random Walks! There is significant interest in the problem of random walks! Markov chain analysis! Computer

More information

Statistics 253/317 Introduction to Probability Models. Winter Midterm Exam Monday, Feb 10, 2014

Statistics 253/317 Introduction to Probability Models. Winter Midterm Exam Monday, Feb 10, 2014 Statistics 253/317 Introduction to Probability Models Winter 2014 - Midterm Exam Monday, Feb 10, 2014 Student Name (print): (a) Do not sit directly next to another student. (b) This is a closed-book, closed-note

More information

Markov Chains Absorption (cont d) Hamid R. Rabiee

Markov Chains Absorption (cont d) Hamid R. Rabiee Markov Chains Absorption (cont d) Hamid R. Rabiee 1 Absorbing Markov Chain An absorbing state is one in which the probability that the process remains in that state once it enters the state is 1 (i.e.,

More information

10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections )

10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections ) c Dr. Igor Zelenko, Fall 2017 1 10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections 7.2-7.4) 1. When each of the functions F 1, F 2,..., F n in right-hand side

More information

1 Stochastic Dynamic Programming

1 Stochastic Dynamic Programming 1 Stochastic Dynamic Programming Formally, a stochastic dynamic program has the same components as a deterministic one; the only modification is to the state transition equation. When events in the future

More information

Markov chains. Randomness and Computation. Markov chains. Markov processes

Markov chains. Randomness and Computation. Markov chains. Markov processes Markov chains Randomness and Computation or, Randomized Algorithms Mary Cryan School of Informatics University of Edinburgh Definition (Definition 7) A discrete-time stochastic process on the state space

More information

Finite-Horizon Statistics for Markov chains

Finite-Horizon Statistics for Markov chains Analyzing FSDT Markov chains Friday, September 30, 2011 2:03 PM Simulating FSDT Markov chains, as we have said is very straightforward, either by using probability transition matrix or stochastic update

More information

Censoring Technique in Studying Block-Structured Markov Chains

Censoring Technique in Studying Block-Structured Markov Chains Censoring Technique in Studying Block-Structured Markov Chains Yiqiang Q. Zhao 1 Abstract: Markov chains with block-structured transition matrices find many applications in various areas. Such Markov chains

More information

Markov chains. 1 Discrete time Markov chains. c A. J. Ganesh, University of Bristol, 2015

Markov chains. 1 Discrete time Markov chains. c A. J. Ganesh, University of Bristol, 2015 Markov chains c A. J. Ganesh, University of Bristol, 2015 1 Discrete time Markov chains Example: A drunkard is walking home from the pub. There are n lampposts between the pub and his home, at each of

More information

The Theory behind PageRank

The Theory behind PageRank The Theory behind PageRank Mauro Sozio Telecom ParisTech May 21, 2014 Mauro Sozio (LTCI TPT) The Theory behind PageRank May 21, 2014 1 / 19 A Crash Course on Discrete Probability Events and Probability

More information

Markov-modulated interactions in SIR epidemics

Markov-modulated interactions in SIR epidemics Markov-modulated interactions in SIR epidemics E. Almaraz 1, A. Gómez-Corral 2 (1)Departamento de Estadística e Investigación Operativa, Facultad de Ciencias Matemáticas (UCM), (2)Instituto de Ciencias

More information

Lectures on Markov Chains

Lectures on Markov Chains Lectures on Markov Chains David M. McClendon Department of Mathematics Ferris State University 2016 edition 1 Contents Contents 2 1 Markov chains 4 1.1 The definition of a Markov chain.....................

More information

Markov Model. Model representing the different resident states of a system, and the transitions between the different states

Markov Model. Model representing the different resident states of a system, and the transitions between the different states Markov Model Model representing the different resident states of a system, and the transitions between the different states (applicable to repairable, as well as non-repairable systems) System behavior

More information

Numerical solution of stochastic epidemiological models

Numerical solution of stochastic epidemiological models Numerical solution of stochastic epidemiological models John M. Drake & Pejman Rohani 1 Introduction He we expand our modeling toolkit to include methods for studying stochastic versions of the compartmental

More information

An Introduction to Entropy and Subshifts of. Finite Type

An Introduction to Entropy and Subshifts of. Finite Type An Introduction to Entropy and Subshifts of Finite Type Abby Pekoske Department of Mathematics Oregon State University pekoskea@math.oregonstate.edu August 4, 2015 Abstract This work gives an overview

More information

(b) What is the variance of the time until the second customer arrives, starting empty, assuming that we measure time in minutes?

(b) What is the variance of the time until the second customer arrives, starting empty, assuming that we measure time in minutes? IEOR 3106: Introduction to Operations Research: Stochastic Models Fall 2006, Professor Whitt SOLUTIONS to Final Exam Chapters 4-7 and 10 in Ross, Tuesday, December 19, 4:10pm-7:00pm Open Book: but only

More information

LAW OF LARGE NUMBERS FOR THE SIRS EPIDEMIC

LAW OF LARGE NUMBERS FOR THE SIRS EPIDEMIC LAW OF LARGE NUMBERS FOR THE SIRS EPIDEMIC R. G. DOLGOARSHINNYKH Abstract. We establish law of large numbers for SIRS stochastic epidemic processes: as the population size increases the paths of SIRS epidemic

More information

Modelling in Systems Biology

Modelling in Systems Biology Modelling in Systems Biology Maria Grazia Vigliotti thanks to my students Anton Stefanek, Ahmed Guecioueur Imperial College Formal representation of chemical reactions precise qualitative and quantitative

More information

LTCC. Exercises solutions

LTCC. Exercises solutions 1. Markov chain LTCC. Exercises solutions (a) Draw a state space diagram with the loops for the possible steps. If the chain starts in state 4, it must stay there. If the chain starts in state 1, it will

More information

STAT STOCHASTIC PROCESSES. Contents

STAT STOCHASTIC PROCESSES. Contents STAT 3911 - STOCHASTIC PROCESSES ANDREW TULLOCH Contents 1. Stochastic Processes 2 2. Classification of states 2 3. Limit theorems for Markov chains 4 4. First step analysis 5 5. Branching processes 5

More information

Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank

Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank David Glickenstein November 3, 4 Representing graphs as matrices It will sometimes be useful to represent graphs

More information

Statistics 150: Spring 2007

Statistics 150: Spring 2007 Statistics 150: Spring 2007 April 23, 2008 0-1 1 Limiting Probabilities If the discrete-time Markov chain with transition probabilities p ij is irreducible and positive recurrent; then the limiting probabilities

More information

Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations

Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2

More information

CDA5530: Performance Models of Computers and Networks. Chapter 3: Review of Practical

CDA5530: Performance Models of Computers and Networks. Chapter 3: Review of Practical CDA5530: Performance Models of Computers and Networks Chapter 3: Review of Practical Stochastic Processes Definition Stochastic ti process X = {X(t), t T} is a collection of random variables (rvs); one

More information

STA 624 Practice Exam 2 Applied Stochastic Processes Spring, 2008

STA 624 Practice Exam 2 Applied Stochastic Processes Spring, 2008 Name STA 624 Practice Exam 2 Applied Stochastic Processes Spring, 2008 There are five questions on this test. DO use calculators if you need them. And then a miracle occurs is not a valid answer. There

More information