AN INTRODUCTION TO STOCHASTIC EPIDEMIC MODELS-PART I

Size: px
Start display at page:

Download "AN INTRODUCTION TO STOCHASTIC EPIDEMIC MODELS-PART I"

Transcription

1 AN INTRODUCTION TO STOCHASTIC EPIDEMIC MODELS-PART I Linda J. S. Allen Department of Mathematics and Statistics Texas Tech University Lubbock, Texas U.S.A Summer School on Mathematical Modeling of Infectious Diseases University of Alberta May 1-11, 2008

2 Outline of Presentation-PART I I. What is a Stochastic Model and What is the Difference Between a Deterministic and Stochastic Model? II. Some Stochastic Models are Illustrated Through Study of an SIS Epidemic Model. (a) Discrete Time Markov Chain DTMC (b) Continuous Time Markov Chain CTMC (c) Diffusion Process and Stochastic Differential Equations SDE III. Some Differences Between the Stochastic SIS and SIR Epidemic Models.

3 I. What is a Stochastic Model? A stochastic model is formulated in terms of a stochastic process. A stochastic process is a collection of random variables {X t (s) t T,s S}, where T is the index set and S is a common sample space. The index set often represents time, such as T = {0, 1, 2,...} or T = [0, ) Time can be discrete or continuous. The study of stochastic processes is based on probability theory.

4 How do Stochastic Epidemic Models Differ from Deterministic Epidemic Models? A deterministic model is often formulated in terms of a system of differential equations or difference equations. A stochastic model is formulated as a stochastic process with a collection of random variables. A solution of a deterministic model is a function of time or space and is generally uniquely dependent on the initial data. A solution of a stochastic model is a probability distribution for each of the random variables. One sample path over time or space is one realization from this distribution. Stochastic models are often used to show the variability inherent due to the demographics or environment variablility are particularly important when quantities in the processes are small- small population size or initial number of infectives.

5 The Following Graphs Illustrate the Dynamics of a Deterministic versus a Stochastic Epidemic Model Number of Infectives, I(t) Time Steps

6 Whether the Random Variables Associated with The Stochastic Process are Discrete or Continuous Distinguishes Some of the Different Types of Stochastic Models. A random variable X(t) of a stochastic process assigns a real value to each outcome A S in the sample space and a probability, Prob{X(t) A} [0, 1]. The values of the random variable constitute the state space, X(t; S). For example, the number of cases associated with a disease may have the following discrete or continuous set of values for its state space: {0, 1, 2,...} or [0, N]. The state space can be discrete or continuous and correspondingly, the random variable is discrete or continuous. For simplicity, the sample space notation is suppressed and X(t) is used to denote a random variable indexed by time t. The stochastic process is completely defined when the set of random variables {X(t)} are related by a set of rules.

7 We will Study Stochastic Processes that have the Markov Property. A stochastic process with the Markov property is one where the future state of the process depends only on the current state, and not on the past. That is, for a discrete-time stochastic process, a d Prob{X(t + t) X(t),X(t t),...,x(0)} = Prob{X(t + t) X(t)}. At a fixed time t, each random variable X(t) has an associated probability distribution. Discrete: Prob{X(t) = i} = p i (t), i {0, 1,2...} Continuous: Prob{X(t) [a,b]} = b a p(x,t)dx

8 The Choice of Discrete or Continuous Random Variables with a Discrete or Continuous Index Set Defines the Type of Stochastic Model. Discrete Time Markov Chain (DTMC): t {0, t, 2 t,...} X(t) is a discrete random variable. X(t) {0,1, 2,...,N} The term chain implies that the random variable is discrete. Continuous Time Markov Chain (CTMC): t [0, ), X(t) is a discrete random variable. X t {0, 1,2,...,N} Diffusion Process, Stochastic Differential Equation (SDE): t [0, ), X(t) is a continuous random variable. X(t) [0,N]

9 II. Before we Formulate the Stochastic SIS Epidemic Models, we Review the Dynamics of the Deterministic SIS Epidemic Model. Deterministic SIS: S I ds dt di dt = β SI + (b + γ)i N = β SI (b + γ)i N where β > 0, γ > 0, N > 0 and b 0, S(t) + I(t) = N.

10 The Dynamics of the Deterministic SIS Epidemic Model Depend on the Basic Reproduction Number. The parameter values represent β = transmission rate b = birth rate = death rate γ = recovery rate N = total population size = constant. Basic Reproduction Number: R 0 = β b + γ If R 0 1, then lim t I(t) = 0. If R 0 > 1, then lim t I(t) = N (1 1R0 ).

11 We will Formulate the Three Types of Stochastic SIS Epidemic Models by Defining Relationships Among the Random Variables Assuming the Markov Property Holds. S(t) = random variable for the number of susceptible individuals. I(t) = random variable for the number of infected individuals. S(t) + I(t) = N = maximum population size. Discrete Time Markov Chain (DTMC): t {0, t, 2 t,...}, I(t) is a discrete random variable, I(t) {0, 1, 2,...,N} Continuous Time Markov Chain (CTMC): t [0, ), I(t) is a discrete random variable. I(t) {0, 1, 2,...,N} Diffusion Process, SDEs: t [0, ), I(t) is a continuous random variable. I(t) [0, N]

12 First, We Formulate a DTMC SIS Epidemic Model. Let I(t) denote the discrete random variable for the number of infected (and infectious) individuals with associated probability function p i (t) = Prob{I(t) = i} where i = 0,1, 2,...,N is the total number infected at time t. The probability distribution is p(t) = (p 0 (t), p 1 (t),...,p N (t)) T for t = 0, t, 2 t,.... Now we relate the random variables {I(t)} indexed by time t by defining the probability of a transition from state i to state j, i j, in time t as p ji ( t) = Prob{I(t + t) = j I(t) = i}.

13 Assume that t is Sufficiently Small, Such that the Number of Infectives Changes by at Most One in Time t. That is, i i + 1, i i 1 or i i. Either there is a new infection, birth, death, or a recovery. Therefore, the transition probabilities are p ji ( t) = βi(n i)/n t = b(i) t, j = i + 1 (b + γ)i t = d(i) t, j = i 1 1 [βi(n i)/n + (b + γ)i] t = 1 [b(i) + d(i)] t, j = i 0, j i + 1,i, i 1,

14 The Probability Distribution Associated with the Epidemic Process Over Time is Found by Repeated Multiplication of the Transition Matrix. Matrix P( t) = (p ji ( t)) is known as the transition matrix: p(t + t) = P( t)p(t), where p(t) = (p 0 (t),...,p N (t)) T is the probability distribution and P( t) is 1 d(1) t [b(1) + d(1)] t d(2) t 0 0 b(1) t 1 [b(2) + d(2)] t b(2) t d(n) t d(n) t Matrix P( t) is stochastic, the column sums equal one.

15 The Stochastic Process for the DTMC SIS Model is known as a Finite State Markov Chain with the Following Properties. The stochastic process {I(t)} for t {0, t, 2 t,...} is timehomogeneous (transition probabilities do not depend on time) and has the Markov property. The probability of no infections p 0 is an absorbing state N For any initial distribution p(0) = (p 0 (0),...,p N (0)) T, zero through a total of N infections lim p(t) = (1,0,...,0)T lim t p 0 (t) = 1. t

16 Three Sample Paths of the DTMC SIS Model are Compared to the Solution of the Deterministic Model. A sample path or stochastic realization of a stochastic process {I(t)} for t {0, t, 2 t,...} is an assignment of a possible value to I(t) for each value of t. R 0 = Number of Infectives Time t = 0.01, N = 100, β = 1, b = 0.25, γ = 0.25, I(0) = 2, and p 2 (0) = 1.The MATLAB program is in the Appendix.

17 The Probability Distribution p(t) for the Number of Infected Individuals in the DTMC SIS Model can be Approximated. Probability Time, n State Probability distribution for the DTMC SIS model, t = 0.01, N = 100, β = 1, b = 0.25, γ = 0.25, R 0 = 2, I(0) = 2 and p 2 (0) = 1. MATLAB program is in the Appendix. Note: Asymptotically, lim t p 0 (t) = 1, the epidemic ends with probability one. But it may take a long time before p 0 1, if N and I(0) are large. In this example, p 0 (t) 1 R 0 «I(0) = 1 2 «2 = 1 4.

18 Next, We Formulate a CTMC SIS Model. This type of model is most often used to study stochastic epidemic processes, time is continuous, but the random variable for number of infected individuals is discrete. The discrete random variable I(t), t [0, ) has an associated probability function p i (t) = Prob{I(t) = i} The probability of a transition for small t satisfies p ji ( t) = where o( t) 0 as t 0. βi(n i)/n t + o( t) = b(i) t + o( t), j = i + 1 (b + γ)i t + o( t) = d(i) t + o( t), j = i 1 1 [βi(n i)/n + (b + γ)i] t + o( t) = 1 [b(i) + d(i)] t + o( t), j = i o( t), otherwise, i i + 1, i i 1, or i i.

19 A System of Differential Equations for the Probabilities Can be Derived Based on the Transition Probabilities. For small t, p i (t + t) = p i 1 (t)[b(i 1) t] + p i+1 (t)[d(i + 1) t] p i (t)[1 (b(i) + d(i)) t] + o( t) Subtracting p i (t), dividing by t, and letting t 0, dp i dt dp 0 dt = p i 1 b(i 1) + p i+1 d(i + 1) p i [b(i) + d(i)] = p 1 d(1) for i = 1, 2,...,N, where b(i) = βi(n i)/n, d(i) = (b + γ)i. These differential equations are known as the forward Kolmogorov differential equations.

20 The Epidemic Process is Captured by a System of Differential Equations Expressed in Matrix Form. In matrix notation, dp dt = Qp, where p(t) = (p 0 (t),...,p N (t)) T and Q is known as the generator matrix: Q = d(1) [b(1) + d(1)] d(2) 0 0 b(1) [b(2) + d(2)] b(2) B d(n) A d(n) b(i) = βi(n i)/n and d(i) = (b + γ)i

21 The DTMC Transition Matrix and CTMC Differential Equations are Closely Related when t is Small. In the DTMC Model, p(t + t) = P( t)p(t), where P( t) is the transition matrix. Letting t 0, we obtain the Kolmogorov differential equations for the CTMC model, p(t + t) p(t) t dp dt = P( t) I t = Qp p(t) where Q = lim t 0 P( t) I t. The Discrete-Time Process can be used to Approximate the Continuous- Time Process.

22 Because of the Markov Property, the Inter-Event Time in a CTMC Model Has an Exponential Distribution. The exponential distribution has what is known as the memoryless property. Let I(t) = n and T n denote the inter-event time, a continuous random variable for the time to the next event. Take the sum of all the probabilities of all possible events where there is a change in state, i i + 1, i i 1: and j=0,j n p jn ( t) = a(n) t + o( t) p nn ( t) = 1 a(n) t + o( t). Then the interevent time has an exponential distribution with parameter a(n), T n E(a(n)) Prob{T n t} = 1 exp( a(n)t).

23 For the SIS Epidemic Model, with I(t) = n, p jn ( t) = [b(n) + d(n)] t + o( t) j=0,j n = [βn(n n)/n + (b + γ)n] t + o( t) a(n) = β n(n n) + (b + γ)n N

24 To Numerically Simulate the Inter-Event Time in a CTMC Model, We Use a Uniform Random Variable. The inter-event time, waiting time until an event occurs, can be numerically computed using a uniform random variable and the cumulative distribution for T n. Let U be uniform random variable on [0,1] and F n (t) the cumulative distribution for T n Then F n (t) = Prob{T n t} = 1 exp(a(n)t). Prob{F 1 n (U) t} = Prob{F n(f 1 (U)) F n(t)} = Prob{U F n (t)} = F n (t) The inter-event time T n, given I(t) = n satisfies n T n = Fn 1 ln(1 U) (U) = a(n) = ln(u) a(n).

25 Three Sample Paths of the CTMC SIS Model are Compared to the Deterministic Solution. R 0 = 2 Number of Infectives Time b = 0.25, γ = 0.25, β = 1, N = 100, I(0) = 2, R 0 = 2. For t small, the dynamics of the DTMC and the CTMC Models are Similar. The DTMC model can be used as an approximation for the CTMC model. MATLAB program is in the Appendix.

26 Next, We Formulate the Third Type of Stochastic Model, a SDE Model. The number of infectives, I(t), is continuous random variable and the time, t [0, ), is also continuous. The random variable I(t) has an associated probability density function (pdf), p(x,t), Prob{I(t) [a, b]} = b a p(x, t)dx. We can derive a system of differential equations satisfied by the pdf. This system of equations is also known as the forward Kolmogorov differential equations: p t = {[βx(n x)/n (b + γ)x]p} x {[βx(n x)/n + (b + γ)x]p} 2 x 2, x [0, N], t [0, ). The first term is known as the drift and second term diffusion.

27 The Stochastic Differential Equation (SDE) Depends on the Drift and Diffusion Terms. The Stochastic Differential Equation (SDE) follows directly from the differential equations, di dt = β r β I(N I) (b + γ)i + N N I(N I) + (b + γ)idw dt, where W(t) is a Wiener process (white noise), normally distributed, with mean zero and variance t: W(t) Normal(0, t), W(t + t) W(t) Normal(0, t). Sample paths for a Wiener process are continuous but not differentiable W(t) Time

28 The SDE Depends on Relationship Between Births and Deaths and Drift and Diffusion Let b(i) =births (new infection or birth) and d(i) =deaths (recovery or death). Then the probability density p(x, t), where Prob{I(t) [a, b]} = b p(x, t)dx satisfies the differential equation a p(x, t) t ([b(x) d(x)]p(x,t)) = x ([b(x) + d(x)]p(x,t)) 2 x 2 and the stochastic differential equation (SDE) satisfies di dt = b(i) d(i) + b(i) + d(i) dw dt = drift + diffusion

29 The Drift and Diffusion Terms Determine the Change in Number of Infections Over Time SDE: di dt = b(i) d(i) + b(i) + d(i) dw dt I(t) is approximately normally distributed with mean b(i) d(i)] t and variance, b(i) + d(i)] t : I(t) = I(t + t) I(t) Normal([b(I) d(i)] t,[b(i) + d(i)] t). The Wiener process W(t) (white noise) is normally distributed with mean 0 and variance t: W(t) = W(t + t) W(t) = t η Normal(0, t).

30 In General, the SDE is Expressed in Terms of the Parameters for Recovery, Transmission and Birth. SDE: di dt = β I(N I) (b + γ)i + N β N I(N I) + (b + γ)idw dt, where W(t) is a Wiener process (white noise), normally distributed, with mean zero and variance t: W(t) Normal(0, t), W(t + t) W(t) Normal(0, t).

31 Three Sample Paths for the SDE SIS Model are Computed Numerically and Compared to the Deterministic Solution. R 0 = 2 Number of Infectives Time b = 0.25, γ = 0.25, β = 1, N = 100, I(0) = 2. MATLAB program is in the Appendix. Note: For large N and I(0), then the SDE model is a good approximation to the CTMC model. However, for small N or I(0), the CTMC model is a better model.

32 Some Advantages of the Stochastic Models Over the Deterministic Model for the SIS Epidemic Model: The SIS Deterministic Model Does Not capture (i) The Variability Inherent in the Transmission, Recovery, Birth, and Death Processes (ii) The Probability of No Epidemic Occurrence when R 0 > 1. The Stochastic Models Do Capture these Features.

33 III. The SIS is a Simple Epidemic Model Because the Dynamics Reduce to a Single Variable. This is not the Case for the SIR Epidemic Model. First, we review the dynamics of the deterministic SIR Epidemic model. Then we will illustrate some of the differences in the stochastic formulation for the SIS versus the SIR epidemic model. Deterministic SIR: S I R

34 Deterministic SIR: S(t) + I(t) + R(t) = N ds dt di dt dr dt = β SI + b(i + R) N = β SI (b + γ)i N = γi br Basic Reproduction Number: R 0 = β b + γ If R 0 > 1 and b > 0, then lim t I(t) = Ī > 0. If R 0 > 1 and b = 0, then lim t I(t) = 0. There is an epidemic if R 0 S(0) N > 1. If R 0 1, then lim t I(t) = 0.

35 Formulation of a DTMC SIR Epidemic Model Results In A Multivariate Process. S(t) + I(t) + R(t) = N = maximum population size. Let S(t) and I(t) denote discrete random variables for the number of susceptible and infected individuals, respectively. These two variables have a joint probability function p (s,i) (t) = Prob{S(t) = s,i(t) = i} where R(t) = N S(t) I(t). For this stochastic process, we define transition probabilities as follows: p (s+k,i+j),(s,i) ( t) = Prob{( S, I) = (k, j) (S(t), I(t)) = (s, i)} 8 βi(n i) t/n, (k, j) = ( 1, 1) γi t, (k, j) = (0, 1) >< bi t, (k, j) = (1, 1) = b(n s i) t, (k, j) = (1, 0) 1 [βi(n i)/n + γi + b(n s)] t, (k, j) = (0, 0) >: 0, otherwise

36 Three Sample Paths of the DTMC SIR Epidemic Model are Compared to the Solution of the Deterministic Model. 35 R 0 = 2, b = 0 R 0 S(0) N = Number of Infectives, I(t) Time Steps t = 0.01, N = 100, β = 1, b = 0, γ = 0.5, S(0) = 98, and I(0) = 2.

37 The SDE Model for the SIR Epidemic is a System of Two Itô SDEs. For example, in the case with b = 0, ds dt di dt = β N SI + B dw 1 11 dt = β N SI γi + B dw 1 21 dt dw 2 + B 12 dt + B 22 dw 2 dt where W 1 and W 2 are two independent Wiener processes and B = (B ij ) is the square root of the following covariance matrix, B = Σ, Σ = ( ) βsi/n βsi/n. βsi/n βsi/n + γi Notice that matrix V is positive definite and thus, has a unique positive definite square root, Σ = B.

38 Three Stochastic Sample Paths of the SDE SIR Epidemic Model Are Compared to the Deterministic Solution. R 0 = 2, b = 0 R 0 S(0) N = Number of Infectives Time t = 0.01, N = 100, β = 1, b = 0, γ = 0.5, I(0) = 2.

39 To Summarize the Main Points: Stochastic epidemic models capture the variability inherent in the transmission, recovery, birth and death processes. Here we did not consider environmental variability. For small population sizes or small number of infected individuals, CTMC or DTMC models with discrete random variables more accurately capture the variability in the epidemic process than deterministic models. The DTMC model may be used to approximate the CTMC model when the time interval t is small. The SDE model may be used to approximate the CTMC model when the population size and initial values are large.

40 (Part II) Stochastic SIS and SIR Epidemic Models are Useful for Quantifying the Following: (a) Probability of No Epidemic (b) Stationary or Quasistationary Distribution (c) Final Size of an Epidemic (e) Expected Duration of an Epidemic

41 References and MATLAB programs: 1. Allen, L. J. S An Introduction to Stochastic Processes with Applications to Biology. Prentice Hall, Upper Saddle River, N.J. 2. Allen, L. J. S. and A. Burgin Comparison of deterministic and stochastic SIS and SIR models in discrete time. Mathematical Biosciences. 163: Andersson, H. and T. Britton Stochastic Epidemic Models and Their Statistical Analysis. Lecture Notes in Statistics. Springer-Verlag, New York, Inc. 4. Daley, D. J. and J. Gani Epidemic Modelling An Introduction. Cambridge Studies in Mathematical Biology, Vol. 15. Cambridge University Press, Cambridge. 5. Gard, T. C Introduction to Stochastic Differential Equations. Marcel Dekker, Inc., New York and Basel. 6. Mode, C. J. and C. K. Sleeman Stochastic Processes in Epidemiology. HIV/AIDS, Other Infectious Diseases and Computers. World Scientific, Singapore, New Jersey.

42 MATLAB Programs For: (1) Three Sample Paths for DTMC SIS Model (2) Probability Distribution for the DTMC SIS Model (3) Three Sample Paths for CTMC SIS Model (4) Three Sample Paths for the SDE SIS Model. (1) % Matlab Program % DTMC SIS Epidemic Model % Three Sample Paths clear set(0, DefaultAxesFontSize, 18) beta=1; g=0.25; b=0.25; N=100; init=2; dt=0.01; time=25; sim=3; for j=1:sim

43 i(1)=init; for t=1:time/dt r=rand; birth=beta*i(t)*(n-i(t))/n*dt; death=(b+g)*i(t)*dt; if r<=birth i(t+1)=i(t)+1; elseif r>birth & r<=birth+death i(t+1)=i(t)-1; else i(t+1)=i(t); end end if j==1 plot([0:dt:time],i, r-, LineWidth,2); hold on elseif j==2 plot([0:dt:time],i, g-, LineWidth,2); else plot([0:dt:time],i, b-, LineWidth,2 end end

44 % Euler s Method for Deterministic SIS Model y(1)=init; for k=1:time/dt y(k+1)=y(k)+dt*(beta*(n-y(k))*y(k)/n-(b+g)*y(k)); end plot([0:dt:time],y, k--, LineWidth,2); hold off xlabel( Time ); ylabel( Number of Infectives ); Number of Infectives Time

45 (2) % Matlab Program % Discrete Time Markov Chain % Stochastic SIS epidemic model % Transition matrix and Graph of Probability Distribution clear all set(gca, FontSize,18); set(0, DefaultAxesFontSize,18); time=2000; dtt=0.01; % Time step beta=1*dtt; b=0.25*dtt; gama=0.25*dtt; N=100; % Total population size en=50; % plot every enth time interval T=zeros(N+1,N+1); % T is the transition matrix, defined below v=linspace(0,n,n+1); p=zeros(time+1,n+1); p(1,3)=1; % Two individuals initially infected bt=beta*v.*(n-v)/n; dt=(b+gama)*v; for i=2:n % Define the transition matrix T(i,i)=1-bt(i)-dt(i); % diagonal entries T(i,i+1)=dt(i+1); % superdiagonal entries

46 T(i+1,i)=bt(i); % subdiagonal entries end T(1,1)=1; T(1,2)=dt(2); T(N+1,N+1)=1-dt(N+1); for t=1:time y=t*p(t,:) ; p(t+1,:)=y ; end pm(1,:)=p(1,:); for t=1:time/en; pm(t+1,:)=p(en*t,:); end ti=linspace(0,time,time/en+1); st=linspace(0,n,n+1); mesh(st,ti,pm); xlabel( number infected ); ylabel( time steps ); zlabel( probability of infection ); view(140,30); axis([0,n,0,time,0,1]);

47 Probability Time, n State

48 (3) % Matlab Program % Continuous Time Markov Chain % SIS Epidemic Model % Three Sample Paths Compared to the Deterministic Model clear set(0, DefaultAxesFontSize, 18); set(gca, fontsize,18); beta=1; b=0.25; gam=0.25; N=100; init=2; time=25; sim=3; for k=1:sim clear t s i t(1)=0; i(1)=init; s(1)=n-init; j=1; while i(j)>0 & t(j)<time u1=rand; u2=rand;

49 a=(beta/n)*i(j)*s(j)+(b+gam)*i(j); probi=(beta*s(j)/n)/(beta*s(j)/n+b+gam); t(j+1)=t(j)-log(u1)/a; if u2 <= probi i(j+1)=i(j)+1; s(j+1)=s(j)-1; else i(j+1)=i(j)-1; s(j+1)=s(j)+1; end j=j+1; end plot(t,i, r-, LineWidth,2) hold on end % Euler s Method Applied to the Deterministic SIS Epidemic Model dt=0.01; x(1)=n-init; y(1)=init; for k=1:time/dt x(k+1)=x(k)+dt*(-beta*x(k)*y(k)/n+(b+gam)*y(k)); y(k+1)=y(k)+dt*(beta*x(k)*y(k)/n-(b+gam)*y(k)); end plot([0:dt:time],y, k--, LineWidth,2);

50 axis([0,time,0,80]); xlabel( Time ); ylabel( Number of Infectives ); hold off Number of Infectives Time

51 (4) % Matlab Program % SDE SIS Epidemic Model % Three Sample Paths using Euler s Method clear beta=1; b=0.25; gam=0.25; N=100; init=2; dt=0.01; time=25; sim=3; for k=1:sim clear i, t j=1; i(j)=init; t(j)=dt; while i(j)>0 & t(j)<25 mu=beta*i(j)*(n-i(j))/n-(b+gam)*i(j); sigma=sqrt(beta*i(j)*(n-i(j))/n+(b+gam)*i(j)); rn=randn; i(j+1)=i(j)+mu*dt+sigma*sqrt(dt)*rn; t(j+1)=t(j)+dt;

52 j=j+1; end plot(t,i, r-, Linewidth,2); hold on end % Euler s method applied to the deterministic SIS epidemic model. y(1)=init; for k=1:time/dt y(k+1)=y(k)+dt*(beta*(n-y(k))*y(k)/n-(b+gam)*y(k)); end plot([0:dt:time],y, k--, LineWidth,2); axis([0,time,0,80]); xlabel( Time ); ylabel( Number of Infectives ); hold off

53 Number of Infectives Time

An Introduction to Stochastic Epidemic Models

An Introduction to Stochastic Epidemic Models An Introduction to Stochastic Epidemic Models Linda J. S. Allen Department of Mathematics and Statistics Texas Tech University Lubbock, Texas 79409-1042, U.S.A. linda.j.allen@ttu.edu 1 Introduction The

More information

Derivation of Itô SDE and Relationship to ODE and CTMC Models

Derivation of Itô SDE and Relationship to ODE and CTMC Models Derivation of Itô SDE and Relationship to ODE and CTMC Models Biomathematics II April 23, 2015 Linda J. S. Allen Texas Tech University TTU 1 Euler-Maruyama Method for Numerical Solution of an Itô SDE dx(t)

More information

Stochastic modelling of epidemic spread

Stochastic modelling of epidemic spread Stochastic modelling of epidemic spread Julien Arino Centre for Research on Inner City Health St Michael s Hospital Toronto On leave from Department of Mathematics University of Manitoba Julien Arino@umanitoba.ca

More information

Stochastic modelling of epidemic spread

Stochastic modelling of epidemic spread Stochastic modelling of epidemic spread Julien Arino Department of Mathematics University of Manitoba Winnipeg Julien Arino@umanitoba.ca 19 May 2012 1 Introduction 2 Stochastic processes 3 The SIS model

More information

AARMS Homework Exercises

AARMS Homework Exercises 1 For the gamma distribution, AARMS Homework Exercises (a) Show that the mgf is M(t) = (1 βt) α for t < 1/β (b) Use the mgf to find the mean and variance of the gamma distribution 2 A well-known inequality

More information

Probability Distributions

Probability Distributions Lecture : Background in Probability Theory Probability Distributions The probability mass function (pmf) or probability density functions (pdf), mean, µ, variance, σ 2, and moment generating function (mgf)

More information

Probability Distributions

Probability Distributions Lecture 1: Background in Probability Theory Probability Distributions The probability mass function (pmf) or probability density functions (pdf), mean, µ, variance, σ 2, and moment generating function

More information

Stochastic process. X, a series of random variables indexed by t

Stochastic process. X, a series of random variables indexed by t Stochastic process X, a series of random variables indexed by t X={X(t), t 0} is a continuous time stochastic process X={X(t), t=0,1, } is a discrete time stochastic process X(t) is the state at time t,

More information

6 Continuous-Time Birth and Death Chains

6 Continuous-Time Birth and Death Chains 6 Continuous-Time Birth and Death Chains Angela Peace Biomathematics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology.

More information

Irreducibility. Irreducible. every state can be reached from every other state For any i,j, exist an m 0, such that. Absorbing state: p jj =1

Irreducibility. Irreducible. every state can be reached from every other state For any i,j, exist an m 0, such that. Absorbing state: p jj =1 Irreducibility Irreducible every state can be reached from every other state For any i,j, exist an m 0, such that i,j are communicate, if the above condition is valid Irreducible: all states are communicate

More information

Markov-modulated interactions in SIR epidemics

Markov-modulated interactions in SIR epidemics Markov-modulated interactions in SIR epidemics E. Almaraz 1, A. Gómez-Corral 2 (1)Departamento de Estadística e Investigación Operativa, Facultad de Ciencias Matemáticas (UCM), (2)Instituto de Ciencias

More information

Lecture 4a: Continuous-Time Markov Chain Models

Lecture 4a: Continuous-Time Markov Chain Models Lecture 4a: Continuous-Time Markov Chain Models Continuous-time Markov chains are stochastic processes whose time is continuous, t [0, ), but the random variables are discrete. Prominent examples of continuous-time

More information

Discrete Time Markov Chain of a Dynamical System with a Rest Phase

Discrete Time Markov Chain of a Dynamical System with a Rest Phase Discrete Time Markov Chain of a Dynamical System with a Rest Phase Abstract A stochastic model, in the form of a discrete time Markov chain, is constructed to describe the dynamics of a population that

More information

Recap. Probability, stochastic processes, Markov chains. ELEC-C7210 Modeling and analysis of communication networks

Recap. Probability, stochastic processes, Markov chains. ELEC-C7210 Modeling and analysis of communication networks Recap Probability, stochastic processes, Markov chains ELEC-C7210 Modeling and analysis of communication networks 1 Recap: Probability theory important distributions Discrete distributions Geometric distribution

More information

Introduction to SEIR Models

Introduction to SEIR Models Department of Epidemiology and Public Health Health Systems Research and Dynamical Modelling Unit Introduction to SEIR Models Nakul Chitnis Workshop on Mathematical Models of Climate Variability, Environmental

More information

Problems 5: Continuous Markov process and the diffusion equation

Problems 5: Continuous Markov process and the diffusion equation Problems 5: Continuous Markov process and the diffusion equation Roman Belavkin Middlesex University Question Give a definition of Markov stochastic process. What is a continuous Markov process? Answer:

More information

Transmission in finite populations

Transmission in finite populations Transmission in finite populations Juliet Pulliam, PhD Department of Biology and Emerging Pathogens Institute University of Florida and RAPIDD Program, DIEPS Fogarty International Center US National Institutes

More information

Solutions to Homework 8 - Continuous-Time Markov Chains

Solutions to Homework 8 - Continuous-Time Markov Chains Solutions to Homework 8 - Continuous-Time Markov Chains 1) Insurance cash flow. A) CTMC states. Since we assume that c, d and X max are integers, while the premiums that the customers pay are worth 1,

More information

LECTURE #6 BIRTH-DEATH PROCESS

LECTURE #6 BIRTH-DEATH PROCESS LECTURE #6 BIRTH-DEATH PROCESS 204528 Queueing Theory and Applications in Networks Assoc. Prof., Ph.D. (รศ.ดร. อน นต ผลเพ ม) Computer Engineering Department, Kasetsart University Outline 2 Birth-Death

More information

Modeling with Itô Stochastic Differential Equations

Modeling with Itô Stochastic Differential Equations Modeling with Itô Stochastic Differential Equations 2.4-2.6 E. Allen presentation by T. Perälä 27.0.2009 Postgraduate seminar on applied mathematics 2009 Outline Hilbert Space of Stochastic Processes (

More information

Stochastic Differential Equations in Population Dynamics

Stochastic Differential Equations in Population Dynamics Stochastic Differential Equations in Population Dynamics Numerical Analysis, Stability and Theoretical Perspectives Bhaskar Ramasubramanian Abstract Population dynamics in the presence of noise in the

More information

The concentration of a drug in blood. Exponential decay. Different realizations. Exponential decay with noise. dc(t) dt.

The concentration of a drug in blood. Exponential decay. Different realizations. Exponential decay with noise. dc(t) dt. The concentration of a drug in blood Exponential decay C12 concentration 2 4 6 8 1 C12 concentration 2 4 6 8 1 dc(t) dt = µc(t) C(t) = C()e µt 2 4 6 8 1 12 time in minutes 2 4 6 8 1 12 time in minutes

More information

The Transition Probability Function P ij (t)

The Transition Probability Function P ij (t) The Transition Probability Function P ij (t) Consider a continuous time Markov chain {X(t), t 0}. We are interested in the probability that in t time units the process will be in state j, given that it

More information

CDA6530: Performance Models of Computers and Networks. Chapter 3: Review of Practical Stochastic Processes

CDA6530: Performance Models of Computers and Networks. Chapter 3: Review of Practical Stochastic Processes CDA6530: Performance Models of Computers and Networks Chapter 3: Review of Practical Stochastic Processes Definition Stochastic process X = {X(t), t2 T} is a collection of random variables (rvs); one rv

More information

SMSTC (2007/08) Probability.

SMSTC (2007/08) Probability. SMSTC (27/8) Probability www.smstc.ac.uk Contents 12 Markov chains in continuous time 12 1 12.1 Markov property and the Kolmogorov equations.................... 12 2 12.1.1 Finite state space.................................

More information

1.1 Review of Probability Theory

1.1 Review of Probability Theory 1.1 Review of Probability Theory Angela Peace Biomathemtics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,

More information

Introduction to Stochastic SIR Model

Introduction to Stochastic SIR Model Introduction to Stochastic R Model Chiu- Yu Yang (Alex), Yi Yang R model is used to model the infection of diseases. It is short for Susceptible- Infected- Recovered. It is important to address that R

More information

Part I Stochastic variables and Markov chains

Part I Stochastic variables and Markov chains Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)

More information

Construction of Equivalent Stochastic Differential Equation Models

Construction of Equivalent Stochastic Differential Equation Models Stochastic Analysis and Applications, 26: 274 297, 2008 Copyright Taylor & Francis Group, LLC ISSN 0736-2994 print/1532-9356 online DOI: 10.1080/07362990701857129 Construction of Equivalent Stochastic

More information

Outlines. Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC)

Outlines. Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC) Markov Chains (2) Outlines Discrete Time Markov Chain (DTMC) Continuous Time Markov Chain (CTMC) 2 pj ( n) denotes the pmf of the random variable p ( n) P( X j) j We will only be concerned with homogenous

More information

Data analysis and stochastic modeling

Data analysis and stochastic modeling Data analysis and stochastic modeling Lecture 7 An introduction to queueing theory Guillaume Gravier guillaume.gravier@irisa.fr with a lot of help from Paul Jensen s course http://www.me.utexas.edu/ jensen/ormm/instruction/powerpoint/or_models_09/14_queuing.ppt

More information

MARKOV PROCESSES. Valerio Di Valerio

MARKOV PROCESSES. Valerio Di Valerio MARKOV PROCESSES Valerio Di Valerio Stochastic Process Definition: a stochastic process is a collection of random variables {X(t)} indexed by time t T Each X(t) X is a random variable that satisfy some

More information

(b) What is the variance of the time until the second customer arrives, starting empty, assuming that we measure time in minutes?

(b) What is the variance of the time until the second customer arrives, starting empty, assuming that we measure time in minutes? IEOR 3106: Introduction to Operations Research: Stochastic Models Fall 2006, Professor Whitt SOLUTIONS to Final Exam Chapters 4-7 and 10 in Ross, Tuesday, December 19, 4:10pm-7:00pm Open Book: but only

More information

SIR Epidemic Model with total Population size

SIR Epidemic Model with total Population size Advances in Applied Mathematical Biosciences. ISSN 2248-9983 Volume 7, Number 1 (2016), pp. 33-39 International Research Publication House http://www.irphouse.com SIR Epidemic Model with total Population

More information

CDA5530: Performance Models of Computers and Networks. Chapter 3: Review of Practical

CDA5530: Performance Models of Computers and Networks. Chapter 3: Review of Practical CDA5530: Performance Models of Computers and Networks Chapter 3: Review of Practical Stochastic Processes Definition Stochastic ti process X = {X(t), t T} is a collection of random variables (rvs); one

More information

The SIS and SIR stochastic epidemic models revisited

The SIS and SIR stochastic epidemic models revisited The SIS and SIR stochastic epidemic models revisited Jesús Artalejo Faculty of Mathematics, University Complutense of Madrid Madrid, Spain jesus_artalejomat.ucm.es BCAM Talk, June 16, 2011 Talk Schedule

More information

Kolmogorov Equations and Markov Processes

Kolmogorov Equations and Markov Processes Kolmogorov Equations and Markov Processes May 3, 013 1 Transition measures and functions Consider a stochastic process {X(t)} t 0 whose state space is a product of intervals contained in R n. We define

More information

On the Spread of Epidemics in a Closed Heterogeneous Population

On the Spread of Epidemics in a Closed Heterogeneous Population On the Spread of Epidemics in a Closed Heterogeneous Population Artem Novozhilov Applied Mathematics 1 Moscow State University of Railway Engineering (MIIT) the 3d Workshop on Mathematical Models and Numerical

More information

An Overview of Methods for Applying Semi-Markov Processes in Biostatistics.

An Overview of Methods for Applying Semi-Markov Processes in Biostatistics. An Overview of Methods for Applying Semi-Markov Processes in Biostatistics. Charles J. Mode Department of Mathematics and Computer Science Drexel University Philadelphia, PA 19104 Overview of Topics. I.

More information

Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and

Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and private study only. The thesis may not be reproduced elsewhere

More information

Lecture 4: Introduction to stochastic processes and stochastic calculus

Lecture 4: Introduction to stochastic processes and stochastic calculus Lecture 4: Introduction to stochastic processes and stochastic calculus Cédric Archambeau Centre for Computational Statistics and Machine Learning Department of Computer Science University College London

More information

Three Disguises of 1 x = e λx

Three Disguises of 1 x = e λx Three Disguises of 1 x = e λx Chathuri Karunarathna Mudiyanselage Rabi K.C. Winfried Just Department of Mathematics, Ohio University Mathematical Biology and Dynamical Systems Seminar Ohio University November

More information

DISCRETE-TIME STOCHASTIC MODELS, SDEs, AND NUMERICAL METHODS. Ed Allen. NIMBioS Tutorial: Stochastic Models With Biological Applications

DISCRETE-TIME STOCHASTIC MODELS, SDEs, AND NUMERICAL METHODS. Ed Allen. NIMBioS Tutorial: Stochastic Models With Biological Applications DISCRETE-TIME STOCHASTIC MODELS, SDEs, AND NUMERICAL METHODS Ed Allen NIMBioS Tutorial: Stochastic Models With Biological Applications University of Tennessee, Knoxville March, 2011 ACKNOWLEDGEMENT I thank

More information

Brownian Motion. An Undergraduate Introduction to Financial Mathematics. J. Robert Buchanan. J. Robert Buchanan Brownian Motion

Brownian Motion. An Undergraduate Introduction to Financial Mathematics. J. Robert Buchanan. J. Robert Buchanan Brownian Motion Brownian Motion An Undergraduate Introduction to Financial Mathematics J. Robert Buchanan 2010 Background We have already seen that the limiting behavior of a discrete random walk yields a derivation of

More information

Simulating stochastic epidemics

Simulating stochastic epidemics Simulating stochastic epidemics John M. Drake & Pejman Rohani 1 Introduction This course will use the R language programming environment for computer modeling. The purpose of this exercise is to introduce

More information

Gaussian, Markov and stationary processes

Gaussian, Markov and stationary processes Gaussian, Markov and stationary processes Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ November

More information

Mathematical Modeling and Analysis of Infectious Disease Dynamics

Mathematical Modeling and Analysis of Infectious Disease Dynamics Mathematical Modeling and Analysis of Infectious Disease Dynamics V. A. Bokil Department of Mathematics Oregon State University Corvallis, OR MTH 323: Mathematical Modeling May 22, 2017 V. A. Bokil (OSU-Math)

More information

Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes

Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes Front. Math. China 215, 1(4): 933 947 DOI 1.17/s11464-15-488-5 Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes Yuanyuan LIU 1, Yuhui ZHANG 2

More information

Continuous time Markov chains (week 8) Solutions

Continuous time Markov chains (week 8) Solutions Continuous time Markov chains (week 8) Solutions 1 Insurance cash flow. A CTMC states. Because c and d are assumed to be integers, and the premiums are each 1, the cash flow X(t) of the insurance company

More information

Introduction to Queuing Networks Solutions to Problem Sheet 3

Introduction to Queuing Networks Solutions to Problem Sheet 3 Introduction to Queuing Networks Solutions to Problem Sheet 3 1. (a) The state space is the whole numbers {, 1, 2,...}. The transition rates are q i,i+1 λ for all i and q i, for all i 1 since, when a bus

More information

A Generic Multivariate Distribution for Counting Data

A Generic Multivariate Distribution for Counting Data arxiv:1103.4866v1 [stat.ap] 24 Mar 2011 A Generic Multivariate Distribution for Counting Data Marcos Capistrán and J. Andrés Christen Centro de Investigación en Matemáticas, A. C. (CIMAT) Guanajuato, MEXICO.

More information

Birth and Death Processes. Birth and Death Processes. Linear Growth with Immigration. Limiting Behaviour for Birth and Death Processes

Birth and Death Processes. Birth and Death Processes. Linear Growth with Immigration. Limiting Behaviour for Birth and Death Processes DTU Informatics 247 Stochastic Processes 6, October 27 Today: Limiting behaviour of birth and death processes Birth and death processes with absorbing states Finite state continuous time Markov chains

More information

Stochastic Modelling Unit 1: Markov chain models

Stochastic Modelling Unit 1: Markov chain models Stochastic Modelling Unit 1: Markov chain models Russell Gerrard and Douglas Wright Cass Business School, City University, London June 2004 Contents of Unit 1 1 Stochastic Processes 2 Markov Chains 3 Poisson

More information

Chapter 6: Random Processes 1

Chapter 6: Random Processes 1 Chapter 6: Random Processes 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.

More information

Lecture 7: Simulation of Markov Processes. Pasi Lassila Department of Communications and Networking

Lecture 7: Simulation of Markov Processes. Pasi Lassila Department of Communications and Networking Lecture 7: Simulation of Markov Processes Pasi Lassila Department of Communications and Networking Contents Markov processes theory recap Elementary queuing models for data networks Simulation of Markov

More information

Multistate Modelling Vertical Transmission and Determination of R 0 Using Transition Intensities

Multistate Modelling Vertical Transmission and Determination of R 0 Using Transition Intensities Applied Mathematical Sciences, Vol. 9, 2015, no. 79, 3941-3956 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ams.2015.52130 Multistate Modelling Vertical Transmission and Determination of R 0

More information

Approximation of epidemic models by diffusion processes and their statistical inferencedes

Approximation of epidemic models by diffusion processes and their statistical inferencedes Approximation of epidemic models by diffusion processes and their statistical inferencedes Catherine Larédo 1,2 1 UR 341, MaIAGE, INRA, Jouy-en-Josas 2 UMR 7599, LPMA, Université Paris Diderot December

More information

The SIR Disease Model Trajectories and MatLab

The SIR Disease Model Trajectories and MatLab The SIR Disease Model Trajectories and MatLab James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University November 17, 2013 Outline Reviewing the SIR

More information

ELEMENTS OF PROBABILITY THEORY

ELEMENTS OF PROBABILITY THEORY ELEMENTS OF PROBABILITY THEORY Elements of Probability Theory A collection of subsets of a set Ω is called a σ algebra if it contains Ω and is closed under the operations of taking complements and countable

More information

6.207/14.15: Networks Lectures 4, 5 & 6: Linear Dynamics, Markov Chains, Centralities

6.207/14.15: Networks Lectures 4, 5 & 6: Linear Dynamics, Markov Chains, Centralities 6.207/14.15: Networks Lectures 4, 5 & 6: Linear Dynamics, Markov Chains, Centralities 1 Outline Outline Dynamical systems. Linear and Non-linear. Convergence. Linear algebra and Lyapunov functions. Markov

More information

Stochastic Models. John M. Drake & Pejman Rohani

Stochastic Models. John M. Drake & Pejman Rohani Epidemiological data are noisy Two types of noise: Observation error: the data are probabilistically related to the true state of the system Process noise: the system progresses probabilistically Environmental

More information

The Effect of Stochastic Migration on an SIR Model for the Transmission of HIV. Jan P. Medlock

The Effect of Stochastic Migration on an SIR Model for the Transmission of HIV. Jan P. Medlock The Effect of Stochastic Migration on an SIR Model for the Transmission of HIV A Thesis Presented to The Faculty of the Division of Graduate Studies by Jan P. Medlock In Partial Fulfillment of the Requirements

More information

Comparison of Two Parameter Estimation Techniques for Stochastic Models

Comparison of Two Parameter Estimation Techniques for Stochastic Models East Tennessee State University Digital Commons @ East Tennessee State University Electronic Theses and Dissertations 8-2015 Comparison of Two Parameter Estimation Techniques for Stochastic Models Thomas

More information

Derivation of SPDEs for Correlated Random Walk Transport Models in One and Two Dimensions

Derivation of SPDEs for Correlated Random Walk Transport Models in One and Two Dimensions This article was downloaded by: [Texas Technology University] On: 23 April 2013, At: 07:52 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered

More information

ENSC327 Communications Systems 19: Random Processes. Jie Liang School of Engineering Science Simon Fraser University

ENSC327 Communications Systems 19: Random Processes. Jie Liang School of Engineering Science Simon Fraser University ENSC327 Communications Systems 19: Random Processes Jie Liang School of Engineering Science Simon Fraser University 1 Outline Random processes Stationary random processes Autocorrelation of random processes

More information

ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process

ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Definition of stochastic process (random

More information

Poisson Processes for Neuroscientists

Poisson Processes for Neuroscientists Poisson Processes for Neuroscientists Thibaud Taillefumier This note is an introduction to the key properties of Poisson processes, which are extensively used to simulate spike trains. For being mathematical

More information

Lecture 11: Introduction to Markov Chains. Copyright G. Caire (Sample Lectures) 321

Lecture 11: Introduction to Markov Chains. Copyright G. Caire (Sample Lectures) 321 Lecture 11: Introduction to Markov Chains Copyright G. Caire (Sample Lectures) 321 Discrete-time random processes A sequence of RVs indexed by a variable n 2 {0, 1, 2,...} forms a discretetime random process

More information

Elementary Applications of Probability Theory

Elementary Applications of Probability Theory Elementary Applications of Probability Theory With an introduction to stochastic differential equations Second edition Henry C. Tuckwell Senior Research Fellow Stochastic Analysis Group of the Centre for

More information

Lecture Notes 7 Random Processes. Markov Processes Markov Chains. Random Processes

Lecture Notes 7 Random Processes. Markov Processes Markov Chains. Random Processes Lecture Notes 7 Random Processes Definition IID Processes Bernoulli Process Binomial Counting Process Interarrival Time Process Markov Processes Markov Chains Classification of States Steady State Probabilities

More information

Age-dependent branching processes with incubation

Age-dependent branching processes with incubation Age-dependent branching processes with incubation I. RAHIMOV Department of Mathematical Sciences, KFUPM, Box. 1339, Dhahran, 3161, Saudi Arabia e-mail: rahimov @kfupm.edu.sa We study a modification of

More information

Journal of Computational and Applied Mathematics

Journal of Computational and Applied Mathematics Journal of Computational and Applied Mathematics 233 (2010) 2563 2574 Contents lists available at ScienceDirect Journal of Computational and Applied Mathematics journal homepage: wwwelseviercom/locate/cam

More information

The story of the film so far... Mathematics for Informatics 4a. Continuous-time Markov processes. Counting processes

The story of the film so far... Mathematics for Informatics 4a. Continuous-time Markov processes. Counting processes The story of the film so far... Mathematics for Informatics 4a José Figueroa-O Farrill Lecture 19 28 March 2012 We have been studying stochastic processes; i.e., systems whose time evolution has an element

More information

1 Introduction. 2 Diffusion equation and central limit theorem. The content of these notes is also covered by chapter 3 section B of [1].

1 Introduction. 2 Diffusion equation and central limit theorem. The content of these notes is also covered by chapter 3 section B of [1]. 1 Introduction The content of these notes is also covered by chapter 3 section B of [1]. Diffusion equation and central limit theorem Consider a sequence {ξ i } i=1 i.i.d. ξ i = d ξ with ξ : Ω { Dx, 0,

More information

Gillespie s Algorithm and its Approximations. Des Higham Department of Mathematics and Statistics University of Strathclyde

Gillespie s Algorithm and its Approximations. Des Higham Department of Mathematics and Statistics University of Strathclyde Gillespie s Algorithm and its Approximations Des Higham Department of Mathematics and Statistics University of Strathclyde djh@maths.strath.ac.uk The Three Lectures 1 Gillespie s algorithm and its relation

More information

Exponential Distribution and Poisson Process

Exponential Distribution and Poisson Process Exponential Distribution and Poisson Process Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 215 Outline Introduction Exponential

More information

STAT STOCHASTIC PROCESSES. Contents

STAT STOCHASTIC PROCESSES. Contents STAT 3911 - STOCHASTIC PROCESSES ANDREW TULLOCH Contents 1. Stochastic Processes 2 2. Classification of states 2 3. Limit theorems for Markov chains 4 4. First step analysis 5 5. Branching processes 5

More information

Stochastic differential equation models in biology Susanne Ditlevsen

Stochastic differential equation models in biology Susanne Ditlevsen Stochastic differential equation models in biology Susanne Ditlevsen Introduction This chapter is concerned with continuous time processes, which are often modeled as a system of ordinary differential

More information

2 Discrete-Time Markov Chains

2 Discrete-Time Markov Chains 2 Discrete-Time Markov Chains Angela Peace Biomathematics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,

More information

Epidemics in Networks Part 2 Compartmental Disease Models

Epidemics in Networks Part 2 Compartmental Disease Models Epidemics in Networks Part 2 Compartmental Disease Models Joel C. Miller & Tom Hladish 18 20 July 2018 1 / 35 Introduction to Compartmental Models Dynamics R 0 Epidemic Probability Epidemic size Review

More information

The decoupling assumption in large stochastic system analysis Talk at ECLT

The decoupling assumption in large stochastic system analysis Talk at ECLT The decoupling assumption in large stochastic system analysis Talk at ECLT Andrea Marin 1 1 Dipartimento di Scienze Ambientali, Informatica e Statistica Università Ca Foscari Venezia, Italy (University

More information

Thursday. Threshold and Sensitivity Analysis

Thursday. Threshold and Sensitivity Analysis Thursday Threshold and Sensitivity Analysis SIR Model without Demography ds dt di dt dr dt = βsi (2.1) = βsi γi (2.2) = γi (2.3) With initial conditions S(0) > 0, I(0) > 0, and R(0) = 0. This model can

More information

Numerical solution of stochastic epidemiological models

Numerical solution of stochastic epidemiological models Numerical solution of stochastic epidemiological models John M. Drake & Pejman Rohani 1 Introduction He we expand our modeling toolkit to include methods for studying stochastic versions of the compartmental

More information

Lecture 10: Semi-Markov Type Processes

Lecture 10: Semi-Markov Type Processes Lecture 1: Semi-Markov Type Processes 1. Semi-Markov processes (SMP) 1.1 Definition of SMP 1.2 Transition probabilities for SMP 1.3 Hitting times and semi-markov renewal equations 2. Processes with semi-markov

More information

Stochastic Numerical Analysis

Stochastic Numerical Analysis Stochastic Numerical Analysis Prof. Mike Giles mike.giles@maths.ox.ac.uk Oxford University Mathematical Institute Stoch. NA, Lecture 3 p. 1 Multi-dimensional SDEs So far we have considered scalar SDEs

More information

Introduction. Stochastic Processes. Will Penny. Stochastic Differential Equations. Stochastic Chain Rule. Expectations.

Introduction. Stochastic Processes. Will Penny. Stochastic Differential Equations. Stochastic Chain Rule. Expectations. 19th May 2011 Chain Introduction We will Show the relation between stochastic differential equations, Gaussian processes and methods This gives us a formal way of deriving equations for the activity of

More information

István Faragó, Róbert Horváth. Numerical Analysis of Evolution Equations, Innsbruck, Austria October, 2014.

István Faragó, Róbert Horváth. Numerical Analysis of Evolution Equations, Innsbruck, Austria October, 2014. s István, Róbert 1 Eötvös Loránd University, MTA-ELTE NumNet Research Group 2 University of Technology Budapest, MTA-ELTE NumNet Research Group Analysis of Evolution Equations, Innsbruck, Austria 14-17

More information

Quantitative Model Checking (QMC) - SS12

Quantitative Model Checking (QMC) - SS12 Quantitative Model Checking (QMC) - SS12 Lecture 06 David Spieler Saarland University, Germany June 4, 2012 1 / 34 Deciding Bisimulations 2 / 34 Partition Refinement Algorithm Notation: A partition P over

More information

Chapter 6 - Random Processes

Chapter 6 - Random Processes EE385 Class Notes //04 John Stensby Chapter 6 - Random Processes Recall that a random variable X is a mapping between the sample space S and the extended real line R +. That is, X : S R +. A random process

More information

The Wiener Itô Chaos Expansion

The Wiener Itô Chaos Expansion 1 The Wiener Itô Chaos Expansion The celebrated Wiener Itô chaos expansion is fundamental in stochastic analysis. In particular, it plays a crucial role in the Malliavin calculus as it is presented in

More information

Formulas for probability theory and linear models SF2941

Formulas for probability theory and linear models SF2941 Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms

More information

Lecture 1: Brief Review on Stochastic Processes

Lecture 1: Brief Review on Stochastic Processes Lecture 1: Brief Review on Stochastic Processes A stochastic process is a collection of random variables {X t (s) : t T, s S}, where T is some index set and S is the common sample space of the random variables.

More information

Lecture 6: Multiple Model Filtering, Particle Filtering and Other Approximations

Lecture 6: Multiple Model Filtering, Particle Filtering and Other Approximations Lecture 6: Multiple Model Filtering, Particle Filtering and Other Approximations Department of Biomedical Engineering and Computational Science Aalto University April 28, 2010 Contents 1 Multiple Model

More information

Handbook of Stochastic Methods

Handbook of Stochastic Methods Springer Series in Synergetics 13 Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences von Crispin W Gardiner Neuausgabe Handbook of Stochastic Methods Gardiner schnell und portofrei

More information

SDE Coefficients. March 4, 2008

SDE Coefficients. March 4, 2008 SDE Coefficients March 4, 2008 The following is a summary of GARD sections 3.3 and 6., mainly as an overview of the two main approaches to creating a SDE model. Stochastic Differential Equations (SDE)

More information

Continuous-Time Markov Chain

Continuous-Time Markov Chain Continuous-Time Markov Chain Consider the process {X(t),t 0} with state space {0, 1, 2,...}. The process {X(t),t 0} is a continuous-time Markov chain if for all s, t 0 and nonnegative integers i, j, x(u),

More information

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015 Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.

More information

Final Project Descriptions Introduction to Mathematical Biology Professor: Paul J. Atzberger. Project I: Predator-Prey Equations

Final Project Descriptions Introduction to Mathematical Biology Professor: Paul J. Atzberger. Project I: Predator-Prey Equations Final Project Descriptions Introduction to Mathematical Biology Professor: Paul J. Atzberger Project I: Predator-Prey Equations The Lotka-Volterra Predator-Prey Model is given by: du dv = αu βuv = ρβuv

More information

Stochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno

Stochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Stochastic Processes M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno 1 Outline Stochastic (random) processes. Autocorrelation. Crosscorrelation. Spectral density function.

More information

MCMC 2: Lecture 3 SIR models - more topics. Phil O Neill Theo Kypraios School of Mathematical Sciences University of Nottingham

MCMC 2: Lecture 3 SIR models - more topics. Phil O Neill Theo Kypraios School of Mathematical Sciences University of Nottingham MCMC 2: Lecture 3 SIR models - more topics Phil O Neill Theo Kypraios School of Mathematical Sciences University of Nottingham Contents 1. What can be estimated? 2. Reparameterisation 3. Marginalisation

More information