Stochastic modelling of epidemic spread

Stochastic modelling of epidemic spread Julien Arino Department of Mathematics University of Manitoba Winnipeg Julien Arino@umanitoba.ca 19 May 2012

1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

Deterministic SIR Use the standard KMK as in Fred s lectures: S 0 = SI I 0 = SI I R 0 = I Basic reproduction number R 0 = S 0 If R 0 < 1, no outbreak If R 0 > 1, outbreak Use =1/4, N = S + I + R = 100, I (0) = 1, R 0 = {0.5, 1.5} and so that this is true

R 0 =0.5 R 0 =1.5 100 100 S S 90 I 90 I R R 80 80 70 60 50 40 30 20 10 70 60 50 40 30 20 10 0 0 50 100 150 Time (days) 1 0 0 50 100 150 Time (days) 8 0.9 7 0.8 6 Number infectious 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Number infectious 5 4 3 2 1 0 0 50 100 150 Time (days) 0 0 50 100 150 Time (days)

Stochastic DTMC SIR Consider S(t) andi (t) (R(t) = N S(t) I (t)). Define transition probabilities p (s 0,i 0 ),(s,i)( t) = P{(S(t + t), I (t + t)) = (s + s 0, i + i 0 ) (S(t), I (t)) = (s, i)} by 8 si t (s 0, i 0 )=( 1, 1) >< i t (s 0, i 0 )=(0, 1) p (s 0,i 0 ),(s,i)( t) = 1 [ si + i] t (s 0, i 0 )=(0, 0) >: 0 otherwise Parameters as previously and t = 0.01

7 R 0 =0.5 R 0 =1.5 25 6 5 20 Number infectious 4 3 Number infectious 15 10 2 5 1 0 0 50 100 150 Time (days) 0 0 50 100 150 Time (days) 1.4 3.5 1.2 3 1 2.5 Number infectious 0.8 0.6 Number infectious 2 1.5 0.4 1 0.2 0.5 0 0 50 100 150 Time (days) 0 0 50 100 150 Time (days)

1000 Number of outbreaks 900 Number of simulations with an outbreak (/1000) 800 700 600 500 400 300 200 100 R 0 =0.5 R 0 =1.5 0 0 10 20 30 40 50 60 70 80 90 100 Value of I 0 outbreak: 9t s.t. I (t) > I (0)

Why stochastic models? Important to choose right type of model for given application Deterministic models Mathematically easier Perfect reproducibility For given parameter set, one sim su ces Stochastic models Often harder Each realization di erent Needs many simulations In early stage of epidemic, very few infective individuals: Deterministic systems: behaviour entirely governed by parameters Stochastic systems: allows chance to play a role

In the context of the SIR model Deterministic R 0 strict threshold R 0 < 1 ) I (t)! 0 monotonically R 0 > 1 ) I (t)! 0 after a bump (outbreak) Stochastic R 0 threshold for mean R 0 < 1 ) I (t)! 0 monotonically (roughly) on average but some realizations have outbreaks R 0 > 1 ) I (t)! 0 after a bump on average (outbreak) but some realizations have no outbreaks

1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

Theoretical setting of stochastic processes Stochastic processes form a well defined area of probability, a subset of measure theory, itself a well established area of mathematics Strong theoretical content Very briefly presented here for completeness, will not be used in what follows Used here: A. Klemke, Probability Theory, Springer 2008 DON T PANIC!!!

-algebras Definition 1 ( -algebra) Let 6= ; be a set and A 2, the set of all subsets of, bea class of subsets of. A is called a -algebra if: 1 2 A 2 A is closed under complements, i.e., A c := \ A 2 A for any A 2 A 3 A is closed under countable unions, i.e., [ 1 n=1 A n 2 A for any choice of countably many sets A 1, A 2,...2 A is the space of all elementary events and A the system of observable events

Probability space Definition 2 1 Apair(, A), with a nonempty set and A 2 a -algebra is a measurable space, withsetsa 2 A measurable sets. If is at most countably infinite and if A =2,thenthemeasurablespace(, 2 )isdiscrete 2 Atriple(, A,µ)isameasure space if (, A) isa measurable space and µ is a measure on A 3 If in addition, µ( ) = 1, then (, A,µ)isaprobability space and the sets A 2 A are events. µ is then usually denoted P (or Prob)

Topological spaces and Borel Definition 3 (Topology) -algebras Let 6= ; be an arbitrary set. A class of sets is a topology on if: 1 ;, 2 2 A \ B 2 for any A, B 2 3 ([ A2F A) 2 for any F 2 (, ) isatopological space, setsa 2 are open and A 2 with A c 2 are closed Definition 4 (Borel -algebra) Let (, ) be a topological space. The -algebra B(, ) generated by the open sets is the Borel -algebra on, with elements A 2 B(, ) theborel sets

Polish spaces Definition 5 A separable topological space whose topology is induced by a complete metric is a Polish space R d, Z d, R N,(C([0, 1]), k k 1 ) are Polish spaces

Stochastic process Definition 6 (Stochastic process) Let (, F, P) be a probability space, (E, ) be a Polish space with Borel -algebra E and T R be arbitrary (typically, T = N, T = Z, T = R + or T an interval). A family of random variables X =(X t, t 2 T ) on (, F, P) with values in (E, E) iscalledastochastic process with index set (or time set) T and range E.

OKAY, DONE WITH HARD THEORY!!!

Types of processes So a stochastic process is a collection of random variables (r.v.) {X (t,!), t 2 T,! 2 } The index set T and r.v. can be discrete or continuous Discrete time Markov chain (DTMC) hast and X discrete (e.g., T = N and X (t) 2 N) Continuous time Markov chain (CTMC) hast continuous and X discrete (e.g., T = R + and X (t) 2 N) In this course, we focus on these two. Also exist Stochastic di erential equations (SDE), with both T and X (t) continuous

Definition 7 (Markov property) Markov property Let E be a Polish space with Borel -algebra B(E), T R and (X t ) t2t an E-valued stochastic process. Assume that (F t ) t2t = (X ) is the filtration generated by X. X has the Markov property if for every A 2 B(E) andalls, t 2 T with s apple t, P[X t 2 A F s ]=P[X t 2 A X s ] Theorem 8 (The usable one) If E is countable then X has the Markov property if for all n 2 N, all s 1 < < s n < tandalli 1,...,i n, i 2 Ewith P[X s1 = i 1,...,X sn = i n ] > 0, there holds P[X t = i X s1 = i 1,...,X sn = i n ]=P[X t = i X sn = i n ]

1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

SIS model We consider an SIS model with demography. In ODE, S 0 = b ds SI + I I 0 = SI ( + d)i Total population N 0 = b dn asymptotically constant (N(t)! b/d =: N as t!1). Can work with asymptotically autonomous system where N = N (and S = N I )

S 0 = b SI ds + I I 0 = SI ( + d)i DFE: (S, I )=(b/d, 0). EEP: (S, I )= +d, b d +d VdD&W: F = D I [ S] DFE = N, V = D I [( + d)i ] DFE = + d ) R 0 = (FV 1 )= + d N If R 0 < 1, lim t!1 (S(t), I (t)) = (N, 0) If R 0 > 1, lim t!1 (S(t), I (t)) = N R 0, N N R 0

Reduction to one equation As N is asymptotically constant, we can write S = N only need I 0 = (N I )I ( + d)i I and so We can then reconstruct dynamics of S from that of I

1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

Discrete time Markov chain Definition 9 (DTMC Simple definition) An experiment with finite number of possible outcomes S 1,...,S N is repeated. The sequence of outcomes is a discrete time Markov chain if there is a set of N 2 numbers {p ij } such that the conditional probability of outcome S j on any experiment given outcome S i on the previous experiment is p ij,i.e.,for1apple i, j apple N, t =1,..., p ji =Pr(S j on experiment t +1 S i on experiment t). The outcomes S 1,...,S N are the states, andthep ij are the transition probabilities. ThematrixP =[p ij ]isthetransition matrix.

Homogeneity One major distinction: If p ij does not depend on t, thechainishomogeneous If p ij (t), i.e, transition probabilities depend on t, thechainis nonhomogeneous

Markov chains operate on two level: Description of probability that the system is in a given state Description of individual realizations of the process Because we assume the Markov property, we need only describe how system switches (transitions) from one state to the next We say that the system has no memory, thenextstatedepends only on the current one

States The states are here the di erent values of I. I can take integer values from 0 to N The chain links these states Infection Infection Infection I=0 I=1 I=2 I=3 I=N-1 I=N Recovery or Death Recovery or Death Recovery or Death Recovery or Death

Probability vector Let p i (t) =P(I (t) =i) is the probability distribution We must have P i p i(t) =1 0 1 p 0 (t) p 1 (t) p(t) = B C @. A p N (t) Initial condition at time t = 0: p(0)

Transition probabilities We then need description of probability of transition In general, suppose that S i is the current state, then one of S 1,...,S N must be the next state. If then we must have p ji = P{X (t + 1) = S j X (t) =S i } p i1 + p i2 + + p in =1, 1 apple i apple N Some of the p ij can be zero

Transition matrix The matrix 0 1 p 11 p 12 p 1N P = Bp 21 p 22 p 2N C @ A p N1 p N2 p NN has nonnegative entries, p ij 0 entries less than 1, p ij apple 1 column sum 1, which we write NX p ij =1, i=1 j =1,...,N or, using the notation 1l =(1,...,1) T, 1l T P = 1l T

In matrix form p(t + 1) = Pp(t), t =1, 2, 3,... where p(t) =(p 1 (t), p 2 (t),...,p N (t)) T is a (column) probability vector and P =(p ij )isan N transition matrix, 0 1 p 11 p 12 p 1N P = Bp 21 p 22 p 2N C @ A p N1 p N2 p NN

Stochastic matrices Definition 10 (Stochastic matrix) The nonnegative N N matrix M is stochastic if P N i=1 a ij =1for all j =1, 2,...,N. In other words, 1l T M = 1l T Theorem 11 Let M be a stochastic matrix M. Then the spectral radius (M) apple 1, i.e., all eigenvalues of M are such that apple 1. Furthermore, =1is an eigenvalue of M M has all column sums 1, 1l T M = 1l T e.value of M with associated left e.vector v, v T M = v T ) 1l T M = 1l T :lefte.vector1l T and e.value 1..

Long time behavior Let p(0) be the initial distribution (column) vector. Then p(1) = Pp(0) p(2) = Pp(1) = P(Pp(0)) = P 2 p(0) Iterating, we get that for any t 2 N, Therefore, lim p(t) = t!+1 p(t) =P t p(0) lim t!+1 Pt p(0) = lim t!+1 Pt p(0) if this limit exists (it does if P is constant and chain is regular)

Regular Markov chain Regular Markov chains not covered here, as most chains in the demographic context are absorbing. For information, though.. Definition 12 (Regular Markov chain) A regular Markov chain is one in which P k is positive for some integer k > 0, i.e., P k has only positive entries, no zero entries. Theorem 13 If P is the transition matrix of a regular Markov chain, then 1 the powers P t approach a stochastic matrix W, 2 each column of W is the same (column) vector w =(w 1,...,w N ) T, 3 the components of w are positive. So if the Markov chain is regular, lim p(t) =Wp(0) t!+1

Interesting properties of matrices Definition 14 (Irreducibility) The square nonnegative matrix M is reducible if there exists a permutation matrix P such that P T MP is block triangular. If M is not reducible, then it is irreducible Definition 15 (Primitivity) A nonnegative matrix M is primitive if and only if there is an integer k > 0 such that M k is positive

Definition 16 (Digraph) Directed graphs (digraphs) A directed graph (or digraph) G =(V, A) consists in a set V of vertices and a set A of arcs linking these vertices 5 1 2 3 4

Connecting graphs and matrices Adjacency matrix M has m ij = 1 if arc from vertex j to vertex i, 0 otherwise 1 2 3 4 5 0 1 0 1 0 0 0 1 0 0 0 0 ) M = B1 0 0 1 0 C @ 0 1 0 1 0A 0 0 0 0 0 Column of zeros (except maybe diagonal entry): vertex is a sink (nothing leaves)

Paths of length exactly 2 Given adjacency matrix M, M 2 has number of paths of length exactly two between vertices 1 2 3 4 5 0 1 1 0 0 0 0 ) M 2 0 1 0 0 0 = B0 2 0 1 0 C @ 0 0 0 1 0A 0 0 0 0 0 Write m k ij the (i, j) entryinm k m 2 32 =2since2! 1! 3and2! 4! 3..

Connecting matrices, graphs and regular chains Theorem 17 A matrix M is irreducible i the associated connection graph is strongly connected, i.e.,thereisapathbetweenanypair(i, j) of vertices Irreducibility: 8i, j, 9k, m k ij > 0 Theorem 18 A matrix M is primitive if it is irreducible and there is at least one positive entry on the diagonal of M Primitivity: 9k, 8i, j, m k ij > 0 Theorem 19 Markov chain regular, transition matrix P primitive

Strong connectedness (1) 1 7 2 4 5 3 6 8 Not strongly connected

Strong connectedness (2) 1 7 2 4 5 3 6 8 Strongly connected

Strong components 1 7 2 4 5 3 6 8 Strong component 1 Strong component 2

Absorbing states, absorbing chains Definition 20 AstateS i in a Markov chain is absorbing if whenever it occurs on the n th generation of the experiment, it then occurs on every subsequent step. In other words, S i is absorbing if p ii =1and p ij =0fori 6= j Definition 21 A Markov chain is said to be absorbing if it has at least one absorbing state, and if from every state it is possible to go to an absorbing state In an absorbing Markov chain, a state that is not absorbing is called transient

Here, we focus on absorbing strong components containing only one vertex Absorbing strong components 1 7 2 4 5 3 6 8 Absorbing strong component Transient states

Some questions on absorbing chains 1 Does the process eventually reach an absorbing state? 2 Average number of times spent in a transient state, if starting in a transient state? 3 Average number of steps before entering an absorbing state? 4 Probability of being absorbed by a given absorbing state, when there are more than one, when starting in a given transient state?

Reaching an absorbing state Answer to question 1: Theorem 22 In an absorbing Markov chain, the probability of reaching an absorbing state is 1

Standard form of the transition matrix For an absorbing chain with k absorbing states and N transient states, the transition matrix can be written as Ik 0 P = R Q k with following meaning, Absorbing states Transient states Absorbing states I k 0 Transient states R Q with I k the k k identity matrix, 0 an k (N k) matrix of zeros, R an (N k) k matrix and Q an (N k) (N k) matrix.

The matrix I N k Q is invertible. Let F =(I N k Q) 1 be the fundamental matrix of the Markov chain T i be the sum of the entries on row i of F B = FR Answers to our remaining questions: 2 F ij is the average number of times the process is in the jth transient state if it starts in the ith transient state 3 T i is the average number of steps before the process enters an absorbing state if it starts in the ith transient state 4 B ij is the probability of eventually entering the jth absorbing state if the process starts in the ith transient state

Back to the SIS model ODE: I 0 = (N I )I ( + d)i This equation has 3 components: 1 (N I )I new infection 2 I recovery 3 di death In the DTMC, want to know what transitions mean and what their probabilities are Infection Infection Infection I=0 I=1 I=2 I=3 I=N-1 I=N Recovery or Death Recovery or Death Recovery or Death Recovery or Death

Consider small amount of time t, su ciently small that only one event occurs: new infection, loss of infectious individual, nothing.. Let B(i) := (N i)i D(i) :=( + d)i be rates of birth and death of infectious. Then p ji ( t) =P {I (t + )=j I(t) =i} 8 B(i) t j = i +1 >< D(i) t j = i 1 = 1 [B(i)+D(i)] t j = i >: 0 otherwise β( N 1)Δ t β( N 2)2 Δ t β( N 1)Δ t Infection Infection Infection I=0 I=1 I=2 I=3 I=N-1 I=N Recovery Recovery Recovery Recovery or Death or Death or Death or Death (γ+d )Δ t (γ+d )2 Δt (γ+d )3 Δ t (γ+d ) N Δ t

Transition matrix 0 1 1 D(1) t 0 1 (B(1) + D(1)) t 0 B(1) t B.... C @ A D(N) t 0 0 0 B(N 1) t 1 D(N) t Note that indices in the matrix range from 0 to N here

Properties p 0 is an absorbing state For any initial distribution p(0) = (p 0 (t), p 1 (0),...,p N (0)), lim p(t) =(1, t!1 0,...,0)T lim p 0 (t) =1 t!1 Exists quasi-stationary distribution conditioned on non-extinction q i (t) = p i(t) 1 p 0 (t), i =1,...,N

Consequence of absorption We have lim p(t) =(1, t!1 0,...,0)T lim p 0 (t) =1 t!1 but time to absorption can be exponential

1 Introduction 2 Stochastic processes 3 The SIS model used as an example 4 DTMC 5 CTMC

Several ways to formulate CTMC s A continuous time Markov chain can be formulated in terms of infinitesimal transition probabilities branching process time to next event Here, time is in R +

For small t, p ji ( t) =P {I (t + )=j I(t) =i} 8 B(i) t + o( t) j = i +1 >< D(i) t + o( t) j = i 1 = 1 [B(i)+D(i)] t + o( t) j = i >: o( t) otherwise with o( t)! 0as t! 0

Forward Kolmogorov equations Assume we know I (0) = k. Then p i (t + t) =p i 1 (t)b(i 1) t + p i+1 (t)d(i + 1) t + p i (t)[1 (B(i)+D(i)) t]+o( t) Compute (p i (t + t) p i (t))/ t and take lim t!0, giving d dt p 0 = p 1 D(1) d dt p i = p i 1 B(i 1) + p i+1 D(i + 1) p i [B(i)+D(i)] i =1,...,N Forward Kolmogorov equations associated to the CTMC

Write previous system as In vector form p 0 = Qp with 0 1 0 D(1) 0 0 0 (B(1) + D(1)) D(2) 0 Q = 0 B(1) (B(2) + D(2)) 0 B C @ D(N) A D(N) Q generator matrix. Of course, p(t) =e Qt p(0)

Linking DTMC and CTMC for small t DTMC: p(t + t) =P( t)p(t) for transition matrix P( t). Let t! 0, obtain Kolmogorov equations for CTMC d dt p = Qp where P( t) Q = lim t!0 t I = P 0 (0)

Going ODE to CTMC Easy to convert compartmental ODE model to CTMC!!

The general setting: flow diagram b S βsi γ I I ds di

ODE approach b S βsi γ I I ds di Focus on compartments b S βsi γ I βsi γ I I ds di

CTMC approach b S βsi γ I I ds di b Focus on arrows (transitions) S ds βsi γ I I di

Focus on transitions b S βsi γ I I ds di b: S! S +1 birth ds: S! S 1 death of susceptible SI : S! S 1, I! I + 1 new infection I : S! S + 1, I! I 1 recovery di : I! I 1 death of infectious Will use S = N I and omit S dynamics

Gillespie s algorithm (SIS model) 1: Set t t 0 and I (t) I (t 0 ) 2: while t apple t f do 3: t (N i)i + i + di 4: Draw t from T s E( t ) 5: v [ (N i)i/ t, (N i)i/ t + i/ t, 1] 6: Draw t from U([0, 1]) 7: Find pos such that t v(pos) 8: switch (pos) 9: case 1: New infection, I (t + t )=I(t)+1 10: case 2: End of infectious period, I (t + t )=I(t) 1 11: case 3: Death, I (t + t )=I(t) 1 12: end switch 13: t t + t 14: end while

IMPORTANT: in Gillespie s algorithm, we do not consider p ii,the event nothing happens

Drawing at random from an exponential distribution Want t from T s E( t ), i.e., T has probability density function f (x, t )= t e tx 1 x 0 Use cumulative distribution function F (x, t )= R x1 f (s, t) ds F (x, t )=(1 e tx )1 x 0 which has values in [0, 1]. So draw from U([0, 1]) and solve F (x, t )= for x: F (x, t )=, 1 e tx =, e tx =1, t x = ln(1 ), x = ln(1 ) t