Introduction to stochastic multiscale modelling of tumour growth

Introduction to stochastic multiscale modelling of tumour growth Tomás Alarcón ICREA & Centre de Recerca Matemàtica T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 1 / 40

Outline Motivation: Why multiscale? Motivation: Why stochastic? Master Equation Law of mass action Steady vs absorbing states Analytical methods: Characteristic function T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 2 / 40

Competition between cellular populations in Biomedicine Competition between cells occur in a number of situations, including... Activation of the immune system during infection Somatic evolution in cancer Via mutations which make cells impervious to homeostatic control a By hijacking physiological competition mechanisms of normal tissues b a Hanahan & Weinberg. Cell. 144, 646 (2011) b L.A. Johnston. Science. 26, 1679 (2009) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 4 / 40

The magic bullet approach to cancer therapy The magic bullet This approach consists of finding a drug with a specific therapeutic target, e.g. a gene or protein implicated in a particular stage of the development of the disease, that would kill only diseased cells. T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 5 / 40

This idea has a long history Time line of the magic bullet T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 6 / 40

However... This approach has had modest success The most widely used cancer therapies continue to be unspecific agents such as cytotoxic drugs and radiotherapy or surgery The efficiency of targetted therapies has been modest with some notorious failures (e.g. anti-angiogenic therapy) Medical practitioners and experimental biologists are aware of this Combination therapies Personalised medicine T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 7 / 40

However... This approach has had modest success The most widely used cancer therapies continue to be unspecific agents such as cytotoxic drugs and radiotherapy or surgery The efficiency of targetted therapies has been modest with some notorious failures (e.g. anti-angiogenic therapy) Medical practitioners and experimental biologists are aware of this Combination therapies Personalised medicine Our research aim Understanding why the magic bullet approach has such difficulties In doing so, formulating better models which leads to improved therapeutic strategies T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 7 / 40

Multiscale phenomena: Coupling between different levels of tissue organisation Biological processes are mutiscale phenomena T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 8 / 40

Global biological behaviour as an emergent property The global behaviour of biological tissues is... 1 An emergent property of the non-linear coupling between processes characterised by different time and length scales (intracellular, cell-to-cell interactions, whole-tissue properties) 2 Difficult to predict when a perturbation is applied at a particular level due to the non-linear character of the system 3 We therefore need multiscale models able to integrate the different levels of biological organisation and predict its emergent behaviour T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 9 / 40

Multiscale modelling of vascular tumour growthin 3D 1 Schematic organisation of the model 1 Perfahl et al. PLoS One. 6, e14790 (2011) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 10 / 40

Hybrid multiscale modelling of tumour growth This model couples: T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 11 / 40

Hybrid multiscale modelling of tumour growth This model couples: Continuous models for blood flow, vascular adaptation, and remodelling of the vascular network (angiogenesis) Reaction-diffusion PDEs for the concentration of oxygen and signalling cues Cellular automata models of the competition between normal and cancer cells for space and resources ODE models intracellular pathways, e.g. coupling between extra-cellular oxygen and the cell-cycle machinery ODE models of signalling pathways (eg secretion VEGF) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 11 / 40

Hybrid multiscale modelling of tumour growth This model couples: Continuous models for blood flow, vascular adaptation, and remodelling of the vascular network (angiogenesis) Reaction-diffusion PDEs for the concentration of oxygen and signalling cues Cellular automata models of the competition between normal and cancer cells for space and resources ODE models intracellular pathways, e.g. coupling between extra-cellular oxygen and the cell-cycle machinery ODE models of signalling pathways (eg secretion VEGF) In this model, the birth and death rates emerge from the intracellular models rather than being fitted from population data T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 11 / 40

Hybrid multiscale modelling of tumour growth This model couples: Continuous models for blood flow, vascular adaptation, and remodelling of the vascular network (angiogenesis) Reaction-diffusion PDEs for the concentration of oxygen and signalling cues Cellular automata models of the competition between normal and cancer cells for space and resources ODE models intracellular pathways, e.g. coupling between extra-cellular oxygen and the cell-cycle machinery ODE models of signalling pathways (eg secretion VEGF) In this model, the birth and death rates emerge from the intracellular models rather than being fitted from population data The complexity of this model poses a barrier to its analysis but opens very interesting avenues for its application T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 11 / 40

Simulation results in 3D Hybrid multiscale modelling of tumour growth T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 12 / 40

However... This model is very complex and not amenable to analysis T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 13 / 40

However... This model is very complex and not amenable to analysis The population dynamics is assumed to be deterministic T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 13 / 40

However... This model is very complex and not amenable to analysis The population dynamics is assumed to be deterministic There are several situations of biological interest in which populations are small (eg dynamics of tumour micro-metastases) and therefore stochastic effects associated to finite-size effects are likely to be dominant T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 13 / 40

However... This model is very complex and not amenable to analysis The population dynamics is assumed to be deterministic There are several situations of biological interest in which populations are small (eg dynamics of tumour micro-metastases) and therefore stochastic effects associated to finite-size effects are likely to be dominant This model is computationally intensive and therefore can only be used to simulate small tumour masses or small regions of fully developed tumours T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 13 / 40

Outline of the course 1 Brief introduction to stochastic modelling 2 Asymptotic large system size approximation: path integral approach 3 Basic simulation techniques 4 Hybrid modelling of a stochastic multi-scale model of tumour growth (work in progress) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 14 / 40

Motivation There exists the general idea that randomness and noise simply add an unsystematic perturbation to a well-defined average behaviour I will present several examples of systems in which noise contributes to the behaviour of the system in a non-trivial manner and it is fundamental to understanding the system From these examples I will extract rules of thumb for ascertaining when randomness plays a fundamental roles T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 16 / 40

Large fluctuations in bistable systems Mean-field model of the G 1 /S transition model a a Tyson & Novak. JTB. 210, 249-263 (2001) 10 1 dy dt = k1 (k 2 + k 2 x)y dx dt = k 3(1 x) J 3 + (1 x) k4myx J 4 + x Cdh1 10 0 10 1 10 2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 CycB T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 17 / 40

Finite-size Tyson-Novak system Separatrix turns into a barrier 1 1 0.9 0.9 0.8 0.8 0.7 0.7 0.6 0.6 Cdh1 0.5 Cdh1 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 CycB 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 CycB T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 18 / 40

The Moran process The Moran process, named after the australian statistician Pat Moran, is a widely-used variant of the Wright-Fisher model and is commonly used in population genetics Moran process N individuals of two types. N is kept fixed. n: number of normal individuals. m: number of mutant individuals. N = n + m At each time step: 1 n n + 1 and m m 1 with probability rate W +(n) = n ( ) N 1 n N 2 n n 1 and m m + 1 with probability rate W (n) = ( 1 n ) n N N T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 19 / 40

n Simulation results The Moran process 50 45 40 35 30 25 20 15 10 5 0 0 10 20 30 40 50 60 70 80 90 100 Time Note that W +(n) = W (n), i.e. n = (n(t + t) n(t)) = 0 This implies that: d n dt = 0 (1) The system has two absorving states: W +(n = 0) = W (n = 0) = 0 and W +(n = N) = W (n = N) = 0 This means that lim P(n(t) = 0 n(t) = N) = 1 (2) t This behaviour is not at all captured by the deterministic equation (1) which predicts that the population will stay constant T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 20 / 40

Stochastic logistic growth The logistic equation, dm (1 dt = m m ), (3) K has two steady states: m = 0 unstable and m = K stable, i.e. regardless of the value of K and for any initial condition such that m(t = 0) > 0, m(t) will asymptotically approach K. Consider now a continuous-time Markov process n t whose dynamics are given by the following transition rate: 1 n n + 1 with probability rate W +(n) = n 2 n n 1 with probability rate W (n) = n(n 1) K This stochastic process has a unique absorbing state: n = 0, and therefore we expect the stochastic dynamics to show strong discrepancies with Eq. (8) when randomness is dominant T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 21 / 40

Stochastic logistic growth Simulation results n/k 2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0 10 20 30 40 50 60 70 80 90 100 Time Red line K = 10, green K = 50, blue K = 100, black K = 1000 For small K fluctuations dominate the behaviour of the system. Extinctions are common for small K, in contradiction to the behaviour predicted by the logistic equation Eq. (8), and become rarer as K is allowed to increase. T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 22 / 40

Summary We have seen two examples in which two different mathematical descriptions of the same physical/biological process give different answers as to what is their behaviour We have seen a model (the Moran model) where the deterministic description predicts the population to stay constant. To the contrary, the stochastic description predicts that the population never stays constant: Eventually, the system evolves to n = 0 or n = N Another model (logistic growth) presents the same dilemma: Its deterministic description preditcs that n = 0 is always unstable, whereas the stochastic formulation shows that very often the system evolves to n = 0 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 23 / 40

Summary We have seen two examples in which two different mathematical descriptions of the same physical/biological process give different answers as to what is their behaviour We have seen a model (the Moran model) where the deterministic description predicts the population to stay constant. To the contrary, the stochastic description predicts that the population never stays constant: Eventually, the system evolves to n = 0 or n = N Another model (logistic growth) presents the same dilemma: Its deterministic description preditcs that n = 0 is always unstable, whereas the stochastic formulation shows that very often the system evolves to n = 0 The question naturally arises: How is this possible? How come two mathematical descriptions of the same phenomenon offers so different answers? T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 23 / 40

General bibliography General references on stochastic processes 1 N. Van Kampen. Stochastic processes in physics and chemistry. Elsevier (2007) 2 C.W. Gardiner. Stochastic methods. Springer-Verlag (2009) Specific to mathematical biology 1 L.J.S. Allen. An introduction to stochastic processes with applications to biology. CRC Press (2003) 2 P. Bressloff. Stochastic Processes in Cell Biology. Springer-Verlag (2014) References to specialised literature relevant to specific issues will be given as we go T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 24 / 40

The Master Equation The Master Equation is our fundamental mathematical description of an stochastic process and the starting point for any attempt to analyse a particular model It is obtained as a probability balance for all the events that can occur during the time interval (t, t + t) Mathematically, it is a set of ordinary differential equations for the probability distribution P(X, t) i.e. the probability that the number of individuals in the population at time t to be X T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 26 / 40

The Master Equation where: dp(x, t) dt = R (W i (X r i, t)p(x r i, t) W i (x, t)p(x, t)) (4) i=1 X is a (vector-valued) Markov process The Master Equation can be interpreted as a balance equation where the first term corresponds to positive flux into state X from every allowed state and the second term as negative flux from state X into every possible state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 27 / 40

The Master Equation where: dp(x, t) dt = R (W i (X r i, t)p(x r i, t) W i (x, t)p(x, t)) (4) i=1 X is a (vector-valued) Markov process R is the number of possible processes (e.g. birth, death, mutation) The Master Equation can be interpreted as a balance equation where the first term corresponds to positive flux into state X from every allowed state and the second term as negative flux from state X into every possible state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 27 / 40

The Master Equation where: dp(x, t) dt = R (W i (X r i, t)p(x r i, t) W i (x, t)p(x, t)) (4) i=1 X is a (vector-valued) Markov process R is the number of possible processes (e.g. birth, death, mutation) W i (X, t) is the probability rate of event i to occur The Master Equation can be interpreted as a balance equation where the first term corresponds to positive flux into state X from every allowed state and the second term as negative flux from state X into every possible state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 27 / 40

The Master Equation where: dp(x, t) dt = R (W i (X r i, t)p(x r i, t) W i (x, t)p(x, t)) (4) i=1 X is a (vector-valued) Markov process R is the number of possible processes (e.g. birth, death, mutation) W i (X, t) is the probability rate of event i to occur r i is the change in the value of X when event i occurs, i.e. The Master Equation can be interpreted as a balance equation where the first term corresponds to positive flux into state X from every allowed state and the second term as negative flux from state X into every possible state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 27 / 40

The Master Equation where: dp(x, t) dt = R (W i (X r i, t)p(x r i, t) W i (x, t)p(x, t)) (4) i=1 X is a (vector-valued) Markov process R is the number of possible processes (e.g. birth, death, mutation) W i (X, t) is the probability rate of event i to occur r i is the change in the value of X when event i occurs, i.e. P(X (t + dt) = X (t) + r i X (t)) = W i (X, t)dt The Master Equation can be interpreted as a balance equation where the first term corresponds to positive flux into state X from every allowed state and the second term as negative flux from state X into every possible state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 27 / 40

Modelling the transition rates: The law of mass action To specify an stochastic model we need to give an expression for the probability rate for each of the events involved in the dynamics of the system (for example, birth and death) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 29 / 40

Modelling the transition rates: The law of mass action To specify an stochastic model we need to give an expression for the probability rate for each of the events involved in the dynamics of the system (for example, birth and death) The standard modelling assumption used to write down expressions for these rates is the so called Law of Mass Action This assumption, which originates in chamical kinetics, consists of assuming that the probability rate of a particular event involving j molecular species, of the same type or of different types, is proportional to (i) the number of ways in which the corresponding molecular species can combine and (ii) a rate constant which accounts for the probability that an encounter of the elements participating in the reaction actually produces the corresponding product T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 29 / 40

Examples X Product: W (x) = k 1X T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 30 / 40

Examples X Product: W (x) = k 1X X + Y Product: W (x) = k 2 XY 2 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 30 / 40

Examples X Product: W (x) = k 1X X + Y Product: W (x) = k 2 XY 2 X + X Product: W (x) = k 2 X (X 1) 2 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 30 / 40

Examples X Product: W (x) = k 1X X + Y Product: W (x) = k 2 XY 2 X + X Product: W (x) = k 2 X (X 1) 2 X + Y + Z Product: W (x) = k 3 XYZ 3! T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 30 / 40

Examples X Product: W (x) = k 1X X + Y Product: W (x) = k 2 XY 2 X + X Product: W (x) = k 2 X (X 1) 2 X + Y + Z Product: W (x) = k 3 XYZ 3! X + X + X Product: W (x) = k 3 X (X 2)(X 3) 3! T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 30 / 40

Example: Birth-and-death process 1 Birth: n n + 1 with probability rate W +(n) = λn. Death: n n 1 with probability rate W (n) = σn 2 Probability balance: P(n, t+ t) = λ(n 1) tp(n 1, t)+σ(n+1) tp(n+1, t)+(1 (λn t+σn t))p(n, t) (5) 3 When t 0: dp(n, t) dt = λ(n 1)P(n 1, t) + σ(n + 1)P(n + 1, t) (λn + σn)p(n, t) (6) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 31 / 40

Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 33 / 40

Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) We will avoid these technicalities, at least for the moment, and adopt a more intuitive and practical approach (although not terribly rigorous) For concreteness, consider (again) the stochastic logistic growth, i.e. a process n t such that: 1 n n + 1 with probability rate W +(n) = n 2 n n 1 with probability rate W (n) = n(n 1) K Consider a state of the system, n s, is, roughly speaking, a state of the process such that W +(n s) = W (n s). 1 W +(n s) is the number of births within a population of n s individuals 2 Likewise, W (n s) = the number of deaths within a population of n s individuals 3 So an steady state of our population dynamics is reached when n t = n s, since death rate is balanced by birth rates and therefore the population stays roughly constant T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 33 / 40

Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) We will avoid these technicalities, at least for the moment, and adopt a more intuitive and practical approach (although not terribly rigorous) For concreteness, consider (again) the stochastic logistic growth, i.e. a process n t such that: 1 n n + 1 with probability rate W +(n) = n 2 n n 1 with probability rate W (n) = n(n 1) K Consider a state of the system, n s, is, roughly speaking, a state of the process such that W +(n s) = W (n s). 1 W +(n s) is the number of births within a population of n s individuals 2 Likewise, W (n s) = the number of deaths within a population of n s individuals 3 So an steady state of our population dynamics is reached when n t = n s, since death rate is balanced by birth rates and therefore the population stays roughly constant n s = K + 1 which coincides with the deterministic stable fixed point (if K 1) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 33 / 40

Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) We will avoid these technicalities, at least for the moment, and adopt a more intuitive and practical approach (although not terribly rigorous) For concreteness, consider (again) the stochastic logistic growth, i.e. a process n t such that: 1 n n + 1 with probability rate W +(n) = n 2 n n 1 with probability rate W (n) = n(n 1) K Consider a state of the system, n s, is, roughly speaking, a state of the process such that W +(n s) = W (n s). 1 W +(n s) is the number of births within a population of n s individuals 2 Likewise, W (n s) = the number of deaths within a population of n s individuals 3 So an steady state of our population dynamics is reached when n t = n s, since death rate is balanced by birth rates and therefore the population stays roughly constant n s = K + 1 which coincides with the deterministic stable fixed point (if K 1) Note that W +(n) W (n) > 0 if n < n s and W +(n) W (n) < 0 if n > n s T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 33 / 40

Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 34 / 40

Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: 1 Steady states are in general not absorbing states 2 W +(n s) 0 and W +(n s) 0 3 If n = 0 then W +(0) = W (0) = 0 therefore n = 0 is an abosrbing state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 34 / 40

Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: 1 Steady states are in general not absorbing states 2 W +(n s) 0 and W +(n s) 0 3 If n = 0 then W +(0) = W (0) = 0 therefore n = 0 is an abosrbing state 4 Also, n s belongs to the set of accessible states of n = 0 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 34 / 40

Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: 1 Steady states are in general not absorbing states 2 W +(n s) 0 and W +(n s) 0 3 If n = 0 then W +(0) = W (0) = 0 therefore n = 0 is an abosrbing state 4 Also, n s belongs to the set of accessible states of n = 0 n s belonging to the set of accessible states of n = 0 means that there is at least one consecutive set of transitions that connects n s and n 0. For example: K K 1 K 2 1 0 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 34 / 40

Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: 1 Steady states are in general not absorbing states 2 W +(n s) 0 and W +(n s) 0 3 If n = 0 then W +(0) = W (0) = 0 therefore n = 0 is an abosrbing state 4 Also, n s belongs to the set of accessible states of n = 0 n s belonging to the set of accessible states of n = 0 means that there is at least one consecutive set of transitions that connects n s and n 0. For example: K K 1 K 2 1 0 However, if K 1 the probability of such a chain of events is vanishingly small T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 34 / 40

Summary I n s is an steady state in the sense that births and deaths are balanced. Moreover, W +(n) W (n) > 0 if n < n s and W +(n) W (n) < 0 if n > n s. This is essentially equivalent to what happens in the deterministic logistic growth model. However, n s is not an absorbing state of the stochastic dynamics. The only absorbing state is n = 0 Stochastic extinctions are relatively rare provided K is big. If this is the case, the deterministic system provides a reasonable approximation to the behaviour of the model. T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 35 / 40

Summary I n s is an steady state in the sense that births and deaths are balanced. Moreover, W +(n) W (n) > 0 if n < n s and W +(n) W (n) < 0 if n > n s. This is essentially equivalent to what happens in the deterministic logistic growth model. However, n s is not an absorbing state of the stochastic dynamics. The only absorbing state is n = 0 Stochastic extinctions are relatively rare provided K is big. If this is the case, the deterministic system provides a reasonable approximation to the behaviour of the model. If, on the contrary, K is small stochastic extinctions are relatively common and the deterministic description is not an accurate one T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 35 / 40

Summary II We have seen several examples of stochastic systems in which noise and randomness are the dominating factors. Their behaviours are not captured by their deterministic or mean-field conterparts In general, we should expect non-trivial random effects for: 1 Small populations: Large fluctuations/rare events have probability P e ΩS for large system size Ω 2 Dynamics with absorbing states T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 36 / 40

Analytical solutions Analytical solutions to the Master Equation are rare. General situations need to be dealt with by means of perturbative methods (Lecture 2) or numerical simulation (Lecture 3) For linear systems, it is possible to write a PDE for the so called probability generating function: G(s, t) = P(n, t)s n (7) n Example: Birth-and-death process 1 Master Equation: dp(n, t) dt = λ(n 1)P(n 1, t) + σ(n + 1)P(n + 1, t) (λn + σn)p(n, t) (8) 2 Multiply by s n and sum over all n n s n dp(n, t) dt = s 2 n s n λ(n 1)P(n 1, t)s n 2 + n σ(n + 1)P(n + 1, t)s n (λn + σn)p(n, t)s n 1 (9) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 38 / 40

Analytical solutions Example: Birth-and-death process (cont.) 1 PDE for the characteristic function: G(s, t) t G(s, t) = (λs σ)(s 1) s 2 This PDE can be solved by the method of characteristics. If G(s, t = 0) = s I (10) G(s, t) = ( ) I σ(s 1) (σ λs) exp( t(λ µ)) (11) λ(s 1) (σ λs) exp( t(λ µ)) 3 By Cauchy s formula: P(n, t) = n! G(z, t) dz (12) 2πi z n+1 Note Even when an analytical, closed solution for the characteristic function is not available, this function and its associated PDE are the corner stone for asymptotic analysis, specially WKB asymptotics T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 39 / 40

Outline of next lecture 1 Formal definition of a Markov process The Markov property Chapman-Kolmogorov equation Derivation of the Master Equation 2 Asymptotic methods and rare events Path integral solution Large deviations and optimal theory 3 Monte Carlo simulations methods: Gillespie s stochastic simulation algorithm T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016 40 / 40