Introduction to stochastic multiscale modelling of tumour growth

Transcription

1 Introduction to stochastic multiscale modelling of tumour growth Tomás Alarcón ICREA & Centre de Recerca Matemàtica T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

2 Outline Motivation: Why multiscale? Motivation: Why stochastic? Master Equation Law of mass action Steady vs absorbing states Analytical methods: Characteristic function T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

3 Outline Motivation: Why multiscale? Motivation: Why stochastic? Master Equation Law of mass action Steady vs absorbing states Analytical methods: Characteristic function T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

4 Competition between cellular populations in Biomedicine Competition between cells occur in a number of situations, including... Activation of the immune system during infection Somatic evolution in cancer Via mutations which make cells impervious to homeostatic control a By hijacking physiological competition mechanisms of normal tissues b a Hanahan & Weinberg. Cell. 144, 646 (2011) b L.A. Johnston. Science. 26, 1679 (2009) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

5 The magic bullet approach to cancer therapy The magic bullet This approach consists of finding a drug with a specific therapeutic target, e.g. a gene or protein implicated in a particular stage of the development of the disease, that would kill only diseased cells. T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

6 This idea has a long history Time line of the magic bullet T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

7 However... This approach has had modest success The most widely used cancer therapies continue to be unspecific agents such as cytotoxic drugs and radiotherapy or surgery The efficiency of targetted therapies has been modest with some notorious failures (e.g. anti-angiogenic therapy) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

8 However... This approach has had modest success The most widely used cancer therapies continue to be unspecific agents such as cytotoxic drugs and radiotherapy or surgery The efficiency of targetted therapies has been modest with some notorious failures (e.g. anti-angiogenic therapy) Medical practitioners and experimental biologists are aware of this Combination therapies Personalised medicine T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

9 However... This approach has had modest success The most widely used cancer therapies continue to be unspecific agents such as cytotoxic drugs and radiotherapy or surgery The efficiency of targetted therapies has been modest with some notorious failures (e.g. anti-angiogenic therapy) Medical practitioners and experimental biologists are aware of this Combination therapies Personalised medicine Our research aim Understanding why the magic bullet approach has such difficulties In doing so, formulating better models which leads to improved therapeutic strategies T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

10 Multiscale phenomena: Coupling between different levels of tissue organisation Biological processes are mutiscale phenomena T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

11 Global biological behaviour as an emergent property The global behaviour of biological tissues is... 1 An emergent property of the non-linear coupling between processes characterised by different time and length scales (intracellular, cell-to-cell interactions, whole-tissue properties) 2 Difficult to predict when a perturbation is applied at a particular level due to the non-linear character of the system 3 We therefore need multiscale models able to integrate the different levels of biological organisation and predict its emergent behaviour T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

12 Multiscale modelling of vascular tumour growthin 3D 1 Schematic organisation of the model 1 Perfahl et al. PLoS One. 6, e14790 (2011) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

13 Hybrid multiscale modelling of tumour growth This model couples: T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

14 Hybrid multiscale modelling of tumour growth This model couples: Continuous models for blood flow, vascular adaptation, and remodelling of the vascular network (angiogenesis) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

15 Hybrid multiscale modelling of tumour growth This model couples: Continuous models for blood flow, vascular adaptation, and remodelling of the vascular network (angiogenesis) Reaction-diffusion PDEs for the concentration of oxygen and signalling cues T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

16 Hybrid multiscale modelling of tumour growth This model couples: Continuous models for blood flow, vascular adaptation, and remodelling of the vascular network (angiogenesis) Reaction-diffusion PDEs for the concentration of oxygen and signalling cues Cellular automata models of the competition between normal and cancer cells for space and resources T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

17 Hybrid multiscale modelling of tumour growth This model couples: Continuous models for blood flow, vascular adaptation, and remodelling of the vascular network (angiogenesis) Reaction-diffusion PDEs for the concentration of oxygen and signalling cues Cellular automata models of the competition between normal and cancer cells for space and resources ODE models intracellular pathways, e.g. coupling between extra-cellular oxygen and the cell-cycle machinery T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

18 Hybrid multiscale modelling of tumour growth This model couples: Continuous models for blood flow, vascular adaptation, and remodelling of the vascular network (angiogenesis) Reaction-diffusion PDEs for the concentration of oxygen and signalling cues Cellular automata models of the competition between normal and cancer cells for space and resources ODE models intracellular pathways, e.g. coupling between extra-cellular oxygen and the cell-cycle machinery ODE models of signalling pathways (eg secretion VEGF) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

19 Hybrid multiscale modelling of tumour growth This model couples: Continuous models for blood flow, vascular adaptation, and remodelling of the vascular network (angiogenesis) Reaction-diffusion PDEs for the concentration of oxygen and signalling cues Cellular automata models of the competition between normal and cancer cells for space and resources ODE models intracellular pathways, e.g. coupling between extra-cellular oxygen and the cell-cycle machinery ODE models of signalling pathways (eg secretion VEGF) In this model, the birth and death rates emerge from the intracellular models rather than being fitted from population data T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

20 Hybrid multiscale modelling of tumour growth This model couples: Continuous models for blood flow, vascular adaptation, and remodelling of the vascular network (angiogenesis) Reaction-diffusion PDEs for the concentration of oxygen and signalling cues Cellular automata models of the competition between normal and cancer cells for space and resources ODE models intracellular pathways, e.g. coupling between extra-cellular oxygen and the cell-cycle machinery ODE models of signalling pathways (eg secretion VEGF) In this model, the birth and death rates emerge from the intracellular models rather than being fitted from population data The complexity of this model poses a barrier to its analysis but opens very interesting avenues for its application T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

21 Simulation results in 3D Hybrid multiscale modelling of tumour growth T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

22 However... This model is very complex and not amenable to analysis T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

23 However... This model is very complex and not amenable to analysis The population dynamics is assumed to be deterministic T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

24 However... This model is very complex and not amenable to analysis The population dynamics is assumed to be deterministic There are several situations of biological interest in which populations are small (eg dynamics of tumour micro-metastases) and therefore stochastic effects associated to finite-size effects are likely to be dominant T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

25 However... This model is very complex and not amenable to analysis The population dynamics is assumed to be deterministic There are several situations of biological interest in which populations are small (eg dynamics of tumour micro-metastases) and therefore stochastic effects associated to finite-size effects are likely to be dominant This model is computationally intensive and therefore can only be used to simulate small tumour masses or small regions of fully developed tumours T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

26 Outline of the course 1 Brief introduction to stochastic modelling 2 Asymptotic large system size approximation: path integral approach 3 Basic simulation techniques 4 Hybrid modelling of a stochastic multi-scale model of tumour growth (work in progress) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

27 Outline Motivation: Why multiscale? Motivation: Why stochastic? Master Equation Law of mass action Steady vs absorbing states Analytical methods: Characteristic function T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

28 Motivation There exists the general idea that randomness and noise simply add an unsystematic perturbation to a well-defined average behaviour I will present several examples of systems in which noise contributes to the behaviour of the system in a non-trivial manner and it is fundamental to understanding the system From these examples I will extract rules of thumb for ascertaining when randomness plays a fundamental roles T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

29 Large fluctuations in bistable systems Mean-field model of the G 1 /S transition model a a Tyson & Novak. JTB. 210, (2001) 10 1 dy dt = k1 (k 2 + k 2 x)y dx dt = k 3(1 x) J 3 + (1 x) k4myx J 4 + x Cdh CycB T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

30 Finite-size Tyson-Novak system Separatrix turns into a barrier Cdh1 0.5 Cdh CycB CycB T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

31 The Moran process The Moran process, named after the australian statistician Pat Moran, is a widely-used variant of the Wright-Fisher model and is commonly used in population genetics Moran process N individuals of two types. N is kept fixed. n: number of normal individuals. m: number of mutant individuals. N = n + m At each time step: 1 n n + 1 and m m 1 with probability rate W +(n) = n ( ) N 1 n N 2 n n 1 and m m + 1 with probability rate W (n) = ( 1 n ) n N N T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

32 n Simulation results The Moran process Time Note that W +(n) = W (n), i.e. n = (n(t + t) n(t)) = 0 This implies that: d n dt = 0 (1) The system has two absorving states: W +(n = 0) = W (n = 0) = 0 and W +(n = N) = W (n = N) = 0 This means that lim P(n(t) = 0 n(t) = N) = 1 (2) t This behaviour is not at all captured by the deterministic equation (1) which predicts that the population will stay constant T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

33 Stochastic logistic growth The logistic equation, dm (1 dt = m m ), (3) K has two steady states: m = 0 unstable and m = K stable, i.e. regardless of the value of K and for any initial condition such that m(t = 0) > 0, m(t) will asymptotically approach K. Consider now a continuous-time Markov process n t whose dynamics are given by the following transition rate: 1 n n + 1 with probability rate W +(n) = n 2 n n 1 with probability rate W (n) = n(n 1) K This stochastic process has a unique absorbing state: n = 0, and therefore we expect the stochastic dynamics to show strong discrepancies with Eq. (8) when randomness is dominant T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

34 Stochastic logistic growth Simulation results n/k Time Red line K = 10, green K = 50, blue K = 100, black K = 1000 For small K fluctuations dominate the behaviour of the system. Extinctions are common for small K, in contradiction to the behaviour predicted by the logistic equation Eq. (8), and become rarer as K is allowed to increase. T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

35 Summary We have seen two examples in which two different mathematical descriptions of the same physical/biological process give different answers as to what is their behaviour T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

36 Summary We have seen two examples in which two different mathematical descriptions of the same physical/biological process give different answers as to what is their behaviour We have seen a model (the Moran model) where the deterministic description predicts the population to stay constant. To the contrary, the stochastic description predicts that the population never stays constant: Eventually, the system evolves to n = 0 or n = N T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

37 Summary We have seen two examples in which two different mathematical descriptions of the same physical/biological process give different answers as to what is their behaviour We have seen a model (the Moran model) where the deterministic description predicts the population to stay constant. To the contrary, the stochastic description predicts that the population never stays constant: Eventually, the system evolves to n = 0 or n = N Another model (logistic growth) presents the same dilemma: Its deterministic description preditcs that n = 0 is always unstable, whereas the stochastic formulation shows that very often the system evolves to n = 0 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

38 Summary We have seen two examples in which two different mathematical descriptions of the same physical/biological process give different answers as to what is their behaviour We have seen a model (the Moran model) where the deterministic description predicts the population to stay constant. To the contrary, the stochastic description predicts that the population never stays constant: Eventually, the system evolves to n = 0 or n = N Another model (logistic growth) presents the same dilemma: Its deterministic description preditcs that n = 0 is always unstable, whereas the stochastic formulation shows that very often the system evolves to n = 0 The question naturally arises: How is this possible? How come two mathematical descriptions of the same phenomenon offers so different answers? T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

39 General bibliography General references on stochastic processes 1 N. Van Kampen. Stochastic processes in physics and chemistry. Elsevier (2007) 2 C.W. Gardiner. Stochastic methods. Springer-Verlag (2009) Specific to mathematical biology 1 L.J.S. Allen. An introduction to stochastic processes with applications to biology. CRC Press (2003) 2 P. Bressloff. Stochastic Processes in Cell Biology. Springer-Verlag (2014) References to specialised literature relevant to specific issues will be given as we go T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

40 Outline Motivation: Why multiscale? Motivation: Why stochastic? Master Equation Law of mass action Steady vs absorbing states Analytical methods: Characteristic function T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

41 The Master Equation The Master Equation is our fundamental mathematical description of an stochastic process and the starting point for any attempt to analyse a particular model T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

42 The Master Equation The Master Equation is our fundamental mathematical description of an stochastic process and the starting point for any attempt to analyse a particular model It is obtained as a probability balance for all the events that can occur during the time interval (t, t + t) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

43 The Master Equation The Master Equation is our fundamental mathematical description of an stochastic process and the starting point for any attempt to analyse a particular model It is obtained as a probability balance for all the events that can occur during the time interval (t, t + t) Mathematically, it is a set of ordinary differential equations for the probability distribution P(X, t) i.e. the probability that the number of individuals in the population at time t to be X T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

44 The Master Equation where: dp(x, t) dt = R (W i (X r i, t)p(x r i, t) W i (x, t)p(x, t)) (4) i=1 X is a (vector-valued) Markov process The Master Equation can be interpreted as a balance equation where the first term corresponds to positive flux into state X from every allowed state and the second term as negative flux from state X into every possible state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

45 The Master Equation where: dp(x, t) dt = R (W i (X r i, t)p(x r i, t) W i (x, t)p(x, t)) (4) i=1 X is a (vector-valued) Markov process R is the number of possible processes (e.g. birth, death, mutation) The Master Equation can be interpreted as a balance equation where the first term corresponds to positive flux into state X from every allowed state and the second term as negative flux from state X into every possible state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

46 The Master Equation where: dp(x, t) dt = R (W i (X r i, t)p(x r i, t) W i (x, t)p(x, t)) (4) i=1 X is a (vector-valued) Markov process R is the number of possible processes (e.g. birth, death, mutation) W i (X, t) is the probability rate of event i to occur The Master Equation can be interpreted as a balance equation where the first term corresponds to positive flux into state X from every allowed state and the second term as negative flux from state X into every possible state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

47 The Master Equation where: dp(x, t) dt = R (W i (X r i, t)p(x r i, t) W i (x, t)p(x, t)) (4) i=1 X is a (vector-valued) Markov process R is the number of possible processes (e.g. birth, death, mutation) W i (X, t) is the probability rate of event i to occur r i is the change in the value of X when event i occurs, i.e. The Master Equation can be interpreted as a balance equation where the first term corresponds to positive flux into state X from every allowed state and the second term as negative flux from state X into every possible state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

48 The Master Equation where: dp(x, t) dt = R (W i (X r i, t)p(x r i, t) W i (x, t)p(x, t)) (4) i=1 X is a (vector-valued) Markov process R is the number of possible processes (e.g. birth, death, mutation) W i (X, t) is the probability rate of event i to occur r i is the change in the value of X when event i occurs, i.e. P(X (t + dt) = X (t) + r i X (t)) = W i (X, t)dt The Master Equation can be interpreted as a balance equation where the first term corresponds to positive flux into state X from every allowed state and the second term as negative flux from state X into every possible state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

49 Outline Motivation: Why multiscale? Motivation: Why stochastic? Master Equation Law of mass action Steady vs absorbing states Analytical methods: Characteristic function T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

50 Modelling the transition rates: The law of mass action To specify an stochastic model we need to give an expression for the probability rate for each of the events involved in the dynamics of the system (for example, birth and death) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

51 Modelling the transition rates: The law of mass action To specify an stochastic model we need to give an expression for the probability rate for each of the events involved in the dynamics of the system (for example, birth and death) The standard modelling assumption used to write down expressions for these rates is the so called Law of Mass Action T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

52 Modelling the transition rates: The law of mass action To specify an stochastic model we need to give an expression for the probability rate for each of the events involved in the dynamics of the system (for example, birth and death) The standard modelling assumption used to write down expressions for these rates is the so called Law of Mass Action This assumption, which originates in chamical kinetics, consists of assuming that the probability rate of a particular event involving j molecular species, of the same type or of different types, is proportional to (i) the number of ways in which the corresponding molecular species can combine and (ii) a rate constant which accounts for the probability that an encounter of the elements participating in the reaction actually produces the corresponding product T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

53 Examples X Product: W (x) = k 1X T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

54 Examples X Product: W (x) = k 1X X + Y Product: W (x) = k 2 XY 2 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

55 Examples X Product: W (x) = k 1X X + Y Product: W (x) = k 2 XY 2 X + X Product: W (x) = k 2 X (X 1) 2 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

56 Examples X Product: W (x) = k 1X X + Y Product: W (x) = k 2 XY 2 X + X Product: W (x) = k 2 X (X 1) 2 X + Y + Z Product: W (x) = k 3 XYZ 3! T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

57 Examples X Product: W (x) = k 1X X + Y Product: W (x) = k 2 XY 2 X + X Product: W (x) = k 2 X (X 1) 2 X + Y + Z Product: W (x) = k 3 XYZ 3! X + X + X Product: W (x) = k 3 X (X 2)(X 3) 3! T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

58 Example: Birth-and-death process 1 Birth: n n + 1 with probability rate W +(n) = λn. Death: n n 1 with probability rate W (n) = σn 2 Probability balance: P(n, t+ t) = λ(n 1) tp(n 1, t)+σ(n+1) tp(n+1, t)+(1 (λn t+σn t))p(n, t) (5) 3 When t 0: dp(n, t) dt = λ(n 1)P(n 1, t) + σ(n + 1)P(n + 1, t) (λn + σn)p(n, t) (6) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

59 Outline Motivation: Why multiscale? Motivation: Why stochastic? Master Equation Law of mass action Steady vs absorbing states Analytical methods: Characteristic function T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

60 Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

61 Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) We will avoid these technicalities, at least for the moment, and adopt a more intuitive and practical approach (although not terribly rigorous) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

62 Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) We will avoid these technicalities, at least for the moment, and adopt a more intuitive and practical approach (although not terribly rigorous) For concreteness, consider (again) the stochastic logistic growth, i.e. a process n t such that: 1 n n + 1 with probability rate W +(n) = n 2 n n 1 with probability rate W (n) = n(n 1) K T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

63 Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) We will avoid these technicalities, at least for the moment, and adopt a more intuitive and practical approach (although not terribly rigorous) For concreteness, consider (again) the stochastic logistic growth, i.e. a process n t such that: 1 n n + 1 with probability rate W +(n) = n 2 n n 1 with probability rate W (n) = n(n 1) K Consider a state of the system, n s, is, roughly speaking, a state of the process such that W +(n s) = W (n s). T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

64 Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) We will avoid these technicalities, at least for the moment, and adopt a more intuitive and practical approach (although not terribly rigorous) For concreteness, consider (again) the stochastic logistic growth, i.e. a process n t such that: 1 n n + 1 with probability rate W +(n) = n 2 n n 1 with probability rate W (n) = n(n 1) K Consider a state of the system, n s, is, roughly speaking, a state of the process such that W +(n s) = W (n s). 1 W +(n s) is the number of births within a population of n s individuals 2 Likewise, W (n s) = the number of deaths within a population of n s individuals 3 So an steady state of our population dynamics is reached when n t = n s, since death rate is balanced by birth rates and therefore the population stays roughly constant T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

65 Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) We will avoid these technicalities, at least for the moment, and adopt a more intuitive and practical approach (although not terribly rigorous) For concreteness, consider (again) the stochastic logistic growth, i.e. a process n t such that: 1 n n + 1 with probability rate W +(n) = n 2 n n 1 with probability rate W (n) = n(n 1) K Consider a state of the system, n s, is, roughly speaking, a state of the process such that W +(n s) = W (n s). 1 W +(n s) is the number of births within a population of n s individuals 2 Likewise, W (n s) = the number of deaths within a population of n s individuals 3 So an steady state of our population dynamics is reached when n t = n s, since death rate is balanced by birth rates and therefore the population stays roughly constant n s = K + 1 which coincides with the deterministic stable fixed point (if K 1) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

66 Steady states in stochastic systems The definition of equilibrium states in stochastic systems is a bit technical and there are several definitions of equilibrium (with or without detailed balance, equilibrium or non-equilibrium, etc.) We will avoid these technicalities, at least for the moment, and adopt a more intuitive and practical approach (although not terribly rigorous) For concreteness, consider (again) the stochastic logistic growth, i.e. a process n t such that: 1 n n + 1 with probability rate W +(n) = n 2 n n 1 with probability rate W (n) = n(n 1) K Consider a state of the system, n s, is, roughly speaking, a state of the process such that W +(n s) = W (n s). 1 W +(n s) is the number of births within a population of n s individuals 2 Likewise, W (n s) = the number of deaths within a population of n s individuals 3 So an steady state of our population dynamics is reached when n t = n s, since death rate is balanced by birth rates and therefore the population stays roughly constant n s = K + 1 which coincides with the deterministic stable fixed point (if K 1) Note that W +(n) W (n) > 0 if n < n s and W +(n) W (n) < 0 if n > n s T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

67 Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

68 Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

69 Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

70 Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

71 Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: 1 Steady states are in general not absorbing states T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

72 Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: 1 Steady states are in general not absorbing states 2 W +(n s) 0 and W +(n s) 0 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

73 Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: 1 Steady states are in general not absorbing states 2 W +(n s) 0 and W +(n s) 0 3 If n = 0 then W +(0) = W (0) = 0 therefore n = 0 is an abosrbing state T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

74 Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: 1 Steady states are in general not absorbing states 2 W +(n s) 0 and W +(n s) 0 3 If n = 0 then W +(0) = W (0) = 0 therefore n = 0 is an abosrbing state 4 Also, n s belongs to the set of accessible states of n = 0 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

75 Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: 1 Steady states are in general not absorbing states 2 W +(n s) 0 and W +(n s) 0 3 If n = 0 then W +(0) = W (0) = 0 therefore n = 0 is an abosrbing state 4 Also, n s belongs to the set of accessible states of n = 0 n s belonging to the set of accessible states of n = 0 means that there is at least one consecutive set of transitions that connects n s and n 0. For example: K K 1 K T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

76 Absorbing states in stochastic systems The concept of absorbing state is the stochastic equivalent of the concept of attractor in deterministic systems An absorbing state, n 0, is characterised by W i (n 0) = 0, i.e. once the system has reached the absorbing state, it cannot leave anymore The set of accessible states of the abosrbing state is the equivalent of the basin of attraction of an attractor in deterministic systems: If n a is an accessible state of n 0 the stochastic dynamics will take the system from n a to n 0 with probability one Consider again, the stochastic logistic growth: 1 Steady states are in general not absorbing states 2 W +(n s) 0 and W +(n s) 0 3 If n = 0 then W +(0) = W (0) = 0 therefore n = 0 is an abosrbing state 4 Also, n s belongs to the set of accessible states of n = 0 n s belonging to the set of accessible states of n = 0 means that there is at least one consecutive set of transitions that connects n s and n 0. For example: K K 1 K However, if K 1 the probability of such a chain of events is vanishingly small T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

77 Summary I n s is an steady state in the sense that births and deaths are balanced. Moreover, W +(n) W (n) > 0 if n < n s and W +(n) W (n) < 0 if n > n s. This is essentially equivalent to what happens in the deterministic logistic growth model. T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

78 Summary I n s is an steady state in the sense that births and deaths are balanced. Moreover, W +(n) W (n) > 0 if n < n s and W +(n) W (n) < 0 if n > n s. This is essentially equivalent to what happens in the deterministic logistic growth model. However, n s is not an absorbing state of the stochastic dynamics. The only absorbing state is n = 0 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

79 Summary I n s is an steady state in the sense that births and deaths are balanced. Moreover, W +(n) W (n) > 0 if n < n s and W +(n) W (n) < 0 if n > n s. This is essentially equivalent to what happens in the deterministic logistic growth model. However, n s is not an absorbing state of the stochastic dynamics. The only absorbing state is n = 0 Stochastic extinctions are relatively rare provided K is big. If this is the case, the deterministic system provides a reasonable approximation to the behaviour of the model. T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

80 Summary I n s is an steady state in the sense that births and deaths are balanced. Moreover, W +(n) W (n) > 0 if n < n s and W +(n) W (n) < 0 if n > n s. This is essentially equivalent to what happens in the deterministic logistic growth model. However, n s is not an absorbing state of the stochastic dynamics. The only absorbing state is n = 0 Stochastic extinctions are relatively rare provided K is big. If this is the case, the deterministic system provides a reasonable approximation to the behaviour of the model. If, on the contrary, K is small stochastic extinctions are relatively common and the deterministic description is not an accurate one T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

81 Summary II We have seen several examples of stochastic systems in which noise and randomness are the dominating factors. Their behaviours are not captured by their deterministic or mean-field conterparts In general, we should expect non-trivial random effects for: 1 Small populations: Large fluctuations/rare events have probability P e ΩS for large system size Ω 2 Dynamics with absorbing states T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

82 Outline Motivation: Why multiscale? Motivation: Why stochastic? Master Equation Law of mass action Steady vs absorbing states Analytical methods: Characteristic function T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

83 Analytical solutions Analytical solutions to the Master Equation are rare. General situations need to be dealt with by means of perturbative methods (Lecture 2) or numerical simulation (Lecture 3) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

84 Analytical solutions Analytical solutions to the Master Equation are rare. General situations need to be dealt with by means of perturbative methods (Lecture 2) or numerical simulation (Lecture 3) For linear systems, it is possible to write a PDE for the so called probability generating function: G(s, t) = P(n, t)s n (7) n T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

85 Analytical solutions Analytical solutions to the Master Equation are rare. General situations need to be dealt with by means of perturbative methods (Lecture 2) or numerical simulation (Lecture 3) For linear systems, it is possible to write a PDE for the so called probability generating function: G(s, t) = P(n, t)s n (7) n Example: Birth-and-death process 1 Master Equation: dp(n, t) dt = λ(n 1)P(n 1, t) + σ(n + 1)P(n + 1, t) (λn + σn)p(n, t) (8) 2 Multiply by s n and sum over all n n s n dp(n, t) dt = s 2 n s n λ(n 1)P(n 1, t)s n 2 + n σ(n + 1)P(n + 1, t)s n (λn + σn)p(n, t)s n 1 (9) T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

86 Analytical solutions Example: Birth-and-death process (cont.) 1 PDE for the characteristic function: G(s, t) t G(s, t) = (λs σ)(s 1) s 2 This PDE can be solved by the method of characteristics. If G(s, t = 0) = s I (10) G(s, t) = ( ) I σ(s 1) (σ λs) exp( t(λ µ)) (11) λ(s 1) (σ λs) exp( t(λ µ)) 3 By Cauchy s formula: P(n, t) = n! G(z, t) dz (12) 2πi z n+1 T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

87 Analytical solutions Example: Birth-and-death process (cont.) 1 PDE for the characteristic function: G(s, t) t G(s, t) = (λs σ)(s 1) s 2 This PDE can be solved by the method of characteristics. If G(s, t = 0) = s I (10) G(s, t) = ( ) I σ(s 1) (σ λs) exp( t(λ µ)) (11) λ(s 1) (σ λs) exp( t(λ µ)) 3 By Cauchy s formula: P(n, t) = n! G(z, t) dz (12) 2πi z n+1 Note Even when an analytical, closed solution for the characteristic function is not available, this function and its associated PDE are the corner stone for asymptotic analysis, specially WKB asymptotics T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40

88 Outline of next lecture 1 Formal definition of a Markov process The Markov property Chapman-Kolmogorov equation Derivation of the Master Equation 2 Asymptotic methods and rare events Path integral solution Large deviations and optimal theory 3 Monte Carlo simulations methods: Gillespie s stochastic simulation algorithm T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June / 40