Formal Modeling of Biological Systems with Delays

Transcription

1 Universita degli Studi di Pisa Dipartimento di Informatica Dottorato di Ricerca in Informatica Ph.D. Thesis Proposal Formal Modeling of Biological Systems with Delays Giulio Caravagna Abstract Delays in biological systems may appear at any level of detail of the modeled system, in particular delays may be used to model events for which the underline dynamics can not be completely observed. There exist constant and variable (e.g. time dependent, state dependent) forms of delays and different modeling techniques for interpreting delays. In this thesis we address the problem of formal modeling biological systems with delays in all their variants and interpretations. In the first part of the thesis we introduce the framework for deterministic modeling of biological systems (e.g. Delay Differential Equations) and we define Delayed Chemical Master Equations. In complete accordance with the standard approach for formal modeling biological systems without delays, we also extend the framework for stochastic modeling by defining some variants of Delayed Stochastic Simulation Algorithms. In the second part of the thesis we address the problem of both qualitative and quantitative formal modeling of biological systems with delays by using existing formal languages theory and by extending well known formal languages for modeling biological systems without delays. January 2, 2009

2 Preface This proposal is developed as a thesis in progress. Sections are structured as, at the moment, I expect they will be in the final version: some of them contain results obtained in the first year of my Ph.D. studies, and others contain some ideas on further developments and on results I hope to obtain in the future. Contents 1 Motivations Scenario Deterministic Models of Biological Systems Stochastic Models of Biological Systems Simulation of Biological Systems with Delays Deterministic Models with Delays Deterministic Models with Constant Delays Deterministic Models with Variable Delays Stochastic Models with Delays Stochastic Models with Constant Delays Stochastic Models with Variable Delays Approximation Techniques for Stochastic Models Examples Epidemics Models Cellular Models Evolutionary Models Formal Modeling of Biological Systems With Delays Qualitative Modeling of Biological Systems With Delays Quantitative Modeling of Biological Systems With Delays Conclusions 27 1

3 1 Motivations Biochemistry, often conveniently described as the study of the chemistry of life, is a multi faceted science that includes the study of all forms of life and that utilizes basic concepts derived from Biology, Chemistry, Physics and Mathematics to achieve its goals. Biochemical research, which arose in the last century with the isolation and chemical characterization of organic compounds occurring in nature, is today an integral component of most modern biological research. Most biological phenomena of concern to biochemists occur within small, living cells. In addition to understanding the chemical structure and function of the biomolecules that can be found in cells, it is equally important to comprehend the organizational structure and function of the membrane limited aqueous environments called cells. Attempts to do the latter are now more common than in previous decades. Where biochemical processes take place in a cell and how these systems function in a coordinated manner are vital aspects of life that cannot be ignored in a meaningful study of biochemistry. Cell biology, the study of the morphological and functional organization of cells, is now an established field in biochemical research. Computer Science and Mathematics can help the research in cell biology in several ways. For instance, it can provide biologists with models and formalisms able to describe and analyze complex systems such as cells. This rather new interdisciplinary field of research is named Systems Biology. The approach which is used to solve a problem of system biology is typically the following: firstly the biological system has to be identified in all of its components, if possible. In particular all the involved elements and the interesting events have to be identified; this is generally one of the major problems because often not all these informations are available due to the non observability of some events or to the non full knowledge of the biological system itself. Furthermore, we say components and events rather than molecules and reactions because of the general applicability of systems biology to different biological systems (e.g. biochemistry, cell study, epidemics, population dynamics). Whenever the system has been identified, then it is possible to build a deterministic or a stochastic model; the model is obviously built on the data which has been carried out by observing the biological system and, potentially, by some biologically meaningful conjectures. These models differ from the frameworks on which they are based: in particular, the deterministic model is, in general, a system of ordinary differential equations (ODEs) which models the variation of concentrations of the involved components. Theoretically the ODEs model may be studied analytically (e.g. the solution of the equation, the equilibrium and the bifurcation points) or via numeric simulations. Practically, for a complex and real model, the analytical solution is difficult to be studied or may be impossible at all, differently, the numerical simulation is always possible. These kinds of models, although very usefull when dealing with biological systems involving a huge number of components, are not always satisfactory. In particular, when the size of the biological systems is small, their simulation does not show some behaviors which are observed in the real biological systems. To overcome this incompleteness, the stochastic models can be defined. The stochastic model can be defined only if it is possible to model the biological events with a stochastic behavior, and this is typical in biological systems due to the chaotic dynamics of the systems itselves. Such a model, which is built with the same objectives and by the same observations used to build the deterministic one, consists in defining a chemical master equation (CME), a special kind of differential equation, which describes the time evolution of the probability of the system to occupy each one of a discrete set of states. Although formally correct, this last approach is, in general, not usable because the CME is prohibitively difficult to be solved analytically [31]. To fill 2

4 this gap, from the definition of the CME, it has been defined the stochastic simulation algorithm (SSA) by Gillespie [31] which computes one trajectory in state space of the modeled system; this algorithm is exact in the sense that produces one exact time evolution of the modeled system, given an initial conditions. Unfortunately, the SSA suffers from scalability problems with respect to the size of the modeled system and with respect to the kinetics of the modeled events. In other words, if the concentrations of the involved components are huge (e.g. a cellular interaction model may contain billions of molecules), then the simulation via the SSA may be prohibitively slow. Some variants of this algorithm exist [30, 18] and, to fix this scalability problem, some approximations, [33, 17], have been developed which are not exact but practically usable under some assumptions. Whatever model is built if it fits, when it is simulated to trace its temporal evolution, the real observed data, then it can be used as a base for compositionally developing more complex models. In particular, it can be used to make predictions on the biological systems by simply adding to the model the modeling of the predictions themselves. This approach can be usefull to make biological conjectures on the behavior of more complex systems and, furthermore, to have a simulable model of the biological system. The following schema summarizes the discussed approach. This work aims at defining a wider class of biological systems which can be modeled by following the outlined approach. In particular, the ordinary differential equations, the CME and the SSA can be used only in presence of systems where the modeled events have no delay. It is the aim of this thesis to extend the frameworks to be able to model also biological systems with delays. Delays in biological systems may appear at any level of detail of the modeled system, in particular delays may be used to model events for which the underline dynamics can not be completely observed. In particular, if it would be possible to define the event in terms of the all the sub events which compose it, then that would be the best low level description of the modeled event. If this is not possible, namely there is not full knowledge of all the underline dynamics composing the main event, then it is reasonable to argue a fixed maximum time in which the sub events are completed and, consequently, it is reasonable to model the main event with a delay equal to the argued time. Notice that this a quite general idea of interpreting the delays when modeling biological systems, in particular there exist some biological systems for which the biological interpretations of delays are quite different. Some examples of this kind of systems, together with their interpretation of delays, are presented in Section 3. This work addresses the modeling of biological systems with delays by following this approach: as regards deterministic models, it is possible to define a more general framework of differential equations, the delay differential equations (DDEs) which can naturally model delays. In particular, this framework is very general with respect to the kind of delays it can models, it is of interest to study whether all this kind of different forms of delay can be biologically meaningful. In Section 2.1 some of these variants are discussed. For the same motivations we had to build a stochastic simulation framework from the deterministic one, it is worth defining, dependently on the type of delay we are interested 3

5 in, a delayed chemical master equation (DCME) conceptually analogous to the CME. Also the DCME suffers from the disadvantages of the CME and, consequently, in the same fashion of non delayed systems, it has to be defined a delayed stochastic simulation algorithm (DSSA) conceptually analogous to the SSA. Dependently on the kind of delays involved in the system, it is possible to define an algorithm as shown in Section 2.2. Summarizing, all the Chapter 2 is dedicated to the extention of both deterministic and stochastic modeling of biological systems with delays. From a computer science perspective, Chapter 4 is devoted to the study of formal language for the modeling of biological systems with delays. In particular, the formal computer science approach, with all the background theory on concurrent systems, has been used in the last years to define formal languages for systems biology. Many formalisms have been proposed, some as adaptations of the existing ones and others considered to be biologically inspired. As regards the former class of languages, it is worth mentioning the Stochastic π calculus [42], namely a stochastic extention of the π calculus [39], a process algebra for modeling concurrent processes. In this approach chemical reacting entities can be described by processes and biological reactions are modeled as communications on channels, the synchronization of communicating processes is interpreted as the firing of a reaction. As regards the latter class of formal languages, the wider one, it is worth mentioning κ calculus [24], BioAmbients [44], Brane Calculi [19], P Systems [40], Stochastic CLS [38] and Stochastic String Multiset Rewriting [9]. This languages differ for the theory on which they are built on, namely are based on process algebras theory, on concurrent systems theory, on rewriting systems theory or on their possible combinations. From a biological perspective, these languages permit to easily define more complex biological systems then simple chemical reacting systems. In fact, they have generally the possibility of expressing, by using some ad hoc synctactic operators, biological aggregations of components, arbitrarily nested membranes and more complex and general biological components. Furthermore, they provide some primitive operations for easily modeling biological events, for instance the creation, the dissolution, the merging of membranes and of their content. All the mentioned languages permit to define models on which properties con be verified via model checking or abstract interpretation, but these models can be simulated in a stochastic framework only if it has been defined a stochastic semantics. For some of them the stochastic semantics has been defined and this permitted to develop specific simulators (e.g. SPiM [53] based on the Stochastic π calculus, and CytoSim and PSym [52] based on P Systems and the CLSm [50] based on Stochastic CLS). These simulators permit, in the same fashion as the approximation techniques for the ODEs, to trace the time evolution of the modeled biological systems with respect to some simulation algorithms. The former approach is commonly named qualitative modeling and the latter quantitative modeling. The algorithm which is mainly used for simulations is, as expected the SSA, which can be applied on the result of applying the semantics to the modeled system. To this extent, the semantics of these languages are typically given in terms of inference rules which, when applied to a term describing the model, build a label transition systems (LTS) where the labels represent the stochastic rates of the events which bring from one state to another. From the initial model of the biological systems, by applying the rule of the semantics, it is possible to compute all the possible reachable states, namely the full LTS. This LTS can be used to build a Continuous Time Markov Chain (CTMC), namely a matrix of all the possible different states enriched with the probability of moving from on state to another. These probabilities, namely the stochastic behavior of the system, depend only on the current state which describe the modeled systems by definition of the CTCM. For this reason, on the CTMC it is possible to apply an algorithm, the SSA, 4

6 on which the probability of choosing one event depends only on the current state of the simulation. We can notice that whenever considering models with delays, under some interpretations of the delays, the probabilities may depend on some past states of the system and not only on the current one. Consequently, the corresponding stochastic process built by the SSA would not be correct due to the non markovian behavior of this kind of process. In other words, depending on the past state of the systems corresponds to having a non memoryless process, which is the main feature of a Markov process and, consequently, the current semantics with the corresponding LTSs cannot be used any more. Summarizing, in order to model both qualitatively and quantitatively biological systems with delays, we will discuss whether some of the well known formal languages can be extended to address qualitative modeling. Moreover, we would like to extend the semantics of some of the mentioned formalisms to quantitative modeling. 1.1 Scenario In this section we introduce the scenario in which we can define both the deterministic and the stochastic modeling of biological systems. We consider a well stirred system of molecules of N chemical species {S 1,..., S N } interacting through M chemical reaction channels R 1,..., R M. We assume the system to be confined in a constant volume and to be in thermal equilibrium at some constant temperature. We denote the number of molecules of species S i in the system at time t with X i (t) and we want to study the evolution of the state vector X(t) = (X 1 (t),..., X N (t)), assuming that the system was initially in some state X(t 0 ) = x 0. A reaction channel R j is characterized mathematically by two quantities. The first is its state change vector ν j = (ν 1j,..., ν Nj ), where ν ij is defined to be the change in the S i molecular population caused by one R j reaction; thus, if the system is in state x and a reaction R j occurs, the system jumps to state x + ν j. The other characterizing quantity for reaction channel R j is its propensity function a j (x); this is defined, accordingly to [31], so that a j (x)dt, given X(t) = x, is the probability of reaction R j to fire in state x. The probabilistic definition of the propensity function finds its justification in physical theory [31]. In order to correctly define the propensity functions of the reactions we recall the fundamental empirical law governing reaction rates in biochemistry: the law of mass action. This states that for a reaction in a homogeneous medium, the reaction rate will be proportional to the concentrations of the individual reactants involved. For example, given the simple molecular reaction 2A k B, namely a reaction which transforms the two reactants A into the single product B with kinetic constant k, is such that, by the law of mass action, the rate of the production of molecule B is: and the rate of destruction of A is: db dt = k[a]2 db dt = k[a]2 where [A] and [B] are the chemical standard notations of the concentrations (i.e. moles over volume unit) of the respective molecules. To the extent of defining the propensity function for such a reaction, accordingly to [31], would be correct to define it as a(x) = k[a] 2 where [A] denotes the concentration of A in x. This kinetic law is the fundamental and mainly used to develop models, both stochastic and deterministic. Other laws exist (e.g.michaelis Menten) and will appear later in the example of biological systems discussed in this thesis. 5

7 1.2 Deterministic Models of Biological Systems Deterministic models of biological systems are the most used and widespread models of biological systems since the last century. This kind of models consist of a set of ordinary differential equations (ODEs) which describe, in general, the time evolution of the concentrations of the involved species in a given volume. In mathematics, an ordinary differential equation is a relation that contains functions of only one independent variable, and one or more of its derivatives with respect to that variable. The general form of an ordinary differential equation for X(t) R n is dx dt = f x(t, X(t)), where dx/dt may depend on the state of the system at time t, X(t), and not on any previous states. Much study has been devoted to the solution of ordinary differential equations. In the case where the equation is linear, it can be solved by analytical methods. Unfortunately, most of the interesting differential equations are non linear and, with a few exceptions, cannot be solved exactly. Approximate solutions are obtained by numerical simulation algorithm developed in the last century. As an example of deterministic model, let us consider one of the first mathematical models of tumor immunotherapy [35], namely a model of the dynamics between tumor cells, T (t), immune effector cells, E(t), and IL-2, I L, a cytokine interleukin-2 which is observed to boost the immune system to fight tumors. For a close examination of the model we refer to [35], here we simply reproduce the model and discuss how it is built. The model consisting of the set of ODEs is the following: de dt = ct (t) µ 2E(t) + p 1E(t)I L (t) g 1 + I L (t) dt dt = r ae(t)t (t) 2T (t) r 2 bt T (t) g 2 + T (t) di L dt = p 2E(t)T (t) g 3 + T (t) µ 3I L (t). The first equation describes the rate of change for the effector cell population. Effector cells are stimulated to grow based on two terms. One is a recruitment term, ct, due to the direct presence of the tumor, where the parameter c models the antigen of the tumor. The other growth term, (p 1 EI L )/(g 1 + I L ), is a proliferation term whereby effector cells are stimulated by IL-2 that is produced by effector cells. This term is of Michaelis Menten form to indicate the saturated effects of the immune response. The last term models the natural lifespan of an average 1/k2 days of the effector cells. Second equation marks the rate of change of the tumor cells. This is described by a logistic limiting growth term r 2 T r 2 bt T. The loss of tumor cells is represented by an immune effector cell interaction at rate a. This rate constant, a represents the strength of the immune response and is modeled by Michaelis Menten kinetics to indicate the limited immune response to the tumor. Last equation gives the rate of change for the concentration of IL-2. Its source is the effector cells that are stimulated by interaction with the tumor and also has Michaelis Menten kinetics to account for the self limiting production of IL-2. The term I L µ 3 represents degraded rate of IL-2. In [35] the model is fully examined and, by looking at the time evolution of the numerical simulations and both the analytical analysis of the equilibrium of this dynamical systems, many results are discovered. In particular, four cases dependently on the parameters and on the initial configuration are observed; in one single case there is a prevalence in the tumor mass and, in the others three cases, oscillations of the tumor mass and of the 6

8 effector cells are observed. These oscillations reach very small values of the tumor mass and this is a suitable case in which a stochastic model can show interesting behaviors. In fact, the stochastic behavior can result in cases in which the tumor mass becomes 0 and the immune system eradicates spontaneously the tumor. This phenomenon can be observed in practice but is not observable by examining these equations. This kind of results hide the main motivation for defining stochastic models, namely the fact that under certain conditions, these models show behaviors which are not observable in the deterministic counterparts. 1.3 Stochastic Models of Biological Systems In this section we will briefly discuss on the stochastic modeling of biological systems. In particular, we will recall firstly both the definition of the chemical master equation and the stochastic simulation algorithm by Gillespie [31] and, secondly, we will show, via an example, why the stochastic framework is suitable as the deterministic one for the modeling of biological systems. The Chemical Master Equation In this section we recall the definition, given by Gillespie in [31], of the chemical master equation (CME). The CME is a set of first order differential equations (ODEs) describing the time evolution of the probability of a system to occupy each one of a discrete set of states; in the definition given by Gillespie [31], the crucial quantity is P (x, t x 0, t 0 ), namely the probability that, given the initial configuration X(t 0 ) = x 0, at time t the system is described by the state vector x, X(t) = x. In order to define the CME, namely the differential equation which describes the variation of such a probability in the infinitesimal time dt, the quantity P (x, t + dt x 0, t 0 ) is defined. Such a probability, assuming that the dt is chosen so small that at most one reaction can fire in the time interval [t; t + dt[, is defined in terms of these two events: - at time t the system is already in state x and in the infinitesimal time [t; t + dt[ no reaction fires; - at time t the system is in state x ν j and reaction R j fires. Summing up the probabilities of these two events, we get M P (x, t + dt x 0, t 0 ) = P (x, t x 0, t 0 ) 1 a j (x)dt + j=1 M P (x ν j, t x 0, t 0 ) a j (x ν j ) j=1 where the first term represents the probability of the former event, and the second term represents the probability for the latter event. By subtracting P (x, t x 0, t 0 ), dividing by dt, and taking the limit dt 0 we get the CME P (x, t x 0, t 0 ) t = M P (x ν j, t x 0, t 0 ) a j (x ν j ) P (x, t x 0, t 0 ) a j (x). j=1 As shown in [31], this ODE is generally difficult to solve, in particular it can be solved analytically only for a very few simple systems and, furthermore, numerical solutions may be prohibitively difficult. These difficulties justified the introduction of alternative 7

9 simulation techniques for the stochastic simulation of biological systems although the CME is an importante conceptual base for the studying of the mathematical foundations of the stochastic simulation algorithm. The Stochastic Simulation Algorithm In this section we are going to recall the definition of the stochastic simulation algorithm (SSA) by Gillespie [31]. This algorithm addresses the following problems, given the system in state x at time t, compute the time instant at which the next reaction fires and choose, accordingly to some policy, the reaction to fire. As regards the former problem, it is shown in [31] how the putative time for the next reaction can be chosen by sampling an exponentially distributed random variable with mean a 0 (x) = M j=1 a j(x). The sampling of such variable can be obtained by inverse Montecarlo algorithm for generating exponentially distributed values. Similarly, the reaction to fire is chosen accordingly to the following inequalities, j 1 i=1 a i(x) < r 2 a 0 (x) j i=1 a i(x). For a proof of correctness of these choices see [31]. Having solved this problem it is possible to state that, given an initial configuration X(t 0 ) = x 0, one possible time evolution of the modeled biological systems is given by applying the following algorithm. Algorithm SSA 1. Initialize the time t = t 0 and the system state x = x With the system in state x at time t, evaluate all the a j (x) and their sum a 0 (x) = M j=1 a j(x). 3. Given two random numbers r 1 and r 2 uniformally distributed in [0; 1], generate values for τ and j accordingly to τ = 1 a 0 (x) ln( 1 r 1 ) j 1 j a i (x) < r 2 a 0 (x) a i (x) i=1 i=1 then update x = x + ν j and t = t + τ, go to step 2. Notice that this algorithm is exact in the sense that produces one exact trajectory in the state space of the system. Furthermore, it is worth mentioning that exist many equivalent variants of the SSA, in particular they differ from the way in which they compute the putative time for next reaction and on the way in which they choose reaction to fire. The variant we presented here is named Direct Method [31], the other presented in the literature is the First Reaction Method [32]. Other algorithms for the stochastic simulation of biological systems can be found in [30, 18]. All this algorithms are similar and based on the ideas defined by Gillespie, furthermore, they suffer from the same disadvantages of this version of the SSA as explained in the previous sections. Example As an example of stochastic model let us examine the immunotherapy model of tumor presented in Section 1.2. The deterministic model, accordingly to the technique shown in [20], can be translated in the following set of reactions, 8

10 ODE term reaction propensity function ct T c T + E c[t ] µ 2 E E µ 2 µ 2 [E] (p 1 EI L )/(g 1 + I L ) I L + E I L + 2E (p 1 [E][I L ])/(g 1 + [I L ]) r 2 T T r2 2T r 2 [T ] r 2 bt T 2T r2b T r 2 b[t ] 2 (aet )/(g 2 + T ) T + E E (a[e][t ])/(g 2 + [T ]) (p 2 ET )/(g 3 + T ) E + T E + T + I L (p 2 [E][T ])/(g 3 + [T ]) µ 3 I L µ 3 I L µ3 [I L ] As expected, the modeled species are T, E and I L and is obtained by defining one reaction from each term of the equations. Consider for instance the term ct, it appears only in the first equation and, consequently, it models the introduction of an effector cell E, namely it semantics, whenever it fires, is to add 1 cell E to the population of Es. Furthermore, its kinetic in the ODE is ct where c is the antigen constant and T is the number of tumoral cells in the current state of the system. It is trivial to recognizer this as a law of mass action kinetics and, consequently, to model the ODE term it is enough to use the reaction T c T + E which, when applied, produces 1E and consumes no cells T as they appear with the same quantity as reactants and products, with kinetics c[t ]. Notice that for those reaction whose propensity function is defined accordingly to the law of mass action the kinetic constant is specified on top of the rewriting arrow, differently for those who have different kinetics, in this case the terms with Michaelis Menten kinetics, the kinetic constant is not specified in the reaction. By simulating this model instantiated with some appropriate parameters which can be found in [35], it is possible to observe that for some initial configuration of the model the tumor is eradicated by the immune system. We recall that this kind of behavior, although realistic, was not observable in the deterministic model which showed always limit cycles, namely configurations of the model which correspond to cyclic behaviors that, in this case, did not correspond to the spontaneous eradication of the tumor. This kind of result, which appears in [8], is a practical proof of how the stochastic models are as interesting as the deterministic ones. 2 Simulation of Biological Systems with Delays In this section we define both the deterministic and the stochastic frameworks for modeling biological systems with delays. Deterministic models are presented as extension of the ODEs framework and stochastic models are presented analogously as the SSA. At the end of the chapter some application examples are given. 2.1 Deterministic Models with Delays In mathematics, delay differential equations (DDEs) are a kind of differential equation in which the derivative of the unknown function at a certain time is given in terms of the values of the function at previous times. DDEs are a more general framework than ODEs and it is easy to note that ODEs are a particular case of DDEs in absence of delay. However, although the framework of DDEs is more expressive than the one of the ODEs, not all the theoretical mathematical results on the ODEs are valid for the DDEs as stated in [25]. The general form of a time delay differential equation for X(t) R n is dx dt = f x(t, X(t), X t ), 9

11 where X t = {X(t ) : t t} represents the trajectory of the solution in the past. Notice that this formulation is general enough to capture all the possible forms of delays we are going to analyze in the next sections of this thesis. As already said, the justification for the study of different forms of delays is due to the general framework of the DDEs and to the many models of biological systems which can be found in the literature, based on these different kinds of DDEs Deterministic Models with Constant Delays In this section we study DDEs in their easiest form, namely we refer to DDEs with constant delays, namely equations of the form dx dt = f x(t, X(t), X(t σ 1 ),..., X(t σ n )) with σ 1 >... > σ n 0 and σ i R. In order to introduce this framework we start by giving an example. Consider the model of tumor immunotherapy presented in [8], an extention of the model firstly presented in [35] and discussed in Section 1.2 and in Section 1.3. This model describes the interaction between two different kind of cells, tumor and effector cells, and one molecule, the interleukin IL-2 for stimulating immunotherapy. The model in [35] has been extended by adding a constant delay τ in the response of the immune system in presence of tumor cells. This kind of delay is realistic and justified by experimental observations which showed a delayed response of the immune system. The model with delays is then the following: de dt = ct (t τ) µ 2E(t) + p 1E(t)I L (t) g 1 + I L (t) dt dt = r ae(t)t (t) 2T (t) r 2 bt T (t) g 2 + T (t) di L dt = p 2E(t)T (t) g 3 + T (t) µ 3I L (t) where the term with delay is ct (t τ) which models the response of the immune system to the presence of tumor cells. For a close analysis of this model we refer to [8]. Other interesting examples of use of this kind of DDEs regard biological system at any level of abstraction, these models will be examined in Section 3 and, in the same section, it will be of interest to discuss the interpretation of delays considered in these models Deterministic Models with Variable Delays In this section we propose to study deterministic models of biological systems with variable delays. In particular, in the literature it is possible to observe two classes of delays: the first regarding the delays which depend on time or on concentrations of species, namely the DDEs of the form dx dt = f x(t, x(t), x(t τ 1 (t, x(t)),..., x(t τ n (t, x(t))) where τ i (t, x(t)) : R + R n R + is a function of both time and x(t) which represents the delay. The second regarding distributed delays, namely DDEs of the from ( dx t ) dt = f x t, x(t), γ(t, x(t ))dt 10

12 where t γ(t, x(t ))dt is the integral over the time of a function γ : R R n R + of both time and state. These kind of models can be a very useful and, although non trivial, extension of those presented in the previous section. From an application point of view we may want to model a biological system with memory in presence of external impulses. In the case in which the time needed to elaborate the response to the external impulses is fixed, then a discrete delay would be enough to model such a time quantity. Differently, as the system has got a memory, and this is typical in most complex cellular systems (e.g. the immune system [8]), then it can be observed that the response for subsequent similar impulses requires different time quantities to be elaborated. In many cases the time quantities depend on the time elapsed between the impulses or depend on the number of responses to the impulses. To model this kind of behaviors, the first form of variable delays can be used. For interesting models of DDEs with variable delays we refer to [5, 6, 36] and to the references therein where a population dynamics model of mammals is presented by using DDEs with state dependent delays. Differently, as regards distributed delays, they are a more precise generalization of the fact that an event depends on a single past state of the system. In particular, as the integral models all the area of the integration interval, then these models take into account a dense set of past states rather than a single one. For the use of integro differential equations and example models we refer to [7] and to the references therein. 2.2 Stochastic Models with Delays In this section we discuss on the stochastic models with delay. In particular, with respect to the different forms of delays we discussed in the previous sections of this chapter, namely within the deterministic modeling of delayed systems, we will try to define, if possible, equivalent chemical master equations with delays and stochastic simulation algorithm with delays. The motivation for defining stochastic algorithms are the same explained in Section 1.3 and in Section 1.3, namely the fact that the intrinsic non deterministic nature of these kind of simulations can show, and consequently justify, behavior naturally observed in real experiments but which are not captured by the deterministic models Stochastic Models with Constant Delays In this section we extend the scenario introduced in Section 1.1 by assuming that each reaction R j has a constant delay σ j R such that σ j 0. Notice that this kind of delay is the same presented for DDEs of Section In the first part of this section we will define a delayed chemical master equation in a similar way as we defined the chemical master equation of Section 1.3. After analyzing the equation, in the last part of this section, we will define some algorithms for the stochastic simulation of models with constant delays. In particular, we will give two different interpretations to constant delays. The first is considering delays as fixed time quantities which are needed, in addition to a stochastic time quantity, to fire a reaction. The second is, in the same sense of the DDEs, the fact that a reaction depends upon a state in the past history fo the system. By following the first interpretation we will define, accordingly to [13, 16, 54], a DSSA with scheduled reactions; differently, by the second interpretation, we will define the DSSA with delayed propensity functions [9]. As a future work, will be of interest to formally investigate wether these two variants of the DSSA are equivalent. Notice that these different algorithms, which differ from their interpretation of delays and from some strategies they adopt, are suitable to be used to simulate the time evolution of models of biological systems with delays but, as expected, being different implies that some systems may be simulated more correctly, from a biological interpretation of delays, 11

13 by one version of the algorithm rather than another. This lets the modeler choose, in accordance with the delays he is modeling in the system, the proper version of the algorithm to be used. A Delayed Chemical Master Equation In this section we derive, via the same principles used to derive the CME in Section 1.3 and similarly to how it is done in [13], a delayed chemical master equation (DCME) for systems with constant delays. In order to define the DCME we change the use of the propensity function with respect to the CME. In particular, in the CME the propensity function, which depends on the state at the current time t, is computed in the state x given X(t) = x, namely the value a j (x) is computed for any reaction R j. Differently, in the DCME we want to compute the propensity function in a state x which represents a past state of the system, namely X(t ) = x with t t, then we will compute the value a j (x ) for any reaction R j. The choice of the state x has to be, as expected, consistent with the delay σ j. Notice also that, having possibly different delays for each reaction, each propensity function depends, potentially, on a different past state of the system. Having pointed out this difference, we can now try to identify the quantities in which we are interested in order to define the DCME. Also in the case of the DCME these quantities are conceptually similar to those present in the definition of the CME. In particular we denote as P (x, t x 0, t 0 ; ω) the probability that the system is in state x at time t given these two facts: the initial configuration is X(t 0 ) = x 0 and, as we have delays, the value of the state for time instants preceding t 0 is given by the function ω, namely X(t) = ω(t) if t < t 0. Notice that the need of the function ω is due to the fact that, in the time interval [t 0 ; t 0 +σ j ], the value for the propensity function of reaction R j depends on the state at an instant t < t 0 and, consequently, could not be computed without using ω. We note also that this choice is strictly related to the standard mathematical technique [25] used to solve systems of DDEs by infinite approximations of set of ODEs. Analogously as for the CME, we want to define the quantity P (x, t + dt x 0, t 0 ; ω) assuming that dt is small enough that at most one reaction fires in time [t; t + dt[. We have the consider the following events: - at time t the system is already in state x and, at delayed time t σ j, the system was in state x i, no reactions fire; - at time t the system is in state x ν j and, at delayed time t σ j, the system was in state x i, reaction R j fires. Formally, denoting with I(x) the set of all possible states in which the system can be, we can define P (x, t + dt x 0, t 0 ; ω) as follows: P (x, t + dt x 0, t 0 ; ω) = M P (x, t x 0, t 0 ; ω) 1 + j=1 x i I(x) M P (x ν j, t x 0, t 0 ; ω) j=1 P (x i, t σ j x, t; x 0, t 0 ; ω) a j (x i )dt x i I(x) P (x i, t σ j x ν j, t; x 0, t 0 ; ω) a j (x i )dt Notice that the quantities P (x i, t σ j x, t; x 0, t 0 ; ω) and P (x i, t σ j x ν j, t; x 0, t 0 ; ω) denote the probability that the system is in state x i at time t σ j when knowing that at time t it is in state x and x ν j, respectively, and knowing that the initial configuration is 12

14 described by x 0, t 0 and ω. Furthermore, notice that the need of coupling all the possible system states justifies the introduction of the set I(x); in particular, for those states x i which have no connection with state x or x ν j such a probability will be equal to 0. As regards the outlined events, the probability of the former is given by the first term of the equation and the probability of the latter is given by the second one. In order to define the DCME we have to make some algebraic rearrangements of this last equation, in particular, from probability theory, we can prove the following theorem. Theorem 1. Given events A, B and C, it holds that P (A B; C) P (B C) = P (A; B C) Proof. By definition of conditioned probability we have P (A B; C) P (B C) = P (A; B; C) P (B; C) P (B; C) P (C) = P (A; B; C) P (C) = P (A; B C) By applying this theorem to the terms of the previous equation we get the following equalities P (x i, t σ j x, t; x 0, t 0 ; ω) = P (x, t; x i, t σ j x 0, t 0 ; ω) P (x, t x 0, t 0 ; ω) P (x ν j, t x 0, t 0 ; ω) P (x i, t σ j x ν j, t; x 0, t 0 ; ω) = P (x ν j, t; x i, t σ j ; x 0, t 0 ; ω). Consequently, the probability of the former event can be rewritten as P (x, t x 0, t 0 ; ω) M j=1 x i I(x) and the probability of the latter event becomes P (x, t; x i, t σ j x 0, t 0 ; ω) a j (x i )dt. M P (x ν j, t; x i, t σ j ; x 0, t 0 ; ω) a j (x i )dt j=1 x i I(x) Summarizing, the quantity P (x, t + dt x 0, t 0 ; ω) can be rewritten as follows: P (x, t + dt x 0, t 0 ; ω) = P (x, t x 0, t 0 ; ω) M P (x, t; x i, t σ j x 0, t 0 ; ω) a j (x i )dt + j=1 x i I(x) M j=1 x i I(x) P (x ν j, t; x i, t σ j x 0, t 0 ; ω) a j (x i )dt Finally, by simple algebraic rearrangement of this last equation it is possible to get the following DCME: P (x, t x 0, t 0 ; ω) t = M j=1 x i I(x) M j=1 x i I(x) P (x ν j, t; x i, t σ j x 0, t 0 ; ω) a j (x i ) P (x, t; x i, t σ j x 0, t 0 ; ω) a j (x i ) 13

15 We discuss now about the relation between the DCME and the CME. In particular, we may expect to have that in absence of delays the DCME reduces to the CME. The following proposition holds. Theorem 2. In absence of delays the delayed chemical master equation reduces to the chemical master equation Proof. In order to prove the theorem we have to prove that j = 1,..., n. σ j = 0 P (x, t x 0, t 0 ; ω) t = P (x, t x 0, t 0 ) t where the left side of the equality is the DCME end the right side is the CME. If we note that the probability of being in two different states at the same time instant is 0, namely P (x, t; x, t ) = 0 if t t, and, furthermore, as we have no delays, than we don t need ω in out initial configuration, x 0, t 0, then P (x ν j, t; x i, t x 0, t 0 ; ω) = P (x ν j, t x 0, t 0 ) because x ν j = x i. Consequently, as x ν j = x i then a(x ν j ) = a(x i ) and we get the following equation M j=1 x i I(x) P (x ν j, t; x i, t x 0, t 0 ; ω) a j (x i ) = M P (x ν j, t x 0, t 0 ) a j (x ν j ). With the same considerations it is possible to show that, without delays, it holds M j=1 x i I(x) j=1 P (x, t; x i, t σ j x 0, t 0 ; ω) a j (x i ) = M P (x, t x 0, t 0 ) a j (x). By combining these two observations, it holds that, without delays, the DCME reduces, as expected, to the CME. Finally, we can notice that the DCME has the same disadvantages of the CME, namely it cannot be always solved analytically and its numerical simulations may be difficult, however it is an importante conceptual base for the studying of the mathematical foundations of the DSSA with constant delays. The DSSA with Scheduled Reactions In this section we introduce the DSSA with scheduled reactions. Its definition appears in the works by [13, 16, 46] although the proof of its correctness is given in [47] which build a stochastic process which behaves exactly as the one built by the algorithm. In this section we introduce a formalization of the DSSA presented in the literature with some simple differences, in particular in our scenario we have every reaction with a non negative delay. Differently, in the literature is presented a scenario where the reactions ar divided between delayed and non delayed. We remark that this difference is not crucial as these DSSAs are equivalent. The main idea at the base of interpreting delays as scheduled reactions is the following: the delay of a reaction is not interpreted as the fact that propensities have to be computed in the past states of the system, but as an intrinsic minimum quantity of time which has to be spent in order to fire a reaction. In other words, the delay of a reaction is interpreted as a constant time quantity which has to be summed to a stochastic time quantity in order to fire the reaction. In particular, the stochastic time quantity is computed, as expected, as in the SSA, namely it is an exponentially distributed number with mean a 0 (t). We have to note that, given the system in state x at time t, the propensity functions, with j=1 14

16 this interpretation of delays, are computed in the current state x rather than in a past state of the system. This is in accordance with the fact that the delay will be part of the time spent to fire the reaction. This is the main difference between the version of the DSSA we present here and in the next sections. As regards delays, if τ is the stochastic time quantity computed for the next reaction to fire, say R j, then the reaction will complete its firing not at time t + τ as in the SSA, but will be scheduled to fire at time t+σ j +τ and the clock will be update, as in the SSA, at time t + τ. This scheduling of the firings yields to the fact that, sometimes, within the time interval [t; t + τ] a reaction was already scheduled to fire and, in this case, a proper strategy has to be adopted which we will discuss later. In the literature are unformally discussed two different scheduling strategies, [54]. The first strategy, the full scheduling, completely applies a reaction (e.g. removes its reactants and inserts its products) at the moment in which it is found to be candidate for firing (e.g. it was scheduled some instants before). The second one divides the firing of a reaction in two different time moments. More precisely, the reaction, and consequently its state change vector, is divided accordingly to its reactants and to its products. At the moment in which the reaction is scheduled, the state of the system changes accordingly to the portion of the state change vector regarding the reactants. At the moment in which the is found to be scheduled, namely at the moment in which the full scheduling completely applied it, the state of the system changes accordingly to the portion of the state change vector regarding the products. It is to note that, as regards the first strategy, it may schedule a reaction too many times with respect to the number of times it could fire in the current state of the system. This problem, which typically is observed when a consuming reaction has a big delay with respect to average time steps of the DSSA, can be easily solved if in this scenario we check for applicability of the scheduled reactions when we have to perform their application. The biological motivations which justify these two different scheduling strategies are to be discussed. In particular, in the case of full scheduling, as we do not remove the reactants from the state of the simulation when we schedule the reaction, we may think of having a reaction which, when starts firing, it does not inhibit the reactants from having other interactions within the modeled system. A typical example of this kind of interaction is common in the modeling of epidemiological systems [15, 55]; in particular, if we are modeling the infection process with a delay, which can be interpreted as the minimum time quantity to get infected after a contact with the illness, then whenever the process of becoming infected is started between two individuals, those individuals are not removed from the population and can interact with the other individuals of the population spreading the illness. This is a typical use of the delay in the definition of epidemiology models. Differently, as regards the partial scheduling strategy, in this case we remove the reactants from the population at the time in which we schedule the reaction. From a biological perspective this corresponds to inhibiting the involved reactants to have any other reaction within the current modeled system. An example application could be an evolutionary model [37] in which the reproduction is a process which take place in a separate and safe area; in this case we may think the the reactants which are going to give birth to a juvenile, are moving in this safe place and, consequently, do not take part into any other event which may happen within the modeled population until they come back. Notice that whenever modeling, the choice of the DSSA version to use depends on the biological interpretations of the delays we have in the model with respect to the events of the system. In order to formally define these two different strategies and, consequently, to define two different DSSAs, let us denote each state change vector ν j as a the composition of the state change vector for reactants, νj r, and the state change vector for products, νp j, 15

17 noting that ν j = ν r j + νp j. The former strategy for scheduling reactions yields to the definition of the following version of the DSSA with full scheduling. Algorithm DSSA with full scheduling 1. Initialize the time t = t 0 and the system state x = x With the system in state x at time t, evaluate all the a j (t) and their sum a 0 (t) = M j=1 a j(x(t)); 3. Given two random numbers r 1, r 2 uniformally distributed in the interval [0; 1], generate values for τ and j accordingly to τ = 1 a 0 (t) ln( 1 r 1 ) j 1 j a i (X(t)) < r 2 a 0 (t) a i (X(t)) i=1 i=1 (a) If delayed reaction R k is scheduled at time t + τ k and τ k < τ and it can be correctly applied, then update x = x+ν k and t = t+τ k ; (b) else, schedule R j at time t + σ j + τ, set time to t + τ; 4. go to step 2. Notice that this algorithm, which is, in its schema, similar to the SSA, contains its full scheduling strategy in its step (3a), namely when executes x = x + ν k. Furthermore, in order to check if the scheduled rule is applicable, we state an informal constraint which can be easily implemented by checking if the rectants of R k are a sub multiset of x. This control was not present in the DSSA versions studied in the literature [13, 16, 46] as they considered simply delays in non consuming reactions which can be always applied in any state of the system. This can be seen as a more general version of their DSSA. Similarly, the latter strategy for scheduling reactions yields to the definition of the following version of the DSSA with partial scheduling. Algorithm DSSA with partial scheduling 1. Initialize the time t = t 0 and the system state x = x With the system in state x at time t, evaluate all the a j (t) and their sum a 0 (t) = M j=1 a j(x(t)); 3. Given two random numbers r 1, r 2 uniformally distributed in the interval [0; 1], generate values for τ and j accordingly to τ = 1 a 0 (t) ln( 1 r 1 ) j 1 j a i (X(t)) < r 2 a 0 (t) a i (X(t)) i=1 i=1 (a) If delayed reaction R k is scheduled at time t+τ k and τ k < τ then update x = x + ν p k and t = t + τ k; (b) else, schedule R j at time t + σ j + τ, update x = x + νk r and set time to t + τ; 4. go to step 2. Notice that this second DSSA is, as the previous one, similar to the SSA, and it contains its partial scheduling strategy in steps (3a) and (3b). In particular, it removes 16