arxiv: v2 [q-bio.pe] 5 Jan 2018

Transcription

1 Phenotyic switching of oulations of cells in a stochastic environment arxiv: v [-bio.pe] 5 Jan 8 Peter G. Hufton,, Yen Ting Lin,,, and Tobias Galla, Theoretical Physics, School of Physics and stronomy, The University of Manchester, Manchester M3 9PL, United Kingdom Theoretical Division and Center for Nonlinear Studies, Los lamos National Laboratory, Los lamos, New Mexico 875, US In biology henotyic switching is a common bet-hedging strategy in the face of uncertain environmental conditions. Existing mathematical models often focus on eriodically changing environments to determine the otimal henotyic resonse. We focus on the case in which the environment switches randomly between discrete states. Starting from an individual-based model we derive stochastic differential euations to describe the dynamics, and obtain analytical exressions for the mean instantaneous growth rates based on the theory of iecewise-deterministic Markov rocesses. We show that otimal henotyic resonses are non-trivial for slow and intermediate environmental rocesses, and systematically comare the cases of eriodic and random environments. The best resonse to random switching is more likely to be heterogeneity than in the case of deterministic eriodic environments, net growth rates tend to be higher under stochastic environmental dynamics. The combined system of environment and oulation of cells can be interreted as host-athogen interaction, in which the host tries to choose environmental switching so as to minimise growth of the athogen, and in which the athogen emloys a henotyic switching otimised to increase its growth rate. We discuss the existence of Nash-like mutual best-resonse scenarios for such host-athogen games. I. INTRODUCTION Outside of laboratory conditions, microbial cells are often subject to unredictable adverse environmental changes. s a mechanism to survive such changes, we now understand that cells have evolved to switch stochastically between henotyic states [ 5]. In contrast to more familiar forms of resistance caused by ermanent genetic mutations, these switches are eigenetic in nature, reversible, and lead to heterogeneity in the oulation. Microbial cells have been shown to use this mechanism as a survival strategy to react to changing environments, for examle induced by the administration of antibiotics or other short-term changes in environmental conditions [6, 7]. Due to the continued interest in antibiotic resistance, understanding the use of henotyic switching as a survival strategy is essential. This aer focuses on modelling the dynamics of henotyic switching in bacterial oulations. Within such a oulation, variability from cell to cell can lead to an fitness advantage. classic study by igger [8] suggests the existence of a suboulation of bacterial more resistant to antibiotics than other members of the oulation; these more resilient cells are often referred to as ersisters [9 ]. The difference between normal and ersister cells is henotyic rather than genotyic. Conseuently, a cell may switch back and forth between the normal and ersister states, and a given oulation will contain suboulations of both henotyes simultaneously. This heterogeneity in the oulation constitutes a strategy to imrove resistance to environmental stresses, a dynamic sometimes described as bet-hedging []. iological systems use this form of risk-balancing against a number of different environmental stresses, including differences in temerature, concentrations of nutrients and toxins, or the state of a host immune resonse [6, 3 5]; Thattai et al. [6], for examle, suggest a model considering the growth of cells in the urinary tract where the environment relates to the state of urination. The effects of environmental sources of noise on biochemical networks has been an imortant area of theoretical research, where it has been shown to modify switching rates [7 9] and induce bistability [, ]. Mathematical models of such bet-hedging strategies often focus on eriodically varying environments, mainly to kee the analysis manageable. Examles can be found in [7, 3, 9]. To a lesser extent, bet-hedging has also been studied in stochastically varying environments [6, 3 35]. In this aer, we resent a mathematical framework to model henotyic switching as a mechanism of ersistence in a fluctuating environment. We begin by considering an individual-based model, where ossible reactions include random birth and death events and stochastic henotyic switching of individual cells, both in a eriodically and stochastically changing environments. We use a recently-develoed mathematical aroach [36] to characterise the joint random rocesses and derive formulae for the fitnesses of the suboulations and the overall growth rate. Further eter.hufton@ostgrad.manchester.ac.uk yentingl@lanl.gov tobias.galla@manchester.ac.uk

2 (a) environment α γ δ λ λ environment α β γ δ β (b) (c) ln a t, ln b t t t t FIG.. (a) Schematic overview of the model. Each cell can exress either henotye or henotye, and switches from to with rate, and vice versa with rate. Cells of tye dulicate with rate α σ and die with rate γ σ, where σ {, } is the state of the environment. The rates for henotye are β σ and δ σ. The environment switches from state to state with rate λ, and vice versa with rate λ. The rates are such that the growth rate in environmental state is higher for cells with henotye, and the growth rate in environmental state is higher for cells with henotye (µ > µ, µ < µ ). (b) tyical trajectory of the oulations as a iecewise-deterministic Markov rocess (see text for details, henotye orange, henotye urle). The background shading indicates whether the environment is in state (light) or state (dark). The oulation of henotye is tyically greater than that of henotye in environment and vice versa in environment. (c) The corresonding trajectory of the roortion of the oulation exressing henotye. s the environment switches states the roortion of henotye alternately tends towards the two fixed oints, and, discussed in the text. Simulations use arameter set (b) (see text and endix ). Environmental switching rates are λ =., λ =. and henotyic switching rates are =.8, =.3. analysis shows that heterogeneity is advantageous for slowly switching environments, whereas homogeneity conveys a fitness advantage for uickly switching environments. y comaring our results to the case of a eriodic environment, we find that environmental stochasticity can, in rincile, lead to strategies of henotyic switching which are very different from those in eriodic external environments. The imlications of our analysis are twofold. Firstly, the method advances the mathematical framework for analysing ecological models in random environments. It rovides not only a more efficient comutational scheme for studying growth dynamics than direct simulations, but also offers mechanistic insights into the roblem. Secondly, the method can be used to investigate effective rotocols of administering treatment, such as antibiotics to bacterial infection in a control-theoretic framework. We demonstrate both of these alications in idealised scenarios in the later sections of our aer. We also identify a scenario where the cometition between the switching dynamics of a host environment and the henotyic resonse of the athogen constitute a game-theoretic scenario. Our analysis allows us to identify the otimal control strategy for the host and the otimal survival strategy for the athogen. We show how this can result in mutual best resonses, akin to what is known as Nash euilibria in game theory [37]. The remainder of the aer is organised as follows. In Sec. II we set u an individual-based model of henotyic heterogeneity in a stochastic environment. We then aroximate the dynamics of this model and obtain a solution for the average growth rate of the oulation of cells (Sec. III). Using this solution, we analyse how the average growth rate changes with the arameters in Sec. IV. Secifically, we look into the otimum bet-hedging strategy for the bacteria to maximise growth rate, and conversely the environmental switching strategy that best hinders the bacteria s growth rate. In Sec. V we summarise our results, and discuss ossible directions for future work. II. MODEL DEFINITIONS. Individual-based model We study an individual-based model of a well-mixed oulation of a secies with henotyic switching in a changing environment. Figure (a) resents a summary of the model. We consider a simlified scenario of two henotyes and two environmental states. Each cell in the oulation exresses either henotye or henotye, reresenting the normal and ersister henotyes, resectively. The birth and death dynamics of the henotyes deend on an environmental state, which we label σ and which takes values or. State reresents the growth state ; in this

3 state external stresses on the oulation are absent and the normal henotye outerforms the ersister henotye. State on the other hand, is the stress state and indicates that an external stress acts on the oulation. In this state, the ersister henotye outerforms the normal henotye. This stress can for examle be the resence of antibacterial agents, the absence of nutrients, unfavourable temeratures, or the state a host immune resonse. In our model individuals exressing henotye dulicate with rate α σ and die with rate γ σ. The corresonding rates for henotye are β σ and δ σ. We define the net growth rates µ σ = α σ γ σ and µ σ = β σ δ σ. Cells exressing henotye are better suited to environment, whereas cells exressing henotye are better suited to environment. Secifically, arameters are chosen such that µ > µ and µ < µ. Individuals switch between the two henotyes stochastically [5]: individuals of henotye switch to henotye with rate, switches in the oosite direction occur with rate. The rates and constitute the switching strategy of the oulation. These rates are assumed to be indeendent of the environment, i.e., cells switch between tyes and without sensing the state of the environment. The mathematical formalism we develo, however, does not rely on this assumtion and can readily be extended to include environmental sensing, as considered in Refs. [6, 3, 38 ]. More secifically, all euations in this section can be modified to consider environmental sensing under the relacement of henotyic switching rates and with environment-deendent uantities σ and σ. We focus mainly on the case when the environmental switching follows a Markov rocess, i.e., the environmental rocess is memoryless. We also assume that the dynamics of the environment are external, in articular it is indeendent of the oulation. Switches from state σ = to state σ = occur with constant rate λ, and switches from to occur with constant rate λ. s a conseuence, the lengths of the eisodes sent in either environment are identically and indeendently distributed random variable samled from exonential distributions with rate arameters λ and λ, resectively. Mathematically, the environment is described by an asymmetric random telegrah rocess []. In real-world systems the environmental switching rocess will often have memory and will not occur at constant rates. Instead, transitions of the environmental state will be more likely to occur at certain moments in time. In the extreme case of a laboratory setting they may be strictly rescribed by a definite exerimental rotocol. The case of strictly eriodic switching between environmental states is regularly assumed in the literature, mostly for mathematical convenience. ut, just like the Markovian case this not entirely realistic either. The main motivation for comaring the cases of a Markovian environment on the one hand, and a strictly eriodic environment on the other, is that these cases serve as two extremes; they delimit the regime of real-world controlled environmental switching statistics, and by studying the two extremes, we shed light on more realistic scenarios oerating in-between. In rincile, the model described above could be simulated using the Gillesie algorithm [3]. In ractice, this method is slow, in articular if the total oulation grows with time. Instead we therefore use a set of aroximations which allow us to simulate the rocesses more efficiently, and also to obtain analytical results. Starting from the master euation for the model, we aroximate the dynamics in the limit of a large but finite system size. This leads to a diffusion rocess with Markovian switching [], and allows us to erform the so-called linear-noise aroximation [5]. From this we obtain exlicit exressions for the average growth rate of the oulation. 3. Diffusion rocess with Markovian switching and linear-noise aroximation. Since our model is Markovian, the evolution of the robabilities of the system to be in a secific state is described by a master euation [, 5, 6]. Excet for a few secial cases, however, there exists no analytical solution for a general master euation. We build on recent work [36, 7, 8] to aroximate the dynamics of systems with switching environments, and erform a Kramers Moyal exansion [, 5, 6] in the limit of a large oulation of cells, but maintaining the discreteness of the environment. Further details can be found in endix. The result of this exansion for our model is a set of two couled Ito stochastic differential euations (SDEs) for the number of individuals a t and b t with henotyes and resectively. These euations are da t = ( µ σ a t a t b t ) dt σ (a t, b t )dw () t σ (a t, b t )dw () t, (a) db t = ( µ σ b t a t b t ) dt σ (a t, b t )dw () t σ (a t, b t )dw () t. (b) In the limit of large oulations, the formerly-discrete numbers of individuals a t and b t are aroximated as continuous variables in Es. (). The uantities W () t and W () t are indeendent Wiener rocesses (rownian motion). The subscrit t is used throughout this aer to indicate a random rocess. The coefficients σij (a, b) characterise the strength of the demograhic noise arising from the random birth-and-death rocess and the henotyic switching events [, 5]. We rovide the exact forms of σij in endix. Euations () describe the random evolution of the oulation of bacteria between switching events of the environment []. In articular the uantities on the right-hand side with a subscrit σ deend on the current state of

4 the environment. For Markovian switching of the environment as described above, the environmental rocess σ t is governed by the master euation d dt P σ(t) = λ σ P σ (t) λ σ P σ (t), () where P σ (t) denotes the robability that the environment is in state σ {, } at time t. In our analysis, we also consider a eriodic environment. In this case the state of the environment is a deterministic function given by { for t < λ σ(t) =, for λ t < λ (3) λ, and then reeated with eriod T = /λ /λ. This is chosen such that the duration of eisodes sent in environments and resectively in the eriodic case match the mean eisode durations of the random environmental rocess. The above euations rovide us with a comutational scheme for simulating trajectories of the system. For stochastically switching environments, environmental switching times are generated randomly from an exonential distribution, using rate arameters λ and λ in environments and, resectively. Euations () are numerically integrated using the current σ until the next switching time is reached. When the rescribed switching is reached, we switch environmental state and reeat the rocess. This rocedure is more efficient than direct simulations of the individual-based system, esecially for large oulations. In articular the time to simulate Es. () does not increase with increasing oulation size, in contrast to the comuting time reuired by the Gillesie algorithm. y virtue of the central limit theorem, the strength of the demograhic noise scales with the inverse suare root of the total size of the oulation. Hence in the limit of large oulations, the contribution of the diffusive terms in the SDEs [Es. ()] becomes negligible. Ignoring the diffusive terms leads to a so-called iecewise-deterministic Markov rocess (PDMP) [9, 5], da t = ( µ σ a t a t b t ) dt, db t = ( µ σ b t a t b t ) dt. (a) (b) This rocess evolves deterministically between stochastic environmental switching events, which occur at discrete times. This simlification allows for analytical solutions, and much of our further analysis will be carried out in this limit. Figure (b) shows a tyical trajectory for both henotyes and the environment using the PDMP method in a stochastically switching environment. The size of both oulations increases aroximately exonentially with time. This rovides an a osteriori justification for our aroximation to neglect intrinsic demograhic noise. We note that the oulation of henotye tends to be larger in environmental state, and in state henotye is usually more abundant, at least once enough time has assed since the last environmental switch. III. NLYSIS. Stochastic differential euations for oulation size and comosition The model above is decetively simle. y ignoring demograhic noise, the oulation dynamics of E. () is a linear dynamical system, described by two couled ordinary differential euations with Markovian switching. The standard techniue to solve the dynamics of couled linear ODEs is to find the eigenvalues and eigenvectors of the roagating matrix. long the eigenvectors, growth or decay is exonential with rates eual to the corresonding eigenvalues. There are two subtleties in our articular case. First, when we have Markovian environmental switching, the time between environmental switches is random, and so the matrix describing the roagation of the oulation from the start of an eisode to the end is also random. Second, the eigenvectors of the roagating matrix in one environment do generally not align with those in the other environmental state. s a result, the roagation of the oulation in the long run is described by roducts of a seuence of non-commuting random matrices. This makes further analysis difficult. Progress can however be made by introducing the rocesses n t = a t b t, t = a t. a t b t (5a) (5b)

5 They describe the total number of individuals in the oulation, and the roortion of individuals of henotye, resectively. Together, the rocesses n t and t rovide an alternative coordinate system for describing the state of the system; such coordinates have been used reviously to describe cometition in growing oulations [5, 5]. With these definitions, Es. (a) and (b) become dn t =n t [ µ σ t µ σ ( t ) ] dt n t C σ ( t )dw () t n t C σ ( t )dw () t, (6a) d t = [ σ t ( t ) t ( t )] dt n t C σ ( t )dw () t n t C σ ( t )dw () t, (6b) where σ = µ σ µ σ. Details of the coefficients C σij () are given in the endix. Euation (6a) describes the growth of the total oulation; the average er caita growth rate is given by the exression in the suare bracket. Euation (6b) describes the evolution of the roortion of cells exressing henotye. Suressing again the demograhic noise leads to a PDMP descrition of n t and t, valid in the limit of a large system. The corresonding dynamics are given by Es. (6), with the noise terms dw () t and dw () t removed. Interestingly, in this limit the evolution of the fraction t decoules from the evolution of the total oulation: the roortion of each henotye in the oulation is indeendent of the size of the total oulation and follows the nonlinear logistic euation obtained from E. (6b). It is imortant to stress that the decouling of the two rocesses is a conseuence of the linearity of the model. More general dynamics will not have this roerty for these models the dynamics of t and the growth rate may deend on the oulation size n t. Nevertheless, our aroach may still offer insight into cases with non-linear interactions. standard techniue for analysing cometition in laboratory exeriments involves successive dilution stes of a oulation, in order to limit the oulation size to some region of biological interest and exerimental racticability [53, 5]. The dynamics and growth rates are thus considered in this limited region of n t only. For our mathematical framework we roose an analogous techniue as a otential area of future research, namely to fix the oulation size at some secific n, and then to study the effective growth rate the contents of the suare bracket in E. (6a). This techniue may also be comared to a birth-death model such as the Moran rocess, in which the oulation size is fixed through aired birth and death events [55]. 5. Calculation of average growth rate The right-hand side of E. (6a) shows that the instantaneous growth rate is given by µ t µ σ t µ σ ( t ). (7) We are interested in comuting the average growth rate of the system, which we write as E (µ). This uantity can be understood as the instantaneous growth rate ṅ/n, averaged over many realisations of the random rocesses (after a transient time has assed). Since the rocess t is ergodic, this ensemble average is euivalent to an average over a long time. Furthermore, it is easy to show that this average instantaneous growth rate is euivalent to the average rate of the exonential growth of the oulation over a long time, the so-called dominant Lyaunov exonent [56]. To roceed, it is useful to first consider the average growth rate in a given environmental state σ, for which we write E (µ σ). This is the instantaneous growth rate averaged over all oints in time at which the environment is in state σ. In order to comute this object and given E. (7) it is sufficient to know E ( σ) the average value of in environmental state σ. In order to obtain this object, an average over intrinsic and environmental fluctuations is reuired. We note that these intrinsic fluctuations are reresented in the stochastic differential euations Es. (6) by multilicative noise. Further simlification can be achieved in the linear-noise aroximation (LN) [, 5], valid also for large but finite oulations. The LN consists in relacing t in the noise correlators (the coefficients C σij ) by the trajectory of the PDMP. In this limit, all fluctuations about the PDMP trajectory due to intrinsic noise will then be symmetrical, and so erforming the average of with resect to intrinsic fluctuations is euivalent to considering the PDMP rocess only, and to comute E( σ) as an average over the environmental rocess (see also Ref. [57]). Thus, for the remainder of the analysis we concentrate on the PDMP descrition of t, d t = [ σ t ( t ) t ( t )] dt. (8) Figure (c) shows a samle ath of the PDMP for the roortion of the oulation with henotye, t. The rocess tends towards two different fixed oints deending on environmental state. n analysis of E. (8) shows that there are two fixed oints for each environmental state: one stable σ, and one unstable σ. These are given by ± σ = σ ± ( σ ) σ σ. (9)

6 6 (a) (d) (g) ⲫ - ⲫ state ⲫ state - ⲫ (b) (e) (h) 3 3 stochastic environment ⲫ -.5 eriodic environment.5 fixed oints ⲫ ⲫ state state - ⲫ (c) (f) (i) 3 5 ⲫ ⲫ ⲫ state state - ⲫ.5 ⲫ.5 ⲫ.5 ⲫ FIG.. Panels (a), (b), and (c) show the stationary distributions of the roortion of cell in henotye for the case of a stochastic environment. These distributions are obtained from E. (). From left to right, anels corresond to increasing environmental switching rates. The orange line shows Π (), and the urle line Π () (see text for details). In the case of slow environment switching (a), the distribution is sharly eaked at the two fixed oints whereas in the case of fast environmental switching (c) the distribution is eaked in the central region between the two fixed oints. Panels (d), (e), and (f) show the stationary distributions for the case of a eriodic environment. Panels (g), (h), and (i) deict the ositions of stable and unstable fixed oints and the direction of flow in each state. Parameters are from set (a) (see text and endix ), along with: (a),(d), and (g) λ, λ =.,, =.6; (b),(e), and (h) λ, λ =.,, =.; (c),(f), and (i) λ, λ =,, =.. For our model, < since state favours henotye and state favours henotye. s the environment switches between states, the rocess t alternates between tending towards these two stable fixed oints. fter sufficient time has assed, the rocess will be confined to the interval between the two stable fixed oints (, ). Euation (8) describes a single-variable PDMP in a Markovian, two-state environment. Processes of this form have been the subject of recent studies, articularly in the context of gene regulatory networks [36, 7, 58, 59]. Reference [36] rovides a general solution for the stationary distribution Π σ() for such a rocess. In the context of the resent model, we find Π () = N ( ) g ( ) g ( ) h ( ) h, (a) Π () = N ( ) g ( ) g ( ) h ( ) h, (b) for (, ). These densities are to be be interreted as follows: Π (u)du is the robability to find the system in environmental state and with (u, u du] in the stationary state. similar interretation alies to Π (). The exonents g and h are given by g = λ ( ), h = λ ( () ), and N is determined by normalisation, [Π () Π ()] d =. Panels (a) (f) in Figure show the resulting stationary distributions for three different rates of environmental and henotyic switching. In anels (a) (c) we deict results for stochastic environmental dynamics, and in anels (d) (f) the environment is eriodic. The calculation of the stationary distribution for the eriodic case is outlined in the endix C. The orange line in each anel shows Π () and the urle line shows Π (). s seen in the figure, the distributions are eaked near the two fixed oints when environmental switching is slow [anels (a) and (d)]. t fast

7 7 (a) (d) growth rate growth rate (b) (e) 6 stochastic environment - - growth rate 6 eriodic environment - - growth rate (c) (f) growth rate growth rate FIG. 3. Distribution of the instantaneous growth rate for the case of stochastic environmental dynamics [anels (a), (b), and (c)], and eriodic environment [anels (d), (e), and (f)]. Environmental switching rates increase from the anels on the left to those on the right. Each distribution consists of two distinct comonents, corresonding to the two environments. The dashed line indicates the average growth rate. Parameters are the same as in Fig.. environmental switching rates, the system sends most of its time in the central region away from the fixed oints [anels (c) and (f)]. The distributions in Es. () can be used to calculate the of growth rates via E. (7). For the case of stochastic environments these are shown in Fig. 3(a c). We roceed to study the fitness of a given henotyic switching strategy (i.e., given switching rates and ). To do this we focus on the average growth rate where we have E (µ) = P E (µ ) P E (µ ), () E (µ σ) = µ σ E ( σ) µ σ [ E ( σ)], (3) and where P σ is the robability of finding the environment in state σ in the stationary state. In the two-state system these are given by P = λ /(λ λ ) and P = λ /(λ λ ). The exectation value E ( σ) is given by E ( σ) = Π σ() P σ d. () Using Es. () we therefore have a closed integral euation for the average growth rate of the oulation. Evaluating E. () numerically, and substituting into E. () rovides a very efficient way of calculating the average growth rate in terms of the model arameters. Results for the mean growth rate are indicated as dashed vertical lines in Fig. 3. The analysis below is for henotyic switching rates, and, which do not deend on the state of the environment. We note that the mathematical formalism alies to the case of environment-deendent switching rates as well; the calculation of the average growth rate can still be carried out if we relace and with environment-deendent rates σ and σ. The dynamics are still reducible to a two-state PDMP for a single degree of freedom, and the stationary distribution can be obtained. Thus, the theory above rovides an analytical formalism to investigate this case. In the following section we use our solution to investigate the otimum switching strategy, i.e., the switching rates, which allows the cells to best roliferate. Conversely, we also identify the otimum environmental switching rates, i.e., the switching rates λ, λ which best inhibit the growth of cells.

8 8. (b) stochastic environment (c) (a) eriodic environment. (f) (e) (d) FIG.. Heatma lot of the average growth rates for different values of the henotyic switching rates and. Panels (a), (b), and (c) show the case of a stochastic environment, and anels (d), (e), and (f) show that of a eriodic environment. Environmental switching rates λ and λ increase from left [(a), (d)] to right [(c), (f)]. The black circle in each anel indicates the switching strategy which otimises the growth rate. When the environmental switching rates are slow, there is a non-trivial and maximising the growth rate. This means heterogeneity has a fitness advantage. When the environmental switching is fast, homogeneity is favoured. The stochastic case is found to have a higher growth rate than the eriodic case for every value of and. Parameters set (a) (see endix ) is used, along with: (a) and (d) λ =., λ =.33; (b) and (e) λ =., λ =.33; (c) and (f) λ =, λ = 3.3. IV. RESULTS In the context of our model, the biological strategy of bet-hedging refers to heterogeneity in the oulation to increase resilience against environmental changes, while at the same time maintaining the caability of growth. Secifically, we will uantify temoral heterogeneity below the extent to which both henotyes are resent over the course of time, but not necessarily simultaneously. For examle, a oulation may largely consist of only one henotye at any one time, but as the environment changes state, both henotyes may be exressed in turn. Viewed over time such a oulation would be heterogeneous. In our model, in the absence of external stress (state σ = ), henotye has a higher fitness. On the other hand, not having any individuals of tye in the oulation is risky, as henotye is more suscetible to the stresses in environment σ =. The bet is hence hedged (the risk limited) by maintaining the caacity for a oulation of ersister cells to establish itself should an external stress occur. The analysis in Sec. III rovides the tools necessary to investigate the mechanics of bet-hedging. In articular, we have obtained an exression for the average growth rate, which can be evaluated numerically by a single integration. This significantly reduces the comuting time reuired to analyse the dynamics, in contrast to direct Monte Carlo simulation of the individual-based model [3]. This allows us to exlore a wide range of arameter sace to investigate bet-hedging strategies.. Otimal rate of henotyic switching rate for stochastic environments We first study the deendence of the average growth rate on henotyic switching rates, for given environmental switching dynamics. We consider different regimes of environmental dynamics, ranging from slow to fast, relative to the time scale of the growth of the oulation.

9 henotyic switching rates henotyic switching rates mean growth rate μ 9 (a) - stochastic environment heterog. (b) - eriodic environment heterog. (c) stochastic eriodic environmental switching rate environmental switching rate environmental switching rate λ FIG. 5. (a) The otimum henotyic resonse as a function of environmental switching rates for a stochastic environment. The ratio of the environmental switching rates is fixed at λ = λ. The vertical axis shows the henotyic switching rates 3 and which maximise the average growth rate. The grey, dashed lines show the environmental switching rates λ and λ for comarison. When the environment switches slowly, the otimum strategy is given by matching the environmental rates: = λ, = λ. t faster environmental switching rates, the otimum strategy involves switching more slowly than the environment. t very fast environmental switching rates, the best strategy is to stay in one articular henotye ( ). (b) The otimum henotyic as a function of environmental switching rates for a eriodic environment. The eriods the environments sends in state and have fixed durations, /λ and /λ, resectively. t slow environmental switching rates, the otimum strategy is for the henotyic switching rates to match the environment. t faster switching rates, the otimum strategy is to stay in a articular henotye. For intermediate switching rates, henotye switching rates slightly faster than the environment maximise the growth rate. The transition from heterogeneity to homogeneity occurs at slower environmental dynamics as in the stochastic case. The green dotted lines in anels (a) and (b) show the measure of temoral heterogeneity defined in the text. (c) Mean growth rate achieved by otimum choice of henotyic switching rates and as we roortionally increase environmental switching rates. The markers indicate the results of Monte Carlo simulation of the PDMP and the eriodic rocess. The mean growth rate in the case of stochastic environments is found to be universally larger than in eriodic environments. Parameters µ σ and µ σ are as in the revious figures (set (a)). Figure shows how the average growth rate deends on the henotyic switching rates and. The arameters µ σ and µ σ, describing the growth rates of each henotye, are fixed; anels (a), (b) and (c) corresond to regimes with increasingly fast environmental switching rates λ and λ. When the environment switches relatively slowly, as in anels (a) and (b), there is a non-trivial henotyic switching strategy which maximises the average growth rate, indicated by a black circle. When the environment switches very uickly, as in anel (c), the otimum strategy is found at the extremes of and, indicating that it is best for the oulation to exress only a single henotye [henotye in the case of Fig. (c)]. These results are in agreement with reviously reorted work in similar systems [7, 6, 6]. In other words, when environmental changes are slow, heterogeneity conveys a fitness advantage, whereas homogeneity is more beneficial in the face of fast environmental changes. This behaviour is further illustrated in Fig. 5(a). Here, we show how the otimum henotyic switching rates vary as we increase the environmental switching rates λ and λ, but keeing their ratio λ /λ fixed, (i.e., the relative amount of time sent in each environment). We identify three regimes. When the environmental switching is slow, the otimum switching strategy is achieved when the henotyic switching rates match the environmental rates: = λ and = λ. For environmental switching in an intermediate regime, the otimum strategy involves henotyic switching rates which are much slower than the environmental switching rates. When the environment switches very uickly, the otimum strategy is not to switch between henotyes at all, as discussed above. Instead, it is best for the cells to kee exressing whichever henotye is better on average. That is, always exress henotye if P µ P µ < P µ P µ, and always henotye otherwise. The growth rate is then P µ P µ, or P µ P µ, resectively. Our analytical findings for the regime of a slow environmental rocess are consistent with revious results by Thattai et al. [6], who studied both eriodic and stochastic environments in numerical simulations. Results for eriodically-switching environments can also be found in Refs. [7, 6]. Kussell and Leibler [3] reort analytical results for randomly switching environments, using an aroach based on maximising the average growth rate by minimising the transient time it takes a oulation to reach its environment-deendent uasi-stationary state. Several uantitative measures of heterogeneity can in rincile be considered. We focus on measuring temoral heterogeneity, i.e., the extent to which both henotyes exist over the course of time. To this end, we use a variation of Simson s diversity index [6], and define heterogeneity as the robability that two cells selected at random from the oulation at two different times are of different henotyes. ssuming that the individuals are samled at two

10 widely sread times, this robability is given by ( ). We have indicated this uantity in Fig. 5(a) and (b). The data confirms that the oulation is heterogeneous for slow environmental switching, but homogeneous when the environmental dynamics are fast. Our calculation of the mean growth rates and comosition of the oulation allows us to address a related uestion concerning the deendence of henotyic heterogeneity and the otimal henotyic switching rate on the ratio of the two environments λ /λ. We summarise our findings here, rather than resent the details. In the regime of very slow environmental switching, the otimum strategy involves henotyic switching for all values of the ratio λ /λ. This is because enough time asses in each environmental eisode for the oulation to reach and benefit from its otimising uasi-stationary distribution. In the intermediate-switching regime, however, if a much longer time is sent on average in one environment than in the other it becomes otimal to exress only the henotye which erforms best in this more freuent environment.. Otimal rate of henotyic switching rate for eriodic environments We erform a arallel study of the model with eriodic environmental switching, described by E. (3). When the environment switches eriodically and the infinite-oulation limit is taken, the dynamical system becomes deterministic. In the long run, the trajectory of (t) then converges to a limit cycle. In environment henotye is favoured; (t) increases until it reaches a turning oint high when the environment switches. Then, in environment henotye is favoured; (t) decreases until turning oint low, at which oint the environment returns to state. It is straightforward to numerically evaluate high and low. s the trajectory tends to a limit cycle, we can comute the stationary distribution, i.e., the of finding the variable (t) at a given oint in the interval [ low, high ] at a randomly chosen time. Further details of the mathematical method can be found in the endix C. comarison between the distributions of for stochastic and eriodic switching environments is resented in Fig.. This figure shows that the domain in which the distribution of is non-zero is larger in the case of stochastic switching than for eriodic environments. The effect is less ronounced for slow switching than for fast switching. When the environment switches stochastically, it is ossible for the duration of each eisode in a fixed state σ to last longer than in the eriodic case. s a result the trajectory of the PDMP can exceed the deterministic extremes low and high of the eriodic case. This difference has significant conseuences for the growth rates and resulting henotyic switching strategies. Figure 3 shows the distribution of growth rates for both cases. s one can see, these distributions are broader for the case of stochastic environments than for eriodic environmental changes, esecially for intermediate and fast switching. Figure (d) (f) shows the numerically comuted average growth rate in eriodically switching environments as a function of henotyic switching rates and (and for given λ, λ ). The growth rates for each and is generally lower than that of the system with stochastic environmental dynamics with the same arameters. Similarly to Figure 5(a), Figure 5(b) shows the otimum henotyic switching strategy, but for the case of a eriodically switching environment. Suerficially, the two figures look similar. In each we identify two extreme regimes of behaviour: (i) the limit of slow environmental switching where the otimum henotyic strategy is to switch with the same rates as the environment, and (ii) the fast environmental regime where homogeneity is referred. In the case of stochastic environmental dynamics, Figure 5(a), an intermediate regime can be detected; this is however largely absent in the case of deterministic eriodic environmental dynamics, where the transition between regimes (i) and (ii) is more abrut [Figure 5(b)]. closer analysis reveals further differences between the cases of stochastic and eriodic environments. For environmental dynamics at intermediate rates, i.e., between regimes (i) and (ii), the otimum henotyic switching rates differ by as much as an order of magnitude between the stochastic and eriodic cases. We also see that the stochastic case favours heterogeneity at much higher switching rates than the eriodic case. Figure 5(c) shows how the average growth rates comare in the cases of a Markovian environment and a eriodically switching environment when the otimum henotyic strategy is in effect. Stochastic switching roduces a growth rate that is universally larger than in the eriodic switching case. We found the same result for all tested arameters and for all values of henotyic switching; similar results were reviously also reorted in Ref. [6]. We note that even small differences in the growth rate can lead to significant differences in oulation size over long times, as the growth is exonential. The exlanation for this enhancement in growth rate is as follows: t any oint in time, the instantaneous growth rate is given by E. (7). The distribution of hence directly translates into a distribution of µ, as illustrated in Figs. and 3. The difference between the growth rates for stochastic and eriodic environments is most ronounced in the intermediate switching regime; the distributions of in this regime are shown Figs. (b) and (e), for the cases of stochastic and eriodic environments resectively. Figs. 3(b) and (e) show the corresonding distributions of growth rates. For eriodic switching the distribution of is largely flat, with only minor eaks near the fixed oints when the flow field in either environment is slow. Conversely, stochastic environmental dynamics ermit long eisodes of

11 .5 λ (b) µ > µ and µ < µ (c) with otimum and.5 λ µ > µ and µ > µ λ (a) λ - - λ - - λ. FIG. 6. (a) Heatma lot of the average growth rate as a function of λ and λ using arameter set (a) as in the revious figures. The henotyic switching rates are =. = =.. For these arameters the lowest growth rate is achieved when ermanently in the stress state, λ =. (b) The same uantity for arameter set (b), where µ > µ and µ < µ. The henotyic switching rates are fixed at =.75, =.5. The growth rate decreases as both switching rates are decreased in roortion. For a given λ there is a non-trivial value of λ which minimises the growth rate, shown here as a white line. It tend towards a straight line a constant ratio which is given by E. (5): λ /λ =.557 resulting in an average growth rate of hµi =.9. (c) Heatma lot for the otimum average growth rate as a function of the environmental switching rates: at each osition on the grah the oulation of cells uses the otimum henotyic switching strategy for the given environmental rocess. Parameters µ σ and µσ as in anel (b). When the switching is sufficiently fast, there is a ratio of environmental switching rates which minimises the cell s best ossible growth rate. This ratio is given by E. (6). time sent in a articular environment. In such long eisodes the variable will mostly reside near the relevant fixed oint, generating the eaked behaviour seen in Fig. (b). Recalling that henotye grows more uickly than in environment and vice versa in environment, we conclude that the system is well-adated to the environmental state at each of these eaks. This leads to an overall enhanced growth rate, relative to the eriodic case. The case of Markovian switching and the case of eriodic switching are two extreme descritions of the environmental rocess; we might exect a real-world environment to change with dynamics somewhere in-between [6]. Our analysis suggests that, for an environment with dynamics between these two extremes, the average growth rate would fall somewhere in-between as well, and similarly for the otimum henotyic switching rate. C. Otimal environmental switching strategy to inhibit oulation growth The environmental switching dynamics can be used to model a host immune resonse [, 6]. y switching between the two states, we assume the host aims to minimise the average growth rate of the oulation. In another alication, the henotyes can be wildtye bacteria and antibiotic-resistant strain, and the two environmental states reresent the absence and resence of antibiotic treatment. The goal is then again to minimise growth, by varying the environmental rotocol. Our analysis allows us to investigate the deendence of the growth rate on environmental switching rates λ and λ. In this Section we investigate which environmental switching strategy best inhibits the growth of the oulation. The figures shown here are for stochastic environmental switching, however we found ualitatively similar results for eriodic environments. Figure 6(a) and (b) show how the average growth rate changes if we vary the environmental switching rates for fixed henotyic switching rates and. Each anel reresents a different set of arameters. The case shown in Figure 6(a) is a reresentation of bacteria in which both henotyes are disadvantaged by the resence of an antibacterial agent, but in which the ersister henotye is affected less by the external stress than the normal henotye. Mathematically one has µ > µ, µ > µ, µ > µ, and µ > µ. Secifically, we use the arameters set used in revious studies [], and we will refer to this as arameter set (a) in the text which follows; the numerical values can be found in the endix. Unsurrisingly, the average growth rate is minimised when the environment stays in environmental state (i.e., λ = ), which can be seen as reresenting the antibiotic state. nother trend shown by our results is that roortionally increasing both environmental switching rates universally decreases the average growth rate. That is, roortionally faster switching rates better inhibit the growth of a oulation. This result has been found for all tested arameters, suggesting that the best way aly antibiotics may involve many short eriods of antibiotic treatment. The arameters used in Fig. 6(b) differ from those in Fig. 6(a) in that here µ > µ and µ < µ. We use the arameters of Ref. [3], and will refer to this as arameter set (b) in the text which follows; numerical values are again given in endix. Here, henotye erforms better in environment than in environment, and henotye

12 erforms better in environment in environment. In this case, the otimum environmental strategy involves non-trivial switching. The white line in anel Fig. 6(b) indicates the otimum choice of λ, for given λ. When both λ and λ are large growth is minimised for a non-trivial ratio, λ /λ. The limit of infinitely fast environmental switching can be characterised analytically. In this limit, the dynamics can be modelled using ordinary differential euations, using effective mean birth and death rates [63, 6]. Minimising the growth rate in this limit leads to an euation describing the otimum environmental strategy to inhibit the growth of the oulation. The minimal growth rate is found when P, the robability of being in environmental state, is given by P = ( ) / (µ µ )(µ µ ). (5) The argument of the suare root indicates a necessary condition (u to relabelling) for the existence of this otimum, µ > µ and µ < µ. This exlains why we have no non-trivial minimum in the fast-switching limit for the arameters used in Fig. 6(a), but there is one for the arameters used in Fig. 6(b). The analytical otimum given by E. (5) agrees with the results shown in Fig. 6(b). D. Co-evolution of host and athogen The interlay between the growth-maximising strategy of the cells and the growth-minimising strategy of the environment rovides an interesting scenario. Given a set of environmental switching rates, there can be henotyic switching rates which maximise the average growth rate. Conversely, for given henotyic switching rates, there can be non-trivial environmental switching rates which minimise the growth rate. This suggests the ossibility of a combined set of rates, from which the growth rate cannot be increased by unilaterally changing and (the strategy of the oulation), and from which the growth rate cannot be decreased by unilaterally changing λ and λ (the strategy of the hosting environment). From a game-theoretic ersective, neither layer of the game (the oulation of cells and the immune system) can imrove their erformance by changing their strategy. This can be viewed as a Nash euilibrium [37]. For a given set of model arameters, µ σ, µ σ and given environmental switching rates, otimal henotyic switching rates can be found by evaluating E. () for different values of and, and then identifying the maximum. This maximum can then be studied as a function of λ and λ. For arameter set (a) the evolutionarily otimising strategy is found to be trivial: the environment stays in the stress state and the cell exresses the ersister henotye only, leading to an overall growth rate µ. For arameter set (b), however, the results are less trivial, since each henotyes refers a different environment. Figure 6(c) shows the average growth rate given otimal henotyic switching in this case. This figure was constructed as follows: for each combination of λ and λ we found the values of and maximising the growth rate. The value of this maximum is then lotted in the figure. For each value of λ the white line in the figure deicts the value of λ, which minimises this maximum achievable growth rate; from the oint of view of the hosting environment this line shows the best choice of λ for each λ. s before, the growth rate decreases as the environmental switching rates are increased roortionally. Hence, the environment s best strategy is to switch states uickly, and with a choice of λ /λ so that switching rates fall on the white line in Figure 6(c). s discussed in earlier sections the maximising henotyic strategy in the limit of fast environments leads to homogeneity, i.e., it is best for the oulation to only exress the henotye does better in a (weighted) average over both environmental states. This gives a growth rate which is the greater of P µ P µ and P µ P µ. Insecting these exressions, one finds that changing the ratio of λ and λ increases one of the uantities and decreases the other. It follows that the greater of the two growth rates is minimised when both growth rates are eual. This the case when λ λ =. (6) This gives us the roortion of each environmental state which minimises the average growth rate in the limit of fast environmental switching, given an otimally switching oulation. This minimum otimum growth rate is given by µ µ µ µ µ = µ µ µ. (7) µ The henotyic switching strategy of the cell cannot change to increase the growth rate. Similarly, if the environment assumes that the cell oulation will otimise its henotyic switching strategy, the environment cannot move to