Demographic noise slows down cycles of dominance in ecological models

J. Math. Biol manuscript No. (will e inserted y the editor) Demographic noise slows down cycles of dominance in ecological models Qian Yang Tim Rogers Jonathan H.P. Dawes Received: date / Accepted: date Astract We study the phenomenon of cyclic dominance in the paradigmatic Rock Paper Scissors model, as occuring in oth stochastic individual-ased models of finite populations and in the deterministic replicator equations. The meanfield replicator equations are valid in the limit of large populations and, in the presence of mutation and unalanced payoffs, they exhiit an attracting limit cycle. The period of this cycle depends on the rate of mutation; specifically, the period grows logarithmically as the mutation rate tends to zero. However, this ehaviour is not reproduced in stochastic simulations with a fixed finite population size. Instead, demographic noise present in the individual-ased model dramatically slows down the progress of the limit cycle, with the typical period growing as the reciprocal of the mutation rate. Here we develop a theory that explains these scaling regimes and delineates them in terms of population size and mutation rate. We identify a further intermediate regime in which we construct a stochastic differential equation model descriing the transition etween stochastically-dominated and mean-field ehaviour. Keywords cyclic dominance ecology limit cycle mean field model replicator equation stochastic differential equation stochastic simulation PACS 87.23.Cc 87.0.Ed 89.75.-k Introduction Many mathematical models in ecology are well-known to e capale of generating oscillatory dynamics in time; important examples stretch right ack to the initial work of Lotka and Volterra on predator-prey interactions [3, 6,, 3, 7, 23]. Such models, although dramatic simplifications when compared to real iological Q. Yian, T. Rogers, and J.H.P. Dawes Centre for Networks and Collective Behaviour, and Department of Mathematical Sciences University of Bath Bath BA2 7AY, UK E-mail: Q.Yang2@ath.ac.uk E-mail: T.C.Rogers@ath.ac.uk E-mail: J.H.P.Dawes@ath.ac.uk

2 Qian Yang et al. systems, have a significant impact in shaping our understanding of the modes of response of ecological systems and are helpful in understanding implications of different strategies for, for example, iodiversity management, and the structure of food wes [9]. Competition etween species is a key driver of complex dynamics in ecological models. Even very simple competitive interactions can yield complex dynamical ehaviour, for example the well documented example of the different strategies adopted y three distinct kinds of side-lotched lizard [2]. Similar cyclical interactions occur in acterial colonies of competing strains of E. coli [9,0,26]. Evolutionary Game Theory (EGT) provides a useful framework for modelling competitive interaction, in particular the replicator equations [22, 20] give a dynamical systems interpretation for models posed in game-theoretic language. Work y many authors, including in particular Hofauer and Sigmund [8, 7] has resulted in a very good understanding of replicator equation models for competing species. Recent work has extended these deterministic approaches to consider stochastic effects that emerge from consideration of finite, rather than infinite, populations. The classic Rock Paper Scissors (RPS) provides an important example of stochastic phenomena in ecological dynamics. When mutation (allowing individuals to spontaneous swap strategies) is added to the replicator equations for the RPS game, the deterministic can exhiit damped oscillations that converge to a fixed point. In ref. [5], it was shown that stochastic effects present in finite populations cause an amplification of these transient oscillations, leading to so-called quasi-cycles [4]. For smaller values of mutation rate, the deterministic system passes through a Hopf ifurcation, and a limit cycles appears. Some past studies exist on the role of noise around limit cycles, such as [,2], in which small-scale fluctuations around the mean-field equations are explored using Floquet theory. More recently, it has een discovered that noise an induce much stronger effects including counterrotation and istaility [6]. In this paper we comine deterministic and stochastic approaches in order to present a complete description of the effect of demographic fluctuations around cycles of dominance in the RPS model. We determine three regimes, depending on the scaling of population size N and mutation rate µ. The asic link etween stochastic individual-ased dynamics and population-level ODEs is a theorem of Kurtz [2], allowing us to construct a self-consistent set of individual-level ehaviours corresponding to the mean-field replicator dynamics for the RPS model that we take as ourt starting point. Between these two views of the dynamics lies a third: the construction of a stochastic differential equation (SDE) that captures the transition etween them. Through the SDE, and the use of Ito s formula to compute an SDE for an angle variale descriing the evolution of trajectories around the limit cycle, we are ale to compute the effect of stochastic ehaviour on the limit cycle in the ODE prolem. This reveals that the contriution of the stochasticity is quantiatively to speed up some parts of the phase space dynamics and to slow down others ut that the overall effect is to markedly increase the oscillation period. Our central conclusion is that as the stochastic effects ecome more important, the period of the oscillations increases rapidly, and this slowing down is a significant departure from the prediction of oscillation periods made on the asis of the mean-field ODE model. The structure of the remainder of the paper is as follows. In section 2 we introduce the replicator dynamical model for the rock-paper-scissors game with

Demographic noise slows down cycles of dominance 3 mutation. The mean-field ODE version of the model is well-known and we derive a self-consistent individual-ased description; this is not as straightforward as one might initially imagine. We show numerically that the two models give the same mean period for the cyclic dynamics when the mutation rate is large ut disagree when µ is small. In section 3 we summarise the computation to estimate the period of the limit cycle when µ is small. This follows the usual approach, dividing up trajectories into local ehaviour near equilirium points, and gloal maps valid near the unstale manifolds of these saddle points. Section 4 turns to the stochastic population model and analyses the dynamics in terms of a Markov chain. This leads to a detailed understanding of the individual-level ehaviour in the limit of small mutation rate µ. Section 5 then fills the gap etween the analyses of sections 3 and 4 y deriving an SDE that allows us to understand the relative contriutions of the stochastic ehaviour and the deterministic parts in an intermediate regime. Finally, section 6 discusses our results and concludes. 2 Models for Rock-Paper-Scissors with mutation 2. Rock-paper-scissors with mutation x x y : x eats y y : x mutates into y B(Paper) C(Scissors) A(Rock) Fig. Illustration of the Rock Paper Scissors interaction in the presence of mutations. Solid (lue) arrows indicate cases in which the state at the tail of the arrow wins over the state at the head of the arrow. Thin (red) arrows indicate possile mutations etween states. Rock Paper Scissors (RPS) is a simple two-player, three-state game which illustrates the idea of cyclic dominance: a collection of strategies, or unchanging system states, in which each state in turn is unstale to the next in the cycle. In mathematical neuroscience dynamical switches etween such a collection of system states has een referred to as winnerless competition since there is no est-performing state overall [8, 24]. In detail: playing the strategy rock eats the strategy scissors ut loses to the strategy paper ; similarly, scissors eats paper ut loses to rock. When the two players play the same strategy the contest is a draw.

4 Qian Yang et al. In game theory, the usual way to summarise this information is via a payoff matrix that specifies the payoff to one player from all possile cominations of actions y oth players. In the two-player RPS game, we define the payoff matrix P y setting the entry (P ij ) to e the payoff to player when player plays strategy i and player 2 plays strategy j. For the RPS game we summarise the payoffs in the following tale and payoff matrix P : A B C A 0 β B β 0 C β 0 0 β P := β 0 () β 0 where β 0 is a parameter that indicates that the loss incurred in losing contests is greater than the payoff gained from winning them. When β = 0, the row and column sums of P are zero: this is the simplest case. When β > 0, the game ecomes more complicated, particularly when we would like to relate the ehaviour at the population level to the individual level, as we discuss later in sections 2.2 and 2.3. 2.2 Deterministic rate equations Setting the RPS game in the context of Evolutionary Game Theory (EGT), one considers a large well-mixed population of N players playing the game against opponents drawn uniformly at random from the whole population. For the population as a whole, we are then interested in the proportions of the total population who are playing different strategies at future times. The state of the whole population is given y the population fractions (x a (t), x (t), x c (t)) := (N A (t), N B (t), N C (t))/n where N A,B,C (t) are the numers of players playing strategies A, B and C respectively. In the mean field limit of an infinite population, the proportions of the population playing each strategy can e expected to evolve in time according to the payoffs P ij, comparing the payoff due to playing each strategy on its own compared to the average payoff over all strategies. The simplest model for this dynamical evolution of the population state as descried y (x a (t), x (t), x c (t)) are the replicator equations x i = x i (t) j P ij x j (t) j,k P jk x j (t)x k (t), (2) where the suscripts i, j {a,, c} and the proportions x i sum to unity. Note that the term j P ijx j (t) represents the payoff earned y strategy i at time t, and the term j,k P jkx j (t)x k (t) (which is independent of strategy i) represents the average payoff of the whole system. Hence, when the payoff earned y strategy i is larger than the average, the proportion of players who play strategy i will increase, and when the payoff is less than the system average the proportion of players will reduce. Another common variant of the model introduces the additional mechanism of mutation etween the three strategies, occurring etween any pair with equal

Demographic noise slows down cycles of dominance 5 frequency. Mutation affects the rate of change of strategy i over time since the strategies other than i will contriute new players of i at rates µ while i will lose players at a rate given y 2µx i as these players switch to a different strategy. The comined effects of the replicator dynamics together with mutations etween strategies gives rise to the model equations (for an infinitely large population) ẋ a = x a [x c ( + β)x + β(x a x + x x c + x a x c )] + µ(x + x c 2x a ), ẋ = x [x a ( + β)x c + β(x a x + x x c + x a x c )] + µ(x a + x c 2x ), ẋ c = x c [x ( + β)x a + β(x a x + x x c + x a x c )] + µ(x a + x 2x c ). (3) We note that these equations are to e solved in the region of R 3 where all coordinates are non-negative. This region is clearly invariant under the vector field (3). Moreover, the constraint x A + x B + x C = is required to hold at all times. We can now proceed to carry out standard investigations of the dynamics of (3). We oserve that there is only one interior equilirium point, x = (/3, /3, /3). To investigate staility of this equilirium point we write x c = x a x, sustitute this into (3) and then look at the resulting 2-dimensional system for x a, x ). The Jacoian matrix at x can e easily calculated to e ( J = 3 3µ 2 3 3 β 2 3 + 3 β 3 + 3 β 3µ The eigenvalues of J are λ ± = λ ± iω, with λ = β 6 3µ and ω = 2 β + 3 3. If λ < 0, then the equilirium point is linearly stale and there are no periodic orits. In the case λ > 0, the equilirium point x is unstale; in terms of the parameters the critical value of µ is µ c = β 8, and x is unstale when µ < β 8 which is the parameter regime we will focus on. Previous work y Moilia [5] has shown that the system undergoes a Hopf ifurcation at µ = µ c and that for µ < µ c trajectories of (3) spiral out away from x and are attracted to a unique periodic orit which is stale, i.e a limit cycle. This ehaviour is illustrated in fig.2. In this and susequent figures we represent the 2-dimensional phase space using the coordinates (y, y 2 ) defined to e y := x a + 2 x, y 2 := 3 2 x. ). 2.3 Stochastic chemical reactions In section 2.2 we discussed a deterministic, population-level model for the RPS game, ased on the replicator equations. In a finite population, the ehaviour of many competing individuals is more appropriately modelled using a set of stochastic transitions etween these states of individuals: essentially a scheme of chemical reactions. In the limit of an infinitely large population, one would expect to recover the dynamics of the deterministic ODEs (3) descriing the system ehaviour. In this section we explore the dynamics of a simple stochastic model that gives rise to the mean-field ODEs (3) in the limit of large population sizes. It is interesting to note that different stochastic individual-ased models may give rise to the same mean-field ODEs, so that the question of constructing a stochastic reaction scheme starting from a particular set of ODEs does not have a unique answer. Moreover, the construction of the stochastic model is suject to a numer

6 Qian Yang et al. 0.9 0.8 ODEs result 0.7 0.6 0.5 y 2 0.4 0.3 0.2 0. 0 0 0.2 0.4 0.6 0.8 y Fig. 2 A typical trajectory of the ODEs (3) in the (y, y 2 ) plane, showing spiralling outwards from the equilirium point x and convergence to the limit cycle. Parameter values: β = 0.5, µ = 26 < µc = 36. of natural constraints, most importantly that all reaction rates are at all times non-negative. With β = µ = 0 the following set of stochastic reactions is the simplest that corresponds to these ODEs in the mean-field limit: A + B B + B, B + C C + C, C + A A + A. (4) This information can e summarised in terms of an integer jump matrix S descriing the changes to x caused y the various reactions (so element S ij represents the increment or decrement of species i taking place in reaction j), and vector r(x) descriing the reaction rates. For the scheme in (4) we have 0 S = 0, r = (x a x, x x c, x c x a ) T. (5) 0 A theorem of Kurtz[2] tells us that in the limit N of large populations, the stochastic process descried y these reactions converges to the deterministic dynamical system ẋ = Sr(x). It is easy to check that the aove scheme thus reproduces the equations (3) with β = µ = 0.

Demographic noise slows down cycles of dominance 7 When β > 0 the ODEs contain additional cuic terms, and the chemical reaction rules ecome more complicated. A reaction scheme was proposed in [5] that provides reasonale results in the neighourhood of the interior fixed point x, however, this scheme contained rates that can take negative (unphysical) values near the system oundaries. Our first task here, therefore, is to propose a consistent reaction scheme that permits us to study the whole of state space. Following the argument aove, we seek to express the ODEs (3) with β > 0 and µ = 0 in the form ẋ = Sr(x). After careful analysis we otain the (minimal) jump matrix S and reaction rate vector r as follows: 0 0 S = 0 0, (6) 0 0 r = (x a x, x x c, x c x a, βx a x 2, βx 2 ax c, βx x 2 c) T. (7) This jump matrix and reaction rate vector corresponds then to chemical reactions that can e written explicitly in the form: A + B B + B, B + C C + C, C + A A + A, (8) A + B + B β B + B + B, (9) A + A + C β A + A + A, (0) B + C + C β C + C + C, () where the reaction rates are unity and β, oth non-negative. In addition, mutations can e included in the model through the additional six reactions A µ B, B µ C, C µ A, A µ C, B µ A, C µ B. (2) A complete reaction scheme valid in the case β > 0 in which the payoff matrix is not zero-sum can therefore e otained y comining the 2 individual reactions written out in (8)-(2). 2.4 Simulations To gain an initial insight into the differences etween the deterministic and stochastic viewpoints for the RPS game with mutation,we use the Gillespie algorithm, the standard and widely used stochastic simulation algorithm (SSA), to simulate the chemical reactions (8)-(2) in order to illustrate the typical dynamics in the stochastic case and to compare that with the deterministic case. Figure 3 presents results comparing the stochastic and deterministic cases for two different values of the mutation rate µ. In each plot we show a typical realisation of the stochastic simulation algorithm, for two different finite, ut large, population sizes N = 2 8 (green dashed line) and N = 2 6 (red dashed line), together with a trajectory of the ODEs (lue solid line). The solution to the ODEs shown y the lue line in 3(a) is the same as that shown in figure 2. The red dashed line in 3(a) starts from a very similar initial

8 Qian Yang et al. condition and evolves similarly: spiralling out towards the oundary of phase space and at large times occupying a region of phase space near to the limit cycle that is visile in 2 ut with small fluctuations around it. The green dashed line in 3(a) shows much larger fluctuations around the limit cycle, including excursions that take the simulation onto the oundaries of the phase space, and much closer to the corners. Note that ecause of the mutations, the oundaries of the phase space are not asoring states for the random process (or invariant lines for the ODE dynamics). Figure 3() illustrates the ehaviour for a significantly larger value of µ for which the limit cycle for the ODEs (lue curve) lies much closer to the centre of the phase space. The SSA for N = 2 6 lies close to the limit cycle ut fluctuates around it; for N = 2 8 the fluctuations are much larger and the sample path of the stochastic process lies outside the limit cycle for a large proportion of the simulation time. To quantify the differences etween the three results shown in each part of figure 3, we focus on one specific aspect of the dynamics: the period of the oscillations around the central equilirium point at (/3, /3, /3). For the ODEs, the period of the limit cycle can e defined to e the smallest elapsed time etween successive crossings of a hyperplane in the same direction, for example the plane x a = 2. We denote y T ODE the period of the deterministic limit cycle. In stochastic simulations, a trajectory may y chance cross a hyperplane ack and forth several times in quick succession, thus the time etween crossings may not represent a full transit of the cycle. We avoid this complication y measuring the period as three times the transit time etween a hyperplane and its 2π/3 rotation. The expected value of this oscillation period in the stochastic case we denote y T SSA. y2 y2.0 SSAresultofN=2 8.0 SSAresultofN=2 8 SSAresultofN=2 SSAresultofN=2 0.8 ODEs result 0.8 ODEs result 0.6 0.6 0.4 0.4 0.2 0.2 0.2 0.4 0.6 0.8.0 (a) y 0.2 0.4 0.6 0.8.0 () y Fig. 3 Illustrative comparisons etween trajectories of the ODEs (lue solid lines) and realisations of the stochastic simulations (red and green dashed lines). Unalanced means that the payoff matrix is not zero-sum, i.e. β > 0. (a) When µ µ c the trajectories lie close to the oundary of the phase space, β = 0.5, µ = 26. () When µ is only slightly smaller than µc, trajectories lie much closer to the central equilirium point, β = 0.5, µ = 5 98.

Demographic noise slows down cycles of dominance 9 Figure 4 provides a quantitative comparison of the different dependencies of the average period T SSA of the stochastic simulations and the period T ODE of the deterministic ODEs on the mutation rate µ, for 0 < µ < µ c. The lue solid line indicates the relatively slow increase in ther period T ODE as µ decreases. The red error ars indicate the range of values of the oscillation period in the stochastic case, with the averages of those values shown y the red dots. The data in this figure for the stochastic simulations was otained from simulations at a fixed value N = 2 7 with µ varying, ut for presentational reasons that will ecome clearer later in the paper we have chosen to plot the scaled quantity µn ln N on the horizontal axis. We then oserve that if µ is sufficiently small then the difference etween T ODE and T SSA is very significant: the oscillations in the stochastic simulation have a much longer period, on average, than that predicted y the ODEs, while if µn ln N is larger than unity, the agreement in terms of the oscillation period, etween the deterministic and stochastic simulations is very good. We lael these two regimes Region III and Region I respectively. The cross-over region where µn ln N we lael Region II. 0 5 T 0 4 000 ODE Simulation N=3072 00 0 0.00 000 0 6 μnlnn Fig. 4 Comparison etween the period T ODE of the limit cycle in the deterministic case and the average period T SSA of oscillations in the stochastic case, as a function of the parameter µ at fixed N = 2 7. The lue line shows T ODE and red dots indicate T SSA. As µ decreases, the two periods start to separate from each other in Region II where µn log N. In the following sections of the paper we will focus on each of the three regions in turn. In section 3 we study the period of the limit cycle in the ODEs (region I). In section 4 we analyse the stochastic dynamics to determine the average period of cycles in region III. In section 5 the cross-over etween the deterministic and stochastic regions is understood through the analysis of an SDE that comines oth deterministic and stochastic effects, thus descriing region II.

0 Qian Yang et al. 3 Analysis of the periodic orit in region I Region I is defined to e the right hand side of figure 4; more precisely, it is the regime in which N and N log N µ < µ c. In this region, stochastic simulation results show only very small fluctuations aout a mean value, and this mean value coincides well with numerical solutions to the ODEs. This is evidence that the ehaviour of the deterministic ODEs provides a very good guide to the stochastic simulation results in this region. Hence our goal in this section is to explain the asymptotic result that the period T ODE 3 log µ for µ small, ut for large, even infinite, N as shown in figure 5. 30 20 0 ODE Simulation N=3072 Simulation N=32768 Simulation N=892 0-7 0-4 μ Fig. 5 Period T SSA for three different values of N, compared with the result for T ODE (lue solid line) on a log-linear plot indicating the scaling law T ODE 3 log µ which applies in region I where the stochastic simulations ehave in a similar way to the ODEs. The traditional approach to the analysis of trajectories near the limit cycle, in the regime where it lies close to the corners of the phase space, is to construct local maps that analyse the flow near the corners, and gloal maps that approximate the ehaviour of trajectories close to the oundaries. Figure 6 shows the construction of a local map in the neighourhood of the corner where x c =, following y the gloal map from this neighourhood to a neighourhood of the corner where x a =. In this section we consider local and gloal maps in turn, in sections 3. and 3.2, and then in section 3.2 we study the composition of local and gloal maps, and deduce an estimate for the period t ODE of the limit cycle. 3. Local map Using the notation as in figure 6, we egin y defining a neighourhood of the corner x c = y setting 0 < h to e a small positive constant. We assume that the trajectory for the ODEs (3) starts at the point (x (0) a, h, x (0) c ) at time

Demographic noise slows down cycles of dominance (0,,0) gloal map (0,0,) local map (,0,0) Fig. 6 This picture shows the local map and the gloal map starting from initial position (x 0 a, h, x 0 c). t = 0, and arrives at the point (h, x (), x () c ) at time t = T > 0. For the whole time 0 < t < T the trajectory lies close to the corner (0, 0, ), so we suppose that 0 < x a, x, u where u := x c. Note that we have x a + x u = 0 since x a + x + x c =. Then the ehaviour of the ODEs (3) in this neighourhood is very similar to that of their linearisation otained y dropping terms higher than linear order in the small quantitites 0 < x a, x, u. For the linearised system we otain ẋ a = µ x a (3µ ), ẋ = µ x ( + β + 3µ), u = x ( + β) + x a + 2µ 3µu. (3) The equations (3) are linear and constant coefficient and so can e solved analytically; note that the first and second equation are actually decoupled from each other and from the u equation. Integrating from t = 0 to the time t = T, we denote the point on the trajectory y (x a (T ), x (T ), x c (T )) (x () a = h, x (), x () c ) where x () = µ γ ( ) e γt x () a = h = µˆγ + e ˆγT + he γt, (4) ( x 0 a µˆγ ) (5)

2 Qian Yang et al. where γ := + β + 3µ and ˆγ := 3µ. Equation (5) allows us to express T in terms of x (0) a : ( ) T = ˆγ log h µ/ˆγ, (6) µ/ˆγ and we can now use (6) to eliminate T from (4) and otain a relationship etween x (0) a and x () which takes the form x (0) a x () = µ (h γ + µ ) ( h µ/ˆγ γ µ/ˆγ x (0) a ) γ/ˆγ. (7) This relationship is the key part of the local map near the point (x a, x, x c ) = (0, 0, ) that we will use in what follows. 3.2 Gloal map For the gloal map, we oserve that trajectories remain close to one of the oundaries (in this case, the oundary x = 0) and so we propose that the trajectory starting from (x () a = h, x (), x () c ) arrives at the point (x (2) a, x (2), h) at time t = T 2. Referring to (3), the ODE for x near the oundary can e well approximated y taking just ẋ = µ (since x a + x c = when x = 0), so its solution is x (2) = x () + (T 2 T )µ = x () + C 0 µ, (8) where we let the elapsed time taken y C 0 := T 2 T. As is typical in these analyses, trajectories take a relatively short time to arrive at the hyperplane x (2) c = h starting from x () c, compared to the time taken to move along the part of the trajectory from x (0) c to x () c ; this is intuitively ecause the asolute value of ẋ c on the gloal part of the map etween T and T 2 is much larger than on the local part, i.e. when 0 < t < T. As a result, the time taken on the gloal part of the map, C 0 := T 2 T is much less than the local travel time T ; the majority of the time spent on the limit cycle is taken up with travel near the corners. The composition of local and gloal maps near the corner x c = and oundary x = 0 is now straightforward: we comine (7) and (8) to otain: x (2) = C 0 µ + µ (h γ + µ ) ( h µ/ˆγ γ µ/ˆγ x (0) a ) γ/ˆγ. (9) We can now use the permutation symmetry inherent in the dynamics to complete the analysis. Due to the fact that the model is rotationally symmetric, the next stage of the evolution is local map again with the same parameter values. The trajectory will start from (x (2) a, x (2), h) and stay near the corner (, 0, 0) for long time efore arriving at a point, say, (x (3) a, h, x (3) c ) where we will construct another gloal map, and so on: x (0) a local x () gloal x (2) local x (3) c gloal x (4) c local x (5) a gloal x (6) a (20)

Demographic noise slows down cycles of dominance 3 which can e summarised further as x 0 local & gloal x local & gloal x 2 local & gloal x 3 (2) Equations (20) and (2) define a one-to-one correspondence etween the points {(x (n) a, x (n), x (n) c )} on a trajectory and a sequence {x n }. From the previous discussion on local and gloal maps, the map that generates the sequence {x n } takes the form x n+ = C 0 µ + µ ( γ + h µ ) ( ) γ/ˆγ h µ/ˆγ, (22) γ x n µ/ˆγ where, as efore, γ := + β + 3µ and ˆγ := 3µ. If iterates of the map (22) converge to a fixed point then this corresponds to a stale limit cycle for the ODE dynamics. We now estimate the location of this fixed point and deduce an estimate for the period of the resulting limit cycle. Let y n := x n /µ e a scaled version of x n, then (22) can e written as: y n+ = C 0 + ( γ + h µ ) γ µ = C 0 + ( γ + h µ ) µ +3µ+β γ ( µyn + µ ) +3µ+β 3µ 3µ h + µ 3µ ( yn ( 3µ) + 3µ h( 3µ) + µ ) +3µ+β 3µ, (23) where γ := + 3µ + β. Denoting the fixed point of the map y y, from (23) we see that y = C 0 + γ + o(µ) in the limit µ 0. This oservation is crucial in order to ensure that we otain the correct leading-order ehaviour and distinguish carefully etween the various small quantities in the prolem. Then it follows that x = µy C µ, where C = C 0 + +β, in the limit µ 0. Introducing this leading-order approximation for x into (6), we otain an estimate of the time spent in a neighourhood of the corner T of the stale limit cycle: ( ) h + µ( 3µ) T = 3µ log. (24) C µ + µ( 3µ) Since in this case µ is assumed to e very small, (24) takes the form, at leading order, ( ) T = 3µ log + B 0 as µ 0. (25) µ where B 0 is a constant. Finally, as remarked on aove, ecause trajectories remain near each corner for large parts of the period of the orit, the local map travel time T is the dominant contriution, compared to the time spent on the gloal map. Hence the period of the limit cycle is given at leading order y considering only the contriutiosn from the three local maps required in one full period of the limit cycle. Hence our estimate for T ODE ecomes as µ 0. T ODE 3T = 3 log µ + B, (26)

4 Qian Yang et al. 4 Analysis of the periodic orit in region III We now turn our attention to Region III, where the period of the orit in the stochastic simulations increases much more rapidly as µ decreases, at fixed finite N, than the prediction from the analysis of the ODEs in section 3 aove suggests. Figure 7 illustrates this y plotting the period T SSA as a function of µn ln N for three different values of N. By plotting, on a log-log scale, the mean values of the periods and the error ars from an ensemle of stochastic simulations we oserve that the period T SSA (µn ln N) for small µ, with a constant that does not demonstrate any systematic dependence on N. In fact, the range of values of N presented here is small: we cannot distinguish from these numerical results the precise form of the dependence on N. T 0 5 0 4 000 00 ODE Simulation N=3072 Simulation N=32768 Simulation N=892 0 0.00 000 0 6 μnlnn Fig. 7 Log-log plot of the period T SSA as a function of the mutation rate µ for three different values of N, showing that in region III the mean period scales roughly as T SSA (µn ln N). As well as the dependence of the period T SSA on µ it is also of interest to determine the extent of Region III in which this scaling ehaviour applies; in other words how small is µ required to e in order to move into this regime? The numerical data in figure 7 indicate that the cross-over from the ODE result to this new stochastic scaling arises when µn log N =. In this section, then, our aim is to explain firstly why this new scaling regime in Region III exists, and secondly, why it extends as far as µn ln N =. We will find that in fact the period should scale as T SSA (µn) for small µ, ut that the oservation that the cross-over occurs when µn ln N = is correct; this explains why we have chosen to plot figure 7 in the form that it appears. Careful examination of the numerical simulations in this regime show that their ehaviour is qualitatively different from that in Region I discussed aove. In stochastic simulations the system ecomes strongly attracted to the corner states (which would e asoring states in the asence of mutation) and then can only escape from a corner when a mutation occurs; mutations are rare when

Demographic noise slows down cycles of dominance 5 µ is small. Our analysis later in this section shows that the oscillation time is dominated y the contriution from the time needed to escape one step from a corner. Trajectories then typically move along a oundary towards the next corner. We show elow that, although this latter part requires at least N steps, the expected total time required is less than the waiting time to escape from the corner. Our schematic approach is illustrated in figure 8 whih sketches this separation into a first step away from a corner, following y movement along the adjoining oundary. Fig. 8 Sketch of the dynamics in the small-µ stochastic limit. The system is strongly attracted to corner states where the population is all in one state. Mutation is then the only mechanism for escape, and the expected time for the first mutation (curved arrows) is longer than the expected time required for susequent steps along a oundary towards the next corner (straight lines with arrows). In the following susections we consider in detail three issues: firstly, in section 4. we show that the proaility that the motion along a oundary is towards the next corner in the sequence, and that the systems state hits this next corner with a proaility that tends to as N. Since we are interested in the regime where the mutation rate is very small, we carry out this calculation in the limit µ = 0. In section 4.2 we compute the expected time until the system hits this next corner, again setting µ = 0. Finally, in section 4.3 we compare this expected time for the system state to evolve along the oundary with the expected time until a mutation occurs. Together, this analysis confirms the intuitive picture outlined in the previous paragraph.

6 Qian Yang et al. 4. Proaility of hitting the asored status In this section we consider the Markov Chain dynamics of the system evolving along a oundary, ignoring the effect of mutation. In this case the system is a one dimensional chain of N + states with transition proailities p i and q i defined diagramatically as follows: p q p 2 q 2 p 3 q N 2 p q 0 2 N N The two end states laelled 0 and N are asoring states, since we assume here that µ = 0. Let h i = P i (hit 0) e the proaility that the system hits (and is therefore asored y) state 0 having started at node i, and similarly let h i = P i (hit N) e the proaility of hitting state N having started at node i. When the system is in state i there are two possile moves: jumps to the left or to the right. The rate at which jumps to the left occur is p i = (N i)i2 N ; the rate 2 at which jumps to the right occur is q i = (N i)i2 N ( + β) + (N i)i 2 N. The transition proailities of moving to the left and to the right are then l i = p i p i +q i and r i = q i p i +q i ; clearly l i + r i =. We remark that the ratio of transition proailities can e simplified to e l i r i = p i q i = = (N i)i 2 /N 2 (N i)i 2 ( + β)/n 2 + (N i)i/n i i( + β) + N = ( + β) + N. i (27) This allows us to derive a recurrence relation for the proaility h i that the system hits the state 0 starting from state i: h 0 =, h i = l i h i + r i h i+, for i =... N, (28) h N = 0. From the recurrence formulas (28), and using r i + l i = we have the relation r i (h i h i+ ) = l i (h i h i ) which implies that h i h i+ = l i r i (h i h i ) = l il i r i r i (h i 2 h i ) = = l il i... l r i r i... r (h 0 h ) =: γ i (h 0 h ), defining the ratio γ i for i N, where we also define γ 0 =, o that we can write the recurrence relation etween the h i as h i h i+ = γ i (h 0 h ). (29)

Demographic noise slows down cycles of dominance 7 We note that the following inequality, ounding γ i, will e very useful: γ i = i j= l j r j = i j= j j( + β) + N < i! N i. Summing (29) from i = 0 to i = N enales us now to compute h in terms of h 0 and h N : h 0 h N = (h i h i+ ) i=0 = ( + γ + γ 2 + + γ )(h 0 h ). Finally, h can e calculated directly since h 0 = and h N = 0: h = γ i + γ. (30) i Similarly, h ( = h = +. i) γ We now examine the limiting ehaviour of this result and prove that γ i 0 as N. Since 0 < γ i+ = p i+ γ i, q i+ and p i+ q i+ <, it follows that γ i+ < γ i. Also, since γ i < i! N i, we conclude that 0 < γ i = γ + N which clearly tends to zero as N. Hence lim N i=2 γ i + (N 2) 2 N 2, γ i = 0. (3) In conclusion, we have shown that, given that it starts at state, the proaility of the system hitting the asoring state at N tends to as N. 4.2 Average hitting time Having shown that the system reaches state N with high proaility, we now consider the expected time until this happens. Let T i e is the first time at which the system hits an asoring state starting from state i at time 0, i.e. the time at which the system hits either 0 or N. Define τ i = E(T i ) for 0 i N, the expected hitting time starting from state i. Clearly we have that τ 0 = E(T 0 ) = 0 and τ N = E(T N ) = 0. Through a similar calculation to setion 4., we now compute τ, the mean time taken to reach either 0 or N, starting from.

8 Qian Yang et al. The recurrence formula for expected hitting times τ i can e computed as the sum of the expected time spent in state i efore jumping either left or right, plus the expected future time required when in that new state: τ i = p i + q i + l i τ i + r i τ i+, for i =... N. (32) This recurrence relation can e rearranged to give (p i + q i )τ i = p i τ i + q i τ i+ + = q i (τ i τ i+ ) = p i (τ i τ i ) + = τ i τ i+ = p i q i (τ i τ i ) + q i (33) Repeated sustitution of the term τ i τ i in (33) gives τ i τ i+ = p ( i pi (τ i 2 τ i ) + ) + q i q i q i q i = p ip i (τ i 2 τ i ) + p i + (34) q i q i q i q i q i = = p ip i p 2 p (τ 0 τ ) + p ip i p 3 p 2 q i q i q 2 q q i q i q 3 q 2 q + p ip i p 4 p 3 + c + p i +. (35) q i q i q 4 q 3 q 2 q i q i q i To simplify notation, note that (35) can also written in the form: τ i τ i+ = γ i (τ 0 τ ) + i j= γ i γ j q j In order manipulate this, we write out the first and last terms explicitly, for clarity: τ 0 τ = τ 0 τ τ τ 2 = γ (τ 0 τ ) + q. τ N 2 τ = γ N 2 (τ 0 τ ) + γ N 2 γ q τ τ N = γ (τ 0 τ ) + γ γ q + γ N 2 γ 2 q 2 + + γ N 2 γ N 3 q N 3 + q N 2 + γ γ 2 q 2 + + γ γ N 2 q N 2 + q Summing the equations in (36), and rewriting the summation of the terms on the right hand side, we now write this as ( ) τ 0 τ N = + γ i (τ 0 τ ) + γ j (37) q i γ i Since τ 0 = τ N = 0 we can solve this to otain τ directly: ( ) τ = + γ i γ j. (38) q i j=i γ i j=i (36)

Demographic noise slows down cycles of dominance 9 We now wish to examine the asymptotic ehaviour of this expression for the expected hitting time, in the limit when N. Initial numerical explorations lead us to propose that τ ln N when N is sufficiently large. In the remainder of this section we will deduce this estimate systematically. First, note that from (3), we know that lim N ( + γ i ) =. (39) Now, turning to the doule sum in (38), we have q i = i(n i) [ ] i ( + β) + N N N = q i i(n i) + ( + β) i. (40) N Since 2 + β < + ( + β) i N <, (as i N ), we can ound the sum as follows = 2 + β N i + i 2 + β (N i)i ) ( N i + i = 2 2 + β i q i q i q i 2 i N i + i (N i)i ( N i + ) i = 2 2 + β ln N q i 2 ( + ln(n )), (4) Next we prove that, for any i N, γ j j=i γ i is ounded as N. From the definition of γ j we see that γ j γ i = j k=i+ ( + β) + N k j < k=i+ k N + k = j! (N + i)! i! (N + j)!. (42) Therefore, j=i γ j < γ i j=i j! (N + i)! i! (N + j)! = [ i + N 2N N ] N!(N + i)!, i!(2n)! and lim N j=i j! (N + i)! i! (N + j)! = lim N [ i + N 2N N ] N!(N + i)! 2. i!(2n)!

20 Qian Yang et al. We note that if we set i = N in the aove, then the limiting value is small, due to the influence of the negative term that is a ratio of factorials, ut the limit must still e positive; in the case i = we have a limit that is closer to. In all cases, since i N the limiting value must e non-negative, and at most 2. Hence we see that γ j lim 2. (43) N γ i j=i Putting the results (39), (4) and (43) together we finally otain the result τ 4 ( + ln(n )) (44) for the expected time, starting in state, until the system hits one of the two asoring states 0 or N. Together with the conclusion of the previous section, in which we computed that the proaility that the system arrives in state N tends to as N, we can conclude that the expected time required for the system to hit state N is no greater than τ 4( + ln N). In using this result later, we will omit the sudominant constant term since we are concerned primarily with values of N for which ln N, and the limit N. 4.3 The effect of mutation on the stochastic dynamics The analysis in the previous two susections ignored the role of mutation in order to understand the dynamics on the oundary of phase space and, in particular, to estimate the time required to travel along the oundary to a corner. Since in the asence of mutation the corners are asoring states, mutation plays an important role in moving the system from a corner (state 0) to state on the oundary, allowing it then to travel further towards the next corner. The mutation rate µ, together with our assumptions on the stochastic dynamics, imply that, for any system state, the time until the next mutation event M occurs is exponentially distriuted: P(M > t) = e µnt For system states on the oundary of phase space, we would expect that mutations would move them away, into the interior, where the analysis in the previous sections might ecome less useful. This is unlikely to happen if mutations are not expected during the time taken for the system to move along the whole oundary, i.e. if P(hit state N efore mutation) = P(M > 4 ln N) = e 4µN ln N (45) is close to. From the form of (45), if N is fixed and µ 0, then this proaility of hitting state N efore any mutation occurs tends to, which implies that the system remains in oundary states and the analysis of sections 4. and 4.2 applies. On the other hand, if N, and µ remains fixed, then mutation is expected to occur efore the system reaches the next corner, and so the system state tends to leave the oundary. The intermediate alance etween these two regimes occurs when µ (N ln N).

Demographic noise slows down cycles of dominance 2 An equivalent discussion can e framed in terms of the sketch of the dynamics indicated in figure 8. The time required for a full period of the oscillation is composed of two contriution on each oundary piece: the first contriution τ m is the time required to jump, via mutation, from a corner to a state with one new individual of the appropriate kind. This mutation takes an expected time τ m Nµ since there are N individuals and each mutates independently at a rate µ. The second contriution τ is the time required to traverse the oundary starting from state. This is approximately τ = 4 ln N. If τ m τ, i.e. µ (4N ln N), then the largest contriution to the oscillation time is from the mutation events, and so we expect in this regime to have the period of the orit eing dominated y the time required for three independent mutations to occur, i.e. T SSA 3/(Nµ). 5 Analysis of the periodic orit in region II Region II is the cross-over region etween large µ where the ODE approximation is valid, and small µ where the stochastic approach ased on a Markov Chain, is appropriate. Using the analysis of the previous section we see mathematically N log N speaking that region II arises where µ. In region II the stochastic simulations show large fluctuations around the ODE predictions, ut the system does not spend time always near the oundaries of phase space, so it is not clear that the analysis of region III should apply directly. In this section we examine region II, and explain why the fluctuations in the stochastic system act to increase the period of the oscillations rather than to decrease it. This involves a third approach to the dynamics, using a stochastic differential equation derived from the Fokker-Planck Equation for the evolution of a proaility density of initial conditions. We show that the SDE approach captures, in some detail, the transition etween the deterministic and the fully stochastic regimes descried in previous sections of the paper. 5. The Fokker Planck equation and stochastic differential equation A Fokker Planck equation (FPE) is a linear partial differential equation that gives the time evolution of a proaility density function of system states, under the influence of advection through phase space, and a diffusion term that captures random effects in an underlying stochastic process. Individual paths for the dynamics can e computed y solving a related stochastic differential equation (SDE) for the path x(t) through phase space. These viewpoints allow us to use additional mathematical tools to ridge the gap etween the stochastic and deterministic cases in the model. Let p(x, t) represent a proaility density function over system states x at time t. The general form of the FPE is: t p(x, t) = i [ i A i (x, t)p] + 2N i j [B ij (x, t)p], (46) in which the vector A(x, t) represents the advection of proaility density p around trajectories of the deterministic dynamical system ẋ = A(x, t), and the matrix i,j

22 Qian Yang et al. B(x, t) represents diffusion of p across and along trajectories. In the case we consider in this paper, i, j 2 correspond to the suscripts {a, } ecause our phase space is two dimensional (recall that x c = x a x, so the c coordinate is not independent). The vector A and matrix B can e derived from the underlying set of stochastic chemical reactions using the Kramers Moyal, or Van Kampen system size expansion, in a standard way. We do not give the details of the calculation here, ut refer readers to the textooks [25, 5] and recent papers, for example [4] where complete details are given. The vector A is exactly as given previously for the ODEs (3); the matrix B must also e computed y considering the O(/N) terms in the expansion. We otain A = x a [x c ( + β)x + β(x a x + x x c + x a x c )] + µ(x + x c 2x a ), A 2 = x [x a ( + β)x c + β(x a x + x x c + x a x c )] + µ(x a + x c 2x ), (47) B = x a x + x c x a + (x a x 2 + x c x 2 a)(2 + β) + µ(2x a + x + x c ), B 2 = B 2 = (x a x + x a x 2 (2 + β) + µ(x a + x )), B 22 = x a x + x x c + (x a x 2 + x x 2 c)(2 + β) + µ(x a + 2x + x c ). The FPE (46) can e interpreted as descriing the ensemle-average ehaviour of solutions to the following SDE: (48) dx = A(x, t) + N G(x, t)dw(t), (49) where A is the vector in (47), G is defined y the relation GG T = B as given in (48) and dw t is a vector, each element of which is an independent Wiener process [5]. 5.2 Analysis of the angular velocity ϕ In order to explain the increase in the period of the orit as the fluctuations grow, we use the SDE (49) to derive an equation for the angular velocity around the limit cycle, and then compute the period of the limit cycle y integrating the angular velocity. In this section we extend this idea y deriving an SDE for the angular velocity. This allows us to investigate the effects of noise on the period of the orit. A fully analytic approach is unfortunately not possile, so our approach is a comination of numerical and analytic methods. Identifying the correct scalings for features of the limit cycle however enales us to confirm the various asymptotic scalings found in regions I and III and to see how they oth contriute in this region, region II. Since the phase space is two dimensional, we follow the presentation started in section 2.4, showing the periodic orit in the plane R 2 using coordinates y = (y, y 2 ) = ( x a + 2 x, 3 2 x These, and other definitions for our coordinate systems, are illustrated in figure 9. Let Γ R 2 e the set of points on the limit cycle, i.e. the red circle in figure 9. ) (50)

Demographic noise slows down cycles of dominance 23.0 0.8 0.6 0.4 0.2 0.2 0.4 0.6 0.8.0.2 Fig. 9 The red circle is the limit cycle when β = 0.5, µ =, the lue triangle is the limit 26 cycle when β = 0.5, µ = 0 6. When µ is very small, it is hard to tell the difference etween triangle oundary and limit cycle without zooming-in. We further define the polar coordinates (ρ, θ) ased on the centre of the triangle, in the y coordinates, i.e. let ( ρ(y) = y ) 2 ( ) 2 3 + y 2 2 6 and ( ) y /2 θ(y) = arctan 3/6 y2 Let T e the period of limit cycle, i.e. T is the smallest positive value such that y Γ, y(t) = y(t+t ) ut for any 0 < T < T, y(t) y(t+t ). All points in the interior of the limit cycle (except the equilirium point at y = ( 2, 3 6 ) are attracted to the limit cycle, enaling us to define a landing point on Γ to which they are asymptotically attracted. Although the ω-limit set of a point y 0 R 2 would clearly e the entire orit Γ, y looking at the sequence defined y advancing for multiples of the period T we can identify a single limit point p (y 0 ). Specifically we write the time-evolution map for the ODEs as φ t (y 0 ) := y(t) where y(t) solves the ODEs (3) suject to the initial condition y(0) = y 0. Then we define the sequence {p n } n 0 y p n = φ nt (y 0 ), and p 0 = y 0.

24 Qian Yang et al. and the limit point p (y 0 ) := lim n φ nt (y 0 ). Note that p (y 0 ) Γ always, and that if y 0 Γ then p (y 0 ) = y 0. We can now define the asymptotic phase ϕ(y 0 ) of a point near, ut not necessarily on, the limit cycle y setting ϕ(y 0 ) := θ(p (y 0 )). (5) So that if y 0 Γ, then ϕ(y 0 ) = θ(y 0 ), and curves on which ϕ(y 0 ) is constant cross through Γ at these points. We can use ϕ(y 0 ) to consider the influence of noise, which pushes trajectories off the limit cycle, causing time advances or delays, as illustrated in figure 0. The three-fold rotation symmetry of the prolem suggests that it is enough to focus on the interval ϕ [ π 3, π 3 ]. limit cycle oundary Fig. 0 Illustration of the idea of the asymptotic phase of small perturations, using the lower part of Γ as shown in the (y, y 2 ) plane. Each of the three lack points lies on the limit cycle Γ : trajectories evolve along Γ from left to right in the figure. Perturations of these points (in green) towards the oundary lead to trajectories that have longer periods than Γ has, hence the green arrows indicate the asymptotic convergence of trajectories ack to the limit cycle to points that lie to the left of the lack dots. In contrast, perturations (red dots) towards the interior of Γ lead to states that are accelerated y trajectories and converge asymptotically to points on Γ that are ahead of the lack dots. 5.3 A stochastic differential equation for ϕ Since ϕ is a function on the phase space, we can derive an SDE for the evolution of ϕ using (49) and Ito s formula [5]. We otain dϕ(x) = A i (x, t) i ϕ(x) + [B(x, t)] 2N ij i j ϕ(x) dt i i,j + G ij (x, t) i ϕ(x)dw j (t). (52) N i,j Note that our notation ϕ(x) really means ϕ(y(x)) since ϕ is defined y (5) which uses the coordinates y defined in (50). The advection vector A and the matrix B GG T are those given previously in (47) and (48).