On Information Sharing and the Evolution of Collectives

Transcription

1 On Information Sharing and the Evolution of Collectives Guy Sella Michael Lachmann SFI WORKING PAPER: SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent the views of the Santa Fe Institute. We accept papers intended for publication in peer-reviewed journals or proceedings volumes, but not papers that have already appeared in print. Except for papers by our external faculty, papers must be based on work done at SFI, inspired by an invited visit to or collaboration at SFI, or funded by an SFI grant. NOTICE: This working paper is included by permission of the contributing author(s) as a means to ensure timely distribution of the scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the author(s). It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may be reposted only with the explicit permission of the copyright holder. SANTA FE INSTITUTE

2 On the Dynamic Persistence of Cooperation: How Lower Fitness Induces Higher Survivability Running Headline: The Dynamic Persistence of Cooperation Guy Sella Λy Michael Lachmann z March 9, 1999 Summary Sub-populations of cooperators and defectors inhabit sites on a lattice. The interactions among the individuals at a site, in the form of a prisoners-dilemma (PD) game, determine their fitnesses. The PD pay-off parameters are chosen so that cooperators are able to maintain a homogeneous population, while defectors are not. Individuals mutate to become the other type and migrate to a neighboring site with low probabilities. Both density dependent and density independent versions of the model are studied. The dynamics of the model can be understood by considering the life-cycle of a population at a site. This life-cycle starts with one cooperator establishing a population. Then defectors invade and eventually take over, resulting finally in the death of the population. During this life-cycle new cooperator populations are founded by single cooperators that migrate out to empty neighboring sites. The system can reach a steady state where cooperation prevails if the global birth" rate of populations is equal to the rate of their death". This steady state is dynamic in nature cooperation persists although every single population of cooperators dies out. These dynamics enable the persistence of cooperation in a large section of the model's parameter space. 1 Introduction Explaining the evolution and persistence of cooperation is a central problem in evolutionary biology and in the social sciences (Axelrod, 1984). The difficulty can be presented as follows: A cooperating individual in a group of cooperators individuals may have a higher fitness than one in isolation. Nevertheless, cooperative behavior is often costly for the cooperator. Consequently a defecting" individual, that enjoys the cooperation of others but abstains from cooperative behavior, will often have an immediate selective advantage over the cooperators. This advantage renders a population cooperators susceptible to the invasion and take-over by defecting individuals. Therefore, it seems that the evolution and persistence of cooperation faces an intrinsic instability. The interaction between cooperators and defectors is often formalized in terms of the game known as the Prisoners' Dilemma (PD) (Weibull, 1995). The PD payoff matrix is shown in table 1. This is a symmetric game between two players, where each player has two possible strategies: defect or cooperate. The game is set up so that for any strategy of the opponent a defector would have a greater payoff, but if both players cooperate they would have a greater payoff than if both defect. This game is extended Israel. Λ School of Mathematics, Raymond and Beverly Sackler Faculty of Exact Sciences, Tel-Aviv University, Tel-Aviv 69978, y Department of Biological Sciences, Stanford University, Stanford, CA 94305, USA. z Santa Fe Institute, Santa Fe, NM 87501, USA. 1

3 for the purpose of studying the evolution and persistence of cooperation in the framework of evolutionary game theory (Weibull, 1995). In such an extension, one could study the dynamic behavior of a population consisting of cooperators and defectors interacting randomly in pairs. The fitness of an individual is determined by the payoffs it receives in its interactions. The population dynamics in such a framework is characterized by two principal features: 1. A population consisting only of cooperators grows faster than a population consisting only of defectors (this follows from the relation ff>fiin the payoff matrix). 2. In any population with both types, a defector has a higher fitness than a cooperator (This follows from the relations ffi>ffand fi >fl). c d c (ff; ff) (ffi;fl) d (fl; ffi) (fi;fi) Table 1: The prisoner's dilemma payoff matrix: Each box describes the payoff for a possible two player interaction. The left entry belongs to the player employing the strategy above, while the second belong to the player employing the strategy listed on the left. The payoffs are set so that: ffi > ff > fi > fl. Under this condition, the best strategy is to defect independent on the other player's strategy. However, if both players defect they both receive lower payoff than if they both cooperate. How then can cooperation evolve and persist if we accept that defection is always the locally favored strategy? There are three main categories of answers to this question, though the distinction between the categories is not sharp. One which is referred to as the individual centered approach, where cooperation persists because it eventually does confer a fitness advantage on the level of the individual. Models incorporating reciprocity (Axelrod, 1984), partnership (Cooper and Wallace, 1998), or the handicap principle (Roberts, 1998) may be included in this class. The second incorporates kin selection (Hamilton, 1964) in the wide sense. This includes models assuming kin recognition (Axelrod, 1984) as well as models where kin interaction results from individual behavior in a spatial context, or more generally what is sometimes referred to as statistical kinship (Eshel and Cavalli-Sforza, 1982). The third category may be called structured population dynamic models. It includes the hey stack model (Maynard-Smith, 1964; Wilson, 1987), models for the founder effect (Cohen and Eshel, 1976), the neighbor effect (Eshel, 1971) and many others. Our model falls into the third category, where it is distinguished by a few features that will be discussed in section 5. In the framework of structured population dynamic models, we adopt a new perspective and consider the persistence of cooperation at an intermediate point of a process characterized by a shift in the level of selection. A transition in which a new level of individuality emerges, where the individuals preceding the transition are incorporated into a higher level of organization (Száthmáry and Maynard-Smith, 1995; Maynard-Smith and Szathmáry, 1995; Buss, 1987; Jablonka, 1994). In this context we examine how cooperation persists after the transition is completed. After the transition, the lower level individual will be considered a defector if it increases its own fitness at the expense of the higher level individual it belongs to. On the other hand, after the transition a shift in the level of selection has already taken place and selection acts on the individuals of the higher level; and will therefore remove individuals consisting of defectors. Defection will then go extinct in spite of its local advantage within any higher level individual. The closest analogy to this kind of behavior before a shift has occurred would be to take parameters such that a population consisting solely of defectors will die out. In terms of the PD formulation, this would occur if fi < 1. The question we address in this paper is whether introducing such a qualitative asymmetry 2

4 between cooperators and defectors, where homogeneous cooperator populations can maintain themselves (ff > 1) while homogeneous defector populations cannot (fi < 1), can enable the persistence of cooperation even before a transition to a new level of individuality takes place. Under conditions studied in this paper this asymmetry will be shown to give rise to a dynamic persistence of cooperation. In section 2 we present a model which incorporates these dynamics. In the model, populations of cooperators and defectors cohabit sites on a lattice. The random interactions among the individuals at a site determine their fitness, which corresponds to the PD payoffs, where: ffi>ff>1 >fi>fl. Individuals also mutate to become the other type and migrate to a neighboring site with a low probability. The fitness function at a site may also depend on the population density at the site. It is important to stress that in this model the population size at a site is finite and varies over time. Moreover, the population at a given site may die out leaving the site empty until it is settled again by a migrating individual. In sections 3-4 we analyze these models. Beginning with the local behavior at a site, we show that the population dynamics can be described in terms of a life-cycle. The life-cycle begins when a cooperator migrates into an empty site, and founds a cooperator population. This population is later invaded by a defector, which is either a mutant or a migrant. Defectors then take over, after which the population dies and the life-cycle ends. The life-cycle is studied for both the density independent and the density dependent models. Then we move on to the global behavior, during a life-cycle new cooperator populations are founded by cooperators that migrate to empty neighboring sites. The system can reach a steady state where cooperation persists, if the global birth" rate of cooperating populations balances their death" rate, or if on average every population gives rise to one other population during its life-cycle. In section 4 we show that such a balance is stable and enables the persistence of cooperation in a large section of the model's parameter space. In section 5 we briefly discuss how the mode of cooperation suggested in this paper may originate and how it may continue to evolve once it is established. We conclude with comments on how this model fits into the context of other work done in the field, and the bearing it may have on explaining evolutionary transitions in which there is a shift in the level of selection. The technical aspects of the work, are separated into appendixes. 2 The Model The model studied in this paper describes the evolution of finite sub-populations inhabiting sites on an infinite 2-dimensional rectangular lattice. Each individual in a sub- population is either cooperator or a defector, where these behaviors are genetically determined (variables referring to these will be marked with subscript c for cooperators and d for defectors). The dynamics consists of selection with absolute fitnesses, mutation and diffusion. The fitness of an individual is determined by the results of its interactions with other individuals at the same site on the lattice, in terms of a PD payoff matrix, like the one in Table 1. For the interaction to be a PD-type interaction, with the asymmetry described in the introduction, the payoffs are taken to satisfy the following conditions: ffi (dc) >ff (cc) > 1 >fi (dd) >fl (cd) (1) (from here on we will drop the subscripts on these parameters). Assuming individuals at a site with n c cooperators and n d defectors interact at random, the absolute fitnesses of cooperators and defectors at time t will be: f c (t) = g(n(t))(ff n c(t) n(t) + fl n d(t) n(t) ) (2) f d (t) = g(n(t))(ffi n c(t) n(t) + fi n d(t) ); (3) n(t) 3

5 where n = n c +n d is the total population at the site, and g(n) is a function reflecting the density dependence. We will assume g(n)» 1, but that ffg(n c ) > 1 for a population size smaller than the carrying capacity n Λ > 0 (the density independent model corresponds to g(n) = 1). These fitness functions are characterized by two principal features : 1. A homogeneous cooperator population grows faster than a homogeneous defector population. Moreover, a homogeneous cooperator population is capable of maintaining itself while a homogeneous defector population is not. These properties follow from the relations f d (0;n d )=fig(n d ) < 1 for any n d 6= 0, and f c (n c ; 0) = ffg(n c ) > 1 for n c below the carrying capacity n Λ. 2. A defector has a higher fitness than a cooperator in any population structure, since ffi>ffand fi >fl imply f d (n c ;n d )>f c (n c ;n d ) for all n c and n d. Mutation and migration are incorporated by having a time step consist of the following stages: 1. The sub-populations at all sites grow stochastically according to the fitness functions. 2. Each individual may mutate to become the other type with probability μ fi Each individual may migrate to one of its neighboring site with probability D fi 1. The expectation values for n c (t + )and n d (t + )ata site given n c (t) and n d (t) are then: E(n c (t + )jn c (t);n d (t)) = (1 D)[(1 μ)f c (t)n c (t)+μf d (t)n d (t)] + " # 1 X D f n:n: c (t)n n:n: c (t) f c (t)n c (t) + O[(D + μ) 2 ] 4 n:n: E(n d (t + )jn c (t);n d (t)) = (1 D)[(1 μ)f d (t)n d (t)+μf c (t)n c (t)] + " # 1 X D f d n:n:(t)n n:n: d (t) f d (t)n d (t) + O[(D + μ) 2 ]; (4) 4 n:n: where superscript n:n: denotes values at nearest neighboring sites. We will take fi1, so that the fitness coefficients ff, fi, fl, ffi and the corresponding fitness functions are close to 1. This means the stochastic process approaches a process continuous in time. However, when fi 1, μ fi 1 and D fi 1, the scheme we described becomes equivalent to any other reasonable scheme incorporating selection, mutation and diffusion, as well as any reasonable scheme for the stochastic selection. 3 Local Behavior The Life-Cycle We begin the analysis by considering the local behavior at one site. This behavior can be described in terms of a typical life-cycle. A life-cycle for the density independent model (d.i.) is described in Fig. 1. It begins when one cooperator migrates to an empty site. The population of cooperators then begins growing. The growth rate depends on f c (n c ; 0) = ffg(n c ). At some time we denote by t f, the first defector appears from mutation or migration and defectors begin to take over. At some stage when defectors dominate the population, the fitness of cooperators drops below 1 and their number starts decreasing. Some time afterwards, as the frequency of cooperators decreases the fitness of defectors approaches fi and in the process becomes less than 1. The life-cycle ends at time t e when the last defector dies, some time after the last cooperator disappeared. Every life-cycle ends with the death of the whole population at a site after a finite time. Therefore, for cooperation to persist, new life-cycles have to be founded at a rate that balances their termination. The 4

6 population size: n, n c d t f time (generations) t end Figure 1: A schematic representation of a life-cycle: population size vs. time. Changing mutation rate scales the whole cycle, leaving the relations between the the number of cooperators and defectors the same. conditions for the existence of such a steady state are studied in section 4. These conditions will be stated in terms of parameters characterizing the life-cycle. A natural choice of parameters relevant for the global dynamics is the number of cooperator and defectors that migrate out of a site during a life-cycle; we denote them M c and M d. In order to derive these parameters from a description of the life-cycle, we define: M c and M d are then given by: S c S d Z t e 0 Z t e 0 n c (t)dt (5) n d (t)dt: (6) Two equivalent parameters which turn out to be useful are: M c = DS c (7) M d = DS d : (8) M M c + M d = D(S c + S d ) (9) R M c M d = DS c DS d = S c S d : (10) Where M corresponds to the total number of units migrating from a site during a life-cycle and R corresponds to the ratio of cooperators to defectors in the life-cycle. The dependence of the global dynamics and in particular of the persistence of cooperation on R and M, will be studied in section 4. In this section we study the population dynamics at a site, focusing on the factors determining R and M. We find that in the d.i. model R depends on ff,fi,fl and ffi, whereas S d depends mainly on μ and D; while in the density dependent (d.d.) models R depends on μ and D, and S d is essentially constant. 5

7 3.1 The Density Independent Model In the d.i. models the dynamics (equation 4) are homogeneous to the first order in n c and n d. Thus scaling n c and n d by a factor Λ at some time will just scale S c and S d by the same factor, leaving R unchanged. It is not hard to show, that taking two life-cycles that vary only in the time in which the first defector invades, i.e. taking fi f >t f, is equivalent to scaling S c and S d by the factor 1 : Λ=ff» 1+ logff log ff ffi (fi f t f ) : (11) The only significant effect (i.e. not O[μ; D]) that mutation and migration has on the life-cycle is in determining t f. Therefore, we conclude that R = R(ff; fi; fl; ffi)+o[μ; D]. Result 1 provides an explicit expression for R(ff; fi; fl; ffi): Result 1 In the density independent model the ratio R of the average number of migrating cooperators to the average number of migrating defectors in a life-cycle is: R(ff; fi; fl; ffi;μ;d)= E(M c) E(M d ) = DE(S c) DE(S d ) = 1 fi ff fl + O[μ; D]: (12) ff 1 ffi fi We prove this in Appendix A. A careful look at equation 12 reveals that reducing ff (but maintaining the condition ff > 1) while leaving every other parameter fixed can yield a larger ratio of cooperators to j fi;fl;ffi < 0 for ffi>ff>1>fi>fl: (13) It seems reasonable and it will be shown later, that the larger R is, the more likely it is that cooperation could persist in the system. This hints at the possibility that in certain parameter regions of the d.i. model, decreasing ff while leaving all the other parameters fixed will transform the global behavior from a state where cooperation cannot persist to a state where it can. Such behavior is seen in simulation results in Figs. 3, 4, 5, 9, in section 4. Thus, in this model lower cooperator fitness may induce higher survivability! This effect would not have been anticipated from an individual centered perspective. On the other hand when the life-cycle is considered, there is a simple explanation for this effect: The integral number of defectors during the life-cycle S D strongly depends on the number of cooperators at the time the first defector invades n c (t f ). Therefore, cooperators can increase their fraction by maximizing their integral S c keeping n c (t f ) fixed. This explains why R increases when ff is smaller. Roughly speaking the moral is that when surrounded by defectors, keeping a low profile might be a good idea. 3.2 The Density Dependent Model We consider an example of a density depended (d.d.) model with: ρ 1 n< nmax ffi g(n) = nmax ffin n nmax ffi In this model the population size is bounded by n max. The life-cycle for this model is described in Fig. 2. Unlike the d.i. life-cycle, in this case the number of cooperators stabilizes after a finite time t(ff) onc(ff)n max, where c(ff) ff. The shape of this life-cycle ffi 1 In this argument we are ignoring the tails T c and T d shown in Fig. 1. These tails represent the parts of an extrapolated life-cycle in which n c and n d drop below one. Changing t f moves parts of these tails into S c and S d. However, this effect can be shown to change R only by O[μ; D]. : 6

8 time (generations) c d c d population size: n, n population size: n, n time (generations) Figure 2: The density-dependent life-cycle at a site for the model in the example. The graph shows population size vs. time, for two different mutation rates μ = 0:00001 and μ = 0:

9 implies that as long as t f > t(ff), changing parameters so that t f increases, will increase S c, leaving S d constant. This means that one could make R bigger than any R Λ > 0by picking a large enough t f. Taking a large t f simply means taking a small enough μ and D. Hence, for this example we conclude, that R can assume any large value and S d remains essentially constant if μ and D are taken to be small enough. This behavior is generic, it characterizes a class of d.d. models we define in Appendix B as models of type 1. In Appendix B we apply the same reasoning used in the example, to prove the following result: Result 2 Given a density dependent model of type 1 and R Λ 0, taking μ and D such that D + μ» c(ff) c(ff)n max t(ff)+f Sd R Λ ensures that R is bounded from below by R Λ, where f Sd is E(S d ) for the density independent model with the same parameters ff, fi, fl, ffi, and initial conditions en c (0) = n max, en d (0) = 1. This means that for any such model and parameters ff, fi, fl, and ffi, any desired ratio R may be attained by taking small enough μ and D. 4 Global behavior In the systems we studied there are two types of steady states for the global behavior: the trivial steady state where all sites are empty, and non-trivial steady states in which cooperation persists globally. The behavior of two simulations of systems with d.i. dynamics is presented in Figs. 3 and 4. In the first simulation we start with a few sites inhabited by one cooperator each, and cooperation spreads to establishes a non-trivial steady-state. This steady state is dynamic in nature; cooperation persists even though each cooperator population eventually dies out. This can be seen in Fig. 5, where the population size at one site is described as a function of time. In the system described in Fig. 4, the population also starts with a few sites inhabited by one cooperator, but in this case, populations do not seed new ones at a rate that balances their rate of destruction by defectors from within and without. All the sub-populations in this system eventually die out, leaving it in the steady state where all the sites are empty. In these models, the global dynamics derive from an interplay between the local dynamics at a site and the interaction of the sub-populations in this site with its environment. The local dynamics at a site were described in the previous section. They are affected by its environment through the inflow of cooperators and defectors. When a site is empty, this inflow determines when it will become inhabited by a cooperator; and when a site is inhabited by cooperators with no defectors, this inflow will affect how long it will take it to be invaded by a defector t f. The environment, on the other hand, is generated by the local dynamics at sites. Essentially, the more intricate are the population dynamics at a site, the more complex is the analysis of the global dynamics. In the d.d. models, the rough temporal structure of the life-cycle is rather simple, and it is possible to approximate its global behavior by dividing the life-cycle into three main stages: empty, cooperation and defection, where in each the population could be considered as being in one state. In each of these states, the internal dynamics at each site and its interactions with its environment can be described as a Markov process switching between states. For a simplified model of this nature we can obtain an analytical approximation of the conditions for the existence of a non-trivial steady state. We briefly outline a simplified model and its analysis in sections 4.1, 4.2 and in Appendix C. The life-cycle for the d.i. model described in Figs. 1 and 5 consists of many different states, since each population structure n c, n d, affects the evolution at the site and the sites effect on its environment differently. Each of these states is characterized by a different outflow of cooperators and defectors. This makes the global analysis of these models much more complicated. For this reason we restrict the study of their global behavior to simulations. The results from the analysis and simulations of the simplified d.d. model and from the simulations of the d.i. model are presented in section

10 population size 0 1 gen. 25 gen. 555 gen gen gen. color legend Figure 3: Simulation of the density independent model on a lattice with periodic boundary conditions. The parameters for this simulation were: ff = 1:01, fi = 0:95, fl = 0:8, ffi = 1:2, μ = 0:0001 and D = 0:0003. The state of the sites of the lattice are given according to the color key on the right, which corresponds to the different stages in the life-cycle for these parameter values. The simulation begins with single cooperators inhabiting a few sites. The system reaches a non-trivial steady state, where cooperation persists dynamically population size gen. 13 gen. 57 gen. 250 gen. 299 gen. color legend Figure 4: In this simulation we took: ff = 1:06, where all the other parameters are identical to those in the simulation in Fig. 3. In this case, the system reaches the trivial all empty steady state. This is a case where taking higher cooperator fitness results in the extinction of cooperation. This effect is discussed further in section

11 population size: n c, n d ime (generations) Figure 5: The population at a specific site as a function of time, for the same simulation presented in Fig. 3. The life-cycles vary in size due to the stochasticity in the t f s. This stochasticity is caused by variations in the environment and in mutation. However, the shape of the different life-cycles is similar, in correspondence with the scaling properties discussed in section A Simplified Density Dependent Model The life-cycle at a site for the simplified d.d. model is described in Fig. 6. In this model, for which the life-cycle is a simplification of the d.d. life-cycle shown in Fig. 2, when a cooperator enters an empty site it immediately establishes a population of n c cooperators. After some time, the population is invaded by a defector that is either a mutant from within or a migrant from without. Once a defector invades, it instantaneously takes over and establishes a constant population of n d defectors. This population has a probability P d per unit time to die and leave the site empty. Diffusion and mutation are stochastic as in the non-simplified models. The simplified model can be seen as an interacting particle system, where a site (i; j) (which corresponds to the particle) can be in one of three states: empty (S ij = e), cooperation (S ij = c) and defection (S ij = d). The dynamics of this system can be described as a Markov process, written here in terms of the transition probabilities for a site (i; j) during a time step : and A ij e!c(t) P (S ij (t + )=cjs ij (t) =e)= D 4 n ci ij c (t) (14) A ij c!d (t) P (S ij(t + )=djs ij (t) =c)= D 4 n di ij d (t)+μn c (15) A ij d!e P (S ij(t + )=ejs ij (t) =d)=p d ; (16) (17) A ij e!e(t) = 1 A ij e!c(t) c!c(t) = 1 A ij c!d (t) = 1 Aij d!e A ij A ij d!d 10 (t): (18)

12 population size defectors cooperators n d n c 0 time Figure 6: A life-cycle in the simplified model: population as a function time. Notice the similarity to the density-dependent life-cycle (Fig. 2). Here, the number of (i; j)'s nearest neighbors in state c is denoted I ij c, and the number of nearest neighbors in state d was denoted I ij d. In writing these dynamics it was assumed that the time step fi 1, so that effects that are 2nd order in μ and D can be ignored. Equation 14 describes how a site changes its state from e to c, by way of a nearest neighbor interaction corresponding to diffusion. Equation 15 describes how a site changes its state from c to d, either by nearest neighbor interaction corresponding to diffusion, or spontaneously in a way which corresponds to mutation. Finally, equation 16 describes how a site changes its state from d to e, spontaneously, inawaythat corresponds to the death of the defector population. Equations have 4 parameters: Dn c, Dn d, μn c and P d. As D, μ, and P d are all homogeneous to the 1st order in the time scale, so are the right hand sides in equations This means one of the 4 parameters, such as P d, could be taken to determine the time scale. The other 3 could be taken to be independent of the time scale, for example Dn d P d, D nc and μ n d (which are independent and homogeneous with degree 0in ). This model captures the qualitative features of the local behavior of the explicit d.d. models of section 2. The establishment of cooperation, the defector take-over and the populations' extinction which derive from the population dynamics at a site in the explicit d.d. models are assumed in the simplified model. The interactions between a site and its environment, however, are of same form in both simplified and general d.d. models. The environment affects when the empty site becomes inhabited and when the defector take-over occurs. On the other hand a site affects its environment by diffusing out cooperators and defectors. 4.2 A Mean-Field Approximation to the Simplified Density Dependent Model We would like to find the region in the model's parameter space in which a non-trivial steady state exists. One way to do this, is to solve the model analytically. A solution is a stationary probability distribution on the space of all possible lattice configurations P (fs ij g i;j2z ) as a function of the model's parameters. Using a mean field approximation one can find the best solution within a restricted class of distributions. Roughly speaking, as the class of distributions becomes larger the approximations become better. In this paper we will not evaluate the accuracy of the approximations other than by comparing their predictions with simulations. A systematic evaluation of these approximations, as well as a more accurate analysis using Renormalization Groups, has been done for other particle systems (Goldenfeld, 1992; Baxter, 1982). 11

13 The 1st order mean field approximation is restricted Y to probability distributions of the form: P (fs i;j (t)g i;j2z )= P (S i;j (t)): (19) This means that the probability of finding the system in a certain configuration can be decomposed into a product of the probabilities of finding each site in its state. One further assumes that the probabilities of finding a site in state c, d or e are uniform across the lattice. Under these assumptions the system's description reduces to the probabilities of finding any site in each one of the possible states 2. Denoting these probabilities which are independent of the site by p e, p c, and p d, the system's dynamics reduces to: i;j2z p e (t + ) = p e (t)(1 A e!c (t)) + p d (t)a d!e (t) (20) p c (t + ) = p c (t)(1 A c!d (t)) + p e (t)a e!c (t) p d (t + ) = p d (t)(1 A d!e (t)) + p c (t)a c!d (t); Where A e!c, A c!d and A d!e denote the transition probabilities, which can be derived from equations 14: A e!c (t) = D 4 n c4p c (t) (21) A c!d (t) = D 4 n d4p d (t)+μn c A d!e (t) = P d : Here we set I d =4p d (t)and I c =4p c (t). The fixed point and stability analysis for this system is straight forward. A non-trivial fixed point (one where p e 6= 1) exists if: D μ > 1: (22) When this condition holds, the system has two meaningful fixed points, with one trivial (p e =1)and the other not. In this case only the non-trivial fixed point is stable. For this non-trivial fixed point, expressions for R, M or any other dynamic parameter of the system, as functions of Dn d P d, D, nc μ n d and P d can be derived. Deriving condition 22 from general considerations will help in understanding the scope of the 1st order approximation. For a non-trivial steady state to be maintained, every life-cycle has on average to establish exactly one new life-cycle. This requirement takes the form: DS c ρ c e = M 1+ 1 R ρ c e=1; (23) where ρ c e denotes the density of empty sites near a site in state c. This density equals the probability that a cooperator leaving a site will establish a new life-cycle. Condition 22 could be derived from equation 23 by putting trivial bounds on ρ c e and S c : ρ c e» 1 (24) S c = 1 n c t f» n c μ : (25) The bound on ρ c e is realized only when all the neighboring sites are empty. The bound on t f is also realized when all the neighboring sites are empty, i.e. when the first defector is always a mutant. These two bounds 2 Note, that both this and the second order" mean field approximations, can be treated as models for the persistence of cooperation by their own right. 12

14 imply that condition 22 is equivalent to the requirement that at least one cooperator diffuses out in a life-cycle at a site surrounded by empty neighbors. As the number of cooperators in a life-cycle at an isolated site depends only on μ, the number of cooperators diffusing from it depends only on μ and D. Condition 22 indicates that the 1st order mean-field approximation cannot incorporate the harmful effects of migration, an important feature of the model. In the 1st order mean field approximation, the density ρ c e can approach 1 enabling cooperators to survive as long as on average one cooperator migrates during a life-cycle. This means that in this approximation defector migration doesn't really affect whether cooperation prevails or not, because the density of inhabited sites can always be so low that no defector ever invades it. In the spatial model the density ρ c e can never reach 1, because near a population of cooperators there is always a finite probability ofhaving the population from which the founding cooperator migrated. The neighboring population, in this case, will be in either state c or d during some part of the life-cycle of its daughter sub-population. This discrepancy between the spatial model and the 1st order mean-field approximation, is demonstrated in Figs. 3 and 4. The second picture in Fig. 4 (25 generations) indicates that a life-cycle at a site surrounded by empty sites produced more than 1 diffusing cooperator. Yet cooperation does not prevail due to the effects of extensive defector migration into sites inhabited by cooperators. An approximation incorporating such effects would have to describe the correlations between the states of nearest neighbors. Such an approximation is outlined in Appendix C. Results presented in the next section will hint at the possibility that as the phase transition between persistence and non-persistence of cooperation is approached the correlation length in the system goes to infinity. This would imply that near the parameters at which the transition happens, the reliability ofsuch mean field approximations is questionable. 4.3 Results The d.i. model has 6 parameters: ff, fi, fl, ffi, μ and D, while the simplified d.d. model has 4: D μ, Dn d P d, nc n d and P d. A point atwhich a non-trivial steady state is maintained, is characterized by a stationary probability distribution on all possible lattice configurations. We would like to present these 6/4 dimensional phasespaces in a comprehensible way, that permits comparison with the global behavior of models which derive from different local parameters. In doing so we will necessarily lose some information, information that can be further explored using different representations. To the extent that the fine temporal and spatial structure of a steady state in these models can be ignored, the basic variables characterizing the global dynamics would be M the average number of migrants during a life-cycle, and R the cooperator to defector ratio among these migrants. Phase spaces in the R-M coordinates, which were derived from analytical approximations to the simplified d.d. model, from simulations of the simplified d.d. model and from simulations of the d.i. model are presented in Figs. 78.The solid lines in Figs. 7, 8 correspond to the boundaries in R-M space below which a non-trivial steady state does not exist according to 1st/2nd order approximations. We will refer to such a boundary as a phase boundary. The thick lines in Figs. 7, 8 represent the phase boundaries derived from simulations. They were derived as described in appendix D. In the simplified d.d. model DS d = Dn d P d is one of the basic parameters of the model, while DS c derives from the dynamics (see sections ). As M = DS d + DS c and R = DSc DS d, one component of R and M is a parameter whereas the other is an outcome of the dynamics depending on the other parameters. In the d.i. model the situation is similar (3.1), R is a function of ff, fi, fl and ffi, and thus can be considered to be a parameter, whereas M derives from the dynamics which depends on the other parameters. The shape of the phase boundaries from the analysis and simulations can be roughly understood from the heuristic derivation in the last section, equation 23, that predicts the phase boundary takes the form: M = C(1 + 1 ); (26) R 13

15 M R Figure 7: The R-M phase space for the simplified d.d. model: The graph presents regions of cooperation persistence according to the 1st and 2nd order mean-field approximations (above the phase boundaries pictured); and according to simulations. For details on how the phase boundaries were derived from simulations, see Appendix D. 14

16 M R Figure 8: The R-M phase space for the d.i. model: The graph presents regions of cooperation persistence according to simulations of the d.i. model. The phase boundaries according to the mean-field approximations to the simplified d.d. model are also presented, as reference. For details on how the phase boundaries were derived from simulations, see Appendix D. 15

17 where C is some constant. The differences in the shape and position of the phase boundaries reflect the effects of the fine spatio-temporal dynamic structure. As we discussed at the end of the last section, one can state roughly that the effect of spatial correlations, i.e. spatio-temporal structure, is to increase the damage defectors inflict thus imposing stronger restrictions on the region in the R-M space where a nontrivial steady state can be maintained. This causes the phase boundaries resulting from the simulations to be above those resulting from the 2nd order approximation; as well as for 2nd order phase boundary to be above the 1st order. A systematic study of the factors effecting the phase boundaries requires the study of higher order correlations. The drawback of using R-M phase-spaces to study a specific model is that some information concerning the relation between the system's behavior and its basic parameters is lost. One interesting dependence on local parameters in the d.i. model mentioned in section 3.1 is that of the persistence of cooperation on the fitness of cooperators ff. We explained why, when ff is reduced, a larger cooperator to defector ratio R, can be expected. The functional dependence of R on ff was derived. In that section we conjectured that as ff increases and R decreases, one might expect that the disturbance from defectors will grow to a point where cooperation cannot be maintained. In Figs. 3, 4 an example for exactly that kind of behavior is described. Two systems in which all the parameters other than ff were taken to be the same were simulated. In the system with the smaller ff a non-trivial steady state was established and thus cooperation persisted, whereas in the system with the larger ff the trivial steady state with no cooperation was reached. In Figs. 9 A-C the behaviors of a few dynamic parameters were studied as a function of ff, while all the other parameters were fixed. As ff was increased the number of migrating defectors increases while the number of migrating cooperators remains approximately constant Fig. A. Thus R decreases Fig. B, and so does the density of occupied sites Fig. C. Around ff = 1:0175 the rate of destruction by migrating defectors reaches a point in which a non-trivial steady state can no longer be maintained, like in the example described in Fig. 4. Note that as the phase boundary is approached, the density of occupied sites drops Fig. C. This suggests that near the boundary the correlation distance in the system grows. As mentioned in the previous section, this might imply that the reliability of the mean-field approximations near the boundary becomes questionable. 5 Discussion We have demonstrated that cooperation may persist in a system in a steady state where populations of cooperators and defectors are constantly appearing and disappearing. This can be understood by considering the life-cycle of a population at a site. The life cycle starts with one cooperator that establishes a population, then defectors invade and take over, and it ends with the death of the population. During this life-cycle, new populations of cooperators are founded by single cooperators that migrate to empty neighboring sites. The system reaches a steady state where cooperation persists, if the global "birth" rate of populations is equal to the rate of their "death", or if on average every population gives rise to one other population during its life-cycle. This steady state is dynamic in nature cooperation persists even-though every single population of cooperators eventually dies out. In section 4 we demonstrated that these dynamics enable the persistence of cooperation in a large section of the model's parameter space. Furthermore, we demonstrated and explained how in the d.i. models, taking the local fitness of cooperators to be lower can enable the persistence of cooperation; and within the region of persistence it can increase the global density of sites inhabited by cooperators. We have described how cooperation persists, but we did not describe how it may originate in the first place, or continue to evolve once it is established. Although a proper treatment of these questions will require the expansion of the models presented above, a few comments are in place here. We begin with the origin. Consider a system consisting of reproductively isolated sub-populations, where new subpopulations are founded by individuals that leave existing sub-populations. Assume further that individuals 16

18 A B number of migrators Number of migrators (M, DS c, DS d ) M D Sc S Sd Ratio of cooperators to defectors (R) α R α C 0.5 Density of occupied sites density α Figure 9: Dynamic behavior in the d.i. model as a function of ff: Figs. A-C describe different dynamic variables of the system as they result from simulations of the d.i. model with fi =0:95, fl =0:8, ffi =1:2, μ = 0:0001 and D = 0:0003 where ff varies between 1:001 1:035. In Fig. A: the average number of migrating cooperators (DS c ), defectors (DS d ) and their sum (M) in a life-cycle are given as a function of ff. In Fig. B: the average ratio of cooperators to defectors migrating during a life-cycle (R) is given as a function of ff; and in Fig. C: the density of occupied sites is given as a function of ff. 17

19 with cooperative behavior appear in the system, after which defectors appear. The defectors may well be the preexisting type. It is well known that cooperative and defective behaviors, are often relative terms. In an homogeneous population the appearance of an individual that displays more altruistic behavior than its peers, may elicit a dynamic where the preexisting type is redefined as a defector. If cooperation is costly, the pre-existing type benefits from interacting with cooperators without having to pay the cost, and will have the characteristics of defectors once cooperators appear. For the type of dynamic we describe to emerge, one additional condition must hold: sub-populations of defectors must not be able to maintain themselves. This condition seems unreasonable at first, if defectors may be the a preexisting type which existed and maintained sub-populations before cooperators appeared. How then can such a condition be accounted for after the appearance of cooperation? We suggest two options. One is to consider a system with an inhomogeneous environment containing harsh areas where sub-populations of solitary individuals are not able to survive. However, as cooperator sub-populations are more efficient they can inhabit some of these niches. Once they do, the preexisting type may invade these areas, by taking advantage of the cooperators, and here the conditions for the dynamic persistence of cooperation may appear once they are invaded by cooperators. Another possibility isto consider a system in which reproductively isolated subpopulations share common resources. Once cooperating sub-populations appear, the conditions for solitary sub-population change, again due to the higher efficiency of sub-populations consisting of cooperators. Consequently, sub-populations of defectors can not be maintained and the stage is set for the establishment of a steady state of the type we described. Next consider the direction of evolution once a steady state with cooperation is established. Let us assume the following model: Sub-populations share common resources, as we have just described. At some point "improved" cooperators appear, and consequently both the old cooperators and defectors assume the role of defectors. The long term evolution (Eshel et al., 1997; Eshel et al., 1998) of the system then becomes relevant. Specifically, one must consider the invasion conditions for the new cooperators to take over. On the one hand, "improved" cooperators mayinvade by establishing conditions, through the shared resources, under which sub-populations of the preceding cooperators cannot maintain themselves. If this is indeed the case, after the new cooperators take over, a new steady state will be established, where the carrying capacity of the environment has been increased, and the fitnesses within sub-populations are renormalized. On the other hand, as described in section 3, taking a higher cooperator fitness and leaving everything else the same can lead to a breakdown in the persistence of cooperation. This suggests that invasibility conditions for improved" cooperators in these systems can be very subtle and should be studied more carefully. We conjecture that such a system may regulate the rate at which cooperation evolves, so that it cannot improve too fast in a single take-over step. If this is indeed true, it will be reflected in the invasibility criteria on the one hand, and in the way fitness is renormalized after a take over occurs, on the other hand. How is this model related to other models for the persistence of cooperation? One way in which cooperation is explained is in terms of selection on the level of the individual favoring it. In this category one may include models for reciprocal altruism (Axelrod, 1984), models for partnership (Cooper and Wallace, 1998) and models where cooperation is based on kin recognition (Hamilton, 1964; Axelrod, 1984). In our model, cooperation persists even-though at any site at any time defection is locally advantageous. Moreover, we have shown that lowering the fitness of cooperators may increase their survivability. Considering these phenomena, it is self evident that our model cannot be explained in terms of selection at the level of the individual. It is a structured population dynamic model, in the spirit of other models for the persistence of cooperation, including: the hay stack model (Maynard-Smith, 1964; Wilson, 1987), the founder effect (Cohen and Eshel, 1976) model and the neighbor effect model (Eshel, 1971). These models are sometimes referred to as models for group selection, although they do not meet most definitions of group selection (Wilson and Sober, 1989; Jablonka, 1994). Specifically, the persistence of cooperation in our model cannot be explained by group selection: there is no meaningful variation between sub-populations, 18

20 as they all go through the same life-cycle. Moreover, there are no heritable group properties, as all the sub-populations start from an identical single cooperator. Furthermore, the standard PD condition that 2ff > fl + ffi, meaning that a group of cooperators has an average fitness higher than any other group, doesn't hold. The lower ff effect indicates that in this respect, lowering the fitness of a cooperator group below fl+ffi may actually improve its survivability. Within the category of structured population dynamic 2 models, ours is distinguished by the emergence of the temporal population structure we referred to as the local life-cycle. Considering the properties of the local life-cycles, and their global interactions, had enabled us to understand how and when cooperation persists. Száthmáry and Maynard Smith (Száthmáry and Maynard-Smith, 1995; Maynard-Smith and Szathmáry, 1995), Buss (Buss, 1987), and others have claimed that the evolutionary transitions, in which a new level of organization emerges and where a shift takes place in the level at which selection acts, must be explained in terms of the immediate selective advantage to the individual replicators, i.e. to the individual existing before the transition. However, a model for a transition, characterized by a shift in the level at which selection acts, would have at least two relevant levels of individuality in it. One may ask, when during the the transition does the shift in the level of selection occur? It seems that at the later stages of a transition, the next level is at least as relevant as the previous one. More generally, it seems that both levels, the future level of individuality and the level of individuality preceding the transition, may havesimilar explanatory footings. Indeed, in the construction of the model presented here, we have considered both levels (see section 1). However, if the way cooperation persists in our model is a logically possible description of an intermediate stage of a transition, then selection on neither the future level nor the preceding one can explain it. Hence we suggest that as useful as these considerations may be, it is possible that neither of them can provide a comprehensive explanation of the behavior of the evolutionary dynamics during the transition itself. References Axelrod, R. (1984). The Evolution of Cooperation. Basic Books, New-York. Baxter, R. J. (1982). Exactly Solved Models in Statistical Mechanics. Academic Press, London; New York. Buss, L. W. (1987). The Evolution of Individuality. Princeton UP, Princeton NJ. Cohen, D. and Eshel, I. (1976). Founder effect and evolution of altruistic traits. Theoretical Population Biology, 10: Cooper, B. and Wallace, C. (1998). Evolution, partnership and cooperation. J. Theor. Biol., 195: Eshel, I. (1971). On the neighbor effect and the evolution of altruistic traits. Theoretical Population Biology, 3: Eshel, I. and Cavalli-Sforza, L. L. (1982). Assortment of encounters and evolution of cooperativeness. Proc. Natl. Acad., 79: Eshel, I., Feldman, M. W., and Bergman, A. (1998). Long-term evolution, short-term evolution, and population genetic theory. J. Theor. Biol., 191: Eshel, I., Motro, U., and Sansone, E. (1997). Continuous stability and evolutionary convergence. J. Theor. Biol., 185: Goldenfeld, D. (1992). Lectures on Phase Transitions and the Renormalization Group. Adison-Wesley, Massachusetts. 19