Evolutionary stability on graphs

Transcription

1 RTICLE IN PRESS Journal of Theoretical iology 251 (2008) Evolutionary staility on graphs Hisashi Ohtsuki a,, Martin. Nowak a,,c a Program for Evolutionary Dynamics, Harvar University, Camrige, M 02138, US Department of Organismic an Evolutionary iology, Harvar University, Camrige, M 02138, US c Department of Mathematics, Harvar University, Camrige, M 02138, US Receive 1 Octoer 2007; receive in revise form 9 January 2008; accepte 11 January 2008 vailale online 18 January 2008 stract Evolutionary staility is a funamental concept in evolutionary game theory. strategy is calle an evolutionarily stale strategy (ESS), if its monomorphic population rejects the invasion of any other mutant strategy. Recent stuies have reveale that population structure can consieraly affect evolutionary ynamics. Here we erive the conitions of evolutionary staility for games on graphs. We otain analytical conitions for regular graphs of egree k42. Those theoretical preictions are compare with computer simulations for ranom regular graphs an for lattices. We stuy three ifferent upate rules: irth eath (D), eath irth (D), an imitation (IM) upating. Evolutionary staility on sparse graphs oes not imply evolutionary staility in a well-mixe population, nor vice versa. We provie a geometrical interpretation of the ESS conition on graphs. r 2008 Elsevier Lt. ll rights reserve. Keywors: Evolutionary game theory; Evolutionary graph theory; ESS; Structure population; Spatial games 1. Introuction Evolutionary game theory is the stuy of frequency epenent selection (Maynar Smith, 1982; Hofauer an Sigmun, 1998; Nowak an Sigmun, 2004; Nowak, 2006). The fitness of an iniviual is not constant, ut epens on interactions with other iniviuals. These interactions can e escrie y a game. The payoff from the game affects fitness, which is reprouctive success. Reprouction can e genetic or cultural. n important concept in evolutionary game ynamics is that of an evolutionarily stale strategy (ESS) (Maynar Smith an Price, 1973; Maynar Smith, 1974, 1982). If all players of a population aopt that strategy, then no mutant strategy can invae. The traitional ESS conition is efine for infinitely large, well-mixe populations. In a well-mixe population, every iniviual interacts with every other iniviual equally likely. Real populations are, of course, neither infinitely large nor well-mixe. Various attempts have een mae to exten the ESS Corresponing author. Tel.: ; fax: aress: ohtsuki@fas.harvar.eu (H. Ohtsuki). concept to populations of finite size (Maynar Smith, 1988; Schaffer, 1988; Ficici an Pollack, 2000; Neill, 2004; Nowak et al., 2004; Wil an Taylor, 2004; Traulsen et al., 2006). In spatial evolutionary game theory (Nowak an May, 1992), the players of a population are arrange on a spatial gri an interact with their nearest neighors. Spatial games can lea to very ifferent evolutionary ynamics than games in well-mixe populations (Nowak an May, 1992, 1993; Wilson et al., 1992; Ellison, 1993; Herz, 1994; Lingren an Norahl, 1994; Nowak et al., 1994; Killingack an Doeeli, 1996; Nakamaru et al., 1997, 1998; Eshel et al., 1998, 1999; Szao an T+oke, 1998; van aalen an Ran, 1998; Szao et al., 2000, 2005; Hauert, 2001; Irwin an Taylor, 2001; Hauert et al., 2002; Szao an Hauert, 2002; Le Galliar et al., 2003; Hauert an Doeeli, 2004; Ifti et al., 2004; Santos an Pacheco, 2005; Santos et al., 2006; Szao an Fa th, 2007). Spatial moels have also een stuie in ecology (Levin, 1974; Levin an Paine, 1974; Durrett an Levin, 1994; Hassell et al., 1994; Tainaka, 1994; Durrett an Levin, 1997, 1998; Tilman an Karieva, 1997; Iwasa et al., 1998; Haraguchi an Sasaki, 2000; Neuhauser, 2001; Pastor-Satorras an /$ - see front matter r 2008 Elsevier Lt. ll rights reserve. oi: /j.jti

2 RTICLE IN PRESS H. Ohtsuki, M.. Nowak / Journal of Theoretical iology 251 (2008) Vespignani, 2001; Wootton, 2001; May, 2006) an population genetics (Wright, 1943; Kimura, 1953; Kimura an Weiss, 1964; Maruyama, 1970, 1971; Nagylaki, 1992; Epperson, 2003). Literature of kin selection is useful in analyzing spatial games (Taylor, 1992; Taylor an Irwin, 2000; Rousset, 2004; Lehmann et al., 2007; Taylor et al., 2007). Evolutionary graph theory (Lieerman et al., 2005; Ohtsuki et al., 2006; Ohtsuki an Nowak, 2006a, ; Pacheco et al., 2006a, ; Taylor et al., 2007) is the extension of spatial evolutionary ynamics to general graphs an networks. The memers of a population occupy the vertices of a graph. Interactions occur etween connecte iniviuals. Many ifferent upate rules are possile. Competition for reprouction an playing the game can e escrie y the same graph or y two ifferent graphs (Ohtsuki et al., 2006; Ohtsuki an Nowak, 2006a, ; Ohtsuki an Nowak, 2007a, ). well-mixe population is efine y a complete graph with ientical weights. Spatial games are typically escrie y regular lattices. The purpose of this paper is to erive the ESS conition for games on graphs. The paper is structure as follows. In Section 2, we review the efinition of evolutionary staility in a wellmixe population. In Section 3, we introuce three ifferent upate rules an formulate the replicator equation on graphs (Ohtsuki an Nowak, 2006). Section 4 escries our main results. We give geometrical representations of the ESS conitions on graphs in Sections 5 an 6. We offer examples an computer simulations in Section 7. Finally, we provie a iscussion in Section Evolutionary staility in a well-mixe population Consier a game etween two strategies, an. The payoff matrix is given y a c : In a well-mixe population every player meets every other player equally likely. Let x an x e the frequencies of an players in the population. The average payoffs of an players are given y P ¼ ax þ x, P ¼ cx þ x. (2) population of iniviuals is challenge y a small fraction of invaers. The relative aunance of players is x ¼, where 0o51. The fraction of players is x ¼ 1. Strategy is evolutionary stale if P 4P for ðx ; x Þ¼ð1 ; Þ. This conition leas to ða cþð1 Þþð Þ40. (3) (1) For! 0, the left han sie of (3) is positive if an only if ESS: a4c or a ¼ c an 4. (4) The conition, 4, is only use in the knife-ege case, a ¼ c. The evolutionary staility of a strategy in a game with n strategies can e efine in a similar way. strategy is ESS if an only if conition (4) hols in pairwise comparison with each of the n 1 other strategies. The traitional staility concept in game theory is the Nash equilirium (Nash, 1950; Luce an Raiffa, 1957; Fuenerg an Tirole, 1991; inmore, 1994; Weiull, 1995; Samuelson, 1997). For payoff matrix (1), strategy is calle a Nash equilirium if an only if Nash: axc. (5) Conition (5) implies that is a est reply to itself. In aition, strategy is calle a strict Nash equilirium if an only if Strict Nash: a4c. (6) Conition (6) implies that is the unique est response to itself. Note that if is a strict Nash equilirium then it is an ESS. If is an ESS then it is a Nash equilirium. While the conition for a Nash equilirium epens only on the payoff matrix, a meaningful concept of evolutionary staility is affecte y population size (Schaffer, 1988; Ficici an Pollack, 2000; Nowak et al., 2004; Wil an Taylor, 2004; Traulsen et al., 2006) an population structure (Nowak an May, 1992, 1993; Nakamaru et al., 1997; Le Galliar et al., 2003). 3. Evolutionary game ynamics on graphs In this section, we introuce three ifferent upate rules for evolutionary games on graphs (Ohtsuki et al., 2006; Ohtsuki an Nowak 2006a, ). Then we iscuss the replicator equation on graphs (Ohtsuki an Nowak, 2006). We consier an infinitely large population. The structure of the population is escrie y an infinite, connecte, an regular graph of egree k. graph is calle regular when each noe has exactly the same numer of neighors; that numer is calle the egree of the graph. Each noe represents a player whose strategy is either or. There are no empty noes. In this paper, we stuy k42. For games on cycles, k ¼ 2, we refer to Ohtsuki an Nowak (2006a). Each player interacts with all k neighors, an otains an accumulate payoff, enote y P. The accumulate payoff is translate into fitness, F, y the following formula: F ¼ð1 wþþwp. (7) Here 0pwp1 represents the intensity of selection. If w ¼ 0 then fitness is constant, F ¼ 1, an inepenent of the payoff. Throughout the paper, we consier the case of small w, given y 0ow51.

3 700 RTICLE IN PRESS H. Ohtsuki, M.. Nowak / Journal of Theoretical iology 251 (2008) Given the fitnesses of all players, we upate the strategy of one player in each elementary time step. Therefore upating is asynchronous. We consier the following three upate rules (Ohtsuki et al., 2006; Ohtsuki an Nowak, 2006a, ): irth Death (D) upating. n iniviual is chosen for reprouction proportional to fitness; the offspring replaces a ranomly chosen neighor. Death irth (D) upating. ranom iniviual is chosen to ie; the k neighors compete for the empty site proportional to their fitness. Imitation (IM) upating. ranom player is chosen for upating his strategy; he either aopts a strategy of one of his k neighors or remains with his own strategy, proportional to fitness. Using the pair approximation metho (Matsua et al., 1987, 1992; Nakamaru et al., 1997, 1998; Keeling, 1999; van aalen, 2000), Ohtsuki an Nowak (2006) have shown that for small w the frequencies of strategies on a regular graph of egree k can e escrie y a ifferential equation. For a n n game with the payoff matrix, ½a ij Š,it is given y " # X n _x i ¼ x i x j ða ij þ ij Þ f. (8) j¼1 Here x i enotes the frequency of ith strategy, a ot represents time erivative, an f ¼ P n i;j¼1 x ix j ða ij þ ij Þ. For each upate rule, the value of ij in Eq. (8) is calculate from the original payoff matrix, ½a ij Š,as D: D: ij ¼ a ii þ a ij a ji a jj, k 2 ij ¼ ðk þ 1Þa ii þ a ij a ji ðk þ 1Þa jj, ðk þ 1Þðk 2Þ IM: ij ¼ ðk þ 3Þa ii þ 3a ij 3a ji ðk þ 3Þa jj. (9) ðk þ 3Þðk 2Þ Interestingly, ifferential equation (8) has the form of a replicator equation (Taylor an Jonker, 1978; Hofauer et al., 1979; Zeeman, 19; Weiull, 1995; Hofauer an Sigmun, 1998; Nowak, 2006) with a transforme payoff matrix ½a ij þ ij Š. Therefore, many aspects of evolutionary ynamics on graphs can e analyze y stuying a stanar replicator equation with a transforme payoff matrix, a socalle replicator equation on graphs. For example, for the 2 2 payoff matrix (1), Ohtsuki an Nowak (2006) have shown that the transforme payoff matrix is a þ h. (10) c h The moifier h epens on the upate rule. It is given y D: h ¼ a þ c, k 2 D: ðk þ 1Þa þ c ðk þ 1Þ h ¼, ðk þ 1Þðk 2Þ IM: ðk þ 3Þa þ 3 3c ðk þ 3Þ h ¼. ðk þ 3Þðk 2Þ (11) These results hol for infinitely large population size an for 0ow51. Regaring a finite population of size N, Ohtsuki an Nowak (2006) foun that the replicator equation on graphs gives a goo approximation if Nw1. In the next section, we use Eqs. (10) an (11) to erive the concept of evolutionary staility for graph selection. 4. ESS conitions on graphs In orer to characterize evolutionary staility on graphs, we ask whether rare mutants (of fraction) have a selective avantage over resients. ccoring to the moifie payoff matrix (10), if a4c h then rare mutants are selecte against in an -population. In this case, is ESS when compare to. If aoc h then can invae an, therefore, is not an ESS. We o not iscuss the evolutionary staility of the knife-ege case, a ¼ c h, ecause it is ungeneric. Throughout the paper we call ESS on graphs: a4c h (12) the ESS conition on graphs. In contrast to the ESS conition on graphs, the conitions for (strict) Nash equilirium on graphs are not otaine from the moifie payoff matrix (10). y analogy to its counterpart in a well-mixe population, a straightforwar efinition of (strict) Nash equilirium on graphs is as follows: a strategy is a (strict) Nash equilirium on graphs if no one gains a (strictly) higher payoff y switching to the other strategy. Therefore, the conitions for (strict) Nash equilirium on graphs are the same as those in a well-mixe population, Eqs. (5) an (6). On graphs, it is possile that a strict Nash equilirium is not an ESS. It is also possile that an ESS is not a Nash equilirium. We will iscuss this issue further in Section 7.1. For the three upate rules, conition (12) is rewritten as follows: ESS on graphs (D upating): ðk 1Þa þ 4ðk 1Þc þ. (13) ESS on graphs (D upating): ðk 2 1Þa þ 4ðk 2 k 1Þc þðk þ 1Þ. (14) ESS on graphs (IM upating): ðk 2 þ 2k 3Þa þ 34ðk 2 þ k 3Þc þðk þ 3Þ. (15) ll these conitions converge to a4c for k!1. This makes sense, ecause a well-mixe population is escrie y a fully connecte graph.

4 RTICLE IN PRESS H. Ohtsuki, M.. Nowak / Journal of Theoretical iology 251 (2008) The geometry of evolutionary staility We can provie a eautiful geometrical representation of the ESS conitions (13) (15). Fig. 1 illustrates the invasion of a homogeneous population y an fraction of players. Initially the players are sprinkle ranomly over the entire population: each vertex changes from to with proaility, which is very small. The following evolutionary ynamics have two time scales: (i) on a fast time scale the population equilirates locally an - clusters are forme, (ii) on a slower time scale the gloal frequency of (an ) is changing. This separation of time scales is a consequence of the weak intensity of selection, 0ow51 (Ohtsuki et al., 2006). fter the local equiliration process, we fin that, a player, whose one neighor is alreay specifie, has on average one neighor among his k 1 other neighors. Thus, in orer to intuitively unerstan the ESS conitions on graphs (13) (15), it is convenient to use a schematic rawing where a half-line of players is emee in a sea of players (Fig. 2). If the tip of the half-line is more likely to shrink than to exten, then is ESS. Otherwise selection favors the invasion of. For example, let us stuy the invasion ynamics for D upating. The half-line of -players extens in length y one if (i) one of the ðk 1Þ -neighors of the -player on the tip is chosen to ie an (ii) the -player, whose payoff is ðk 1Þc þ, wins the competition over the vacancy against Initial invasion Equilirium configuration : fraction of ε Fig. 1. Local configuration of rare mutants quickly equilirates into clusters, without changing the total numer of initial players. Half-line of invaers exten? or shrink? ka (k - 1)a + (k - 1)c + (k - 2)c + 2 Fig. 2. schematic rawing where rare -mutants invae the population of -players in a half-line shape. The numer shown next to player is his payoff. When the half-life is more likely to shrink than to exten, strategy is an ESS.

5 702 RTICLE IN PRESS H. Ohtsuki, M.. Nowak / Journal of Theoretical iology 251 (2008) ðk 1Þ -players, whose payoff is ka. These events occur with a proaility that is proportional to ð1 wþþwfðk 1Þc þ g ðk 1Þ ½ð1 wþþwfðk 1Þc þ gš þ ðk 1Þ½ð1 wþþwkaš. (16) similar calculation shows that the half-line of -players shrinks in length y one with a proaility proportional to (up to the same constant) ðk 1Þ½ð1 wþþwfðk 1Þa þ gš 1 ðk 1Þ½ð1 wþþwfðk 1Þa þ gš þ ½ð1 wþþwfðk 2Þc þ 2gŠ. (17) For small w, it is easy to see that proaility (16) eing smaller than proaility (17) is equivalent to the ESS conition for D upating (14). The ESS conitions for D an IM upating rules, (13) an (15), can e erive in the same way. 6. Intuitive counting over one conteste ege There is a simple, intuitive way to erive the ESS conitions on graphs (13) (15). gain we consier the tip of the half-line of players. The trick to erive the ESS conitions (13) (15), is to sum up the payoffs of all players involve in the movement of an ege extening from the tip (¼ a ounary etween an players), ut separately for -players an for -players. Then we compare these total payoffs to see which replacement is more likely to occur at the ounary. If the total payoff of strategy excees that of, then strategy is an ESS on graphs. Fig. 3 shows the calculation for D upating. The two players at the ounary are involve in the contest (they are marke with circles). The -player has payoff ðk 1Þþ. The -player has payoff ðk 1Þc þ. Comparison of these two payoffs immeiately leas to the ESS conition for D upating (13). Fig. 4 escries D upating. We focus on the movement of the ounary which is shown as a otte line. There are two possiilities (i) If the -player at the ounary ies (top panel), his ðk 1Þ -neighors an one -neighor compete for the irth-eath (D) upating (k 1)c + (k 1)a + is an ESS on graphs if (k - 1)a + >(k 1)c + payoff: {(k 1)a + } 1 payoff: {(k 1)c + } 1 Fig. 3. simple way to reprouce the ESS conition for D upating. The focal ounary is rawn in a otte line. Those who are involve in the movement of the ounary are marke in circles. Payoffs are shown next to players. ka Death-irth (D) upating is an ESS on graphs if payoff: {(k - 1)a + } 1 payoff: {(k - 2)c + 2} 1 Fig. 4. simple way to reprouce the ESS conition for D upating. The focal ounary is rawn in a otte line. Those who are involve in the movement of the ounary are marke with circles. Payoffs are shown next to players. Top: when the -player at the ounary ies. ottom: when the -player at the ounary ies. ka Imitation (IM) upating payoff: payoff: is an ESS on graphs if payoff: payoff: {(k - 1)a + } 2 {(k - 1)c + } 1 {(k - 2)c + 2} 1 Fig. 5. simple way to reprouce the ESS conition for IM upating. The focal ounary is rawn in a otte line. Those who are involve in the movement of the ounary are marke with circles. Payoffs are shown next to players. Top: when the -player at the ounary ies. ottom: when the -player at the ounary ies. empty site. Each of the ðk 1Þ -neighors has payoff ka, an has one -opponent. The -player has payoff ðk 1Þc þ. He has ðk 1Þ -opponents. Therefore, his weight is k 1. (ii) If the -player at the ounary ies (ottom panel), one -neighor an one -neighor (marke with circles) are relevant for the movement of the ounary. Notice that other -neighors are NOT involve in the movement of the ounary ecause if one of them replaces the vacancy a ifferent ounary moves. The -player has payoff ðk 1Þa þ an has one

6 RTICLE IN PRESS H. Ohtsuki, M.. Nowak / Journal of Theoretical iology 251 (2008) opponent. The -player has payoff ðk 2Þc þ 2 an has one -opponent. Comparing the two weighte total payoffs reprouces our ESS conition for D upating (14). Fig. 5 shows how to erive the ESS conition for IM upating. 7. Numerical examples an computer simulations 7.1. Examples We will now stuy some examples, which eluciate the ifference etween ESS in well mixe populations an on sparse graphs. Let us first consier the following game: 1 3 : 2 0 (18) In this example, strategy is not an ESS in a well-mixe population. However, it is an ESS on a regular graph of egree k ¼ 3 for all upate rules (D, D, anim). The reason for this iscrepancy etween well-mixe populations an graph-structure populations can e unerstoo as follows. Imagine that strategy ominates the population an that a -mutant tries to invae it. The average payoff of the -mutant per game is 2. In a well-mixe population, the average payoff of players is 1. Hence, strategy is not evolutionarily stale. However, in a graph-structure population, the payoff of the -neighors of the -mutant affects selection. These -neighors gain payoff, 3, from the interaction with the -mutant, which excees the mutant s payoff, 2. Therefore, the -mutant fails to invae the population. The game (18) is an example where strategy is not a Nash equilirium, ut is an ESS on graphs. s a secon example, consier the game: 2 0 : 1 3 (19) Strategy is an ESS in a well-mixe population. However, it is not an ESS on a regular graph of egree k ¼ 3 for all three upate rules (D, D, an IM). Imagine that players ominate the population. Their payoff per game is 2. When a mutant invaes the population, his payoff per game is 1. Therefore, strategy resists invasion y strategy in a well-mixe population. However, we notice that players in the neighorhoo of the invaing -player gain payoff, 0, from the game with. Strategy consieraly reuces the payoff of its - neighors. On graphs it can e aaptive to weaken neighors with whom one competes (Nakamaru et al., 1997; Nakamaru an Iwasa, 2005). The payoff matrix (19) is an example for a game where strategy is a strict Nash equilirium, ut is not an ESS on graphs. s a thir example, consier the Prisoner s Dilemma game (Rapoport an Chammah, 1965; xelro an Hamilton, 1981). Cooperation (C) costs c for the onor an yiels enefit,, for the recipient. Defection (D) yiels zero payoff to oth players. The payoff matrix is C D C c c (20) : D 0 In a well-mixe population, efection (D) is the unique ESS of the game. It is also true for games on graphs uner D upating. However, as is shown in Ohtsuki an Nowak (2006), cooperation is the only ESS on a graph of egree k uner D upating whenever =c4k is satisfie. For IM upating, =c4k þ 2 is the ecisive conition for cooperation to e the unique ESS on graphs. For the more general Prisoner s Dilemma payoff matrix, C C R D T D S ; P (21) where T4R4P4S, it is possile that oth cooperation an efection are ESSs on graphs. It is also possile that neither of them is an ESS on graphs (see Ohtsuki an Nowak, 2006) epening on the parameter values an the egree of the graph, k Computer simulations In orer to test the valiity of our analytic results, Eqs. (13) (15), we have run extensive computer simulations for each of the three upate rules. We have stuie ranom regular graphs of egrees k ¼ 3; 4; 5; 6 an lattices of egrees k ¼ 3; 4; 6. We stuy a population of size N ¼ t the eginning of each run, ranom vertices are change from to. We run simulations for five generations, or equivalently, 5N asynchronous upating steps. This means that each player in the population experiences, on average, five potential upating events. fter five generations, we count the numer of players in the population. We conucte simulations 10 4 times for each set of parameters. Each ata point in our result represents the average numer of players after five generations over 10 4 runs. In stuying ranom regular graphs, we generate a new graph every 10 2 runs, in orer to avoi the effect of a particular configuration of a graph. Throughout our simulations, we use w ¼ 0:01. Therefore we have Nw ¼, which meets the requirement of Nw1. Fig. 6 shows the result of computer simulations for ranom regular graphs. Note that the analytic preictions, Eqs. (13) (15), agree well with the simulations. Each

7 RTICLE IN PRESS 704 H. Ohtsuki, M.. Nowak / Journal of Theoretical iology 251 (2008) k=3 k=4 k=5 k=6 D upating D upating IM upating numer of players numer of players numer of players numer of players numer of players numer of players Fig. 6. Computer simulation results for ranom regular graphs of egrees k ¼ 3 (lue), k ¼ 4 (green), k ¼ 5 (orange), an k ¼ 6 (re). Each column of panels in the figure represents one of the three upate rules, D (panels in the left), D (panels in the center), an IM (panels in the right). In the top three panels, we stuy the parameter ða; ; c; Þ ¼ð0; ; 1; 0Þ so is the only free parameter, which is shown in each x-axis. In the ottom three, we stuy the parameters ða; ; c; Þ ¼ð1; 0; 0; Þ, so is the free parameter shown in each x-axis. The y-axis of each panel represents the average numer of players after five generations over 10 4 runs. In each small panel, the initial numer of players, that is n ¼, is shown y the lack horizontal otte line. Thus, the numer of players after five generations eing elow this threshol, n ¼, implies that strategy is an ESS. For each egree k, simulation results are plotte in a corresponing color with x -symols. colore vertical otte line represents the theoretical preiction, Eqs. (22) an (23). The simulation ata show a perfect agreement with theoretical preictions if oth ata plots an a vertical otte line in the same color intersect with the lack horizontal otte line exactly at the same point. column of panels in Fig. 6 correspons to one of the three upate rules, D (left), D (center), an IM (right). In the upper three panels, we stuy the payoff matrix, ða; ; c; Þ ¼ð0; ; 1; 0Þ. Eqs. (13) (15) preict that strategy is an ESS if D: 4k 1, D: 4k 2 k 1, IM: 4 k2 þ k 3. (22) 3 In the lower three panels, we stuy ða; ; c; Þ ¼ð1; 0; 0; Þ. In this case our preictions, Eqs. (13) (15), tell us that strategy is an ESS if ok 1 (23) for all three upate rules. In Fig. 7, we show the results of computer simulations for lattices of egrees k ¼ 3 (triangular), k ¼ 4 (square), an k ¼ 6 (hexagonal). We fin very goo agreement etween the simulations an the theoretical preictions for the triangular lattice, fairly goo for the square lattice, ut no goo agreement for the hexagonal lattice. The reason for this eviation can e unerstoo as follows. The calculations of Ohtsuki an Nowak (2006) are ase on pair approximation, which is mathematically correct for ethe lattices ð¼ Cayley treesþ, that have no loops. However, oth ranom regular graphs an lattices contain many loops. The existence of loops causes a iscrepancy etween simulation results an analytic conitions. The precision of the pair approximation epens on the length, L, of the existing loops. The smaller L is, the worse pair approximation tens to e. For ethe lattices we have L ¼1. Thus pair approximation is correct. For large ranom regular graphs, L is usually very large. For triangular, square, an hexagonal lattices, however, there are many loops with length L ¼ 6; 4 an 3, respectively (see Fig. 8). Therefore, for those lattices we expect that preictions ase on pair approximation o not work well.

8 RTICLE IN PRESS H. Ohtsuki, M.. Nowak / Journal of Theoretical iology 251 (2008) k=3 k=4 k=6 D upating D upating IM upating numer of players numer of players numer of players numer of players numer of players numer of players Fig. 7. The average numer of players after five generations, for lattices of egrees k ¼ 3 (triangular, in lue), k ¼ 4 (square, in green), an k ¼ 6 (hexagonal, in re). ll the others conitions, such as upate rules an parameters use, are the same an in the same orer as in Fig. 6. We fin that results for hexagonal lattices (in re) show a particularly poor agreement with theoretical preictions. triangular, L=6 square, L=4 hexagonal, L=3 Fig. 8. The length of the minimal loop, L, is shown for triangular, square, an hexagonal lattices, respectively. 8. Conclusion We have erive ESS conitions for games on regular graphs of egree k. resient strategy,, can resist invasion y a small fraction of players if D upating: ðk 1Þa þ 4ðk 1Þc þ, D upating: ðk 2 1Þa þ 4ðk 2 k 1Þc þðk þ 1Þ, IM upating: ðk 2 þ 2k 3Þa þ 34ðk 2 þ k 3Þc þðk þ 3Þ. (24) The parameters, a; ; c;, enote the entries of the payoff matrix, (1), which efines the game etween strategies an. The ESS conitions (24) hol for a weak intensity of selection, 0ow51, an for infinitely large population size. For well-mixe populations, which are given y the complete graph, k!1, all three conitions converge to a4c. Thus, for infinite k, the ecisive criterion is what oes the resient get from itself, a, compare to what oes the invaer get from the resient, c. ut for finite k, the payoff values an also matter; it is crucial to know, what the resient gets from the invaer,, an what the invaer gets from itself,. For D upating, a is as important as c (oth parameters have the same weight, k 1, in the ESS conition), an is as important as. For D an IM upating, however, a is more important than c,an is less important than. For all three upate rules, we fin that a an c enter into the ESS conitions with greater weights than an. The traitional ESS criterion of well-mixe populations is neither necessary nor sufficient to guarantee evolutionary staility in structure populations. cknowlegments Support from the John Templeton Founation an the NSF/NIH joint program in mathematical iology (NIH Grant R01GM078986) is gratefully acknowlege. The

9 706 RTICLE IN PRESS H. Ohtsuki, M.. Nowak / Journal of Theoretical iology 251 (2008) Program for Evolutionary Dynamics at Harvar University is sponsore y Jeffrey Epstein. References xelro, R., Hamilton, W.D., The evolution of cooperation. Science 211, inmore, K., Game Theory an the Social Contract, Vol. 1. Playing Fair. MIT Press, Camrige. Durrett, R., Levin, S.., The importance of eing iscrete (an spatial). Theor. Popul. iol. 46, Durrett, R., Levin, S.., llelopathy in spatially istriute populations. J. Theor. iol. 185, Durrett, R., Levin, S.., Spatial aspects of interspecific competition. Theor. Popul. iol. 53, Ellison, G., Learning, local interaction, an coorination. Econometrica 61, Epperson,.K., Geographical Genetics. Princeton University Press, Princeton. Eshel, I., Samuelson, L., Shake,., ltruists, egoists, an hooligans in a local interaction moel. m. Econ. Rev. 88, Eshel, I., Sansone, E., Shake,., The emergence of kinship ehavior in structure populations of unrelate iniviuals. Int. J. Game Theory 28, Ficici, S., Pollack, J., Effects of finite populations on evolutionary stale strategies. In: Whitley, D. (E.), Proceeings of the 2000 Genetic an Evolutionary Computation Conference. Morgan-Kaufmann, San Francisco, pp Fuenerg, D., Tirole, J., Game Theory. MIT Press, Camrige. Haraguchi, Y., Sasaki,., The evolution of parasite virulence an transmission rate in a spatially structure population. J. Theor. iol. 203, Hassell, M.P., Comins, H.N., May, R.M., Species coexistence an self-organizing spatial ynamics. Nature 370, Hauert, C., Funamental clusters in spatial 2 2 games. Proc. R. Soc. Lon. 268, Hauert, C., Doeeli, M., Spatial structure often inhiits the evolution of cooperation in the snowrift game. Nature 428, Hauert, C., De Monte, S., Hofauer, J., Sigmun, K., Volunteering as re queen mechanism for cooperation in pulic goos game. Science 296, Herz,.V.M., Collective phenomena in spatially extene evolutionary games. J. Theor. iol. 169, Hofauer, J., Sigmun, K., Evolutionary Games an Population Dynamics. Camrige University Press, Camrige, UK. Hofauer, J., Schuster, P., Sigmun, K., note on evolutionarily stale strategies an game ynamics. J. Theor. iol. 81, Ifti, M., Killingack, T., Doeeli, M., Effects of neighourhoosize an connectivity on the spatial Continuous Prisoner s Dilemma. J. Theor. iol. 231, Irwin,., Taylor, P.D., Evolution of altruism in a stepping-stone population with overlapping generations. Theor. Popul. iol. 60, Iwasa, Y., Nakamaru, M., Levin, S.., llelopathy of acteria in a lattice population: competition etween colicin-sensitive an colicinproucing strains. Evol. Ecol. 12, Keeling, M.J., The effects of local spatial structure on epiemiological invasions. Proc. R. Soc. Lon. 266, Killingack, T., Doeeli, M., Spatial evolutionary game theory: hawks an oves revisite. Proc. R. Soc. Lon. 263, Kimura, M., Stepping stone moel of population. nn. Rep. Nat. Inst. Genetics (Japan) 3, Kimura, M., Weiss, G.H., The stepping stone moel of population structure an the ecrease of genetical correlation with istance. Genetics 49, Le Galliar, J., Ferrière, R., Dieckman, U., The aaptive ynamics of altruism in spatially heterogeneous populations. Evolution 57, Lehmann, L., Keller, L., Sumpter, D.J.T., The evolution of helping an harming on graphs: the return of the inclusive fitness effect. J. Evol. iol. 20, Levin, S.., Dispersion an population interactions. m. Nat. 108, Levin, S.., Paine, R.T., Disturance, patch formation, an community structure. Proc. Natl ca. Sci. US 71, Lieerman, E., Hauert, C., Nowak, M.., Evolutionary ynamics on graphs. Nature 433, Lingren, K., Norahl, M.G., Evolutionary ynamics of spatial games. Physica D 75, Luce, R.D., Raiffa, H., Games an Decision. Wiley, New York. Maruyama, T., On the fixation proaility of mutant genes in a suivie population. Gen. Res. 15, Maruyama, T., n invariance property of a structure population. Gen. Res. 18, Matsua, H., Tamachi, N., Sasaki,., Ogita, N., lattice moel for population iology. In: Teramot, E., Yamaguti, M. (Es.), Mathematical Topics in iology, Morphogenesis an Neurosciences. Springer Lecture Notes in iomathematics, vol. 71, pp Matsua, H., Ogita, N., Sasaki,., Sato, K., Statistical mechanics of population the lattice Lotka Volterra moel. Prog. Theor. Phys. 88, May, R.M., Network structure an the iology of populations. Trens. Ecol. Evol. 21, Maynar Smith, J., The theory of games an the evolution of animal conflicts. J. Theor. iol. 47, Maynar Smith, J., Evolution an the Theory of Games. Camrige University Press, Camrige. Maynar Smith, J., Can a mixe strategy e stale in a finite population? J. Theor. iol. 130, Maynar Smith, J., Price, G., The logic of animal conflicts. Nature 246, Nagylaki, T., Theoretical Population Genetics. Springer, erlin. Nakamaru, M., Iwasa, Y., The evolution of altruism y costly punishment in the lattice structure population: score-epenent viaility versus score-epenent fertility. Evol. Ecol. Res. 7, Nakamaru, M., Matsua, H., Iwasa, Y., The evolution of cooperation in a lattice structure population. J. Theor. iol. 184, Nakamaru, M., Nogami, H., Iwasa, Y., Score-epenent fertility moel for the evolution of cooperation in a lattice. J. Theor. iol. 194, Nash, J.F., Equilirium points in n-person games. Proc. Natl ca. Sci. US 36, Neill, D.., Evolutionary staility for large populations. J. Theor. iol. 227, Neuhauser, C., Mathematical challenges in spatial ecology. Not. m. Math. Soc. 48, Nowak, M.., Evolutionary Dynamics. Harvar University Press, M. Nowak, M.., May, R.M., Evolutionary games an spatial chaos. Nature 359, Nowak, M.., May, R.M., The spatial ilemmas of evolution. Int. J. ifur. Chaos 3, Nowak, M.., Sigmun, K., Evolutionary ynamics of iological games. Science 303, Nowak, M.., onhoeffer, S., May, R.M., More spatial games. Int. J. ifurc. Chaos 4, Nowak, M.., Sasaki,., Taylor, C., Fuenerg, D., Emergence of cooperation an evolutionary staility in finite populations. Nature 428, Ohtsuki, H., Nowak, M.., 2006a. Evolutionary games on cycles. Proc. R. Soc. 273, Ohtsuki, H., Nowak, M.., The replicator equation on graphs. J. Theor. iol. 243,

10 RTICLE IN PRESS H. Ohtsuki, M.. Nowak / Journal of Theoretical iology 251 (2008) Ohtsuki, H., Hauert, C., Lieerman, E., Nowak, M.., simple rule for the evolution of cooperation on graphs an social networks. Nature 441, Ohtsuki, H., Nowak, M.., Pacheco, J., 2007a. reaking the symmetry etween interaction an replacement in evolutionary ynamics on graphs. Phys. Rev. Lett. 98, Ohtsuki, H., Pacheco, J., Nowak, M.., Evolutionary graph theory: reaking the symmetry etween interaction an replacement. J. Theor. iol. 246, Pacheco, J.M., Traulsen,., Nowak, M.., 2006a. ctive linking in evolutionary games. J. Theor. iol. 243, Pacheco, J.M., Traulsen,., Nowak, M.., Coevolution of strategy an structure in complex networks with ynamical linking. Phys. Rev. Lett. 97, Pastor-Satorras, R., Vespignani,., Epiemic spreaing in scalefree networks. Phys. Rev. Lett. 86, Rapoport,., Chammah,.M., Prisoners Dilemma: Stuy in Conflict an Cooperation. The University of Michigan Press, nn ror, MI, US. Rousset, F., Genetic Structure an Selection in Suivie Populations. Princeton University Press, Princeton. Samuelson, L., Evolutionary Games an Equilirium Selection. MIT Press, Camrige. Santos, F.C., Pacheco, J.M., Scale-free networks provie a unifying framework for the emergence of cooperation. Phys. Rev. Lett. 95, Santos, F.C., Pacheco, J.M., Lenaerts, T., Evolutionary ynamics of social ilemmas in structure heterogeneous populations. Proc. Natl ca. Sci. US 103, Schaffer, M., Evolutionarily stale strategies for a finite population an a variale contest size. J. Theor. iol. 132, Szao, G., Fa th, G., Evolutionary games on graphs. Phys. Rep. 446, Szao, G., Hauert, C., Phase transitions an volunteering in spatial pulic goos games. Phys. Rev. Lett. 89, Szao, G., T+oke, C., Evolutionary prisoner s ilemma game on a square lattice. Phys. Rev. E 58, Szao, G., ntal, T., Szao, P., Droz, M., Spatial evolutionary prisoner s ilemma game with three strategies an external constraints. Phys. Rev. E 62, Szaó, G., Vukov, J., Szolnoki,., Phase iagrams for an evolutionary prisoner s ilemma game on two-imensional lattices. Phys. Rev. E 72, Tainaka, K., Vortices an strings in a moel ecosystem. Phys. Rev. E 50, Taylor, P.D., ltruism in viscous populations an inclusive fitness approach. Evol. Ecol. 6, Taylor, P.D., Irwin,.J., Overlapping generations can promote altruistic ehavior. Evolution 54, Taylor, P.D., Jonker, L., Evolutionary stale strategies an game ynamics. Math. iosci. 40, Taylor, P.D., Day, T., Wil, G., Evolution of cooperation in a finite homogeneous graph. Nature 447, Tilman, D., Karieva, P. (Es.), Spatial Ecology: The Role of Space in Population Dynamics an Interspecific Interactions. Monographs in Population iology. Princeton University Press, Princeton, NJ. Traulsen,., Pacheco, J.M., Imhof, L.., Stochasticity an evolutionary staility. Phys. Rev. E 74, van aalen, M., Pair approximations for ifferent spatial geometries. In: Dieckmann, U., Law, R., Metz, J..J. (Es.), The Geometry of Ecological Interactions. Camrige University Press, Camrige, UK, pp van aalen, M., Ran, D.., The unit of selection in viscous populations an the evolution of altruism. J. Theor. iol. 193, Weiull, J., Evolutionary Game Theory. MIT Press, Camrige. Wil, G., Taylor, P.D., Fitness an evolutionary staility in game theoretic moels of finite populations. Proc. R. Soc. Lon. 271, Wilson, D.S., Pollock, G.., Dugatkin, L.., Can altruism evolve in purely viscous populations? Evol. Ecol. 6, Wootton, J.T., Local interactions preict large-scale pattern in empirically erive cellular automata. Nature 413, Wright, S., Isolation y istance. Genetics 28, Zeeman, E.C., 19. Population ynamics from game theory. In: Nitecki,., Roinson, C. (Es.), Proceeings of an International Conference on Gloal Theory of Dynamical Systems. Lecture Notes in Mathematics, vol Springer, erlin.