Repeated games and direct reciprocity under active linking

Transcription

1 Journal of Theoretical iology 25 (28) eeated games and direct recirocity under active linking Jorge M. Pacheco a,, Arne Traulsen b, Hisashi Ohtsuki b, Martin A. Nowak b,c a ATP-Grou and FT, eartamento de Física da Faculdade de iências, P Lisboa odex, Portugal b Program for Evolutionary ynamics, Harvard University, ambridge, MA 238, USA c eartment of Organismic and Evolutionary iology, eartment of Mathematics, Harvard University, ambridge, MA 238, USA eceived 7 Setember 27; received in revised form 3 October 27; acceted 3 October 27 Available online 6 November 27 Abstract irect recirocity relies on reeated encounters between the same two individuals. Here we examine the evolution of cooeration under direct recirocity in dynamically structured oulations. Individuals occuy the vertices of a grah, undergoing reeated interactions with their artners via the edges of the grah. Unlike the traditional aroach to evolutionary game theory, where individuals meet at random and have no control over the frequency or duration of interactions, we consider a model in which individuals differ in the rate at which they seek new interactions. Moreover, once a link between two individuals has formed, the roductivity of this link is evaluated. Links can be broken off at different rates. Whenever the active dynamics of links is sufficiently fast, oulation structure leads to a simle transformation of the ayoff matrix, effectively changing the game under consideration, and hence aving the way for recirocators to dominate defectors. We derive analytical conditions for evolutionary stability. r 27 Elsevier Ltd. All rights reserved. Keywords: Evolutionary game theory; Structured oulations; oevolution; ynamically structured oulations. Introduction Game theoretic ideas were first introduced to biology by Hamilton (964) and Trivers (97), but the field of evolutionary game theory was founded by Maynard Smith and Price (973) and Maynard Smith (982). The relicator equation (Taylor and Jonker, 978; Hofbauer et al., 979; Zeeman, 98) constitutes the mathematical foundation of evolutionary game dynamics. It is a system of ordinary differential equations describing how the relative abundances (frequencies) of strategies change over time as a consequence of frequency-deendent selection. The ayoff from the game is interreted as biological fitness. Individuals reroduce roortional to their fitness. eroduction can be genetic or cultural. The exected ayoff of an individual is a linear function of the frequencies of all strategies; the coefficients of this function are the entries of the ayoff matrix. For detailed reviews of the relicator equation and other aroaches to evolutionary game orresonding author. Tel.: ; fax: address: acheco@cii.fc.ul.t (J.M. Pacheco). dynamics, see Fudenberg and Tirole (99), Weibull (995), Samuelson (997), Hofbauer and Sigmund (998, 23), Gintis (2), owles (23), ressman (23), Nowak and Sigmund (24) and Nowak (26a). The act of cooeration tyically involves a cost c to the rovider and a benefit b to the reciient. In the absence of a secific mechanism for the evolution of cooeration, natural selection favors defectors. There are at least five mechanisms that can lead to the evolution of cooeration: kin selection, grou selection, direct recirocity, indirect recirocity and network recirocity ð¼ grah selectionþ. In this aer, we study the interaction between direct and network recirocity; however, unlike conventional network recirocity, as defined in Nowak (26b), here the network is adative, as discussed below. The study of the evolution of cooeration under direct recirocity on dynamical networks deserves secial attention, given the recent results which show that co-evolution of oulation structure with individual strategy rovide an efficient mechanism for the evolution of cooeration under simle one-shot games (Pacheco et al., 26a, b; Santos et al., 26c) /$ - see front matter r 27 Elsevier Ltd. All rights reserved. doi:.6/j.jtbi.27..4

2 724 J.M. Pacheco et al. / Journal of Theoretical iology 25 (28) irect recirocity is based on the idea of reeated encounters between two individuals (Trivers, 97) according to the rincile, I scratch your back and you scratch mine. The game theoretic framework of direct recirocity is the reeated Prisoner s ilemma (P), which has been the subject of numerous studies across various discilines (aoort and hammah, 965; Axelrod and Hamilton, 98; Axelrod, 984; Selten and Hammerstein, 984; Milinski, 987; May, 987; Axelrod and ion, 988; Fudenberg and Maskin, 99; Imhof et al., 25). A large number of strategies for laying the reeated P have been analyzed. The most rominent ones are tit-for-tat (Axelrod, 984), generous-tit-for-tat (Nowak and Sigmund, 992), contrite-tit-for-tat (Sugden, 986; oerlijst et al., 997) or win-stay, lose-shift (Nowak and Sigmund, 993). In general, it is a very difficult task to find successful strategies for laying the reeated P (Axelrod, 984; Kraines and Kraines, 988; Fudenberg and Maskin, 99; Lindgren, 99). ut if what we want is to investigate if cooeration has any chance to evolve by direct recirocity at all, then a very simle game can be studied. We only need to consider two strategies: unconditional defectors (), defect all the time; recirocators () start cooerating and then continue to cooerate as long as the oonent cooerates, but defect if the oonent defects. Such individuals can be thought of as laying a strategy like tit-for-tat or Grim. Tit-for-tat cooerates on the first move and then does whatever the oonent has done on the revious move. Grim cooerates until the oonent defects once and then ermanently switches to defection. esite the difference between these two strategies, when laying against an unconditional defector, tit-for-tat and grim lead to the same sequence of cooeration in the first round and unconditional defection from then on. Only if errors or more comlex strategy sets are considered, differences between the strategies arise. Hence, a recirocator will only cooerate once against a defector and will behave as an unconditional cooerator against another recirocator. Let us denote by w the robability of laying another round. The average number of rounds between the same two layers is given by =ð wþ. The ayoff matrix for recirocators () versus unconditional defectors () is given by b w c A; b that is, recirocators ay the cost c once, and unconditional defectors receive the benefit b only once. One-shot and reeated games on satial lattices have been studied by many authors (Nowak and May, 992, 993; Wilson et al., 992; Nowak et al., 994; Lindgren and Nordahl, 994; Killingback and oebeli, 996; Nakamaru et al., 997, 998; van aalen and and, 998; Szabo and T+oke, 998; Hauert et al., 22; Szabo and Hauert, 22; () randt et al., 23; Hauert and oebeli, 24; Hauert and Szabo, 25; Szabo et al., 25; Nowak, 26a; Szabo and Fa th, 27). Evolutionary grah theory is an extension of this aroach to general oulation structure and networks (Lieberman et al., 25; Pacheco and Santos, 25; Santos and Pacheco, 25; Santos et al., 25, 26a, b; Santos and Pacheco, 26; Ohtsuki and Nowak, 26a, b, 27; Ohtsuki et al., 26, 27a, b; Pacheco et al., 26a, b). It is usually assumed that the oulation structure is constant in the time scale of the evolutionary udating. ecently, Ohtsuki and Nowak (27) have investigated the evolutionary feasibility of cooeration under direct recirocity for static networks. The combination of direct recirocity with (static) network recirocity was shown to oen the way for recirocators to invade (even when rare) unconditional defectors, which is never ossible in a wellmixed oulation. The effect of network recirocity is strongest if eole have few neighbors (or if most interactions occur only with a subset of very close friends ). In many real-world social and biological networks (Amaral et al., 2; orogovtsev and Mendes, 23; May, 26; Santos et al., 26c); however, the average connectivity of individuals is not small. In addition to static networks, one-shot-games on dynamical grahs have also been investigated (ala and Goyal, 2; Skyrms and Pemantle, 2; Zimmermann et al., 24; Eguı luz et al., 25; Santos et al., 26c). It has been recently shown (Pacheco et al., 26a, b; Santos et al., 26c) that the limitation to small connectivity may be overcome if one evolves simultaneously individual strategy and oulation structure. Here we investigate the imact of co-evolution of strategy and structure in the evolution of cooeration under direct recirocity. In Section 2 we introduce relevant concets of evolutionary game dynamics in finite and infinite oulations, as well as results related to direct recirocity in well-mixed oulations. In Section 3 we introduce the model of active linking dynamics, in which individuals seek new artners and break existing ties at different rates. In Sections 4 and 5 we discuss our results for direct recirocity on dynamical grahs. In Section 6 we offer conclusions. 2. Evolutionary stability and risk-dominance in well-mixed oulations onsider a game between two strategies, A and, given by the ayoff matrix A A AA A (2) : A An infinitely large oulation of A layers cannot be invaded by layers if AA 4 A, that is, A is both a strict Nash equilibrium and an evolutionarily stable strategy (ESS). In an infinite well-mixed oulation, both strategies are ESS whenever AA 4 A and A o. The relicator

3 J.M. Pacheco et al. / Journal of Theoretical iology 25 (28) equation (Taylor and Jonker, 978; Hofbauer et al., 979; Weibull, 995; Hofbauer and Sigmund, 998) admits an unstable mixed equilibrium, located at x ¼ð A Þ= ð AA A A þ Þ, where x is the equilibrium frequency of A layers in the oulation. Strategy A is risk-dominant () if it has the bigger basin of attraction, that is, whenever AA þ A 4 A þ. In finite, well-mixed oulations, a crucial quantity is the fixation robability of a strategy, that is, the robability that the lineage arising from a single mutant of that strategy will take over the entire oulation (Nowak et al., 24; Taylor et al., 24). If AA þ 2 A 4 A þ 2 then the fixation robability of strategy A is greater than the fixation robability of a neutral mutant ð=nþ. Tobemore secific, the roduct of oulation size and the intensity of selection has to be small (Traulsen et al., 26b). This means selection favors the relacement of by A, and therefore a single A-layer in a oulation of -layers is an advantageous mutant. The condition can be exressed as a 3-rule: if the fitness of the invading A at a frequency of 3 is greater than the fitness of the resident then the fixation robability of A is greater than =N (Nowak et al., 24; Imhof and Nowak, 26; Ohtsuki et al., 27c). This condition holds in the limit of weak selection where the ayoff from the game is small comared to a constant background fitness (Traulsen et al., 26a; 27a,b). Furthermore, if A is comared to, then the fixation robability of A is greater than the fixation robability of for weak selection and large oulation size (Nowak et al., 24; Imhof and Nowak, 26). Given the ayoff matrix associated with direct recirocity, Eq. (), we can immediately write down the following conditions (Ohtsuki and Nowak, 27): The recirocator strategy is an ESS if b c 4 w. (3a) In this case, a defector in an infinitely large oulation of cooerators has a lower fitness. The unstable fixed oint is located at x ¼ c w b c w. (3b) In a finite oulation, however, it is still ossible that the fixation robability of a single defector, r, is greater than that of a neutral mutant ð=nþ. Hence, if we want defectors to be disadvantageous, we must require that r o=n. For weak selection and large oulation size the condition reads (Ohtsuki and Nowak, 27) b c 4 3 w 2w. (3c) In this case, the basin of attraction of recirocators is greater than 3. ecirocators become when b c 4 2 w w, (3d) that is, r 4r for large oulations and weak selection. Finally, recirocators become advantageous if r 4=N; for large oulations and weak selection, this is equivalent to (Ohtsuki and Nowak, 27) b 3 2w 4 c w. (3e) 3. asic model and transformation of ayoff matrices Let us study a game between two strategies, A and,in a oulation of fixed size, N. There are N A layers who use strategy A, andn layers who use strategy. 3.. Unconditional strategies in finite, well-mixed oulations First consider the case without dynamical linking or conditional strategies. Strategies A and are unconditional and ure strategies of the 2 2 game with ayoff matrix A A AA A (4) : A In each round of the game, A layers choose action A, and layers choose action. Suose that layers kee laying the game with all other layers simultaneously. Each A-layer interacts with N A many A-oonents and N many -oonents. Each -layer interacts with N A many A-oonents and N many -oonents. When it takes an amount of time for layers to comlete a round of game, the ayoffs er unit time are calculated as W A ¼ðN A Þ AA W ¼ N A A þ N A, þðn Þ. ð5þ If N A and N are large, we can neglect in Eq. (5) and we obtain W A ¼ N AA A xa. (6) W A x Here x A and x reresent relative abundances of strategies, A and, namely, x A ¼ N A =N, x ¼ N =N, such that x A þ x ¼ Unconditional strategies in oulations with dynamical linking Next we incororate the effect of dynamical linking into the ayoff matrix. onsider two layers in the oulation. These layers are able to lay games only when there is a link between them. It is ossible for a layer to have multile links and to lay games with different artners at the same time. Let f ij reresent the average fraction of time a link is resent between an ið¼ A; Þ-layer and a jð¼ A; Þ-layer. In this case, the ayoffs er unit

4 726 J.M. Pacheco et al. / Journal of Theoretical iology 25 (28) time become W A ¼ðN A Þf AA AA W ¼ N A f A A We have W A W þ N f A A, þ N f. ð7þ ¼ N f AA AA f A A f A A f xa x. (8) Eq. (8) suggests that the linking dynamics introduces a simle transformation of the ayoff matrix. We can study standard evolutionary game dynamics using the modified ayoff matrix (Pacheco et al., 26a, b). The fractions of time that different tyes of links are active, f, are calculated as follows. Links are formed at certain rates and have secific life-times. enote by XðtÞ the number of AA links at time t. Similarly, Y ðtþ and ZðtÞ denote the number of A and links at time t. The maximum ossible number of AA, A and links is, resectively, given by X m ¼ N A ðn A Þ=2, Y m ¼ N A N, Z m ¼ N ðn Þ=2. ð9þ Suose A and layers have a roensity to form new links denoted by a A and a, such that AA links are formed at a rate a 2 A, A links are formed at a rate a Aa and links are formed at a rate a 2. Also suose that the average life-times of links are given by t AA, t A and t ðb Þ. Linking dynamics can then be described by a system of three ordinary differential equations for the number of links (Pacheco et al., 26a, b): _X ¼ a 2 A ðx m XÞ X, t AA _Y ¼ a A a ðy m YÞ Y, t A _Z ¼ a 2 ðz m ZÞ Z. ðþ t In the steady state, the number of links of the three different tyes is given by Examles for cumulative degree distributions of oulation structures attained under steady-state dynamics for different combinations of the relevant arameters are shown in Fig.. Indeed, this simle model of linking dynamics leads to single-scale networks as defined by Amaral et al. (2), with associated cumulative degree distributions exhibiting fast decaying tails (Santos et al., 26c). Such tails which decay exonentially or faster than exonential, leading to what are known as broad-scale and single-scale networks, resectively, are features which, together with a large variability in the average connectivity (orogovtsev and Mendes, 23; May, 26), characterize most real-world social networks. The resent model only encomasses single-scale networks. In order to describe the broad-scale networks often encountered in social systems, more refined models should be develoed. The vertical arrows in Fig. indicate the average connectivity of the associated grahs, showing that connectivity values similar to those measured emirically (orogovtsev and Mendes, 23) are easily obtained with the resent model. Note, in articular, that the deendence of the stationary networks on the frequency of individuals of a given tye will automatically coule network dynamics with the frequency-deendent evolutionary dynamics we introduce in the following. umulative degree distribution egree a2 A t AA X ¼ a 2 A t AA þ X m, Y ¼ a Aa t A a A a t A þ Y m, Z ¼ a2 t a 2 t þ Z m. Hence we may write f AA ¼ X ¼ a2 A t AA X m a 2 A t AA þ, f A ¼ f A ¼ Y ¼ a Aa t A Y m a A a t A þ, f ¼ Z ¼ a2 t Z m a 2 t þ. ðþ ð2þ Fig.. umulative degree distributions (defined as ðkþ ¼ P jxk N j=n, with N j the number of nodes with degree j) for networks generated with the resent model, for oulations of size N ¼ 3 and two different tyes of individuals. The fast decaying tails correlate well with the observed tails of real social networks (Amaral et al., 2; orogovtsev and Mendes, 23; May, 26). The resent model, however, leads to single-scale networks (Amaral et al., 2), broad scale networks being out of its scoe (for details of the degree distributions, see Pacheco et al., 26a). On the other hand, the deendence of the final network on the frequency of each tye of individuals leads to a natural couling between network dynamics and frequency-deendent strategy evolution. The vertical arrows indicate the average connectivity of each grah, which is far greater than those tyically associated with static grahs where cooeration under direct recirocity thrives (Ohtsuki and Nowak, 27). Parameters used: N A =N ¼ :5, a A ¼ a ¼, b AA ¼ b A ¼ b ¼ 5 (red solid curve), N A =N ¼ :35, a A ¼ :, a ¼ :75, b AA ¼ b A ¼ b ¼ 5 (blue dashed curve) and N A =N ¼ :5, a A ¼ a ¼ :2, b AA ¼ b A ¼ b ¼ (black dash dot curve).

5 3.3. onditional strategies in oulations with dynamical linking So far we have assumed that strategies A and are ure strategies in a single game. What if they are strategies in a reeated game? onsider recirocators () and unconditional defectors (). Each time a new link is established, a recirocator cooerates in the first round while an unconditional defector never cooerates. Once a recirocator faces defection by the oonent, he kees defecting until the link is broken. Interactions with two layers last on average for time t. Since it takes time to comlete a round, they lay on average t = rounds of Prisoner s ilemma game within the lifetime of that link. Suose that the ayoff matrix of the single-round Prisoner s ilemma game is W W J.M. Pacheco et al. / Journal of Theoretical iology 25 (28) Under the same assumtion, the average ayoff of defectors er unit time is given by þ t t ¼ þ. (7) t t Taking into account the fraction of time when links are absent, we find that the average ayoffs er unit time of recirocators and unconditional defectors are W ¼ðN Þf þ N f t þ, t W ¼ N f t f f þ ð ¼ Nt t Þ f þ A ð t Þ f þ t þðn Þf. ð8þ Therefore, for large oulations we obtain x x. (9) given by : (3) oth recirocators gain the ayoff of ðt = Þ in time t. Therefore, given a link remains established, a ayoff er unit time is given by t t ¼. (4) t A similar consideration yields that the ayoff er unit time between two unconditional defectors is given by t t ¼. (5) t When a link is established between a recirocator and a defector, the link lasts for an average time t, so that these layers on average lay t = rounds of Prisoner s ilemma game. In the first round, the recirocator cooerates whereas the unconditional defector defects, which yields the ayoff of to the recirocator and to the defector. From the second round on, both kee defecting and gain er round. The average number of rounds of mutual defection is ðt = Þ. Since the whole reeated game takes time t, the average ayoff of recirocators er unit time is, under the assumtion of the link remaining established, given by þ t t ¼ t þ t. (6) In the following, we will study the ayoff f f þ t ð t Þ A f f þ ð t Þ (2) as if associated with the evolutionary dynamics of a wellmixed oulation. emember that f s in (2) are determined by Eq. (2). In addition to the entries of the 2 2 ayoff matrix, we have six arameters in total, a ; a ; t ; t ; t and. 4. esults Let us investigate how the frequencies of strategies and change under evolutionary dynamics. The simultaneous evolution of strategy and structure will deend on the time scales associated with strategy evolution (T) and structural evolution ðt ij Þ (Pacheco et al., 26a, b; Santos et al., 26c). Whenever T5t ij strategies evolve in an immutable network, which leads to the framework investigated by Ohtsuki and Nowak (27). Whenever Tbt ij grah dynamics always attains a steady state before the next strategy udate takes lace. This limit, which has been shown to extend to a range of time scales which is wider than exected (Santos et al., 26c; Pacheco et al., 26b), is the novel one we shall investigate here. In the following, we always assume that 5t ij 5T holds. Fig. 2 illustrates the magnitudes of the different time scales that aear in the resent aer.

6 728 J.M. Pacheco et al. / Journal of Theoretical iology 25 (28) T defectors link lifetime tye of link recirocators Fig. 2. haracteristic time scales associated with direct recirocity under active linking dynamics. We assume that a tyical interaction between two individuals has an average duration. For direct recirocity to be effective, the characteristic duration of links between recirocators ðt Þ, between defectors ðt Þ and between recirocators and defectors ðt Þ should be larger than. Nonetheless, each of this tye of links may have different characteristic lifetimes, as illustrated in the left anel. Thus, the average number of rounds between airs of individuals with different strategies may be different, as well as the average number of links between individuals of different tyes, as illustrated in the right anel. Finally, our analytical results rely on the assumtion that the characteristic time scale of active linking of the order of any of ft ; t ; t g must be much smaller than that associated with strategy evolution (T), as illustrated in the left anel. Let us study a standard Prisoner s ilemma game ¼ b c c (2) b (we rovide the general conditions for the case in which a in Aendix). Suose, for simlicity, that both recirocators and unconditional defectors share the same roensity, a a ¼ a, to form a new link. Matrix (2) simlifies to t t þ a 2ðb cþ t t þ a 2ð cþ t þ a 2b A : (22) Multilying (22) by ðt þ a 2 Þ= gives us s e ðb cþ c ; b where (23) s e ¼ t þ t a 2 þ t a2. (24) 5. iscussion As seen in (23) (comare with Eq. ()), the arameter s e reresents the effective number of rounds of mutual cooeration. The larger the value of s e the easier it is for recirocators to invade the entire oulation under active linking. For fixed a, and t, s e is an increasing function of t, which conveys the message that the more long-lived the links are between recirocators, the better for cooeration. On the other hand, for fixed a, and t, s e is also an increasing function of t. In other words, the longer the lifetime of links between recirocators and defectors, the better for cooeration. This result seems counter-intuitive. However, one may understand it if one considers the tye of interaction on this link in detail. Once an link is established, the recirocator obtains the sucker s ayoff c once. After that, both individuals receive nothing. For the recirocator, it is better to kee this link active than breaking it, since otherwise the link might be re-established again and the defector would exloit him once more. Thus, for recirocators a long lifetime of links is advantageous. If it is an link, the mutual cooeration leads to a higher ayoff. An active link avoids multile acts of exloitation by the defector. We now study how s e behaves with a. When the roensity to form a new link, a, is very small, s e becomes s e t, (25) which is exactly the same as the average number of rounds layed by two recirocators. On the other hand, when a is very large we obtain s e t, (26)

7 J.M. Pacheco et al. / Journal of Theoretical iology 25 (28) which is the average number of rounds layed between a recirocator and an unconditional defector. The feasibility of cooeration relies on the roensity to form new links. When this value is high, s e is determined by the lifetime of recirocator defector links. Since it is often the case in reality that t 4t, we find that the smaller the roensity to establish new links the better for cooeration, given that t contributes more to s e than t. Indeed, when the roensity to form a new link is high, defectors, who tend to lose a link more frequently than recirocators, are able to re-establish the link quickly and exloit a recirocator in a new first round, which is unfavorable for cooeration. When we write s e in terms of the effective discounting factor, w e s e ¼ or w e ¼, (27) w e s e all the results from Eq. (3a) (3e) hold for w ¼ w e, rovided the oulation size N is large such that the underlying mean-field treatment used here remains valid. For examle, the recirocating strategy is an ESS against unconditional defection whenever b c 4 ¼ s e (28) w e s e holds. In this work we took into account the time scale associated with a single round of a reeated game, as well as the lifetimes of different tyes of links, together with the ossibility that existing links are severed and new links are established. As a result, and in the limit in which link dynamics is faster than evolutionary dynamics of strategies, we have obtained a game-theoretical roblem equivalent to a conventional evolutionary game in a wellmixed oulation, with a rescaled ayoff matrix. This equivalence, however, is only mathematical, in the sense that the roblem under consideration does not allow us to regain a well-mixed oulation limit easily. learly, the model introduced here catures some of the stylized features of social networks, in which individuals change their social ties in time, and in which rewarding links tend to last longer than unleasant ones. On the other hand, one may exect that random rewiring does not cature the detailed mechanism(s) underlying social network dynamics (Santos et al., 26c). While the resent model allows one to assess the role of dynamic linking in the evolution of cooeration under direct recirocity, more elaborate models should be considered in order to describe realistic social dynamics. Our model shows that, in what concerns the evolution of cooeration under direct recirocity, the ath to cooeration is facilitated by active linking dynamics. ooeration is most viable when links last long enough and the roensity to form new links is not too high. ertainly this model recovers the message already obtained before that sarse static grahs favor cooeration (Ohtsuki and Nowak, 27). Yet, dynamic linking enlarges the scoe of feasibility of cooeration. 6. onclusions Whenever single round interactions of a Prisoner s ilemma game are swift, and the re-adjustment of different tyes of links occurs much faster than the re-adjustment of strategies, we find that the role of link rewiring dynamics is to introduce a rescaling of the ayoff matrix associated with direct recirocity. The rescaling obtained widens the scoe of feasibility of cooeration already set forward by Ohtsuki and Nowak (27). Without dynamical linking, recirocators mutually cooerate in consecutive rounds in a reeated game, whereas unconditional cooerators take advantage of exloiting recirocators only in the first round. In the traditional framework of studying the iterated Prisoner s ilemma game, one usually assumes that the number of reeated games that one lays is the same among individuals in the oulation, and so is the number of the first round of reeated games. When active rewiring and time scales are exlicitly taken into consideration; however, this homogeneous assumtion is lost, and one must take into consideration the cometition between the lifetime of recirocator recirocator links and recirocator defector links and the rates of link formation. As shown in Fig., arameter values which ensure the feasibility of cooeration under active linking dynamics lead also to social grahs exhibiting realistic features. Active linking oens a way for cooeration by direct recirocity to evolve on these realistic networks. Acknowledgments Suort from FT, Portugal (J.M.P.), the eutsche Akademie der Naturforscher Leooldina (A.T., Grant no. MF-LP 99/8-34), the John Temleton Foundation and the NSF/NIH joint rogram in mathematical biology (NIH Grant GM78986) (M.A.N.) is gratefully acknowledged. The Program for Evolutionary ynamics at Harvard University is sonsored by Jeffrey Estein. Aendix For the general case in which a, Eq. (23) now reads s e Z þð Þ (A.) ; Z þð Þ r e where s e has been defined before, r e ¼ t þ t a 2 þ t a2, and Z ¼ t =. For Prisoner s ilemma we know that Hence, direct recirocity and active linking may

8 73 J.M. Pacheco et al. / Journal of Theoretical iology 25 (28) effectively lead to a coordination game whenever s e 4Z þð Þ and r e 4Z þð Þ. eferences (A.2) (A.3) Amaral, L.A.N., Scala, A., arthelemy, M., Stanley, H.E., 2. lasses of small-world networks. Proc. Natl Acad. Sci. USA 97, 49. Axelrod,., 984. The Evolution of ooeration. asic ooks, New York, USA. Axelrod,., ion,., 988. The further evolution of cooeration. Science 242, Axelrod,., Hamilton, W.., 98. The evolution of cooeration. Science 2, ala, V., Goyal, S., 2. A noncooerative model of network formation. Econometrica 68, oerlijst, M.., Nowak, M.A., Sigmund, K., 997. The logic of contrition. J. Theor. iol. 85, owles, S., 23. Microeconomics: ehavior, Institutions, and Evolution. Princeton University Press, Princeton, NJ. randt, H., Hauert,., Sigmund, K., 23. Punishment and reutation in satial ublic goods games. Proc.. Soc. London 27, ressman,., 23. Evolutionary ynamics and Extensive Form Games. MIT Press, ambridge, USA. orogovtsev, S.N., Mendes, J.F.F., 23. Evolution of Networks: From iological Nets to the Internet and WWW. Oxford University Press, Oxford. Eguíluz, V., Zimmermann, M.G., ela-onde,.j., Miguel, M.S., 25. ooeration and the emergence of role differentiation in the dynamics of social networks. Am. J. Sociol., Fudenberg,., Maskin, E., 99. Evolution and cooeration in noisy reeated games. Am. Econ. ev. 8, Fudenberg,., Tirole, J., 99. Game Theory. MIT Press, ambridge, USA. Gintis, H., 2. Game Theory Evolving. Princeton University Press, Princeton, USA. Hamilton, W.., 964. The genetical evolution of social behavior. J. Theor. iol. 7, 6, Hauert,., oebeli, M., 24. Satial structure often inhibits the evolution of cooeration in the snowdrift game. Nature 428, Hauert,., Szabó, G., 25. Game theory and hysics. Am. J. Phys. 73, Hauert,., e Monte, S., Hofbauer, J., Sigmund, K., 22. Volunteering as red queen mechanism for cooeration in ublic goods game. Science 296, Hofbauer, J., Sigmund, K., 998. Evolutionary Games and Poulation ynamics. ambridge University Press, ambridge, USA. Hofbauer, J., Sigmund, K., 23. Evolutionary game dynamics. ull. Am. Math. Soc. 4, Hofbauer, J., Schuster, P., Sigmund, K., 979. Evolutionary stable strategies and game dynamics. J. Theor. iol. 8, Imhof, L.A., Nowak, M.A., 26. Evolutionary game dynamics in a Wright Fisher rocess. J. Math. iol. 52, Imhof, L.A., Fudenberg,., Nowak, M.A., 25. Evolutionary cycles of cooeration and defection. Proc. Natl Acad. Sci. USA 2, Killingback, T., oebeli, M., 996. Satial evolutionary game theory: Hawks and oves revisited. Proc.. Soc. London 263, Kraines,., Kraines, V., 988. Pavlov and the risoner s dilemma. Theor. ecision 26, Lieberman, E., Hauert,., Nowak, M.A., 25. Evolutionary dynamics on grahs. Nature 433, Lindgren, K., 99. Evolutionary henomena in simle dynamics. In: Langton,.G., et al. (Eds.), Artificial Life II. Lindgren, K., Nordahl, M.G., 994. Evolutionary dynamics of satial games. Physica 75, May,., 987. More evolution of cooeration. Nature 327, 5 7. May,.M., 26. Network structure and the biology of oulations. Trends Ecol. Evol. 2, 394. Maynard Smith, J., 982. Evolution and the Theory of Games. ambridge University Press, ambridge, USA. Maynard Smith, J., Price, G.., 973. The logic of animal conflict. Nature 246, 5. Milinski, M., 987. Tit for tat in sticklebacks and the evolution of cooeration. Nature 325, Nakamaru, M., Matsuda, H., Iwasa, Y., 997. The evolution of cooeration in a lattice structured oulation. J. Theor. iol. 84, Nakamaru, M., Nogami, H., Iwasa, Y., 998. Score-deendent fertility model for the evolution of cooeration in a lattice. J. Theor. iol. 94, 24. Nowak, M.A., 26a. Evolutionary ynamics. Harvard University Press, MA. Nowak, M.A., 26b. Five rules for the evolution of cooeration. Science 34, Nowak, M.A., May,.M., 992. Evolutionary games and satial chaos. Nature 359, Nowak, M.A., May,.M., 993. The satial dilemmas of evolution. Int. J. ifurcat. haos 3, Nowak, M.A., Sigmund, K., 992. Tit for tat in heterogeneous oulations. Nature 355, Nowak, M.A., Sigmund, K., 993. A strategy of win-stay, lose-shift that outerforms tit for tat in Prisoner s ilemma. Nature 364, Nowak, M.A., Sigmund, K., 24. Evolutionary dynamics of biological games. Science 33, Nowak, M.A., onhoeffer, S., May,.M., 994. More satial games. Int. J. ifurcat. haos 4, Nowak, M.A., Sasaki, A., Taylor,., Fudenberg,., 24. Emergence of cooeration and evolutionary stability in finite oulations. Nature 428, Ohtsuki, H., Nowak, M.A., 26a. Evolutionary games on cycles. Proc.. Soc. 273, Ohtsuki, H., Nowak, M.A., 26b. The relicator equation on grahs. J. Theor. iol. 243, Ohtsuki, H., Nowak, M.A., 27. irect recirocity on grahs. J. Theor. iol. 247, Ohtsuki, H., Hauert,., Lieberman, E., Nowak, M.A., 26. A simle rule for the evolution of cooeration on grahs and social networks. Nature 44, Ohtsuki, H., Nowak, M.A., Pacheco, J., 27a. reaking the symmetry between interaction and relacement in evolutionary dynamics on grahs. Phys. ev. Lett. 98, 86. Ohtsuki, H., Pacheco, J., Nowak, M.A., 27b. Evolutionary grah theory: breaking the symmetry between interaction and relacement. J. Theor. iol. 246, Ohtsuki, H., ordalo, P., Nowak, M.A., 27c. The one-third law of evolutionary dynamics. J. Theor. iol. 249 (27), Pacheco, J.M., Santos, F.., 25. Network deendence of the dilemmas of cooeration. In: Mendes, J.F.F. (Ed.), Science of omlex Networks: From iology to the Internet and WWW, vol AIP onference Proceedings,. 9. Pacheco, J.M., Traulsen, A., Nowak, M.A., 26a. Active linking in evolutionary games. J. Theor. iol. 243, Pacheco, J.M., Traulsen, A., Nowak, M.A., 26b. oevolution of strategy and structure in comlex networks with dynamical linking. Phys. ev. Lett. 97, aoort, A., hammah, A.M., 965. Prisoners ilemma: A Study in onflict and ooeration. The University of Michigan Press, Ann Arbor, USA.

9 J.M. Pacheco et al. / Journal of Theoretical iology 25 (28) Samuelson, L., 997. Evolutionary Games and Equilibrium Selection. MIT Press, ambridge, USA. Santos, F.., Pacheco, J.M., 25. Scale-free networks rovide a unifying framework for the emergence of cooeration. Phys. ev. Lett. 95, 984. Santos, F.., Pacheco, J.M., 26. A new route to the evolution of cooeration. J. Evol. iol. 9, Santos, F.., odrigues, J.F., Pacheco, J.M., 25. Eidemic sreading and cooeration dynamics on homogeneous small-world networks. Phys. ev. E 72, Santos, F.., odrigues, J.F., Pacheco, J.M., 26a. Grah toology lays a determinant role in the evolution of cooeration. Proc.. Soc. iol. Sci. 273, Santos, F.., Pacheco, J.M., Lenaerts, T., 26b. Evolutionary dynamics of social dilemmas in structured heterogeneous oulations. Proc. Natl Acad. Sci. USA 3, Santos, F.., Pacheco, J.M., Lenaerts, T., 26c. ooeration revails when individuals adjust their social ties. PloS omut. iol. 2, 284. Selten,., Hammerstein, P., 984. Gas in Harley s argument on evolutionarily stable learning rules and in the logic of tit-for-tat. ehav. rain Sci. 7, 5 6. Skyrms,., Pemantle,., 2. A dynamic model of social network formation. Proc. Natl Acad. Sci. USA 97, Sugden,., 986. The Economics of ights, o-oeration and Welfare. lackwell, Oxford. Szabo, G., Fa th, G., 27. Evolutionary games on grahs. Phys. e. 446, Szabo, G., Hauert,., 22. Phase transitions and volunteering in satial ublic goods games. Phys. ev. Lett. 89, 8. Szabo, G., T+oke,., 998. Evolutionary risoner s dilemma game on a square lattice. Phys. ev. E 58, Szabó, G., Vukov, J., Szolnoki, A., 25. Phase diagrams for an evolutionary risoner s dilemma game on two-dimensional lattices. Phys. ev. E 72, 477. Taylor, P.., Jonker, L., 978. Evolutionary stable strategies and game dynamics. Math. iosci. 4, 45. Taylor,., Fudenberg,., Sasaki, Nowak, M.A., 24. Evolutionary game dynamics in finite oulations. ull. Math. iol. 66, Traulsen, A., Nowak, M.A., Pacheco, J.M., 26a. Stochastic dynamics of invasion and fixation. Phys. ev. E 74, 99. Traulsen, A., Pacheco, J.M., Imhof, L., 26b. Stochasticity and evolutionary stability. Phys. ev. E 74, 295. Traulsen, A., Nowak, M.A., Pacheco, J.M., 27a. Stochastic ayoff evaluation increases the temerature of selection. J. Theor. iol. 244, Traulsen, A., Pacheco, J.M., Nowak, M.A., 27b. Pairwise comarison and selection temerature in evolutionary game dynamics. J. Theor. iol. 246, Trivers,.L., 97. The evolution of recirocal altruism. Q. ev. iol. 46, van aalen, M., and,.a., 998. The unit of selection in viscous oulations and the evolution of altruism. J. Theor. iol. 93, Weibull, J., 995. Evolutionary Game Theory. MIT Press, ambridge, USA. Wilson,.S., Pollock, G.., ugatkin, L.A., 992. an altruism evolve in urely viscous oulations? Evol. Ecol. 6, Zeeman, E.., 98. Poulation dynamics from game theory. In: Lecture Notes in Mathematics, vol. 89. Sringer, erlin. Zimmermann, M., Eguíluz, V.M., Miguel, M.S., 24. oevolution of dynamical states and interactions in dynamic networks. Phys. ev. E 69, 652.