Physica A. Promoting cooperation by local contribution under stochastic win-stay-lose-shift mechanism

Transcription

1 Physica A 387 (2008) Contents lists available at ScienceDirect Physica A journal homepage: Promoting cooperation by local contribution under stochastic win-stay-lose-shift mechanism Xiaojie Chen a, Feng Fu b, Long Wang a, a Center for Systems and Control, College of Engineering and Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing , China b Program for Evolutionary Dynamics, Harvard University, Cambridge, MA 02138, USA a r t i c l e i n f o a b s t r a c t Article history: Received 5 January 2008 Received in revised form 15 April 2008 Available online 3 June 2008 Keywords: Prisoner s Dilemma Cooperation Stochastic win-stay-lose-shift (WSLS) We introduce a stochastic win-stay-lose-shift (WSLS) mechanism into evolutionary Prisoner s Dilemma on small-world networks. At each time step, after playing with all its immediate neighbors, each individual gets a score to evaluate its performance in the game. The score is a linear combination of an individual s total payoff (i.e., individual gain from the group) and local contribution to its neighbors (i.e., individual donation to the group). If one s actual score is not larger than its desired score aspiration, it switches current strategy to the opposite one with the probability depending on the difference between the two scores. Under this stochastic WSLS regime, we assume that each focal individual gains its fixed score aspiration under the condition of full cooperation in its neighborhood, and find that cooperation is significantly enhanced under some certain parameters of the model by studying the evolution of cooperation. We also explore the influences of different values of learning rate and intensity of deterministic switch on the evolution of cooperation. Simulation results show that cooperation level monotonically increases with the relative weight of the local contribution to the score. For much low intensity of deterministic switch, cooperation is to a large extent independent of learning rate, and full cooperation can be reached when relative weight is not less than 0.5. Otherwise, cooperation level is affected by the value of learning rate. Besides, we find that the cooperation level is not sensitive to the topological parameters. To explain these simulation results, we provide corresponding analytical results based on mean-field approximation, and find out that simulation results are in close agreement with the analytical ones. Our work may be helpful in understanding the cooperative behavior in social systems based on this stochastic WSLS mechanism Elsevier B.V. All rights reserved. 1. Introduction According to the fundamental principles of Darwinian selection, evolution is based on a fierce competition between individuals and should therefore reward only selfish behavior. Yet, cooperative (altruistic) behavior is widespread in natural and social systems [1]. How to understand the emergence of cooperation is a fundamental problem. Fortunately, evolutionary game theory has provided a powerful framework to investigate cooperative behavior in systems consisting of competitive individuals [2,3]. As a common paradigm, the Prisoner s Dilemma game (PDG) has received much attention to study the evolution of cooperation in the literature (for example, see a recent review in Ref. [4] and references therein). In the traditional version of PDG, two individuals simultaneously adopt one of the two available strategies, cooperate (C) Corresponding author. addresses: xjchen@pku.edu.cn (X. Chen), longwang@pku.edu.cn (L. Wang) /$ see front matter 2008 Elsevier B.V. All rights reserved. doi: /j.physa

2 5610 X. Chen et al. / Physica A 387 (2008) or defect (D). For mutual cooperation both receive R, and only P for mutual defection, while a cooperator receives S when confronted to a defector, which in turn receives T, such that T > R > P > S and T + S < 2R. Under these conditions it is best to defect for rational individuals in a single round of the PDG, regardless of the opponent strategy. However, mutual cooperation leads to a higher payoff than mutual defection, but cooperation is irrational. Therefore, the dilemma is caused by the selfishness of the individuals. For the emergence of sustainable cooperation, other suitable extensions based on the traditional PDG need to be explored. During recent decades, different mechanisms which favor the emergence of cooperation are summarized in some recent reviews [4 6]. The following mechanisms, e.g., kin selection [7], direct [8] and indirect [9] reciprocity, network reciprocity [10,11], group selection [12], tag-based models [13], are found to support the emergence of cooperative behavior in biological and ecological systems as well as within human societies [14]. Indeed, the graph model should be more meaningful in realistic systems because most interactions among individuals are localized. In the evolutionary games with network structure, each individual occupying a node of the network can follow one of the pure strategies (C or D), and collect payoffs by playing the game with its immediate neighbors. It is well accepted that network topology plays an important role in the evolution of cooperation, and scale-free networks provide a favorable framework for the promotion of cooperation if the updating rule is based on the total payoff difference [15,16]. Here, we would like to stress that this promotion of cooperation in heterogeneous networks is valid for the total payoff difference, but remarkably inhibited if averaged payoffs instead of total ones [17 22]. It is because that the difference in players total payoffs which are proportional to the number of their neighbors is favorable to cooperation, whereas the averaged payoffs diminish the positive effect of degree heterogeneity on the evolution of cooperation. Moreover, the promotion of cooperation on graphs is also dependent on other factors, e.g., the effects of noise and uncertainties [23 26], the updating rules (deterministic [10] or stochastic [27]). And even for these specific updating rules, such as Best-takes-over and Betters-possess-chance, different levels of cooperation among them would exhibit (for more details, see Ref. [20]). Therefore, it has been confirmed that cooperation is not very robust under these rules on graphs, hence it is worth developing other updating rules or new frameworks to study games on graphs for robust cooperation in future work. To describe the system in a more realistic manner, the concept of WSLS mechanism has been adopted for sophisticated individuals in order to study the evolution of cooperation in realistic situations, which remarkably enhances cooperation in repeated game [28,29]. Adopting this similar concept, some reinforcement learning models were also proposed for studying game dynamics [30 32]. Noteworthy, Macy et al. [32] proposed a simple reinforcement learning model for two-person games, and studied the learning dynamics of cooperation in repeated games. In this work, individuals have their outcome aspirations, which are used to evaluate whether the individual satisfies its current behavior or not. If the outcome induced by the individual s behavior exceeds aspirations, the probability increases that the behavior will be repeated rather than searching for a superior alternative. Whereas if the outcome falls below aspirations, the probability decreases that the behavior will be repeated. Furthermore, these above principles have been introduced into networked evolutionary games to study the evolution of cooperation. Recently, Gulyás [33] investigated the adaptation of cooperating strategies in an iterated PDG with individually learning agents on Watts-Strogatz networks. Szolnoki et al. [34] studied evolutionary PDG with quenched teaching and learning activity of players on a two-dimensional lattice. Enlightened by these works, it may be worth studying the evolution of cooperation in structured population according to these principles under the concept of WSLS mechanism. In our present work, based on the original WSLS mechanism, we introduce a modified WSLS mechanism into the networked evolutionary PDG, and focus on the evolution of cooperation in Newman-Watts (NW) networks [35]. Indeed, real social networks are small-world implying the combination of abundant localized interaction and sufficient shortcuts, not similar to regular or random graphs. As a typical model of small-world networks, NW networks are close to real social networks, and can describe the population structure to be realistic. Moreover, NW networks facilitate extensions of mean-field approximation to study evolutionary dynamics [36,37]. To our knowledge, an alternative way to escape from the dilemma is to consider more sophisticated individuals. For instance, individuals should feel strongly about immediate benefit that affects them directly, but they should also take into account the affairs of others [5,38], and different possible measures of success in strategy adoption for individuals should be used to assess their performance in games [39,40]. Actually, in realistic systems individuals do not always consider maximizing their immediate benefit as a first goal; they also take into account their social responsibility, i.e., contributions to the group individuals belong to. That is to say, individuals are endowed with heuristic thinking ability in order to go beyond the dilemma, evidently they could not always consider taking advantage of others help, sometimes they should help others as donors. Thus, our starting point is realizing that, and in this paper, for long-term interactions with neighbors, individuals not only wish to increase their current total payoffs coming from PDG with their neighbors, but also consider the local contribution to their neighbors. We will show that this is an effective way to promote and maintain cooperation at favorable levels. To pursue as much generality as possible without sacrificing the simplicity, this local contribution of a given individual denotes the sum of all the payoffs its neighbors collect against it. Besides, the total payoff and local contribution constitute the score through a linear combination. Similarly, the score aspiration can also be described. Individuals are allowed to alter their strategies according to their actual scores and score aspirations. Interestingly, we find that the introduction of local contribution under the modified WSLS mechanism promotes cooperation substantially. Also, we present theoretical analysis based on mean-field approximation, and find that analytical results are in good agreement with numerical simulations.

3 X. Chen et al. / Physica A 387 (2008) The model We consider the evolutionary PDG with players located on the NW network, where a parameter p controls the fraction of edges randomly added to the regular ring graph [35]. Herein, in the original graph each node links with its four nearest neighbors. In evolutionary game on graphs, each player who occupies one site of the graph can only follow two simple strategies: C and D, and interacts only with its neighbors in each round. Let us represent the individuals strategies with twocomponent vector, taking the value s = (1, 0) T for C-strategist and s = (0, 1) T for D-strategist. For one certain individual x, the total payoff P x is collected from its neighbors, and the local contribution T x is the sum of all the payoffs its neighbors collect against it. Therefore, they can be respectively written as P x = y Ω x s T x As y, T x = y Ω x s T y As x, where the two sums both run over all the neighboring sites of x (this set is indicated by Ω x ), and A is the payoff matrix [ ] R S A =. T P (1) (2) The best-studied set of payoff values are T = 5, R = 3, P = 1, S = 0 [8,41], which is also adopted in this study. The score U x of a certain individual x is a weighted mean of the total payoff P x and the local contribution T x, that is, U x = (1 h)p x + ht x, (3) where 0 h 1 is a parameter characterizing the relative weight between T x and P x in the score U x. With h = 0, individuals only consider current total payoff coming from PDG with their neighbors, namely, the local contribution is ignored. With h = 1, individuals only consider the local contribution to the neighbors, that is, the current total payoff is ignored. Under this present heuristic updating rule, considering its own payoff and local contribution simultaneously, each individual expects itself and all its neighbors to play C strategy at final stage, and thus has a score aspiration U xa. Hence, the aspiration score U xa of each individual is defined as the value of U x under full cooperation in the neighborhood, namely, U xa = (1 h)k x R + hk x R = k x R, (4) where k x is the neighbor number of individual x. Accordingly, for a fixed value of R, the aspiration level for one focal individual in our study is fixed during the evolutionary process, and players update their strategies at this fixed aspiration level. Herein, the aspiration level provides the benchmark which is used to evaluate whether the individual satisfies its current strategy or not. To our knowledge, under the concept of WSLS mechanism individuals can update their strategies with a deterministic probability which is independent of payoff differences [28,42,43], or with a stochastic probability which is dependent of the difference between actual payoff and payoff aspiration [32]. Here, in our model each agent tends to switch to the opposite strategy from the current one if unsatisfied with its performance in games according to the actual score and score aspiration. Such tendency is characterized by a balanced weight between deterministic and stochastic switch of which the probability is proportional to the difference between the two scores. The reason for considering this heuristic updating regime is twofold: individuals sometimes are so myopic that they immediately choose the opposite strategy if the original one does not pay (i.e. outcome falls below the inspiration level); usually, individuals are also so sophisticated that they become reluctant to change current strategy if the present income is only a bit less than desired one (because they are not sure that they would gain more by switching to the opposite one. Hence they switch with a probability proportional to the difference between the two scores). We think that these two factors are both present in individual s decision-making, thus we have the above assumption regarding individual heuristic thinking. Specifically, in our model, at each time step each individual updates its strategy as follows: if U x U xa, individual x keeps its original strategy. Otherwise, individual x adopts the opposite strategy with a probability given as W = (1 δ)l U xa U x + δ, (5) U xa where l is the learning rate (0 < l < 1), and δ measures the intensity of deterministic switch, that is, the deterministic probability of adopting the opposite strategy. δ = 1 denotes completely deterministic switch, in this case the updating rule is analogous to the original WSLS mechanism, individuals thus decide to switch to the opposite strategy if they only know U x < U xa. δ = 0 denotes that this switch depends exactly on differences between actual score and score aspiration. The rationale for this rule is that when the score exceeds the aspiration level, the individual satisfies and keeps its original state. Otherwise, the actual score does not reach the aspiration level, the individual adopts the opposite strategy with a stochastic probability W, and W characterizes the exact extent of dissatisfying with its original state, and controls the individual to adopt the opposite strategy in order to reach the aspiration level. Such updating is different from the deterministic switch in original WSLS. Comparing to original WSLS, this mechanism makes full use of the individual s own information so that players can update their states more accurately. In our study, we would like to explore how this stochastic WSLS mechanism affects the evolution of cooperation, hence we mainly concentrate on the corresponding situations for small δ.

4 5612 X. Chen et al. / Physica A 387 (2008) Fig. 1. (Color online) The density of cooperators as a function of h for different learning rates in the case of (a) δ = and (b) δ = 0.1. Simulations are carried out on a NW network with p = 0.5. The inset in (b) shows the detail around the point (0.3, 0.5). 3. Simulations and analysis Simulations are carried out for a population of N = 2000 players occupying the nodes of a NW network. Initially, the two strategies of C and D are randomly distributed among the players with the equal probability 0.5. The key quantity for characterizing the cooperative behavior of the system is the density of cooperators ρ c, which is defined as the fraction of cooperators in the whole population. In all simulations, ρ c is obtained by averaging over last 1000 generations of the entire 11,000 generations, and we have confirmed that averaging over larger periods or using different transient times did not qualitatively change the simulation results. And each date point results from 10 different network realizations with 10 runs for each realization. The above model is simulated with synchronous update. Moreover, we have checked that the qualitative results do not change if we adopt random sequential strategy update. Fig. 1 shows the simulation results of ρ c as a function of h for different values of learning rate l and intensity of deterministic switch δ. We can find that the density of cooperators monotonously increases with the increasing of h regardless of the values of l and δ. In addition, corresponding to a fixed value of h, there are nearly no differences between the results of ρ c for different values of learning rate l as shown in Fig. 1(a), namely, ρ c is almost independent of the values of l. Whereas in Fig. 1(b), ρ c is affected by the values of l. One can find that when h < 0.3, the density of cooperators for small learning rate is a little greater than for high learning rate. While for h > 0.3, cooperation becomes much more favorable as learning rate increases. Interestingly and surprisingly, for un-neglectable values of δ, we figure out that the ρ c curves for different l all intersect at the point (0.3, 0.5). In what follows, to explain the nontrivial phenomena shown in Fig. 1, we provide theoretical predictions based on mean-field approximation. Considering the payoff matrix in Eq. (2) and assuming cooperators are distributed uniformly among the networks, the average scores of cooperators and defectors are respectively given as U C = (1 h)[rkρ c + Sk(1 ρ c )] + h[rkρ c + Tk(1 ρ c )], U D = (1 h)[tkρ c + Pk(1 ρ c )] + h[skρ c + Pk(1 ρ c )], and the score aspiration of the population is U a = (1 h)kr + hkr = kr, where k is the average degree of NW network. Subsequently, the transition rates can be written as W C D = l(1 δ) U a U C U a + δ, (6) (7)

5 X. Chen et al. / Physica A 387 (2008) Table 1 Simulation results of ρ c for different values of h with l = 0.0 (row 2), and for different values of l with h = 0.3 (row 4), respectively h ρ c l ρ c The data points are computed for δ = 0.1 on NW networks with p = 0.5. W D C = l(1 δ) U a U D U a + δ, and the dynamical equation of the density of cooperators becomes [4] c = (1 ρ c ) W D C ρ c W C D, with T = 5, R = 3, P = 1, and S = 0, this yields c = l 3 (1 δ)(1 ρ c)(2 7ρ c + 10ρ c h) + (1 2ρ c )δ. (8) For very small δ, deterministic switch in strategy updating process can be ignored, thus Eq. (8) simplifies as c = l 3 (1 ρ c)(2 7ρ c + 10ρ c h). (9) Considering Eq. (9), if h < 0.5, cooperators and defectors coexist in stable equilibrium, and the only stable equilibrium is ρ c = h. For h 0.5, cooperators dominate defectors, and the only stable equilibrium is ρ c = 1. Therefore, it indicates that ρ c monotonously increases with h. Also, from Eq. (9), one can easily see that the stable equilibrium is independent of learning rate l. Besides, in the case of learning rate l = 0, the dynamical equation of the density of cooperators simplifies as c = 0. Thus, the stationary solution satisfies ρ c = ρ 0 = 0.5, where ρ 0 denotes the initial density of cooperators. The comparison between simulation results and theoretical analysis of ρ c is plotted in Fig. 1(a). We can find that the analytical results are in very good agreement with the simulation ones. Furthermore, we would like to stress that, under small values of δ, the steady state ending up with full cooperation could be reached with h = 0.5. That is to say, in order for the emergence of cooperation, individuals do not necessarily consider others income more than themselves. It is very interesting and fascinating. Of course, if they tend to take into account more contributions than incomes (i.e. h > 0.5), full cooperation would be easier to sustain. When δ is not very small, the effect of deterministic switch on the strategy updating process can not be ignored. It is difficult to obtain the equilibria in Eq. (8). Nevertheless, in particular for l = 0, Eq. (8) can be simplified as c = (1 2ρ c )δ, hence ρ c = 0.5 is the only stable equilibrium [43]. More interestingly, for h = 0.3, Eq. (8) turns out as: [ ] 2l c = (1 2ρ c ) 3 (1 δ)(1 ρ c) + δ. In this situation, the unique stable equilibrium ρ c = 0.5 is independent of learning rate l. Please see Table 1 for detailed simulation results of ρ c for these two cases. One can find that numerical results are well consistent with the corresponding theoretical predictions. We present the role of different values of δ in the evolution of cooperation in Fig. 2. And also, we can see that ρ c curves all intersect at (0.3, 0.5) (as shown in the above theoretical analysis for small δ). When h < 0.3, cooperation level for small δ is a little greater than for high δ. While for h > 0.3, the density of cooperators is much more favorable as δ decreases. Intuitively, when δ 1, the ratio of the deterministic probability to the strategy updating probability tends to be one, hence cooperators and defectors will be distributed with equal percentage in the population [43]. Additionally, when the ratio of the local contribution to the score is high (h 1), individuals tend to adopt C strategy. Accordingly, the ρ c monotonically increases with h for a fixed δ as predicted by the mean-field approximation for small δ. Thus far, we have studied the evolution of cooperation for the structured populations under the stochastic updating rule. We summarize our findings as follows: for fixed values of l and δ, the higher the value of h is, the more favorable cooperation becomes. This is because, under this update rule, when the relative weight h of the local contribution to the score is high, in order to reach the aspiration level, most of individuals would adopt strategy C to make the local contribution to neighbors greater. Otherwise, most of individuals would adopt strategy D to collect much more payoff from their neighbors. Interestingly, in all situations, h = 0.3 is the critical weight. If h > 0.3 and with intermediate value of δ, then individuals are more inclined to play C strategy with their neighbors, thus it results in favorable cooperation. Nonetheless, the worst case of h 0, the system can still maintain a cooperation level around 25%. Especially, for very low δ, the system would eventually reach full cooperation when h 0.5, and the learning rate l would not influence the long run behavior of the system.

6 5614 X. Chen et al. / Physica A 387 (2008) Fig. 2. (Color online) The density of cooperators as a function of h for different values of δ. Open symbols are the simulation results, and closed symbols are the corresponding analytical results for δ = We make p = 0.5 for NW networks and l = 0.5. Fig. 3. (Color online) The density of cooperators as a function of h for different values of p on NW networks with l = 0.6 and δ = As a consequence, it is demonstrated that our proposed stochastic WSLS mechanism with local contribution substantially promotes cooperation level in Prisoner s Dilemma, thus solving the dilemma of cooperation in a way. Finally, let us consider the effect of the NW network parameter p on cooperation. Detailed simulation results are plotted in Fig. 3, from which we can see that the influence of different values p (p from 0 to 1, resulting in different network structural organization features), on cooperation is very limited. From Eq. (8), we can find that the cooperation level is independent of average degree. In fact, under this updating rule individuals update their strategies just depending on the differences between the actual and aspiration score, independent of the correlations between their nearest neighbors, therefore the connectivity structures do not influence the evolution of cooperation. Moreover, we would like to stress that in our work we only study the evolution of cooperation for the fixed payoff aspirations of players, and the results for different payoff aspirations were investigated elsewhere [44], where it is shown that appropriate intermediate payoff aspirations can promote cooperation. And we have confirmed that these simulation results remain valid for different network size N on NW networks. It turns out that our proposed updating rule still works very well. Therefore, cooperation based on this stochastic WSLS mechanism with local contribution is robust; in other words, the cooperation level is not sensitive to the population structure. Comparing with other different updating rules, our rule may provide a unifying framework to promote cooperation. 4. Conclusions We have studied the cooperative behavior in evolutionary PDG under the stochastic WSLS mechanism on NW networks. It is shown that cooperation is promoted when the relative weight h of the local contribution to the score is larger than 0.3. Additionally, when δ in the strategy updating probability is very small, it is found that the cooperation level is nearly independent of the learning rate, and full cooperation can be reached when h is not less than 0.5. We have also provided a theoretical analysis by using mean-field approximation, and found that the analytical results are well consistent with numerical simulations.

7 X. Chen et al. / Physica A 387 (2008) To conclude, we have illustrated the validity and efficiency of our proposed stochastic WSLS rule in solving the dilemma (escape from the trap of the dilemma). In fact, this updating rule provides a simple closed-loop feedback for individuals during the strategy updating process; that is, individuals update their strategies based on the differences between their actual scores and aspiration scores. This model is an extension of the previous strategy updating rules adopted in Refs. [28, 43]. Furthermore, this stochastic WSLS rule is indeed widespread in the realistic world and is the basis by which creatures (living beings) adapt to harsh nature. Here, individuals play, observe, and then respond based on the observations (feedback). The introduced feedback between actual score and score aspiration enables individuals to update their strategies more accurately, and our method might be a plausible stride to explore the important role of this ubiquitous mechanism in studying the evolution of cooperation in evolutionary games. Acknowledgments F.F. gratefully acknowledges the financial support from China Scholarship Council (2007U01235). This work was partially supported by National Natural Science Foundation of China (NSFC) under grant No , No and No , National 973 Program under Grant No. 2002CB312200, National 863 Program under Grant No. 2006AA04Z258 and 11-5 project under Grant No. A References [1] L.A. Dugatkin, Cooperation among Animals: An Evolutionary Perspective, Oxford University Press, Oxford, [2] J.M. Smith, Evolution and the Theory of Games, Cambridge University Press, Cambridge, [3] J. Hofbauer, K. Sigmund, Evolutionary Games and Population Dynamics, Cambridge University Press, Cambridge, [4] G. Szabó, G. Fáth, Phys. Rep. 446 (2007) 97. [5] M.A. Nowak, Science 314 (2006) [6] M. Doebeli, C. Hauert, Ecol. Lett. 8 (2005) 748. [7] W.D. Hamiltion, J. Theoret. Biol. 7 (1964) 17. [8] R. Axelrod, The Evolution of Cooperation, Basic Books, New York, [9] M.A. Nowak, K. Sigmund, Nature (London) 437 (2005) [10] M.A. Nowak, R.M. May, Nature (London) 359 (1992) 826. [11] H. Ohtsuki, C. Hauert, E. Lieberman, M.A. Nowak, Nature (London) 441 (2006) 502. [12] A. Traulsen, M.A. Nowak, Proc. Natl. Acad. Sci. U.S.A. 103 (2006) [13] A. Traulsen, M.A. Nowak, PLoS ONE 2 (2007) e270. [14] M.A. Nowak, K Sigmund, Science 303 (2004) 793. [15] F.C. Santos, J.M. Pacheco, Phys. Rev. Lett. 95 (2005) [16] J. Gómez-Gardeñes, M. Campillo, L.M. Floría, Y. Moreno, Phys. Rev. Lett. 75 (2007) [17] F.C. Santos, J.M. Pacheco, J. Evol. Biol. 19 (2006) 726. [18] F. Fu, L.-H. Liu, L. Wang, Eur. Phys. J. B 56 (2007) 367. [19] M. Tomassini, L. Luthi, E. Pestelacci, arxiv:physics/061225v1. [20] Z.-X. Wu, J.-Y. Guan, X.-J. Xu, Y.-H. Wang, Physica A 379 (2007) 672. [21] N. Masuda, Proc. R. Soc. B 274 (2007) [22] A. Szolnoki, M. Perc, Z. Danku, Physica A 387 (2008) [23] G. Szabó, J. Vulov, A. Szolnoki, Phys. Rev. E 72 (2005) [24] J. Vukov, G. Szabó, A. Szolnoki, Phys. Rev. E 73 (2006) [25] J. Ren, W.-X. Wang, F. Qi, Phys. Rev. E 75 (2007) (R). [26] M. Perc, Europhys. Lett. 75 (2006) 841; New J. Phys. 8 (2006) 22; New J. Phys. 8 (2006) 142; New J. Phys. 8 (2006) 183; Phys. Rev. E 75 (2007) [27] G. Szabó, C. Töke, Phys. Rev. E 58 (1998) 69. [28] M.A. Nowak, K. Sigmund, Nature (London) 364 (1993) 56. [29] M. Posch, A. Pichler, K. Sigmund, Proc. R. Soc. London, Ser. B 266 (1999) [30] D. Fudenberg, D. Levine, The Theory of Learning in Games, MIT Press, Cambridge, [31] T. Borgers, R. Sarin, Int. Econ. Rev. 41 (2000) 921. [32] M.W. Macy, A. Flache, Proc. Natl. Acad. Sci. USA 99 (2002) [33] L. Gulyás, Physica A 378 (2007) 110. [34] A. Szolnoki, G. Szabó, Europhys. Lett. 77 (2007) [35] M.E.J. Newman, D.J. Watts, Phys. Rev. E 60 (1999) [36] C.-L. Tang, W.-X. Wang, X. Wu, B.-H. Wang, Eur. Phys. J. B 53 (2006) 411. [37] J. Vukov, G. Szabó, A. Szolnoki, Phys. Rev. E 77 (2008) [38] J.M. Pacheco, F.C. Santos, F.A.C.C. Chalub, PLoS Comput Biol 2 (2006) e178. [39] H. Fort, N. Pérez, J. Artif. Soc. Soc. Simul. 8 (2005) 3. [40] H. Fort, N. Pérez, Eur. Phys. J. B 44 (2005) 109. [41] R. Axelrod, W.D. Hamilton, Science 211 (1981) [42] H. Posch, J. Theoret. Biol. 198 (1999) 183. [43] J. Alonso, A. Fernández, H. Fort, J. Stat. Mech. (2006) P [44] X.-J. Chen, L. Wang, Phys. Rev. E 77 (2008)