Behavior of Collective Cooperation Yielded by Two Update Rules in Social Dilemmas: Combining Fermi and Moran Rules

Transcription

1 Commun. Theor. Phys. 58 (2012) Vol. 58, No. 3, September 15, 2012 Behavior of Collective Cooperation Yielded by Two Update Rules in Social Dilemmas: Combining Fermi and Moran Rules XIA Cheng-Yi ( ), 1, WANG Lei ( ), 1 WANG Juan ( ), 2 and WANG Jin-Song ( Ø) 1 1 Key Laboratory of Computer Vision and System (Ministry of Education) and Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin , China 2 School of Automation, Tianjin University of Technology, Tianjin , China (Received May 9, 2012; revised manuscript received June 18, 2012) Abstract We combine the Fermi and Moran update rules in the spatial prisoner s dilemma and snowdrift games to investigate the behavior of collective cooperation among agents on the regular lattice. Large-scale simulations indicate that, compared to the model with only one update rule, the cooperation behavior exhibits the richer phenomena, and the role of update dynamics should be paid more attention in the evolutionary game theory. Meanwhile, we also observe that the introduction of Moran rule, which needs to consider all neighbor s information, can markedly promote the aggregate cooperation level, that is, randomly selecting the neighbor proportional to its payoff to imitate will facilitate the cooperation among agents. Current results will contribute to further understand the cooperation dynamics and evolutionary behaviors within many biological, economic and social systems. PACS numbers: Le, Ge, Xx Key words: spatial prisoner s dilemma game, Fermi update rule, Moran update rule, spatial snowdrift game, cooperation promotion, regular lattice 1 Introduction How to understand and characterize the collective cooperation among selfish individuals has been a great challenge of evolutionary biology and behavioral science. [1] Over the past decades, evolutionary game theory provides a powerful framework for us to partly interpret the ubiquity of cooperative behaviors within real systems, ranging from natural, biological to economic and social systems. [2] Generally, prisoner s dilemma game (PDG) is often utilized as a typical metaphor to explore the maintenance and emergence of cooperation. [3] Due to the difficulties in acquiring the specific payoffs, PDG often faces some limitations and thus the snowdrift game (SDG) becomes a good candidate. [4] During the game process, two players must simultaneously determine their own strategy, Cooperation (C) or Defection (D), without knowing its opponent s decision beforehand. The mutual cooperation leads to the reward (R) and mutual defection generates the punishment (P) for each individual, while the defector wins the highest payoff (i.e., temptation, (T)) and the cooperator gets the sucker s (S) payoff when a cooperator meets with a defector. The game is termed as PDG only if T > R > P > S and 2R > T + S, or SDG when T > R > S > P and 2R > T + S. [3 7] From the perspective of an individual player, the best strategy is to defect for PDG or adopt the opposite strategy of its opponent for SDG. However, in the whole, the cooperation can result in the greater payoff. Thus, the social dilemmas could emerge and even lead to the tragedy of the commons. [8 13] At present, many avenues or methods to promote or enhance the cooperation in the population have been proposed so as to understand the widespread cooperation behaviors in natural and social systems. On one hand, various game mechanisms, such as kin selection, [14] direct and indirect reciprocity, [15 16] group selection, [17 18] reputation, [19 20] reward and punishment [21 22] and payoff aspiration, [23 26] neighborhood size, [27 28] memory effect, [29] age structure, [30] payoff weighting mechanism, [31 32] voluntary participation, [33 34] environmental information [35 36] and so on, have been considered as effective means to counteract the adverse situations in the social dilemmas. On the other hand, beyond the well-mixed assumptions, the topological structure has also been identified as an important evidence to support the collective cooperation among agents. For example, Nowak and May [37] seminally found that the cooperation will be largely promoted if all players are arranged on a regular lattice and can only interact with their nearest neighbors. After that, the game behaviors on the regular lattice have received the greatest concern in the scientific communities, and spatial structure can largely promote the cooperative behaviors. [3 4] In the very recent years, the complex topology has been found in many natural, Supported by the National Natural Science Foundation of China under Grant No , Tianjin Municipal Natural Science Foundation under Grant No. 11JCYBJC06600, the Development Fund of Science and Technology for the Higher Education in Tianjin under Grant No and the 7th Overseas Training Project for the Young and Middle Teachers in Tianjin Municipal Universities xialooking@163.com c 2011 Chinese Physical Society and IOP Publishing Ltd

2 344 Communications in Theoretical Physics Vol. 58 biological, technological and social systems, it has been indicated that complex networks can further facilitate the emergence of cooperation. [38 40] That is, spatial and network reciprocity has possibly become another origin to promote the cooperation between players within real systems. However, whether the cooperation can be enhanced is also closely correlated with the update rule when we discuss the game dynamics. Different update dynamics can often lead to different outcomes, and even create the contradictory results about the same mechanism. For instance, spatial structure has been found to promote the cooperation of PDG in Nowak and May s seminal works, [37] but Hauert and Doebeli [41] indicated that the cooperation of SDG can not benefit from the spatial structure. These distinct results cast a puzzle for us to understand the role of spatiality in the evolution of cooperation. Since PDG and SDG have the same Nash equilibrium, the role of spatial structure should be consistent in these two games, why do not the final results coincide? If we carefully consider these two seminal works, it is found that the unconditional imitation (UI) rule is applied in Nowak and May works, whereas the replicator dynamics are used to update the individual strategy by Hauert and Doebeli. In reality, the qualitative results will be identical if we apply the same update dynamics in PDG and SDG models. In addition, Li et al. [42] applied the unconditional imitation rule to study the cooperative behavior in evolutionary snowdrift games, and they observe that the step-like behavior of cooperator s density with respect to cost-to-benefit ratio (r) is exhibited and the stationary state is very sensitive to the initial setup of cooperators and defectors in the lattice. That is, the role of the update rule should not be neglected when we talk about the evolutionary game dynamics. Up to now, to the best of our knowledge, nearly all works assume that players which participate in the game take the same update rule in the whole population, such as Fermi, Replicator dynamics or UI, when they make the decisions. In fact, it is not exact and far away from the reality for all players to update their strategies only according to one update rule, especially for the large scale real systems. When confronting the different environments, individuals may often make the decisions based on different conditions or criterions, that is, agents perhaps adopt different update rules. Thus, it is necessary for us to investigate the game dynamics in which two or more update rules are adopted by the interacting players. In this paper, we will combine the Fermi rule [43] with the Moran rule, borrowing from the Moran process in biology, [44] to update the individual strategy during the evolution of cooperation, and further to explore the cooperation dynamics in the spatial game of regular lattice. Based on this model, we try to mimic the individual decision behavior during the course of the game interaction. We find that the introduction of Moran process, which characterizes some kind of randomness, largely facilitates the persistence and survival of cooperators, and provides some new insights into the update dynamics in game cooperation behaviors. 2 Model In this paper, two different update rules are combined to investigate the cooperative behaviors based on the prisoner s dilemma game in which the payoff matrix can be written as follows. C D C D ( ) 1 0, b 0 (1) where b is a sole parameter of PDG model, termed as the defection parameter in the PDG. Although Eq. (1) can not strictly observe the ranking order T > R > P > S, it captures nearly all the behaviors as pointed out by Nowak and May. At the same time, we also consider the SDG model by the normalized payoff, i.e. R = 1, T = 1 + r, S = 1 r, and P = 0, the payoff matrix can be described as follows, C D C D ( ) 1 1 r, 1 + r 0 (2) where r denotes the cost-to-benefit ratio in the SDG model. Our model is implemented as follows. At first, all players are placed on the square L L lattice with the periodic conditions, and each player takes the cooperation (S i = 0) or defection (S j = 0) with the equal probability. Then, in each Monte Carlo (MC) simulation step, we first compute the game payoff according to the PDG or SDG payoff matrix described in Eq. (1) or Eq. (2). After that, we asynchronously update the strategy of each individual, which means that agents are selected in a random sequence and each agent in the whole lattice, on average, is chosen once for a strategy update in one MC time step. Different from previous works, however, for each player we adopt two kinds of update rules here, we perform the Fermi rule with the probability (1 q) and Moran rule with the probability (q) in which q is a tunable parameter changing from 0 to 1. Fermi rule: agent i will randomly select one of his neighbors, say j, player i will copy the strategy of player j with the following probability 1 Prob(S i S j ) = 1 + exp[(π i Π j )/K], (3) where Π i and Π j represent the payoffs that player i and j have obtained in the current game round, S i and S j denote their strategy. Moran rule: agent i will imitate one of his neighbors, say j, with the following probability, Π j Prob(S i S j ) =, (4) k N i Π k

3 No. 3 Communications in Theoretical Physics 345 where j, k is one of neighbors of agent i, N i represents the set of neighbors of agent i. Π j and Π k stand for the payoff of agent j and k, respectively. It is obvious that the higher the payoff of i s neighbors, the larger the possibility of being selected to imitate. In particular, the agents will only adopt the Fermi rule to update their strategies when q = 0, while the update rule is reduced to the Moran update when q = 1. Large scale simulations will be presented for 0 < q < 1 in Sec to 0.2 and F C is largely enhanced, and the extinction threshold b c is varied from to around 1.1. Since Moran update rule integrates the information with respect to all neighbors of a focal player, and can markedly promote the cooperation among agents on the square lattice. Thus, the cooperative behavior can always be kept into a higher level when compared to the Fermi rule. 3 Monte Carlo Simulation Results In this paper, all Monte Carlo simulation results are obtained after averaging over 20 independent runs on L L square lattice, in which periodic boundary conditions are satisfied. Fig. 1 (Color online) Relationship between F c and b for different q in the spatial PDG model. All these results are obtained for L = 200, K = 0.1 and initial frequency of cooperators is f c(0) = 0.5. All data points are obtained after averaging over 20 independent realizations and f c is averaged over 2000 time steps after transient time steps. First, we illustrate the relationship between the fraction of cooperators (F C ) and defection parameter b in Fig. 1. It is obvious that the cooperation behavior of current model is interpolated between two extreme cases, in which q = 0 represents the Fermi dynamics and q = 1.0 corresponds to the Moran update dynamics. These two extreme cases in our model are also consistent with previous results. [3,43] The probability that an individual adopts the Moran update rule will naturally increase as q grows, that is, the likelihood that randomly select the neighbor with the highest payoff to imitate becomes larger. The higher the neighbor s payoff, the more the probability that can be selected to learn. In the whole population, each individual tends to imitate the neighbor with the highest neighbor and the aggregate cooperation level will become higher and higher as more and more players apply the Moran dynamics. For example, q is only increased from Fig. 2 (Color online) The characteristic evolutionary patten between cooperators and defectors in the square lattice for a typical run. In the left panels, the defection parameter b is 1.06, q is set to be 0,0.2 and 0.4 from top to bottom, respectively; In the right panels, the defection parameter b is 1.5, q is set to be 0.6, 0.8 and 1.0 from top to bottom, respectively. The noise strength K is always 0.1. The Green (light gray) points denote defectors and Red (dark gray) points denote cooperators. Second, the evolutionary pattern between cooperators and defectors after sufficient MC steps is depicted in Fig. 2. When the defection parameter b is smaller (e.g., b = 1.06), it can be observed, in the left panels of Fig. 2, that the cooperators can resist the exploitations of defectors and form the small cooperative clusters even if the Moran rule is only adopted with the probability q = 0.2. When q > 0.2, the cooperative clusters become larger and larger, indicated as in the left bottom panel of Fig. 2 in which q is set to be 0.4. The cooperators even completely take over the whole lattice when q is beyond about 0.5. While for larger defection parameter (e.g., b = 1.5), the

4 346 Communications in Theoretical Physics Vol. 58 cooperators can not resist the complete invasion and be exploited by the defectors. Finally, the cooperators will be extinct even if q is up to 0.4. In the right panels of Fig. 2, from top to bottom, we show three characteristic snapshots for q = 0.6, 0.8 and 1.0, respectively. Likely, the higher the probability of Moran rule being applied, the larger the cooperative clusters, and also the more the fraction of cooperators in the lattice. the higher payoff neighbors are easier to be selected and imitated. Thus, combining Moran rule or integrating the neighborhood information will greatly facilitate the collective cooperation, and the persistence and emergence of cooperation can be arrived at in the whole population. Fig. 3 (Color online) The normalized average cluster size as a function of tunable parameter q in the lattice with L = 200, 400, 600 respectively. The defection parameter b is set to be 1.06, and K = 0.1. In order to further understand the cooperative patterns created commonly by the Fermi and Moran rules, we can calculate the average cluster size S of cooperators in the lattice, and S is defined as follows, S = 1 t+t 1 1 L 2 T t nc i=1 S i, (5) nc where S i denotes the size of i-th cluster and nc is the total number of cooperative connected clusters at each time step, t is the beginning of sampling steps and T stands for the sampling length and L 2 is used to normalize the average cluster size. In Fig. 3, we describe the normalized average size of cooperative clusters as a function of q for different lattice size L = 200, 400, and 600. Here we set the MC steps to be and start to sample at t = and the sampling length is set to be T = As q increases, the average cluster size of cooperators S will quickly grow and little by little dominate the lattice, and even take over the whole lattice when q goes beyond a specific value, termed as the cooperation threshold (denoted by p c ), which lies around 0.5. Beyond this threshold the cooperators will totally dominate the population and the clusters are all composed of cooperators. Furthermore, the cooperation threshold (p c ) is independent of the lattice size L, and S is nearly kept to be same for different lattice size L under the condition p < p c. Altogether, the clusters formed by cooperators are absolutely impervious to defector s attack at moderate values of p since Fig. 4 (Color online) F C as a function of Fermi parameter K in the lattice with L = 200. (a) b is set to be 1.06; (b) b is set to be 1.5. Figure 4 illustrates the frequency of cooperators F C in dependency on K for different values of b (for explicitly showing the effect, the case of F C = 0 for the whole interval of K is not included). For b = 1.06, the non-monotonic behaviors exists in the classical Fermi rule (i.e. q = 0). This kind of behavior, however, will disappear when the Moran update rule is integrated into the update dynamics of agents with the given probability, and the cooperation will be monotonically increased as K grows only if K is no more than 2.0. Interestingly, for larger defection parameters (e.g., b = 1.5), the influence of K can nearly be ignored when K 0.1, afterwards the fraction of cooperators will continuously increase. Meanwhile, the complete Moran rule will not be affected by the noise strength since the Moran rule does not cover the noise and the selection of imitating objects are randomly assigned proportional to their payoff. That is, introduction of this kind of ran-

5 No. 3 Communications in Theoretical Physics 347 domness will foster the cooperation among agents. Fig. 5 (Color online) Relationship between F c and r for different q in the spatial SDG model. All these results are obtained for L = 200, K = 0.1 and initial frequency of cooperators is f c(0) = 0.5. All data points are obtained after averaging over 20 independent realizations and f c is averaged over 2000 time steps after transient time steps. However, whether the above-mentioned mechanism promotes the cooperation in the SDG model is an open question. Here, we further verify the generality that integrating the two update dynamics can promote the cooperation behavior in the SDG model as well. We report the simulation results in the SDG model in Fig. 5. Similarly, we can observe that the cooperation can be largely enhanced when compared to only Fermi rule. That is, it is also positive for us to integrate the Moran process into the SDG model. In addition, the results are surprisingly inspiring for us to understand the cooperative behaviors inside the population since an individual can be allowed to take different update rules confronting the changing conditions, which maybe represent a more real situation and help to shed further light on the cooperation dynamics in real systems. 4 Conclusions In summary, we investigate the game behaviors among agents on the square lattice in which two update rules, Fermi rule or Moran rule, are commonly utilized to update the individual strategy. Each agent updates its strategy by the Fermi rule with the probability 1 q or Moran rule with the probability q, and q = 0 rightly corresponds to the Fermi rule, whereas q = 1 represents the original Moran rule. 0 < q < 1 can characterize a more realistic case, which allows the agents to adopt different update rules when faced with different environments, and provides a richer phenomenon about the game dynamical behaviors. We can observe that integrating the Moran rule into the game will largely promote the collective cooperation level among agents on the square lattice, in which an agent randomly chooses the imitated object proportional to its payoff. It is obvious that the Moran rule requires that each agent needs to know about its all environmental information, such as the payoff of every neighbor, which helps to prevent the cooperative clusters from being exploited by the defectors. Furthermore, the introduction of Moran rule will support the formation of cooperative clusters and keep the fraction of cooperators robust for the noise strength K of Fermi rule. At the same time, combing the Moran rule with Fermi rule will also present an active role in the snowdrift game model. Altogether, the current model benefits us to deeply analyze and understand the origin of emergence of cooperation in social dilemmas within many natural, biological, economic and social systems. References [1] H. Gintis, Game Theory Evolving, Princeton University Press, Princeton (2000). [2] M.A. Nowak, Evolutionary Dynamics: Exploring the Equations of Life, Harvard Universtiy Press, Cambridge (2006). [3] G. Szabó and G. Fáth, Phys. Rep. 446 ( 2007) 97. [4] C.P. Roca, J.A. Cuesta, and A. Sánchez, Phys. Life Rev. 6 (2009) 208. [5] Y. Chen, S.M. Qin, L. Yu, and S.L. Zhang, Phys. Rev. E 77 (2008) [6] W.B. Du, X.B. Cao, R.R. Liu, and C.X. Jia, Int. J. Mod. Phys. C 21 (2010) [7] Z.X. Wu, X.J. Xu, Z.G. Huang, S.J. Wang, and Y.H. Wang, Phys. Rev. E 74 (2006) [8] G. Harding, Science 162 (1968) [9] X.B. Cao, W.B. Du, and Z.H. Rong, Physica A 389 (2010) [10] Z. Wang, A. Szolnoki, and M. Perc, Phys. Rev. E 85 (2012) [11] Z.H. Rong and Z.X. Wu, Europhys. Lett. 87 (2009) [12] C.Y. Xia, J.J. Zhang, Y.L. Wang, and J.S. Wang, Commun. Theor. Phys. 56 (2011) 638. [13] Z.J. Xu, Z. Wang, and L.Z. Zhang, Phys. Rev. E 80 (2009) [14] W.D. Hamilton, J. Theor. Bio. 7 (1964) 1. [15] R.L. Trivers, Q. Rev. Biol. 46 (1971) 35. [16] M.A. Nowak and K. Sigmund. Nature (London) 393 (1998) 573. [17] A. Traulsen and M.A. Nowak, Proc. Natl. Acad. Sci. (USA) 103 (2006) [18] M. Perc. New J. Phys. 13 (2011)

6 348 Communications in Theoretical Physics Vol. 58 [19] M. Milinski, D. Semmann, and H.J. Krambeck, Nature (London) 415 (2002) 424. [20] F. Fu, C. Hauert, M.A. Nowak, and L. Wang, Phys. Rev. E 78 (2008) [21] R. Jiménez, H. Lugo, J.A. Cuesta, and A. Sánchez, J. Theor. Bio. 250 (2008) 475. [22] Z. Wang, Z.J. Xu, and L.Z. Zhang, Chin. Phys. B 19 (2010) [23] X.J. Cheng and L. Wang, Phys. Rev. E 77 (2008) [24] Z. Wang and M. Perc, Phys. Rev. E 82 (2010) [25] H.F. Zhang, R.R. Liu, Z. Wang, and B.H. Wang, EPL 94 (2011) [26] W.B. Du, X.B. Cao, M.B. Hu, H.X. Yang, and H. Zhou, Physica A 388 (2009) [27] J.J. Zhang, J. Wang, S.W. Sun, L. Wang, Z. Wang, and C.Y. Xia, Commun. Theor. Phys. 57 (2012) 541. [28] J. Wang, C.Y. Xia, Y.L. Wang, S. Ding, and J.Q. Sun, Chin. Sci. Bul. 57 (2012) 724. [29] W.X. Wang, J. Ren, G.R. Chen, and B.H. Wang, Phys. Rev. E 74 (2006) [30] Z. Wang, Z. Wang, and J.J. Arenzon, Phys. Rev. E 85 (2012) [31] C.Y. Xia, Z.Q. Ma, Y.L. Wang, J.S. Wang, and Z.Q. Chen, Physica A 390 (2011) [32] C.Y. Xia, J. Zhao, J. Wang, Y.L. Wang, and H. Zhang, Phys. Scrip. 84 (2011) [33] G. Szabó and C. Hauert, Phys. Rev. Lett. 89 (2002) [34] C. Hauert, S. De Monte, J. Hofbauer, and K. Sigmund, Science 296 (2002) [35] Z. Wang, W.B. Du, X.B. Cao, and L.Z. Zhang, Physica A 390 (2011) [36] Z. Wang, A. Murks, W.B. Du, Z.H. Rong, and M. Perc, J. Thero. Bio. 277 (2011) 19. [37] M.A. Nowak and R.M. May, Nature (London) 359 (1992) 826. [38] F.C. Santos and J.M. Pacheco, Phys. Rev. Lett. 95 (2005) [39] Z.H. Rong, H.X. Yang, and W.X. Wang, Phys. Rev. E 82 (2010) [40] Z. Wang, A. Szolnoki, and M. Perc, Sci. Rep. 2 (2012) 369. [41] C. Hauert and M. Doebeli, Science 428 (2004) 643. [42] P.P. Li, J.H. Ke, Z.Q. Lin, and P.M. Hui, Phys. Rev. E 85 (2012) [43] G. Szabó and C. Töke, Phys. Rev. E 58 (1998) 69. [44] P.A.P. Moran, The Statistical Process of Eolutionary Theroy, Oxford University, Clarendon, Oxford (1962).