Complex networks and evolutionary games

Volume 2 Complex networks and evolutionary games Michael Kirley Department of Computer Science and Software Engineering The University of Melbourne, Victoria, Australia Email: mkirley@cs.mu.oz.au Abstract The Hawk Dove game is a well known non-repeating evolutionary game often used as a simple model of biological or economic phenomenon. In the spatial version of this game, complex spatial and temporal dynamics emerge as a direct consequence of agents adopting one of two strategies in order to gain a valuable resource. In this study, the population dynamics are investigated in terms of the underlying structural properties of the network on which the game is played. Simulations using alternative network topologies regular, small-world, random and scale-free networks suggest that the mode of connectivity within the spatial model is the most significant factor affecting the system dynamics. To explore the robustness of the network models, results are reported for more general cases involving stochasticity, asynchronous updating and varying interaction matrices.. Introduction Evolutionary game theory is concerned with the evolutionarily stable outcomes of particular phenotypic frequencies in a population (Maynard Smith and Price, 973; Maynard Smith, 982). Evolutionary games are often modelled where individuals confront each other, and the result of the encounter is determined according to a fixed payoff matrix. The strategic interactions between individuals and the resulting frequency-dependent effects on the overall evolutionary dynamics are key concepts examined. One of the most basic and interesting games from evolutionary biology is the Hawk Dove game (Maynard Smith, 982). In this game, hawk and dove are alternative strategies that agents (or players) have the option to play in order to gain a valuable resource. The standard evolutionary outcome is a polymorphic population of hawks and doves. Yet, this is not the most optimal configuration for all individuals. All doves would lead to the highest reproductive success for all individuals. Analysis of the Hawk Dove game provides valuable insight into the conflict between group beneficial behaviour and individual behaviour and the corresponding population dynamics. In the classic version of the game, it is assumed that a given individual is equally likely to interact with any other member of the population and that the success of any individual depends on the frequency of all other strategies represented in the population. In many biological situations, individuals occupy well defined territores, with interaction restricted to local neighbourhoods. Subsequently, the inclusion of a spatial dimensions into evolutionary game theory has proven to be a very fertile extension (Killingback and Doebeli, 996; Killingback and Doebeli, 998; Lindgren and Nordah, 994; Nowak and May, 993). A valid criticism of Copyright 25

the games on grids models is that results are dependent on the type of neighbourhood rules adopted (Hauert, 2). In most spatial evolutionary games reported in the literature, regularity in the connection topology is usually assumed. However, some models have been extended to more general cases involving stochasticity, asynchronous updating, and irregularity in lattices (Huberman and Glance, 993; Killingback and Doebeli, 998; Hauert, 2). In this study, the evolutionary games are played on a number of different complex networks. Specifically, networks with regular, random, small-world, and scale-free architectures are used (see section 3). Here, agents are mapped to the nodes of the networks. The rationale behind the use of alternative network structures is based on the fact that network topology plays a crucial role in determining the system s dynamic properties (Barabâsi and Albert, 999; Watts, 999; Dorogovtsev and Mendes, 23). Agents play the game with their local neighbours over a fixed number of interations (or rounds). An agent updates it s strategy if the neighbouring sites are doing better. A number of simulation models are used to investigate the population dynamics. The results suggest that the mode and level of connectivity of the network have a significant impact on the equilibrium proportion of strategies in the population. The remainder of the paper is organised as follows: In the next section, the Hawk Dove game is formally defined. The concept of evolutionary stable strategies and spatial evolutionary games are also described. In section 3, complex networks are introduced, including a brief overview of the particular network models used in this study. Section 4 describes the simulation models and presents the results. In section 5, the results are discussed. The paper concludes with a summary of the implications of this work. 2. Evolutionary Game Theory 2. The Hawk Dove Game Within an evolutionary games framework, a population consists of N interacting agents. Although in many models the number of agents is very large, their strategic interactions are usually decomposed into a sum of two player games. Each interaction is a simplified contest. That is, an agent s reproductive success is based on the encounter, and this payoff determines the number of offspring they produce. In the Hawk Dove game, the population consists of two groups: H and D. These groups are best characterized by their behaviour in contests. We can think of H as representing an all-out aggressive strategy. That is, agents playing strategy H will always fight for the resource. D on the other hand, is to be considered as a passive strategy. In this case, agents playing D will never fight for a resource. Each agent must select a strategy before the game starts. This strategy does not change over the course of the game. It is also assumed that an agent has no memory of previous encounters. The payoff matrix for the Hawk Dove game is shown in Table. Here, β is a positive real parameter, which represents the cost incurred by an individual who adopts the strategy H against H opponents. Typically, a value of β =3is used. The cumulative payoffs identify rates of reproduction of particular strategies in the population. That is, the proportion of each strategy in the population grows at a rate equal to the difference between the average payoff of that strategy and the average payoff of the population as a whole. 2 Copyright 25

Table. Hawk Dove Game Payoff Matrix Hawk Dove Hawk β 2 Dove 2.2 Evolutionary Stable Strategies The concept of an evolutionary stable strategy (ESS) is a central component of evolutionary game theory (Maynard Smith, 982). An ESS is one that, once pervasive in the population, can repel any (sufficiently small) mutant advance. Let E(I,J) be the expected payoff of playing strategy I against opponent J. IfI is a stable strategy, it must have the property that, if almost all members of the population adopt I then the fitness of these typical members is greater than that of any possible mutant. If not, the mutant strategy could invade the population consisting mainly of I, with a small frequency p either E(I,J) > E(J, I) or E(I,I) =E(J, I) and E(I,J) >E(J, J) In the Hawk Dove game, if β< then H is the only ESS. However, if β> then neither H or D is an ESS. Consider the scenario where a single individual playing H is introduced into a population consisting entirely of D. This situation is unstable, due to the fact that the mutant displaying hawk like behavior would bully the D out of existence. Here, the H strategy has an unfair advantage. Similarly, an all H population would also be unstable. A single individual playing D (introduced by mutation) would have a long-term advantage as a direct results of the aggressive behaviour among the H individuals. More generally, given particular frequencies of H and/or D in the population, one strategy will be fitter (greater utility across the population as a whole) while at other frequencies the other strategy will be fitter. This is a direct consequence of the fact that the two strategies do not have identical payoffs. Thus, in the case of mixed strategies, an ESS exists when H is played with probability p and D with probability p, where p =/β (Maynard Smith, 982; Killingback and Doebeli, 996). This mixed strategy corresponds to a stable polymorphism. An ESS is not necessarily the best strategy in all cases. That is, on a local scale the strategy may result in lower average payoffs to both agents than some other combination of strategies. However, once one or both agents adopt their ESS, they will be selected against if they change their strategy. It is also possible for a game to have several ESS. In this situation, chance determines which of the strategies increases in frequency, such that it is impregnable. It should also be noted that an important assumption of the ESS concept is that of an infinite population. However, under the more realistic conditions of finite populations with high selection pressure, the ratio of H to D can diverge from the ESS and may exhibit limit cycles (Fogel et. al, 997). 2.3 Spatial Models In spatial evolutionary games, the agents (players) are usually mapped onto a regular lattice. Interactions between agents are restricted to the local neighbourhood. The utility (fitness) of each individual is determined by summing its payoffs in games against each of its neighbours. The scores in the neighbourhood, including the individual s own score are typically ranked. In the following round (iteration) the individual adopts the strategy of the most successful agent from the local neighbourhood. From a biological perspective, the utility of an individual is 3 Copyright 25

interpreted in terms of reproductive success. Alternatively, from an economic perspective the utility refers to individuals adapting their strategy to mimic a successful neighbour. Most of the analysis of spatial evolutionary games reported in the literature has focused on the emergence of cooperation in the well-known Prisoner s Dilemma game (Huberman and Glance, 993; Lindgren and Nordah, 994; Nowak and May, 992; Nowak and May, 993). In contrast, there has only been a limited number of studies of the Hawk Dove game (Killingback and Doebeli, 996; Killingback and Doebeli, 998). In these studies, it was reported that, in general, the long-term proportion of H is smaller than the equilibrium proportion predicted by classical evolutionary game theory. In addition, Killingback and Doebeli have observed a type of complex dynamics that is different from the spatio-temporal chaos seen in the spatial Prisoner s Dilemma. They conjectured that suitable spatial extensions of any evolutionary game with a mixed evolutionarily stable strategy will exhibit critical dynamics for appropriate parameters. In a recent Nature article (Hauert and Doebeli, 24), it was shown that spatial structure might actually hinder cooperative behaviour in certain circumstances. Hauert and Doebeli have suggested that under certain conditions, spatial structure might degrade the level of cooperative behaviour in the population. In the case of the Hawk Dove game, when benefit outweighs cost, the emergent clusters of like minded individuals on a regular lattice often leads to increased vulnerability of cooperators. 3. Complex Networks A network can be modelled as a graph G(N,E) where N is a finite set of nodes (vertices) and E a finite set of edges (links) such that each edge is associated with a pair of nodes i and j. G can be represented by simply giving the N N adjacency (or connection) matrix whose entry a ij is if there is an edge joining node i to node j and is otherwise. The dynamic properties of a particular network are determined fundamentally by the topology (Watts and Stogatz, 998; Dorogovtsev and Mendes, 23). Typically, two measures are used to characterise the structural properties of the network: a global property the average path length (L), and a local property the clustering coefficient (C). L, measures the average separation between two nodes in the graph. The distance d ij between two nodes, labelled i and j respectively, is defined as the number of edges along the shortest path connecting them. Thus L = N(N ) i =j d ij. C is the probability that two nearest neighbours of a node are also nearest neighbours of each other. C i of node i is then defined as the ratio between the number E i of edges that actually exist among these k i nodes and the total possible number k i (k i )/2 that is C i =2E i /k i (k i ). The clustering coefficient C of the whole network is the average of C i over all i. Clearly, C< and C =if and only if the network is globally coupled, which means that every node in the network connects to every other node. A third structural property often consider is the degree k i of a node i. The degree is usually defined to be the total number of links. Thus, the larger the degree, the more important the node is in a network. The spread of node degrees over a network is characterized by a distribution function P (k), which is the probability that a randomly selected node has exactly k edges. 3. Network Classification A comprehensive discussion of alternative network architectures can be found in (Dorogovtsev and Mendes, 23). A brief description of four networks that provide the scaffolding for the 4 Copyright 25

Hawk Dove games investigated in this study is provide below. Regular A regular network is defined as a nearest-neighbour coupled network in which every node is joined by a few of its neighbours. Regular networks are clustered. Typically, a -D circular or a 2-D square grid or lattice is used. In this study, a 2-D lattice, with Von Neumann neighbourhood and periodic boundary conditions is used. Random At the opposite end of the spectrum from regular networks are completely random graphs (Erdös and Rényi, 959). Here, the nodes are randomly connected with some probability p. Random networks are rather homogeneous, that is, most of the nodes have approximately the same number of links. The degree distribution of the nodes is typically Poisson. The nodes are not clustered and have short average path lengths (Watts and Stogatz, 998). Small World The transition from a regular lattice to a random graph best describes smallworld networks (Watts and Stogatz, 998; Watts, 999). In small world networks, each link is re wired with some probability p. The effect of rewiring is the substitution of some short-range connections with long-range connections. Small world networks are highly clustered and the minimum path length is relatively short. Scale Free Scale-free networks are characterised by their connectivity distributions, which are in a power-law form, that is, independent of the network scale (Barabâsi and Albert, 999; Barabâsi et. al, 999). Here, most nodes have very few links and yet a small number of nodes are very highly connected. The average path length is smaller than in a corresponding random graph. The clustering coefficient of the scale-free model is typically larger than the corresponding value of a random graph. 4. Simulations 4. Network Model In the spatial Hawk Dove games examined in this study, the agents (players) are mapped onto the nodes of alternative complex networks (see Table 2). The network is initialized randomly with equal proportions of the two strategies. At each iteration, an agent plays either H or D. This model assumes that an individual located at node k, playing strategy i at time t, receives the a ij payoff if it interacts with a neighbour playing strategy j (see Table ). An agent s utility is therefore the sum of the payoff contributions to strategy i from all different j strategies in the local neighbourhood. Whenever a strategy at node k is updated, a neighbour y is drawn from among the n neighbours. The strategy at node y is then copied to node k. Two alternative techniques can be used to select y. In the deterministic model, the score of each individual is compared to all of its neighbours and only the nodes with the highest local maximum score are allowed to reproduce. In case of a tie between two competitors a random node is chosen. The second model is a stochastic technique. Here, the probability of reproduction is given by the scaled value of an individual s own score divided by total score in the neighbourhood. Such local competition allows strategies with lower fitness to have a chance of reproducing, especially when they are locally superior in numbers. An additional parameter is used to control the update sequence in the network. In the synchronous model, all nodes in the network are updated at the same time exactly once in each iteration. In the alternative asynchronous model, nodes to be updated are chosen randomly. 5 Copyright 25

Table 2. Proportion of Hawks in the population for a given network. Network C L Mean SDev Regular.2 6..2. Small World..7.39.4 Random. 5.2.42.2 Scale Free.2 4.3.32.8 4.2 Experiments and Results Extensive computational simulations were carried out to investigate the population dynamics of the alternative games played.the number of nodes in each network for all simulations was fixed at 24. The most important population dynamic metric considered was the proportion of H in the model at time t. Consequently, equilibrium frequencies of H and D were obtained after, iterations. All configurations/trials were repeated times with statistical results reported below. In the first simulation, the impact that different network topologies have on the proportion of H in the population was examined using the standard payoff matrix. In particular, this simulation provides some insight into the role the clustering coefficient and average path length plays in determining the population dynamics. Table 2 lists the mean and standard deviation values for each network architecture for β =3. Also listed are the values for the clustering coefficient and average path length. In all cases, there was a statistical significant difference (p-value <., t-test) between the observed data values for the proportion of H in the population and the theoretical ESS value. However, it is interesting to note that the proportion of H is significantly smaller for the regular network (largest average path length). Figure plots the proportion of H versus time from a typically trial. Both the initial phases and the equilibrium values are plotted. A number of different scenarios were then examined to investigate populations dynamics for games played on complex networks where the payoff matrix was varied. For each network architecture (see Table 2 for network characteristics), the value of β was incremented from to 4.25 in steps of.25. Four different simulations were conducted for each combination of network class and β value: (a) deterministic selection with synchronous update, (b) stochastic selection with synchronous update, (c) deterministic selection with asynchronous update, and (d) stochastic selection with asynchronous update. Figure 2 illustrates the variation of the proportion of H as a function of β for each scenario. Also plotted on each graph is the theoretical value of the ESS mixed-strategy proportion of H in the population. 5. Discussion In the spatial evolutionary games domain, the idea of cooperating with neighbours is certainly not a new concept. It is well understood that population structure can be a mixed blessing for cooperation, because the gains that it provides through positive assortment are countered by competition between like individuals (Hauert, 2; Nowak and May, 992; Nowak and May, 993). However, the games are usually played on 2-D lattices, possibly with varying neighbourhood sizes. In this paper, the investigation has focussed on the impact that alternative complex 6 Copyright 25

.75 Proportion of Hawks vs Time.75 Proportion of Hawks vs Time RG RN SF SM.5.5.25.25 5 5 2 Time 5 Time Figure. Proportion of H versus time for each network architecture. (left) the initial 2 iterations, and (right) equilibrium levels after, iterations. network topologies have on population dynamics. The specific aim has been to examine the variation in the proportion of particular strategies in the agent population playing the Hawk Dove game. Here, the role of (a) the payoff incentives that govern individual behaviour, which are subject to the strategies played by neighbours, (b) the values of the clustering coefficient and average path length of particular networks, and (c) the selection and update mechanisms employed, are of interest. Killingback and Doebeli reported that for β 3 the proportion of H present in the spatial game was significantly lower than predicted by classical theory (Killingback and Doebeli, 996). In the first simulation conducted, this hypothesis was examined using spatial models with different modes/levels of connectivity. In Table 2, it can be seen that for all spatial models used the proportion of H was significantly lower that the theoretical ESS. The expected average fitness of the standard polymorphic equilibrium values are different for each network. The higher selection pressure associated with higher clustering coefficients and average path lengths has the effect of reducing the proportion of H in the population. The differing overlapping neighbourhoods in a given network structure appears to be a driving force behind the the results observed. In Figure, time series plots illustrating the trajectory of the proportion of H are given. In the early stages, the proportion of H in each network model fluctuates with similarities between the random and small-world networks and the regular and scale-free networks. As the number of iterations increases, the scale-free and small-world converge to some extent. However, the scale-free proportion tends to oscillate around the small-world value. In the second phase of the study, the payoff for a particular combination of strategies in the game were varied for each topology examined. This approach provided useful insights and accurate estimates of any ESS behaviour evident in the games. The plots in Figure 2 illustrate the role that a particular network structure has in determining population dynamics. The general tendency is for the proportion of H to decrease as β increases in accordance with theoretical predications. The plots indicate that stable mixed population are possible for values of <β<4. It appears likely that two alternative strategies can coexist under conditions in 7 Copyright 25

(a) (b).8.8.6.4.6.4.2.2 2 3 4 Beta 2 3 4 Beta.8.6.4 (c).8.6.4 (d) RG RN SF SM NS.2.2 2 3 4 Beta 2 3 4 Beta Figure 2. Proportion of Hawks vs β. Each plot shows results for the four network architectures and the theoretical ESS value (NS), for a range of β values. (a) deterministic selection with synchronous update, (b) stochastic selection with synchronous update, (c) deterministic selection with asynchronous update, and (d) stochastic selection with asynchronous update. which, on average, one strategy is more successful than another. However, the dynamics of the population appear to be dependent on the control parameters. Further observation of the plots in Figure 2 show that for each configuration, the proportion of H in the population was smallest for the regular network. In the case of stochastic selection with synchronous updates (b) the proportion of H settles to a value of approximately.8. The results for the random network most closely approximated the theoretical ESS values. The greatest deviation occurring for values β>4. For the small-world network, the proportion of H in the population tends to be greater than the theoretical ESS values for β<4. In the case of asynchronous updating this difference is more pronounced. The rewiring of some of the short range links to long range links has the effect of altering selection pressure in local regions. The inherent time delay associated with asynchronous updated reinforces levels of cooperation. In scale-free networks, the proportion of H in the population follows the general trend of the theoretical ESS values for values of β < 3. As β increases the proportion deviates down for the deterministic, asynchronous and the stochastic, asynchronous models. A defining characteristic 8 Copyright 25

of the scale-free network is that a small number of nodes are very highly connected. Thus, it is to be expected that if one of these highly connected nodes is active in a local neighbour, and the strategy played at this node happens to be different to other strategies in the neighbour, the cumulative payoff will have a dominating effect. 6. Conclusion In this study, computational simulations have shown that strategic interactions in spatial models can influence the dynamics of the evolutionary process. The analysis shows that both the network architecture and the induced behavior associated with strategy payoff values are crucial to outcome of the game. It is possible for particular strategies, which are considered poorly adapted globally, to survive and increase in number given alternative network models. The results suggest that the theoretical ESS values are not necessarily the one that returns the best results for the participating players for a given network architecture. Different values of the average path length and clustering coefficient may lead to the establishment and persistence of different types of populations. However, when interpreting the simulation results, it is important to take into account the stability of possible equilibria depending on chosen payoff matrix, strategy space, and the network topology. References Barabâsi, A.L. and Albert, R. (999). Emergence of scaling in random networks. Science, 286:59-52. Barabâsi, A.L., Albert, R. and Jeong, H. (999). networks. Physica A, 272:73-87. Mean-field theory for scale free random Dorogovtsev S.N. and Mendes, J.F.F. (23). Evolution of Networks: From Biological Nets to the Internet and WWW. Oxford University Press, Oxford. Erdös, P. and Rényi, A. (959). On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci., 5:7-6. Fogel, D.B., Fogel, G.B., and Andrews, P.C. (997). On the instability of evolutionary stable strategies. Biosystems, 44:35-52. Hauert, C. (2). Fundamental clusters in spatial 2 2 games. Proc. R. Soc. Lond. B, 268:76-769. Hauert, C. and Doebeli, M. (24). Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature, 428:643-646. Huberman, B.A., and Glance, N.S. (993). Evolutionary games and computer simulations. Proceedings of the National Academy of Sciences of the USA, 9: 776-778. Killingback, T. and Doebeli, M. (996). Spatial evolutionary game theory: Hawks and Doves revisited. Proc. R. Soc. Lond. B, 262:35-44. Killingback, T. and Doebeli, M. (998). Self-organized Criticality in Spatial Evolutionary Game Theory. J. Theor. Biol., 9:335-34. 9 Copyright 25

Lindgren, K. and Nordahl, M.G. (994). Evolutionary dynamics of spatial games. Physica D, 75:292-39. Maynard Smith, J. and Price, G. (973). The logic of animal conflict. Nature, 246:5. Maynard Smith, J. (982). Evolution and the theory of games. Cambridge University Press. Nowak, M.A. and May, R.M. (992). Evolutionary games and spatial chaos. Nature, 359:826-829. Nowak, M.A. and May, R.M. (993). The spatial dilemmas of evolution. Int. J. Bifurcation Chaos, 3:35-78. Strogatz, S. (2). Exploring complex networks. Nature, 4:268-276. Watts, D.J. (999). Small Worlds: The Dynamics of Networks between Order and Randomness. Princeton University Press. Watts, D. and Stogatz, S.H. (998). Collective dynamics of small-world networks. Nature, 393:44-44. Copyright 25