EVOLUTIONARY GAME THEORY AND THE MODELING OF ECONOMIC BEHAVIOR**

Transcription

1 DE ECONOMIST 146, NO. 1, 1998 EVOLUTIONARY GAME THEORY AND THE MODELING OF ECONOMIC BEHAVIOR** BY GERARD VAN DER LAAN AND XANDER TIEMAN* Key words: noncooperative symmetric bimatrix game, evolutionary stable strategy, replicator dynamics, learning and imitation, metastrategy, stable population 1 INTRODUCTION Starting with the famous Nash equilibrium for noncooperative games, game theory became a popular field of research in the 1950s. The general feeling was that it would be possible to solve a lot of previously unsolvable problems using game theory. Although game theory was indeed applied to a wide variety of problems, attention for game theory diminished throughout the 1960s and the early 1970s. In the late 1970s and the early 1980s game theory boomed again, especially after Ariel Rubinstein s 1982 article in which he proved that a particular noncooperative bargaining model had a unique subgame perfect equilibrium. At the end of the 1980s however, there was a general feeling of discomfort about the use of the notion of Nash equilibrium to predict outcomes. The mode of research at the time was to use a different and very specific notion of Nash equilibrium for every problem and players were assumed to be hyperrational. The number of Nash equilibrium refinements had grown enormously and it was not clear in advance which refinement best suited which situation. What was clear was the fact that in the real world people did not act in the way behavior was postulated in the models. Before a new decline in interest in game theory could take place, game theorists picked up the idea of evolution from biology. In retrospect, the 1973 article by the biologists Maynard Smith and Price 1973 in which they defined the concept of Evolutionary Stable Strategies, was the most important in transferring evolutionary thinking from biology to game theory. The book Evolution and the Theory of Games by Maynard Smith 1982 explicitly introduced evolutionary selection pressure in a game-theoretic setting. The notion of evolution of strategies in a repeatedly played game closely resembles certain models in which players learn from past behavior, thus facilitating the shift of * G. van der Laan, Department of Econometrics and Tinbergen Institute, Free University, De Boelelaan 1105, 1081 HV Amsterdam, The Netherlands. A.F. Tieman, Department of Econometrics and Tinbergen Institute, Free University, De Boelelaan 1105, 1081 HV Amsterdam, The Netherlands. ** This research is part of the Research program Competition and Cooperation of the Faculty of Economics and Econometrics, Free University, Amsterdam. De Economist 146, 59 89, Kluwer Academic Publishers. Printed in the Netherlands.

2 60 G. VAN DER LAAN AND A.F. TIEMAN attention in the field towards evolutionary models. In the last decade evolutionary models have become increasingly popular in game theory and other fields of economics. Special issues of leading journals have been devoted to the subject, e.g. the issue of Games and Economic Behavior, Volume introduced by Selten, the issue in Journal of Economic Theory, Volume introduced by Mailath, and the issue in Journal of Economic Behavior and Organization, Volume Moreover, at many conferences in the field of game theory or economics special sessions were devoted to the subject, see for instance Van Damme In this paper we discuss the main concepts of evolutionary game theory and their applicability to economic issues. Traditional game theory focusses on the static Nash equilibria of the game and therefore implicitly assumes rational and knowledgeable players being able to anticipate correctly on the behavior of the other players in equilibrium. In evolutionary game theory it is assumed that initially all individuals are preprogrammed with some strategy and that the number of individuals playing a particular strategy is adjusted over time by some prespecified process according to the successfulness of the various strategies. This results in a dynamic process which may lead to an equilibrium and may give us a better understanding of how equilibrium strategies emerge within a society of interacting individuals. We start this survey article by reviewing the technical basis of the standard Nash equilibrium theory in section 2. This theory will be linked to the notion of Evolutionary Stable Strategies in section 3. In section 4 the system of differential equations specifying a biological evolutionary process, the replicator dynamics, will be discussed. Following that we discuss the economic significance of the ESS and the replicator dynamics in sections 5, 6 and 7. In section 8 we draw attention to an alternative branch of modeling, the so-called local interaction models. Finally, we pose some concluding remarks in section 9. 2 NASH EQUILIBRIUM Noncooperative game theory has become a standard tool in modeling conflict situations between rational individuals. Such a model describes the set of strategies of each individual or player and the payoff to each player for any strategy profile the list of strategies chosen by the players. The concept of Nash equilibrium is the cornerstone in predicting the outcome of a noncooperative game. In a Nash equilibrium each player s strategy maximizes his utility given the strategies played by the other players. In many situations the Nash equilibrium is not unique. Therefore many papers in game theory have been devoted to the issue of equilibrium selection. Refining the concept of Nash equilibrium allows one to discard certain Nash equilibria as not satisfying certain types of rationality. In this section we review some of the standard Nash equilibrium theory and its refinements. This enables us in the next section to show how the basic concepts of evolutionary game theory fit into this framework.

3 EVOLUTIONARY GAME THEORY 61 The basic model in noncooperative game theory is known as the n-person game in normal form and is characterized by a set of n players, denoted by I n 1,..., n, and for any player j a nonempty finite set of actions and a payoff function assigning the payoff to player j at any n-vector of actions specifying an action for each player. For j I n, let m j be the number of actions available to player j, indexed by k 1,..., m j. Then a mixed strategy for player j is a probability distribution over his set of actions and can be represented by a nonnegative vector x j R m j, with its kth coordinate xjk the probability assigned by player j to his action k, k 1,..., m j. The set of all mixed strategies of player j is the m j 1 -dimensional unit simplex S m j defined as S m j x j R m j k 1 m j x jk 1. The kth unit vector in R m j is denoted by e j k and is the vertex of S m j in which player j plays his action k with probability 1, i.e. when he plays a pure strategy n by choosing action k for sure. The set S j 1 S m j is the mixed strategy set of the game, i.e. an element x x 1,..., x n S specifies a mixed strategy x j S m j for each j I n. Thus x ji denotes the probability with which player j, j I n, plays action i, i 1,..., m j. For j I n, let a j be the payoff to player j when the vector of actions 1,..., n is played, where h 1,..., m h is the action chosen by player h, h I n. Let be the collection of all such vectors of actions. The payoff function u j : S R assigning the payoff to player j at any vector of mixed strategies x S, is defined as the expected value of the payoff for player j at any mixed strategy profile x x 1,..., x n S, i.e. u j n x h 1 x h h a j, 1 n where h 1 x h h is the probability with which the vector of actions 1,..., n is played under the mixed strategy profile x S. Finally, at a mixed strategy profile x S, the value u j e j k, x j denotes the expected marginal payoff for player j, when j changes his mixed strategy x j into his kth pure strategy of playing action k for sure and the other players h j stick to their mixed strategy x h, h j, i.e., u j e j k, x j j k h j x h ha j. From equations 1 and 2 it follows straightforwardly that for any x S and any j I n it holds that m j u j x k 1 x jku j e j k, x j, 2

4 62 G. VAN DER LAAN AND A.F. TIEMAN saying that u j x is the weighted sum over all expected marginal payoffs and therefore the payoff function u j x is linear in the components x jk, k 1,..., m j, of the mixed strategy vector x j. Observe that this linearity follows from the assumption that the number m j of actions available to player j is finite. We denote the non-cooperative n-person game in mixed strategies by G I n, S, u. For n 2 this game reduced to a so-called two-player bimatrix game denoted by G A, B, where A and B, respectively is the m 1 m 2 payoff matrix of player 1 and 2, respectively, i.e. the element a kh of matrix A and the element b kh of matrix B, respectively is the payoff to player 1 2 when player 1 plays his action k and player 2 plays his action h. Clearly for x S S m 1 S m 2 we have that u1 x x 1 Ax 2 and u 2 x x 1 Bx 2. Within this framework it is assumed that each player behaves rational and searches to maximize his own payoff. This is expressed in the equilibrium concept by John Nash The Nash equilibrium concept is the most fundamental idea in noncooperative game theory. A strategy profile x* S is a Nash equilibrium if no player can gain by unilaterally deviating from it. This means that in the Nash equilibrium concept it is implicitly taken as given that the players make their choices simultaneous and independent. Definition 2.1 Nash equilibrium Let the n-person game G I n, S, u be given. Then a strategy profile x* S is a Nash equilibrium when for every j I n holds that u j e jk, x j * u j x* for all k 1,..., m j. According to the linearity of u j x in the variables x jk, the definition implies that for any player j it holds that at a Nash equilibrium x* there is no mixed strategy y j S m j that gives player j a higher payoff than x j *, given that all the other players h j stick to their equilibrium strategy x h *. In a Nash equilibrium every player plays a best reply to the strategy of the others. Example 2.2 Inspection game The inspection game is a two player game: player 1 the worker works for player 2 the principal. The worker can choose between two actions: shirk S and work W. The principal can also choose out of two actions: inspect I and not inspect N. The principal pays a wage w 10 unless he has evidence obtained from inspection that the worker shirks. In the latter case he pays nothing. The inspection costs are c. When the worker chooses to work he produces output with value 20, otherwise he produces nothing. The effort of working is equal to a value e 5. The payoff matrices A for the worker and B for the principal are given by

5 and EVOLUTIONARY GAME THEORY A 0 w w e w e B c w w c w c c 10, 63 with the first player worker choosing a row either S or W and the second player principal a column either I ot N. For c 10 the principal s action N always yields a higher outcome than this action I. Since the worker s action S is the unique best reply to N we have that in this case the strategy profile S, N, i.e. the worker always plays his first action and the principal always plays his second action, is the unique Nash equilibrium in pure strategies. For 0 c 10 there is no Nash equilibrium in pure strategies. Suppose the principal does not inspect. Then it is optimal for the worker to shirk and therefore the principal is better off if he inspects. However, when the principal inspects, the agent prefers to work and then it is optimal for the principal to choose the action not inspect. So, in equilibrium both the principal and the worker have to play a mixed strategy. For 0 c 10 the unique Nash equilibrium in mixed strategies is given by 10 x* x 1 *, x 2 * with x 1 * c 10, 10 c and x 2 * 1 2, 1. In this equilibrium 2 the worker plays S with probability x 11 * c and W with probability c x 12 * 10. By analogy, the principal plays I with probability x* and N with probability x 22 * 21. In this case, every mixed strategy of the worker is a best reply against x 2 * and every mixed strategy of the principal is a best reply against x 1 *. However, the Nash equilibrium strategies are the only mixed strategies being a best reply against each other. The elegance of the Nash equilibrium concept and the underlying rational behavior of the agents has inspired many economists in formulating economic problems as noncooperative n-person games. Since the 1970s the Nash equilibrium concept has been applied to a wide range of problems. However, in applying the concept game theorists became aware of a serious drawback of the Nash equilibrium, namely that a noncooperative n-person game may have many Nash equilibria and therefore one particular arbitrarily chosen equilibrium does not make much sense as a prediction of the outcome of the problem. However, in many cases not all outcomes are consistent with the intuitive notion about what should be the outcome of the game. From the 1970s on several game theorists have

6 64 G. VAN DER LAAN AND A.F. TIEMAN addressed the problem of equilibrium selection by putting more requirements on the rational behavior of the players. Assuming highly rational players may eliminate the less intuitive outcomes. Several results have been obtained along this line of research. For an excellent survey on Nash equilibrium refinements we refer to Van Damme Here we only consider the concepts of perfect and proper Nash equilibria for normal form games. The notion of a trembling hand perfect Nash equilibrium of a normal form game has been introduced by Reinhard Selten 1975 and is one of the most fundamental results in the theory of Nash equilibrium refinements. For some real number 0 1, a completely mixed strategy profile x S, i.e. x jk 0 for all j and k, isa -perfect Nash equilibrium if x jk when the strategy e j k is not a best reply of player j against the strategies x h of the other players h j. This means that at a -perfect Nash equilibrium each pure strategy is played with a positive probability, but only pure strategies in the best reply set can have a higher probability than. In other wordt, at a -perfect equilibrium it is possible that a non-optimal strategy is played with positive probability, but the probability of playing such a non-optimal strategy is not larger than. Players are allowed to make errors, but for close to zero the probability of making an error by playing a non-optimal strategy is only small and hence the probability of playing a best reply is close to one. A perfect equilibrium is now defined as the limit of a sequence of -perfect equilibria when goes to zero. Definition 2.3 Perfect Nash equilibrium A strategy profile x* S is a perfect Nash equilibrium for the game G I n, S, u if for some sequence r 0, r N, converging to zero, there is a sequence of r -perfect Nash equilibria converging to x*. From the definition it follows immediately that any Nash equilibrium with completely mixed strategies is a perfect Nash equilibrium. Furthermore, Selten 1975 proved that any game G I n, S, u has a perfect Nash equilibrium, even if the game has no Nash equilibrium in completely mixed strategies. Theorem 2.4 Selten Theorem Any game G I n, S, u has at least one perfect Nash equilibrium. Moreover, the set of perfect Nash equilibria is a subset of the set of Nash equilibria. Example 2.5 Perfect Nash equilibrium We take the inspection game of example 2.2 with c 10. Then the payoff matrices are given by A and B

7 EVOLUTIONARY GAME THEORY 65 In this case it is optimal for the worker to play action S against any mixed strategy vector x 2 of the principal satisfying x As long as the probability of not inspect is at least as high as the probability of playing inspect, it is optimal for the worker to shirk. On the other hand, as the worker shirks, any strategy of the principal gives him an expected payoff equal to 10, so any mixed strategy of the principal is a best reply against the action S of the worker. So, any x* x 1 *, x 2 * with x 11 * 1 x 12 * 1 the worker 1 plays S for sure and 0 x 21 * 1 x 22 * 21 the probability that the principal inspects is at most 1 2 is a Nash equilibrium. However, if there is a small probability that the worker makes an error by choosing to work, it is optimal for the principal not to inspect, because the action not inspect gives him a higher payoff than the action inspect when the error indeed occurs, while both actions give him the same payoff if the error does not occur. So, as soon as there is an arbitrarily small probability that the worker by mistake will choose to play W, then playing a mixed strategy is no longer optimal for the principal and his unique best reply is to play N. The equilibria in which the principal plays a mixed strategy are not robust against the small probability that the worker will deviate from his optimal strategy of playing S. Only the Nash equilibrium in which the principal plays N for sure is a perfect Nash equilibrium. Although the notion of perfect equilibrium wipes out Nash equilibria which are not robust with respect to small probabilities of mistakes by error-making players and hence are not credible under the assumption of rational behavior of the players, Myerson 1978 argued that the probability with which a rational player plays a strategy by mistake will depend on the detrimental effect of the non-optimal strategy. More costly mistakes will be less probable than less costly mistakes. To further refine the set of perfect Nash equilibria, Myerson introduced the notion of a proper Nash equilibrium. Again we have that a proper Nash equilibrium is the limit of a sequence of -proper Nash equilibria as goes to zero. As in a -perfect equilibrium, at a -proper Nash equilibrium each pure strategy is played with a positive probability. However, if for some player j his kth action is a worse response against the strategies of the other players than some other action h, then the probability that strategy k is played is at most, 0 1, times the probability with which strategy h is played and so worse responses are played with smaller probabilities. Formally, for some real number 0 1, a completely mixed strategy profile x S is a -proper Nash equilibrium if for any player j I n, x jk x jh if u j e j k, x j u j e j h, x j. For example, let x* x 1 *,..., x n * be a 0.1-proper equilibrium and suppose that player 1 has three pure actions satisfying u 1 e j 1, x j * u 1 e j 2, x j * u 1 e j 3, x j *. Then player plays his optimal third action with a probability of at least 111, his not optimal second action with a probability of at most one-tenth of the probability of the optimal action, and his worst first action with a probability of at most one-tenth of the probability of the second action.

8 66 G. VAN DER LAAN AND A.F. TIEMAN Definition 2.6 Proper Nash equilibrium A strategy profile x* S is a proper Nash equilibrium for the game G I n, S, u if for some sequence r 0, r N, converging to zero, there is a sequence of r -proper Nash equilibria converging to x*. Again it immediately follows that any interior i.e. completely mixed Nash equilibrium is a proper Nash equilibrium. Moreover, each proper Nash equilibrium satisfies the conditions of a perfect equilibrium, because the notion of properness further restricts the set of allowable mistakes. Myerson 1978 proved that any game G I n, S, u has at least one proper Nash equilibrium. Theorem 2.7 Myerson Theorem Any game G I n, S, u has at least one proper Nash equilibrium. Moreover, the set of proper Nash equilibria is a subset of the set of perfect Nash equilibria. The concepts of perfect and proper Nash equilibria are nicely illustrated by a well-known example of Myerson Let the two-person bimatrix game be given by A B This game has three Nash equilibria in pure strategies, namely each strategy profile in which the two players play the same pure action. However, the Nash equilibrium in which both players play the third strategy and both receive a payoff of 7 is very unlikely. This equilibrium is ruled out by the notion of perfectness, which only allows for the two equilibria in which either both players play their first action, giving both players a payoff of 1 or both players play their second action, giving both players a payoff of 0. The latter equilibrium is ruled out by the notion of properness. So, properness selects the equilibrium in which both players use the pure strategy of playing their first action for sure as the unique outcome of the game. In this section we have discussed the concept of Nash equilibrium and some of its refinements for games in which all players have a finite set of strategies, resulting in payoff functions which are linear in their own strategy variables. Noncooperative games and in particular the two-player games have a wide applicability in micro economic theory in general and in particular in the fields of industrial organization for instance spatial location models as introduced by Hotelling 1929, see Goeree 1997 or Webers 1996 and the references mentioned in these dissertations, or auction theory, see McAfee and McMillan 1987, international trade see for instance Krishna 1992, Spencer and Jones 1992 or

9 EVOLUTIONARY GAME THEORY 67 Choi 1995 on the theory of optimal tariffs and protection, or policy analysis see for instance Barro and Gordon 1983 on monetary policy or Van der Ploeg and De Zeeuw 1992 on environmental policy. In essence, game-theoretic models are needed to analyse multilateral decision-making. Well-known examples are the inspection game, the entry-deterrence game, the coordination game, the staghunt game and the advertisement game prisoner s dilemma. In the next section we restrict ourselves to 2-person games with two identical players. 3 EVOLUTIONARY STABLE STRATEGY The theory of refinements has proven to be very helpful in eliminating inadequate outcomes of the game. However, this theory also has its drawbacks. Not only have many different concepts of refinements been developed, the theory also assumes that players are acting according to a very high level of rationality. This leads Ken Binmore in his foreword to the monograph Evolutionary Game Theory of Jörgen W. Weibull to the following conclusion: However, different game theorists proposed so many different rationality definitions that the available set of refinements of Nash equilibria became embarrassingly large. Eventually, almost any Nash equilibrium could be justified in terms of someone or other s refinement. He then continues with: As a consequence a new period of disillusionment with game theory seemed inevitable by the late 1980s. Fortunately the 1980s saw a new development. Maynard Smith s book Evolution and the Theory of Games directed game theorists attention away from their increasingly elaborate definitions of rationality. After all, insects can hardly be said to think at all, and so rationality cannot be so crucial if game theory somehow manages to predict their behavior... the 1990s have therefore seen a turning away from attempts to model people as hyperrational players. This brings us to the question what economists can learn from evolutionary game theory, introduced by biologists in studying the evolution of populations and the individual behavior of its members. Where has evolutionary game theory brought us and where might it be applicable? We will see that evolutionary game theory is able to analyse and to predict the evolutionary selection of outcomes in interactive environments in which the behavior of the players is conditioned or preprogrammed to follow biological, sociological or economical rules. Evolutionary or biological game theory originated from the seminal paper The logic of animal conflict by Maynard Smith and Price 1973, see also Maynard Smith 1974 and Maynard Smith considers a population in which mem-

10 68 G. VAN DER LAAN AND A.F. TIEMAN bers are randomly matched in pairs to play a bimatrix game. The players are anonymous, that is any pair of players plays the same symmetric bimatrix game and the players are identical with respect to their set of strategies and their payoff function. So, for any member of the population, let m be the number of actions and S S m the set of mixed strategies. Furthermore, the payoff function u: S S R 2 assigns to any pair of two players the payoff pair u 1 x, u 2 x, x S S. The assumed symmetry of the bimatrix game implies that u 1 x, y u 2 y, x for any pair x, y S S, which states that the payoff of the first player when he plays x S and his opponent plays y S is equal to the payoff of the second player when the latter plays x S and the first player plays y S. So, any pair plays the symmetric bimatrix game G A, A with A an m m matrix. Observe that it is not assumed that A is symmetric. In the following we denote with x, y u 1 x, y u 2 y, x x Ay the payoff of a player playing x S against an opponent playing y S. In this way all members of the population are symmetric, except in their strategy choice. In biological game theory it is not assumed that the members animals in the population behave rationally. Instead it is assumed that any member is preprogrammed with an inherited, possibly mixed, strategy and that this strategy is fixed for life. Now let x S be the vector of average frequencies with which the strategies are played by the members of the population. So, x k is the average probability or frequency over all members of the population that action k, k 1,..., m, is played. Assuming that the population is very large, the differences between the expected strategy frequencies faced by different members are negligible and the average expected payoff of an arbitrarily chosen member of the population when paired at random with one of the other members is given by x, x x Ax. However, now suppose that a perturbation of the population occurs and that at random a small fraction of the population is replaced by individuals which are all going to play a so-called mutant strategy q. Then the vector of average frequencies becomes y 1 x q and hence the average expected payoff of the incumbent individuals of the population when matched at random with a member of the perturbed population becomes x, y x A 1 x q 1 x, x x, q 3 and the expected payoff of a mutant individual becomes q, y q A 1 x q 1 q, x q, q. 4

11 EVOLUTIONARY GAME THEORY 69 Now the population is said to be stable against mutants if for all q x there is an q 0 such that for all 0 q x, y q, y, where y 1 x q. 5 A strategy x satisfying 5 for any q x with some q 0 is called an Evolutionary Stable Strategy ESS. So, an evolutionary stable strategy is stable against any mutant strategy as long as the number of mutants is small enough. The reasoning above allows for several viewpoints. Originally, the stability condition was only applied for monomorphic populations, i.e. populations in which all individuals are endowed with the same strategy x. In this framework the frequency x k is the probability with which any member of the population plays action k. So, any incumbent individual plays x and hence the expected payoff of an incumbent is equal to the average expected payoff. The concept of Evolutionary Stable Strategy is the central equilibrium concept in the biological game theory about monomorphic populations. From another opponent viewpoint the stability condition can also be applied to a polymorphic population in which each member is preprogrammed with one of the pure strategies. Within this framework, which originated from mathematicians like Taylor and Jonker 1978 see also Zeeman 1981 the frequency x k is the fraction of members preprogrammed with action k. Of course, intermediate cases are possible as well in which multiple mixed strategies are present in the population. Although both viewpoints are allowed, for the moment we restrict ourselves to the case of the monomorphic population and suppose that x S satisfies the best reply condition q, x x, x for all q S. Clearly, this is a sufficient and necessary condition for the strategy pair x, x to be a Nash equilibrium for the symmetric bimatrix game A, A. In this equilibrium both players play the same strategy x. The equilibrium x, x S S is called a symmetric Nash equilibrium and x a symmetric equilibrium strategy. It is well-known that any symmetric bimatrix game has at least one symmetric equilibrium, see e.g. Weibull It should be noted that a symmetric bimatrix game may also have nonsymmetric Nash equilibria in which the players use different strategies. Now, suppose that for a symmetric equilibrium strategy x S the stronger condition q, x x, x holds for any mutant strategy q S. Then it follows from equations 3 and 4 that there is some q 0 such that equation 5 holds and hence the strategy x S is also ESS. However, no symmetric equilibrium strategy x is ESS. To see this, observe that every strategy q is a best reply to x and hence q, x x, x for any q S, when x is a completely mixed strategy. More precisely, for any mixed equilibrium strategy x S it holds that q, x x, x for any q S satisfying q k 0 if x k 0. So, when x is a mixed equilibrium strategy, q, x x, x does not hold for all q x. However, in case that q, x x, x, it follows from equations 3 and 4 that equation 5 still holds

12 70 G. VAN DER LAAN AND A.F. TIEMAN if q, q x, q, i.e. if the incumbent strategy x performs better against the mutant strategy q than the mutant against itself. This is called the second-order best reply condition. This gives us the following formal definition of ESS as originally formulated by Maynard Smith and Price Definition 3.1 Evolutionary Stable Strategy A strategy x S is an Evolutionary Stable Strategy of the symmetric bimatrix game G A, A if it satisfies the two inequalities: i First-order best reply condition: q Ax x Ax for all q S, ii Second-order best reply condition: when q Ax x Ax, then q Aq x Aq for all q x. Clearly, x S satisfies the conditions of definition 3.1 if and only if for any q S, x satisfies equation 5 for some q 0, see e.g. Weibull Furthermore, the first-order condition i of definition 3.1 shows that x, x is a Nash equilibrium for the bimatrix game G A, A if x is ESS. However, the reverse is not true. The symmetric Nash equilibrium strategy x is ESS only if x also satisfies the second-order condition ii. Example 3.2 Coordination game Consider the 2 2 payoff matrix A given by A In this case both players receive a payoff of eight when both play their first action and a payoff of five when both play their second action. When the players play different actions, then the payoff is equal to one for the player using his first action and zero for the player using his second action. Therefore it is optimal for the players to coordinate on the first action, while coordinating on the second action is still better for both players than playing different actions. This bimatrix game has three symmetric equilibrium strategies, given by the two pure strategies e 1 both players play the first action and e 2 both players play the second action and the mixed equilibrium strategy x* 31, 32 both players play their first action with probability equal to 1 3 and their second action with probability equal to 2 3. However, only the two pure equilibrium strategies are ESS. To see this, consider the equilibrium strategy e 1 and let q q 1, q 2 be any mixed strategy with q 1 1. Then q Ae 1 8q 1 8 e 1 Ae 1 and so conditions i and ii of definition 3.1 are satisfied. Analogous, for any mixed strategy q q 1, q 2 with q 2 1 we have q Ae 2 q 1 5q 2 1 4q 2 5 e 2 Ae 2 and again the conditions i and ii of definition 3.1 are satisfied. Now consider the strategy x*. Then, for any q S we have that q Ax* 10 3 x* Ax* and the first-order condition i is satisfied with equality. Now, take q e 1. Then

13 EVOLUTIONARY GAME THEORY 71 q Aq 8 x* Aq 8 3 and hence the second-order condition ii is not satisfied. It follows that x* is not ESS. By excluding equilibria not satisfying the conditions of definition 3.1 we see that the ESS condition yields a refinement on the set of Symmetric Nash equilibria. More precisely, we find that if x is ESS, then x, x is a symmetric proper Nash equilibrium, see e.g. Van Damme 1987, page 224. According to Weibull 1995, page 42, we can say that an ESS satisfies the equilibrium condition and is cautious, i.e. is robust with respect to low-probability mistakes such that more costly mistakes are less probable than less costly mistakes. Again, the reverse implication is not true. When x, x is a symmetric proper Nash equilibrium, then x is not necessarily an ESS. Even stronger, not any symmetric bimatrix game has an ESS for instance the unique symmetric equilibrium strategy of the Rock-Scissors-Paper game is not ESS, see Weibull 1995, page 39, while it has been proven recently by Van der Laan and Yang 1996 that any symmetric bimatrix game has a symmetric proper equilibrium. To summarize the above results let NE denote the set of symmetric Nash equilibrium strategies, PE the set of symmetric proper equilibrium strategies and ES the set of Evolutionary Stable Strategies. Then we have that ES PE NE and PE Ø. For further characterization results on the set of Evolutionary Stable Strategies we refer to e.g. Bomze 1986, Van Damme 1991 or Weibull In these references many results can be found with respect to the structure of the set of ESS, the relation between this set and other refinements of the Nash equilibrium and the conditions guaranteeing the existence of ESS. For instance, any nondegenerate symmetric bimatrix game with two pure strategies has at least one ESS. In this paper we also skip the discussion about weaker evolutionary criteria. We only mention the weaker concepts of neutral stability introduced by Maynard Smith 1982 and robustness against equilibrium entrants introduced by Swinkels Setwise evolutionary stability criteria have been given by e.g. Thomas 1985 and Swinkels Furthermore, Binmore and Samuelson 1994 provide alternative stability concepts for games in which ESS does not exist. 4 REPLICATOR DYNAMICS Evolutionary game theory combines the static concept of Evolutionary Stable Strategy with the dynamic concept of replicator dynamics, a notion formalized by Taylor and Jonker 1978, see also e.g. Zeeman 1981, Bomze 1986, Van Damme 1987 and Weibull Within the framework of replicator dynamics or population dynamics, it is assumed that all individuals are preprogrammed to play a pure strategy. Within such a framework a strategy vector x S has to be

14 72 G. VAN DER LAAN AND A.F. TIEMAN interpreted as the state of the population with x k the proportion of individuals playing action k, k 1,..., m, when paired with an opponent to play the symmetric bimatrix game G A, A. Within this framework individuals are assumed to be paired at random and each member of the population is assumed to be engaged in exactly one contest at the time. Furthermore, the payoff to an individual is assumed to represent fitness, measured by the number of offspring. So, more succesful individuals get more offspring. Finally it is assumed that in this asexual world the individuals breed true, so that each child inherits its single parent s strategy. Then in the next generation the fraction of more successful members in the population will be higher and the fraction of less successful members will be lower. Modeling this process in continuous time results in a system of differential equations known as the replicator dynamics. Given a population state x x t S at time t, the expected payoff of a member playing action k, k 1,..., m, is given by e k Ax. The growth rate ẋ k x k of the share of k players is given by comparing the payoff or fitness of their strategy with the average fitness of the population x Ax. This gives the system of differential equations: ẋ k e k Ax x Ax x k, k 1,..., m. 6 Taking for granted the theory of differential equations see for instance the introduction given in Hirsch and Smale 1974 this system has a unique solution x t, x 0, t 0, for any initial point x 0 x 0 S. Furthermore, summing up the equations 6 over all k 1,..., m, we get that m k 1 ẋ k 0 because m k 1 x k 1 and hence we have that S and all its faces is invariant, i.e. any trajectory starting in a face of S stays in the same face of S. So, m k 1 x k t 1 for all t and x k t 0 for any t 0ifx k s 0 for some s 0. The latter property says that if at a certain time the fraction of members playing action k is equal to zero, then it will always remain zero and it has always been zero. On the other hand x k t 0 for all t 0ifx k0 0, so an action will survive forever if it is available at t 0. Of course, this does not exclude that x k t converges to zero if t goes infinity, i.e. it may happen that a trajectory starting in the interior of S converges to the boundary. Now, a Nash equilibrium strategy x S of the monomorphic population in which all members play x is said to be asymptotically stable for the polymorphic population in which x k denotes the fraction of members playing action k, if there is a neighborhood X of x, i.e. an open set X in S containing x, such that any trajectory of the replicator dynamics starting at x 0 X converges to x. The following result is due to Taylor and Jonker 1978, see also Hines 1980 or Zeeman 1981, and can be seen as the basic result relating replicator dynamics with Evolutionary Stable Strategies.

15 EVOLUTIONARY GAME THEORY 73 Theorem 4.1 Every Evolutionary Stable Strategy x S is asymptotically stable for the replicator dynamics given in equation 6. The theorem says that for any ESS there is a neighborhood such that any trajectory of the replicator dynamics starting in this neighborhood converges to this ESS. However, the reverse is not true. The replicator dynamics may converge to a strategy not being ESS. Moreover, a trajectory does not always converge to some limit point x* iftgoes to infinity. It may happen that the trajectory path is a cycle on S or moves outwards toward a hyperbola. For other properties of the trajectories, for instance when in the limit the replicator dynamics wipes out all strictly dominated strategies if initially all strategies are present, we refer to e.g. Weibull 1995 or Van Damme In these references many results are stated about the relation between asymptotically stable equilibria of the replicator dynamics and refinements of the Nash equilibrium, for instance that any asymptotically stable strategy is a perfect Nash equilibrium strategy. For a further discussion about the replicator dynamics see also Mailath 1992 and Friedman Here we want to restrict ourselves to a more detailed discussion of the results for 2 2 symmetric matrix games. Let the payoff matrix A be given by A a b d c. It is well-known that the set of Nash equilibrium strategies is invariant with respect to adding the same constant to the payoffs of one player for a given pure strategy of the other player. From this it follows that we can distinguish three types of 2 2 games see also e.g. Weibull 1995 or Friedman Type I: a c 0 d b. In this case the second action strictly dominates the first one and hence e 2 is the unique symmetric equilibrium strategy of the game. Moreover, x e 2 is ESS and it can be shown that the replicator dynamics converges to e 2 for any strict positive initial strategy x 0. In case d a and hence c b this type represents the well-known Prisoner s Dilemma Game PDG and the Nash equilibrium yields the nonefficient payoff d for both players. So, in this case the replicator dynamics leads to a nonefficient stable state and starting from a state x 0 close to e 1 the average payoff or fitness of the population is decreasing along the trajectory from almost equal to a at the starting point to d at the limit point. The case d b 0 a c is similar with the first strategy as the dominant strategy.

16 74 G. VAN DER LAAN AND A.F. TIEMAN Type II: a c 0 and d b 0. This class of games is known as Coordination Games CG and has three symmetric equilibrium strategies, namely the two pure strategies e 1 and e 2 and the mixed equilibrium strategy d b x* a d b c, a c. The highest payoffs are obtained in the a d b c pure strategy equilibria. Although all three equilibrium strategies are proper, only the two equilibrium strategies are ESS and are asymptotically stable see also example 3.2. The replicator dynamics converges to e 1 e 2, respectively if d b x 10 a d b c. Type III: a c 0 and d b 0. A typical example of this class of games is the classic Hawk-Dove Game HDG of Maynard Smith and Price Taking d 1 2b, a 0 and c 0, a member of the population gets in a contest with his rival a payoff of a 0 of he fights plays hawk and 0 if he does not plays dove when his rival plays hawk, while the payoff is equal to b 0, and 1 2b, respectively, when his rival plays dove. In this case there are two asymmetric equilibria, namely e 1, e 2 and e 2, e 1. Furthermore there is one symmetric equilibrium x*, x* with x* as given under type II games, i.e. x* is the unique symmetric equilibrium strategy. This strategy is also the unique ESS and the replicator dynamics converges to this strategy for any strictly positive initial strategy vector x 0. Observe that for type I and type II games the replicator dynamics converges to a pure evolutionary stable strategy, whereas for type III games the dynamics converges to a mixed ESS. In the latter case the dynamics converges to a population in which a fraction x 1 * plays hawk and a fraction 1 x 1 * play dove, so that in such a stable population two types of individuals can be distinguished. Of course, in equilibrium both types have the same average fitness. We have also seen that for 2 2 games the replicator dynamics always leads to an ESS when starting at an interior point of the strategy space. So, for the 2 2 case the replicator dynamics always converges. This does not hold in case of games with more than two actions. Until now we have discussed the notion of Evolutionary Stable Strategy and the concept of replicator or population dynamics. We now come to the question where this biological game theory has brought us. What is the meaning of this theory for economics? This question will be addressed in the next sections. 5 PARETO IN EFFICIENCY OF ESS We have seen that the concept of Evolutionary Stable Strategy gives a further refinement of the Nash equilibrium concept. If an ESS exists, then it is a proper

17 EVOLUTIONARY GAME THEORY 75 equilibrium strategy and therefore ESS is cautious and robust against trembles. Moreover, an ESS is stable against mutants and is asymptotically stable with respect to the replicator dynamics. This latter result shows that rational behavior is not necessary to obtain such a sophisticated equilibrium. A population of genetic players inherited with the behavior of their parents is able to reach an ESS when the number of offspring is determined by the fitness of the strategy of the players. However, the application of this result in economics is not straightforward. First of all, the replicator dynamics always leads to Nash equilibria. Unfortunately, Nash equilibria often reveal bad outcomes for the society as a whole. They may suffer from the tragedy of the commons. This is nicely illustrated by the PDG. The Nash equilibrium yields the worst possible outcome, both players can obtain higher payoffs by playing the dominated strategy. In contrast, in empirical prisoner s dilemma-like situations cooperative behavior can be observed rather frequently. This has inspired many authors to develop game-theoretic models in which the players cooperate by playing the dominated strategy. Many papers on repeated games are devoted to this topic. Unfortunately, biological evolutionary game theory is not very helpful in explaining cooperative behavior. Even worse, in the Hawk-Dove game the Nash equilibrium selected by the ESS outcome is the worst possible Nash equilibrium. Taking a d 0 and hence b 0 and c 0, it can easily be seen that both players obtain more payoff in either of the two equilibria in pure strategies. So, ESS does not sustain cooperative behavior in type I or type III 2 2 games. This might explain that many theoretical papers on evolutionary games in economic journals have been focussed explicitly on coordination games. Indeed, for such games coordination is sustained by ESS. Furthermore, the most efficient outcome has the largest region of attraction. From this it follows that to bring the population from one of the ESS outcomes in coordination games to the other, considerably less mutants are needed to go from the worst ESS to the best ESS than vice versa. Adding small probabilities of mutations to the dynamic process, Kandori, Mailath and Rob 1993, see also Young 1993, show that in the long run the evolutionary process converges to the best possible outcome. Within this framework mutants can be seen as individuals who are experimenting by deviating from the preprogrammed strategy. For further results on this topic we also refer to Ellison 1993 and 1995, Robson and Vega-Redondo 1996, and Bergin and Lipman For type I and type III 2 2 games we may conclude that the replacement of the usual assumption of rationality in economics by the biological fitness criterion is not of any help in sustaining cooperative behavior, because basically both assumptions lead to best reply strategies and therefore the only possible outcomes are the Nash equilibria, which are not Pareto-efficient. Other ideas have to be exploited to model the sustaining of cooperation. Recent literature in the field of sociology and psychology offers a way out by replacing the economic rationality approach by procedural rationality, see e.g. Simon 1976, specifying a rule of behavior. The Tit-for-Tat strategy, well-known from repeated game theory,

18 76 G. VAN DER LAAN AND A.F. TIEMAN can be seen as an example of such a rule of behavior. We return to this subject in section 8. 6 LEARNING AND IMITATION AS REPLICATOR DYNAMICS As indicated by e.g. Smith 1991, most theories on convergence in economic settings still leave ample room for improvement. Unfortunately, the biological concept of fitness driving the dynamics of the replicator dynamics cannot be applied directly to economics. The genetic mechanism of natural selection has to be replaced by a social mechanism of learning and imitation. Here we encounter several problems. First, natural selection by fitness results in a dynamic process in which frequencies of the strategies played by the members are adjusted through differences in the number of offspring. Members with more successful strategies get more offspring, implying that the adjustment of the frequencies is an autonomous process. Individual members of the population do not make rational choices. But adjusting of behavior on an individual level and hence rational behavior is an essential feature of learning in economic settings. The implications of replacing the fitness mechanism by individual learning have been discussed in e.g. Crawford Crawford 1995 considers the repeated play of coordination games and addresses the question whether experimental results reported by for instance Van Huyck, Battalio and Beil 1990 and 1991 can be explained by a learning process in order to adjust the strategies. Second, in the replicator dynamics the strategy frequencies in the population are adjusted according to their fitness. As a consequence the frequencies of all strategies with a fitness above the average level are increasing, also when these strategies are not a best reply. However, within a framework of learning and imitation is it usually assumed that rational players will replace their strategies by best reply strategies. So, in such a process only the frequencies of the best reply strategies may be expected to increase. This raises the question whether a learning or imitation process can justify the replicator dynamics as defined in equation 6. The research on this topic has gone into several directions. A number of authors have addressed the form of the replicator dynamics. In the original system 6 the growth rate of the frequencies is given by the average expected payoff of the strategies. It has been shown that the weaker form of monotonicity of the growth rates in the expected payoffs is sufficient to preserve most of the results. Results along these lines have been obtained by e.g. Nachbar 1990, Friedman 1991, Matsui 1992, Samuelson and Zhang 1992, Björnerstedt 1993, and Joosten On the other hand, research has been done addressing the question whether it is possible to formulate a learning or imitation process resulting in the replicator dynamics as given in equation 6, see e.g. Björnerstedt and Weibull 1993, Gale, Binmore and Samuelson 1995, and Schlag Inthe latter paper the players show imitative behavior: never imitate an individual that

19 EVOLUTIONARY GAME THEORY 77 is performing worse than oneself, and imitate individuals doing better with a probability proportional to how much better they perform. It is shown that this behavioral rule results in an adjustment process that can be approximated by the replicator dynamics. Finally, we would like to mention a third problem when replacing the genetic mechanism by a story of learning or imitation. To formulate this problem, we quote Mailath 1992 when he discusses the problem of the often assumed bounded rationality of the players. He states: Players in these models are often quite stupid. For example, in evolutionary models, players are not able to detect cycles that may be generated by the dynamics. The criticism is that the models seem to rely on the players being implausibly stupid. Why are the players not able to figure out what the modeler can? Recall that in the traditional theory, players are often presumed to be computationally superior to the modeler. Here we come to the question that when frequencies are adjusted by a learning process, why then would players adapt their strategies according to a nonsophisticated process like the replicator dynamics? As we have seen before, convergence of the replicator dynamics cannot be guaranteed. In fact, the replicator dynamics is equivalent with the Walrasian tatonnement process in general equilibrium theory and suffers from the same weaknesses. However, in general equilibrium theory several globally convergent price adjustment processes based on path-following methods have been designed, see for instance Van der Laan and Talman 1987 and Kamiya In these processes cycling is prevented by taking the starting price vector into account. In Van den Elzen and Talman 1991 and Yang 1996 the path-following technique is modified to convergent strategy adjustment processes for noncooperative games in normal form. The latter process always leads to a proper Nash equilibrium. Instead of these deterministic processes convergence can also be obtained by using stochastic differential equations. For these so-called urn processes we refer to e.g. Arthur 1988 or Dosi, Ermoliev and Kaniovski It might be interesting to apply both techniques to the framework of a monomorphic population and to search for an interpretation of such processes as sophisticated learning or imitation processes. 7 EVOLUTIONARY STABILITY IN NONSYMMETRIC GAMES Applying evolutionary game theory to economic situations we often encounter the problem that the conditions of a single symmetric population and pairwise random matching are not met. In many economic situations we have to deal with interaction between more than two individuals and/or interaction between individuals from several distinct populations. Nevertheless, as argued by e.g. Fried-