Evolutionary Stability of Pure-Strategy Equilibria in Finite Games

Transcription

1 GAMES AND ECONOMIC BEHAVIOR 2, ARTICLE NO GA Evolutionary Stability of Pure-Strategy Equilibria in Finite Games E Somanathan* Emory Uniersity, Department of Economics, Atlanta, Georgia Receive January 8, 996 Sufficient conitions for pure-strategy Nash equilibria of finite games to be Ž Lyapunov stable uner a large class of evolutionary ynamics, the regular monotonic selection ynamics, are iscusse In particular, it is shown that in almost all finite extensive-form games, all the pure-strategy equilibria are stable In such games, all mixe-strategy equilibria close to pure-strategy equilibria are also stable Journal of Economic Literature Classification Numbers: C70, C Acaemic Press INTRODUCTION One way of approaching the equilibrium selection problem in games that has attracte much attention recently is to moel evolution A common approach is to suppose that a game is playe by large populations, one for each player The shares of the populations playing ifferent strategies varies over time in a payoff-relate manner The vector of shares for each population is moele as a ynamical system with the growth rates of the shares being relate to payoffs, an the limiting behavior of the system is examine For one such class of ynamics, the regular monotonic selection ynamics, it has been shown that a necessary an sufficient conition for pure-strategy equilibria to be asymptotically stable is that they be strict Ž Samuelson an Zhang, 992 This is a strong restriction for extensive-form games, since no equilibrium in which there are information sets not reache in equilibrium can be strict * I am grateful to Eric Maskin an to an anonymous referee an eitor for very helpful comments an suggestions An earlier version of this paper was part of my PhD thesis at Harvar University $2500 Copyright 997 by Acaemic Press All rights of reprouction in any form reserve

2 254 E SOMANATHAN But examples have also been given to show that the replicator ynamics Ž which fall into the above class nee not remove weakly ominate strategies Ž van Damme, 987; Samuelson an Zhang, 992; Sethi, 996 This paper shows that such situations, far from being rare, are in fact typical A theorem giving sufficient conitions for stability of pure-strategy Nash equilibria of finite games in regular monotonic selection ynamics is presente Here, stability Ž efine below refers to Lyapunov stability, which is weaker than asymptotic stability The theorem implies that all the pure-strategy Nash equilibria of almost all finite extensive-form games are stable Examples of games to which the theorem applies inclue the well-known Chain-store Game, an the Ultimatum Game Žwhen the possible ivisions are restricte to finitely many values An extension to some kins of mixe-strategy equilibria is iscusse 2 STABILITY 2 The Moel Consier a finite two-person game in normal form with strategies a,,an an b,,bn available to players an 2, respectively Exten- 2 sions to any finite number of players are straightforwar, an the two-player case is use for notational convenience only The game is playe by two large populations of players of type an 2 Only pure strategies are playe Let the shares of the populations playing strategies ai an bj at time t be enote by s Ž t an u Ž i j t, respectively The payoff to strategy ai for a type- player, Ž i, u, is a continuous function of the state of population 2, ut Ž Žu Ž Ž t,,un t 2 2 The payoff to a type-2 player using strategy j is the analogous expression Ž s, j 2, assume to be continuous in s Evolutionary pressure exerte by payoff ifferentials on the popula- Stability has receive less attention than asymptotic stability because the latter is always robust to small perturbations of the ynamics, unlike the former ŽSee, for example, the iscussion in Weibull, 995, pp 2425 However, recent work has shown that it is not generally true that the aition of payoff-unrelate rift to the moel will lea to a stable equilibrium being estabilize See Gale, Binmore, an Samuelson Ž 995 on the Ultimatum Ž one-stage ivie-the-ollar Game, Sethi an Somanathan Ž 996 on a common-pool-resource exploitation game, an Binmore an Samuelson Ž 995 for general results, further examples, an a iscussion of empirical tests 2 A special case of this occurs in the ranom-matching moel in which pairs of players n encounter each other at ranom, so that i, u 2 j Biju j, where Bij enotes the payoff to strategy ai for player when 2 uses b j Examples in which payoffs are possibly nonlinear functions of u, the playing the fiel moel, are in Maynar Smith Ž 982 An economic example is in Sethi an Somanathan Ž 996

3 EVOLUTIONARY STABILITY IN GAMES 255 tion compositions or state Ž s, u is moele by supposing that the shares of strategies with higher payoffs grow faster This is mae precise below using terminology that follows Samuelson an Zhang Ž 992 Let f: S n S n 2 R n an g: S n S n 2 R n 2, where S k is the Ž k -imensional simplex The system i s f i Ž s,u, i,,n, Ž u j g j Ž s,u, j,,n 2, Ž 2 is a selection ynamic if it satisfies for all s, u S S 2 : f an g are Lipschitz continuous; ie, M 0 st s, u, s, u S S 2, 4 max fž s,u fž s,u, gž s,u gž s,u M Ž s,u Ž s,u, Ž 3 where enotes the Eucliean norm, n n2 i f 0 g, Ž 4 i j Ž s,u S n S n 2, s 0f Ž s,u 0, Ž 5 i Ž s,u S n S n 2, u 0g Ž s,u 0 Ž 6 j j i j The Lipschitz conition ensures that for any initial state, the selection ynamic efines a unique path originating from the state The other conitions ensure that the shares always a up to one an remain nonnegative The following regularity conition is also use: A selection ynamic Ž f, g satisfies regularity if for all i, s*, u* S S 2 with s i 0: lim Ž s, u Ž s*, u* i fiž s,u fiž s*, u* s 0 exists an is finite, an a symmetric conition hols for g Regularity allows the proportional growth rates of the shares to be efine an continuous on the whole of S n S n 2, not just its interior Finally, the key iea that evolution results in strategies with higher payoffs increasing their share at the expense of strategies with lower payoffs is capture as follows: A regular selection ynamic Ž f, g is

4 256 E SOMANATHAN 2 monotonic if for any s, u S S an i, i, j, j, fiž s,u fiž s,u Ž i,u Ž Ž i,u Ž, s s gjž s,u gjž s,u 2Ž s, j Ž 2Ž s, j Ž u u This says that the proportional growth rate of a strategy with a higher payoff will be greater than that of a strategy with a lower payoff Note that the replicator ynamic, i j s s Ž i, u Ž s, u, i,,n, Ž 7 i i u u Ž s, j Ž s,u, j,,n, Ž 8 j j n i i where s, u s i, u, erive from biological moels of evolution, is a regular monotonic selection ynamic The escription of how play will evolve is now complete To arrive at a preiction about long-run behavior, one nees a notion of stability of the ynamics A point Ž s*, u* is sai to be stable in the ynamics Ž f, g if, for any neighborhoo U of Ž s*, u*, there exists a neighborhoo V of Ž s*, u* with V U such that ŽsŽ 0,u0 VŽst,ut U for all t 0 Stable points therefore have the property that paths starting close to them o not rift away from them A point Ž s*, u* isasymptotically stable if it is stable an there exists a neighborhoo U of Ž s*, u* such that every path starting in U converges to Ž s*, u* By efinition, asymptotic stability implies stability i j 22 Sufficient Conitions In looking for stable states of regular monotonic selection ynamics one can confine attention to Nash equilibria since all stable states are Nash equilibria Ž Samuelson an Zhang, 992 It is well known that strict Nash equilibria are asymptotically stable in ynamics of the type escribe here In fact, Samuelson an Zhang Ž 992 have shown that a necessary conition for pure-strategy equilibria to be asymptotically stable in regular monotonic selection ynamics is that they be strict But in extensive-form games, strictness of equilibria is the exception, not the rule Given a pure-strategy equilibrium, the payoff to a strategy that iffers from the equilibrium strategy only at information sets that are never reache in equilibrium must be equal to the payoff to the equilibrium strategy In such games, asymptotic stability may be too restrictive as a solution concept

5 EVOLUTIONARY STABILITY IN GAMES 257 The following result provies conitions uner which pure-strategy equilibria that are not strict satisfy the weaker requirement of stability These conitions are quite weak In fact, all the pure-strategy Nash equilibria in almost all finite extensive-form games satisfy them Žsee Corollary 2 Almost all or generic is efine as follows: Take a game an consier the class of games with the same game tree but with possibly ifferent payoff vectors at the terminal noes One can ientify each game in this class with the vector of payoff vectors at its terminal noes an, thus, the class itself with a Eucliean space A property hols for almost all games if, for each class, the subset of games for which it hols is open an ense in the corresponing Eucliean space Before stating the theorem, some further notation is introuce: e i will enote the vector in S n that has a in the ith place an zeros elsewhere an e 2 will enote the vector in S n 2 j that has a in the jth place an zeros elsewhere THEOREM Suppose a, b is a Nash equilibrium such that, e 2 i, e 2 i 2,,k, Ž 9, e 2 i, e 2 i k,,n, Ž 0 e, e, j j2,,k, Ž e, e, j jk,,n Ž An suppose there exists a neighborhoo X S S 2 of Že, e 2 such that for all Ž s, u X satisfying si 0 only if i k an u j 0 only if j k 2, the following inequalities hol: Ž, u Ž i, u i k, Ž 3 2Ž s, 2Ž s, j jk 2 Ž 4 Then Ž a, b is stable in eery regular, monotonic selection ynamic on S n S n 2 Proof Let f, g n be a regular, monotonic selection ynamic on S S n 2 It will be convenient to efine k n s s, s s, c i i i2 ik k2 n2 u u, u u, c j j j2 jk2

6 258 E SOMANATHAN k n f f, f f, c i i i2 ik k2 n2 g g, g g c j j j2 jk2 We will refer to strategies satisfying Ž 9 an Ž, that is, a2 through ak an b through b, as type-c strategies an to strategies satisfying Ž 0 2 k 2 an Ž 2, ak through an an bk through b n, as type- strategies 2 2 Note that the proportional growth rates of s c, s, u c, an u are weighte averages of the proportional growth rates of the shares of corresponing type-c an type- strategies Ž 2 Let U be a neighborhoo of e, e in S S 2 We will fin a neighborhoo V U of this point such that no path starting in V ever leaves V By continuity of the payoffs an ŽŽ 9 2, there exists a close neighborhoo N of Že, e 2, N X, U in which s s, u u, an c c Ž i,u Ž i,u ik, ik, Ž 5 2Ž s, j 2Ž s, j jk 2, jk 2 Ž 6 We will procee as follows: The assumptions on the payoff structure, together with monotonicity guarantee that the shares of the equilibrium strategies o not fall when type- strategies are not present Monotonicity an the other properties of the selection ynamic will be use to show that Ž s s u an Ž u u s c c are boune above in N These relative bouns on the growth of nonequilibrium strategy shares are then use to construct V By Ž 6 an monotonicity, g gc g, u uc u within N Using the continuity of the proportional growth rates Žwhich follows from regularity an the fact that N is close an boune, we get g gc g, for some 0 u uc u We now make the following claim: gž s,u u 0 Ž s,u N Ž 7

7 EVOLUTIONARY STABILITY IN GAMES 259 Proof of claim Suppose there exists Ž s, u N for which gu 0 Then g 0 Also, g u, g u 0 So, g 0 an g 0 Ž c c c since u 0 in N Therefore, g g g 0, a contraiction to Ž 4 c This proves the claim We note for future reference that Ž 7 implies that g s,u 0 s,u N 8 Now Ž 7 also implies g u within N for some 0 Hence, g u within N Ž 9 Now consier s, u in N Suppose f s,u 0 Then, fcž s,u Ž fcfž s,u f f s,u f f s, u u,u,,u,0,,0 c c 2 k 2 f f s,u f f s, u u,u,,u,0,,0 c c 2 k 2 M Ž s,u Ž s, Ž uu,u 2,,u k,0,,0 ' 2 2 for some M 0 n j ' 2 ž / jk2 M u u M uu ' 2 Mu The secon inequality above is obtaine as follows: fs fcsc fs when u 0 Žby Ž 3, Ž 5, an monotonicity So f 0 when u 0 Žotherwise Ž 4 woul be violate Hence, fc fscs when u 0 Since f 0, this means fc f,or fc f 0 when u 0 This establishes the secon inequality The fourth inequality uses the Lipschitz continuity of f So, f s,u Mu s, u N for which f s,u 0 20 c

8 260 E SOMANATHAN So by Ž 9 an Ž 20, within N, g M fc, as long as f 0 Since f 0in NŽby Ž 8 an symmetry ; therefore, within N, g M fcf, if f0 Ž 2 An for those Ž s, u N at which f Ž s,u 0, we have f Ž s,u f Ž s,u c 0 an so, g M fcf0 Putting this together with Ž 2 we get gž s,u M fcž s,u fž s,u gž s,u B Ž s,u N Ž 22 So Ž s s an Ž by a symmetric argument Ž u u c c are boune above in N by Ž u B an Ž s B 2, respectively We will now construct V Let V Ž s, u S n S n 2 Ž 23 Ž 25 hol 4, where 0s u x Ž 23 0scBŽ xu xs Ž 24 0ucB2Ž xs xu, Ž 25 where x is positive an chosen small enough that V N Ž 2 Note that V is a neighborhoo of e, e in S S 2 Next we show that any path starting in V cannot leave V Let Ž s, u V So Ž s, s, u, u satisfies Ž 23 Ž 25 We show that Ž 23 Ž 25 c c must continue to hol as time passes First, note that Ž 23 must continue to hol since u 0 by Ž 8 an s 0 by symmetry Next, note that Ž 24 can be rewritten as s c s Bu Bxx Now since s s Ž u c B, ie, s c s Bu 0, therefore Ž 24 must also continue to hol An by symmetry, Ž 25 must continue to hol as well Hence, any path starting in V cannot leave V This completes the proof

9 EVOLUTIONARY STABILITY IN GAMES 26 The conitions Ž 3 an Ž 4 of Theorem are not necessary for stability as the following example shows 3 The pure-strategy Nash equilibrium where both players play their first strategy in the bimatrix normal-form game,,, 2 2, is stable in the replicator ynamic Ž this is easily checke, but oes not satisfy Ž 3 in the hypotheses of the theorem COROLLARY 2 All the pure-strategy Nash equilibria of almost all finite extensie-form games are stable in eery regular monotonic selection ynamic Proof In almost all finite extensive-form games, the vector of payoff vectors has istinct elements Consier any pure-strategy equilibrium in such a game Strategies that involve eviations from the equilibrium path, when everyone is playing the equilibrium strategies, must yiel strictly lower payoffs These are type- strategies in the terminology of the theorem All other strategies that eviate from the equilibrium yiel payoffs equal to the equilibrium payoffs; hence, they are type-c strategies Žthat is, they satisfy conitions Ž 9 an Ž In fact, the payoff to a type-c strategy against any type-c strategy in the opposing population equals the equilibrium payoff So type-c strategies satisfy conitions Ž 3 an Ž 4 with equality Now we can apply the theorem to conclue that the equilibrium is stable In extensive-form games that are not generic, conitions Ž 3 an Ž 4 from the hypotheses of Theorem may not hol, an so Nash equilibria may not be stable, as the following example shows In Fig, the game shown has a nongeneric extensive form In every neighborhoo of the equilibrium Ž T, L, the payoffs to the other strategies are strictly higher than the payoffs to T an L at all points which place positive probability on D an R So Ž 3 an Ž 4 from the hypotheses of the theorem o not hol Clearly, Ž T, L is unstable in regular monotonic selection ynamics An argument similar to that in Theorem can be mae to show that in almost all finite extensive-form games, each pure-strategy equilibrium has a neighborhoo in the subspace of shares of type-c strategies that consists of stable points More precisely, 3 I am grateful to an anonymous referee for proviing this simple example

10 262 E SOMANATHAN FIG A nongeneric extensive form with an unstable equilibrium THEOREM 3 Let Ž a, b be a pure-strategy Nash equilibrium in a generic two-person finite extensie-form game, where, in the normal form of the game, player has strategies a,,a n, an player 2 has strategies b,,b n 2 Suppose a 2,,a k, an b 2,,bk iffer from a an b, respectiely, only 2 at information sets not reache in equilibrium, while all other strategies inole eiations from the equilibrium path Then there exists a neighborhoo N of Že, e 2 in the subspace of shares of strategies a,,a,b,,b, such k k2 that eery point in N is stable in eery regular monotonic selection ynamic on S n S n 2 Proof Strategies a,,a an b,,b clearly satisfy Ž 9 an Ž 2 k 2 k ; 2 that is, they are type-c strategies Since the game is generic, a,,a k n an b,,b must satisfy Ž 0 an Ž 2 k n ; that is, they are type- 2 2 strategies Hence, by the continuity of the payoff functions, there exists an Ž 2 open neighborhoo N of e, e in S S 2 such that for all Ž s, u 0 N0 Ž 5 an Ž 6 hol Notice that, Ž, u Ž i, u i k,if u0, Ž 26 2Ž s, 2Ž s, j jk 2,if s0 Ž 27 Let f, g be a regular monotonic selection ynamic on S S 2 It will be shown that the set N Ž s, u N : s 0 u 4 0 consists of points that are stable in Ž f, g Let Ž s*, u* N Let U be a neighborhoo of Ž s*, u* We shall fin a neighborhoo V U of Ž s*, u* such that no path starting in V ever leaves V Using the regularity an monotonicity of f an g an Ž 5 an Ž 6, we conclue, as in the proof of Theorem, that there exists a close

11 EVOLUTIONARY STABILITY IN GAMES 263 neighborhoo V U of Ž s*, u* an, 0, such that for all Ž s, u 0 2 V 0, fž s,u s0 Ž 28 gž s,u 2u0 Ž 29 Note that V 0 can be chosen so that Ž s, u V0 Ž Ž ss, s 2,,s k,0,,0, Ž uu,u 2,,u k,0,,0 2 V 0 Ž 30 Next, note that 26 an monotonicity imply that when u 0, Ž Now, by 4, fi f ik s s i i i k f f So when u 0, i k, f k i i fi f Ms Ž 3 k k k s s s s i i i i i i i for some M 0, where the inequality follows from the continuity, hence, bouneness of f s on the simplex Hence, when u 0, fims ik Ž 32 By Lipschitz continuity of f, for any Ž s, u, there exists M 0 such that Ž f Ž s,u f s, u u,u,,u,0,,0 Mu Ž 33 i i 2 k2 2 2

12 264 E SOMANATHAN So there exists B 0 such that for any s, u V, i k, 0 Ž f s,u f s,u f s, u u,u,,u,0,,0 i i i 2 k 2 Ž f s, u u,u,,u,0,,0 i 2 k 2 Ž Ž Mu f s, u u,u,,u,0,,0 by Ž 33, 2 i 2 k 2 2 i Ž 2 k 2 Ž by Ž 28, Ž 30, Ž 3 Mu f s, u u,u,,u,0,,0 Mu 2 Ms Ž by 32 M2 M gž s,u fž s,u by Ž 29, Ž 28 2 B f Ž s,u g Ž s,u, Ž 34 with a symmetric inequality for g 2 4 Let V s, u S S : 35, 36 hol, where s u x, Ž 35 s s, u u xbž xs u ik, jk, Ž 36 i i j j 2 an x 0 is chosen small enough that V V 0 VVU is a neighborhoo of Ž s*, u* Let ŽŽŽ s 0,u0 Ž 0 V To complete the proof, it is enough to show that ŽŽst,ut Vt0 This follows irectly from Ž 28, Ž 29, an Ž 34, an the efinition of V It may be note that Theorem 3 implies that in finite games with generic extensive-forms, there exists a neighborhoo of each pure-strategy equilibrium such that all the mixe-strategy equilibria within it are stable To see this, consier the equilibrium Ž a, b in the game mentione in the statement of Theorem 3 In the light of the theorem, it is enough to show that there exists a neighborhoo U of Že, e 2 such that no point Ž s, u in U with a positive weight on any type- strategy ai can be a Nash equilibrium For a type- strategy a, Ž, e 2 Ži, e 2 i or B B i, where Bij enotes the payoff to a for player when 2 uses b For Ž s, u i j to be an equilibrium with si 0, it must be that n2 n2 j B u B u, j j ij j j which is impossible if U is chosen small enough

13 EVOLUTIONARY STABILITY IN GAMES CONCLUSION It was shown that for a large class of evolutionary ynamics, the regular monotonic selection ynamics, all the pure-strategy Nash equilibria of almost all finite extensive-form games are stable These inclue imperfect equilibria involving the use of weakly ominate strategies Mixe-strategy equilibria in the neighborhoo of a pure-strategy equilibrium in a generic finite extensive-form game are also seen to be stable REFERENCES Binmore, K, an Samuelson, L Ž 995 Evolutionary Drift an Equilibrium Selection SSRI Working Paper 9529, University of Wisconsin, Maison; Games Econ Beha, to appear Gale, J, Binmore, K, an Samuelson, L Ž 995 Learning to be Imperfect: The Ultimatum Game, Games Econ Beha 8, 5690 Maynar Smith, J Ž 982 Eolution an the Theory of Games Cambrige: Cambrige Univ Press Samuelson, L, an Zhang, J Ž 992 Evolutionary Stability in Asymmetric Games, J Econ Theory 57, Sethi, R Ž 996 Evolutionary Stability an Social Norms J Econ Beha Organ 29, 340 Sethi, R, an Somanathan, E Ž 996 The Evolution of Social Norms in Common Property Resource Use, Amer Econ Re 86, van Damme, E Ž 987 Stability an Perfection of Nash Equilibria Berlin: Springer-Verlag Weibull, J W Ž 995 Eolutionary Game Theory Cambrige, MA: The MIT Press