April 29, 200 CHAPTER : EVOLUTIONARY EQUILIBRIUM Some concepts from biology have been applied to game theory to define a type of equilibrium of a population that is robust against invasion by another type of strategy or state. In evolutionary game theory, there is assumed to be a large population that plays a mixed strategy as a whole: either each player plays a mixed strategy or the mixed strategy gives the probability that a random person plays a given strategy. An individual or invading sub-population plays against a randomly chosen person from the population as a whole in a given symmetric game. Rather than assume rationality of the players, the population is assumed to maintain a state with the maximum fitness, which is measured by the payoff: therefore, the population is assume to adjust to the maximum payoff. We assume that a symmetric game is fixed with a finite number of possible actions {s,..., s n } and payoffs u(s i, s j ) = a i j. Let { = p = (p,..., p n ) T : p i 0 for all i, } p = i be the simplex of probability vectors giving mixed strategies. In the context of evolutionary game theory, an element p is called a state (or strategy) of the population. A state p can be thought of as either (i) the mixed strategy played by each individual in the population or (ii) the probability distribution of an individuals within a large population playing each action s j with probability p j. We consider the repeated random pairing of an individual against a population in state p, so the individual encounters the states s j with probability p j. The payoff of an individual who takes action s i against a population with state p is assumed to be U(s i, p) = a i j p j = (e i ) T Ap = e i Ap, j where e i is the standard unit vector with a in the i th -coordinate and 0s in the other coordinates. In the same way, the payoff of a state q playing against a state p is assumed to be U(q, p) = i j q ia i j p j = q T Ap = q Ap. The latter quantity U(q, p) can be thought of as either (i) an individual playing mixed strategy q against the whole population playing strategy p or (ii) an invading sub-population which plays the various strategies s j with probability q j. Definition (29.2). A state ˆp is called a Nash equilibrium provided that it is a symmetric Nash equilibrium, i.e., U(ˆp, ˆp) U(q, ˆp) for all q, and the state q = ˆp has the largest payoff in response to a population in the state ˆp, or ˆp B(ˆp). Definition (cf. 97.). A state ˆp is an evolutionary stable state (or evolutionary stable strategy), ESS, provided that for any q in with q ˆp, there exists an ɛ(q) > 0 such that U(ˆp, ɛq + ( ɛ)ˆp) > U(q, ɛq + ( ɛ)ˆp) and for all 0 < ɛ ɛ(q). This means that if a small amount of a new state q invades to form a new state q ɛ = ɛq + ( ɛ)ˆp then the intruder q has a lower payoff playing against than new state q ɛ than the original state ˆp has playing against q ɛ. In the definition of an ESS, we can think of q representing either (i) a mutation my individuals making a random choice of a strategy and ɛ the proportion of the population with this mutation, or (ii) an invading sub-population with state q with size ɛ in relation to the original population.
2 CHAPTER : EVOLUTIONARY EQUILIBRIUM Rational Game Theorem Set of player Payoff measured in payoff Strategy is chosen by a player Equilibrium: no player can do better Comparison Evolutionary Game Theory Population from which the set of players is drawn Fitness of state Strategy is inherited or endowed by a player ESS: no small mutation in the population can survive The following theorem gives conditions for a Nash equilibrium and an ESS. Osborne takes the assumptions of part (b) of this theorem as the definition (00.) of ESS. Theorem. a. A state ˆp is a Nash equilibrium if and only if the following two conditions hold: (i) e i Aˆp ˆp Aˆp for all i, and (ii) e i Aˆp = ˆp Aˆp for all i with ˆp i > 0. b. A state ˆp is an ESS if and only if the following two conditions hold: (i) ˆp Aˆp q Aˆp for all q (i.e., ˆp is a Nash equilibrium), and (ii) ˆp Aq > q Aq whenever q ˆp and ˆp Aˆp = q Aˆp. c. Let ˆp with ˆp i > 0 for all i. The state ˆp is an ESS iff ˆp Aq > q Aq for all q with q ˆp. d. Any state ˆp is an ESS iff ˆp Aq > q Aq for all q ˆp sufficiently close to ˆp in. Proof. a. If ˆp is a Nash equilibrium, then the inequality (i) holds for all q and so certainly for for all e i. Assume that ˆp i > 0. Then by (i), ˆp Aˆp = ˆp i e i Aˆp ˆp i ˆp Aˆp = ˆp Aˆp. i I i I Since the first and last terms are equal, the inequality is an equality and we must have condition (ii) for all i for which ˆp i > 0. If we assume conditions (i) and (ii), then for any q, q Aˆp = i q ie i Aˆp i q i ˆp Aˆp = ˆp Aˆp. This shows that ˆp is a Nash equilibrium. b. Assume that ˆp is an ESS and q ˆp. Expanding the condition defining an ESS and collecting terms, gives (*) ( ɛ) (ˆp Aˆp q Aˆp ) + ɛ (ˆp Aq q Aq ) > 0. Since (*) holds for small ɛ > 0, we get conditions (i) and (ii) of part (b). Conversely, if conditions (i) and (ii) hold, then (*) holds for all ɛ > 0 sufficiently small. (Note that the second term can be negative so the inequality is not necessarily true for all ɛ.) c. Assume that ˆp is an ESS with ˆp i > 0 for all i. Since ˆp is a Nash equilibrium with ˆp i > 0 for all i, ˆp Aˆp = e i Aˆp for all i. Take any q ˆp and sum with weights q i, ˆp Aˆp = i q i ˆp Aˆp = i q ie i Aˆp = q Aˆp. By condition (ii) of ESS, ˆp Aq > q Aq for this arbitrary q ˆp. This is what we wanted to prove. For the converse, for q ˆp, set q ɛ = ( ɛ)ˆp + ɛq. So by assumption, ˆp Aq ɛ > q ɛ Aq ɛ for all ɛ > 0. Subtracting ( ɛ)ˆp Aq ɛ from both sides of the inequality yields ɛ ˆp Aq ɛ > ɛq Aq ɛ. Dividing the last inequality by ɛ gives the condition for an ESS. d. Since part (c) covered the case with all the ˆp i > 0, we assume that ˆp is an ESS with some ˆp i = 0. Let I (ˆp) = { i : ˆp i > 0 }, I (ˆp) = {q : q i > 0 if i I (ˆp), q j = 0 if j / I (ˆp) }, and I (ˆp) = {q : q i = 0 for some i I (ˆp), }. The set I (ˆp) is a vertex, edge, or face of of where certain q j are zero. The set I (ˆp) is the union of all the faces opposite to I (ˆp). In fact, let r be any state not equal to ˆp. Since i r i = i ˆp i =, i (r i ˆp i ) = 0 and some of the r i ˆp i must be negative. Let j be the index
CHAPTER : EVOLUTIONARY EQUILIBRIUM and t > 0 be such that 0 = ˆp j + t(r j ˆp j ) and all the ˆp i + t(r i ˆp i ) 0. Let q = ˆp + t(r ˆp). Note that q I (ˆp). Since the line segment from ˆp to q is in, t. Thus, r = ( t q + ) ˆp, t with 0 < /t. This shows all states in can be written in this form. For q I (ˆp), q ˆp. Let q ɛ = ɛq + ( ɛ)ˆp. By the definition of ESS, q Aq ɛ < ˆp Aq ɛ for ɛ(q) ɛ > 0. Multiplying this inequality by ɛ and adding ( ɛ)ˆp Aq ɛ to both sides gives q ɛ Aq ɛ < ˆp Aq ɛ. The ɛ(q) can all be taken greater than the same ɛ 0 > 0 for all q I (ˆp) (where we have compactness ). The union of the line segments from ˆp to all q I (ˆp) fill up all of, so we get the inequality for all q ɛ ˆp in a neighborhood of ˆp. The proof of the converse is like the converse for part (c), but is only for small ɛ > 0. Remark. From part (c) of the theorem, it follows that if an interior Nash equilibrium is an ESS, then there cannot be any other Nash equilibrium. In other words, if there is an interior Nash equilibrium and another Nash equilibrium, then the interior Nash equilibrium is not an ESS. Examples ( ) 2 Example. Consider the game with matrix A =. For a state p = (p, p) 0 T, the expected payoffs of e and e 2 are E(e, p) = 2 p and E(e 2, p) = p. These are equal for = 2p, or p = /2. For p < /2, e is preferred and e 2 is preferred for p > /2. The best response in terms of the value q of (q, q) is {} if p < /2 B(p, p) = [0, ] if p = /2 {0} if p > /2. Therefore, the only Nash equilibrium is p = /2. Let ˆp = ( /2, /2) T. Because ˆp is an interior Nash equilibrium, ˆp Aˆp = q Aˆp for all q. Letting q = (q, q) T, Is q Aq = (q, q) T (2 q, q) T = 2q q 2 + 2q + q 2 = 2q 2 and ˆp Aq = ( /2, /2) T (2 q, q) T = /2 2q. 2q 2 < /2 2q, or 0 < 2q 2 2q + ( 2 = 2 q 2 q + ) ( = 2 q 2 2) This last inequality is true for all q /2. Therefore, ˆp is an ESS. ( ) c 2 Example (Example 0.2). The Dove-Hawk game can be modeled with the matrix A = for 0 c > 0, where the first choice is an aggressive bird or hawk and the second choice is passive bird or dove. In comparison to the book, we have taken v = 2 and replaced c by 2c (to eliminate fractions). The expected payoffs for a state p = (p, p) T are E(e, p) = ( c)p+2( p) = 2 (+c)p and E(e 2, p) = p. If 0 < c <, then E(e, p) E(e 2, p) = cp > 0 for all p, e is always preferred, and the only Nash equilibrium is p =. Since e is a strict Nash equilibrium, it an ESS. This equilibrium corresponds to only hawks or aggressive birds surviving.
CHAPTER : EVOLUTIONARY EQUILIBRIUM If c, then E(e, p) = E(e 2, p) for 2 ( + c)p = p, = cp, or ˆp c = /c. The best response in terms of the value q of (q, q) is {} if p < /c B(p, p) = [0, ] if p = /c {0} if p > /c. The only Nash equilibrium where p B(p) is ˆp c = ( /c, (c ) /c). B(p) /c p FIGURE. Best responses For c =, it can be checked directly that ˆp = e is an ESS. For c >, the equilibrium ˆp = ˆp c is interior, so to be a ESS, we need ˆp Aq > q Aq for all q ˆp. Letting q = (q, q), ( c ˆp Aq =, + c ) q = 2q + + c and c c c Therefore, we need q Aq = ( c)q 2 + 2q( q) + ( q) 2 = cq 2 +. 2q + c + c > cq 2 +, cq 2 2q + > 0, or c c (cq )2 > 0. This last inequality is true for all q /c = ˆp c, so ˆp c is an ESS for c >. This equilibrium corresponds to both aggressive and passive birds surviving. Example. Consider the game with matrix A = 0 0. For a state p, the expected payoffs are 0 E(e, p) = p 2 + p, E(e 2, p) = p + p, and E(e, p) = p + p 2. A straight forward consideration shows that these expected payoffs are equal only for p = p 2 = p = /. Let ˆp = ( /, /, T /). Because ˆp is an interior Nash equilibrium, ˆp Aˆp = q Aˆp for all q. We need to show that q Aq < ˆp Aq for q ˆp. The payoffs are ˆp Aq = ( 2 /, 2 /, 2 /)q = 2 / and q Aq = 2q q 2 + 2q q + 2q 2 q. Substituting q = x + /, q 2 = x 2 [ + /, and ] q = / x x 2, q Aq = 2x 2 2x 2 2 2x x 2 + 2 /. The quadratic 2 terms have the matrix, which has determinant > 0 and negative upper left entry. By the test for 2 negative definite, q Aq < 2 / for (x, x 2 ) (0, 0), i.e., for (q, q 2, q ) ˆp. This shows that ˆp is an ESS. Since ˆp is a completely mixed ESS, there cannot be any other Nash equilibrium.
CHAPTER : EVOLUTIONARY EQUILIBRIUM 5 Example. Consider the game given by the matrix A = 0 6 0 5. The expected payoffs against 0 a state of p are E(e, p) = 6p 2 p, E(e 2, p) = p + 5p, and E(e, p) = p + p 2. When p = e, E(e, e ) = 0, which is greater than E(e 2, e ) = and E(e, e ) =, so e = (, 0, 0) is one Nash equilibrium. Since q Ae = q 2 q < 0 = e Ae for all q e, e is a strict Nash equilibrium. Since there are no states q e, with q Ae = e Ae, e is automatically an ESS. For an interior equilibrium, the expected payoffs must all be equal. A direct calculation shows that p = p 2 = p = /, and ˆp = ( /, /, /) is an interior Nash equilibrium. Since there is another Nash equilibrium, ˆp cannot be an ESS. Alternatively, since ˆp Ae = / < 0 = e Ae, ˆp is not an ESS. This example has both an interior Nash equilibrium and one on the boundary; the one on the boundary is an ESS and the one in the interior is not an ESS. Example (Two ESS). Consider the game with 5 7 2 8 6 5. 8 There are three Nash equilibria, one completely mixed equilibrium and two mixed between two states. The expected values in playing against p = (p, p 2, p ) are E(e, p) = 5p + 7p 2 + 2p E(e 2, p) = 8p + 6p 2 + 5p E(e, p) = p + 8p 2 + p. If all three are equal, then we have the equations 5p + 7p 2 + 2p = 8p + 6p 2 + 5p = p + 8p 2 + p p + p 2 + p =. This leads to the equations and = p + p 2 + p 0 = p p 2 + p 0 = 7p + 2p 2 p. Solving the augmented system by row reduction is as follows: 0 0 0 7 2 0 0 9 6 7 5 0 0 2 0 0 0 0 0 0 6 0 0 We get the completely mixed Nash equilibrium of ( 5 2,, 2). If E(e, p) = E(e 2, p) > E(e, p), then p = 0, p = p, and p 2 = p. We get the equation 5p + 7( p) = 8p + 6( p) = p p =. 2
6 CHAPTER : EVOLUTIONARY EQUILIBRIUM Then E(e, p) = E(e 2, p) = 5 + 7 = 26 and E(e, p) = + 8 = 25 < 26. Thus, (,, 0) is a Nash equilibrium. Similarly, if E(e 2, p) = E(e, p) > E(e, p), then p = 0, p 2 = p, and p = p. We get the equation 6p + 5( p) = 8p + ( p) = p p =. Then E(e 2, p) = E(e, p) = 6 + 5 2 = 6 and E(e, p) = 7 + 2 2 = < 6. Thus, ( 0,, ) 2 is a Nash equilibrium. It can be checked that there are not any other Nash equilibria, pure or with E(e, p) = E(e, p) > E(e 2, p). Since there are other Nash equilibria, the completely mixed Nash equilibrium cannot be an ESS. Consider the Nash equilibrium ˆp = (,, 0). If U(ˆp, ˆp) = U(q, ˆp), then q = (q, q, 0). We get the following payoffs, U(ˆp, q) = [ ] q (5 + 8, 7 + 6) = q (29 25) q + 25 = q + 25, U(q, q) = 5q 2 + (8 + 7)q( q) + 6( q) 2 = 5q 2 + 5q 5q 2 + 6 2q + 6q 2 = q 2 + q + 6. To be an ESS, for q, we need q + 25 > q 2 + q + 6 q 2 2q + ( q 2 2 q + ) 6 > 0 > 0 ( q ) 2 > 0. This is indeed true, so this is an ESS. The case for the Nash equilibrium ( 0,, ) 2 is similar. The only p that give the same payoff with U(ˆp, ˆp) = U(q, ˆp) have q = (0, q, q). We get the following payoffs, U(ˆp, q) = (6q + 5( q)) + 2 (8q + ( q)) = ( 6 5 + 6 8 = q +, ) q + 5 + 8 U(q, q) = 6q 2 + (8 + 5)q( q) + ( q) 2 = 6q 2 + q q 2 + 8q + q 2 = q 2 + 5q +. To be an ESS, for q, we need q + > q 2 + 5q + q 2 2q + > 0 ( q 2 2 q + ) ( = q 2 > 0. 9 )
CHAPTER : EVOLUTIONARY EQUILIBRIUM 7 This is indeed true, so this is also an ESS. Example (.2.6: No ESS). Some games have no ESS. Consider γ γ. γ with 0 < γ <. The expected payoffs against a state p are E(e, p) = γ p p 2 + p, E(e 2, p) = p + γ p 2 p, E(e, p) = p + p 2 + γ p. and (i) No pure strategy is a NE. (ii) If p = 0 and p, p 2 > 0, then γ p p 2 = p +γ p 2 iff 0 = ( γ )p +(+γ )p 2, so p = p 2 = 0. It is impossible for all the p i = 0. (iii) The case of p = 0 or p 2 = 0 is similar to case (ii). (iv) The expected values are all equal for p = p 2 = p = /. A calculation shows that these are the only values with all three expected values equal. Thus, p = ( /, /, /) is the only Nash equilibrium. For any pure strategies has a payoff U(e i, p ) = γ / = U(p, p ). Since U(e i, e i ) = γ > γ / = U(e i, p ), p I not an ESS. This shows that the only Nash equilibrium is not an ESS and there are no ESS. Example (.2.5 Bach versus Stravinsky). Because we want to set the problem up in a symmetric manner, we do not use the possible actions of Bach or Stravinsky, but rather of (i) choosing favorite concert F and (ii) deferring to the preference of your friend D. The payoffs are then assumed to be given by the following matrix: F D The payoffs in response to a state of p = (p, p) are [ F (0, 0) D ] (2, ). (, 2) (0, 0) E(e, p) = 2( p) E(e 2, p) = p. They are equal for 2 2p = p, 2 = p, or p = 2 /. For p < 2 /, e is preferred so p = is the best response. For p > 2 /, e 2 is preferred so p = 0 is the best response. The best response correspondence is {} if p < 2 B(p, p) = [0, ] if p = 2 {0} if p > 2. The only symmetric equilibrium is ˆp = (2 /, /). There is no pure strategy symmetric equilibrium, even though we earlier found non-symmetric ones. To check for an ESS, we need to compare the payoffs U(q, q) = 2q( q) + ( q)q = q q 2 U(ˆp, q) = 2 2 ( q) + q = q. and
8 CHAPTER : EVOLUTIONARY EQUILIBRIUM Is q q 2 < q, or 0 < q 2 q + ( = q 2 q + ) 9 ( = q 2 2 ) This latter quantity is positive for q 2 = ˆp. Therefore, this is an ESS.