Outline for today Stat155 Game Theory Lecture 16: Evolutionary game theory Peter Bartlett October 20, 2016 Nash equilibrium 1 / 21 2 / 21 A strategy profile x = (x1,..., x k ) S 1 Sk is a Nash equilibrium for utility functions u 1,..., u k if, for each player j {1,..., k}, max u j (x j, x j) = u j (xj, x j). x j Sj If the players play these mixed strategies xj, nobody has an incentive to unilaterally deviate: each player s mixed strategy is a best response to the other players mixed strategies. There is a population of individuals. There is a game played between pairs of individuals. Each individual has a pure strategy encoded in its genes. The two players are randomly chosen individuals. A higher payoff gives higher reproductive success. This can push the population towards stable mixed strategies. 3 / 21 4 / 21
Consider a two-player game with payoff matrices A, B. Suppose that it is symmetric (A = B ). Consider a strategy x. Think of x as the proportion of each pure strategy in the population. If x is invaded by a small population of mutants z x, will x survive? x s utility: z s utility: x A (ɛz + (1 ɛ)x) = ɛx Az + (1 ɛ)x Ax z A (ɛz + (1 ɛ)x) = ɛz Az + (1 ɛ)z Ax 1 z Ax x Ax (x, x) is a Nash equilibrium. 2 If z Ax = x Ax then z Az < x Az. Hawks and Doves Payoff Fix v = 2, c = 2. H D H (-1,-1) (2,0) D (0,2) (1,1) x = (1/2, 1/2) is a Nash equilibrium. Is it an ESS? Consider a mutant pure strategy z. For z = (1, 0) (that is, H), z Az = 1 < 1/2 = x Az. For z = (0, 1) (that is, D), z Az = 1 < 3/2 = x Az. So x is an ESS. Rock-Paper-Scissors 5 / 21 Outline 6 / 21 Payoff R P S R 0-1 1 P 1 0-1 S -1 1 0 x = (1/3, 1/3, 1/3) is a Nash equilibrium. Is it an ESS? For a mutant pure strategy z, z Ax = 0 = z Az. So x is not an ESS. Cycles can occur, with the population shifting between strategies. 7 / 21 8 / 21
Theorem Every ESS is a Nash equilibrium. This follows from the definition: 1 z Ax x Ax 2 If z Ax = x Ax then z Az < x Az. Proof: If every pure strategy z satisfies z Ax x Ax, then every mixed strategy z (mixture of pure strategies) must obey the same inequality. Hence, (x, x) is a Nash equilibrium. The converse is true for strict Nash equilibria. A strategy profile x = (x1,..., x k ) S 1 Sk is a strict Nash equilibrium for utility functions u 1,..., u k if, for each player j {1,..., k}, for all x j Sj that is different from xj, u j (x j, x j) < u j (x j, x j). A Nash equilibrium has u j (x j, x j ) u j(x j, x j ). By the principle of indifference, only a pure Nash equilibrium can be a strict Nash equilibrium. 9 / 21 Outline 10 / 21 Theorem Every strict Nash equilibrium is an ESS. Proof: A strict Nash equilibrium has z Ax < x Ax for z x, so both conditions defining an ESS are satisfied. 1 z Ax x Ax 2 If z Ax = x Ax then z Az < x Az. 11 / 21 12 / 21
Pure versus mixed An ESS is a Nash equilibrium (x, x ) satisfying, for all e i x, if e i Ax = x Ax, then e i Ae i < x Ae i. What about invasion by a mixed strategy? Say that a symmetric strategy (x, x ) is evolutionarily stable against mixed strategies (ESMS) if it is a Nash equilibrium and, for all mixed strategies z x, if z Ax = x Ax, then z Az < x Az. (ESS is sometimes defined this way, e.g., Leyton-Brown and Shoham) Clearly, every ESMS strategy is an ESS. Theorem For a two-player 2 2 symmetric game, every ESS is ESMS. Proof When there are only two pure strategies, the space of mixed strategies, 2, is one-dimensional. So insisting that a small perturbation in the direction of one or other pure strategies leads to a strict decrease in utility is equivalent to insisting that a perturbation in the direction of any mixture of those pure strategies. For instance, if q > p, ( ) ( ) p q (1 ɛ) + ɛ 1 p 1 q ( ) p + ɛ(q p) = 1 p + ɛ(1 q (1 p)) ( ) p + ɛ q p 1 p (1 p) = 1 p + ɛ q p 1 p (0 (1 p)) ( = 1 ɛ q p ) ( ) ( p + 1 ɛ q p ) ( ) 1. 1 p 1 p 1 p 0 13 / 21 Outline 14 / 21 ESS might not be ESMS Consider a game with three actions, with an ESS at e 1 : 1 1 1 A = 1 0 20. 1 20 0 1 (e 1, e 1 ) is a (pure) Nash equilibrium: e 1 Ae 1 = 1 1 = e j Ae 1. 2 e 1 is an ESS: e j Ae 1 = e 1 Ae 1 = 1, and e j Ae j = 0 < 1 = e 1 Ae j. 3 e 1 is not ESMS: For x = (1/3, 1/3/1/3), x Ax = 5 > 1 = e 1 Ax. The text gives an example of a mixed ESS that is not ESMS. 15 / 21 16 / 21
Multiplayer evolutionarily stable strategies Multiplayer evolutionarily stable strategies Consider a symmetric multiplayer game (that is, unchanged by relabeling the players). Suppose that a symmetric mixed strategy x is invaded by a small population of mutants z: x is replaced by (1 ɛ)x + ɛz. Will the mix x survive? x s utility: u 1 (x, ɛz + (1 ɛ)x, ɛz + (1 ɛ)x) = ɛu 1 (x, z, x) + ɛu 1 (x, x, z) + (1 2ɛ)u 1 (x, x, x) + O(ɛ 2 ) z s utility: ɛu 1 (z, z, x) + ɛu 1 (z, x, z) + (1 2ɛ)u 1 (z, x, x) + O(ɛ 2 ). (Suppose, for simplicity, that the utility for player i depends on s i and on the set of strategies played by the other players, but is invariant to a permutation of the other players strategies.) 1 u 1 (z, x 1 ) u 1 (x, x 1 ) x is a Nash equilibrium. 2 If u 1 (z, x 1 ) = u 1 (x, x 1 ) then for all j 1, u 1 (z, z, x 1, j ) < u 1 (x, z, x 1, j ). Example: Polluting Factories 17 / 21 Example: Polluting Factories 18 / 21 ESS? (p, p, p) with p = Pr(purify). Symmetric Nash equilibria (p, p, p) with p = Pr(purify): p = (3 + 3)/6 0.79. p = (3 3)/6 0.21. p = 0. (Karlin and Peres, 2016) (Karlin and Peres, 2016) (Karlin and Peres, 2016) 1 Is u 1 (purify, purify, x) < u 1 (x, x, purify)? i.e., 1 > p 2 + p(1 p) + 3(1 p) 2? Equivalent to p > (3 3)/3. 2 Is u 1 (pollute, pollute, x) < u 1 (x, x, pollute)? 3 > p 2 + (3 + 4)p(1 p) + 3(1 p) 2? Equivalent to p > 1/3. Equilibrium at p = (3 + 3)/6 is an ESS. The other equilibria are not. 19 / 21 20 / 21
Outline 21 / 21