Zero-Sum Games Public Strategies Minimax Theorem and Nash Equilibria Appendix. Zero-Sum Games. Algorithmic Game Theory.

Transcription

1 Public Strategies Minimax Theorem and Nash Equilibria Appendix 2013

2 Public Strategies Minimax Theorem and Nash Equilibria Appendix Definition Definition A zero-sum game is a strategic game, in which for every state s we have i N u i(s) = 0. In a zero-sum game every utility gain of one player results in a utility loss of another player. For instance, this can be used to model situations in which players must divide a common good.

3 Public Strategies Minimax Theorem and Nash Equilibria Appendix Definition Definition A zero-sum game is a strategic game, in which for every state s we have i N u i(s) = 0. In a zero-sum game every utility gain of one player results in a utility loss of another player. For instance, this can be used to model situations in which players must divide a common good.

4 Public Strategies Minimax Theorem and Nash Equilibria Appendix 2-player Zero Sum Games Two players, player I (row player), player II (column player) Representation as a matrix A R k l with k = S I rows and l = S II columns: a 11 a a 1l a 21 a a 2l a k1 a k2... a kl a ij is utility for player I, a ij for player II

5 Public Strategies Minimax Theorem and Nash Equilibria Appendix Examples Matching Pennies ( ) Rock-Paper-Scissors A game with k l: ( )

6 Public Strategies Minimax Theorem and Nash Equilibria Appendix Utility by Matrix Multiplication We denote mixed strategies by x for I and y for II. Computing the utility u I(x, y): ( ) ( ) = 2.48 u I(x, y) = u II(x, y) = k i=1 l x i a ij y j j=1 = ( ( ) a11 a x 1 x 12 a 13 2 a 21 a 22 a 23 = x T Ay ) y 1 y 2 y 3

7 Public Strategies Minimax Theorem and Nash Equilibria Appendix Utility by Matrix Multiplication We denote mixed strategies by x for I and y for II. Computing the utility u I(x, y): ( ) ( ) = 2.48 u I(x, y) = u II(x, y) = k i=1 l x i a ij y j j=1 = ( ( ) a11 a x 1 x 12 a 13 2 a 21 a 22 a 23 = x T Ay ) y 1 y 2 y 3

8 Public Strategies Minimax Theorem and Nash Equilibria Appendix Public Strategy Choices Suppose player I has to move first and pick a public strategy before player II makes his choice. How should I choose his strategy? ( Player II will hurt player I as much as possible. ) In this game II will always answer with column 1. Hence, optimal choice for I is pure strategy 2 or x = (0, 1) T.

9 Public Strategies Minimax Theorem and Nash Equilibria Appendix Public Strategy Choices Suppose player I has to move first and pick a public strategy before player II makes his choice. How should I choose his strategy? ( Player II will hurt player I as much as possible. ) In this game II will always answer with column 1. Hence, optimal choice for I is pure strategy 2 or x = (0, 1) T.

10 Public Strategies Minimax Theorem and Nash Equilibria Appendix Public Strategy Choices Suppose player I has to move first and pick a public strategy before player II makes his choice. How should I choose his strategy? ( Player II will hurt player I as much as possible. ) In this game II will always answer with column 1. Hence, optimal choice for I is pure strategy 2 or x = (0, 1) T.

11 Public Strategies Minimax Theorem and Nash Equilibria Appendix Maximin Strategies Player I picks x, then player II picks y optimally with max y u II(x, y). So II chooses y by max y x T Ay = min y x T Ay. Hence, player I searches for x that maximizes min y x T Ay. Definition The gain-floor of a 2-player zero-sum game is v I = max min x T Ay. x y A strategy x that yields the gain-floor is an optimal strategy for I, called maximin strategy.

12 Public Strategies Minimax Theorem and Nash Equilibria Appendix Maximin Strategies Player I picks x, then player II picks y optimally with max y u II(x, y). So II chooses y by max y x T Ay = min y x T Ay. Hence, player I searches for x that maximizes min y x T Ay. Definition The gain-floor of a 2-player zero-sum game is v I = max min x T Ay. x y A strategy x that yields the gain-floor is an optimal strategy for I, called maximin strategy.

13 Public Strategies Minimax Theorem and Nash Equilibria Appendix Maximin Strategies Player I picks x, then player II picks y optimally with max y u II(x, y). So II chooses y by max y x T Ay = min y x T Ay. Hence, player I searches for x that maximizes min y x T Ay. Definition The gain-floor of a 2-player zero-sum game is v I = max min x T Ay. x y A strategy x that yields the gain-floor is an optimal strategy for I, called maximin strategy.

14 Public Strategies Minimax Theorem and Nash Equilibria Appendix Example Maximin ( Player II will hurt player I as much as possible. I picks row 1 II picks column 2 I gets utility 1 I picks row 2 II picks column 1 I gets utility 1 ) I picks x = (0.5, 0.5), minimum loss for II is 2.5 in columns 2 and 3 I gets utility 2.5! What is x, how large can v I be?

15 Public Strategies Minimax Theorem and Nash Equilibria Appendix Dual Perspective: Minimax Now suppose player II first picks y, then player I picks x optimally with max x u I(x, y) = max x x T Ay. Hence, II searches for y that minimizes max x x T Ay. Definition The loss-ceiling of a 2-player zero-sum game is v II = min y max x T Ay. x A strategy y that yields the loss-ceiling is an optimal strategy for II, called minimax strategy. What is y, how small can v II be? How do v I and v II compare?

16 Public Strategies Minimax Theorem and Nash Equilibria Appendix Dual Perspective: Minimax Now suppose player II first picks y, then player I picks x optimally with max x u I(x, y) = max x x T Ay. Hence, II searches for y that minimizes max x x T Ay. Definition The loss-ceiling of a 2-player zero-sum game is v II = min y max x T Ay. x A strategy y that yields the loss-ceiling is an optimal strategy for II, called minimax strategy. What is y, how small can v II be? How do v I and v II compare?

17 Public Strategies Minimax Theorem and Nash Equilibria Appendix Dual Perspective: Minimax Now suppose player II first picks y, then player I picks x optimally with max x u I(x, y) = max x x T Ay. Hence, II searches for y that minimizes max x x T Ay. Definition The loss-ceiling of a 2-player zero-sum game is v II = min y max x T Ay. x A strategy y that yields the loss-ceiling is an optimal strategy for II, called minimax strategy. What is y, how small can v II be? How do v I and v II compare?

18 Public Strategies Minimax Theorem and Nash Equilibria Appendix Minimax Theorem Intuitively, if both players play optimally, player I should gain at least v I, player II should not lose more than v II. It is easy to show that Lemma It holds that v I v II. Perhaps surprisingly, von Neumann and Morgenstern proved Theorem (Minimax Theorem) In every 2-player zero-sum game it holds that v = vi = vii. The value v is called the value of the game.

19 Public Strategies Minimax Theorem and Nash Equilibria Appendix Minimax Theorem Intuitively, if both players play optimally, player I should gain at least v I, player II should not lose more than v II. It is easy to show that Lemma It holds that v I v II. Perhaps surprisingly, von Neumann and Morgenstern proved Theorem (Minimax Theorem) In every 2-player zero-sum game it holds that v = vi = vii. The value v is called the value of the game.

20 Public Strategies Minimax Theorem and Nash Equilibria Appendix Minimax Theorem by Linear Programming Duality Consider the optimization problem to find x and vi = max x min y x T Ay. Observe: Player II is assumed to play a best response y against x. For a given x, player II sets y j > 0 if and only if his expected loss k i=1 x ia ij in column j is minimal. Hence, v I = l j=1 k x i a ij y j = i=1 l min j=1 k x i a ij For any x and the resulting utility v I obtained by I we thus know i=1 v I k x i a ij for all j = 1,..., l. i=1

21 Public Strategies Minimax Theorem and Nash Equilibria Appendix Gain-Floor Optimization as a Linear Program Maximize subject to v I v I k x i a ij 0 for all j = 1,..., l i=1 k x i = 1 i=1 x i 0 for all i = 1,..., k v I R (1)

22 Public Strategies Minimax Theorem and Nash Equilibria Appendix Loss-Ceiling Optimization as a Linear Program Similar arguments yield a linear program for loss-ceiling minimization. Minimize subject to v II v II l a ij y j 0 for all i = 1,..., k j=1 l y j = 1 j=1 y j 0 for all j = 1,..., l v II R (2) In the appendix we show that this represents the LP-dual of the Gain-Floor Optimization LP.

23 Public Strategies Minimax Theorem and Nash Equilibria Appendix Implications Finding optimal strategies for players I and II yields linear programs that are duals. Strong duality in Linear Programming: Consider a linear program with a feasible optimum solution Let f be the optimal objective function value Then the dual has a feasible optimum solution, objective function value g Strong Duality: It holds that f = g. Thus, strong duality proves the minimax theorem.

24 Public Strategies Minimax Theorem and Nash Equilibria Appendix Example ( ) Max. v I s.t. v I 5x 1 1x 2 0 v I 1x 1 4x 2 0 v I 2x 1 3x 2 0 x 1 + x 2 = 1 x 1, x 2 0 x = (0.4, 0.6) v I = 2.6 v I R Min. v II s.t. v II 5y 1 1y 2 2y 3 0 v II 1y 1 4y 2 3y 3 0 y 1 + y 2 + y 3 = 1 y 1, y 2, y 3 0 y = (0.2, 0, 0.8) v II = 2.6 v II R Is (x, y ) a mixed Nash equilibrium?

25 Public Strategies Minimax Theorem and Nash Equilibria Appendix Example ( ) Max. v I s.t. v I 5x 1 1x 2 0 v I 1x 1 4x 2 0 v I 2x 1 3x 2 0 x 1 + x 2 = 1 x 1, x 2 0 x = (0.4, 0.6) v I = 2.6 v I R Min. v II s.t. v II 5y 1 1y 2 2y 3 0 v II 1y 1 4y 2 3y 3 0 y 1 + y 2 + y 3 = 1 y 1, y 2, y 3 0 y = (0.2, 0, 0.8) v II = 2.6 v II R Is (x, y ) a mixed Nash equilibrium?

26 Public Strategies Minimax Theorem and Nash Equilibria Appendix Mixed Nash equilibrium Corollary A state (x, y) in a 2-player zero-sum game is a mixed Nash equilibrium x and y are optimal strategies for the players. Proof: Exercise

27 Public Strategies Minimax Theorem and Nash Equilibria Appendix Mixed Nash equilibrium Corollary All mixed Nash equilibria in a 2-player zero-sum game yield the same expected utility of v ( v) for player I (II). Note that we can find optimal strategies by solving the linear programs (1) and (2). There are efficient algorithms for solving linear programs, which proves the following result: Theorem In 2-player zero-sum games a mixed Nash equilibrium can be computed in polynomial time.

28 Public Strategies Minimax Theorem and Nash Equilibria Appendix Literature G. Owen. Game Theory. Academic Press, (Chapter 2) Chapter 1 in the AGT book For background on linear programming, duality, and algorithms see: Cormen, Leiserson, Rivest, Stein. Introduction to Algorithms, 3rd edition. MIT Press, (Chapter 29)

29 Public Strategies Minimax Theorem and Nash Equilibria Appendix Constructing the Dual We construct an upper bound on v I for every solution of (1). Consider a solution (v I, x) of (1). We take a linear combination of the constraints to construct an upper bound. In particular, we use multipliers z j and w I: ( z j v I ) k i=1 x ia ij z j 0 for each j and w I k i=1 x i = w I 1 Here z j 0 to keep the correct inequality.

30 Public Strategies Minimax Theorem and Nash Equilibria Appendix Constructing the Dual Now we try to get an upper bound by using the linear combination: v I = ) l k z j (v I x i a ij + w I j=1 i=1 ( l ) z j v I + ( k w I j=1 i=1 j=1 l z j 0 + w I 1 = w I j=1 k i=1 x i ) l a ij z j x i This works if the first inequality is fulfilled, and this holds if the following conditions for coefficients for the v I and x i on l.h.s. and r.h.s. are true: 1 = l j=1 z j (Same ones because v I R.) 0 w I l j=1 a ijz j (Possibly larger ones, because x i 0.) What is the best upper bound w I that can be obtained in this way?

31 Public Strategies Minimax Theorem and Nash Equilibria Appendix Finding the best Upper Bound Minimize subject to w I w I l a ij z j 0 for all i = 1,..., k j=1 l z j = 1 j=1 z j 0 for all j = 1,..., l w I R (3) This linear program is called the dual program of (1), w I and z j are the dual variables. Note that this represents exactly the optimization problem to find the loss-ceiling and an optimal strategy of player II (with w I = v II and z j = y j )!