Game Theory: Lecture 3

Transcription

1 Game Theory: Lecture 3 Lecturer: Pingzhong Tang Topic: Mixed strategy Scribe: Yuan Deng March 16, 2015 Definition 1 (Mixed strategy). A mixed strategy S i : A i [0, 1] assigns a probability S i ( ) 0 to each action, s.t. A i S i ( ) = 1. A mixed strategy profile is denoted as S = (S 1,, S n ). Under mixed strategy profile S, the expected utility for player i is u i (S) = ( a A i S i( ) ) u i (a). Definition 2 (Mixed strategy Nash equilibrium). S = (S1,, S n) is a mixed strategy Nash equilibrium (MSNE), if for all i, u i (S i, S i) u i (S i, S i) for all S i (A i ) Here, (A i ) is the set of the possible distributions of A i. The advantage of MSNE is that it is guaranteed to exist in any normal-form games. The proof of Nash can be found in Section [1]. 1 Compute an MSNE 1.1 Best response analysis Consider the following game (penny matching game) with two players and each with 2 actions. Let H T H 1,-1-1,1 T -1,1 1,-1 p denote the probability that the row player plays H and q denote the probability that the column player plays H. Moreover, f(q) is the function that given q, compute the best probability for player 1 to play H. g(p) is defined in the similar ways. Given q, the utility for player 1 is a follows H: 1 p + ( 1) (1 p) = 2p 1; T : ( 1) p + 1 (1 p) = 1 2p; Thus, we conclude that f(q) is equal to 0, if 0 q < 1/2; [0, 1], if q = 1/2; 1, if 1/2 < q 1; Similarly, we can compute g(p) in the same way and obtain that g(p) is 0, if 0 p < 1/2;

2 [0, 1], if p = 1/2; 1, if 1/2 < p 1; Notice that, for a MSNE, it must be that p f(q) and q g(p). Thus, the only MSNE for this game is p = 1/2 and q = 1/ Enumerate Support Proposition 1 (Characterization of MSNE). Given a mixed-strategy profile S and suppose the utility of player i is u i. Then, S is MSNE if and only if 1. Given S i, the expected utility of i to every action such that S i () > 0 equals to u i ; 2. Given S i, the expected utility of every action such that S i (), is not larger than u i ; Notice that pure strategy Nash equilibrium (PSNE) is also MSNE. With the probability, let us find all MSNE for the following game. B S X B 4,2 0,0 0,1 S 0,0 2,4 1,3 Notice that (B, B), (S, S) are PSNE; For MSNE, observe that player 1 must mix B and S, and let S 1 (B) = p; For player 2, we enumerate her strategy support, the set of actions with positive probability. 1. (B, S): 2p = 4(1 p) p + 3(1 p); No solution; 2. (B, X): 2p = p + 3(1 p) 4(1 p); Learn that p = 3/4 and let S 2 (B) = q, 4q = 1 q q = 1/5. Thus, one MSNE is [(3/4, 1/4), (1/5, 0, 4/5)]; 3. (S, X): Given (S, X), S dominates B for player 1; No solution; 4. (B, S, X): 2p = 4(1 p) = p + 3(1 p); No solution; To sum up, this game contains three MSNE! Remark 1. However, computing NE in general games is PPAD-complete. The proof can be found in Chapter 2 [2]. 2 Zero-sum game Definition 3 (Zero-sum game). In a zero-sum game, for every action profile a = (a 1, a 2 ) A 1 A 2, the total utility is u 1 (a) + u 2 (a) = 0. Intuitively, two players have exactly opposite preferences. Moreover, zero-sum games contain many desirable features. Commitment = Equilibrium; Nash equilibrium can be computed easily; All Nash equilibria have the same values; Equilibrium strategies are interchangeable; 2

3 {maxmin} = {minmax} = {Nash equilibrium}; Definition 4 (Maxmin strategy). A maxmin strategy for player i is arg max min u i (s i, s i ) s i s i Here, max si min s i u i (s i, s i ) is i s maxmin value. Interpretation: A strategy that s player i s worst-case utility in case where all the others happen to play a strategy profile that cause the greatest harm to this player. In other words, i commits to a strategy s i, given that i choose s i that minimizes u i (In zero-sum game, this is equivalent to u i, what is i s best commitment? Maxmin value is sometimes called security level: a value that can be guaranteed by player i alone. Definition 5 (Minmax strategy). A minmax strategy for player i is arg min s i max u i (s i, s i ) (In zero-sum game, same thing as maxmin!) s i Here, min si max s i u i (s i, s i ) is i s minmax value. Interpretation: the maxmin punishment I can impose on others. (In repeated games, this punishment is used to sustain cooperation.) 2.1 Minimax Theorem, von Neumann, 1928 The proof can also be found in Section [1]. One important application of Minimax theorem is Yao s principle. Theorem 1 (Minimax Theorem). In any finite 2-person, zero-sum game, in any NE, each player receives a utility equal to both his maxmin and minmax value. Proof. Let Si, S i be any NE. i s utility in this NE is V i, his maxmin value is V i and minmax value is V i. We prove for the case V i = V i and the remaining case is left for homework. First of all, we can not have V i > V i, for otherwise, i would deviate from Si to his maxmin strategy and obtains at least V i. However, by definition of NE, we have Thus, it must be V i = V i. V i = max S i u i (S i, S i ) V i = min S i u i (S i, S i ) V i = min S i u i (S i, S i ) Since V i = max min u i (S i, S i ) S i S i V i V i From the proof, we can notice that Si arg max S i min S i u i, so Si must be a maxmin strategy. Following are the corollaries and further results on zero-sum games. Any NE strategy is a maxmin strategy/min max strategy; 3

4 {maxmin} and {minmax} are the same; {maxmin, minmax} is an NE; (homework) Thus, {NE strategies} = {maxmin} = {minmax}. Moreover, in zero-sum games, it does not hurt to announce my strategy before the game starts. 2.2 Computation How to compute a NE/minmax/maxmin strategy? Consider the following rock-paper-scissor game. R P S B 0,0-1,1 1,-1 P 1,-1 0,0-1,1 S -1,1 1,-1 0,0 Denote P rock, P paper, P scissor for the probability of the strategy player 1 takes. Then, we have u 2 (rock) = P scissor P paper ; u 2 (paper) = P rock P scissors ; u 2 (scissor) = P paper P rock ; So, player s minmax strategy is arg min P max{p s P p, P r P s, P p P r } subject to P s + P r + P p = 1 Thus, we get P s = P r = P p = 1/ Linear programming review The standard form of LP subject to n i=1 c ix i n j=1 jx j i = 1,, m x i 0 i = 1,, n Matrix form subject to c T x Ax b x 0 Note: It is ok that we do not have an objective. The duality of the LP is as follows. minimize subject to b T y A T y w y 0 Weak duality: If x is a feasible solution for the primal maximization problem and y is a feasible solution for the dual minimization problem, then weak duality implies f(x) g(y) where f and g are the objective functions for the primal and dual problems respectively. 4

5 Strong duality: primal and dual solutions are equivalent Complementary slackness: y i > 0 A i x = b i, x j > 0 A T j y = c j In general, the maxmin strategy can be obtained by the following LP. z subject to S i ( ) u i (, a i ) z S i ( ) = 1 S i ( ) 0 Similarly, we can write the LP for the minmax strategy as follows. minimize z subject to S i ( ) u i (, a i ) z S i ( ) = 1 S i ( ) Computation: domination by a mixed strategy Problem 1: given a i, does there exist a mixed strategy S i dominates a i in the strict sense? Consider the following LP, subject to S i ( )u i (, a i ) > u i (a i, a i) S i ( ) = 1 S i ( ) 0 However, in LP, strict inequalities, i.e. (< and >), are not allowed, because they may lead to illdefined problems. The correct solution is as follows, minimize S i ( ) subject to S i ( )u i (, a i ) u i (a i, a i) S i ( ) 0 To apply the above program, we need to increase u i first such that for all, a i, u i (, a i ) > 0. It is easy to show that such operation does not change the solution. Lemma 1. Let OBJ denote the optimal value of the second program. S i exists iff OBJ < 1. Proof., if s i = 1 and it is indeed a feasible solution for the first program, then in the second program, the summation of s i can be reduced;, increase s i a little bit so that all constraints becomes strict; Problem 2: given a i, does there exist a mixed strategy S i dominates a i Consider the following LP, in the weak sense? subject to S i ( )u i (, a i ) u i (a i, a i) S i ( ) = 1 S i ( ) 0 5

6 However, at least one constraint in the first line must be strict! In order to fix it, consider the following LP, [ a i S i ( )u i (, a i ) u i (a i, a i) ] subject to S i ( )u i (, a i ) u i (a i, a i) a i S i ( ) = 1 S i ( ) 0 Lemma 2. S i exists iff OBJ > 0. Finally, notice that all the programs in this subsection can be applied to the case when restricted to a subset of S i. 2.4 Computation: optimal strategy to commit to Problem: Suppose i promises to player S i and i best responds to S i. Find the optimal S i. (Notice that, in zero-sum game, S i is just the maxmin strategy) In general, we enumerate each a i and solve the following LP S i ( )u i (, a i ) subject to S i ( )u i (, a i ) S i ( )u i (, a i ) a i A i S i ( ) = 1 S i ( ) 0 Interpretation: We wish a i to be the best response, given S i ( ). After enumeration, choose the optimal solution and its corresponding S i. But, why enumerate all a i is enough? References [1] Yoav Shoham and Kevin Leyton-Brown. Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, New York, NY, USA, [2] Noam Nisan, Tim Roughgarden, Eva Tardos, and Vijay V. Vazirani. Algorithmic Game Theory. Cambridge University Press, New York, NY, USA,