1 Basic Game Modelling

Transcription

1 Max-Planck-Institut für Informatik, Winter 2017 Advanced Topic Course Algorithmic Game Theory, Mechanism Design & Computational Economics Lecturer: CHEUNG, Yun Kuen (Marco) Lecture 1: Basic Game Modelling, Nash Equilibrium, Two-Person Zero-Sum Games and the Minimax Theorem Our course emphasizes the mathematical formulations and the algorithmic aspect of games/markets. We will focus on games first, and discuss markets latter. 1 Basic Game Modelling Without going deep into the formalism and details of games, one should identify that games usually involve two or more players. The players choose strategies and interact, then they know the outcome, which, in the simplest model, is represented numerically by the payoffs to each player. In applications, details can vary. Sometimes the payoffs for every combination of strategy choices are fully known in advance, while in other applications they are not. The outcomes can also be random. Information also plays a huge part in games. Sometimes only statistical information is available to a particular player. As you probably have expected, having more (and more reliable) information might help the player perform better in the game. By the end of this course, as a game analyst, you are supposed to know how to identify the above elements of the game, and determine the appropriate strategic choice of each player. Example 1: Paper-Rock-Scissors n players play this classical game. As the name already suggested, each player has three strategies. The payoffs, however, can have at least two different schemes. When there is a draw (all three strategies are shown in the same round), all players even. Otherwise, Scheme 1: each winner wins $1, and each loser loses $1. Scheme 2: each loser loses $1, and the money is split evenly among the winner(s). Go ask a kindergarten student. They will tell you not to stick to showing the same strategy. Being in a university, we know more vocabulary strategy should be random. And as you may guess, playing the three strategies uniformly randomly sounds like the best strategy. We will see how to justify this formally. (Well, there is arguably a better way; see this link.) Example 2: Asymmetric Paper-Rock-Scissors The classical game is too boring? a Twist it a bit. We build on Scheme 2 in the previous example. The winner-loser rule is same as before, but now a winner by showing Paper wins a double amount, while every loser (who shows Rock ) correspondingly loses double amount. If you are one of the players, will you still play randomly? If so, will you show Paper with probability higher than 1/3? a Actually not. In the link above, it says there is an International Championship! Example 3: Double Chance There are two players, Alice and Bob. First of all, each player throws a fair dice. After seeing the result, Alice has a choice to throw again or not; if she throws again, only her second throw matters. The player with smaller number will pay the other player with the difference of their numbers. What are the strategies of Alice? If in the first round Alice gets 3 and Bob gets 1, should Alice throw again? 1

2 Example 4: Prime Achiever There are two players, Alice and Bob. Each player can choose a number between 1 and 5 (inclusive). If the sum of their choices is a prime number p, then Alice wins $(p 1) and Bob loses $(p + 1). Otherwise, Alice loses $3 and Bob wins the amount of money which is the sum of their two numbers. What are the strategies of the two players? If Alice and Bob cannot communicate before playing the game, what should they do? If Alice and Bob are allowed to cooperate and communicate before playing the game, what should they do? Example 5: Whose Turn? Francis will play a head-to-head game with Will. If Francis plays alone, he can win with 50% of chance. If Doug offers his consultation to Francis, Francis will win with 70% of chance; if Claire offers her consultation to Francis, Francis will win with 90% of chance; if both Doug and Claire offer their consultation to Francis, Francis will win for sure. When Francis win, he will get a money prize of $1 million. What are the strategies of Francis? Should Francis seek consultation from Claire and/or Doug? If so, how should the money prize be split among them? Example 6: Price Discrimination You are a seller of shampoo. You know that Adam is a gentleman of Type A, while Ben is a gentleman of Type B. From previous sales, you know that the how Type A gentleman values the shampoo follows the probability distribution P [v c] = c 5, for 5 c 12, 7 while how Type B gentleman values the shampoo follows the probability distribution c 2 P [v c] =, for 4 c If you are allowed to sell the shampoo at different prices to Adam and Bob (this is called price discrimination), which prices should you set? If you are not allowed to do so, i.e., you can only set one price for both gentlemen, what price should you set? a a The current example has no game-theoretic flavour, since the only player is the seller. This example is more of making use of statistical information to adjust your choice of price(s). In other words, students who know basic probability theory should be able to find out the answer themselves. This example will be illuminating when we go into the topic of Mechanism Design. You may not know all the answers to the questions raised in the above examples. But you will 1, after this course. 2 Game Bimatrices, Mixed Strategies and Nash Equilibrium 2.1 Game Bimatrices In general, there can be many players in a game. Let s focus on the simplest scenario: when there are two players, named Alice and Bob. Each player has her/his own finite set of strategies. Let Alice s set of strategies be {A 1, A 2,, A n }, and Bob s set of strategies be {B 1, B 2,, B k }. As convention, the payoffs 1 At least partially; in game analysis, sometimes there is not a definite answer. 2

3 to the players are represented by a bimatrix: A 1 A 2. A n B 1 B 2... B k (u 11, v 11 ) (u 12, v 12 )... (u 1k, v 1k ) (u 21, v 21 ) (u 22, v 22 )... (u 2k, v 2k ) (u n1, v n1 ) (u n2, v n2 )... (u nk, v nk ), where u ij and v ij are the payoffs to Alice and Bob respectively, when Alice uses strategy A i and Bob uses strategy B j. When there are more players, the game representation will be a d-matrix, where d is the number of players. Even if each player has only two strategies, the matrix size grows exponentially as d 2 d you can feel that such games are not easy to analyze. In theoretical computer science, communication complexity has been used to show that exponential size of the matrix entries are required to be read in order to compute a good approximation of Nash equilibrium (to be defined formally soon). But the situation is worse; for the above form of bimatrix game where the size is just 2nk, finding its Nash equilibrium in general is believed to be computationally hard. Example 7: Asymmetric Paper-Rock-Scissors Suppose there are only two players for this game. The game bimatrix is P R S P R S (0, 0) (2, 2) ( 1, 1) ( 2, 2) (0, 0) (1, 1) (1, 1) ( 1, 1) (0, 0). Observe that the payoff values to the two players always sum up to zero. This is called a two-person zero-sum game. Example 8: Prime Achiever The game bimatrix is (1, 3) (2, 4) ( 3, 4) (4, 6) ( 3, 6) (2, 4) ( 3, 4) (4, 6) ( 3, 6) (6, 8) ( 3, 4) (4, 6) ( 3, 6) (6, 8) ( 3, 8) (4, 6) ( 3, 6) (6, 8) ( 3, 8) ( 3, 9) ( 3, 6) (6, 8) ( 3, 8) ( 3, 9) ( 3, 10). 2.2 Mixed Strategies and Nash Equilibrium As already illustrated by some of the examples, in most cases, when a game is played repeatedly, players should not stick with the same strategy all the time. A natural alternative is to choose strategy randomly, 3

4 which we call mixed strategy. Let p = (p 1, p 2,, p n ) denote a probability distribution on the n strategies of Alice, and let q = (q 1, q 2,, q k ) denote a probability distribution on the k strategies of Bob. For i = 1, 2,, n, if Alice sticks with strategy i, her expected payoff is V i A( q) := q j u ij. Thus, if Alice uses mixed strategy p, by linearity of expectation, her expected payoff is V A ( p, q ) := p i VA( q) i = p i q j u ij. Analogously, one can define V j B ( p) := p i v ij and V B ( p, q ) := q j V j B ( p) = q j p i v ij. Definition 1: Nash Equilibrium The pair ( p, q ) forms a Mixed Nash Equilibrium (NE) if each player cannot raise her/his own payoff by changing her/his own mixed strategy, while the other player remains fixed on his/her choice of mixed strategy. Mathematically, it is formally defined as: p, V A ( p, q ) V A ( p, q ) and q, V B ( p, q ) V B ( p, q ). When there are more than two players, the basic idea is the same. We will use the following notation. Suppose there are l players. The tuple p = ( p 1, p 2,, p l ) forms a Nash Equilibrium if for any Player i = 1, 2,, l, and for any p i, V i ( p i, p i ) V i ( p i, p i ); here, p i denote the collection of all mixed strategies in p of all players, except player i. If there exists some p i = q j = 1, then the Nash Equilibrium is called a Pure Nash Equilibrium. In other words, at NE, it is best for each player to stick with her/his own mixed strategy. While the definition is intuitive enough, it is not clear at all whether NE must exist or not. John Nash proved that it must exist in 1950 s. This is one of the key foundational results in the field of game theory. Example 9: Prime Achiever In the lecture, we will verify that the pair ( ) 38 p = 1427, , , , 0 and q = ( ) , 0, , , forms a NE of the game. The following proposition is useful for verifying whether a given pair ( p, q ) is NE. Proposition 1: Verifying Nash Equilibrium of a Two-Person Game Given the pair ( p, q ), let V A := max n V i A ( q) and V B := max k V j B ( p). ( p, q ) is NE if and only if the followings hold: (a) p i > 0 implies V i A = V A. (b) q j > 0 implies V j B = V B. 4

5 Proof: ( ) By (a), V A ( p, q) = p i VA( q) i = p i V A + p i VA( q) i = V A. i:p i >0 i:p i =0 But for any p, V A ( p, q) = p i VA( q) i p i V A = V A. So we have verified that for any p, V A ( p, q) V A ( p, q). Similarly, we can prove that for any q, V B ( p, q) V B ( p, q ). ( ) Suppose the contrary, i.e., either (i) there exists a strategy i of Alice such that p i > 0 but VA i is strictly less than V A, or (ii) there exists a strategy j of Bob such that q j > 0 but V j B is strictly less than V B. WLOG, let s assume case (i) holds. Let l i be a strategy of Alice such that V l A = V A. Consider p such that i # i, l, p i # = p i # ; p i = 0; p l = p l + p i. In other words, p is the same as p, except that the probability density on strategy i is fully shifted to strategy l. Then V A ( p, q) = V A ( p, q) + p i (V A V i A) > V A ( p, q), thus ( p, q) does not form a NE, a contradiction. Example 10: Prime Achiever Continued Now, forget the exact values of p, q. We only remember that the first four entries of p is non-zero and the fifth entry of p is zero, and we remember that all but the second entries of q is non-zero. Can we compute to resume the NE exactly? Write p = (p 1, p 2, p 3, p 4, 0) and q = (q 1, 0, q 3, q 4, q 5 ). By Proposition 1, we have V 1 A( q) = V 2 A( q) = V 3 A( q) = V 4 A( q) = V A, while V 5 A ( q) V A. The above equality constraints, together with the constraint q 1 + q 3 + q 4 + q 5 = 1, give a linear system with 5 variables (namely q 1, q 3, q 4, q 5 and V A ) and 5 linear equality. Assuming that you have done a linear-algebra course, you should know how to resume the value of q. Similarly, you can do the same thing to resume p. Thus, verifying a NE can be done in polynomial time. However, unlike those algorithms you learnt for computing roots of a quadratic equation or sorting a sequence of real numbers which are typically efficient, we do not know any polynomial-time algorithm for computing a NE of a two-person Nash game. And, there probably does not exist such an efficient algorithm. We have just said verifying a NE is easy, while finding a NE is probably hard. Students familiar with the complexity classes P and NP probably feel an interesting analogue here. There is a key difference, though. The complexity class NP is about decision problems, i.e., to determine a solution exists or not. Take the canonical NP-complete problem 3-SAT as example, the problem is to determine if a satisfying True/False assignment exists or not. However, in our case, a NE is guaranteed to exist by Nash s proof, thus determining the existence of a solution is not a meaningful problem, as the answer is YES for sure. Finding a NE is a search problem. One of the representative complexity classes for search problems is PPAD. We will not go very deep into the complexity-theoretic context, but at least you should know that finding NE for two-person games, which is denoted as a search problem called 2-NASH, is known to be PPAD-complete, indicating that a polynomial-time algorithm is unlikely to exist. If time allows, we will discuss an exponential-time algorithm for solving 2-NASH. 5

6 While in general finding a NE is believed to be computationally hard, for some interesting sub-classes of games, we can still hope for an efficient algorithm. This is what we do next. 3 Two-Person Zero-Sum Games, and the Minimax Theorem 3.1 Two-person Zero-sum Game Two-person zero-sum game is a special type of bimatrix game, in which v ij = u ij for every i, j. By convention, we represent such game by a matrix instead of a bimatrix, and each entry of the matrix is simply the value of u ij. For instance, the bimatrix in Example 7 can be represented by the following simpler matrix: P R S P R S Two-person Zero-sum game is the game with absolute conflict the gain of a player must come entirely at the loss of the other player. There is no room for the two players to cooperate. For instance, Facebook has recently developed a face-recognizer, and surely Facebook wants it be as accurate as possible. However, a privacy-concerning user has a totally opposite objective: the user want to twist photo to make the recognizer inaccurate. This is a scenario of absolute conflict between Facebook and the user. Suppose you are Alice and you want to compute p. Without the knowledge of q, what is a safety choice? One approach is: compute a p such that no matter what q is, you can guarantee an expected payoff of at least U, for an as-large-as-possible U. This can be done via the following linear program (LP): max U such that j [k], u ij p i p i = 1 i [n], p i 0. This is called the lower game value, as it serves a lower bound on Alice s expected payoff. There is one important observation: such computation is not limited to zero-sum game! As a game analyst, you need to switch your identity at wish. Now you are Bob. There is nothing to stop you from doing the same thing as Alice just did. Try to write down the linear program yourself; let LP-B 1 denote this LP. This time, you have a lower bound on the expected payoff of Bob, and its negation serves as an upper bound on Alice s expected payoff. Thus, the negation is called the upper game value. For non-zero-sum games, the upper game value is defined in the following way. Suppose Bob does not concern his own payoff, but his aim is to harm Alice. Then his approach is to compute a q such that no matter what p is, Alice cannot have an expected payoff larger than U, for an as-small-as-possible U. Then his LP, denoted by LP-B 2, will be for computing the upper game value is: min U such that j [k], U u ij q j U q j = 1 j [k], q j 0. 6

7 For zero-sum games, you can observe that LP-B 1 and LP-B 2 are equivalent. This is no surprise, because in the situation of absolute conflict, improving own benefit is just the same as harming your opponent! Example 11: Asymmetric Paper-Rock-Scissors For the game in Example 7, the above linear program becomes Using the substitution p 3 = 1 p 1 p 2, we have max U such that 2p 2 + p 3 U 2p 1 p 3 U p 1 + p 2 U p 1 + p 2 + p 3 = 1 i [3], p i 0. max U such that 1 p 1 3p 2 U 3p 1 + p 2 1 U p 1 + p 2 U p 1 + p 2 1 i [2], p i 0. Observe that a feasible solution to the linear program is p 1 = p 2 = 1/4 and U = 0 (and hence p 3 = 1/2). Can we attain an U which is strictly larger than 0? The answer is NO. To see why, suppose U > 0. Then by the first and the second constraints, we have 3p 1 + p 2 > 1 > p 1 + 3p 2, which implies p 1 > p 2. However, the third constraint implies p 1 < p 2, a contradiction. Indeed, you can check that p = q = (1/4, 1/4, 1/2) together form the unique NE of this game. Thus, perhaps in contrary to some students guesses, albeit the payoff from showing Paper and winning over opponent s Rock will earn double amount, the players should show Paper less probably (than in the classical case). 3.2 LP Strong Duality, and The Minimax Theorem First of all, recall the LP Strong Duality Theorem. 2 The standard form on a LP is Its dual is max c T x subject to Ax b and x 0. min b T y subject to A T y c and y 0. The LP Strong Duality Theorem states that if the first LP is feasible and bounded, then the second LP is also feasible and bounded, and furthermore, the optimal objective values of both LPs are equal. In Alice s LP, there are n + 1 variables, and note that there is no positivity constraint on U. To fit into the standard form, we need to rewrite U = u 1 u 2, and impose the positivity constraint on u 1, u 2. Also the equality constraint n p i = 1 needs to be split into two inequality constraints, namely n p i 1 2 For our purpose, we ignore the cases when an LP is infeasible or unbounded. 7

8 and n p i 1. Alice s (transformed) LP is Its dual is max u 1 u 2 such that j [k], ( u ij ) p i + u 1 u 2 0 p i 1 p i 1 i [n], p i 0, and u 1, u 2 0. min v 1 v 2 such that i [n], ( u ij ) q j + v 1 v 2 0 q j 1 q j 1 j [k], q j 0, and v 1, v 2 0. Yes, this is exactly the same as Bob s LP (modulo a sign switch) after the same transformation as we just did to Alice s original LP! Thus, the LP Strong Duality Theorem implies that the lower game value equals to the upper game value; the value is called the game value of this two-person zero-sum game. The impacts are stated formally in the following Minimax Theorem: Theorem 1: The Minimax Theorem Given a two-person zero-sum game, let p be an optimal solution to Alice s LP, and q be an optimal solution to Bob s LP. Then ( p, q ) is a NE of the game, and at every NE ( p, q) of the game, the expected payoff of Alice V A ( p, q) is the same, which is the game value as defined above. Proof: First, we verify that ( p, q ) forms a NE. Let g be the game value of the game. Bob s LP guarantee that for any p, V A ( p, q ) g. Alice s LP guarantee that for any q, V A ( p, q ) g; in particular, V A ( p, q ) g. Thus, for any p, V A ( p, q ) g V A ( p, q ). Similarly, we can prove that for any q, V B ( p, q ) V B ( p, q ). For the second part, it can be proved by noting the following three (in)equalities. First, observe the identity V B ( p, q) = V A ( p, q) for zero-sum game. Next, note that V A ( p, q) V A ( p, q) g, due to the definition of NE and p is an optimal solution to Alice s LP with optimal value g. Similarly, V B ( p, q) V B ( p, q ) g. Putting them altogether forces V A ( p, q) = g. The importance is that any finite two-person zero-sum game can be unambiguously solved by solving two LPs, which can be done in polynomial time. Note, however, it remains possible to have other NE. One should note that for general sum game, the lower and upper game values are not necessarily equal. 8