6.896 Topics in Algorithmic Game Theory February 8, Lecture 2

Transcription

1 6.896 Topics in Algorithmic Game Theor Februar 8, 2010 Lecture 2 Lecturer: Constantinos Daskalakis Scribe: Yang Cai, Debmala Panigrahi In this lecture, we focus on two-plaer zero-sum games. Our goal is to show that linear programming (LP) dualit implies the eistence of an efficientl computable Nash equilibrium in such games. Before describing this result, we introduce some definitions and terminolog. 1 Preliminaries Two-Plaer (Normal-Form) Games. A two-plaer normal-form game is specified via a pair (R, C) of m n paoff matrices. The two plaers of the game, called the row plaer and the column plaer, have respectivel m and n pure strategies. As the plaers names impl, the pure strategies of the row plaer are in one-to-one correspondence with the rows of the paoff matrices, while the strategies of the column plaer correspond to columns. So if the row plaer plas strateg i and the column plaer strateg j, then their respective paoffs are given b R[i, j] and C[i, j]. Plaers ma also randomize over their strategies, leading to mied as opposed to pure strategies. We represent the simple of mied strategies available to the row plaer b m and those available to the column plaer b n. Thus, if m, then is an m-dimensional vector 1, 2,..., m such that 0 i 1 for each i [m] := {1,..., m} and m i=1 i = 1; similarl for n. For a pair of mied strategies m and n, the epected paoff of the row and column plaer are respectivel T R and T C. Nash Equilibrium. A pair of strategies (, ) is said to be a Nash equilibrium iff neither plaer can increase her epected paoff b unilaterall deviating from her strateg. Mathematicall, T R T R, m ; T C T C, n. Since the epected paoff is linear in the plaers mied strategies, the support of a plaer s strateg in a Nash equilibrium should include onl those pure strategies that maimize her paoff given the other plaer s mied strateg. Conversel, an strateg whose support satisfies the above condition must be a Nash equilibrium. This leads to a pair of alternative conditions defining a Nash equilibrium: i s.t. i > 0, i arg ma {e k T R}; k j s.t. j > 0, j arg ma{ T Ce l }, l where e k m, e l n represent the vectors of appropriate dimensions with a single 1 at the coordinate k and l respectivel; these correspond to the pure strategies k and l respectivel for the row and column plaers. Another pair of equivalent conditions asserts that a plaer cannot improve her paoff b unilaterall switching to a pure strateg, i.e. T R e i T R, i [m] T C T Ce j, j [n], where e i and e j respectivel represent the row plaer s pure strateg i and the column plaer s pure strateg j. 2-1

2 Eamples. We give a few classical eamples of two-plaer games. Each is described b a table whose (i, j)-th entr has a pair of values, the first corresponding to R[i, j] and the second to C[i, j]. Rock Paper Scissors (RPS) This is the familiar school-ard game with the same name. Rock Paper Scissors Rock 0, 0-1, 1 1, -1 Paper 1, -1 0, 0-1, 1 Scissors -1, 1 1, -1 0, 0 It can be checked that his game has a unique Nash equilibrium, namel when both plaers randomize uniforml over their strategies: = = 1 3, 1 3, 1 3. Prisoner s Dilemma Two suspects are arrested b the police. The police have insufficient evidence for a conviction, and, having separated both prisoners, visit each of them to offer the same deal. If one testifies for the prosecution against the other and the other remains silent, the betraer goes free and the silent accomplice receives the full 10-ear sentence. If both remain silent, both prisoners are sentenced to onl si months in jail for a minor charge. If each betras the other, each receives a five-ear sentence. Each prisoner must choose to betra the other or to remain silent. Each one is assured that the other would not know about the betraal before the end of the investigation. How should the prisoners act? Silent Betra Silent 1 2, , 0 Betra 0, -10-5, -5 This game has a pure Nash equilibrium where both plaers Betra. Observe that this is not the social optimum, that is the pair of strategies optimizing the sum of the plaers paoffs, since both plaers stand to gain if the simultaneousl switched to Silent. This eposes a ke feature of Nash equilibria that the ma fail to optimize social welfare and are stable onl under unilateral deviations b the plaers. Battle of the Sees A pair of friends with different interests need to decide where to spend their afternoon. The make simultaneous suggestions to each other and will onl go somewhere if their suggestions match. One prefers football and the other prefers theater. Their paoffs are as follows. Theater! Football fine Theater fine 1, 5 0, 0 Football! 0, 0 5, 1 This game has multiple Nash equilibria. In particular, both the squares with non-zero paoffs represent Nash equilibria. In addition, = 1 6, 5 6 and = 5 6, 1 6 is a Nash equilibrium. Thus, this game has 3 Nash equilibria. In fact, as we will establish later this semester, ever non-degenerate game must have an odd number of Nash equilibria! Two-plaer Zero-sum Games. A two-plaer game is said to be zero-sum if R + C = 0, i.e. R[i, j] + C[i, j] = 0 for all i [m] and j [n]. For eample, the RPS game described above is zero-sum. We give another less smmetrical game below. 2-2

3 Presidential Elections (adopted from [3]) Two candidates for presidenc have two possible strategies on which to focus their campaigns. Depending on their decisions the ma win the favor of a number of voters. The table below describes their gains in millions of voters. Moralit Ta-cuts Econom 3, -3-1, 1 Societ -2, 2 1, -1 Clearl, this is also a zero-sum game. Suppose now that the row plaer announces a strateg of 1 2, 1 2 in advance. This is presumabl a suboptimal wa to pla the game, since it eposes the row plaer s strateg to the column plaer who can now react optimall. Given the row plaer s announced strateg, the column plaer will choose the strateg that maimizes her paoff given the row plaer s declared strateg; in this case, she will choose Ta-cuts. More generall, if the row plaer announces strateg 1, 2, then the column plaer has the following epected paoffs: E[ Moralit ] = E[ Ta cuts ] = 1 2. Hence, the paoff that the row plaer will get after reacting optimall to the row plaer s announced strateg is ma( , 1 2 ). Since this is a zero-sum game, the resulting row plaer s paoff is ma( , 1 2 ) min( , ). So if the row plaer is forced to announce his strateg in advance he will choose ( 1, 2 ) arg ma ( 1, 2) min( , ). In other words, the row plaer s optimal announced strateg ( 1, 2 ) can be computed via the following linear program, whose value will be equal to the row plaer s paoff after the column plaer s optimal response to ( 1, 2 ). ma z s.t z z = 1 1, 2 0. Solving this LP ields 1 = 3 7, 2 = 4 7 and z = 1 7. Conversel, if the column plaer were forced to announce her strateg in advance, she would aim to solve the following LP. ma w s.t w w = 1 1, 2 0, which ields the following solution: 1 = 2 7, 2 = 5 7 and w = 1 7. Here is a remarkable observation. = 2 7, 5 7 then If the row plaer plas = 3 7, 4 7 and the column plaer plas The paoff of the row plaer is 1/7 and the paoff of the column paer is 1/7; in particular, the sum of their paoffs is zero. 2-3

4 Given strateg = 3 7, 4 7 for the row plaer, the column plaer cannot obtain a paoff greater than 1 7 since the row plaer gets a paoff of at least 1 7 irrespective of the column plaer s strateg (b the definition of the first LP) and the game is zero-sum; Similarl, given = 2 7, 5 7 for the column plaer, the row plaer cannot obtain a paoff greater than 1 7 since the column plaer gets a paoff of at least 1 7 irrespective of the row plaer s strateg. Hence, the pair of strategies = 3 7, 4 7 and = 2 7, 5 7 is a Nash equilibrium! This were somewhat unepected since the two LPs were formulated independentl b the two plaers without an care/guessing about what their opponent might do. In particular, it seemed a priori suboptimal/too pessimistic for a plaer to announce her strateg and let the other plaer respond. Moreover, there was no reason to epect that the pessimistic strategies for the two plaers are in fact best responses to each other. As we show below, this coincidence happens for a deep reason, it is true in all two-plaer zero-sum games, and is a ramification of the strong LP dualit. 2 Generalization 2.1 LP Formulation Let us now stud general two-plaer zero-sum games. Recall that a two-plaer game is zero-sum iff the paoff matrices (R, C) satisf R + C = 0. As in the previous section, let us write the LP for the row plaer if he is forced to announce his strateg in advance. s.t. ma z T R z1 T T 1 = 1 i, i 0. Notice that the maimum of z is actuall the same as Now let us consider the dual of LP (1): ma min T R. min z s.t. T R T + z 1 T 0 T 1 = 1 j, j 0. LP (1) LP (2) Let z = z. Using the fact that C = R, we can change LP (2) into another equivalent LP. ma z s.t. C z 1 T 1 = 1 j j 0. LP (3) It is not hard to see that the maimum of z is equal to ma min T C = min ma T R. Here is the interesting fact: LP (3) is nothing but the linear program that the column plaer would solve if she were forced to announce her strateg in advance! Using the strong dualit of linear programming we are going to show net that the solutions to LP (1) and LP (3) constitute a Nash equilibrium of the game. 2-4

5 2.2 LP Strong Dualit Since LP (2) is the dual of LP (1), LP strong dualit implies the following: If (, z) is optimal for LP (1) and (, z ) is optimal for LP (2), then z = z. On the other hand, given our construction of LP (3) it follows that, if (, z ) is optimal for LP (2), then (, z ) is optimal for LP (3). Thus, if (, z) is optimal for LP (1) and (, z ) is optimal for LP (3), then z = z. We show net that the solutions of these LPs actuall give us a Nash equilibrium. Theorem 1. If (, z) is optimal for LP (1), and (, z ) is optimal for LP (3), then (, ) is a Nash equilibrium of (R, C). Moreover, the paoffs of the row/column plaer in this Nash equilibrium are z and z = z respectivel. Proof: Since (, z) is feasible for LP (1), and (, z ) is feasible for LP (3): T R z1 T R z (1) C z 1 T C z, T R z,. (2) From equation (2) we know that, if the column plaer is plaing, the row plaer s paoff is at most z z (since we argued above that z = z from strong LP dualit). Moreover, equation (1) implies that if the row plaer plas against, her paoff will be at least z; in fact, given (2) it is eactl z. Thus, is a best response for the row plaer against strateg of the column plaer, giving paoff of z to the row plaer. Using a similar argument, we can show that is a best response for the column plaer against strateg of the row plaer, giving paoff z to the column plaer. Hence, (, ) is a Nash equilibrium in which the plaers paoffs are z and z = z respectivel. We give a couple of corollaries of the above theorem. Corollar 1. There eists a Nash equilibrium in ever two-plaer zero-sum game. Corollar 2 (The Minma Theorem). ma min T R = min ma T R. Net, we argue that ever Nash equilibrium of the two-plaer zero-sum game can be found b our approach. Theorem 2. If (, ) is a Nash equilibrium of (R, C), then (, T R) is an optimal solution of LP (1), and (, T C) is an optimal solution of LP (2). Proof: Let z = T R = T C. Since (, ) is a Nash equilibrium. T C T Ce j j T R T Re j j T R z1. So we have showed that (, z) is a feasible solution for LP (1); similarl we can argue that (, z) is a feasible solution for LP (2). Since LP (2) is the dual of LP (1), b weak dualit, we know that the value achieved b an feasible solution of LP (2) upper bounds the value achieved b an feasible solution of LP (1). But, these values are equal here. So, (, z) and (, z) must be optimal solutions for LP (1) and LP (2) respectivel. Since all Nash equilibria define a pair of optimal solutions to LP (1) and LP (2) and all pairs of optimal solutions to LP (1) and LP (2) define a Nash equilibrium, we obtain the following important corollaries of Theorems 1 and

6 Corollar 3 (Conveit of Optimal Strategies). The set { there eists such that (, ) is a Nash equilibrium of (R, C)} of equilibrium strategies for the row plaer in zero-sum game is conve. Ditto for the column plaer. Corollar 4 (Anthing Goes). If is an equilibrium strateg for the row plaer in a zero-sum game (R, C) and is an equilibrium strateg for the column plaer, then (, ) is a Nash equilibrium of the game. Corollar 5 (Conveit of Equilibrium Set). The equilibrium set of a zero-sum game is conve. {(, ) (, ) is a Nash equilibrium of (R, C)} Corollar 6 (Uniqueness of Paoffs). The paoff of the row plaer is equal in all Nash equilibria of a zero-sum game. Ditto for the column plaer. Inspired b this last corollar, we define the value of a zero-sum game, to be the paoff of the row plaer in an (all) Nash equilibria of the game. Definition 1 (Value of a Zero-Sum Game). If (R, C) is zero-sum game, then the value of the game is the unique paoff of the row plaer in all Nash equilibria of the game. 3 Homework We conclude with a homework problem inspired b our discussion of zero-sum games. Adjacent Zero-Sum Games (2 points): Plaers 1 and 2 are plaing a zero-sum game with paoff matrices (A 1,2, A 2,1 ); at the same time plaers 1 and 3 are plaing another zero-sum game with paoff matrices (A 1,3, A 3,1 ). Instead of using different strategies in the two games, plaer 1 must use the same strateg in both games. 1. Show that there eists a Nash equilibrium in this game. 2. Give an algorithm that can compute a Nash equilibrium efficientl. 4 Historical Remarks The stud of games from a mathematical perspective goes back to the work of mathematician Émile Borel. The eistence of a Nash equilibrium in two-plaer zero-sum games was established b John von Neumann in 1928, in a paper [5] that arguabl gave birth to the field of Game Theor. Two decades after von Neumann s result it became understood in a discussion between von Neumann and Danzig the father of mathematical programming that the eistence of an equilibrium in zero-sum games is tantamount to Strong Linear Programming dualit (see, e.g., [1], [2]). Hence, as was established another three decades later b Khachian [4], finding an equilibrium in zero-sum games is computationall tractable. 2-6

7 References [1] Donald J. Albers and Constance Reid. An Interview with George B. Dantzig: The Father of Linear Programming. The College Mathematics Journal, 17(4):293314, [2] George B. Dantzig. Linear Programming and Etensions. Princeton Universit Press, [3] Sanjo Dasgupta, Christos Papadimitriou and Umesh Vazirani. Algorithms. McGraw-Hill, [4] Leonid G. Khachian. A Polnomial Algorithm in Linear Programming. Soviet Mathematics Doklad, 20(1):191194, [5] John von Neumann. Zur Theorie der Gesellshaftsspiele. Mathematische Annalen, 100:295320,