Lecture 6. 1 Introduction. 2 Existence of Nash Equilibria. Apple Client 1. Client 2. Client 3. Microsoft

Transcription

1 CS-352 Theoretcal Computer Scence October 4, 2012 Lecture 6 Lecturer: Aleksander Mądry Scrbes: Sddhartha Brahma, Mlan Korda 1 Introducton Today we contnue wth the topc of the prevous lecture game theory. Last tme we ntroduced a noton of Nash equlbrum and showed that all the eamples of games we consdered posses them. Ths begs to wonder: was t a concdence or maybe every game has a Nash equlbrum? Answerng ths queston s the man topc of ths lecture. In partcular, we prove two fundamental theorems n ths regard: the MnMa theorem and the Nash s (Estence) theorem. 2 Estence of Nash Equlbra Let us start wth an eample that shows that not every game has a Nash equlbrum. Consder the stuaton n Fgure 1, where Apple and Mcrosoft want to sell ther products to three clents. There are some restrctons, however. Apple can sell only to Clent 1 and Clent 2, whereas Mcrosoft can sell only to Clent 2 and Clent 3. Let us (rather unrealstcally) assume that the clents do not care about the product vendor and wll always go wth the lower prce. In addton, we requre that there s no prce dscrmnaton,.e., each company has to charge all ther clents the same prce. Now, t s not hard to see that there s no pure 1 Nash equlbrum here. Indeed, for any two prces p M and p A posted by the companes, we can have two cases. Ether these prces are equal, or one of them (say Mcrosoft s one) s strctly lower than the other. In the former case, the company that lost Clent 2 due to te-breakng, has an ncentve to just slghtly reduce ts prce to wn ths clent over (whle losng only a lttle bt on reducton of the prce for the other clent). Smlarly, n the latter case, Mcrosoft has an ncentve to brng ts prce even closer to (but stll below) p A to get a further ncrease of ts proft (ths ncrease can be mnuscule, but stll postve). A more nvolved analyss reveals that a med Nash equlbrum does not est here ether. Apple Clent 1 Clent 2 Mcrosoft Clent 3 Fgure 1: An llustraton for the game wthout Nash equlbrum. Ths eample s a bt worrsome, as t suggests that there mght be a large class of games that do not possess Nash equlbrum and thus the applcablty of the theory we just developed mght be severely lmted. So, s t only somethng wrong wth ths partcular game, or s t really that only some specal type of games has Nash equlbra? Fortunately, t turns out that the culprt here s just pecularty of our game. Specfcally, one can show that once we make our game more realstc by dscretzng the range of prces e.g., by nsstng that they have to be multples of, say, 1 centme the reasonng we appled above does not work anymore. And, f we addtonally, mpose some absolute upper bound on the possble prces (whch agan s completely reasonable), ths game wll have Nash equlbrum. 1 Recall that a Nash equlbrum s pure f t conssts only of pure (.e., determnstc) strateges. 1

2 In fact, the followng fundamental theorem of game theory shows that every game has a Nash equlbrum as long as there s only fntely many actons and fntely many players. Theorem 1 (Nash 1951) Any game wth a fnte number of players and a fnte set of actons has a (med) Nash equlbrum We postpone the proof of ths theorem tll later (see Secton 2.3). 2.1 Two-Player Zero-Sum Games Before we prove the general Nash s theorem, we focus on a smpler (but very mportant) case of twoplayer zero-sum games. Such games model a drect competton, where any gan of one player s the loss of the other one. Formally, a two-player game s zero-sum f the utlty functons of the two players satsfy s S u 1 (s) + u 2 (s) = 0, where we recall that S s the set of the possble outcomes of the game correspondng to (med) strateges of both players. Note that any such game can be convenently represented as an n m payoff matr A, where n s the number of actons avalable to the frst player we wll call hm the row player (as he/she chooses rows) and m s the number of actons offered to the second player that we wll call column player. Now, when the row player s playng acton and column player chooses acton j, the entry A j encodes the gan of the row player, and A j s the gan of the column player. So, for eample, the Penalty Kck game from the prevous lecture (whch s zero-sum), can be represented by a matr [ ] 1 1 A =, 1 1 where the goalkeeper s the row player and the shooter s the column player. Furthermore, one can see that n ths notaton, any med strategy 2 of the row player corresponds to a vector = [ 1,..., n ] T. On the other hand, y = [y 1,..., y m ] T can encode the med strategy of the column player. Therefore, the epected utlty of the row player correspondng to playng these two strateges s smply T Ay for the row player and T Ay for the column player, as T Ay =,j y j A j =,j p j A j, where p j := y j s the probablty of gettng the outcome (, j). Thus we see that, from ths pont of vew, the goal of the row player s to choose strategy that leads to mamzaton of T Ay, whle the goal of the column player s to choose y that mnmzes ths quantty. Now, before the (more general) Nash s theorem was proved, the followng MnMa theorem due to von Neumann was establshed for two-player zero-sum games. Theorem 2 (MnMa Theorem, von Neuman, 1928) For any two-player zero-sum game gven by a matr A R n m, let us defne V R := ma mn y T Ay and V C := mn y ma T Ay. We then have that V R = V C. Moreover, there ests a par of med strateges (, y ) whch acheves the common optmum V (whch s called the value of the game) and t s a Nash equlbrum of that game. 2 Ths s a randomzed strategy, where entry denotes the probablty of choosng -th acton,.e., 0, and = 1. 2

3 Although the statement of ths theorem can look a bt mysterous at frst, t actually turns out to have a very ntutve nterpretaton n game-theoretc terms. To see ths, observe that V R descrbes the best epected utlty of the row player n a stuaton when he/she has to reveal ts med strategy frst. That s, when the column player can choose hs/her strategy y after seeng. (Recall from the dscusson above that the goal of the column player s to choose y that mnmzes T Ay.) On the other hand, V C denotes the analogous best possble epected utlty of the row player f t s the column player that goes frst. Therefore, n the above contet, what the MnMa theorem s tellng us s that n two-person zerosum games, t does not matter who has to reveal hs/her strategy frst. (It s mportant, however, to note here that ths s true only as long as we allow declaraton of med strategy. It s no longer so f one had requred the players to reveal ther pure actons after choosng them randomly based on ther med strategy.) At frst, ths statement mght seem to be of only modest nterest, but t actually has some very deep and mportant consequences that go far beyond game theory. For one, one of ts ramfcatons s estence of Nash equlbrum n two-player zero-sum games. (On the other hand, one can also show that, more generally, the estence of Nash equlbra mples that not havng to reveal one s strategy frst does not gve any beneft.) 2.2 Proof of the MnMa Theorem Assume for the sake of contradcton that the theorem s not true,.e., that there ests a two-person zero-sum game that s descrbed by an n m payoff matr A and has V C V R. Note that as games (and the statement of the theorem) s nvarant under scalng by postve scalars and shftng of all the utltes by the same addtve factor, we can assume wlog that A j [0, 1] for all and j. Now, clearly, f V C V R, we can t have V C < V R, as beng able to reveal one s strategy after the column player does, can be only a beneft to row player. So, we just need to focus on provng that V C > V R cannot be the case ether. To derve our desred contradcton, we wll use the learnng-from-epert-advce framework we ntroduced n Lecture 1 (see Secton 6 n the notes from that lecture) to capture a process of repeated playng of our two-player zero-sum game descrbed by A, from the perspectve of the row player. To ths end, let us have n eperts one epert per each pure acton of the row player and work wth gans nstead of losses. (Note that our framework developed n Lecture 1 can smply model gans as negatve losses.) For a gven T, let us consder the followng T -round nstance of the learnng-from-epert advce framework. In each round t = 1,..., T : We output a probablty dstrbuton p t = (p t 1,..., p t n) over the eperts (actons of row player). Ths dstrbuton p t s treated as a med strategy of the row player. Then, let j t {1,..., m} be the (pure) strategy of the column player that s hs/her best response acton to row player s commtment to play p t,.e., j t t = arg mn ( p ) T Aej, 1 j n where e j s the m-dmensonal vector havng 1 at coordnate j, and 0 elsewhere. For each epert 1 n, hs/her gan n ths round to be g t := A jt. As a result, our gan n round t s g t := p t A j t = ( p t) T Aej. 3

4 Observe that our gan g t, n each round t, corresponds eactly to the utlty of the row player when playng the med strategy p t and havng the column player play the pure acton j t n response. So, we can drectly relate our total gan n the understandng of learnng-from-epert-advce framework to the total utlty we get by repeated playng of our two-player zero-sum game as a row player whle gong frst. In partcular, the above pont of vew mples that we have g t V R, for each t, as that s the best utlty/gan we can hope for whle playng our game and havng to go frst. (Note that from the perspectve of column player, there s no beneft n randomzaton once he/she knows what s the strategy of the row player. So, nsstng that he/she uses a pure acton s not restrctng hm/her n any way.) By summng over all the T rounds, we get that our total gan g, no matter how well we play, s at most T g := g t T V R. (1) t=1 Now, we want to compare our gan to the total gan of the best epert n the hndsght. To ths end, let us defne ŷ R m to be ŷ j := # of tmes jt = j, T for each 1 j m. Note that ths defnton of ŷ mples ŷ j 0, for every 1 j m, and j ŷj = 1. That s, ŷ s a probablty dstrbuton over the actons of the column player. In fact, we can vew ŷ as the emprcal estmaton of the med strategy played by the column player repeatedly throughout the whole game. Usng ŷ, we can relate the gan of the best epert to the value of V C. Namely, we have that g := ma T t=1 g t = ma T t=1 A j t T T = T ma e T Aŷ T V C, (2) where the last nequalty follows from notcng that ma e T Aŷ s just the best-response utlty of the row player when t s the column player that has to go frst (and commts to playng ŷ), and thus t s always at least V C. Once we establshed (1) and (2), the key remanng step s to nterpret these bounds approprately. Namely, what (1) s tellng us s that no matter what algorthm we use to play our game n the above learnng-from-epert-advce framework, our average gan per round wll be never bgger than V R. On the other hand, (2) states that the average gan per round of the best epert n hndsght s at least V C. However, f we just use the multplcatve-weghts-update algorthm (see Lecture 1) to repeatedly play our game n our learnng-from-epert-advce setup, then the performance guarantee of ths algorthm are contradctng the fact that V C > V R. Namely, recall that we proved n Lecture 1 that the performance of the MWU algorthm asymptotcally acheves the performance of the best epert n hndsght. More precsely, once we transfer the bounds from Theorem 6 n Lecture 1 from loss to gan settng (by just multplyng both ts sdes by 1), we have that, for any 0 < ε 1 2, the total gan g MW U of ths algorthm s at least g MW U (1 ε)g ln n ε, where g := ma T t=1 gt s the performance of the best epert n hndsght. (Note that, as A j [0, 1], we have that ρ = 1 here.) 4

5 So, f we dvde both sdes of ths performance bound by T, we wll get that the average per-round gan ḡ MW U of ths algorthm s at least ḡ MW U := g MW U T (1 ε) g T ln n εt = (1 ε)ḡ ln n T ḡ. (3) εt ε 0 That s, t approaches the average per-round gan ḡ of the best epert n hndsght, once ε tends to 0 and T tends to approprately. However, from (1) we know that ḡ MW U V C, whle from (2) we know that ḡ V R. So, (3) gves us a contradcton wth the fact that V C > V R and thus ndeed we need to have V R = V C, as desred. So, t just remans to formalze the above reasonng by pluggng the rght values of ε and T. To ths end, let us defne δ := V C V R > 0 and observe that as A j [0, 1], V C [δ, 1] and V R [0, 1 δ]. Let us then take ε = δ 4 ln n 2 and T > δ. 2 Pluggng these values nto the bound n (3), as well as, usng the fact that V R 1 δ and nequaltes (1) and (2), we get that V R ḡ MW U (1 ε)ḡ ln n εt (1 ε)v C ln n εt > (1 δ 2 )(V R + δ) δ 2 V R, whch s the desred contradcton. Now, once we establshed that V R = V C, we prove the second part of the theorem. To ths end, consder the followng two med strateges = arg ma mn y T Ay and y = arg mn y ma T Ay. Clearly, V = V C T Ay V R = V and thus (, y ) must be a Nash equlbrum of the game. We remark n passng that our proof of the MnMa theorem can be easly adapted to provde us wth an effcent algorthm that computes δ-appromate Nash equlbrum for two-player zero-sum games n O( ln n δ ) rounds of MWU algorthm eecuton. (The runtme dependence on δ s not the best possble as 2 lnear programmng can be used to get an O(ln 1 δ ) dependence.) 2.3 Proof of Nash s Theorem We now proceed to the proof of the Nash s theorem. To ths end, we wll need the followng powerful topologcal result. Theorem 3 (Brouwer s Fed Pont theorem) Let f : C C be a contnuous functon and C a conve and compact set, then there ests C such that f() =. The pont for whch f() = s called a fed pont of f. Proof of Nash theorem. (We wll prove ths theorem only for two-player games. However, a verson for larger number of players s just a smple etenson of eactly the same approach.) Defne C := S, where S = S 1 S 2 s the Cartesan product of the spaces of med strateges each of the players. Note that the set of med strateges can be embedded nto R n where t forms a compact smple. Snce the Cartesan product of (fntely many) compact and conve sets s compact and conve, our set C satsfes the condtons from Brouwer s Fed Pont theorem. Now, we want to fnd a sutable functon f : C C that s contnuous and whose fed ponts correspond to Nash equlbra of the underlyng game. A temptng choce for such functon could be a one that s defned as f((s 1, s 2 )) = (s 1, s 2), where s 1 s the best response strategy of Player 1 to the strategy s 2 of Player 2 and vce versa. Clearly, f some (s 1, s 2) s a fed pont of such functon t must be a Nash equlbrum. Unfortunately, ths functon s not well-defned. To see that, recall the Penalty Shot game that we dscussed. If the 5

6 row player chooses s 1 = (1/2, 1/2) (left and rght wth the same probablty) then any strategy of the column player s the best response. Furthermore, even fng ths problem would not make ths functon sutable, as such f s also not contnuous. Ths s so as f we take agan the eample of Penalty Shot game and look at strateges s 1 = (1/2 ε, 1/2 + ε) and s 1 = (1/2 + ε, 1/2 ε) of the row player, for any ε > 0, then the best response to the former s (1, 0), whle the best response to the latter s (0, 1). Fortunately, there s an easy remedy for the above problems. It just suffces to add to f a dampenng term that prevents t from devatng too rapdly. Formally, let us defne f as where and f((s 1, s 2 )) = (s 1, s 2), s 1 := arg ma u 1 (s 1, s 2 ) s 1 s s 1 S1 s 2 := arg ma u 2 (s 1, s 2) s 2 s s 2 S2 It s not hard to see that fed ponts of ths functon are stll Nash equlbra, as whenever there s a strctly better response to a gven strategy, one s always able to move (by some small but postve amount) n that drecton. Also, now ths functon s contnuous as the quadratc dampenng terms ensures that. So, by the Brouwer fed pont theorem f has a fed pont and thus the underlyng game has at least one Nash equlbrum. 2.4 Dscusson It should be emphaszed that the Nash s theorem only asserts the estence of a Nash equlbrum. The proof of ths theorem s hghly non-constructve and does not gve any hnt on how to effcently fnd them. As t turns out, there s a strong evdence that make us beleve that fndng Nash equlbra n arbtrary games s a problem that s computatonally very hard. As we already mentoned n the last lecture, ths s very troublng f one thnks about the underlyng belef of game theory that nteractons of ratonal agents always converge to a correspondng Nash equlbrum. After all, as Kamal Jan (a promnent researcher n algorthmc game theory) sad If your laptop can t fnd t, then nether can the market. 6