Computing Equilibria in Multi-Player Games

Transcription

1 Computng Equlbra n Mult-Player Games Chrstos H. Papadmtrou Λ Tm Roughgarden y Aprl 9, 2004 Abstract We ntate the systematc study of algorthmc ssues nvolved n fndng equlbra (Nash and correlated) n games wth a large number of players; such games, n order to be computatonally meanngful, must be presented n some succnct, game-specfc way. We develop a general framework for obtanng polynomal-tme algorthms for optmzng over correlated equlbra n such settngs, and show how t can be appled successfully to symmetrc games (for whch we actually fnd an exact polytopal characterzaton), graphcal games, and congeston games, among others. We also present complexty results mplyng that such algorthms are not possble n certan other such games. Fnally, we present a polynomal-tme algorthm, based on quantfer elmnaton, for fndng a Nash equlbrum n symmetrc games when the number of strateges s relatvely small. Λ UC Berkeley, Computer Scence Dvson, Soda Hall, Berkeley, CA Supported by an NSF ITR grant and a France- Berkeley Foundaton grant. Emal: chrstos@cs.berkeley.edu. y UC Berkeley, Computer Scence Dvson, Soda Hall, Berkeley, CA Supported by an NSF Postdoctoral Fellowshp. Emal: tmr@cs.berkeley.edu.

2 1 Introducton The Complexty of Equlbra. A fundamental problem on the ncreasngly actve nterface of game theory and theoretcal computer scence s determnng the computatonal complexty of computng equlbra. For example, the most popular equlbrum concept n noncooperatve game theory s the Nash equlbrum a stable pont among n strategc players from whch no player has a unlateral ncentve to devate. A polynomal-tme algorthm for computng a Nash equlbrum s arguably the holy gral n ths research area (see [34]), and much progress has been made on ths [5, 10, 20, 27, 28, 41] and related problems [7, 8, 9, 18, 29] by the theoretcal computer scence communty n the last few years. Mult-player Games. In ths paper, we wll study the complexty of computng equlbra n games wth many players. Whle two-player games are the most classcal [44] and well-studed type of game, and are possbly the most tractable from a complexty-theoretc perspectve, we nevertheless beleve that multplayer games demand mmedate study. Indeed, much of the current research on game theory n theoretcal computer scence s motvated by large networks, such as the Internet, where games are obvously beng played by a large number of players. Whle mult-player games have been extensvely studed n the game theory lterature (see [22] and the references theren), and ther mportance has long been recognzed by the artfcal ntellgence communty (see e.g. [42]), even less s known about computng equlbra n multplayer games than n the (stll mysterous) specal case of two-player games. There s an mmedate obstacle to dscussng complexty results for general n-player games: massve nput complexty. For example, to specfy a general game n whch n players each have to make a bnary decson, n2 n numbers are requred for each of the 2 n possble outcomes of the game, a payoff to each player. Ths exponental nput complexty s worrsome n two respects. Most obvously, t threatens to render all postve algorthmc results moot who cares about a polynomal-tme algorthm, when the nput sze s already exponental n natural parameters, such as the number of players? Secondly, t llustrates a potental dsconnect between our complexty theory of games and the games that we actually want to study rare s the game that models an applcaton of nterest and yet lacks suffcent structure to be specfed wth a reasonable number of parameters. Compact Representatons. The exponental nput complexty of general mult-player games motvates an mportant research drecton: the complexty of computng equlbra n mult-player games that admt a compact representaton. Many recent papers n the theoretcal computer scence lterature are n essence applyng ths phlosophy to concrete applcatons, such as load-balancng (see [6, 12] and the references theren), network routng (see [40]), faclty locaton [43], and congeston games [11]. In ths paper, we are amng for a more systematc nvestgaton what propertes of a compact representaton permt polynomaltme algorthms for computng equlbra? To llustrate our results, we wll focus on the followng three broad classes of structured mult-player games. Symmetrc Games. In a symmetrc game, all players are dentcal and ndstngushable. They have the same strategy sets, ther utlty functons are the same functon of ther own strategy and the other players actons, and ths functon s symmetrc n the other players actons. Ths functon thus depends only on the number of players choosng each strategy, and on the player s strategy. Symmetrc games have been wdely studed snce the dawn of game theory; together wth zero-sum games, they form one of the most classcal subclasses of games. For example, Nash [32] proved that every symmetrc game must have a symmetrc equlbrum an equlbrum n whch all players play the same strategy. Several of the papers n the frst volume of Contrbutons to the Theory of Games [25] are devoted to symmetrc games. Symmetrc games long tenure n the spotlght s due n large part to the famous examples they have provded: the Prsoner s Dlemma, Chcken, coordnaton games, and so on. More recently, symmetrc games have played 1

3 a central role n evolutonary game theory (see e.g. [45]). Despte ths long hstory, lttle s known about the complexty of computng equlbra of symmetrc games. Mult-player symmetrc games admt a compact representaton. Specfcally, a symmetrc game can, by defnton, be specfed by gvng the payoff of each strategy, gven how many players choose each n+k 1 strategy. If there are n players who choose among k strateges, then there are k 1 dstnct dstrbutons of n players among k strateges (equvalently, ordered parttons of n nto k parts), and the game can be n+k 1 summarzed wth only k k 1 numbers. Ths s always smaller than the nk n numbers requred for the standard representaton, exponentally so f k = O(n). Graphcal Games. Graphcal Games were frst proposed by Kearns, Lttman, and Sngh [20]. (See also Koller and Mlch [23] and the references theren for related concepts.) In a graphcal game, the players are the vertces of a graph, and the payoff of each player only depends on ts strategy and those of ts neghbors. Algorthms that run n tme polynomal n the obvous compact representaton have recently been developed for computng Nash [20, 28] and correlated equlbra [19] for graphcal games defned on trees. Congeston Games. Congeston games are an abstracton of network routng games and were frst defned by Rosenthal [38, 39]. In a congeston game, there s a ground set of elements, and players choose a strategy from a prescrbed collecton of subsets of the ground set. The cost of an element s a functon of the number players that select a strategy that contans t, but ths cost s ndependent of the denttes of these players. The cost (negatve payoff) to a player s then the sum of the costs of the elements n ts strategy. Congeston games enjoy a flexblty useful for modelng dverse applcatons, as well as enough structure to allow non-trval theoretcal analyses, and for these reasons have been extensvely studed n the last 10 years; see [40, x4.4] for a survey and [11] for recent results concernng pure-strategy Nash equlbra. Correlated Equlbra. Whle we also gve algorthms for computng Nash equlbra, our wdest-rangng theory concerns correlated equlbra. Correlated equlbra were frst defned by Aumann [1], and we wll descrbe them n detal n Secton 2. For now, suffce t to say that every Nash equlbrum s a correlated equlbrum and that the set of all correlated equlbra of a game can be descrbed by a system of lnear nequaltes whose sze s polynomal n the length of the game s standard descrpton. Unfortunately, the sze of ths system s generally exponental n that of the compact representatons of all of the games mentoned above. Hence correlated equlbra, currently the only known tractable soluton concept n game theory, long appeared beyond the power of polynomal-tme computaton for these fundamental classes of games. Our Results. For computng correlated equlbra n mult-player games wth a compact representaton, we prove the followng. ffl For symmetrc games we explctly descrbe the set of all correlated equlbra wth a lnear system that has sze polynomal n the natural compact representaton of the game. A correlated equlbrum n fact, one that optmzes an arbtrary lnear functon, such as the expected sum of player payoffs can thus be found effcently. ffl We present a general framework for optmzng over the correlated equlbra of a game n tme polynomal n the sze of a compact representaton. In addton to the class above, ths framework apples to certan congeston games and to graphcal games defned on trees (or more generally, graphs of bounded treewdth). ffl For two mportant classes of games not covered by our general framework general congeston games and general graphcal games we prove that there s no algorthm that optmzes over the set of correlated equlbra n tme polynomal n the sze of the natural compact representaton (assumng P 6= NP). 2

4 None of these results were prevously known, wth the excepton of the tractablty of optmzng over the correlated equlbra of a graphcal game defned on a tree, whch was frst proved by Kakade et al. [19]. Here, we rederve ths result from a much more general perspectve, and also gve the frst complextytheoretc justfcaton of restrctng the topology of graphcal games. Fnally, we present a polynomal-tme algorthm, based on the theory of real closed felds, for fndng a Nash equlbrum n n-player, k-strategy symmetrc games wth k = O(log n= log log n). 2 Prelmnares Games. A normal form game, or smply a game, s a collecton S 1 ;:::;S n of fnte strategy sets and a collecton u 1 ;:::;u n of real-valued utlty functons, each defned on S 1 S n. We dentfy a strategy set S and utlty functon u wth player. A element s of S 1 S n s called a strategy profle. The set of all strategy profles s the state space of the game. For a strategy profle s, s s the strategy of player, s denotes the (n 1)-vector of strateges of players other than, and the value u (s) wll be called the payoff to player. Symmetrc, Congeston, and Graphcal Games. A game s symmetrc f S 1 = = S n, u (s) depends only on s and the other players strateges (but not on ), and u (s) s a symmetrc functon of s. In other words, the payoff to a player depends only on ts strategy and on the number of players choosng each of the dfferent strateges. A symmetrc game can be specfed by gvng, for each ordered partton of the n+k 1 number of players, the payoffs of each player k k 1 numbers, where k s the number of strateges. Ths expresson s (n k 1 ) when k = O(1), polynomal n n k when k = O(log n= log log n), and s super-polynomally smaller than k n unless k =Ω(n 1+ffl ) for some ffl>0. A graphcal game s compactly descrbed by an undrected graph G = (V;E), where each vertex s a player wth an arbtrary strategy set. The payoffs to a player are an arbtrary functon of ts strategy and the strateges of the adjacent players. The number of parameters needed to specfy the payoffs of a graphcal game s therefore exponental n the maxmum degree but polynomal n the number of players. Fnally, n a congeston game there s a ground set E of elements, k collectons S 1 ;:::;S k of subsets of E, and, for =1;:::;k, a postve nteger number n of players wth strategy set S. Each element e 2 E has a real-valued cost functon c e,defned on the postve ntegers, whch descrbes ts cost gven the number of players that select strateges that nclude t. The cost (negatve P payoff) to a player s the sum of the costs of the elements n ts strategy. In a congeston game wth n = n players and m = jej ground elements, payoffs can thus be completely summarzed wth only nm real numbers. Nash Equlbra. Let G =(fs g; fu g) be an n-player game and p 1 ;:::;p n a collecton of probablty dstrbutons on the strategy sets. Dstrbutons p 1 ;:::;p n are a Nash equlbrum f, for each player, pckng a strategy from S accordng to the dstrbuton p maxmzes s expected payoff, assumng that each player j 6= pcks a strategy accordng to the dstrbuton p j. Nash [31] showed that every game admts a Nash equlbrum. In a subsequent paper [32], he showed that every symmetrc game admts a symmetrc Nash equlbrum, meanng a Nash equlbrum n whch p 1 = = p n. (A symmetrc game can also have other Nash equlbra.) Correlated Equlbra. Let G =(fs g; fu g) be an n-person game. Let q be a probablty dstrbuton on S 1 S n. Dstrbuton q s a correlated equlbrum f for each player and each par `; `0 of strateges n S, X X q(s)u (s) q(s)u (s 0 ); (1) s : s =` s : s =` 3

5 where s 0 s obtaned from s by reassgnng s strategy to be `0. One nterpretaton of a correlated equlbrum s as follows. A trusted authorty pcks a strategy profle s at random accordng to q, and recommends strategy s to each player. Each player s assumed to know only ts recommended strategy, and not those for other players. Player then compares the condtonal expected payoffs of ts strateges, assumng that the other players follow ther recommendatons (condtonng on the strategy recommended to ). Inequalty (1) states that ths condtonal expectaton should be maxmzed by the recommended strategy. If ths holds for all players, then no player has a unlateral ncentve to devate from the trusted authorty s recommendaton. The nequaltes (1) evdently descrbe the set of all correlated equlbra, and ths lnear system has sze polynomal n the normal form descrpton of the game. Snce every Nash equlbrum, vewed as a product dstrbuton, s a correlated equlbrum, Nash s theorem [31] mples that ths system s always feasble. A popular concrete example of a correlated equlbrum s a traffc sgnal that recommends red (stop) or green (go) to drvers (see e.g. [33]). For more applcatons of correlated equlbra, see [2, 13, 14]. 3 Explct Descrptons of Correlated Equlbra In ths secton, our ambton wll be to explctly descrbe the correlated equlbra of a game that s represented compactly. Put dfferently, we wll am for a characterzaton that s equally powerful and complete as the classcal one, whle at the same tme demandng that the lnear system be just as economcal as the game s compact descrpton. We wll accomplsh ths goal for the class of symmetrc games. As a consequence, every lnear functon can be effcently optmzed over the set of correlated equlbra and n partcular, one can be found. For smplcty, we wll work prmarly n the k = 2 case; ths suffces to llustrate most of our proof technques. In Subsecton 3.2 we make a few comments about what s requred to extend the analyss to arbtrary k-strategy symmetrc games. 3.1 Symmetrc Games wth Two Strateges Let G = (S = f1; 2g;u 1 ;:::;u n ) be an n-player, 2-strategy symmetrc game. An explctly represented correlated equlbrum of G must specfy a probablty q(s) for each of the 2 n strategy profles s. The varables n our compact representaton of the correlated equlbra of G wll be of the form p (j) (basc varables) and p(j) (auxlary varables) for 2 f1; 2;:::;ng and j 2 f0; 1;:::;ng. We ntend the basc varable p (j) to represent the aggregate probablty assgned to the strategy profles S (j) n whch exactly j players, ncludng player, choose strategy 1. Smlarly, p(j) represents the total probablty of the strategy profles S(j) n whch exactly j players choose strategy 1. We wll sometmes refer to subsets of S n of the form S (j) and S(j) as basc and auxlary sets, respectvely. The constrants are as follows. nx j=0 nx j=0 p (j)u (j; 1) [p(j) p (j)] u (j; 2) nx nx j=0 =1 nx nx j=0 j=0 p (j)u (j 1; 2) for all 2f1; 2;:::;ng (2) [p(j) p (j)] u (j +1; 1) for all 2f1; 2;:::;ng (3) p(j) = 1 (4) p (j) = j p(j) for all j 2f0; 1;:::;ng (5) 0» p (j)» p(j)» 1 for all 2f1; 2;:::;ng, j 2f0; 1;:::;ng, (6) 4

6 where u (j; `) denotes the payoff to player n a strategy profle n whch player chooses strategy ` and a total of j players choose strategy 1. Ths payoff s well defned.e., ndependent of the strategy profle meetng the above crtera by the defnton of a symmetrc game. Wth respect to an n-player, 2-strategy game G = (f1; 2g;u 1 ;:::;u n ), we wll call the equatons and nequaltes (2) (6) the basc lnear system of G. The sze of ths system s polynomal n that of the compact representaton of G. We wll sometmes refer to equatons (5) as the coverng equatons. Snce constrants (2) (3) are effectvely aggregated versons of the correlated equlbrum constrants (1), every correlated equlbrum of an n-player, 2-strategy symmetrc game G (defned on all of S n ) nduces a soluton to G s basc lnear system va the ntended aggregatons of probablty. The nterestng drecton s the converse. Let p be a soluton to the basc lnear system of a 2-strategy symmetrc game G =(S; u 1 ;:::;u n ). We say that p extends to S n f there s a functon q : S n!r + wth Ps2S (j) q(s) =p (j) and P s2s(j) q(s) =p(j) for all and j. It s easy to check that f p extends to Sn, then the extenson s a correlated equlbrum of G. It s not at all obvous, however, that such an extenson must exst; ths s our man result. Theorem 3.1 Let G be a 2-strategy symmetrc game. Then every soluton to G s basc lnear system can be extended to a correlated equlbrum of G. We wll prove Theorem 3.1 n two parts. The glue that holds the two parts together s the noton of a unform soluton to a game s basc lnear system. Defnton 3.2 A j-basc cover s a functon x : fs 1 (j);:::;s n (j)g!r + wth P S (j):s2s (j) x (j) j for all s 2 S(j), where we have wrtten x (j) for x(s (j)). A soluton p to G s basc lnear system s unform f for all j 2 f0; 1;:::;ng, P n =1 p (j)x (j) P n =1 p (j) = j p(j) for every j-basc cover x. (The equalty s the jth coverng equaton (5).) Defnton 3.2 s justfed by the followng two lemmas, whch mmedately mply Theorem 3.1. Lemma 3.3 Let G be a 2-strategy symmetrc game. Then every unform soluton to G s basc lnear system can be extended to a correlated equlbrum of G. Lemma 3.4 Let G be a 2-strategy symmetrc game. Then every soluton to G s basc lnear system s unform. Lemma 3.3 s essentally a consequence of strong lnear programmng dualty, and we postpone ts proof to the Appendx. Before provng Lemma 3.4, we establsh a prelmnary lemma. In ts statement, we wll use the notaton [x] + to denote maxf0;xg for a real number x. Lemma 3.5 Let G = (S = f1; 2g;u 1 ;:::;u n ) be a 2-strategy symmetrc game, and p a soluton to G s basc lnear system. (a) If j 2f0; 1;:::;ng, C s a collecton of `» j dstnct j-basc sets, and C 0 s a collecton of r» n ` dstnct j-basc sets not n C, then some element n C S (j) les n only [r + j n] + sets of C 0. (b) If C s a collecton of r dstnct j-basc sets, then P S (j)2c p (j) [r + j n] + p(j). Proof: Part (b) follows mmedately from constrants (5) and (6). To prove part (a), relabel the players so that C = fs 1 (j);:::;s`(j)g and C 0 = fs n r+1 (j);:::;s n (j)g. Let s be the strategy profle n whch the frst j players choose strategy 1 and the last n j players choose strategy 2. The profle s then les n all sets of C but only n [r + j n] + sets of C 0. Ξ 5

7 Proof of Lemma 3.4: Let G =(S = f1; 2g;u 1 ;:::;u n ) by a symmetrc game and p a soluton to G s basc lnear system. We need to show that p s unform n the sense of Defnton 3.2. For j 2f0; 1;:::;ng and a functon x : fs 1 (j);:::;s n (j)g!r,defne the functon C j by C j (x) = P n =1 p (j)x (j) (as usual, x (j) s shorthand for x(s (j))). For every j 2f0; 1;:::;ng, settng x (j) =1 for all 2 f1; 2;:::;ng yelds a j-basc cover wth C j (x) = P n =1 p (j). We call ths cover the unform j-cover. Provng that p s unform s tantamount to showng that, for each j 2 f0; 1;:::;ng, the unform j-cover mnmzes C j (x) over all j-basc covers x. Toward ths end, let x be a non-unform j-basc cover for some j 2 f0; 1;:::;ng. Let U denote the ndces of the sets underused by x (x (j) < 1) and O the ndces of the sets overused by x (x (j) > 1). We can assume that U s non-empty (else clearly C j (x) s no smaller than n the unform soluton). Snce x s a feasble j-basc cover, O s then non-empty as well. We frst clam that the number juj of underused sets s at most j 1. To see why, note that every j basc sets of the form S (j) have exactly one pont n common the strategy profle s n whch s =1f and only f S (j) s one of the sets and ths pont s contaned n no other basc set. Thus f s s n the common ntersecton of j sets S (j) wth 2 U, then P : s2s (j) x (j) <j, and x s not a j-basc cover. Wthout loss of generalty, x 1 (j) x 2 (j) x n (j). Let O = f1; 2;:::;mg and U = ft; : : : ; ng for 1» m<t» n. The contrbuton of underused sets to the sum P : s2s (j) x (j) for elements s n ther (non-empty) common ntersecton s c P n =t (1 x (j)) less than n the unform soluton. Snce x s a j-basc cover, the extra contrbuton from the overused sets, relatve to the unform soluton, must be at least c for all such elements. Let z m (j) =x m (j) 1 and z r (j) =x r (j) x r 1 (j) for r 2f1; 2;:::;m 1g. The z-varables should be regarded as a decomposton of the x-values that permts the applcaton of Lemma 3.5. We can express the prevous nequalty n terms of the z-varables as follows: X (r;):s2s (j);»r»m z r (j) = X X mx :s2s (j);»m r= z r (j) = X :s2s (j);»m [x (j) 1] c: (7) By Lemma 3.5(a), for every r 2 f1; 2;:::; mg there s an element s n all underused sets for whch»r : s2s (j) z r (j)» [r + j n] + z r (j): (8) The proof also shows that, snce the set f :» rg s ncreasng n r, there s a sngle strategy profle s n all underused sets for whch (8) holds smultaneously for all r 2f1; 2;:::;mg. Summng over all r and combnng wth (7), we fnd that We can now complete the proof. Let u be the unform soluton. Wrte mx r=1 C j (x) =C j (u) + z r (j) [r + j n] + c: (9) mx =1 [x 1]p (j) nx =t [1 x ]p (j): (10) Snce p (j)» p(j) for all, the last term s at most p(j) P n =t (1 x )=c p(j). To lower bound the second term on the rght-hand sde of (10), use Lemma 3.5(b) and (9) to wrte mx =1 [x 1]p (j) = mx r=1 z r (j) rx =1 p (j) mx r=1 z r (j)[r + j n] + p(j) c p(j): (11) The nequalty C j (x) C j (u) now follows from (10) and (11), and the proof s complete. Ξ 6

8 3.2 Symmetrc Games wth Many Strateges It s straghtforward to extend the defnton of a basc lnear system to k-strategy symmetrc games. There are varables p (j; `) and p(j), where j s now an ordered partton of n nto k non-negatve ntegers, and ` 2 S = f1; 2;:::;kg, wth P` p(j; `) = p(j) for every player. Analogs of constrants (2) (6) are straghtforward to descrbe. As n the prevous subsecton, we have the followng theorem. Theorem 3.6 Let G be a symmetrc game. Then p s a soluton to G s basc lnear system f and only f t can be extended to a correlated equlbrum of G. The proof of Theorem 3.6 proceeds as for the 2-strategy case, hngng on an extenson of the noton of unformty (Defnton 3.2) to k-strategy symmetrc games. For a fxed ordered partton j, the j-basc sets are now ndexed by both a player and a strategy `. Ths causes no dffculty for extendng Lemma 3.3, but extendng Lemma 3.4 to these rcher collectons of basc sets requres more sophstcated combnatoral arguments. We omt further detals. Remark 3.7 Nowhere n the proofs above dd we use the fact that the utlty functons are equal. Thus our results apply more generally to symmetrc-lke games where dfferent players have dfferent (but symmetrc w.r.t. other players) utlty functons. 4 Fndng Correlated Equlbra of General Compact Games In ths secton, we contnue to devse algorthms for fndng and optmzng over correlated equlbra that run n tme polynomal n the sze of a game s compact representaton. We wll, however, relax our prevous ambton of explctly descrbng the set of correlated equlbra. As our reward, we wll be able to work n a very general settng, wth essentally arbtrary compact representatons. In Subsecton 4.1, we wll present a general result that shows that the tractablty of optmzng over the correlated equlbra of a game n tme polynomal n a compact representaton s controlled by an optmzaton problem related to the representaton. In Subsecton 4.2, we wll see that for all of the classes of games studed n ths paper, ther natural compact representatons gve rse to combnatoral optmzaton problems wth easly determned computatonal complexty. As a result, we wll be able to derve numerous postve and negatve results wth mnmal effort. 4.1 A General Framework At the hghest level the algorthmc approach of ths secton wll be smlar to that of the prevous one. We wll formulate a lnear program where the number of varables s comparable to the sze of the gven compact representaton, and wll then hope that solutons to the lnear program can be extended to correlated equlbra defned explctly on the set of all strategy profles. At the bare mnmum, to mplement ths dea we wll requre equlbrum constrants analogous to (2) (3). In turn, the essentally mnmal assumptons needed to defne such constrants are gven n the next defnton. Defnton 4.1 Let G = (S 1 ;:::;S n ;u 1 ;:::;u n ) be a game n normal form. For = 1; 2;:::;n, let P = fp 1 ;:::;Pm g be a partton of S nto m classes, where S denotes the (n 1)-fold product of strategy sets other than S. (a) For a player, two strategy profles s and s 0 are -equvalent f s = s 0, and both s and s 0 belong to the same class of the partton P. 7

9 (b) The set P = fp 1 ;:::;P n g of parttons s a compact representaton of G f u (s) =u (s 0 ) whenever s and s 0 are -equvalent. The motvaton of Defnton 4.1 s that t permts a reasonable defnton of the correlated equlbrum constrants. To see ths, let P = fp j g be a compact representaton for a game G. For a player, a class Pj n player s partton, and a strategy ` 2 S, let u (j; `) denote the payoff to player n a strategy profle s wth P ; ths s well defned P by Defnton 4.1. We can then wrte the correlated equlbrum m constrants as j=1 p m (j; `)u (j; `) j=1 p (j; `)u (j; `0) for all and all `; `0 2 S, where p (j; `) s the aggregate probablty assgned to strategy profles s wth s = ` and s 2 P j. In Defnton 4.1, there s one partton of (most of) the state space for each player. In some applcatons, there wll be an obvous partton of the state space that cuts across player types. For example, the state space of an n-player, k-strategy symmetrc game admts an obvous global partton, wth one class of the partton for each ordered partton of n nto k parts. Such a global partton easly defnes a compact representaton n the sense of Defnton 4.1 that has comparable sze. In the symmetrc game example, the partton correspondng to player has one class for each ordered partton of n 1 (the other players) nto k parts (the dstrbuton of ther strateges). We need one further defnton. As our man result n ths secton, we wll show that the tractablty of optmzng over the correlated equlbra of a compactly represented game s controlled by the computatonal complexty of a related optmzaton problem. As we wll see n Subsecton 4.2, ths general reducton wll have mmedate consequences for symmetrc, congeston, and graphcal games. We next defne the relevant optmzaton problem correspondng to a compact representaton. s = ` and s 2 P j Defnton 4.2 Let P = fp j g be a compact representaton of a game G. The separaton problem for P s the followng algorthmc P problem: Gven ratonal numbers y (j; `) for all, j, and ` 2 S, s there a strategy profle s wth (;j;`):s =`;s 2P j y (j; `) < 0? We wll see several concrete examples of such separaton problems n Subsecton 4.2. We now conclude ths subsecton by provng that a tractable separaton problem s all that s requred for the effcent computaton of a correlated equlbrum. We wll state ths result n terms of the sze of a compact representaton, whch s defned n the obvous way (total number of classes n ts parttons, plus the number of bts needed to descrbe player payoffs). Theorem 4.3 Let P be a compact representaton of a game. If the separaton problem for P can be solved n polynomal tme, then a correlated equlbrum of G can be computed n tme polynomal n the sze of P. The proof of Theorem 4.3 s drven by two successve applcatons of the ellpsod algorthm. For detals, see the Appendx. More generally, the proof of Theorem 4.3 shows that every lnear functon can be effcently optmzed over the set of correlated equlbra of such a game. 4.2 Applcatons We now demonstrate the power of Theorem 4.3. We begn by revstng symmetrc games, and then proceed to congeston and graphcal games. Symmetrc Games and Extensons. We begn by reconsderng symmetrc games. Ths wll llustrate the defntons and results of Subsecton 4.1 n a famlar settng. As we have noted, n-player, k-strategy symmetrc games admt a natural compact representaton P = fp j g n the sense of Defnton 4.1, where the classes of P are ndexed by a player and an ordered partton j of n 1 nto k parts correspondng to a dstrbuton of the other n 1 players among the k avalable strateges. The separaton problem for P s 8

10 then: gven ratonal numbers y (j; `) for each player, each ordered P partton j of n 1 nto k parts, and each choce ` for player s strategy, s there a strategy profle s wth (;j;`):s =`;s 2P j y (j; `) < 0? Ths problem can be solved n polynomal tme, for example by a straghtforward applcaton of mn-cost flow, and hence Theorem 4.3 mples the followng. Corollary 4.4 A correlated equlbrum of a symmetrc game can be found n tme polynomal n ts natural compact representaton. Whle Corollary 4.4 s weaker than Theorems 3.1 and 3.6 n that t does not gve an explct descrpton of the set of correlated equlbra, we derved t wth consderably less work. As n Remark 3.7, Corollary 4.4 also holds when dfferent players have dfferent (symmetrc) utlty functons. Graphcal Games. For a graphcal game, ts natural compact representaton P = fp j j g has a class P for each player and for each assgnment j of strateges to the players that are neghbors of. The separaton problem for ths representaton s then the followng: gven ratonal numbers y (j; `) for each player, each set j of P strategy choces of s neghbors, and each choce ` for player s strategy, s there a strategy profle s wth (;j;`):s =`;s 2P j y (j; `) < 0? For graphcal games defned on trees, ths problem can be solved by dynamc programmng. Corollary 4.5 A correlated equlbrum of a graphcal game wth a tree topology can be found n tme polynomal n ts natural compact representaton. As we noted n the ntroducton, Corollary 4.5 was frst proved by Kakade et al. [19], usng tools from probablstc nference. Corollary 4.5 also permts easy generalzatons, for example to graphs of bounded treewdth, that do not seem to trvally follow from the proof technques of [19]. For general topologes, however, the story s dfferent. Frst, a reducton from EXACT COVER BY 3-SETS [16, SP2] shows the followng. Proposton 4.6 The separaton problem for the natural compact representaton of a graphcal game s NP-complete, even n bpartte graphs. In fact, somethng much stronger s true. A smlar reducton, usng the verson of EXACT COVER BY 3-SETS where each element s contaned n only a constant number of sets (see [16, SP2]), shows the followng. Proposton 4.7 Assumng P 6= NP, there s no polynomal-tme algorthm for computng a correlated equlbrum of a compactly represented graphcal game that maxmzes the expected sum of player payoffs. Proposton 4.7 dspels any lngerng concern that we mght have taken the wrong proof approach n our attempt to characterze the correlated equlbra of a graphcal game: there s no lnear system that characterzes the correlated equlbra of a general graphcal game and can be optmzed over n tme polynomal n the game s compact representaton (assumng P 6= NP). Thus no small explct descrpton s possble (cf., Theorem 3.1), nor s there any descrpton amenable to the ellpsod algorthm (cf., Theorem 4.3). Congeston Games. Recall that a congeston game s specfed by a ground set E, strategy sets S 1 ;:::;S k, quanttes n 1 ;:::;n k of players, and cost functons fc e g e2e defned on f1; 2;:::; P n g. Congestons games have the most economcal descrpton of all of the games studed n ths paper, wth nm numbers suffcng to descrbe all of the payoffs, where n and m are the number of players and elements, respectvely. Perhaps because of ths very small descrpton, congeston games are n some sense also the least tractable class of games studed n ths paper: analogously to Proposton 4.7, a reducton from EXACT COVER BY 3-SETS shows the followng. 9

11 Proposton 4.8 Assumng P 6= NP, there s no polynomal-tme algorthm for computng a correlated equlbrum of a compactly represented congeston game that maxmzes the expected sum of player payoffs. Proposton 4.8 holds even for congeston games wth one player type (k = 1). Some postve results for effcently optmzng over the correlated equlbra of a congeston game can be salvaged f somewhat larger representatons are used; we defer a detaled dscusson of ths pont to the full verson of ths paper. 5 Nash Equlbra of Symmetrc Games Fnally, we gve an algorthm for computng a symmetrc Nash equlbrum n symmetrc games. Theorem 5.1 The problem of computng a symmetrc Nash equlbrum n a symmetrc game wth n players and k strateges can be solved to arbtrary precson n tme polynomal n n k, the number of bts requred to descrbe the utlty functons, and the number of bts of precson desred. Theorem 5.1 s a reducton from the so-called frst-order theory of the reals, the detals of whch we provde n the Appendx. A dfferent applcaton of ths dea to games was developed ndependently by Lpton and Markaks [26]. Snce the compact representaton of a symmetrc game has sze Ω(poly(n k )) when k = O(log n= log log n), we have the followng corollary of Theorem 5.1. Corollary 5.2 The problem of computng a Nash equlbrum of a compactly represented n-player k-strategy symmetrc game wth k = O(log n= log log n) s n P. Theorem 5.1 and Corollary 5.2 can be extended to certan types of partally symmetrc games (frst consdered by Nash n [32]), such as games wth a constant number of player types and full symmetry among players of the same type. Corollary 5.2 stands n contrast to the state of the art for general games, where no polynomal-tme algorthm for computng a Nash equlbrum s known, even when all players have only two strateges. We unfortunately have no progress to offer when n s small relatve to k. We note, however, that fndng an algorthm for computng a Nash equlbrum of a symmetrc game n ths case could be dffcult. In partcular, t has long been known that for games wth a constant number of players, there s a polynomal-tme reducton from general games to symmetrc games [3, 15], and hence symmetry affords no computatonal advantage n ths case. Acknowledgements Thanks to Grant Schoenebeck for drectng us to [15], Vangels Markaks for notfyng us about [26], and Vncent Contzer, Nmrod Megddo, Éva Tardos, and Borska Toth for helpful comments. References [1] R. J. Aumann. Subjectvty and correlaton n randomzed strateges. Journal of Mathematcal Economcs, 1(1):67 96, [2] R. J. Aumann. Correlated equlbrum as an expresson of Bayesan ratonalty. Econometrca, 55(1):1 18,

12 [3] J. W. Brown and J. von Neumann. Solutons of games by dfferental equatons. In H. W. Kuhn and A. W. Tucker, edtors, Contrbutons to the Theory Games, volume 1, pages Prnceton Unversty Press, [4] V. Chvátal. Lnear Programmng. Freeman, [5] V. Contzer and T. Sandholm. Complexty results about Nash equlbra. In Proceedngs of Internatonal Jont Conference on Artfcal Intellgence, [6] A. Czumaj. Selfsh routng on the Internet. In J. Leung, edtor, Handbook of Schedulng. CRC Press, Boca Raton, FL, 2004, to appear. [7] X. Deng, C. H. Papadmtrou, and S. Safra. On the complexty of equlbra. In Proceedngs of the 34th Annual ACM Symposum on the Theory of Computng, pages 67 71, [8] N. R. Devanur, C. H. Papadmtrou, A. Saber, and V. V. Vazran. Market equlbrum va a prmaldual-type algorthm. In Proceedngs of the 43rd Annual Symposum on Foundatons of Computer Scence, pages , [9] N. R. Devanur and V. V. Vazran. An mproved approxmaton scheme for computng Arrow-Debreu prces for the lnear case. In 23rd Conference on Foundatons of Software Technology and Theoretcal Computer Scence (FSTTCS), pages , [10] E. Even-Dar, A. Kesselman, and Y. Mansour. Convergence tme to Nash equlbra. In Proceedngs of the 30th Annual Internatonal Colloquum n Automata, Languages, and Programmng (ICALP), volume 2719 of Lecture Notes n Computer Scence, pages , [11] A. Fabrkant, C. H. Papadmtrou, and K. Talwar. The complexty of pure-strategy equlbra. In Proceedngs of the 36th Annual ACM Symposum on the Theory of Computng (STOC), [12] R. Feldmann, M. Garng, T. Lückng, B. Monen, and M. Rode. Selfsh routng n non-cooperatve networks: A survey. In Proceedngs of the Conference on Mathematcal Foundatons of Computer Scence (MFCS), volume 2747 of Lecture Notes n Computer Scence, pages 21 4, [13] D. Foster and R. Vohra. Calbrated learnng and correlated equlbrum. Games and Economc Behavor, 21:40 55, [14] D. Foster and R. Vohra. Regret n the on-lne decson problem. Games and Economc Behavor, 29:7 35, [15] D. Gale, H. W. Kuhn, and A. W. Tucker. On symmetrc games. In H. W. Kuhn and A. W. Tucker, edtors, Contrbutons to the Theory Games, volume 1, pages Prnceton Unversty Press, [16] M. R. Garey and D. S. Johnson. Computers and Intractablty: A Gude to the Theory of NP- Completeness. Freeman, [17] M. Grötschel, L. Lovász, and A. Schrjver. Geometrc Algorthms and Combnatoral Optmzaton. Sprnger-Verlag, Second Corrected Edton, [18] K. Jan, M. Mahdan, and A. Saber. Approxmatng market equlbra. In Proceedngs of the 9th Internatonal on Approxmaton Algorthms for Combnatoral Optmzaton Problems,

13 [19] S. Kakade, M. Kearns, J. Langford, and L. Ortz. Correlated equlbra n graphcal games. In Proceedngs of the 4th ACM Conference on Electronc Commerce, pages 42 47, [20] M. Kearns, M. L. Lttman, and S. Sngh. Graphcal models for game theory. In Proceedngs of the Conference on Uncertanty n Artfcal Intellgence, pages , [21] L. G. Khachyan. A polynomal algorthm n lnear programmng. Sovet Mathematcs Doklady, 20: , [22] M. A. Khan and Y. Sun. Non-cooperatve games wth many players. In R. J. Aumann and S. Hart, edtors, Handbook of Game Theory, volume 3, chapter 46. North-Holland, [23] D. Koller and B. Mlch. Mult-agent nfluence dagrams for representng and solvng games. Games and Economc Behavor, 45: , [24] H. Konsh, M. Le Breton, and S. Weber. Equlbra n a model wth partal rvalry. Journal of Economc Theory, 72: , [25] H. W. Kuhn and A. W. Tucker. Contrbutons to the Theory of Games, volume 1. Prnceton Unversty Press, [26] R. J. Lpton and E. Markaks. Nash equlbra va polynomal equatons. In Proceedngs of the Sxth Conference on Latn Amercan Theoretcal Informatcs (LATIN), pages , [27] R. J. Lpton, E. Markaks, and A. Mehta. Playng large games usng smple strateges. In Proceedngs of the 4th ACM Conference on Electronc Commerce, pages 36 41, [28] M. L. Lttman, M. Kearns, and S. Sngh. An effcent exact algorthm for sngly connected graphcal games. In Advances n Neural Informaton Processng Systems, volume 14, [29] M. L. Lttman and P. Stone. A polynomal-tme Nash equlbrum algorthm for repeated games. In Proceedngs of the 4th ACM Conference on Electronc Commerce, pages 48 54, [30] I. Mlchtach. Congeston games wth player-specfc payoff functons. Games and Economc Behavor, 13: , [31] J. F. Nash, Jr. Equlbrum ponts n N-person games. In Proceedngs of the Natonal Academy of Scence, volume 36, pages 48 49, [32] J. F. Nash, Jr. Non-cooperatve games. Annals of Mathematcs, 54(2): , [33] G. Owen. Game Theory. Academc Press, Thrd Edton. [34] C. H. Papadmtrou. Algorthms, games, and the Internet. In Proceedngs of the 33rd Annual ACM Symposum on the Theory of Computng, pages , [35] C. H. Papadmtrou and K. Stegltz. Combnatoral Optmzaton: Algorthms and Complexty. Prentce-Hall, Dover Edton, [36] T. Qunt and M. Shubk. A model of mgraton. Workng paper, Cowles Foundaton, Yale Unversty, [37] J. Renegar. On the computatonal complexty and geometry of the frst-order theory of the reals, parts I III. Journal of Symbolc Computaton, 13(3): , , ,

14 [38] R. W. Rosenthal. A class of games possessng pure-strategy Nash equlbra. Internatonal Journal of Game Theory, 2:65 67, [39] R. W. Rosenthal. The network equlbrum problem n ntegers. Networks, 3:53 59, [40] T. Roughgarden. Selfsh Routng. PhD thess, Cornell Unversty, [41] R. Savan and B. von Stengel. Long Lemke-Howson paths. Techncal Report LSE-CDAM , London School of Economcs, [42] Y. Shoham. Introducton to Mult-Agent Systems In preparaton. [43] A. Vetta. Nash equlbra n compettve socetes, wth applcatons to faclty locaton, traffc routng and auctons. In Proceedngs of the 43rd Annual Symposum on Foundatons of Computer Scence (FOCS), pages , [44] J. von Neumann and O. Morgenstern. Theory of Games and Economc Behavor. Prnceton Unversty Press, [45] J. W. Webull. Evolutonary Game Theory. MIT Press, A Mssng Proofs Proof of Lemma 3.3: Let G =(S = f1; 2g;u 1 ;:::;u n ) be a symmetrc game, p a unform soluton to G s basc lnear system, and B = fs (j)g, A = fs(j)g the basc and auxlary sets, respectvely. The vector p can be extended to a correlated equlbrum of G f and only f there s a soluton q : S n! R + to the equatons X X s2s (j) s2s(j) q(s) = p (j) q(s) = p(j) for all basc sets S (j) and auxlary sets S(j). By Farkas s Lemma or strong LP dualty (see e.g. [35]), such an extenson exsts f and only f for every functon y : A[B!Rwth y(j) + X : s2s (j) for all j 2f0; 1;:::;ng and s 2 S(j), the nequalty p(j)y(j) + nx =1 y (j) 0 (12) p (j)y (j) 0 (13) holds for every j 2f0; 1;:::;ng, where we have wrtten y(j) for y(s(j)) and y (j) for y(s (j)). We now show that (12) mples (13). Let y satsfy (12). Snce each s n S(j) les n precsely j sets of the form S (j), ncreasng the value of all y (j) s whle decreasng the value of y(j) at j tmes the rate leaves the left-hand sde of (12) unchanged. Moreover, the jth coverng equaton (Equaton (5)) mples that the left-hand sde of (13) s nvarant under ths operaton. We can therefore assume that y (j) 0 for 13

15 all and j. Snce (12) and (13) are nvarant under scalng by a postve constant, we can smlarly assume that y(j) 0 or y(j) = j for each j. If y(j) 0 then (13) clearly holds, so suppose y(j) = j. Hypothess (12) then mples that y 1 (j);:::;y n (j) s a j-basc cover. Snce p s assumed unform, we have P n =1 p (j)y (j) j p(j). Inequalty (13) now follows. Ξ Proof of Theorem 4.3: Let G =(S 1 ;:::;S n ;u ; :::;u n ) be a game and P a compact representaton such that the separaton problem for P s solvable n polynomal tme. Defne u (j; `) as n the dscusson followng Defnton 4.1, and consder the followng system of equatons and nequaltes: Xm j=1 Xm p (j; `)u (j; `) j=1 X Xm j=1 p (j; `)u (j; `0) for all, `; `0 2 S (14) `2S p (j; `) = 1 for all (15) p (j; `) 0 for all ; j 2f1; 2;:::;m g;`2 S. (16) Every correlated equlbrum naturally nduces a feasble soluton to ths lnear system. In contrast to Theorem 3.1, the converse need not hold unless the system s augmented by addtonal nequaltes. We explore ths dea next. As n Lemma 3.3, by Farkas s Lemma there s a matrx A, wth columns ndexed by the exponentally many strategy profles, so that a soluton p to (14) (16) can be extended to a correlated equlbrum of G f and only f for every y wth y T A 0, y T p 0. Note that the vector y s ndexed by the varables n (14) (16). Ths observaton suggests extra nequaltes to add to (14) (16): for every vector y wth y T A 0, nclude the nequalty y T p 0. Every such nequalty s vald n the sense that every correlated equlbrum of G nduces a soluton to (14) (16) that also satsfes ths extra nequalty. Navely, there are nfntely many such extra nequaltes to add. Fortunately, we need only nclude those nequaltes that can arse as an optmal soluton to the followng problem: Gven p satsfyng (14) (16), mnmze y T p subject to y T A 0. (17) Snce (17) s a lnear program, the mnmum s always attaned by one of the fntely many basc solutons [4]. Moreover, n all such basc solutons, the vector y can be descrbed wth a number of bts polynomal n P [17, x6.2]. We have therefore defned a lnear system, whch we wll call the full lnear system for P, so that p s a soluton to the full lnear system f and only f p can be extended to a correlated equlbrum of G. Whle ths full lnear system has many nequaltes, each nequalty has sze polynomal n P. We can therefore effcently compute a soluton to the full lnear system va the ellpsod algorthm [17, 21], provded we can defne a polynomal-tme separaton oracle an algorthm that takes as nput a canddate soluton and, f the soluton s not feasble, produces a volated constrant. Such a separaton oracle s tantamount to a polynomal-tme algorthm for (17), whch s agan a lnear program wth exponentally many constrants (ndexed by strategy profles). We can solve (17) wth a second applcaton of the ellpsod method. Here, the separaton oracle requred s precsely the separaton problem for P (Defnton 4.2) whch, by assumpton, admts a polynomal-tme algorthm. The proof s therefore complete. Ξ Proof of Theorem 5.1: Let G =(S = f1;:::;kg;u 1 ;:::;u n ) be an n-player, k-strategy symmetrc game. As dscussed n Secton 2, there s a symmetrc Nash equlbrum p Λ = (p Λ 1 ;:::;pλ k ). We can guess the 14

16 support of p Λ (.e., try all possbltes) n tme exponental n k but ndependent of n and thus polynomal n n k. (The support of p Λ s the set of strateges for whch p Λ > 0.) So suppose we know the support of pλ, whch wthout loss of generalty s f1; 2;:::;jg for some 1» j» n. Let E` denote the expected payoff to player, f player chooses strategy ` and every other player chooses a strategy at random accordng to the dstrbuton p Λ. Snce G s symmetrc, E` s a polynomal n the j varables p Λ 1 ;:::;pλ j of degree n 1 that s ndependent of the player. Snce p Λ s a Nash equlbrum, t must satsfy the equatons E` = E`+1 for 1» ` < j and the nequaltes E j» E` for ` >j. Conversely, every vector (p 1 ;:::;p j ) wth non-negatve components that sum to 1 and that satsfes these equatons and nequaltes yelds a symmetrc Nash equlbrum. Fndng such a vector amounts to solvng O(k) smultaneous equatons and nequaltes wth degree O(n) n O(k) varables. It s known see Renegar [37] and the references theren that ths problem can be solved n tme polynomal n n k, the number of bts of the numbers n the nput, and n the number of bts of precson desred. Ξ 15