Communication and Cooperation in Repeated Games

Transcription

1 Communcaton and Cooperaton n Repeated Games Yu Awaya y and Vjay Krshna z October, 08 Abstract We study the role of communcaton n repeated games wth prvate montorng. We rst show that wthout communcaton, the set of Nash equlbrum payo s n such games s a subset of the set of "-coarse correlated equlbrum payo s ("-CCE) of the underlyng one-shot game. The value of " depends on the dscount factor and the qualty of montorng. We then dentfy condtons under whch there are equlbra wth "cheap talk" that result n nearly e - cent payo s outsde the set "-CCE. Thus, n our model, communcaton s necessary for cooperaton. Introducton The proposton that communcaton s necessary for cooperaton seems qute natural, even self-evdent. Indeed, n the old testament story of the Tower of Babel, God thwarted the mortals attempt to buld a tower reachng the heavens merely by dvdng the languages. The nablty to communcate wth each other was enough to doom manknd s buldng project. At a more earthly level, anttrust laws n many countres prohbt or restrct communcaton among rms. Agan, the premse s that lmtng communcaton lmts colluson. Organzatons seek to desgn nternal communcaton protocols to mprove performance. Many use subjectve peer evaluatons nonver able communcaton among employees and managers to compensate employees and t s felt that such schemes provde stronger ncentves for hard work (cooperaton). The research reported here was supported by a grant from the Natonal Scence Foundaton (SES-66783). We are grateful to Gabrel Carroll, Tm Roughgarden and Satoru Takahash for ther comments and suggestons. The referees and the edtor also provded valuable nput. y Department of Economcs, Unversty of Rochester, E-mal: yuawaya@gmal.com. z Department of Economcs, Penn State Unversty, E-mal: vkrshna@psu.edu.

2 Despte ts self-evdent nature, t s not clear how one may formally establsh the connecton between communcaton and cooperaton. One opton s to consder the e ects of pre-play communcaton n one-shot games. Ths allows players to coordnate and even correlate ther play n the game. But n many games of nterest, ths does not enlarge the set of equlbra to allow for cooperaton. For nstance, preplay communcaton has no e ect n the prsoners dlemma. It also has no e ect n a d erentated-product prce-settng olgopoly wth lnear demands. In ths paper we study the role of n-play communcaton n repeated games the basc framework for analyzng the prospects for cooperaton among self-nterested partes. The man dea of the theory of repeated games s that players are wllng to forgo short-term gans n order to reap future rewards. But ths reles on the ablty of players to montor each other well. If montorng s poor, cooperatve outcomes are hard to sustan because players can cheat wth mpunty and we study whether, n these crcumstances, communcaton can help. Spec cally, we show how n-play communcaton mproves the prospects for cooperaton n repeated games wth mperfect prvate montorng. In such settngs, players receve only nosy prvate sgnals about the actons of ther rvals. 3 Our man result (Theorem 6.) dent es montorng structures the stochastc mappng between actons and sgnals wth the property that communcaton s necessary for cooperaton. Theorem For any hgh but xed dscount factor, there exsts a non-empty and open set of montorng structures such that there s an equlbrum wth communcaton whose welfare exceeds that from any equlbrum wthout communcaton. What knds of montorng structures lead to ths concluson? Two condtons are needed. Frst, prvate sgnals should be rather nosy so that n the absence of communcaton montorng s poor. Second, prvate sgnals of players should be strongly correlated when they play e cent acton pro les and less so otherwse. 4 The second condton s natural n many economc envronments. In an earler paper (Awaya and Krshna, 06), we explored a prce-settng duopoly n whch rms sales were correlated n ths manner and showed how ths arose naturally because of randomness n consumers search costs. Another class of stuatons n whch the second condton s natural s the followng. Suppose that players can play cooperatvely expend e ort, contrbute to a publc good or gather nformaton or play sel shly and free The set of equlbrum payo s wth pre-play communcaton s contaned wthn wth set of correlated equlbrum payo s of the orgnal game. Ths follows from Aumann (987). Ths s a potental game n the sense of Monderer and Shapley (996) and a result of Neyman s (997) then mples that the set of correlated equlbra concdes wth the unque Nash equlbrum. 3 A classc example s Stgler s (964) model of secret prce cuts where rms choose prces that are not observed by other rms. The prces (actons) then stochastcally determne rms sales (sgnals). Each rm only observes ts own sales and must nfer ts rvals actons only va these. 4 Example below shows that communcaton does not help f the reverse s true that s, f correlaton actually ncreases followng a devaton.

3 rde on other players. These choces result n nosy prvate sgnals about an unknown state of nature. The stochastc structure s such that when a player plays cooperatvely, the sgnal s more nformatve about the state than f he or she were to play sel shly. In any such stuaton, the players sgnals wll be more correlated when all cooperate than when ths s not the case. 5 Ths knd of structure s common to many economc stuatons of nterest. For nstance, Flecknger (0) and Deb, L and Mukherjee (06) study a moral hazard stuaton n whch agents outputs are hghly correlated when all agents work and less so f anyone shrks. Deb et al. study the role of communcaton n the form of peer evaluatons n provdng ncentves. Smlarly, Gromb and Martmort (007) study the ncentves of, say, two experts to gather nformaton. When both experts expend e ort, they learn more about a true state of the world than otherwse. Ths also results n the features outlned above. How exactly does communcaton facltate cooperaton n such settngs? The basc dea s that players can montor each other not only by what they "see" the sgnals but also by what they "hear" the messages that are exchanged. But snce the messages are just cheap talk costless and unver able one may wonder how these can be used for montorng. We construct equlbra n whch the messages are cross-checked to ensure truthful reportng of sgnals. Moreover, the cross-checkng s su cently accurate so that devatng players nd t d cult to le e ectvely. These essental propertes rely on the second condton on the montorng structure mentoned above. We use these deas to construct an equlbrum that s nearly e cent. Our man result requres two steps. The rst task s to nd an e ectve bound on equlbrum payo s that can be acheved wthout communcaton. But the model we study s that of a repeated game wth prvate montorng and there s no known characterzaton of the set of equlbrum payo s. Ths s because wth prvate montorng each player knows only hs own hstory (of past actons and sgnals) and has only nosy nformaton about the prvate hstores of other players. Snce players hstores are not commonly known, these cannot be used as state varables n a recursve formulaton of the equlbrum payo set. In Secton 4, we borrow an equlbrum noton from algorthmc game theory that of a coarse correlated equlbrum and are able to relate (n Proposton 4.) the Nash equlbrum payo s of the repeated game to the coarse correlated equlbrum payo s of the one-shot game. The noton of a coarse correlated equlbrum was ntroduced by Mouln and Val (978). 6 The set of coarse correlated equlbra s larger than the set of correlated equlbra and so has less predctve power n one-shot games. But because t s very easy to compute, n some cases t s nevertheless useful n boundng the set of Nash equlbra for nstance, n congeston games (Roughgarden, 06). Here we show that t s useful 5 In Example below, we provde a smple dervaton of ths structure. 6 Whle a formal de nton appears n Secton, a coarse correlated equlbrum s a jont dstrbuton over players actons such that no player can gan by playng a pure acton under the assumpton that the other players wll follow the margnal dstrbuton over ther actons. 3

4 n boundng the set of Nash equlbra of repeated games as well. Precsely, Proposton The set of Nash equlbrum payo s of the repeated game wthout communcaton s a subset of the set of "-coarse correlated equlbrum payo s of the one-shot game. The """ s determned by the dscount factor and the montorng structure of the repeated game and we provde an explct formula for ths. When the montorng qualty s poor t s hard for other players to detect a devaton " s small and the set of "-coarse correlated equlbrum payo s provdes an e ectve bound to the set of equlbrum payo s n the repeated game. The second task s to show that wth communcaton, equlbrum payo s above the bound can be acheved. To show ths, we construct a cooperatve equlbrum explctly n whch, n every perod, players publcly report the sgnals they have receved (see Proposton 5.). Players reports are aggregated nto a sngle "score" and the future course of play s completely determned by ths summary statstc. Devatons result n low scores and trgger punshments wth hgher probablty. The score functon lets us construct an equlbrum wth communcaton to be of a partcularly smple "trgger-strategy" form. When sgnals are "very nformatve" about each other, the constructed equlbrum s nearly e cent. We emphasze that the analyss n ths paper s of a d erent nature than that underlyng the so-called "folk theorems" (see Sugaya, 05). These show that for a xed montorng structure, as players become ncreasngly patent, near-perfect colluson can be acheved n equlbrum. In ths paper, we keep the dscount factor xed and change the montorng structure so that the set of equlbra wth communcaton s substantally larger than the set wthout. A d culty here s that montorng structures whch are stochastc mappngs from actons to sgnals are hgh-dmensonal objects. We show, however, that only two easly computable parameters one measurng how nosy the sgnals are and the other how strongly correlated they are su ce to dentfy montorng structures for whch communcaton s necessary for cooperaton. Related lterature The current paper bulds on our earler work, Awaya and Krshna (06), where we explored some of the same ssues n the specal context of Stgler s (964) model of secret prce cuts n a symmetrc duopoly wth nosy sales. Ths s, of course, the canoncal example of a repeated game wth prvate montorng. Much of the analyss n our earler paper, however, reled on the assumpton that sales were (log-) normally dstrbuted and that the two rms were symmetrc. Ths paper consders general n-person nte games and general sgnal dstrbutons. More mportant, the bound on payo s wthout communcaton that s developed here s tghter than the 4

5 bound constructed n the earler paper as well as beng easer to nterpret. 7 Moreover, the constructon of equlbra wth communcaton s entrely d erent and does not rely on any symmetry among players. Fnally, we show n ths paper (see Example below) that communcaton does not always help cooperaton. Awaya (04) explores some of the same ssues n a repeated prsoners dlemma n contnuous tme. There s a vast lterature on repeated games under d erent knds of montorng. Under perfect montorng, gven any xed dscount factor, the set of perfect equlbrum payo s wth and wthout communcaton s the same. Under publc montorng, agan gven any xed dscount factor, the set of (publc) perfect equlbrum payo s wth and wthout communcaton s also the same. Thus, n these settngs communcaton does not a ect the set of equlbra. 8 Compte (998) and Kandor and Matsushma (998) study repeated games wth prvate montorng allowng, as we do, for communcaton among players. In ths settng, they show that the folk theorem holds any ndvdually ratonal and feasble outcome can be approxmated as the dscount factor tends to one. These results are derved under spec c assumptons about the detectablty of devatons by other players and can be sats ed only f there are at least three players. Ths lne of research has been pursued by others as well (see Aoyag (00), Fudenberg and Levne (007), Zheng (008) and Obara (009)) n envronments d erent from, and sometmes more general than, those of Compte (998) and Kandor and Matsushma (998). Partcularly related to the current paper s the work of Aoyag (00) and Zheng (008) who assume, as we do, that players sgnals exhbt greater correlaton when e cent actons are played than when actons are ne cent. All of these papers thus show that communcaton s su cent for cooperaton when players are su cently patent. But as Kandor and Matsushma (998) recognze, One thng whch we dd not show s the necessty of communcaton for a folk theorem" (p. 648, ther talcs). In a remarkable paper, Sugaya (05) shows the surprsng result that n very general envronments, the folk theorem holds wthout communcaton. Thus, n fact, communcaton s not necessary for a folk theorem. Although Sugaya s result was preceded by folk theorems for some lmtng cases where the montorng was almost perfect or almost publc, the generalty of ts scope was unantcpated. Unlke the folk theorems, n our work we do not consder the lmt of the set of equlbrum payo s as players become arbtrarly patent. We study the set of equlbrum payo s for a xed dscount factor. Key to our result s a method of boundng the set of payo s wthout communcaton usng the easly computable set of "-coarse correlated equlbra. Pa, Roth and Ullman (06) also develop a bound that depends on a measure of montorng qualty based on the computer-scence noton of "d erental prvacy." But the bound so obtaned apples to equlbrum payo s wth communcaton as well as those wthout, and so does not help n dstngushng between the two. 7 See Secton To be precse, these equvalences requre that n the absence of communcaton, players have access to a publc randomzng devce ("sunspots") and that all communcaton s publc. 5

6 Sugaya and Woltzky (07) nd su cent condtons under whch the equlbrum payo s wth prvate montorng are bounded by the equlbrum payo s wth perfect montorng. Ths bound agan apples whether or not there s communcaton and so s also unable to dstngush between the two. Spector (05) shows that communcaton can be bene cal n a model of prce competton wth prvate montorng. Frms see ther own current sales but, unlke n our model, can see other rms sales wth some delay. Communcaton s helpful n reducng ths delay n montorng. In our model, all communcaton s pure cheap talk prvate sgnals reman so forever. Kandor (003) establshes a folk theorem wth communcaton n a repeated game wth publc montorng. Wth publc montorng, the only useful communcaton concerns the prvately known actons that players have taken. In equlbrum such actons reman prvate when players randomze and the outcomes of these randomzatons are known only to the player n queston. Kandor shows how wth prvate strateges and communcaton, a folk theorem may be establshed under weaker condtons than requred when only publc strateges are used. But agan, ths does not show the necessty of communcaton snce t s not known what can be acheved wth prvate strateges wthout communcaton. Rahman (04) derves a smlar result n a duopoly model. The role of communcaton n fosterng cooperaton has also been the subject of numerous experments n vared nformatonal settngs. Of partcular nterest s the work of Ostrom et al. (994, Chapter 7) who nd that n-play communcaton n repeated common-pool resource games leads to greater cooperaton than does preplay communcaton. In recent work, Aryal, Clberto and Leyden (08) study how publc communcaton a ects colluson among arlnes. The remander of the paper s organzed as follows. The next secton outlnes the formal model of repeated games wth prvate montorng. To motvate the subsequent analyss, n Secton 3 we present some of the man deas, as well as some subtletes, by means of some smple examples. Secton 4 analyzes the repeated game wthout communcaton whereas Secton 5 does the same wth communcaton. The bene ts of communcaton are establshed n Secton 6. Secton 7 concludes. Appendx A contans omtted proofs from Secton 4 regardng the no-communcaton bound. Appendx B analyzes the score functon that forms the bass of the equlbrum wth communcaton that s constructed n Secton 5. Prelmnares As mentoned n the ntroducton, we study repeated games wth prvate montorng. Stage game The underlyng game s de ned by I; (A ; Y ; w ) I ; q where I = f; ; :::; ng s the set of players, A s a nte set of actons avalable to player 6

7 and Y s a nte set of sgnals that may observe. The actons of all the players a (a ; a ; :::; a n ) A A together determne q ( j a) (Y ), a probablty dstrbuton over the sgnals of all players. 9 A vector of sgnals y Y s drawn from ths dstrbuton and player only observes y : Player s payo s then gven by the functon w : A Y! R so that s payo depends on other players actons only va the nduced sgnal dstrbuton q ( j a) : We wll refer to w (a ; y ) as s ex post payo. 0 Pror to any sgnal realzatons, the expected payo of player s then gven by the functon u : A! R; de ned by u (a) = P y Y w (a ; y ) q (y j a) where q ( j a) (Y ) s the margnal dstrbuton of q ( j a) on Y so that q (y j a) = P y Y q (y ; y j a) : As usual, kuk denotes the sup-norm of u: In what follows, we wll merely specfy the expected payo functons u not the underlyng ex post payo functons w : The latter can be derved from the former for generc sgnal dstrbutons spec cally, as long as fq ( j a) : a Ag s a lnearly ndependent set of vectors. We refer to G (A ; u ) I as the stage game. The set of feasble payo s n G s F = co u (A) ; the convex hull of the range of u. A payo vector v F s (strongly) e cent f there does not exst a feasble v 6= v such that v v : The collecton fq ( j a)g aa s referred to as the montorng structure. We suppose throughout that for all ; the margnal dstrbuton over s sgnals, q ( j a) (Y ) has full support, that s, for all y Y and a A; q (y j a) > 0 () Qualty of montorng Let q ( j a) (Y ) be the margnal dstrbuton of q ( j a) (Y ) over the jont sgnals of the players j 6= : The qualty of a montorng structure q s de ned as = max max kq ( j a) q ( j a 0 ; a )k a;a 0 T V () 9 We adopt the followng notatonal conventons throughout: captal letters denote sets wth typcal elements denoted by lower case letters. Subscrpts denote players and unsubscrpted letters denote vectors or cartesan products. Thus, x X and x = (x ; x ; :::; x n ) X = X : Also, x denotes the vector obtaned after the th component of x has been removed and (x 0 ; x ) denotes the vector where the th component of x has been repaced by x 0 : Fnally, (X) s the set of probablty dstrbutons over X: 0 Ths ensures that knowledge of one s ex post payo does not carry any nfomaton beyond that n the sgnal. For nstance, n Stgler s (964) model a rm s pro ts depend only on ts own actons (prces) and ts own sgnal (sales). 7

8 where k k T V denotes the total varaton dstance between the probablty measures and. It s ntutvely clear that when the qualty of montorng s poor, t s hard for players other than to detect a devaton by : Coarse correlated equlbrum The dstrbuton (A) s a coarse correlated equlbrum (CCE) of G f for all and all a A ; u () u (a ; ) where (A ) denotes the margnal dstrbuton of over A (see Mouln and Val, 978). The noton of a CCE s best understood by contrastng t wth the noton of a classcal correlated equlbrum (CE). A correlated equlbrum can be thought of as a "medated soluton" a medator draws a jont acton a A from a dstrbuton (A) and sends a prvate recommendaton to each player. The dstrbuton consttutes a CE f for every a n the support of ; no player can gan by choosng a d erent acton than the one recommended. A coarse correlated equlbrum can also be thought of as a "medated soluton but wth commtment" the players have to decde whether or not to "sgn on" to the medated soluton wthout knowng anythng other than the dstrbuton : A CCE nvolves greater commtment on the part of the players than does a CE they agree to play accordng to a jont agreement knowng nothng else. The dstrbuton (A) s an "-coarse correlated equlbrum ("-CCE) of G f for all and all a A ; u () u (a ; ) " (3) De ne "-CCE (G) = fu () F : s an "-CCE of Gg to be the set of "-coarse correlated equlbrum payo s of G: Repeated game We wll study an n ntely repeated verson of G, denoted by G, de ned as follows. Tme s dscrete and ndexed by t = ; ; ::: and n each perod t, the game G s played. Payo s n the repeated game G are dscounted averages of per-perod payo s usng the common dscount factor (0; ) : Precsely, f the sequence of actons taken s (a ; a ; :::) ; player s ex ante expected payo s ( ) P t t u (a t ) : A (behavoral) strategy for player n the game G s a sequence of functons = ( ; ; :::) where t : A t Y t! (A ). Hence, a strategy determnes a player s current, possbly mxed, acton as a functon of hs prvate hstory hs own past actons and past sgnals. The total varaton dstance between two probablty measures and on X s de ned as k k T V = P xx j (x) (x)j : Ths s, of course, equvalent to the metrc derved from the L norm. Further dscusson along these lnes can be found n Mouln et al. (04). 8

9 Repeated game wth communcaton We wll also study a verson of G, denoted by G com, n whch players can communcate wth each other after every perod by sendng publc messages m from a nte set M : The communcaton phase n perod t takes place after the sgnals n perod t have been observed. Thus, a strategy of player n the game G com conssts of two sequences of functons = ( ; ; :::) and = ( ; ; :::) where t : A t Y t M t! (A ) determnes a player s current acton as a functon of hs own past actons, past sgnals and past messages from all the players. The functon t : A t Y t M t! (M ) determnes a player s current message as a functon of hs own past and current actons and sgnals as well as past messages from all the players. The messages m themselves have no drect payo consequences. Equlbrum noton We wll consder sequental equlbra of the two games. For G ; the repeated game wthout communcaton, the full support condton () ensures that the set of sequental equlbrum payo s concdes wth the set of Nash equlbrum payo s (see Sekguch, 997). In both stuatons, we suppose that players have access to publc randomzaton devces. 3 Some examples Before begnnng a formal analyss of equlbrum payo s n the repeated game G and ts counterpart wth communcaton, G com, t wll be nstructve to consder a few examples. The rst example llustrates, n the smplest terms, the man result of the paper. The other examples then pont to some complextes. A word of warnng s n order. All of the followng examples have the property that the margnal dstrbutons of players sgnals are the same regardless of players actons. Ths means that the expected payo functons u (a) cannot be derved from underlyng ex post payo functons w (a ; y ) and, of course, expected payo s are not observed. The examples have ths property only to llustrate some features of the model n the smplest way possble. Ths s not essental the examples can easly be amended so that underlyng ex post payo functons, whch are observed, exst. Example : Communcaton s necessary for cooperaton. Consder the followng prsoners dlemma as the stage game wth expected payo s: c d c ; ; 3 d 3; 0; 0 9

10 Each player has two possble sgnals y 0 and y 00 and suppose that the montorng structure q s q ( j cc) = y 0 y 00 y 0 " " y 00 " " q ( j : cc) = y 0 4 y 00 4 y 0 y (4) where : cc denotes any acton pro le other than cc: We argue below that wthout communcaton, t s mpossble for players to "cooperate" that s, to play cc and that the unque equlbrum payo s (0; 0). Wth communcaton, however, t s possble for the players to cooperate (wth hgh probablty) and, n fact, attan average payo s close to (; ) : The montorng structure here has two key features. Frst, the margnal dstrbutons q ( j a) are dentcal no matter what acton a s played, so that the qualty of montorng (de ned n ()) s zero. Second, f cc s played, each player s sgnal s very nformatve about the other player s sgnal. If somethng other than cc s played, a player s sgnal s completely unnformatve about the other player s sgnal. 3 At the end of ths example, we derve ths montorng structure from more basc consderatons. Clam Wthout communcaton, cooperaton s not possble for all ; the unque equlbrum payo of G s (0; 0) : Fx any strategy of player : Snce the margnal dstrbuton on player s sgnals s not a ected by what player does, hs ex ante belef on what player wll play n any future perod s also ndependent of what he plays today. Thus, n any perod, player s better o playng d rather than c: Cooperaton s mpossble. 4 Clam Wth communcaton, cooperaton s possble gven any > ; there exsts an " such that for all " < ", there exsts an equlbrum of G com whose payo s are "-close to (; ) : Now suppose that players report ther sgnals durng the communcaton phase that s, M = Y : Consder the followng varant of a "trgger strategy": play c n perod and n the communcaton phase, report the sgnal that was receved. In any perod t; play c f n all past perods, the reported sgnals have agreed that s, f both players reported y 0 or both reported y 00 : If the reports dsagreed n any past perod, play d: In the communcaton phase, report your sgnal. 3 The term "nformatve" s used n the sense of Blackwell (95). From player s perspectve, the sgnals of the other players y consttute the "state of nature" and hs own sgnal carres nformaton about ths. 4 The equalty of the margnals volates one of Sugaya s (05) condtons and so hs folk theorem does not apply. 0

11 To see that these strateges consttute an equlbrum, note rst that f all past reports have agreed, and a player has played c n the current perod, then there s no ncentve to msreport one s sgnal. Msreportng only ncreases the probablty of trggerng a punshment from " to and so there s no gan from devatng durng the communcaton phase. Fnally, f all past reports agreed, a player cannot gan by devatng by playng d: Such a devaton wll trgger a punshment wth probablty ; no matter what he reports n the communcaton phase. It s routne to verfy that when " s small, ths s not pro table. Each player s payo n ths equlbrum s v = + " whch converges to as " converges to zero. The montorng structure n ths example can be derved from more basc consderatons, as outlned n the ntroducton. Suppose that there are two equally lkely states of nature! 0 and! 00 and players get a nosy prvate sgnal y 0 or y 00 about the state. Condtonal on the state, players sgnals are ndependent. If a player plays c n state! 0 ; then he receves sgnal y 0 wth probablty > and sgnal y00 wth probablty : Lkewse, f a player plays c n state! 00 ; then he receves y 00 wth probablty and y 0 wth probablty : On the other hand, f a player plays d; then the two sgnals are equally lkely regardless of state. Thus, cooperatng provdes nformaton about the state whereas defectng does not. A routne computaton shows that the montorng structure n (4) results for " = ( ) : Example : Cooperaton s mpossble even wth communcaton The rst example exhbted some crcumstances n whch cooperaton was not possble wthout communcaton but wth communcaton, t was. Does communcaton always facltate cooperaton? As the next example shows, ths s not always the case the sgnal structure q matters. Consder the prsoners dlemma of Example agan but wth the followng " pped" sgnal structure: q ( j : dd) = y 0 4 y 00 4 y 0 y q ( j dd) = y 0 y 00 y 0 " " y 00 " " where agan, : dd denotes any acton pro le other than dd: The margnal dstrbuton of sgnals q ( j a) s, as before, una ected by players actons s zero agan. But now the sgnal dstrbuton when dd s played s more nformatve than when any other acton s played n fact, the former s completely nformatve. Clam 3 Wth or wthout communcaton, cooperaton s not possble for all, the unque equlbrum payo n both G and G com s (0; 0) :

12 Suppose that there s an equlbrum wth communcaton n whch after some hstory, player s supposed to play c wth probablty one and report hs sgnal truthfully. Suppose player plays d nstead of c and at the communcaton stage, regardless of hs prvate sgnal, reports wth probablty one-half that hs sgnal was y 0 and wth probablty one-half that hs sgnal was y 00 : Now regardless of whether player plays c or d n that perod, the jont dstrbuton over player s reports and player s sgnals s the same as f player had played c that s, q ( j : dd) : Thus, player can devate and "le" n a way that hs devaton cannot be statstcally detected. 5 Thus, wth the " pped" montorng structure no cooperaton s possble even wth communcaton. A fortor, no cooperaton s possble wthout communcaton ether. In ths example, therefore, communcaton s unable to facltate cooperaton. Example 3: Communcaton s not necessary for cooperaton Our nal example llustrates the possblty that "full" cooperaton s possble wthout communcaton even though montorng s very poor n fact, non-exstent. 6 Consder the followng verson of "rock-paper-scssors": r p s r 0; 0 0; ; 0 p ; 0 0; 0 0; s 0; ; 0 0; 0 The stage game has a unque correlated equlbrum, and hence a unque Nash equlbrum as well, n whch players randomze equally among the three actons and results n a payo of (7; 7) : Suppose the montorng structure s: q ( j a = a ) = y r y p y s y r y p y s q ( j a 6= a ) = y r 9 y p 9 y s 9 y r y p y s so that f players coordnate on the same acton, then the sgnals are perfectly nformatve; otherwse, they are unnformatve. Once agan = 0 snce the margnal dstrbutons of sgnals are not a ected by players actons. Clam 4 Cooperaton s possble wthout communcaton for any 3 ; there exsts an equlbrum of G wth a payo of (0; 0) : In what follows, we wll say that the players are "coordnated" f they take the same acton and so the resultng sgnal dstrbuton s q ( j a = a ) : Otherwse, they are sad to be "mscoordnated." 5 Ths argument can be extended to nclude randomzed strateges. Detals can be obtaned from the authors. 6 Ths trvally holds n games where there s an e cent one-shot Nash equlbrum, of course. In ths example that s not the case

13 Consder the followng strategy: n perod ; play r: In perod t; play acton a fr; p; sg f the sgnal receved n the last perod was y a : The average payo from ths strategy s clearly 0 snce the players are always coordnated. Now suppose player devates once from the prescrbed strategy and then reverts back to t. Player s mmedate payo from the devaton s : But the devaton also causes the players to become mscoordnated. So no matter what sgnal player receves, player s equally lkely to play each of hs actons. As a result, once the players are mscoordnated, the contnuaton payo s w = 0 + (( ) + w) + (( ) 0 + w) Ths s because wth probablty ; the players wll become coordnated agan n the 3 next perod and then reman coordnated thereafter. Wth probablty ; they wll 3 reman mscoordnated, player wll get and then the contnuaton payo w; wth probablty ; he wll get 0 and then w agan. Thus the contnuaton payo after 3 mscoordnaton s w = 3 The orgnal devaton s not pro table as long as ( ) + w 0 and ths holds as long as 3 : The one-devaton prncple for games wth prvate montorng (Malath and Samuelson, 006, p. 397), then ensures that the prescrbed strateges consttute an equlbrum. The example thus demonstrates that n a repeated settng, players can sometmes acheve outcomes far superor to the set of correlated equlbra of the stage game even though there s zero montorng. 4 Equlbrum wthout communcaton In ths secton, we develop a method to bound the set of equlbrum payo s n games wth prvate montorng. We wll show that the set of equlbrum payo s of the repeated game wthout communcaton G s contaned wthn the set of "-coarse correlated equlbrum payo s of the stage game G. In our result, we gve an explct formula for " nvolvng (a) the dscount factor; and (b) the qualty of montorng (as de ned n ()). The man result of ths secton s: Proposton 4. where " = kuk : NE (G ) "-CCE (G) 3

14 (0; ) (0; 0) p p p p p p CCE tp p p p p p p p p p p CE (; 0) Fgure : Rock-Scssors-Paper: CE = (7; 7) and CCE = shaded area The proposton s of ndependent nterest because repeated games wth prvate montorng do not have a natural recursve structure and so a characterzaton of the set of equlbrum payo s seems ntractable. So one s left wth the task of ndng e ectve bounds for ths set. Proposton 4. provdes such a bound and one that s easy to compute explctly: the set "-CCE s de ned by P ja j lnear nequaltes. Moreover, the bound does not use any detaled nformaton about the montorng structure t depends only on the montorng qualty parameter : 7 Proposton 4. relates Nash equlbra of the repeated game to coarse correlated equlbra of the one-shot game. It s clear that the sgnals that players receve result n ther actons beng correlated and so the relatonshp to some "medated soluton" s not unnatural. To see why CCE s the rght noton, consder a montorng technology for whch = 0; that s, a stuaton n whch player s actons do not a ect the dstrbuton of sgnals of other players. Ths means, of course, that any devaton by player wll go undetected. Consder a repeated game equlbrum strategy whch s statonary and so results n a xed dstrbuton of actons (A) n every perod. Because devatons cannot be detected, each player can guarantee that hs or her payo s max a u (a ; ) whch mples mmedately that must be a coarse correlated equlbrum. The formal proof below allows for both > 0 as well as non-statonary strateges. The ""-coarse correlated equlbrum" n the statement cannot smply be replaced wth ""-correlated equlbrum." Precsely, f the set of "-correlated equlbrum payo s of G s denoted by "-CE (G) ; then the statement NE (G ) "-CE (G) s false for the 7 Sugaya and Woltzky (07) show that the set of equlbrum payo s wth perfect montorng and a medtor s a bound for large enough : Unlke ours, ther bound s ndependent of the qualty of montorng. Our method results n tghter upper bounds to payo s when the qualty of montorng s poor. 4

15 same value of " as above. For nstance, n Example 3 the set of correlated equlbrum payo s s a sngleton CE (G) = f(7; 7)g whle the set of coarse correlated equlbra s as depcted n Fgure. For the montorng structure n Example 3, = 0 and hence " = 0 as well. But for 3 ; repeated game has an equlbrum payo of (0; 0) whch s n CCE (G) but not n CE (G) : Sketch of Proof We ndcated how the CCE bound arses naturally when there s zero montorng ( = 0) and the equlbrum strategy of the repeated game s statonary. We now show how the argument s extended to permt both > 0 and non-statonary strateges. So consder strategy pro le of the repeated game wth a payo v () that s not an "-CCE payo n the one-shot game. Suppose that some player devates to a strategy n whch chooses a n every perod regardless of hstory that s, conssts of a permanent devaton to a : We decompose the (possble) gan from such a devaton nto two bts. Consder a cttous stuaton n whch the players j 6= are replaced by a non-responsve machne that, n every perod, and regardless of hstory, plays the ex ante dstrbuton t (A ) that would have resulted from the canddate strategy : In the cttous stuaton, player s devaton s unpunshed n the sense that the machne contnues to play as f no devaton had occurred. Note that ths lack of response s what would have occurred f were zero. We can then wrte v ( ; ) v () {z } Gan from devaton = v ( ; ) v ( ; ) {z } Loss from punshment + v ( ; ) v () {z } Gan when unpunshed (5) The rst component on the rght-hand sde represents the payo d erence from facng the real players j 6= versus facng the non-responsve machne. If s an e ectve deterrent to the permanent devaton then ths should be negatve and n Lemma 4. we calculate a lower bound to ths loss. The second component s the gan to player when hs permanent devaton goes unpunshed. As we wll show below n Lemma 4., ths gan can be related to the coarse correlated equlbra of the one-shot game (see (3)). The lemma establshes that a permanent devaton, whch s, of course, statonary, s best deterred by a statonary dstrbuton. Non-responsve strateges We begn wth a formal de nton of the nonresponsve strategy played by the cttous "machne." Gven a strategy pro le ; the nduced ex ante dstrbuton over A n perod t s t () = E j t j h t j (A) and the correspondng margnal dstrbuton over A n perod t s t () = E j6= t j h t j (A ) 5

16 where the expectaton s de ned by the probablty dstrbuton over t hstores determned by : Note that depends on the whole strategy pro le and not just on the strateges of players other than : Note also that because players hstores are correlated, t s typcally the case that t () = j6= (A j ) : Gven ; let () denote the (correlated) strategy of players j 6= n whch they play t () n perod t followng any t perod hstory. The strategy ; whch s merely a sequence t of jont dstrbutons n (A ) ; replcates the ex ante dstrbuton of actons of players j 6= resultng from but s non-responsve to hstores. We now proceed to decompose the gan from a permanent devaton. 4. Loss from punshment In ths subsecton we provde a bound on the absolute value of v ( ; ) v ( ; ) ; the d erence n payo s between beng punshed by strategy of the real players j 6= versus not beng punshed by the cttous machne. It s clear that the magntude of ths d erence depends crucally on how responsve s compared to the and ths n turn depends on how well the players j 6= can detect s permanent devaton. We show below that ths loss can n fact be bounded by a quantty that s a postve lnear functon of : Moreover, the bound s ncreasng n : The followng result provdes an exact formula for the trade-o between the qualty of montorng and the dscount factor. Lemma 4. Suppose plays a always. The d erence n s payo when others play versus when they play the non-responsve strategy derved from sats es Proof. See Appendx A. 4. Gan when unpunshed jv ( ; ) v ( ; )j kuk We now relate the second component n (5) to the "-coarse correlated equlbra of the one-shot game (see (3)). We begn by characterzng the set of "-CCE payo s. For any v F; de ne (v) mn max u a ; u () a subject to max (A) u () = v where (A ) denotes the margnal dstrbuton of over A : In words, no matter how the payo v s acheved va a correlated acton, at least one player can gan at least (v) by devatng. It s easy to see that v "-CCE (G) 6

17 f and only f (v) ": Ths s because (v) " s the same as: there exsts a (A) satsfyng u () = v such that max max u a ; u () " a A and ths s equvalent to v "-CCE (G) : Note that (v) s also the value of an art cal two-person zero-sum game n whch player I ("devator") chooses a par (; a ) and player II ("medator") chooses a jont dstrbuton (A) such that u () = v: The dea s that the medator chooses a jont dstrbuton and the devator chooses a player and a pure strategy for that player as a devaton. The payo to player I s then u a ; u () : The fact that (v) 0 s the same as v CCE (G) s analogous to a result of Hart and Schmedler (989) on correlated equlbra. The followng mportant result shows that the functon, whch measures the statc ncentves to devate, also measures the dynamc ncentves to permanently devate from a non-responsve strategy. Formally, consder a dynamc analogue of the two-person zero-sum game outlned above. In ; player I ("devator") chooses a par (; ) where denotes the constant sequence a (a permanent devaton) and player II ("medator") chooses a non-responsve strategy that s, a sequence of jont dstrbutons (A) such that ts dscounted average v () = v: The payo to player I n s v ( ; ) v () : The lemma shows that the value of ; that s, (v), s also the value of : Ths reles on the fact that the devaton s permanent (statonary). Snce the maxmzed payo functon of player I s convex n player II s strategy, player II optmal response s statonary as well. Ths last step resembles the famlar "consumpton smoothng" argument. Lemma 4. (v) = mn max (A) max [v ( ; ) v ()] subject to the constrant that the dscounted average payo from ; v () = v where v ( ; ) s s payo when he plays a always and others play the nonresponsve strategy = ; ; ::: (A ) derved from = ( ; ; :::) (A) : Proof. It s clear that (v) s at least as large as the rght-hand sde of the equalty above. Ths s because the set of strateges avalable to player II n ncludes all statonary strateges and the latter are equvalent to all strateges n : The set of strateges avalable to player I n and are the same. Gven a sequence = ( ; ; :::) (A), let v t = u ( t ) be the ex ante payo s n perod t: Then, f v () = v we have ( ) P t= t v t = v: 7

18 For any payo vector w F; de ne (w; a ) = mn (A) u subject to u () = w: Then, for for all and all a ; P v ( ; ) v () = ( ) t u t= P ( ) t t= a ; t v t ; a u a ; t u () (6) where the second nequalty follows from the de nton of. Now snce (; a ) s convex 8, t s the case that a soluton to the problem: P mn ( ) t v t t= v t ; a subject to P ( ) t v t = v t= s to set v t = v for all t. Thus, we have that P ( ) t v t ; a (v; a ) t= whch when combned wth (6) yelds that for all and a ; v ( ; ) v () (v; a ) = mn :u()=v u a ; u () Ths mples that mn [v ( ; ) v ()] mn :v()=v :u()=v u a ; u () and thus max ([A j ) mn E [v (a ; ) v ()] max :v()=v mn E u ([A j ) :u()=v a ; u () Applyng the mnmax theorem (Son, 958) n the game n the game on the rght-hand sde, we obtan mn max max v ( ; ) v () mn :v()=v a max :u()=v = (v) on the left-hand sde and max a u a ; u () 8 The convexty of (; a ) s a consequence of the fact that u s lnear n : 8

19 4.3 Payo bound Wth Lemmas 4. and 4. n hand, we can now complete the proof of the result (Proposton 4.) that the set of Nash equlbrum payo s of the repeated game s contaned n the set of "-coarse correlated equlbrum payo s of the one-shot game. Suppose s a strategy pro le n G such that v v () = "-CCE (G) for " = kuk : Then we know that (v) > ": Lemma 4. mples that and so mn max max [v ( ; ) v ()] > " (A) max max [v ( ; ) v ()] > " where s the non-responsve strategy derved from as above. Thus, there exsts a player and a permanent devaton for that player such that v ( ; ) v () > ": Applyng Lemmas 4. and 4. we have v ( ; ) v () = v ( ; ) v ( ; ) + v ( ; ) v () > kuk + " = 0 Thus, s not a Nash equlbrum of G : Ths completes the proof of Proposton Bound Comparson The bound n Proposton 4. s related to but (weakly) tghter than the bound obtaned n our earler work (Awaya and Krshna, 06), whch s called the - bound as t depends on a functon de ned there. The -bound was obtaned by consderng a permanent devaton to a sngle devatng acton a = a BR, a one-perod best response to a whch unquely acheves an e cent payo u. The new CCEbound, derved n ths secton, also consders a permanent devaton but both the dentty of the devator and the devatng acton a are chosen optmally. In the prsoners dlemma, there s no d erence n the bounds but n many other games, the resultng mprovement can be substantal. For nstance, n the game, b c d b 8; 8 0; 0 0; 7 c 0; 0 8; 8 0; 0 d 0; 0 7; 0 0; 0 the -bound s trval. It says merely that all equlbrum payo s of the repeated game satsfy v + v 8 (the symmetrc e cent payo n the one-shot game). The reason the earler -bound s trval s that there s an e cent mxed acton 9

20 whch places probablty on (b; b; ) and probablty on (c; c) resultng n a payo of (8; 8) : But when (b; b) s played, player has the ncentve to devate to d and when (c; c) s played, player has the ncentve to devate to d: The -bound does not allow the devator be chosen dependng on whch of (b; b) and (c; c) s played the sngle devator must be chosen once and for all. Thus, the -bound under-estmates the ncentve to devate. In ths game, however, the unque CCE payo s (0; 0) and the hghest symmetrc "-CCE payo s (6"; 6") : Proposton 4. now mples that equlbrum payo s of the repeated game satsfy v + v 6": The mprovement n the bound can be calculated n any game (both the functon and the set of CCE payo s can be computed va lnear programs). To get some sense of the mprovement n a game of economc nterest, consder a duopoly n whch rms compete by settng prces for d erentated products. Ths was the model studed n Awaya and Krshna (06). Suppose that the rms demands are lnear n prces: for = ; and j 6= ; d = A bp + p j where b >. Note that snce b > ; the "own-prce e ect" on a rm s demand s greater than the "cross-prce e ect." In ths example, the d erence n total pro ts gven by the two bounds when there s zero montorng ( = 0) can be obtaned n closed form: -bound CCE-bound = A 4b 8 b (b ) (7) b The -bound for ths game can be calculated usng the dervaton n Awaya and Krshna (06, p. 305). In the case of lnear demand, the set of CCE of ths game concdes wth the unque Nash equlbrum (see Gérard-Varet and Mouln, 978) and s easly calculated. To get a sense of the mprovement, suppose b = : Then the -bound says that when the montorng s poor n any equlbrum of the repeated game, the total pro ts of the two rms cannot exceed approxmately 44% of the gap between total monopoly pro ts and total Nash pro ts. But the CCE-bound s much tghter. It says that wth poor montorng, total pro ts n any equlbrum cannot exceed approxmately % of the gap. 5 Equlbrum wth communcaton In what follows, we wll consder e cent actons a that Pareto domnate some Nash equlbrum of the stage game, that s, u (a ) u N where N (A ) s a (possbly mxed) Nash equlbrum of the stage game. 9 In Secton 5. below, we wll dsplay a partcular strategy pro le for the game wth communcaton. Then n 9 Ths condton can be easly weakened to requre only that u (a ) Pareto domnate some convex combnaton of one-shot Nash equlbrum payo s. 0

21 Secton 5., we wll dentfy condtons on the sgnal structure q and the dscount factor that guarantee that the pro le consttutes an equlbrum whch s "nearly" e cent (see Proposton 5. below). We emphasze that our result s not a "folk theorem." In the latter, the sgnal structure s held xed and the dscount factor s rased su cently so that any feasble outcome can arse n equlbrum. In our result, the dscount factor s held xed (perhaps at some hgh level) and the montorng structure s vared so that e cent outcomes can be sustaned n equlbrum. In what follows, t wll convenent to assume that all players have the same set of sgnals that s, for all and j; Y = Y j and wthout loss of generalty, we wll suppose that for all ; the set of sgnals Y = f; ; :::; Kg : 5. A smple equlbrum We wll now construct a nearly e cent equlbrum wth communcaton. De ne Y D to be the set of (dagonal) sgnal pro les such that y = y = ::: = y n ; that s, pro les n whch all players sgnals are the same. The proposed equlbrum strategy resembles a trgger strategy and s very smple: In perod, play a and report the sgnal receved truthfully. In any perod t > ; play a and report the sgnal y t truthfully f all players have reported the same sgnal n all past perods, that s, f all players have reported the same sgnal n every perod s < t; that s, f for all s < t; y s Y D : Otherwse, play the one-shot Nash acton N and report y t truthfully. The strategy thus requres that all players unanmously agree on a sgnal n order to contnue cooperatng. Any dssent results n n nte punshment. Another way to wrte ths, useful for later comparsons, s that the strategy calls on players to cooperate n perod t f and only f I Y D (y s ) = n all perods s < t; where I Y D s the ndcator functon of the set of dagonal pro les Y D : Fx the set of sgnal dstrbutons fq ( j a) : a 6= a g and suppose that for all a 6= a ; q ( j a) has full support. As a rst step, consder a sgnal dstrbuton q = q ( j a ) that s degenerate on sgnal pro les n whch all the players sgnals agree and assgns postve probablty to all such pro les. Formally, q (y) > 0 f and only f y = y = ::: = y n : We wll call such a dstrbuton perfectly nformatve snce any player s sgnal provdes perfect nformaton about others sgnals. Thus, the termnology s consstent wth that of Blackwell (95). Frst, suppose that all players follow the suggested strateges. Then the payo of player s u (a ) snce punshments are never trggered. We wll now argue that f no one has devated untl now, then no player has any ncentve () to msreport hs sgnal; or () to devate to another acton. Suppose that the suggested strateges are beng played and we are n a stuaton n whch n all past perods, players reports have agreed. If player plays a n perod t and then receves the sgnal y she s sure that all other players sgnals are

22 the same as hers. Reportng y truthfully s then strctly better than reportng any other sgnal snce dong the latter s sure to trgger a punshment. Thus, no player has the ncentve to le along the equlbrum path. After a dsagreement, the play s ndependent of the reports. Fnally, provded that the dscount factor s hgh enough, no player has any ncentve to devate f we are along the equlbrum path. Ths s because followng any devaton to a 6= a ; the dstrbuton q j a ; a wll assgn postve probablty to all sgnal pro les and n partcular to pro les where not all sgnals agree. Thus any devaton wll trgger punshment wth postve probablty and even by optmally talorng her report followng a devaton, a player cannot reduce ths to zero. Provded the dscount factor s hgh enough, no devaton wll be pro table and all the ncentves can be made strct. Thus we have argued that f q s perfectly nformatve and there s a hgh enough dscount factor so that the repeated game wth communcaton has an equlbrum whch s fully e cent. Now suppose that q s nearly nformatve n the sense that t s close to a perfectly nformatve dstrbuton q 0. Then by contnuty we obtan Proposton 5. Fx fq ( j a) : a 6= a g : There exsts a such that for all > and for all perfectly nformatve q 0 (Y ) ; there exsts a such that f kq 0 q k T V < ; there s a nearly e cent equlbrum of the game wth communcaton. Note the dstrbutons q ( j a) for a 6= a are xed and t s only q ( j a ) that s requred to be nearly nformatve. 0 Some shortcomngs of the equlbrum The equlbrum constructed above, however, has some shortcomngs. Frst, wth the suggested strateges, the lkelhood that cooperaton may break down even though no one has devated may be substantal players wll revert to noncooperaton whenever there s any dscrepancy n the reported sgnals. Such dscrepances wll occur, wth probablty ; even f all players have conformed to the equlbrum strategy. Second, the strateges guarantee truthtellng only when kq 0 q k T V s small, that s, when q s very close to beng perfectly nformatve. The rst de cency s partcularly acute when the number of sgnals or the number of players s large.. To see ths n the case of a large number of sgnals, suppose that there are two players and suppose, as a lmtng case, that there s a contnuum of sgnals n [0; ]. The probablty that the sgnals of the two players are the same s clearly zero and so the probablty of contnung cooperaton s zero as well. A dscrete approxmaton to the contnuous dstrbuton wll have the property 0 Ths means that settng we study s not one wth "almost-publc sgnals" as n Malath and Morrs (00) or Hörner and Olszewsk (009).