Folk Theorem in Repeated Games with Private Monitoring

Transcription

1 Folk Theorem n Repeated Games wth Prvate Montorng Takuo Sugaya y Prnceton Unversty November 15, 2011 The latest verson and the onlne Supplemental Materals are avalable at Abstract We show that the folk theorem genercally holds for N-player repeated games wth prvate montorng when each player s number of sgnals s su cently large. Nether cheap talk communcaton nor publc randomzaton s necessary. Journal of Economc Lterature Class caton Numbers: C72, C73, D82 Keywords: repeated game, folk theorem, prvate montorng tsugaya@prnceton.edu y I thank Stephen Morrs for nvaluable advce, Yuly Sannkov for helpful dscusson, and Dlp Abreu, Eduardo Fangold, Drew Fudenberg, Edoardo Grllo, Johannes Hörner, Yuhta Ish, Mchhro Kandor, Vjay Krshna, George Malath, Mha Manea, Larry Samuelson, Tadash Sekguch, Andrzej Skrzypacz, Satoru Takahash, Yuch Yamamoto and semnar partcpants at 10th SAET conference, 10th World Congress of the Econometrc Socety, 22nd Stony Brook Game Theory Festval and Summer Workshop on Economc Theory for useful comments. The remanng errors are my own responsblty. 1

2 1 Introducton One of the key results n the lterature on n ntely repeated games s the folk theorem: Any feasble and ndvdually ratonal payo can be sustaned n equlbrum when players are su cently patent. Even f a stage game does not have an e cent Nash equlbrum, the repeated game does. Hence, the repeated game gves a formal framework to analyze a cooperatve behavor. Fudenberg and Maskn (1986) establsh the folk theorem under perfect montorng, that s, when players can drectly observe the acton pro le. Fudenberg, Levne and Maskn (1994) extend the folk theorem to mperfect publc montorng, where players can observe only publc nosy sgnals about the acton pro le. Recent papers by Hörner and Olszewsk (2006 and 2009) show that the folk theorem holds n prvate montorng, where players can observe only prvate nosy sgnals about the acton pro le, f the montorng s almost perfect and almost publc, respectvely. The drvng force of the folk theorem n perfect or publc montorng s the coordnaton of future play based on common knowledge of relevant hstores. Spec cally, the publc component of hstores, such as acton pro les n perfect montorng or publc sgnals n publc montorng, reveals past acton pro les (at least statstcally). Snce ths publc nformaton s common knowledge, players can coordnate a punshment contngent on the publc nformaton, and thereby provde dynamc ncentves to choose actons that are not statc best responses. Hörner and Olszewsk (2006 and 2009) show the robustness of ths coordnaton to the lmted classes of prvate montorng. If montorng s almost perfect, then players can beleve that every player observes the same sgnal correspondng to the acton pro le wth hgh probablty. If montorng s almost publc, then players can beleve that every player observes the same sgnal wth hgh probablty. 1 Hence, almost common knowledge about relevant hstores stll exsts. However, wth general prvate montorng, almost common knowledge may not exst and 1 See also Malath and Morrs (2002 and 2006). 2

3 coordnaton s d cult (we call ths problem coordnaton falure ). 2 Hence, the robustness of the folk theorem to general prvate montorng has been an open queston. For example, Kandor (2002) states that [t]hs s probably one of the best known long-standng open questons n economc theory. 3 Ths paper s, to the best of our knowledge, the rst to show that the folk theorem holds n repeated games wth dscountng and generc prvate montorng: In any N-player repeated game wth prvate montorng, f each player s number of sgnals s su cently large, then any feasble and ndvdually ratonal payo s sustanable n a sequental equlbrum for a su cently large dscount factor. 4 Repeated games wth prvate montorng are relevant for many tradtonal economc problems. For example, Stgler (1964) proposes a repeated prce-settng olgopoly, where rms set ther own prce n a face-to-face negotaton and cannot drectly observe ther opponents prces. Instead, a rm obtans some nformaton about opponents prces through ts own sales. Snce the level of sales depends on both opponents prces and unobservable shocks due to busness cycles, the sales level s an mperfect sgnal. Moreover, each rm s sales level s often prvate nformaton. Thus, the montorng s mperfect and prvate. In prncpalagent problems, f the prncpal evaluates the agent subjectvely, then the montorng by the prncpal about the agent becomes prvate. Despte the mportance of these problems, only a lmted number of papers successfully analyze the repeated games wth prvate montorng. 5 Our result o ers a benchmark to analyze these problems n a general prvate-montorng settng. To show the folk theorem under general prvate montorng, we unfy and mprove on three approaches n the lterature on prvate montorng that have been used to show the 2 Malath and Morrs (2002 and 2006) and Sugaya and Takahash (2011) o er the formal models of ths argument. 3 See Malath and Samuelson (2006) for a survey. 4 See Lehrer (1990) for the case of no dscountng. 5 Harrngton and Skrzypacz (2011) show evdence of cooperatve behavor (cartels) among rms n lysne and vtamn ndustres. After argung that these ndustres t Stgler s setup, they wrte a repeated-game model wth prvate montorng and solve a specal case. See also Harrngton and Skrzypacz (2007). Fuchs (2007) apples a repeated game wth prvate montorng to a contract between a prncpal and an agent wth subjectve evaluaton. 3

4 partal results so far: Belef-free, belef-based and communcaton approaches. The belef-free approach (and ts generalzatons) has been successful n showng the folk theorem n the prsoners dlemma. 6 A strategy pro le s belef-free f, for any hstory pro le, the contnuaton strategy of each player s optmal condtonal on the hstory of the opponents. Hence, coordnaton falure never happens. Wth almost perfect montorng, Pccone (2002) and Ely and Välmäk (2002) show the folk theorem for the two-player prsoners dlemma. 7 Wthout any assumpton on the precson of montorng but wth condtonally ndependent montorng, Matsushma (2004) obtans the folk theorem n the two-player prsoners dlemma, whch s extended by Yamamoto (2011) to the N-player prsoners dlemma wth condtonally ndependent montorng. 8 Prevously, attempts to generalze Matsushma (2004) have shown only lmted results wthout almost perfect or condtonally ndependent montorng: For some restrcted classes of the dstrbutons of prvate sgnals, Fong, Gossner, Hörner and Sannkov (2010) show that the payo of the mutual cooperaton s approxmately attanable and Sugaya (2010a) shows the folk theorem n the two-player prsoners dlemma. Sugaya (2010b) shows that the folk theorem holds wth a general montorng structure n the prsoners dlemma f the number of players s no less than four. Several papers construct belef-based equlbra, where players strateges nvolve statstcal nference about the opponents past hstores. That s, snce common knowledge about relevant hstores no longer exsts, each player calculates the belefs about the opponents hstores to calculate best responses. Wth almost perfect montorng, Sekguch (1997) 6 Kandor and Obara (2006) use a smlar concept to analyze a prvate strategy n publc montorng. Kandor (2010) consders weakly belef-free equlbra, whch s a generalzaton of belef-free equlbra. Apart from a typcal repeated-game settng, Takahash (2010) and Deb (2011) consder the communty enforcement and Myagawa, Myahara and Sekguch (2008) consder the stuaton where a player can mprove the precson of montorng by payng cost. 7 See Yamamoto (2007) for the N-player prsoners dlemma. Ely, Hörner and Olszewsk (2004 and 2005) and Yamamoto (2009) characterze the set of belef-free equlbrum payo s for a general game. Except for the prsoners dlemma, ths set s not so large as that of feasble and ndvdually ratonal payo s. 8 The strategy used n Matsushma (2004) s called a belef-free revew strategy. See Yamamoto (2011) for the characterzaton of the set of belef-free revew-strategy equlbrum payo s for a general game wth condtonal ndependence. Agan, except for the prsoners dlemma, ths set s not so large as that of feasble and ndvdually ratonal payo s. 4

5 shows that the payo of the mutual cooperaton s approxmately attanable and Bhaskar and Obara (2002) show the folk theorem n the two-player prsoners dlemma. 9 Phelan and Skrzypacz (2011) characterze the set of possble belefs about opponents states n a ntestate automaton strategy and Kandor and Obara (2010) o er a way to verfy f a nte-state automaton strategy s an equlbrum. Another approach to analyze repeated games wth prvate montorng ntroduces publc communcaton. Folk theorems have been proven by Compte (1998), Kandor and Matsushma (1998), Aoyag (2002), Fudenberg and Levne (2002) and Obara (2009). Introducng a publc element (the result of communcaton) and lettng a strategy depend only on the publc element allow these papers to sdestep the d culty of coordnaton through prvate sgnals. However, the analyses are not applcable to settngs where communcaton s not allowed: For example, n Stgler (1964) s olgopoly example, ant-trust laws prohbt communcaton. Hörner and Olszewsk (2006) argue that communcaton rentroduces an element of publc nformaton that s somewhat at odds wth the motvaton of prvate montorng as a robustness test to the lack of common knowledge. Ths paper ncorporates all three approaches. Frst, the equlbrum strategy to show the folk theorem s phase-belef-free. That s, we see the repeated game as the repetton of long revew phases. Each player has two strateges for the revew phase; one that s generous to the opponent and another that s harsh to the opponent. 10 At the begnnng of each revew phase, for each player, both generous and harsh strateges are optmal condtonal on any realzaton of the opponents hstory. Wthn each revew phase, each player can change the opponent s contnuaton payo from the next revew phase by changng the transton probablty between the two strateges, wthout consderng the other players hstory. Ths equlbrum s mmune to coordnaton falure at the begnnng of each phase and gves us freedom to control the contnuaton payo s. Second, however, the belef-free property does not hold except at the begnnng of the 9 Bhaskar and Obara (2002) also derve a su cent condton for the N-player prsoners dlemma. 10 As wll be seen n Secton 5, for a game wth more than two players, one of player s strateges s generous to player + 1 and another of player s strateges s harsh to player + 1. In addton, players (; + 1) s payo s are constant regardless of whch strategy player pcks from the two. 5

6 phases. Hence, we consder each player s statstcal nference about the opponents past hstores as n the belef-based approach wthn each phase. Fnally, n our equlbrum, to coordnate on the play n the mddle of the phase, the players do communcate but the message exchange s done wth ther actons. The d culty to replace cheap talk wth messages va actons s that, snce the players need to nfer the opponents messages from ther prvate sgnals, common knowledge about the past messages no longer exsts. One of our methodologcal contrbutons s to o er a systematc way to replace the cheap talk wth message exchange va actons n prvate montorng by overcomng the lack of common knowledge. The rest of the paper s organzed as follows: Secton 2 ntroduces the model and Secton 3 states the assumptons and man result. Secton 4 o ers the overvew of the structure of the proof. Secton 5 relates the n ntely repeated game to a ntely repeated game wth an auxlary scenaro (reward functon) and derves a su cent condton on the ntely repeated game to show the folk theorem n the n ntely repeated game. The remanng parts of the paper are devoted to the proof of the su cent condton. Secton 6 explans the basc structure of the ntely repeated game. As wll be seen n Secton 7, we concentrate on the approxmate equlbrum untl Secton 13. Snce the complete proof s long and complcated, for the rest of the man text (that s, from Secton 8 to Secton 15), we concentrate on a specal case explaned n Secton 8 to llustrate the key structure. Namely, we focus on the twoplayer prsoners dlemma wth cheap talk and publc randomzaton, and nterested readers may refer to the Supplemental Materals for the complete proof for a general game wthout cheap talk or publc randomzaton. Secton 9 spec es what assumptons are su cent n ths specal case. After we formally de ne the structure of the ntely repeated game for the two-player prsoners dlemma n Secton 10, we de ne the strategy n Secton 11. Whle de nng the strategy, we de ne many varables. Secton 12 ver es that we take all the varables coherently. Secton 13 shows that the strategy approxmately sats es the su cent condton derved n Secton 5. Fnally, Secton 14 adjusts the strategy further so that t exactly sats es the su cent condton (therefore, we are not consderng an approxmate 6

7 equlbrum. The nal strategy s an exact sequental equlbrum). All proofs are gven n the Appendx (Secton 15). Sectons from 16 to 53 are n the Supplemental Materals. 2 Model 2.1 Stage Game The stage game s gven by I; fa ; Y ; U g 2I ; q. I = f1; : : : ; Ng s the set of players, A wth ja j 2 s the nte set of player s pure actons, Y s the nte set of player s prvate sgnals, and U s the nte set of player s ex-post utltes. Let A Q 2I A, Y Q 2I Y and U Q 2I U be the set of acton pro les, sgnal pro les and ex post utlty pro les, respectvely. In every stage game, player chooses an acton a 2 A, whch nduces an acton pro le a (a 1 ; : : : ; a N ) 2 A. Then, a sgnal pro le y (y 1 ; : : : ; y N ) 2 Y and an ex post utlty pro le ~u (~u 1 ; : : : ; ~u N ) 2 U are realzed accordng to a jont condtonal probablty functon q (y; ~u j a). Followng the conventon n the lterature, we assume that ~u s a determnstc functon of a and y so that observng the ex post utlty does not gve any further nformaton than (a ; y ). If ths were not the case, then we could see a par of a sgnal and an ex post utlty, (y ; ~u ), as a new sgnal. Player s expected payo from a 2 A s the ex ante value of ~u gven a and s denoted u (a). For each a 2 A, let u (a) represent the payo vector (u (a)) 2I. 2.2 Repeated Game Consder the n ntely repeated game of the above stage game n whch the (common) dscount factor s 2 (0; 1). Let a ; and y ;, respectvely, denote the acton played and the prvate sgnal observed n perod by player. Player s prvate hstory up to perod t 1 s gven by h t fa ; ; y ; g t 1 =1. Wth h1 = f;g, for each t 1, let H t be the set of all h t. A 7

8 S strategy for player s de ned to be a mappng : 1 H t! 4(A ). Let be the set of all strateges for player. Fnally, let E() be the set of sequental equlbrum payo s wth a common dscount factor. t=1 3 Assumptons and Result In ths secton, we state two assumptons and the man result (folk theorem). Frst, we state an assumpton on the payo structure. Let F co(fu(a)g a2a ) be the set of feasble payo s. The mnmax payo for player s v mn 2 j6= (A j ) max u (a ; ): a 2A Then, the set of feasble and ndvdually ratonal payo s s gven by F fv 2 F : v v for all g. We assume the full dmensonalty of F. Assumpton 1 The stage game payo structure sats es the full dmensonalty condton: dm(f ) = N. Second, we state an assumpton on the sgnal structure. Assumpton 2 Each player s number of sgnals s su cently large: For any 2 I, we have jy j 2 X j2i ja j j : Under these assumptons, we can genercally construct an equlbrum to attan any pont n nt(f ). Theorem 1 If Assumptons 1 and 2 are sats ed, then the folk theorem genercally holds: For generc q ( j ), for any v 2 nt(f ), there exsts < 1 such that, for all >, v 2 E (). 8

9 See Secton 9 and the Supplemental Materal 1 for exactly what genercty condtons we need n the proof. As wll be seen, Assumpton 2 s more than necessary. What we need for the proof s 8 >< 8 jy j < max : >: ja j + 2 ja f N = 2; ja j + ja +1 j P j6=;+1 (ja jj 1) ; ja 1 j + P j6= 1; (ja jj 1) ; n 1 2 ja 1 j ; max j2i ja 2 jj + P o n n6=j ja 1 nj ; max j2i ja 2 jj + 2 P o n6=;j ja nj f N 3: 1 j 9 = ; From now on, we arbtrarly x v 2 nt(f ) and construct an equlbrum to support v n a sequental equlbrum. 4 An Overvew of the Argument Ths secton provdes some ntuton for our constructon. Followng Hörner and Olszewsk (2006), we see a repeated game as repetton of T P -perod revew phases. T P wll be formally de ned later. In Secton 4.1, we explan that our equlbrum s phase-belef-free and how t makes our equlbrum mmune to coordnaton falure at the begnnng of each phase. Secton 4.2 o ers the basc structure of the revew phase. To explan the detals of the revew phase, t s useful to consder a specal case where addtonal communcaton devces are avalable. Secton 4.3 ntroduces these devces. Wth these communcaton devces, n Sectons 4.4, 4.5 and 4.6, we o er the detaled explanaton of the revew phase. Fnally, we explan how to dspense wth the communcaton devces n Secton Phase-Belef-Free As Hörner and Olszewsk (2006), the equlbrum s phase-belef-free. Each player has two T P -perod- ntely-repeated-game strateges, denoted (G) and (B). At the begnnng of 9

10 each revew phase, for each player, ndependently of her hstory, any contnuaton strategy that adheres to one of the two strateges (G) and (B) n the revew phase s optmal. We say that player takng (x ) wth x 2 fg; Bg n the revew phase s n state x 2 fg; Bg. Intutvely speakng, (G) s a generous strategy that gves a hgh payo to player + 1 (mod N) who takes ether +1 (G) or +1 (B), regardless of the other players state pro le x (;+1) 2 fg; Bg N 2. On the other hand, (B) s a harsh strategy that gves a low payo to player + 1 regardless of player ( + 1) s strategy (ncludng those d erent from +1 (G) and +1 (B)) and x (;+1). Hence, player ( 1) s strategy controls player s value regardless of x ( 1), replacng wth 1 n the prevous two sentences. Snce these two strateges are optmal at the begnnng of the next phase, t s up to player 1 whether player 1 wll take 1 (G) or 1 (B) n the next phase. Therefore, player 1 wth 1 (G) n the current phase can freely reduce player s contnuaton payo from the next revew phase by transtng to 1 (B) wth hgher probablty whle player 1 wth 1 (B) can freely ncrease player s contnuaton payo by transtng to 1 (G) wth hgher probablty. 11 In summary, we do not need to consder player ( 1) s ncentve to punsh player after a bad hstory n state G or to reward player after a good hstory n state B. 4.2 Structure of the Revew Phase The basc structure of the revew phase s summarzed as follows. At the begnnng of the revew phase, the players communcate a state pro le x 2 fg; Bg N. Ths communcaton stage s named the coordnaton block snce the players try to coordnate on x. The detals wll be explaned n Secton 4.4. Based on the result of the coordnaton block, the players play the ntely repeated 11 Here, the changes n the contnuaton payo s are measured by the d erences between player s ex ante value gven x 1 at the begnnng of the revew phase and the ex post value at the end of the revew phase after player 1 observes the hstory n the phase. See Secton 5 for the formal de nton. For example, f player 1 wth x 1 = G does not reduce player s contnuaton value, then t means that the state of player 1 n the next revew phase s G wth probablty one, so that the ex post value s the same as the ex ante value. 10

11 game for many perods. Ths step conssts of multple revew rounds. The detals wll be explaned n Secton 4.6. Fnally, at the end of the phase, the players communcate the hstory n the coordnaton block and revew rounds. Ths stage s named the report block snce the players report the hstory n the revew rounds. The role of ths communcaton wll be explaned n Secton Specal Communcaton Devces Before explanng the detals of the coordnaton block, revew rounds and report block, we ntroduce three specal communcaton devces. We wll dspense wth all three n Secton 4.7. Perfect Cheap Talk Untl Secton 4.6, we assume that the players could drectly communcate n the coordnaton block and report block. We assume that the communcaton were () cheap (not drectly payo -relevant), () nstantaneous and () publc and perfect (t generates the same sgnal as the message to each player). Nosy Cheap Talk In the revew rounds, we assume that the players could drectly communcate va nosy cheap talk. We wll later explan why we use nosy cheap talk rather than the perfect cheap talk n the revew rounds. Nosy cheap talk wth precson p 2 (0; 1) s the communcaton devce that s () cheap and () nstantaneous, but () prvate and mprecse wth probablty exp( O(T p )). 12 Spec cally, when the sender (say player j) sends a bnary message m 2 fg; Bg va nosy cheap talk, the recever (say player ) wll observe a bnary prvate sgnal f[](m) 2 fg; Bg. Wth hgh probablty, the message transmts correctly: f[](m) = m wth probablty 1 exp( O(T p )). Gven the true message m and the recever s prvate sgnal f[](m), the controller of the recever s payo (player 1) stochastcally receves a bnary prvate sgnal 12 In general, when we say y = O(x), t means that there exsts k > 0 such that y = kx. 11

12 g[ 1](m) 2 fm; Eg. If f[](m) 6= m (f the recever receves a wrong sgnal), then g[ 1](m) = E wth probablty 1 exp( O(T p )). That s, g[ 1](m) = E mples that player 1 (the controller of player s payo ) suspects that the communcaton may have an error. Further, we assume that any sgnal par can occur wth probablty at least exp( O(T p )). Hence, the communcaton s nosy. We assume that the sgnals are prvate. Therefore, f[](m) s observable only to the recever (player ) and g[ 1](m) s observable only to the controller of the recever s payo (player 1). There are two mportant features of ths nosy cheap talk: Frst, whenever the recever realzes that her sgnal was wrong: f[](m) 6= m, then she puts a belef no less than 1 exp( O(T p )) on the event that the controller of her payo should have receved the sgnal g[ 1](m) = E and realzed there was an error. 13 Second, any error occurs wth postve probablty exp( O(T p )). It wll be clear n Secton 4.6 that these two features are mportant to construct an equlbrum n the revew rounds. Publc Randomzaton In the report block, we assume that publc randomzaton were avalable n addton to the perfect cheap talk. Wth these specal communcaton devces, Sectons 4.4, 4.5 and 4.6 explan the coordnaton block, the report block and the revew rounds, respectvely. 4.4 Coordnaton Block The role of the coordnaton block s to coordnate on x as n Hörner and Olszewsk (2006). Wth the perfect cheap talk, each player tells the truth about her own state x and the state pro le x 2 fg; Bg N becomes common knowledge. In the revew rounds, based on x, the players play a(x) wth hgh probablty on the equlbrum path. Intutvely, a(x) s the acton pro le taken n the regular hstores when the state pro le s x. See Secton 5 for the formal de nton of a(x). 13 As we wll see, player ( 1) s contnuaton play s ndependent of g[ 1](m) and so player cannot learn g[ 1](m). 12

13 4.5 Report Block We ntroduce the report block where the players communcate the hstory n the coordnaton block and revew rounds. Ths communcaton enables us to concentrate on "-equlbrum untl the end of the last revew round. Suppose that we have constructed a strategy pro le whch s "-equlbrum at the end of the last revew round f we neglect the report block. We explan how to attan the exact equlbrum by usng the report block. As seen n Secton 4.3, suppose that the perfect cheap talk and publc randomzaton are avalable. Each player s pcked by the publc randomzaton wth probablty 1 N.14 pcked player sends the whole hstory n the coordnaton block and revew rounds (denoted h man ) to player 1. That s, h man revew phase to the end of the last revew round. The s player s hstory from the begnnng of the current Assume that player always tells the truth about h man. Player 1 changes the contnuaton payo of player such that, after any perod t n the coordnaton block and revew rounds, after any hstory h t, t s exactly optmal for player to follow the prescrbed acton by (x ). Snce the orgnal strategy pro le was "-equlbrum wth arbtrarly small ", ths can be done by slghtly changng the contnuaton strategy based on h man 1 and h man. 15 The remanng task wth the perfect cheap talk and publc randomzaton s to show the ncentve to tell the truth about h man. Intutvely, wth de nng a lnear space and norm prop- h erly for the hstores, player 1 punshes player proportonally to h man 1 E h man man 2 1 j ^h wth ^h man ^h man beng the reported hstory. The optmal report to mnmze the expected h h man man 2 1 j ^h j h man s to tell the truth: ^hman = h man. 16 punshment E h man 1 E Snce the adjustment for exact optmalty s small, the small punshment s enough to ncentvze player to tell the truth. Therefore, the total changes n the contnuaton payo based on the report block do not a ect the equlbrum payo. 14 For N 3, the precse procedure s slghtly d erent. See Secton 36 n the Supplemental Materal Wth more than two players, player 1 also needs to know the hstores of players ( 1; ). So that players ( 1; ) can send ther hstores to player 1, we ntroduce another communcaton stage after the report block, named the re-report block. Snce ths nformaton sent by players ( 1; ) n the re-report block s used only to control player s contnuaton payo, the truthtellng ncentve for players ( 1; ) s trvally sats ed. See Secton 37 n the Supplemental Materal Note that ths logc s the same as we show the consstency of generalzed-method-of-moments estmators. 13

14 4.6 Revew Rounds Between the coordnaton block and the report block, the players play a T -perod revew round for L tmes. Here, L 2 N s a xed nteger that wll be determned n Secton 12, and T = (1 ) 1 2 so that T! 1 and LT! 1 as! 1: (1) Throughout the paper, we neglect the nteger problem snce t s handled by replacng each varable s that should be an nteger wth mn n2n n. ns The reason why we have T perods n each revew round s to aggregate prvate sgnals for many perods to get precse nformaton as n Matsushma (2004). 17 There are two reasons why we have L revew rounds. The rst reason s new: As we wll explan, the sgnals of the players can be correlated whle Matsushma (2004) assumes that the sgnals are condtonally ndependent. To deal wth correlaton, we need multple revew rounds. The second reason s the same as Hörner and Olszewsk (2006). If we replace each perod of Hörner and Olszewsk (2006) wth a T -perod revew round, then we need a su cently large number of revew rounds so that a devator should be punshed su cently long to cancel out the gans n the nstantaneous utlty from devaton. Below, we o er a more detaled explanaton of the revew rounds. In Secton 4.6.1, we concentrate on the rst role of the L rounds. That s, we consder the case where the block of Hörner and Olszewsk (2006) has one perod, that s, the stage game s the two-player prsoners dlemma. We wll explan a general two-player game and a general more-than-twoplayer game n Sectons and 4.6.3, respectvely, where the second role of the L rounds s mportant. Whenever we consder the two-player case and we say players and j, we assume that player j s player s (unque) opponent unless otherwse spec ed. 17 See also Radner (1985) and Abreu, Mlgrom and Pearce (1991). 14

15 4.6.1 The Two-Player Prsoners Dlemma In the two-player prsoners dlemma, we consder player s ncentve to take (G) when player j takes j (G). The other combnatons of (x ; x j ) are symmetrc. Remember that snce x s communcated va perfect cheap talk, x s common knowledge. So that (G) s generous to player j, player needs to take cooperaton wth ex ante hgh probablty. On the other hand, player j can reduce player s contnuaton payo from the next revew phase based on her hstory wthn the current revew phase (see the explanaton of phase-belef-free n Secton 4.1). Suppose that player j has a good random varable (sgnal) whch occurs wth probablty q 2 when player takes cooperaton and wth probablty q 1 < q 2 when player takes defecton. q 2 > 0 can be very small snce the montorng s mperfect. Assume that the nstantaneous utlty gan of takng defecton nstead of cooperaton s g > 0. If player j needs to ncentvze player to take cooperaton every perod ndependently, then player j needs to reduce player s contnuaton payo by at least g q 2 q 1 (for smplcty, forget about dscountng) after not observng the good sgnal. Then, the ex ante per-perod reducton of the contnuaton payo s g q 2 q 1 (1 q 2 ), whch s too large to attan e cency (f q 2 s bounded away from one). That s, player j swtches to the harsh strategy (whch takes defecton n the prsoners dlemma) from the next revew phase too often. Hence, we need to come up wth a procedure to prevent the ne cent punshment (reducton of the contnuaton payo ). Condtonal Independence Followng Matsushma (2004), assume that player s sgnals were ndependent of player j s sgnals. In ths case, we could see a collecton of L revew rounds as one long revew round. That s, player j montors player for LT perods. Player j wll take the generous strategy wth probablty one n the next revew phase f the good sgnal s observed (q 2 + 2") LT tmes or more. 18 If t s observed less, then player j reduces the contnuaton payo by the shortage multpled by q 2 g q 1. That s, wth X j beng how 18 We wll explan why we use 2" nstead of " later. In addton, ths " s d erent from " for "-equlbrum. 15

16 many tmes player j has observed the good sgnal n the LT perods, the reducton of the contnuaton payo wll be player. g q 2 q 1 f(q 2 + 2") LT X j g We call X j player j s score about Snce player s sgnals were ndependent of player j s sgnals, player could not update any nformaton about player j s score about player from player s prvate sgnals. Hence, by the law of large numbers, for su cently large T, player beleves that (q 2 + 2") LT X j > 0 wth ex post hgh probablty after any hstory. Hence, t s optmal for player to constantly take cooperaton. At the same tme, snce the expected value of X j s q 2 LT, the ex ante per-perod reducton of the contnuaton payo s takng " small. Therefore, we are done. q 2 g q 1 2", whch can be arbtrarly small by Condtonal Dependence Now, we dspense wth condtonal ndependence. That s, player s sgnals and player j s sgnals can be correlated arbtrarly. Intutvely, see one perod as a day and a long revew round as a year: LT = 365. Snce the expected score s q 2 LT, to prevent an ne cent punshment, player j cannot punsh player after the score slghtly exceeds q 2 LT (n the above example, (q 2 + 2") LT ). On the other hand, f the sgnals are correlated, then later n a year (say, November), t happens wth a postve probablty that player beleves that, judgng from her own sgnals and correlaton, player j s score about player has been much more than q 2 LT already (n the above example, more than (q 2 + 2") LT ). Then, player wants to start to defect. More generally, t s mpossble to create a punshment schedule that s approxmately e cent and that at the same tme ncentvzes player to cooperate after any hstory wth arbtrary correlaton. Hence, we need to let player s ncentve to cooperate break down after some hstory. Symmetrcally, player j also swtches her own acton after some hstory. Intutvely, player swtches to defecton f player s expectaton of player j s score about player s much hgher than the ex ante mean. We want to specfy exactly when each player takes defecton based on player s expectaton of player j s score about player. 19 fxg + s equal to X f X 0 and 0 otherwse. 16

17 Chan of Learnng However, ths creates the followng problem: Snce player swtches her acton based on player s expectaton of player j s score about player, player s acton reveals player s expectaton of player j s score about player. Snce both player s expectaton of player j s score about player and player s score about player j are calculated from player s hstory, player j may want to learn player s expectaton of player j s score about player from player j s sgnals about player s acton. If so, player j s decson of actons depends also on player j s expectaton of player s expectaton of player j s score about player. Proceedng one step further, player s decson of actons depends on player s expectaton of player j s expectaton of player s expectaton of player j s score about player. Ths chan contnues n ntely. Nosy Cheap Talk Cuts o the Chan of Learnng We want to construct an equlbrum that s not destroyed by the chan of hgh order expectatons. From the dscusson of the report block, we can focus on "-equlbrum. Ths means that, to verfy an equlbrum, t s enough to show that each player beleves that her acton s strctly optmal or any acton s optmal wth hgh probablty (not probablty one). To prevent the chan of learnng, we take advantage of ths " slack n "-equlbrum and the nose n the nosy cheap talk explaned n Secton 4.3. The basc structure s as follows. We dvde an LT -perod long revew round nto L T -perod revew rounds. We make sure that each player takes a constant acton wthn a revew round. If player j observes a lot of good sgnals n a revew round, then player should take defecton from the next revew round. At the end of each revew round, player j sends a nosy cheap talk message wth precson p = 1 to nform player of the optmal 2 acton n the next revew round. Based on player s own hstory and player s sgnal of player j s message va nosy cheap talk, player may swtch to a constant defecton from the next revew round. That s, the breakdown of ncentves and swtches of actons occur only at the begnnng of each revew round. The remanng questons are () how we can ncentvze player j to tell the truth and () how we can make sure that the chan of learnng 17

18 does not destroy an equlbrum. The ntutve answer to these questons are as follows. In equlbrum, by the law of large numbers, wth ex ante hgh probablty, player at the end of the revew round puts a hgh belef on the event that player j has not observed a lot of good sgnals. In such a case, player beleves that player s optmal acton n the next revew round s cooperaton and dsregards the sgnal of player j s message. That s, the precson of player s nference about player s optmal acton from the revew rounds s usually 1 exp( O(T )) because the length of the revew round s T. Snce ths s hgher than the precson of the sgnal of the nosy cheap talk, 1 exp( O(T 1 2 )), player dsregards the sgnal. Player ncentvzes player j to tell the truth by changng player j s contnuaton payo from the next revew phase only f player does not dsregard the message. Snce player does not dsregard the message only after rare hstores, ncentvzng player j does not a ect e cency. Ths answers queston (). The answer to queston () s as follows: Consder the case where player obeys the sgnal of player j s message and player learns from player j s contnuaton play that player s sgnal of player j s message was wrong. Even after realzng an error, player keeps obeyng the sgnal by the followng reasons: By the de nton of the nosy cheap talk n Secton 4.3, player beleves that player j should have receved E and should have realzed that player s sgnal was wrong. Snce player j s contnuaton play never reveals whether player j receved E or not, player keeps ths belef. As wll be seen, player j after observng E makes player nd erent between any acton pro le. Therefore, t s almost optmal for player to keep obeyng the sgnal. Next, consder the case where player dsregards the sgnal of player j s message and player learns from player j s contnuaton play that player j s acton s d erent from what s consstent wth player s expectaton of player j s score about player and player s message. For example, player sent the message that player j should swtch to defecton but realzes that player j s stll cooperatng. Ths means that, f player s message had transmtted correctly, then n order for player j to keep cooperatng, player j s hstory should 18

19 have told player j that player has not observed a lot of good sgnals about player j yet (that s, player j s expectaton of player s score about player j s low). What f player j s expectaton of player s good sgnal about player j and player j s good sgnal about player are negatvely correlated and ths mples that player j should have observed a lot of good sgnals about player? Does player want to swtch to defecton? The answer s no. Snce player s message dd not transmt correctly wth probablty exp( O(T 1 2 )), player always attrbutes the nconsstency between player j s acton and player s expectaton of player j s acton to the error n player j s sgnal of player s nosy message, rather than the mstake n player s nference. 20 We wll de ne an equlbrum strategy more fully to answer the questons () and () formally. Full Explanaton of the Strategy For each lth revew round, let X j (l) be player j s score about player n the lth revew round, whch denotes how many tmes player j observes the good sgnal n the lth revew round. In each lth revew round, f X j ( ~ l) (q 2 + 2") T for all ~ l l 1, that s, f player j s score about player has not been erroneously hgh n the prevous revew rounds, then player j montors player by player j s score about player. That s, the reducton of the contnuaton payo from the next revew phase 21 s g q 2 q 1 ((q 2 + 2") T X j (l)). Note that ths s proportonal to except that ths ncreases wthout an upper bound wthn a revew round. On the other hand, f X j ( ~ l) > (q 2 + 2") T happens for some ~ l l caused by the lth revew round g q 2 q 1 f(q 2 + 2") LT X j g + 1, that s, f player j s score about player has been erroneously hgh n one of the prevous revew rounds, then player j stops montorng. That s, the reducton of player s contnuaton payo from the next revew phase caused by the lth revew round s xed at gt + for how we determne ths number. q 2 g q 1 2"T. See below 20 See 1-(b) and 2 below to make sure that after any hstory, there s a postve probablty that player j obeys the sgnal of player s message. 21 Note that ths s not a next revew round. 19

20 For notatonal convenence, let j (l) = G denote the stuaton that player j montors player n the lth revew round and let j (l) = B denote the stuaton that player j stops montorng n the lth revew round. X j ( ~ l) (q 2 + 2") T for all ~ l l 1. The total reducton of the contnuaton payo s g q 2 q 1 T + LX l=1 1 f j (l) = Gg In general, 1 fxg s an ndex functon such that That s, j (1) = G and j (l) = G f and only f g ((q 2 + 2") T X j (l)) + 1 f j (l) = Bg q 2 q 1 8 < 1 f X s true, 1 fxg = : 0 f X s not true. Three remarks: Frst, we have a constant term q 2 g g T + g 2"T q 2 q 1 q 2 q 1 q 1 T. Note that the maxmum score X j (l) for one round s T. Snce the ncrement of the decrease n the reducton of the contnuaton payo s q 2 g q 1, ths constant term s su cent to cover the maxmum decrease of the reducton of the contnuaton payo for one revew round. Second, after (q 2 + 2") T X j (l) < 0, that s, after player j s score about player becomes erroneously hgh, n the followng revew g rounds, we have a constant postve reducton q 2 q 1 T + g q 2 q 1 2"T. Thrd, from the rst and second remarks, the total reducton n the contnuaton payo at the begnnng of the next revew phase s always postve. Ths mples that we can nd a transton probablty for player j s state n the next revew phase to acheve ths reducton of the contnuaton payo. If t were negatve, then player j would need to transt to a bad strategy wth a negatve probablty, whch s nfeasble. Consder player s ncentve. If player could know j (l), then player wants to take cooperaton (defecton, respectvely) constantly n the lth revew round f j (l) = G ( j (l) = B, respectvely). Verfy ths by backward nducton: In the last Lth revew round, ths s true snce the decrease n the reducton of the contnuaton payo s always g q 2 q 1 (0, respectvely) for an addtonal observaton of the good sgnal f j (L) = G ( j (L) = B, respectvely). : 20

21 Note that wth ths optmal strategy, player s payo from the Lth revew round (the nstantaneous utltes n the Lth revew round and the reducton of the contnuaton payo caused by the Lth revew round) s equal to u (C; C) j plays cooperaton. 22 q 2 g q 1 2"T regardless of j (L) f player That s, the reducton of the contnuaton payo after j (L) = B s determned so that player s payo s the same between j (L) = G and j (L) = B. Therefore, when we consder the (L the strategy n the (L 1)th revew round, player can neglect the e ect of 1)th revew round on the payo n the Lth revew round. Hence, the same argument establshes the result for the (L untl the rst revew round by backward nducton. 1)th revew round. We can proceed Snce player cannot observe j (l + 1) drectly, after the lth revew round, player wants to know whether j (l + 1) s G or B. nosy cheap talk message m = j (l + 1) wth precson p = 1 2 round. Wth two players, player To nform player of j (l + 1), player j sends a at the end of each lth revew 1 s equal to player j. If player j receves the sgnal g[j](m) = E whch mples that the communcaton may have an error, then player j makes player nd erent between any acton pro le sequence n the followng revew rounds. Intutvely, player takes cooperaton n the next revew round f f[]( j (l + 1)) = G and defecton f f[]( j (l + 1)) = B. However, to ncentvze player j to tell the truth wthout destroyng e cency of the equlbrum and to deal wth the chan of learnng, we need a more complcated strategy. Spec cally, after each lth revew round, player calculates the condtonal belef (dstrbuton) of X j (l) gven player s hstory. By the central lmt theorem, gven player s hstory, the standard devaton of ths condtonal dstrbuton s O(T 1 2 ). If the condtonal expectaton of X j (l) s no more than (q 2 + ") T, then snce (q 2 + 2") T s far from the condtonal expectaton by at least "T, player beleves that player j has not observed an erroneously hgh score wth probablty at least 1 exp( O(T )). That s, player beleves that j (l + 1) = G wth probablty at least 1 exp( O(T )). 23 Therefore, f player s cond- 22 As player swtches to defecton after some hstory, player j does not always take cooperaton. We wll take ths nto account n the formal proof. 23 Precsely speakng, j (l + 1) = B f and only f player j has observed an erroneously hgh score n the 21

22 tonal expectaton of player j s score about player s no more than (q 2 + ") T, then player wll thnk that t s an error wth probablty at least 1 exp( O(T )) f player receves f[]( j (l + 1)) = B. Gven the dscusson above, player wll take the followng strategy: 1. If player s condtonal expectaton of player j s score about player s no more than (q 2 + ") T, then player wll mx the followng two: (a) Wth probablty 1, player dsregards the message and beleves that j (l+1) = G, thnkng that t s an error f player receves f[]( j (l + 1)) = B. (b) Wth probablty, player obeys player s sgnal of player j s message: Player takes cooperaton n the (l + 1)th revew round f f[]( j (l+1)) = G and defecton f f[]( j (l + 1)) = B. 2. If player s condtonal expectaton of player j s score about player s more than (q 2 + ") T, then player always obeys player s sgnal of player j s message: Player takes cooperaton n the (l + 1)th revew round f f[]( j (l + 1)) = G and defecton f f[]( j (l + 1)) = B. In addton, f 1-(b) or 2 happens, then player makes player j nd erent between any acton pro le sequence. Verfy that ths s an "-equlbrum: From player s perspectve at the begnnng of the (l + 1)th revew round, 1-(a) s "-optmal by the reason explaned above. For 1-(b) and 2, t s always "-equlbrum to obey the message snce whenever player s sgnal s wrong: f[]( j (l + 1)) 6= j (l + 1), player j receves g[j]( j (l + 1)) = E and makes player nd erent between any acton pro le sequence wth probablty 1 exp( O(T 1 2 )). Does player want to learn from player j s contnuaton strategy? The answer s no n "-equlbrum. ~ lth revew round for some ~ l l. Hence, even f player j s score n the lth revew round s not erroneously hgh, t s possble to have j (l + 1) = B when player j has observed an erroneous score before the lth revew round. We wll take ths nto account n the formal proof n Secton

23 If 1-(b) or 2 s the case for player, then snce player j s strategy s ndependent of g[j]( j (l+1)), 24 player always beleves that f f[]( j (l+1)) 6= j (l+1), then g[j]( j (l+1)) = E. If 1-(a) s the case for player, then player s belef on j (l + 1) = G at the begnnng of the (l + 1)th revew round s no less than 1 exp( O(T )). On the other hand, 1-(b) or 2 s the case for player j wth probablty at least regardless of player j s hstory. Hence, player j obeys player j s sgnal of player s message wth probablty at least. Snce player j s sgnal of player s message s nosy, regardless of player s true message and g[]( (l + 1)), any realzaton of player j s sgnal s possble wth probablty at least exp( O(T 1 2 )). Thus, player beleves that any acton of player j happens wth probablty at least exp( O(T 1 2 )). Snce the ntal belef on j (l + 1) = G s 1 exp( O(T )), whch s very hgh compared to exp( O(T 1 2 )), player wll not learn from player j s contnuaton play n "-equlbrum. In other words, when player obeys the sgnal, player beleves that f player s sgnal s wrong, then player j should have known that. When player dsregards the message based on her nference from the revew round, then whenever player observes player j s acton d erent from player s expectaton, player attrbutes the nconsstency to an error n player j s sgnals, rather than to player s nference about player j s score about player. Ths s possble snce the nference from the revew round s precse wth probablty 1 exp( O(T )) whle the sgnals of the nosy cheap talk are mprecse wth probablty exp( O(T 1 2 )). Fnally, consder player j s ncentve. The ncentve to tell the truth about j (l + 1) s sats ed snce whenever player s sgnal of player j s message a ects player s contnuaton play, that s, f 1-(b) or 2 s the case for player, then player makes player j nd erent between any acton pro le sequence. We also need to consder player j s ncentve n the lth revew round. If 1-(a) s the case, then player cooperates and player does not make player j nd erent between any acton pro le sequence. Ths s better than 1-(b) or 2, where player makes player j nd erent be- 24 As player s contnuaton play s ndependent of g[]( (l+1)), player j s contnuaton play s ndependent of g[j]( j (l + 1)). 23

24 tween any acton pro le sequence. 25 Therefore, f player j can decrease player s condtonal expectaton of player j s score about player, then player j wants to do so. We construct the good sgnal so that player j cannot manpulate player s condtonal expectaton of player j s score about player. That s, player j s expectaton of player s condtonal expectaton of player j s score about player s constant wth respect to player j s acton. See (19) and (27) for the formal de nton of the good sgnal. Therefore, ths s an "-equlbrum. We are left to check e cency. An erroneously hgh realzaton of player j s score about player or player s condtonal expectaton of player j s score about player does not occur wth hgh probablty. In addton, g[j](m) = E does not happen wth hgh probablty ether. Hence, f we take (the probablty that 1-(b) s the case) su cently small, then wth hgh probablty, player takes cooperaton for all the revew rounds and player j montors player by g q 2 q 1 T + LX l=1 g q 2 q 1 ((q 2 + 2") T X j (l)) : Snce the ex ante mean of X j (l) s q 2 T, the per-perod expected reducton of the contnuaton payo s g q 2 q 1 1 L + 2", whch can be arbtrarly small for large L and small ". Summary Let us summarze the equlbrum constructon. Although the breakdown of cooperaton after erroneous hstores s nevtable, we need to verfy that the chan of learnng does not destroy the ncentves. Frst, we dvde the long revew round nto L revew rounds. We make sure that, n each revew round, the constant acton s optmal. To do so, we have a constant term q 2 g q 1 T for the reducton of the contnuaton payo. Ths s enough to cover the maxmum decrease n the reducton of the contnuaton payo n one revew round. At the same tme, snce the length of one revew round s only 1 L of the total length of the revew phase, the per-perod reducton of the contnuaton payo from ths constant term s 25 Snce player s n the good state, when player makes player j nd erent between any acton pro le sequence, player wll do so by reducng player j s contnuaton payo from the next revew phase so that player j s payo (the summaton of the nstantaneous utltes and the reducton of the contnuaton payo ) s atten at the lowest level wth respect to acton pro les. 24

25 su cently small for large L. So, ths does not a ect e cency. To nform player of the optmal acton n the next revew round, player j sends a nosy message. The nose plays two roles: Frst, player (the recever) dsregards the message wth ex ante hgh probablty (ths s 1-(a) n the above explanaton). To ncentvze player j to tell the truth, player makes player j nd erent between any acton pro le sequence n the followng revew rounds whenever player s sgnal of player j s message a ects player s contnuaton play. Snce player dsregards the message wth hgh probablty, ths does not destroy e cency. Second, snce each player obeys her sgnal of the opponent s message wth a postve probablty, whenever a player observes the opponent s acton d erent from what she expected, she thnks that ths s due to an error n the nosy communcaton. Ths cut down the chan of learnng. Fnally, we construct the good sgnal from player j s prvate sgnals such that player j s expectaton of player s condtonal expectaton of player j s score about player s constant wth respect to player j s acton A General Two-Player Game Now, we consder the second role of L, that s, we consder a game where the block of Hörner and Olszewsk (2006) has more than one perod. We stll concentrate on the two-player case. Imagne that we replace each perod n Hörner and Olszewsk (2006) wth a T -perod revew round. We need L revew rounds so that, when player uses the harsh strategy, regardless of player j s devaton, we can keep player j s value low enough. If player j devates for a non-neglgble part of a revew round, then by the law of large numbers, player can detect player j s devaton wth hgh probablty. If player mnmaxes player j from the next revew round after such an event, then player j can get a payo hgher than the targeted payo only for one revew round. Wth su cently long L, therefore, player j s average payo from a revew phase can be arbtrarly close to the mnmax payo. A known problem to replace one perod n Hörner and Olszewsk (2006) wth a revew round s summarzed n Remark 5 n ther Secton 5. Player s optmal acton n a round 25