A Robust Folk Theorem for the Prisoner s Dilemma

Transcription

1 A Robust Folk Theorem for the Prsoner s Dlemm Jeffrey C. Ely Juuso Välmäk December 23, 1999 Abstrct We prove the folk theorem for the Prsoner s dlemm usng strteges tht re robust to prvte montorng. From ths follows lmt folk theorem: when plyers re ptent nd montorng s suffcently ccurte, but prvte nd possbly ndependent) ny fesble ndvdully rtonl pyoff cn be obtned n sequentl equlbrum. The strteges used cn be mplemented by fnte rndomzng) utomt. Thnks to Görkem Celk for vluble reserch ssstnce. Economcs Deprtment, Northwestern Unversty. ely@nwu.edu Economcs Deprtment, Unversty of Southmpton. vlmk@soton.c.uk 1

2 The folk theorem for dscounted repeted gmes sttes tht every pyoff vector tht s fesble nd ndvdully rtonl s n equlbrum pyoff when plyers re suffcently ptent. A proof of the folk theorem frst ppered n Fudenberg nd Mskn 1986) for subgme-perfect equlbr of repeted gmes wth perfect montorng. Perfect montorng mens tht the hstory of chosen ctons s lwys common knowledge mong the plyers. In mny mportnt economc pplctons, plyers montor one nother mperfectly: ech observes nosy sgnl of the ctons chosen by others. Thus, strtng wth the erly ppers by Rdner 1985) nd Green nd Porter 1984), ttenton turned to repeted gmes wth mperfect montorng. The erly pplctons were to stutons n whch montorng, whle mperfect, ws publc: the rndom pyoff-relevnt outcome n ech stge ws ssumed common knowledge mong the plyers. Explotng the structure of publc montorng, dynmc progrmmng technques cn be used to chrcterze the set of perfect publc equlbrum pyoffs s n Abreu, Perce, nd Stchett 1986) nd Abreu, Perce, nd Stchett 1990). Ths pproch to mperfect montorng culmnted n the folk theorem of Fudenberg, Levne, nd Mskn 1994) whch dentfed condtons on the publc montorng technology whch ensured tht ll fesble nd ndvdully rtonl pyoffs could be supported n equlbrum. In repeted gmes wth montorng by prvtely observed sgnls, these technques do not pply nd whether the folk theorem extends s stll n open queston. Indeed, for some gmes, whether there re ny sequentl equlbr dfferent from repetton of stge-gme Nsh profles s unresolved. 1 Numerous negtve results emphsze the dffcultes nvovled. Mtsushm 1991) consders repeted ply of stge gmes wth unque Nsh equlbrum nd montorng by condtonlly ndependent prvte sgnls. Condtonl ndependence mens tht for ech cton profle, the plyers prvte sgnls re dstrbuted ndependently of one nother. Ech ndvdul s prvte sgnl my be rbtrrly nformtve bout the relzed cton profle, but condtonl on tht cton profle, ech plyer s sgnl s unnformtve bout the sgnls observed by other plyers. Mtsushm shows tht f there s ny pure-strtegy equlbrum dfferent from repetton of the stge gme Nsh profle, t must nvolve condtonng on pyoff-rrelevnt hstory. Specfclly, there must be plyer nd pr of hstores for whch gve rse to dentcl 1 For prtculrly problemtc exmple, see the dscusson n Mlth nd Morrs 1998) of the conventon gme orgnlly studed by Shn nd Wllmson 1991) 2

3 belefs over the opponents contnuton strteges but whch nevertheless nduce dstnct contnuton ply by. In prtculr, ths mples tht equlbr cnnot be strct. 2 Mny of the strteges used to prove folk theorems n envronments wth publc montorng fl to be even pproxmte equlbr when montorng s t ll mperfect but condtonlly ndependent. For exmple, consder the strteges used by Fudenberg nd Mskn 1991) to prove the folk theorem under perfect montorng. These strteges begn n coopertve phse n whch plyers ply determnstc sequence of pure cton profles untl some plyer devtes from tht sequence. Such devton trggers punshment n whch the devtng plyer s mnmxed. Contnuton strteges re constructed so tht the non-devtng plyers hve n ncentve to crry out the punshment nd return to the coopertve phse. When the dscount fctor s close enough to one, ech plyer hs strct ncentve to follow hs equlbrum strtegy fter every hstory. It s ths strctness property tht mples tht even the slghtest condtnlly ndependent prvte montorng mperfectons destroy the equlbrum. To see ths, consder plyer who hs detected devton by plyer n stge 1. Snce knows tht s followng hs equlbrum strtegy, knows tht hs not devted nd n fct tht no plyer hs devted. Snce the equlbrum ws constructed so tht hs strct ncentve to cooperte fter hstores n whch no plyer hs devted, wll not punsh but wll nsted contnue to cooperte. But ths unwllngness to punsh sgnls of bd behvor elmntes the necessry ncentves to cooperte n the frst plce. Ths nturlly rses the queston of wht equlbr of gmes wth perfect montorng re robust to prvte montorng mperfectons. Tht s, for whch f ny) equlbr s there gurnteed to be nerby equlbrum when montorng s nerly perfect, but prvte. In ths pper, we focus on the repeted prsoners dlemm nd prove robust folk theorem: for suffcently ptent plyers, every fesble, ndvdully rtonl pyoff cn be cheved by equlbrum strteges tht re robust to prvte montorng. Ths mples lmt folk-theorem: For montorng technologes suffcently ccurte nd dscount fctors close enough to one, ny ndvdully rtonl pyoff cn be 2 Bhskr 1998) derves n even stronger necessry condton from model n whch ech plyer s pyoffs re rndomly perturbed n ech perod ndependently of hstory nd re prvte nformton to tht plyer. In repeted gme wth overlppng genertons of plyers hence prvte montorng), hs condton mples tht ny equlbrum must be repetton of stge-gme Nsh outcomes. 3

4 cheved n sequentl equlbrum of the prvte montorng gme. Relted results hve been obtned elsewhere. Sekguch 1997) ws the frst to show tht the mutul cooperton pyoff cn be cheved n clss of prsoners dlemm gmes when montorng s nerly perfect. 3 Mlth nd Morrs 1998) show tht some trgger strteges re robust to montorng mperfectons tht re pproxmtely publc. Montorng s pproxmtely publc when plyers relzed sgnl s nformtve of the sgnls observed by others even fter condtonng on the relzed cton.) Condtonl ndependence s therefore ruled out. To prove tht trgger-strtegy nd other strct equlbr re robust, the problem of dscontnuous condtonl belefs must be crcumvented. Condtonl belefs bout opponents contnuton strteges fter hstores nvolvng devton chnge dscontnuously when slght montorng mperfectons re ntroduced. Ths cn be seen n the bove dscusson of the Fudenberg nd Mskn 1991) strteges. When montorng s perfect, when plyer observes devton n stge 1, he s certn tht hs opponents wll move nto the punshment phse. But when montorng s condtonlly ndependent, followng the correspondng prvte) hstory plyer s nerly certn tht hs opponents wll contnue to cooperte. Ths observton, together wth the Mtsushm result, mke t cler tht strct equlbr cnnot be robust to ll prvte montorng mperfectons. Mlth nd Morrs 1998) restrct ttenton to montorng technologes n whch there s suffcent correlton n the plyers prvte sgnls,.e. nerly publc montorng. When montorng s pproxmtely publc, plyer wll be nerly certn tht the opponents hve lso observed devton nd wll begn punshng, even though t s common knowledge mong the plyers tht no devton hs occured. Notce tht condtonl belefs re no longer dscontnuous once the montorng mperfectons re restrcted n ths wy. 4 Our pproch to the folk theorem dels wth dscontnuous condtonl belefs n smpler wy. We construct sttonry strteges whch hve the property tht ech plyer s ndfferent mong ech of hs ctons no mtter wht prvte hstory hs opponnent hs observed. It s then rrelevnt 3 Bhskr 1999) shrpens the nlyss nd shows tht the pyoff restrcton used n Sekguch 1997) s not necessry. When publc correlton devces re vlble, these strteges cn lso be used to prove the folk theorem. 4 There s stll n ddtonl complcton to overcome n estblshng tht strct equlbrum s robust to pproxmte publc montorng. Condtonl belefs must move contnuously unformly cross the nfnte set of hstores. 4

5 how condtonl belefs re ltered by the montorng mperfectons snce the plyers contnuton strteges wll be best-reply to every condtonl belef. Ths n turn mples tht our strteges wll be robust to ll montorng mperfectons, ncludng condtonl ndependence nd even negtve correlton. We show tht lrge set of pyoffs, ncludng mutul cooperton, cn be supported by such equlbr. However, not ll ndvdully rtonl pyoffs cn be supported n ths wy. To obtn the full folk theorem set of pyoffs, we show how these robust equlbr generte contnuton vlues tht cn enforce behvor yeldng pyoffs outsde the orgnl set for suffcently long but fnte length of tme. A recent pper by Pccone 1998) employs technque smlr to ours. In the repeted prsoners dlemm Pccone uses dynmc progrmmng technques over the nfnte stte-spce of prvte hstores to construct the mxed strteges necessry to mntn ndfference. Hs strteges cn be used to pproxmte most of the fesble, ndvdully rtonl pyoff set. Our pproch uses sttonry behvor strteges tht condton only on one perod of hstory. Ths drmtclly smplfes the dynmc progrm yeldng system of four equtons whch cn esly be solved for equlbrum mxtures. 5 Beyond the computtonl smplcty, the smple structure mkes ths more promsng drecton for results n more generl gmes. We demonstrte the flexblty of our pproch by nlyzng some more generl gmes n secton 4. In secton 1, the perfect montorng folk theorem s proven for lrge subset of the fesble pyoff set usng smple two stte mxed strteges. In secton 2, t s shown tht the strteges used re robust, nd the lmt folk theorem s estblshed for these pyoffs. In secton 3 we show how to ugment these strteges to obtn the full set of fesble ndvdully rtonl pyoffs. In secton 4, we consder more generl gmes. For 2 plyer gmes, we provde suffcent condton on stge pyoffs under whch our pproch wll pply. We lso nlyze symmetrc N-plyer prsoners dlemm nd show how to extend our pproch nd obtn nerly coopertve pyoffs for hgh dscount fctors nd suffcently ccurte montorng. Fnlly, secton 5 concludes. 5 We lso obtn the full set of pyoffs exctly. 5

6 1 Perfect Montorng We consder the δ-dscounted nfntely repeted prsoner s dlemm, the normlzed stge-gme pyoffs of whch re dsplyed below, where g, l > 0. C D C 1, 1 l, 1+g D 1+g, l 0, 0 Fgure 1: Normlzed Prsoners Dlemm In ths secton, montorng s ssumed perfect. 6 Consder the fmly of behvor strteges defned s follows. Plyer s =1, 2) plys C n stge 1, nd n ny subsequent stge t, hs mxed cton depends only on the outcome n stge t 1. Denote by π the probblty wth whch plyer plys C condtonl on the outcome, ) occurrng n the prevous perod. Note tht the behvor n perod t>1 s ndependent of t. For consstency wth the followng secton, we wll use lower-cse subscrpts, e.g. π cd.below s dgrm of the two-stte mchne tht plys ths strtegy π cd d c 1- π cc C π π dc dd c D d Fgure 2: Mchne representton of π. To vod clutter, we hve left out the rrows tht return to the precedng stte. 6 Another nterestng queston concerns the robustness of the folk theorem under mperfect publc montorng: wht pyoffs n the publc montorng gme cn be obtned when montorng becomes nerly publc, but remns mperfect? The technques here cn be extended to ths context, nd s subect for further reserch. 7 Obr 1999) ndependently ppled strteges of ths form to repeted gmes wth publc montorng. 6

7 We wll show tht for ny pyoff pr v 1,v 2 ) n the squre V =0, 1] 0, 1], there re strteges of ths form tht consttute n equlbrum of the repeted gme ssumng suffcent ptence) nd obtn verge pyoffs v 1,v 2 ). In the next secton, we wll show tht equlbr of ths form re robust to prvte montorng. Obvously V s not the entre fesble, ndvdully-rtonl set. The constructon wll mke t cler tht more cnnot be cheved usng strteges of ths smple form. In secton 3 we show how to modfy these strteges to obtn ny fesble ndvdully rtonl pyoff n robust equlbrum. Let V denote the nteror of V. Fx vlues VC nd V D n 0, 1], wth V C >V D for =1, 2, nd let δ <1 stsfy mn {VC V D } > 1 δ. The gol s to fnd δ δ, 1) suffcently lrge nd to construct probbltes π such tht 1) n ny perod n whch s plyng C, plyer s ndfferent between ctons C nd D nd obtns contnuton vlue VC nd 2), n ny perod n whch plyer s plyng D, plyer s ndfferent between ctons C nd D nd obtns contnuton vlue VD. From ths t wll follow tht ech plyer s ndfferent between C nd D fter every hstory fct tht s essentl for the robustness rgument), nd plyer gets verge pyoff VC, for =1, 2. The followng four equtons express these condtons: VC =1 δ)+δ [ πcc V C +1 π cc )V ] D =1 δ)1 + g)+δ [ π cd V C +1 π cd )V D VD = l1 δ)+δ [ π dc V C +1 π dc )V ] D = δ [ π dd V C +1 π dd )V ] D ] 1) 2) 3) 4) Equton 1 s equvlent to the followng π cc = V C 1 δ) δv D δv C V D ) Becuse VC δv D >V C V D > 1 δ >1 δ, the numertor nd hence the frcton s greter thn zero. It s no greter thn one ff VC 1 δ) δv C whch s equvlent to VC 1 whch ws ssumed. For future reference, note tht πcc cnbemdenterorfv C < 1. Combnng 1 nd 2 nd mnpultng, we obtn ) ) 1 δ πcc π cd = g δ VC V D 7

8 Snce VC V D > 0, ths s greter thn zero nd cn be mde rbtrrly smll by choosng δ suffcently close to 1. Thus, π cd 0, 1) for suffcently lrge δ. Note lso tht for ny such δ, thsπ cd 0, 1) contnues to hold for ny ˆV C >V C. We hve shown tht then there exst probbltes πcc 0, 1] nd π cd 0, 1) such tht f VD s the vlue to plyer when hs opponent s plyng D, then VC s the vlue to when hs opponent s plyng C, nd s ndfferent between C nd D fter such hstores. Now equton 4 reduces to ) ) 1 δ V π dd = D δ VC V D Ths s lwys postve nd wll be less thn 1 once δ s suffcently close to 1. Fnlly, equtons 3 nd 4 combne to yeld ) ) 1 δ π dc π dd = l δ VC V D whch gn mkes π dc n nteror probblty once δ s suffcently lrge. Agn, f we fx such δ, π dc wll remn nteror for ny V C closer to 1. Thus, nteror probbltes cn be found to ensure tht when plyer s plyng D, plyer s ndfferent between C nd D nd obtns contnuton vlue VD. We conclude tht for δ suffcently close to 1, there exsts strtegy for plyer stsfyng equtons 1-4 for ny vlue ˆV C [V C, 1]. By the symmetrc rgument, there exsts strtegy for plyer whch stsfy the nlogous equtons for plyer. Furthermore, snce these equtons mply tht ech plyer s ndfferent between C nd D fter every hstory, ech strtegy s best-response to the other fter every hstory. Thus, the strteges form subgme-perfect equlbrum of the perfect montorng gme nd snce ech strtegy s ssumed to ply C n the frst stge, plyer obtns verge pyoff v = VC. 2 A Lmt Folk Theorem The focus now turns to the repeted prsoners dlemm wth prvte montorng. In ths settng, plyers do not drectly observe the ctons chosen by ther opponent. Followng ech stge of ply, ech plyer observes hs own 8

9 chosen cton nd prvte sgnl whch depends on the outcome n tht stge. Let Σ be fnte set of sgnls for plyer, ndσ= Σ. Assume tht ech Σ hs t lest two elements. A montorng technology s collecton {m ) : {C, D} 2 } of probblty dstrbutons over sgnl profles, one for ech possble stge-gme outcome. The mrgnl dstrbuton over plyer s sgnl wll be denoted m ). Perfect montorng corresponds to technology m 0 whch stsfes two condtons. Frst, ech m 0 ) exhbts perfect correlton publc montorng); nd second, for ech there s set c Σ such tht f = C then m 0 c ) =1ndf = D then m 0 c ) =0. Sy tht montorng technology s n ε-perturbton of m 0 f for ech there s set c Σ such tht = C mples m c ) > 1 ε nd = D mples m c ) <ε. Note tht ths defnton nvolves no requrement on the correlton n m, nd n prtculr, ncludes s specl cse, ndependent prvte montorng: technologes m for whch m ) s ndependent of m ) for ech. Henceforth, we restrct ttenton to technologes tht re perturbtons of m 0 nd wrte m = m c, ), so for exmple m DC represents the probblty tht observes sgnl n c when plys D nd plys C. Letd denote the complement n Σ of c. The δ-dscounted repeted prsoners dlemm wth prvte montorng technology m wll be denoted G δ, m). A strtegy π n G δ, m) specfes mxed cton for ech hstory of own-ctons nd observed sgnls. We wll restrct ttenton to strteges whch depend only on hstores of length 1 nd condton only on the plyer s own cton nd the events c nd d. By nlogy to the prevous secton, wrte e.g. πdc for the probblty wth whch plyer plys C followng stge n whch plyed D nd observed sgnl n c.setπ =1. For ny two strtegy profles π, π of ths form, defne the dstnce π π to be mx πs π s where the mxmum s over plyers =1, 2ndsttess of the mchne. Defnton 1 A subgme-perfect equlbrum π of G δ, m 0 ) s robust to prvte montorng f for every e>0 there exsts ε >0 such tht for ll ε 0, ε) nd ll ε-perturbtons m ε of m 0, there s sequentl equlbrum π of G δ, m ε ),wthne dstnce of π nd wth pyoffs wthn e of the pyoffs under π. A suffcent condton for pr π 1,π 2 ) to be sequentl equlbrum 9

10 of G δ, m) wthpyoffsv 1,v 2 ) s for the mxtures to solve the followng equtons for V C = v, =1, 2,. [ ] VC =1 δ)+δvc m CC π cc +1 m CC )π cd [ + δvd 1 m CC )1 π cd )+m CC 1 5) π cc )] [ VC =1 δ)1 + g)+δvc m CD π cc +1 m ] CD )π cd [ + δvd 1 m CD )1 π cd )+m CD 1 6) π cc )] [ VD = l1 δ)+δvc m DC π dc +1 ] m DC )π dd [ + δvd 1 m DC )1 π dd )+m DC 1 7) π dc )] [ VD = δvc m DD π dc +1 ] m DD )π dd [ + δvd 1 m DD )1 π dd )+m DD 1 8) π dc )] Equtons 5 nd 6 stte tht plyer gets verge pyoff VC when he plys ether C or D n ny stge n whch plyer plys C. Equtons 5 nd 6 stte tht plyer gets verge pyoff VD when he plys ether C or D n ny stge n whch plyer plys D. If these equtons re stsfed, then plyer s lwys ndfferent between hs two ctons nd s therefore wllng to ply ny mxed strtegy fter every hstory. When montorng s perfect, m,d =1 m,c = 0 nd these equtons reduce to equtons 1-4 from the prevous secton. There t ws shown tht provded δ s suffcently lrge nd VC 0, 1], soluton π, V C, V D ) exsts, nd tht the mxtures π cd nd π dc cn be chosen to be nteror. We cn now fx π cc nd π dd t the soluton nd solve for V C, V D, π cd,ndπ dc : π dc = δπ dd µ Dµ C g1 m CC )) + l1 δ π cc π dd )+π dd m DD ) δgµ D m CC 1) + µ Cµ D lm DD ) π cd = δµ Cπccµ D + lπ dd )+gµ Dδπccm CC π dd ) 1) δgµ D m CC 1) + µ Cµ D lm DD ) V C = V C + gµ Dm CC 1)1 + δπ dd 1)) δµ Clπ cc 1)m DD µ C µ D δ π cc π dd ) 1) V D = V D + δ[gµ Dπ dd 1 m CC ) lµ Cπ cc m DD ] lµ Cm DD µ C µ D δ π cc π dd ) 1) where we hve wrtten µ D = m DC m DD nd µ C = m CC m CD. These equtons defne the left-hnd sde vrbles s contnuous functons of the 10

11 montorng prmeters. Snce π cd nd π dc were nteror, t follows tht for m,d nd 1 m,c suffcently close to zero, there exsts soluton π,vd,v C ) to equtons 5-8 wth mxtures π nd vlues VC,V D rbtrrly close to π, V C, V D ). Plyer therefore hs strtegy gnst whch every strtegy of plyer s best-response nd cheves pyoff VC. Ths estblshes the followng theorem. Theorem 1 Robust folk theorem) Let v 1,v 2 ) V. There exst δ 0, 1) such tht for ll δ δ, 1), there exsts robust subgme perfect equlbrum of G δ, m 0 ) wth pyoffs v 1,v 2 ). Now suppose V C 0, 1) nd recll tht ths mples tht π cc 0, 1). We cn now fx V C, V D ) 0, 1)2 nd solve equtons 5-8 for the followng explct reltons between the equlbrum strteges nd the montorng probbltes. πcc = V C δv D )m CC m CD )+1 δ)g + m CD m CC 1 + g)) δ m CC CD) m V C VD ) π cd = V C δv D )m CC m DD )+1 δ)m CD 1 + g)m CC ) δ m CC CD) m V C VD ) π dc = 1 δ) [ l1 m DD ) V D m DD m DC )] δ m DC DD) m V C VD ) π dd = 1 δ) [ m DC m DD )V D ] lm DD δ m DC DD) m V C VD ) One cn esly verfy tht these defne equlbrum behvor strteges s contnuous functons of the montorng probbltes m. Snce ll mxtures n π re nteror, t follows tht for m,d nd 1 m,c suffcently close to zero, equtons 5-8 cn be solved for probbltes π, nd the symmetrc set of equtons cn be solved for probbltes π. These solutons wll be Nsh equlbr of the prvte montorng gme wth vlues V C 1, V C 2 ). Ths proves the lmt folk theorem Theorem 2 Lmt folk theorem) Let v 1,v 2 ) V. There exst δ, ε 0, 1) such tht for ll δ δ, 1) nd ε 0, ε), fm s n ε-perturbton of m 0, then there exsts sequentl equlbrum of G δ, m) wth pyoffs v 1,v 2 ). 11

12 The order of quntfcton ws not proven bove. In the prevous secton, t ws shown tht f there s soluton for some δ then there s soluton for ny lrger ˆδ. One cn esly verfy tht the sme holds here: f there re probbltes tht solve equtons 5-8 for some ε nd δ, then there re probbltes tht solve the equtons for ε nd ny ˆδ >δ. 3 Extendng to the Full Pyoff Set We hve shown tht ny pyoff pr n V cn be obtned n robust equlbrum. Ths leves out much of the fesble, ndvdully rtonl set of pyoffs. In prtculr, the only effcent pyoff pr n V s the symmetrc pyoff 1, 1). Unfortuntely, nothng outsde of V cn be obtned robustly usng strteges of the smple form consdered bove. To see ths, note tht f n equlbrum, plyer s ndfferent between C nd D fter every hstory, then hs long-run verge pyoff must be equl to the pyoff he would get by plyng C fter every hstory. Obvously such pyoff cnnot exceed 1. However, more cn be cheved usng more complcted yet stll fntestte) strteges. We sketch the de here, nd prove t formlly below. Return to the perfect montorng cse. Fx VD 1 > 0closetozerondV C 2 =1 Let VC 1 be ny vlue n V D 1, 1]. We hve shown n theorem 1 tht for δ close enough to 1, there exst robust equlbrum strteges π 1,π 2 )whchobtn vlues VC 1,V2 C ), nd plyer 2 s strtegy cn be chosen so tht plyer 1 obtns vlue VD 1 whenever plyer 2 plys D. Consder the followng strteges. Plyer 1 plys C n stge 1, then proceeds wth π 1 strtng n stge 2. Plyer 2 plys D n stge 1. In stge 2, plyer 2 swtches to strtegy π 2 but strts n stte D f he observes tht plyer 1 hs plyed D n stge 1. If 2 observes tht 1 hs plyed C, 2 strts n stte C. Let v 1,v 2 )=1 δ) l, 1+g)+δVC 1,V2 C ). The bove strteges form n equlbrum wth pyoffs v 1,v 2 ) f the followng ncentve constrnts re stsfed. 1 δ)1 + g)+δvc 1 > 1 δ)+δv C 1 9) 1 δ) l)+δvc 2 >δv2 D 10) These hold for suffcently lrge δ. Note tht the pyoffs v 1,v 2 ) re outsde of V. Now we clm tht ths equlbrum s robust to prvte montorng. To see ths, note tht n stge two, ech plyer s ndfferent between hs 12

13 two ctons regrdless of the outcome n stge 1. Ech s therefore wllng to behve s the strtegy demnds even fter erroneous sgnls. The contnuton pyoffs VC 1,V2 C,V1 D re contnuous n the montorng perturbton becuse the contnuton strteges re robust. Thus, becuse the frst-stge ncentve constrnts re stsfed wth strct nequltes, they wll contnue to hold for smll perturbtons of m 0. We show below tht by ncresng the number of stges n whch C, D) s plyed, ny fesble ndvdully rtonl pyoff vector cn be sustned n robust equlbrum. To prove ths formlly, we mke n ddtonl ssumpton. We ssume tht g l 1 so tht mutul cooperton s not Preto-domnted by ny other profle. We mke ths ssumpton merely to smplfy the exposton s ths llows us to cheve ny ndvdully rtonl pyoff by convex combnton of the set V nd sngle ddtonl pont. Our result contnues to hold wthout ths ssumpton, wth strteges lterntng between C, D) nd D, C), slghtly complctng the proof. For P R n, denote by CoP ), the convex hull of P. Lemm 1 Let P be convex subset of R n wth non-empty nteror md v R n. For ny u nt cop {v}), theresδ such tht for ll δ δ, 1], there s nturl number N, nd w P such tht u = 1 δ N) v + δ N w. Proof: Fx u nt cop {v}). Snce P s convex wth non-empty nteror, there s λ 0, 1) nd w nt P such tht u =1 λ ) v + λ w. 11) Snce w nt P, there s by contnuty) n ε>0ndε<λ ), such tht for ll λ B ε λ ), there s w λ) P stsfyng u =1 λ) v + λw λ). Let N δ) =mx{n δ n λ }. The remnng tsk s hence to show tht there s δ such tht for ll δ δ, 1], δ Nδ) B ε λ ). By constructon, δ Nδ)+1 <λ nd hence δ Nδ) < λ or δ δnδ) λ < λ δ λ = λ 1 1) nd the δ clm holds for δ = λ ε. λ Theorem 3 For ech v V theres δ 0, 1) such tht for ll δ δ, 1) there s robust subgme perfect equlbrum of G δ, m 0 ) wth pyoffs v. 13

14 Proof: Let V1 be the subset of V n whch plyer 1 s pyoff s t lest 1 nd hence plyer 2 s pyoff s less thn 1). We wll show tht ech v V1 s robust equlbrum pyoff for δ close enough to 1. Together wth the symmetrc rgument when the plyers roles re reversed, ths wll estblsh the theorem. If v V1 then v CoV {1 + g, l)}) see fgure 3). By lemm 1 there s δ such tht for ll δ δ, 1) there s N nd u V such tht v =1 δ N )1 + g, l) +δ N u. Note tht snce u 2 > 0 > l, u 2 >v 2. Choose w 2 so tht 0 <w 2 <v 2 nd δ suffcently close to 1 so tht u nd w := u 1,w 2 ) re robust equlbrum pyoffs. -l, 1+g) u ,1) v w ,0) 1+g, -l) Fgure 3: Prsoner s Dlemm fesblepyoff set. Consder the followng strtegy profle. Plyer 1 plys D ndpendent of hstory for the frst N perods. If plyer 1 observes tht 2 hs plyed C n ech of the frst N stges, plyer 1 contnues wth strtegy s 1 gnst whch every strtegy of plyer 2 s best-response nd cheves pyoff of u 2. Otherwse, plyer 1 contnues n stge N + 1 wth strtegy s 1 gnst whch every strtegy of plyer 2 cheves pyoff of w 2. 8 Plyer 2 plys C n ech of the frst N stges provded he hs never plyed D. If n ny of the frst N 1 stges, 2 hs plyed D, then 2 contnues wth D through stge N. Independent of hstory, begnnng n stge N + 1, 2 8 In workng pper verson, we showed by more complcted rgument tht less drconn strteges cn be used. 14

15 plys strtegy s 2 gnst whch every strtegy of plyer 1 cheves pyoff of u 1. By constructon, ny contnutton for plyer 1 s best response to s 2 begnnng n stge N + 1. And snce 2 contnues wth s 2 s ndependent of hstory, t s best-response for 1 to ply D n ech of the frst N stges. Lkewse, s 2 s best response to both s 1 nd s 1 begnnng n stge N +1. To estblsh tht ths profle s subgme perfect equlbrum, therefore, t remns only to check tht 2 s wllng to ply ccordng to the equlbrum n ech of the frst N stges. If 2 hs plyed D pror to stge N, then ndependent of ny future hstory, 1 wll ply s 1 nd 2 wll receve contnuton pyoff w 2. Therefore, 2 optmlly contnues wth D through stge N. Fnlly, suppose tht n ech stge up to s N, 2hsplyedC. If 2 plys ccordng to the equlbrum, hs pyoff s l1 δ N s+1 )+δ N s+1 u 2 > l1 δ N )+δ N u 2 = v 2. If he nsted plys D n stge s ndthencontnues wth hs equlbrum strtegy, hs pyoff s δ N s+1 w 2 <w 2. Snce w 2 <v 2, plyer 2 optmlly plys C. To conclude the proof, we show tht the bove equlbrum s robust. By Theorem 1, for ny suffcently smll ε-perturbton of m 0, there re contnuton strteges s 1, s 1,nd s 2 rbtrrly close to s 1,s 1,nds 2, genertng contnuton pyoffs rbtrrly close to w 1,u 1, nd u 2 respectvely, nd mkng the opponent ndfferent mong ll strteges. It follows tht begnnng n stge N +1, both s 1 nd nd s 1 re best-responses to s 2 nd s 2 s best-response for 2 whtever hs belef over s 1 nd s 1. The plyers wll ply s before for the frst N stges, nd now contnue wth these contnuton strteges. The pyoffs n the frst N stges re unffected by ε. The dstrbuton over N-stge hstores, nd hence contnuton pyoffs begnnng n stge N + 1 s contnuous n ε. Therefore ε cn be tken suffcently smll so tht the overll pyoffs to these strteges re rbtrrly close to v nd they remn sequentl equlbrum. 4 Extensons In ths secton, we outlne brefly how the method for fndng robust subgme perfect equlbr cn be extended to more generl gmes. 15

16 4.1 Two-Plyer Gmes Let G =A 1,A 2,g 1,g 2 ) be fnte two- plyer norml form gme, where A re the cton sets nd g pyoff functons. Let G δ) the nfntely repeted gme wth G s the stge gme nd δ s the dscount fctor. Consder two strtegy profles 1, 2)nd 1, 2) nd ssume tht the followng condton on the pyoff functons s stsfed: For {1, 2}, there re v nd v >v such tht ) ) mxg, <v nd A ) mn { ) g,,g, )} > v. The frst step s gn the constructon of subgme perfect equlbrum n the gme wth perfect montorng where the plyers mx between, ). In nlogy to the prevous sectons, let π, A ) be the probblty dstrbuton on plyer s pure ctons condtonl on observng outcome, )t the prevous stge, nd let V be s before. Theorem 4 For ny v v 1, v 1 ) v 2, v 2 ), there s δ<1 such tht v cn be supported s SPE pyoff n G δ) for ll δ>δusng strteges where π, ) > 0, π, ) > 0 nd π, )=0for / {, }. Proof: Pck V nd V such tht v <V <V < v. Suppose tht rndomzes between nd n ech perod so tht π, ) = 0 for / { } ),. The clm s proved f there re probbltes π, such tht the followng condtons hold: V =1 δ) g ) [ ), + δ V 1 + π ) ) ] V =1 δ) g, For ll / {, }, V 1 δ) g V =1 δ) g, =1 δ) g, ) + δ [ ) + δ [ ) [, + δ ) [ + δ π, π, π, π, π, ) V ) V + + ) V ) V 16 1 π,, 1 π, 1 + π, + 1 π, ) ) V ]. ) ) ] V. ) ) V ) ) V ] ],

17 nd lso for ll / {, }, V 1 δ) g, g, ) + δ [ π, ) V + 1 π, ) <V ) ) ] V Our condton on pyoffs ensures g, for ll A, nd ) ) >V nd g,, >V. The frst set of equltes nd nequltes cn, n fct, be solved s set of equltes, nd we get: ) π ) V, 1 δ) g =, δv ). δ V V ) Snce V >V nd g, <V, 0 <π, suffcently close to 1. The second set of equltes yelds: ) 1 δ) π, = δ π, ) 1 δ) = δ V V V ) ) g, ) V ) ) g, ). V V ) < 1 whenever δ s Agn, π ), nd π ), re probbltes for δ suffcently close to 1. The second set cnnot be stsfed s equltes, n generl. To see ths solve for the probbltes: π, ) 1 δ)v = g, ) ) ). δ V V If g, ) <V, nd V <V the soluton s negtve., Ths does not cuse ny problems, however, snce we cn pck n rbtrry V wth V {, }: < V <V V =1 δ) g, nd probbltes π ), tht yeld for ll / ) [ + δ π, 17 ) V + 1 π, ) ) ] V.

18 A smlr set of condtons holds for plyer, nd gven tht ech plyer s ndfferent fter ech hstory between ctons nd, nd gven tht ll other ctons yeld wekly lower pyoff, the rndomztons bove re best responses. An rgument smlr to tht n the prevous sectons cn be gven to show tht there s, n fct, robust subgme perfect equlbrum tht yelds v s the pyoff vector. Two corollres follow mmedtely from the result bove. Let e =e 1,e 2 ) be pure strtegy Nsh equlbrum profle of the stge gme. Denote the stge gme equlbrum pyoffs by v1 e nd ve 2 respectvely. Corollry 1 If there s profle = 1, 2 ) such tht mn {g e, ),g e,e )} > v e for {1, 2}, then the set of robust subgme perfect equlbrum pyoffs hs non empty nteror. Furthermore, we cn requre tht plyer uses t ech stge ether or e for =1, 2. As n exmple of ths corollry, consder the dscretzed Cournot competton model wth lner demnds. For concreteness, let A = {0, 1, 2,..., 20} nd g, )= 20 ). A Nsh equlbrum of the stge gme s e = e =7. The monopoly prce s supported by cton profle 1 = 2 =5. Observe tht g e) =42,g ) = 50 nd g,e )=56. As result, we conclude tht ny v 42, 50) 2 cn be supported s robust subgme perfect equlbrum for suffcently hgh δ. For the second corollry, let m A be the cton tht mnmxes plyer. Let v denote the pure strtegy mnmx pyoff of ech plyer. Corollry 2 If there s n 1, 2 ) such tht mn {g m, ),g, )} > v for {1, 2}, then the set of robust subgme perfect equlbrum pyoffs hs non empty nteror. Furthermore, we cn requre tht plyer uses t ech stge ether or m. Usng ths corollry, t s esy to see tht n some gmes, pyoffs below those resultng from the unque domnnt strtegy equlbrum of the stge gme re sustnble n robust subgme perfect equlbrum nd hence n gmes wth smll mperfectons n the montorng technologes. Consder e.g. fgure 4. In ths exmple, m = 2 for =1, 2. Hence the Corollry bove mples tht ny v 1, 0) 2 cn be supported s robust equlbrum pyoff. Observe lso tht the pure strtegy Nsh equlbrum pyoff cnnot be pproxmted by mxed strtegy equlbrum of the type s descrbed bove. 9 9 Although such pyoffs cn be obtned by strteges whch ply 1, 1 ) for suffcently 18

19 , 1 1, 0 2 0, 1 2, 2 Fgure 4: Pyoffs below the domnnt strtegy equlbrum cn be supported. 4.2 N-Plyer Cse We conclude ths secton by nlyzng n exmple of symmetrc N -plyer gme. The bsc dffculty n comprson to the two plyer cse s tht the trnstons between the sttes of the mchnes equvlent to those descrbed n the prevous secton re no longer lner n the rndomztons of the other plyers even fter condtonng on own ctons). As result, the probbltes cnnot be solved for s before by smple lner lgebr. In ths subsecton, we show tht for δ close to unty, we cn recover pproxmte lnerty nd show the robustness by n pplcton of the mplct functon theorem. 10 We consder here the followng verson of the N -plyer Prsoner s Dlemm. A = {C, D} for =1,..., N, nd g C, )=n )=g D, )+1, where n ) s the number of plyers dfferent from tht ply C n profle. Let G δ) stnd for the nfntely repeted verson of ths gme wth dscount fctor δ. Suppose tht ll plyers re usng strteges of the followng form: π,n =Pr{C }. In words, ech plyer bses her own rndomztons solely on her own pst ctons nd the number of other plyers tht cooperted, not ther dentty. The next theorem shows tht outcomes rbtrrly close to the effcent outcome cn be supported Theorem 5 For ny v 0,N) theresδ such tht for ll δ>δ, G δ) hs completely mxed SPE n whch ech plyer obtns pyoff of v. Proof: Construct the followng sequence of numbers: n V 0) = v 0,Vn) =v 0 + N 1) v v 0) for n {1,..., N 1}. 12) mny perods before revertng to n equlbrum n the set 1, 0) 2. These strteges re robust by the rgument mde n secton 3 10 Obr 1999b) lso nlyzes the N-plyer prsoner s dlemm. He uses n extenson of the Sekguch 1997) pproch to obtn n pproxmtely effcent symmetrc equlbrum. 19

20 We show tht ech V n) cn be nterpreted s the vlue to plyer n the gme when n other plyers cooperte t the current stge. To smplfy notton, wrte the strteges s: π C,n =1 γ n nd π D,n = β n. Note tht we hve dropped the superscrpt s we wll now restrct ttenton to symmetrc strteges. We re nterested n the exstence of equlbr n strteges of ths type for δ lrge enough. The dynmc progrmmng equtons become then for n {0,..., N 1}: V n) = 1 δ) n + δe D,n V n ), 13) V n) = 1 δ)n 1) + δe C,n V n ), where E C,n V n ) denotes the expected vlue from tomorrow on condtonl on the cton profle tody The trnston probbltes re nonlner n γ n,β n ) snce they re obtned from two bnoml dstrbutons. Suppose tht the strteges used by the plyers depend on δ s follows: γ n δ) δ=1 =0,β n δ) δ=1 = 0 for ll n {0,..., N 1}. In ths cse, the equtons n 13) re trvlly stsfed t δ = 1, nd n prtculr, the sequence gven n 12 solves the system. Wrte γδ) = γ 0 δ),...,γ N 1 δ)) nd βδ) =β 0 δ),...,β N 1 δ)). We wnt to use the mplct functon theorem to conclude tht the nonlner system, 13 hs soluton γδ), βδ)) for δ n neghborhood of 1. Next, to gurntee tht the solutons re ndeed probbltes, we need to show tht for ll n, γn 1) > 0 δ nd βn 1) > 0 δ To pply the mplct functon theorem, we tret 13) s system of 2N equtons tht mplctly defne the 2N probbltes γδ), βδ)). Wrte ths system s V γ, β; δ) =0, wth γ, β s the endogenous vrbles nd δ s the exogenous vrble. Vewng 13) s system of N 1 prs of equtons, denote by V nd the frst functon of the nth pr nd V nc the second. We now evlute the prtl dervtves of the system t the orgnl soluton for δ =1)wth 20

21 respect to the endogenous vrbles. For 1 n N 1, V nd = V nc = nv n 1) V n)) γ n 1 γ n V nd = V nc =N n 1)V n +1) V n)) β n β n+1 It cn esly be verfed tht ths system hs full rnk. The mplct functon theorem then gurntees the exstence of dfferentble soluton mppng γδ), βδ) n the neghborhood of δ = 1. To check the sgn of the dervtves, we pply the chn rule. Note V 0) V 0) 1 V 1) 1 D δ V γ, β; δ) = V 1) 2. V N 1) N 1) V N 1) N Observe tht the system of equtons, D fl,fv γ, β; δ)d δ γδ), βδ)) = D δ V γ, β; δ) cn be solved n blocks of sze 2. Ths mkes the comprtve sttcs reltvely esy: n V n 1) V n)) N n 1) V n +1) V n)) n +1)V n) V n + 1))) N n 2) V n +2) V n +1)) ) = V n) n +1 V n +1) n 1. ) γn δ Usng the fct tht V n +1) V n) =V k) V k 1) for ll k, n {0, 1,..., N 1}, t s esy to see tht the dervtves hve the rght sgn whenever v 0 s chosen close enough to v. Hence for δ close enough to one, the system cn be solved for probbltes. Ths mples tht for such dscount fctors, the gme hs completely mxed SPE wth pyoff v. 21 β n δ )

22 In order to obtn the result on gmes wth lmost perfect montorng, fx δ for whch there s completely mxed SPE wth pyoff vector v. We cn mke the sme contnuty rgument s n the prevous secton to conclude tht whenever the montorng s close enough to perfect, sequentl equlbrum wth pyoff vector v exsts. 5 Concluson There re mny unresolved questons n the context of dscounted repeted gmes wth prvte montorng. In prtculr, unlke the publc montorng cse, there s no known folk theorem for fxed montorng technologes. 11 In ths note, we hve followed the lterture nd sought fter the weker result: lmt folk theorem for nerly perfect montorng. An mportnt gol for future reserch s to chrcterze equlbrum pyoffs when montorng s prvte but fr from perfect. The strteges we hve used n ths pper cn sustn some cooperton under less thn perfect montorng technologes nd equlbrum cn be chrcterzed by n nlogous system of dynmc progrmmng equtons. The pyoffs tht cn be supported re thus those vlues for whch the system cn be solved for probbltes. Determnng the full potentl of ths pproch s ongong reserch. References Abreu, D., D. Perce, nd E. Stchett 1986): Optml Crtel Montorng wth Imperfect Informton, Journl of Economc Theory, 39, ): Towrd Theory of Dscounted Repeted Gmes wth Imperfect Montorng, Econometrc, 58, Bhskr, V. 1998): Informtonl Constrnts nd the Overlppng Genertons Model: Folk nd Ant-Folk Theorems, Revew of Economcs Studes, 65, ): Sequentl Equlbr n the Repeted Prsoners Dlemm wth Prvte Montorng, mmeo, Unversty of Essex. 11 Lehrer 1989) nd Lehrer 1990) prove folk theorems for non-dscountng crter 22

23 Fudenberg, D., D. Levne, nd E. Mskn 1994): The Folk Theorem wth Imperfect Publc Informton, Econometrc, 625), Fudenberg, D., nd E. Mskn 1986): The Folk Theorem n Repeted Gmes wth Dscountng nd wth Incomplete Informton, Econometrc, 54, ): On the Dspensblty of Publc Rndomzton n Dscounted Repeted Gmes, Journl of Economc Theory, 53, Green, E. J., nd R. H. Porter 1984): Noncoopertve Colluson under Imperfect Prce Informton, Econometrc, 521), Lehrer, E. 1989): Lower Equlbrum Pyoffs n Two-Plyer Repeted Ges wth Non-Observble Actons, Interntonl Journl of Gme Theory, 181), ): Nsh Equlbrum of n-plyer Repeted Gmes wth Sem- Stndrd Informton, Interntonl Journl of Gme Theory, 19, Mlth, G. J., nd S. Morrs 1998): Repeted Gmes wth Imperfect Prvte Montorng: Notes on Coordnton Perspectve, CARESS Workng Pper 98-07, Unversty of Pennsylvn. Mtsushm, H. 1991): On the Theory of Repeted Gmes wth Prvte Informton, prt I: Ant-Folk Theorem wthout Communcton, Economcs Letters, 35, Obr, I. 1999): Prvte Strtegy nd Effcency: Repeted Prtnershp Gme Revsted, mmeo, Unversty of Pennsylvn. 1999b): The Repeted Prsoner s Dlemm wth Prvte Montorng: N-plyer Cse, CARESS Workng Pper #99-13, Unversty of Pennsylvn. Pccone, M. 1998): The Repeted Prsoners Dlemm wth Imperfect Prvte Montorng, mmeo. Rdner, R. 1985): Repeted Prncpl-Agent Gmes wth Dscountng, Econometrc, 535),

24 Sekguch, T. 1997): Effcency n Repeted Prsoners Dlemm wth Prvte Montorng, Journl of Economc Theory, 762), Shn, H. S., nd T. Wllmson 1991): How Much Common Belef s Necessry for Conventon?, Gmes nd Economc Behvor, 13,