Tit-For-Tat Equilibria in Discounted Repeated Games with. Private Monitoring

Transcription

1 1 Tt-For-Tat Equlbra n Dscounted Repeated Games wth Prvate Montorng Htosh Matsushma 1 Department of Economcs, Unversty of Tokyo 2 Aprl 24, 2007 Abstract We nvestgate nfntely repeated games wth mperfect prvate montorng. We focus on a class of games where the payoff functons are addtvely separable and the sgnal for montorng a player s acton does not depend on the other player s acton. Tt-for-tat strateges functon very well n ths class, accordng to whch each player s acton n each perod depends only on the sgnal for the opponent s acton one perod before. Wth almost perfect montorng, we show that even f the dscount factors are fxed low, effcency s approxmated by a tt-for-tat Nash equlbrum payoff vector. JEL Classfcaton Numbers: C72, C73, D82, H41 Keywords: Infntely Repeated Games, Prvate Montorng, Tt-For-Tat Strateges, Fxed Dscount Factors, Approxmate Effcency 1 Ths research was supported by a Grant-In-Ad for Scentfc Research (KAKENHI ) from the Japan Socety for the Promoton of Scence (JSPS) and the Mnstry of Educaton, Culture, Sports, Scence and Technology (MEXT) of the Japanese government as well as a grant from the Center for Advanced Research n Fnance (CARF) at the Unversty of Tokyo. 2 Department of Economcs, Unversty of Tokyo, Hongo, Bunkyo-ku, Tokyo , Japan. E-mal: htosh@e.u-tokyo.ac.p

2 2 1. Introducton We nvestgate two-player nfntely repeated games wth dscountng. We assume that montorng s mperfect n that each player cannot observe the opponent s acton choce but can mperfectly montor t by observng a nosy sgnal. Ths sgnal s randomly determned accordng to the probablty functon condtonal on the opponent s acton choce. We also assume that montorng s prvate n that the sgnals that each player observes cannot be observed by the opponent. We focus on a class of games where the payoff functons are addtvely separable and the sgnals for montorng any player s acton choces are ndependent of the other player s acton choces. Wth ths class restrcton, we nvestgate the possblty that effcency s approxmated by a Nash equlbrum payoff vector when montorng s almost perfect, and we arrve at the followng affrmatve answer. In the case of almost perfect prvate montorng, there exsts a smple tt-for-tat Nash equlbrum that approxmately nduces effcency, accordng to whch, each player s acton choce n each perod depends only on the sgnal for the opponent s acton choce one perod before. The man contrbuton of ths paper s that ths postve result holds even f the dscount factor s fxed and not very hgh. It s well known that f each player perfectly montors the opponent s acton

3 3 choces and the dscount factor s not very low, effcency s acheved by a Nash equlbrum. See Fudenberg and Trole (1991) and Osborne and Rubnsten (1994). In order to clarfy the robustness wth respect to montorng ablty, t s mportant to answer the queston of whether effcency s approxmated by a Nash equlbrum payoff vector f montorng s almost perfect but mperfect. In the case of publc montorng where the sgnals are observable to both players, t s now not dffcult to answer n the affrmatve due to prevous works such as Green and Porter (1984); Abreu, Pearce, and Stachett (1990), and Fudenberg, Levne, and Maskn (1986). In the case of prvate montorng, however, ths robustness ssue remans unresolved and s much more substantal. In fact, the prevous works have gven only partal answers for the prvate montorng case. Frst, Sekguch (1997) showed an example n whch effcency can be approxmated by a Nash equlbrum payoff vector when montorng s almost perfect. Sekguch demonstrated an dea of constructon wth publc randomzaton, whch allowed equlbrum strateges to depend on hstores n a non-recursve manner. Ely and Valmak (2002) and Pccone (2002) demonstrated another dea of constructon on the bass of recursve Markovan technques, whch was applcable to a class of games wder than that presented n Sekguch. By constructng Markovan strateges to whch

4 4 both the cooperatve and non-cooperatve actons are the best responses at all tmes, they showed that effcency s approxmated by a Nash equlbrum payoff vector when the dscount factor s very close to unty. However, ths result s not satsfactory because ther proofs crucally depend on the assumpton of almost no dscountng. Hence, the robustness s regarded as an open queston n the case when the dscount factor s fxed and not very hgh. Ths s the man theme of ths paper. Wth our class restrcton, by usng only tt-for-tat strateges nstead of the more complcated Markovan ones, we can demonstrate the same result as that shown by Ely, Valmak, and Pccone. Tt-for-tat Nash equlbra have a useful property that the least upper bound of tt-for-tat Nash equlbrum payoffs for each player s ndependent of the dscount factor. Ths mples that whenever the approxmate effcency holds wth almost no dscountng, then ths holds even f the dscount factor s not very hgh. Ths s precsely what we wll show as the man theorem. Ths paper s organzed as follows. Secton 2 descrbes the model. Secton 3 defnes tt-for-tat strateges. In Secton 4, we characterze the tt-for-tat Nash equlbra and the least upper bounds. Secton 5 presents the man theorem. Fnally, Secton 6 consders an example where we argue the degree of mplct colluson sustaned n the long run by ntroducng the concept of statonary dstrbuton of acton profles.

5 5 2. The Model A two-person component game s defned by ( Au, ) {1,2}, where A denotes the fnte set of actons for each player {1, 2}, a A, A A1 A2, a ( a1, a2) A, u : A R, and u ( a ) s the payoff for player when the players choose the acton profle a A. Let α : A [0,1] denote a mxed acton for player. Let denote the set of mxed actons for player. Two nosy sgnals ω1 Ω 1 and ω 2 Ω 2 occur after the players acton choces, where Ω denotes the fnte set of possble ω, ω = ( ω1, ω2), and Ω=Ω 1 Ω 2. A sgnal profle ω Ω s randomly determned accordng to the probablty functon f ( a): Ω R + condtonal on a A. Let f( ω a) f( ω a), where. ω Ω We assume that the payoff functons are addtvely separable,.e., u ( a) = v ( a ) + w( a ) for all {1, 2} and all a A, where v : A R and w : A R. We assume that f ( ω a) does not depend on a. We wrte f( ω a) nstead of f( ω a) and call ω Ω the sgnal for player 's acton choce. An economc stuaton relevant to our model s the voluntary contrbuton of publc goods. The players supply ther publc goods that are perfectly dfferentated each other. An acton a A for each player mples the amount of

6 6 publc good that player produces. The producton cost for player 's publc good s gven by v ( a ) w( a ).. Player 's beneft from opponent 's publc good s gven by Fx A 1, A 2, Ω 1, and Ω 2 arbtrarly. We defne an nfntely repeated game by Γ ( u, f, δ) {1,2}, where δ (0,1) denotes the dscount factor for player. We allow the players to have dfferent dscount factors. Let ht () = ( a( τ), ω( τ )) t τ = 1 denote a hstory up to perod t, where a( τ ) = ( a1( τ), a2( τ)) A, and let ω( τ) = ( ω ( τ), ω ( τ)) Ω denote the acton profle and the sgnal profle n perod t, 1 2 respectvely. Let H = { h( t) t = 0,1,...} denote the set of hstores, where h (0) s the null hstory. A strategy for player {1,2} s defned as σ : H, where σ ( ht ( 1)) s the mxed acton for player n perod t when ht ( 1) occurs. Let σ = ( σ, σ ) denote a strategy profle. The payoff for player nduced by σ n Γ 1 2 s gven by t τ 1 ( σ) (1 δ) [ δ ( ( τ)) σ, Γ] τ = 1 v E u a, where E[ σ, Γ ] denotes the expectaton when the players conform to σ n Γ. Let Σ denote the set of strateges for player. A strategy profle σ s sad to be a Nash equlbrum n Γ f v ( σ ) v ( σ, σ ) for all {1, 2} and all σ Σ. If montorng s mperfect and prvate,.e., each player observes only the sgnal for the opponent s acton choce, t s approprate to replace the set of strateges Σ wth a

7 7 subset Σˆ Σ that s defned as the set of strateges σ such that σ ( ht ( )) s ndependent of ( a ( τ ), ω ( τ )) for all τ {1,..., t}. Let Σ ˆ =Σ ˆ ˆ 1 Σ 2. A strategy profle ˆ σ Σ s sad to be a Nash equlbrum n Γ wth prvate montorng f v ( σ ) v ( σ, σ ) for all {1,2} and all σ Σ ˆ. Clearly, a strategy profle s a Nash equlbrum n Γ wth prvate montorng f t s a Nash equlbrum n Γ. 3. Tt-For-Tat Equlbra For each {1, 2}, we arbtrarly fx two actons a A and a A for player. Let a = ( a, a ) A and 1 2 a = ( a, a ) A. We assume that the acton profle 1 2 a s effcent n that u ( a ) + u ( a ) u ( a) + u ( a) for all a A. Further, we assume that for every {1,2}, v a ( ) v( a ) < and w a > w a,.e., ( ) ( ) (1) u a a < u a a and (, ) (, ) u a a > u a a for all a A. (, ) (, ) In the voluntary contrbutons of publc goods, ths assumpton mples that player s producng a at a hgher cost than a but n a manner more benefcal to opponent. We consder strateges that are tt-for-tat n that () each player only chooses a and a at all tmes, and () each player 's mxed acton n any perod t 2 depends only on the sgnal

8 8 ω ( t 1) for opponent 's acton choce n the prevous perod t 1. Formally, a strategy σ for each player {1, 2} s sad to be tt-for-tat f for every t 1 and every ht ( 1) H, σ ( ht ( 1))( a) = 0 for all a A a a \{, } and for every t 2, every ht ( 1) H, and every h ( t 1) H \{ h( t 1)}, σ ( ht ( 1)) = σ ( h ( t 1)) whenever ω ( t 1) = ω ( t 1). A tt-for-tat strategy σ s represented by ( q, s ) where q [0,1], s : Ω [0,1], ( h(0))( a ) q σ =, ( h(0))( a ) 1 q σ =, for every t 2 and every ht ( 1) H, σ ht a = s ω t, and ( ( ))( ) 1 ( ( )) σ ht a = s ω t. ( ( ))( ) ( ( )) Accordng to ( q, s ), player chooses acton a (acton a ) wth probablty q (probablty 1 q ) n perod 1 and acton a (acton a ) wth probablty s ( ω ) (probablty 1 s( ω ) ) f he/she observes ω one perod before. Note that any tt-for-tat strategy for player belongs to the subset Σ ˆ. Let ( qs, ) denote a tt-for-tat strategy profle, where q= ( q1, q2) and s = ( s1, s2). We confne our attenton to the tt-for-tat Nash equlbra ( qs, ) n Γ such that each player chooses a wth postve probabltes n every perod t 2,.e., s( ω ( t)) > 0 for some ω Ω. Note that the necessary and suffcent condton for a

9 9 tt-for-tat strategy profle to be a Nash equlbrum n Γ s that each player has no ncentve to choose any other tt-for-tat strategy. Hence, t follows that a tt-for-tat strategy profle s a Nash equlbrum n Γ wth prvate montorng f and only f t s a Nash equlbrum n Γ. The set of tt-for-tat Nash equlbra also remans unchanged when montorng s mperfect and publc, that s, when the sgnals are observable to both players. 4. Characterzaton The defnton of tt-for-tat strateges, along wth the assumptons on the payoff functons and the sgnals n Secton 3, mples that n every perod t, the ncentve constrant of Nash equlbrum for each player s rrelevant to the hstory other than ω ( t 1) ; further, each player s ' current acton a ( t ) nfluences opponent s ' mxed acton σ ( ht ( )) n the next perod t + 1 through the determnaton of ω ( t). Snce each player s ' choce of a s at all tmes one of the best responses to any tt-for-tat Nash equlbrum ( qs, ), t follows that for every {1,2}, (2) v q s = v a + δ qw a + q w a (, ) ( ) (1 ){ ( ) (1 ) ( )} δ[ w( a) s( ω) w( a ){1 s( ω)}] f( ω a), ω Ω + + and that the ncentve constrant s replaced wth the maxmzaton of

10 10 (3) v a + δ w a s ω + w a s ω f ω a ω Ω ( ) [ ( ) ( ) ( ){1 ( )}] ( ) wth respect to a A. The defnton of tt-for-tat Nash equlbrum automatcally mples that both a and a maxmze ths value. Hence, we have proved the followng proposton. Proposton 1: A tt-for-tat strategy profle ( q, s ) s a Nash equlbrum n Γ f and only f for every {1, 2}, (4) v a + δ w a s ω + w a s ω f ω a ω Ω ( ) [ ( ) ( ) ( ){1 ( )}] ( ) = v a + w a s + w a s f a, ( ) δ[ ( ) ( ω) ( ){1 ( ω)}] ( ω ) ω Ω and for every a A, v a w a s w a s f a ( ) + δ[ ( ) ( ω) + ( ){1 ( ω)}] ( ω ) ω Ω v a + w a s + w a s f a. ( ) δ[ ( ) ( ω) ( ){1 ( ω)}] ( ω ) ω Ω From Proposton 1, t follows that there exsts a tt-for-tat Nash equlbrum n Γ f and only f for every {1,2}, there exsts a functon : Ω {0} such that µ R+ (5) (6), v a f a v a f a ( ) µ ( ω) ( ω ) = ( ) µ ( ω) ( ω ) ω Ω ω Ω for all v a f a v a f a ( ) µ ( ω) ( ω ) ( ) µ ( ω) ( ω ) ω Ω ω Ω a A, and

11 11 (7) for all ω Ω. 0 µ ( ω) δ{ w( a) w( a )} From the compactness and non-emptness of the set of possble µ satsfyng (5), (6), and (7), we can defne a value R ( δ ) = R( δ ; Γ ) R as max { u( a ) µ ( ω) f( ω a)} µ : Ω R+ {0} ω Ω subect to (5), (6), and (7). The followng proposton shows that R ( δ ) s regarded as the least upper bound of the tt-for-tat Nash equlbrum payoffs for each player {1,2}. Proposton 2: Suppose that there exsts a tt-for-tat Nash equlbrum n Γ,.e., for each {1, 2}, there exsts a functon µ that satsfes (5), (6), and (7). Then, there exsts a tt-for-tat Nash equlbrum ( qs ˆˆ, ) n Γ such that v ( qˆˆ, s) = R( δ ) for all {1, 2}. Moreover, for every tt-for-tat Nash equlbrum ( qs, ) n Γ, v ( q, s) R( δ ) for each {1, 2}. Proof: From (2), for every tt-for-tat Nash equlbrum ( qs, ) and every {1,2}, v q s u a w a w a s f a ; (, ) ( ) δ{ ( ) ( )}{1 ( ω)} ( ω ) ω Ω note that n the above equaton, the functon µ defned by µ ω = δ ω for all ω Ω ( ) { w( a) w( a )}{1 s( )}

12 12 satsfes (5), (6), and (7). Ths mples that v ( q, s) R( δ ). For each {1,2}, there exsts ˆ µ satsfyng (5), (6), and (7) such that u a ˆ f a R. ( ) µ ( ω) ( ω ) = ( δ) ω Ω We can construct a tt-for-tat Nash equlbrum ( qs ˆ, ˆ) n a manner that for every {1, 2}, ˆ µ ( ω) q ˆ = 1, and sˆ ( ω) = 1 δ { w( a ) w( a )} for all ω Ω. Clearly, v ( qˆˆ, s) = R( δ ) for each {1, 2}. Q.E.D. Remark 1 (Independence of Dscount Factors): Gven that the dscount factors are suffcently large, we can verfy that the least upper bound R ( δ ; Γ ) for each player does not depend on δ as follows. We defne R R by max { u( a ) µ ( ω) f( ω a)} µ : Ω R+ {0} ω Ω where we must note that R s ndependent of subect to (5) and (6), δ, and that R R( δ ). If δ s large enough for the restrcton (7) not to be bndng, then t { w( a) w( a )} holds that R = R( δ ). Let δ denote the mnmal dscount factor δ such that R = R( δ ). Snce R ( ) δ s nondecreasng wth respect to δ, we have shown that R( δ) = R( δ) = R for all δ δ.

13 13 Remark 2 (Exchangeablty): Proposton 1 mples that f ( qs, ) s a Nash equlbrum n a repeated game Γ= ( u, f, δ) {1,2}, then all tt-for-tat strateges for each player are the best responses to ( q, s ) n any repeated game such that player 's dscount factor s the same as that of Γ,.e., δ. Ths mples that the tt-for-tat Nash equlbrum noton satsfes the followng strong property of exchangeablty across dfferent games. Consder three repeated games gven by Γ ( u, f, δ) {1,2}, Γ ( u, f, δ ) {1,2}, and Γ = ( u1, f1, δ1, u 2, f 2, δ 2). If ( q, s ) s a Nash equlbrum n Γ and ( q, s ) s a Nash equlbrum n Γ, then ( q1, s1, q 2, s 2) s a Nash equlbrum n Γ, where the payoffs are unchanged n that v1( q1, s1, q 2, s 2; Γ ) = v1( q, s ; Γ ) and v 2 ( q 1, s 1, q 2, s 2 ; Γ ) = v 1 ( q, s; Γ ). 5. Approxmate Effcency We assume that for each {1,2}, (8) v a v a < w a w a for all ( ) ( ) δ{ ( ) ( )} a A a, \{ } whch s a necessary condton for (7),.e., the exstence of tt-for-tat Nash equlbra. Note that (8) s suffcent and (almost) necessary for the exstence of the effcent tt-for-tat Nash equlbrum gven that montorng s perfect. We wll show that (8) s also suffcent for the exstence of the approxmately effcent tt-for-tat Nash

14 14 equlbrum gven that montorng s almost perfect. Fx ε > 0 arbtrarly, whch s postve but close to zero. Assume that there exst Ω Ω and Ω Ω such that Ω Ω =, φ f( ω a ) 1 ε, ω Ω f( ω a ) 1 ε, ω Ω f( ω a) ε for all ω ω Ω f( ω a) ε for all Ω a A a \{ } a A a. \{ }, and Ths assumpton along wth a small ε > 0 mples that the sgnals are accurate n montorng. When player chooses a, t s almost certan that the sgnal ω for player 's acton choce belongs to Ω. When player chooses a, t s almost certan that t belongs to Ω. Hence, opponent can almost perfectly montor whether player has chosen a, a, or other actons. We specfy the functon µ as µ ω = δ for all ( ) { w( a) w( a )} ω Ω Ω, and µ ( ω ) = 0 for all ω Ω. By selectng µ ( ω ) from the nterval [0, { w( a) w( a )}] δ for each ω Ω n the approprate manner, we can make µ satsfy (5), (6), and (7), and we can make µ ( ω) f( ω a) close to zero. Ths mples that the least upper bound ( ) ω Ω R δ s approxmated by u a ( ). Hence, we have proved that whenever the sgnals are

15 15 suffcently accurate, effcency s approxmated by a tt-for-tat Nash equlbrum payoff vector even f the dscount factors are fxed and not very hgh. Theorem 3: If ε > 0 s suffcently close to zero and (8) holds for each {1, 2}, then there exsts a tt-for-tat Nash equlbrum ( qs, ) n Γ such that vqs (, ) s approxmated by ua ( ). 6. Example Fx p ( 1,1) and δ (0,1) arbtrarly. We nvestgate an example where 2 A = {0,1}, Ω = {0,1}, f(1 1) = f(0 0) = p, and δ = δ for each {1,2}. The followng matrx llustrates the component game, where we assume Z > Y > 0. Let a = (1,1) and (0,0) a =. All the assumptons necessary for the results of ths paper are satsfed,.e., the payoff functons are addtvely separable, a s effcent, and the nequaltes expressed n (1) hold X X X Z X + Y 0 X + Y X Z X + Y Z X + Y Z From the standard calculaton, a tt-for-tat strategy profle ( qs, ) satsfes (4) f

16 16 and only f (9) s (1) s (0) = Y δ (2p 1) Z for all {1, 2}. Ths along wth Proposton 1 mples that the nequaltes gven n (9) are necessary and suffcent for the tt-for-tat strategy profle to be a Nash equlbrum. Ths mples that there exsts a tt-for-tat Nash equlbrum ( qs, ) f and only f (10) Y δ (2p 1) Z. Suppose that (10) holds. The standard calculaton along wth (2) mples that f ( qs, ) s a tt-for-tat Nash equlbrum, the payoff nduced by ( qs, ) equals 1 p v( q, s) = X Y [(1 δ)(1 q) + δ{1 s(1)}] Z. 2p 1 The average payoff of the players equals v1( q, s) + v2( q, s) 1 p = X Y 2 2p 1 δ q + q s (1) + s (1) δ [(1 )(1 ) + {1 }]. Z Then, t follows that f ( qs, ) and ( q, s ) are tt-for-tat Nash equlbra and q1+ q2 = q 1+ q 2 and s 1 (1) + s 2 (1) = s 1 (1) + s 2 (1), then the average payoffs are the same between ( qs, ) and ( q, s ),.e., v1( q, s) + v2( q, s) v1( q, s ) + v2( q, s ) =. 2 2 Snce the tt-for-tat Nash equlbrum ( qs ˆˆ, ) specfed by (9), qˆ 1 = qˆ 2 = 1, and sˆ (1) = sˆ (1) = 1 nduces the least upper bound R ( δ ) for each player, t follows that p R ( ) ( ˆ, ˆ δ = R = v q s) = X Y. 2p 1

17 17 Ths along wth Proposton 2 mples that the least upper bound s ndependent of the dscount factor δ. As Theorem 3 shows, the least upper bound 1 p X Y 2p 1 converges to the effcent payoff X as p approaches unty. In order to demonstrate the degree of mplct colluson n the long run, t s approprate to exclude the payoffs n the early perods and concentrate on the statonary dstrbuton defned as ρ = ρ( s) : A [0,1], where ρ = ρ ω ω ω, ( a ) ( a) s1( 2) s2( 1) f( a) a A, ω Ω ρ ρ ω ω ω, ( a ) = ( a){1 s1( 2)}{1 s2( 1)} f( a) a A, ω Ω ρ ρ ω ω ω, ( a, a ) = ( a) s( ){1 s( )} f( a) a A, ω Ω and ρ ( a) = 0 for all a { a, a,( a, a ),( a, a )}. Let ρ = ρ( a, a ) denote a A the relatve frequency of player 's acton choce a n the long run. Note that ρ = K ρ + K {1 ρ }, where K s ( ω ) f ( ω a ) and ω Ω K s ( ω ) f ( ω a ). Hence, ω Ω K ( K K ) + K ρ =. 1 ( )( ) K K K K In ths example, the standard calculaton along wth (9) mples that Y(1 p) K = s(1) p+ s(0)(1 p) = s(1), δ Z(2p 1) Yp K = s(1)(1 p) + s(0) p = s(1), δ Z(2p 1) and therefore,

18 18 δ Z{ s(1) δ Z + s(1) Y} Yp ρ = δ Z Y ( δz Y)(2p 1) The average relatve frequency n the long run s expressed as δ Z{ s1(1) + s2(1)} Yp ρ1 + ρ2 =, 2( δz Y) ( δz Y)(2 p 1) whch depends only on s1(1) + s2(1). The least upper bound of the relatve frequences n the long run s acheved by the tt-for-tat Nash equlbrum ( qs ˆ, ˆ) satsfyng sˆ 1 (1) = sˆ 2 (1) = 1, and s equal to δ Z(2p 1) YP. Snce ths value s ncreasng wth respect to δ, t follows that ( δ Z Y)(2p 1) mplct colluson n the long run s more successful when the dscount factor s hgher. References Abreu, D., D. Pearce, and E. Stachett (1990): Toward a Theory of Dscounted Repeated Games wth Imperfect Montorng, Econometrca 58, Ely, J. and J. Välmäk (2002): A Robust Folk Theorem for the Prsoner s Dlemma, Journal of Economc Theory 102, Fudenberg, D., D. Levne, and E. Maskn (1994): The Folk Theorem wth Imperfect Publc Informaton, Econometrca 62, Fudenberg, D. and J. Trole (1991): Game Theory, MIT Press. Green, E. and R. Porter (1984): Non-cooperatve Colluson under Imperfect Prce Informaton, Econometrca 52, Osborne, M. and A. Rubnsten (1994): A Course n Game Theory, MIT Press. Pccone, M. (2002): The Repeated Prsoners Dlemma wth Imperfect Prvate Montorng, Journal of Economc Theory 102, Sekguch, T. (1997): Effcency n Repeated Prsoners Dlemma wth Prvate Montorng, Journal of Economc Theory 76,