1 Tt-For-Tat Equlbra n Dscounted Repeated Games wth Prvate Montorng Htosh Matsushma 1 Department of Economcs, Unversty of Tokyo 2 Aprl 24, 2007 Abstract We nvestgate nfntely repeated games wth mperfect prvate montorng. We focus on a class of games where the payoff functons are addtvely separable and the sgnal for montorng a player s acton does not depend on the other player s acton. Tt-for-tat strateges functon very well n ths class, accordng to whch each player s acton n each perod depends only on the sgnal for the opponent s acton one perod before. Wth almost perfect montorng, we show that even f the dscount factors are fxed low, effcency s approxmated by a tt-for-tat Nash equlbrum payoff vector. JEL Classfcaton Numbers: C72, C73, D82, H41 Keywords: Infntely Repeated Games, Prvate Montorng, Tt-For-Tat Strateges, Fxed Dscount Factors, Approxmate Effcency 1 Ths research was supported by a Grant-In-Ad for Scentfc Research (KAKENHI 15330036) from the Japan Socety for the Promoton of Scence (JSPS) and the Mnstry of Educaton, Culture, Sports, Scence and Technology (MEXT) of the Japanese government as well as a grant from the Center for Advanced Research n Fnance (CARF) at the Unversty of Tokyo. 2 Department of Economcs, Unversty of Tokyo, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan. E-mal: htosh@e.u-tokyo.ac.p
2 1. Introducton We nvestgate two-player nfntely repeated games wth dscountng. We assume that montorng s mperfect n that each player cannot observe the opponent s acton choce but can mperfectly montor t by observng a nosy sgnal. Ths sgnal s randomly determned accordng to the probablty functon condtonal on the opponent s acton choce. We also assume that montorng s prvate n that the sgnals that each player observes cannot be observed by the opponent. We focus on a class of games where the payoff functons are addtvely separable and the sgnals for montorng any player s acton choces are ndependent of the other player s acton choces. Wth ths class restrcton, we nvestgate the possblty that effcency s approxmated by a Nash equlbrum payoff vector when montorng s almost perfect, and we arrve at the followng affrmatve answer. In the case of almost perfect prvate montorng, there exsts a smple tt-for-tat Nash equlbrum that approxmately nduces effcency, accordng to whch, each player s acton choce n each perod depends only on the sgnal for the opponent s acton choce one perod before. The man contrbuton of ths paper s that ths postve result holds even f the dscount factor s fxed and not very hgh. It s well known that f each player perfectly montors the opponent s acton
3 choces and the dscount factor s not very low, effcency s acheved by a Nash equlbrum. See Fudenberg and Trole (1991) and Osborne and Rubnsten (1994). In order to clarfy the robustness wth respect to montorng ablty, t s mportant to answer the queston of whether effcency s approxmated by a Nash equlbrum payoff vector f montorng s almost perfect but mperfect. In the case of publc montorng where the sgnals are observable to both players, t s now not dffcult to answer n the affrmatve due to prevous works such as Green and Porter (1984); Abreu, Pearce, and Stachett (1990), and Fudenberg, Levne, and Maskn (1986). In the case of prvate montorng, however, ths robustness ssue remans unresolved and s much more substantal. In fact, the prevous works have gven only partal answers for the prvate montorng case. Frst, Sekguch (1997) showed an example n whch effcency can be approxmated by a Nash equlbrum payoff vector when montorng s almost perfect. Sekguch demonstrated an dea of constructon wth publc randomzaton, whch allowed equlbrum strateges to depend on hstores n a non-recursve manner. Ely and Valmak (2002) and Pccone (2002) demonstrated another dea of constructon on the bass of recursve Markovan technques, whch was applcable to a class of games wder than that presented n Sekguch. By constructng Markovan strateges to whch
4 both the cooperatve and non-cooperatve actons are the best responses at all tmes, they showed that effcency s approxmated by a Nash equlbrum payoff vector when the dscount factor s very close to unty. However, ths result s not satsfactory because ther proofs crucally depend on the assumpton of almost no dscountng. Hence, the robustness s regarded as an open queston n the case when the dscount factor s fxed and not very hgh. Ths s the man theme of ths paper. Wth our class restrcton, by usng only tt-for-tat strateges nstead of the more complcated Markovan ones, we can demonstrate the same result as that shown by Ely, Valmak, and Pccone. Tt-for-tat Nash equlbra have a useful property that the least upper bound of tt-for-tat Nash equlbrum payoffs for each player s ndependent of the dscount factor. Ths mples that whenever the approxmate effcency holds wth almost no dscountng, then ths holds even f the dscount factor s not very hgh. Ths s precsely what we wll show as the man theorem. Ths paper s organzed as follows. Secton 2 descrbes the model. Secton 3 defnes tt-for-tat strateges. In Secton 4, we characterze the tt-for-tat Nash equlbra and the least upper bounds. Secton 5 presents the man theorem. Fnally, Secton 6 consders an example where we argue the degree of mplct colluson sustaned n the long run by ntroducng the concept of statonary dstrbuton of acton profles.
5 2. The Model A two-person component game s defned by ( Au, ) {1,2}, where A denotes the fnte set of actons for each player {1, 2}, a A, A A1 A2, a ( a1, a2) A, u : A R, and u ( a ) s the payoff for player when the players choose the acton profle a A. Let α : A [0,1] denote a mxed acton for player. Let denote the set of mxed actons for player. Two nosy sgnals ω1 Ω 1 and ω 2 Ω 2 occur after the players acton choces, where Ω denotes the fnte set of possble ω, ω = ( ω1, ω2), and Ω=Ω 1 Ω 2. A sgnal profle ω Ω s randomly determned accordng to the probablty functon f ( a): Ω R + condtonal on a A. Let f( ω a) f( ω a), where. ω Ω We assume that the payoff functons are addtvely separable,.e., u ( a) = v ( a ) + w( a ) for all {1, 2} and all a A, where v : A R and w : A R. We assume that f ( ω a) does not depend on a. We wrte f( ω a) nstead of f( ω a) and call ω Ω the sgnal for player 's acton choce. An economc stuaton relevant to our model s the voluntary contrbuton of publc goods. The players supply ther publc goods that are perfectly dfferentated each other. An acton a A for each player mples the amount of
6 publc good that player produces. The producton cost for player 's publc good s gven by v ( a ) w( a ).. Player 's beneft from opponent 's publc good s gven by Fx A 1, A 2, Ω 1, and Ω 2 arbtrarly. We defne an nfntely repeated game by Γ ( u, f, δ) {1,2}, where δ (0,1) denotes the dscount factor for player. We allow the players to have dfferent dscount factors. Let ht () = ( a( τ), ω( τ )) t τ = 1 denote a hstory up to perod t, where a( τ ) = ( a1( τ), a2( τ)) A, and let ω( τ) = ( ω ( τ), ω ( τ)) Ω denote the acton profle and the sgnal profle n perod t, 1 2 respectvely. Let H = { h( t) t = 0,1,...} denote the set of hstores, where h (0) s the null hstory. A strategy for player {1,2} s defned as σ : H, where σ ( ht ( 1)) s the mxed acton for player n perod t when ht ( 1) occurs. Let σ = ( σ, σ ) denote a strategy profle. The payoff for player nduced by σ n Γ 1 2 s gven by t τ 1 ( σ) (1 δ) [ δ ( ( τ)) σ, Γ] τ = 1 v E u a, where E[ σ, Γ ] denotes the expectaton when the players conform to σ n Γ. Let Σ denote the set of strateges for player. A strategy profle σ s sad to be a Nash equlbrum n Γ f v ( σ ) v ( σ, σ ) for all {1, 2} and all σ Σ. If montorng s mperfect and prvate,.e., each player observes only the sgnal for the opponent s acton choce, t s approprate to replace the set of strateges Σ wth a
7 subset Σˆ Σ that s defned as the set of strateges σ such that σ ( ht ( )) s ndependent of ( a ( τ ), ω ( τ )) for all τ {1,..., t}. Let Σ ˆ =Σ ˆ ˆ 1 Σ 2. A strategy profle ˆ σ Σ s sad to be a Nash equlbrum n Γ wth prvate montorng f v ( σ ) v ( σ, σ ) for all {1,2} and all σ Σ ˆ. Clearly, a strategy profle s a Nash equlbrum n Γ wth prvate montorng f t s a Nash equlbrum n Γ. 3. Tt-For-Tat Equlbra For each {1, 2}, we arbtrarly fx two actons a A and a A for player. Let a = ( a, a ) A and 1 2 a = ( a, a ) A. We assume that the acton profle 1 2 a s effcent n that u ( a ) + u ( a ) u ( a) + u ( a) for all a A. Further, we assume 1 2 1 2 that for every {1,2}, v a ( ) v( a ) < and w a > w a,.e., ( ) ( ) (1) u a a < u a a and (, ) (, ) u a a > u a a for all a A. (, ) (, ) In the voluntary contrbutons of publc goods, ths assumpton mples that player s producng a at a hgher cost than a but n a manner more benefcal to opponent. We consder strateges that are tt-for-tat n that () each player only chooses a and a at all tmes, and () each player 's mxed acton n any perod t 2 depends only on the sgnal
8 ω ( t 1) for opponent 's acton choce n the prevous perod t 1. Formally, a strategy σ for each player {1, 2} s sad to be tt-for-tat f for every t 1 and every ht ( 1) H, σ ( ht ( 1))( a) = 0 for all a A a a \{, } and for every t 2, every ht ( 1) H, and every h ( t 1) H \{ h( t 1)}, σ ( ht ( 1)) = σ ( h ( t 1)) whenever ω ( t 1) = ω ( t 1). A tt-for-tat strategy σ s represented by ( q, s ) where q [0,1], s : Ω [0,1], ( h(0))( a ) q σ =, ( h(0))( a ) 1 q σ =, for every t 2 and every ht ( 1) H, σ ht a = s ω t, and ( ( ))( ) 1 ( ( )) σ ht a = s ω t. ( ( ))( ) ( ( )) Accordng to ( q, s ), player chooses acton a (acton a ) wth probablty q (probablty 1 q ) n perod 1 and acton a (acton a ) wth probablty s ( ω ) (probablty 1 s( ω ) ) f he/she observes ω one perod before. Note that any tt-for-tat strategy for player belongs to the subset Σ ˆ. Let ( qs, ) denote a tt-for-tat strategy profle, where q= ( q1, q2) and s = ( s1, s2). We confne our attenton to the tt-for-tat Nash equlbra ( qs, ) n Γ such that each player chooses a wth postve probabltes n every perod t 2,.e., s( ω ( t)) > 0 for some ω Ω. Note that the necessary and suffcent condton for a
9 tt-for-tat strategy profle to be a Nash equlbrum n Γ s that each player has no ncentve to choose any other tt-for-tat strategy. Hence, t follows that a tt-for-tat strategy profle s a Nash equlbrum n Γ wth prvate montorng f and only f t s a Nash equlbrum n Γ. The set of tt-for-tat Nash equlbra also remans unchanged when montorng s mperfect and publc, that s, when the sgnals are observable to both players. 4. Characterzaton The defnton of tt-for-tat strateges, along wth the assumptons on the payoff functons and the sgnals n Secton 3, mples that n every perod t, the ncentve constrant of Nash equlbrum for each player s rrelevant to the hstory other than ω ( t 1) ; further, each player s ' current acton a ( t ) nfluences opponent s ' mxed acton σ ( ht ( )) n the next perod t + 1 through the determnaton of ω ( t). Snce each player s ' choce of a s at all tmes one of the best responses to any tt-for-tat Nash equlbrum ( qs, ), t follows that for every {1,2}, (2) v q s = v a + δ qw a + q w a (, ) ( ) (1 ){ ( ) (1 ) ( )} δ[ w( a) s( ω) w( a ){1 s( ω)}] f( ω a), ω Ω + + and that the ncentve constrant s replaced wth the maxmzaton of
10 (3) v a + δ w a s ω + w a s ω f ω a ω Ω ( ) [ ( ) ( ) ( ){1 ( )}] ( ) wth respect to a A. The defnton of tt-for-tat Nash equlbrum automatcally mples that both a and a maxmze ths value. Hence, we have proved the followng proposton. Proposton 1: A tt-for-tat strategy profle ( q, s ) s a Nash equlbrum n Γ f and only f for every {1, 2}, (4) v a + δ w a s ω + w a s ω f ω a ω Ω ( ) [ ( ) ( ) ( ){1 ( )}] ( ) = v a + w a s + w a s f a, ( ) δ[ ( ) ( ω) ( ){1 ( ω)}] ( ω ) ω Ω and for every a A, v a w a s w a s f a ( ) + δ[ ( ) ( ω) + ( ){1 ( ω)}] ( ω ) ω Ω v a + w a s + w a s f a. ( ) δ[ ( ) ( ω) ( ){1 ( ω)}] ( ω ) ω Ω From Proposton 1, t follows that there exsts a tt-for-tat Nash equlbrum n Γ f and only f for every {1,2}, there exsts a functon : Ω {0} such that µ R+ (5) (6), v a f a v a f a ( ) µ ( ω) ( ω ) = ( ) µ ( ω) ( ω ) ω Ω ω Ω for all v a f a v a f a ( ) µ ( ω) ( ω ) ( ) µ ( ω) ( ω ) ω Ω ω Ω a A, and
11 (7) for all ω Ω. 0 µ ( ω) δ{ w( a) w( a )} From the compactness and non-emptness of the set of possble µ satsfyng (5), (6), and (7), we can defne a value R ( δ ) = R( δ ; Γ ) R as max { u( a ) µ ( ω) f( ω a)} µ : Ω R+ {0} ω Ω subect to (5), (6), and (7). The followng proposton shows that R ( δ ) s regarded as the least upper bound of the tt-for-tat Nash equlbrum payoffs for each player {1,2}. Proposton 2: Suppose that there exsts a tt-for-tat Nash equlbrum n Γ,.e., for each {1, 2}, there exsts a functon µ that satsfes (5), (6), and (7). Then, there exsts a tt-for-tat Nash equlbrum ( qs ˆˆ, ) n Γ such that v ( qˆˆ, s) = R( δ ) for all {1, 2}. Moreover, for every tt-for-tat Nash equlbrum ( qs, ) n Γ, v ( q, s) R( δ ) for each {1, 2}. Proof: From (2), for every tt-for-tat Nash equlbrum ( qs, ) and every {1,2}, v q s u a w a w a s f a ; (, ) ( ) δ{ ( ) ( )}{1 ( ω)} ( ω ) ω Ω note that n the above equaton, the functon µ defned by µ ω = δ ω for all ω Ω ( ) { w( a) w( a )}{1 s( )}
12 satsfes (5), (6), and (7). Ths mples that v ( q, s) R( δ ). For each {1,2}, there exsts ˆ µ satsfyng (5), (6), and (7) such that u a ˆ f a R. ( ) µ ( ω) ( ω ) = ( δ) ω Ω We can construct a tt-for-tat Nash equlbrum ( qs ˆ, ˆ) n a manner that for every {1, 2}, ˆ µ ( ω) q ˆ = 1, and sˆ ( ω) = 1 δ { w( a ) w( a )} for all ω Ω. Clearly, v ( qˆˆ, s) = R( δ ) for each {1, 2}. Q.E.D. Remark 1 (Independence of Dscount Factors): Gven that the dscount factors are suffcently large, we can verfy that the least upper bound R ( δ ; Γ ) for each player does not depend on δ as follows. We defne R R by max { u( a ) µ ( ω) f( ω a)} µ : Ω R+ {0} ω Ω where we must note that R s ndependent of subect to (5) and (6), δ, and that R R( δ ). If δ s large enough for the restrcton (7) not to be bndng, then t { w( a) w( a )} holds that R = R( δ ). Let δ denote the mnmal dscount factor δ such that R = R( δ ). Snce R ( ) δ s nondecreasng wth respect to δ, we have shown that R( δ) = R( δ) = R for all δ δ.
13 Remark 2 (Exchangeablty): Proposton 1 mples that f ( qs, ) s a Nash equlbrum n a repeated game Γ= ( u, f, δ) {1,2}, then all tt-for-tat strateges for each player are the best responses to ( q, s ) n any repeated game such that player 's dscount factor s the same as that of Γ,.e., δ. Ths mples that the tt-for-tat Nash equlbrum noton satsfes the followng strong property of exchangeablty across dfferent games. Consder three repeated games gven by Γ ( u, f, δ) {1,2}, Γ ( u, f, δ ) {1,2}, and Γ = ( u1, f1, δ1, u 2, f 2, δ 2). If ( q, s ) s a Nash equlbrum n Γ and ( q, s ) s a Nash equlbrum n Γ, then ( q1, s1, q 2, s 2) s a Nash equlbrum n Γ, where the payoffs are unchanged n that v1( q1, s1, q 2, s 2; Γ ) = v1( q, s ; Γ ) and v 2 ( q 1, s 1, q 2, s 2 ; Γ ) = v 1 ( q, s; Γ ). 5. Approxmate Effcency We assume that for each {1,2}, (8) v a v a < w a w a for all ( ) ( ) δ{ ( ) ( )} a A a, \{ } whch s a necessary condton for (7),.e., the exstence of tt-for-tat Nash equlbra. Note that (8) s suffcent and (almost) necessary for the exstence of the effcent tt-for-tat Nash equlbrum gven that montorng s perfect. We wll show that (8) s also suffcent for the exstence of the approxmately effcent tt-for-tat Nash
14 equlbrum gven that montorng s almost perfect. Fx ε > 0 arbtrarly, whch s postve but close to zero. Assume that there exst Ω Ω and Ω Ω such that Ω Ω =, φ f( ω a ) 1 ε, ω Ω f( ω a ) 1 ε, ω Ω f( ω a) ε for all ω ω Ω f( ω a) ε for all Ω a A a \{ } a A a. \{ }, and Ths assumpton along wth a small ε > 0 mples that the sgnals are accurate n montorng. When player chooses a, t s almost certan that the sgnal ω for player 's acton choce belongs to Ω. When player chooses a, t s almost certan that t belongs to Ω. Hence, opponent can almost perfectly montor whether player has chosen a, a, or other actons. We specfy the functon µ as µ ω = δ for all ( ) { w( a) w( a )} ω Ω Ω, and µ ( ω ) = 0 for all ω Ω. By selectng µ ( ω ) from the nterval [0, { w( a) w( a )}] δ for each ω Ω n the approprate manner, we can make µ satsfy (5), (6), and (7), and we can make µ ( ω) f( ω a) close to zero. Ths mples that the least upper bound ( ) ω Ω R δ s approxmated by u a ( ). Hence, we have proved that whenever the sgnals are
15 suffcently accurate, effcency s approxmated by a tt-for-tat Nash equlbrum payoff vector even f the dscount factors are fxed and not very hgh. Theorem 3: If ε > 0 s suffcently close to zero and (8) holds for each {1, 2}, then there exsts a tt-for-tat Nash equlbrum ( qs, ) n Γ such that vqs (, ) s approxmated by ua ( ). 6. Example Fx p ( 1,1) and δ (0,1) arbtrarly. We nvestgate an example where 2 A = {0,1}, Ω = {0,1}, f(1 1) = f(0 0) = p, and δ = δ for each {1,2}. The followng matrx llustrates the component game, where we assume Z > Y > 0. Let a = (1,1) and (0,0) a =. All the assumptons necessary for the results of ths paper are satsfed,.e., the payoff functons are addtvely separable, a s effcent, and the nequaltes expressed n (1) hold. 1 0 1 X X X Z X + Y 0 X + Y X Z X + Y Z X + Y Z From the standard calculaton, a tt-for-tat strategy profle ( qs, ) satsfes (4) f
16 and only f (9) s (1) s (0) = Y δ (2p 1) Z for all {1, 2}. Ths along wth Proposton 1 mples that the nequaltes gven n (9) are necessary and suffcent for the tt-for-tat strategy profle to be a Nash equlbrum. Ths mples that there exsts a tt-for-tat Nash equlbrum ( qs, ) f and only f (10) Y δ (2p 1) Z. Suppose that (10) holds. The standard calculaton along wth (2) mples that f ( qs, ) s a tt-for-tat Nash equlbrum, the payoff nduced by ( qs, ) equals 1 p v( q, s) = X Y [(1 δ)(1 q) + δ{1 s(1)}] Z. 2p 1 The average payoff of the players equals v1( q, s) + v2( q, s) 1 p = X Y 2 2p 1 δ q + q s (1) + s (1) δ 2 2 1 2 1 2 [(1 )(1 ) + {1 }]. Z Then, t follows that f ( qs, ) and ( q, s ) are tt-for-tat Nash equlbra and q1+ q2 = q 1+ q 2 and s 1 (1) + s 2 (1) = s 1 (1) + s 2 (1), then the average payoffs are the same between ( qs, ) and ( q, s ),.e., v1( q, s) + v2( q, s) v1( q, s ) + v2( q, s ) =. 2 2 Snce the tt-for-tat Nash equlbrum ( qs ˆˆ, ) specfed by (9), qˆ 1 = qˆ 2 = 1, and sˆ (1) = sˆ (1) = 1 nduces the least upper bound R ( δ ) for each player, t follows that 1 2 1 p R ( ) ( ˆ, ˆ δ = R = v q s) = X Y. 2p 1
17 Ths along wth Proposton 2 mples that the least upper bound s ndependent of the dscount factor δ. As Theorem 3 shows, the least upper bound 1 p X Y 2p 1 converges to the effcent payoff X as p approaches unty. In order to demonstrate the degree of mplct colluson n the long run, t s approprate to exclude the payoffs n the early perods and concentrate on the statonary dstrbuton defned as ρ = ρ( s) : A [0,1], where ρ = ρ ω ω ω, ( a ) ( a) s1( 2) s2( 1) f( a) a A, ω Ω ρ ρ ω ω ω, ( a ) = ( a){1 s1( 2)}{1 s2( 1)} f( a) a A, ω Ω ρ ρ ω ω ω, ( a, a ) = ( a) s( ){1 s( )} f( a) a A, ω Ω and ρ ( a) = 0 for all a { a, a,( a, a ),( a, a )}. Let ρ = ρ( a, a ) denote 1 2 1 2 a A the relatve frequency of player 's acton choce a n the long run. Note that ρ = K ρ + K {1 ρ }, where K s ( ω ) f ( ω a ) and ω Ω K s ( ω ) f ( ω a ). Hence, ω Ω K ( K K ) + K ρ =. 1 ( )( ) K K K K In ths example, the standard calculaton along wth (9) mples that Y(1 p) K = s(1) p+ s(0)(1 p) = s(1), δ Z(2p 1) Yp K = s(1)(1 p) + s(0) p = s(1), δ Z(2p 1) and therefore,
18 δ Z{ s(1) δ Z + s(1) Y} Yp ρ =. 2 2 2 δ Z Y ( δz Y)(2p 1) The average relatve frequency n the long run s expressed as δ Z{ s1(1) + s2(1)} Yp ρ1 + ρ2 =, 2( δz Y) ( δz Y)(2 p 1) whch depends only on s1(1) + s2(1). The least upper bound of the relatve frequences n the long run s acheved by the tt-for-tat Nash equlbrum ( qs ˆ, ˆ) satsfyng sˆ 1 (1) = sˆ 2 (1) = 1, and s equal to δ Z(2p 1) YP. Snce ths value s ncreasng wth respect to δ, t follows that ( δ Z Y)(2p 1) mplct colluson n the long run s more successful when the dscount factor s hgher. References Abreu, D., D. Pearce, and E. Stachett (1990): Toward a Theory of Dscounted Repeated Games wth Imperfect Montorng, Econometrca 58, 1041 1063. Ely, J. and J. Välmäk (2002): A Robust Folk Theorem for the Prsoner s Dlemma, Journal of Economc Theory 102, 84 105. Fudenberg, D., D. Levne, and E. Maskn (1994): The Folk Theorem wth Imperfect Publc Informaton, Econometrca 62, 997 1040. Fudenberg, D. and J. Trole (1991): Game Theory, MIT Press. Green, E. and R. Porter (1984): Non-cooperatve Colluson under Imperfect Prce Informaton, Econometrca 52, 87 100. Osborne, M. and A. Rubnsten (1994): A Course n Game Theory, MIT Press. Pccone, M. (2002): The Repeated Prsoners Dlemma wth Imperfect Prvate Montorng, Journal of Economc Theory 102, 70 83. Sekguch, T. (1997): Effcency n Repeated Prsoners Dlemma wth Prvate Montorng, Journal of Economc Theory 76, 345 361.