Negatively Correlated Bandits

Transcription

1 Discussion Pape No. 43 Negatively Coelated Bandits Nicolas Klein* Sven Rady** * Munich Gaduate School of Economics, Kaulbachst. 45, D Munich, Gemany; kleinnic@yahoo.com. **Depatment of Economics, Univesity of Munich, Kaulbachst. 45, D Munich, Gemany; sven.ady@lz.uni-muenchen.de August 008 Financial suppot fom the Deutsche Foschungsgemeinschaft though SFB/TR 15 is gatefully acknowledged. Sondefoschungsbeeich/Tansegio 15 Univesität Mannheim Feie Univesität Belin Humboldt-Univesität zu Belin Ludwig-Maximilians-Univesität München Rheinische Fiedich-Wilhelms-Univesität Bonn Zentum fü Euopäische Witschaftsfoschung Mannheim Speake: Pof. D. Us Schweize. Depatment of Economics Univesity of Bonn D Bonn, Phone: +49(08)7390 Fax: +49(08)7391

2 Negatively Coelated Bandits Nicolas Klein Sven Rady This vesion: August 1, 008 Abstact We analyze a two-playe game of stategic expeimentation with two-amed bandits. Each playe has to decide in continuous time whethe to use a safe am with a known payoff o a isky am whose likelihood of deliveing payoffs is initially unknown. The quality of the isky ams is pefectly negatively coelated between playes. In maked contast to the case whee both isky ams ae of the same type, we find that leaning will be complete in any Makov pefect equilibium if the stakes exceed a cetain theshold, and that all equilibia ae in cutoff stategies. Fo low stakes, the equilibium is unique, symmetic, and coincides with the planne s solution. Fo high stakes, the equilibium is unique, symmetic, and tantamount to myopic behavio. Fo intemediate stakes, thee is a continuum of equilibia. Keywods: Stategic Expeimentation, Two-Amed Bandit, Exponential Distibution, Poisson Pocess, Bayesian Leaning, Makov Pefect Equilibium. JEL Classification Numbes: C73, D83, O3. We ae gateful to Philippe Aghion, Patick Bolton, Kalyan Chattejee, Matin Cipps, Matthias Dewatipont, Jan Eeckhout, Floian Englmai, Eduado Faingold, Philipp Kiche, Geoge Mailath, Timofiy Mylovanov, Stephen Ryan, Klaus Schmidt, Lay Samuelson, as well as semina paticipants at Bonn, Munich, UPenn, Yale, HEC Pais, the 007 SFB/TR 15 Summe School in Bonnbach, the 007 SFB/TR 15 Wokshop fo Young Reseaches in Bonn, the 008 SFB/TR 15 Meeting in Gummesbach, the 008 Noth Ameican Summe Meetings of the Econometic Society, the 008 Meeting of the Society fo Economic Dynamics and the Euopean Summe Symposium in Economic Theoy (ESSET) 008 fo helpful comments and suggestions. We thank the Depatment of Economics at the Univesity of Bonn, the Business and Public Policy Goup at the Whaton School, the UPenn Economics Depatment and the Studienzentum Gezensee fo thei hospitality. Financial suppot fom the Deutsche Foschungsgemeinschaft though SFB/TR 15 and GRK 801 is gatefully acknowledged. Munich Gaduate School of Economics, Kaulbachst. 45, D Munich, Gemany; kleinnic@yahoo.com. Depatment of Economics, Univesity of Munich, Kaulbachst. 45, D Munich, Gemany; sven.ady@lz.uni-muenchen.de.

3 1 Intoduction Stating with Rothschild (1974), two-amed bandit models have been used extensively in economics to fomalize the tade-off between expeimentation and exploitation in dynamic decision poblems with leaning; see Begemann and Välimäki (008) fo a suvey of this liteatue. The use of the two-amed bandit famewok as a canonical model of stategic expeimentation in teams is moe ecent: Bolton and Hais (1999, 000) analyze the case of Bownian motion bandits, while Kelle, Rady and Cipps (005) and Kelle and Rady (007) analyze bandits whee payoffs ae govened by Poisson pocesses. These papes assume pefect positive coelation acoss bandits; what is good news to any given playe is assumed to be good news fo eveybody else. Thee ae many situations, howeve, whee one man s boon is the othe one s bane. Think of a suit at law, fo instance: whateve is good news fo one paty is bad news fo the othe. O conside two fims pusuing eseach and development unde diffeent, incompatible woking hypotheses. One phamaceutical company, fo example, might base its dug development stategy on the hypothesis that the cause of a paticula disease is a vius, while the othe might see a bacteium as the cause. An appopiate model of stategic expeimentation in such a situation must assume pefect negative coelation acoss bandits. This we popose to do in the pesent pape. 1 We conside two playes, each facing a continuous-time exponential bandit as in Kelle, Rady and Cipps (005). One am is safe, geneating a known payoff pe unit of time. The othe am is isky, and can be good o bad. If it is good, it geneates lump-sum payoffs afte exponentially distibuted andom times; if it is bad, it neve geneates any payoff. A good isky am dominates the safe one in tems of aveage payoff pe unit of time, wheeas the safe am dominates a bad isky one. At the stat of the game, the playes do not know the type of thei isky am, but it is common knowledge that exactly one isky am is good. Each playe s actions and payoffs ae pefectly obsevable to the othe playe. Stating fom a common pio, the playes posteio beliefs thus agee at all times. The dynamics of these beliefs ae easy to descibe. If both playes play safe, no new infomation is geneated and beliefs stay unchanged. If only one playe plays isky and he has no success, the posteio pobability that his isky am is the good one falls gadually ove time; if he obtains a lump-sum payoff, all uncetainty is esolved and beliefs become 1 Thee is a decision-theoetic liteatue on coelated bandits which analyzes coelation acoss diffeent ams of a bandit opeated by a single agent; see e.g. Camago (007) fo a ecent contibution to this liteatue, o Pastoino (005) fo economic applications. Ou focus hee is quite diffeent, though, in that we ae concened with coelation between diffeent bandits opeated by two playes who inteact stategically. 1

4 degeneate at the tue state of the wold. If both playes play isky, finally, and thee is no success on eithe am, this is again uninfomative about the state of the wold, so beliefs ae constant up to the andom time when the fist success occus. It is impotant to note that a success on one playe s isky am is always bad news fo the othe playe, while lack of success gadually makes the othe playe moe optimistic. We estict playes to stationay Makov stategies with the common posteio belief as the state vaiable. As is well known, this estiction is without loss of geneality in the decision poblem of a single agent expeimenting in isolation, whose optimal policy is given by a cutoff stategy, i.e. has him play isky at beliefs moe optimistic than some theshold, and safe othewise. As the same stuctue pevails in the optimization poblem of a utilitaian planne who maximizes the aveage of the two playes expected discounted payoffs, the Makov estiction is without loss of geneality thee as well. In the non-coopeative expeimentation game, the Makov estiction ules out histoy-dependent behavio that is familia fom the analysis of infinitely epeated games in discete time, yet technically quite inticate to fomalize in continuous time (Simon and Stinchcombe 1989, Begin 199, Begin and McLeod 1993). Imposing Makov pefection allows us to focus on the expeimentation tadeoff that the playes face and makes ou esults diectly compaable to those in the pevious liteatue on stategic expeimentation in bandits. Afte solving the planne s optimization poblem, we chaacteize the Makov pefect equilibia of the expeimentation game. We find that all Makov pefect equilibia ae in cutoff stategies. This is in stak contast to Kelle, Rady and Cipps (005), who find that thee is no such equilibium when all isky ams ae of the same type. On account of the symmety of the situation, it is not supising that thee always exists a symmetic equilibium, whee both playes use the same cutoff. This symmetic equilibium is unique. What is moe, based upon a chaacteization of the beliefs at which best esponses can change, we ae able to detemine the paamete values fo which thee is no othe equilibium besides the symmetic one. This uniqueness esult is again in shap contast to the multiplicity of equilibia in Kelle, Rady and Cipps (005). When stakes (as measued by the payoff advantage of a good isky am ove a safe one) ae sufficiently low, playes espective single-agent cutoffs ae such that a highe than chance of having the good isky am is equied fo a playe to play isky. In this case, the single-agent optimal stategies let at most one playe play isky at any given belief and, in the absence of a success, make this playe switch to the safe am at a point whee the othe playe s theshold fo playing isky is not yet eached. This means that neithe playe eve benefits fom any expeimentation by the othe, and so the single-agent optimal stategies ae mutually best esponses. As expeimentation on one bandit can only make

5 playes incementally moe optimistic about the othe bandit, the same logic applies to the planne s poblem, so both playes using thei single-agent optimal stategies is efficient. Finally, since any equilibium must give each playe at least his single-agent optimal payoff and any payoff highe than that would be incompatible with the uppe bound given by the planne s solution, the Makov pefect equilibium whee both playes use thei single-agent optimal cutoffs is unique. When stakes ae sufficiently high, a lowe than chance is sufficient fo a myopic playe, i.e. one who is meely inteested in maximizing cuent payoffs, to play isky. In this case, equilibium is again unique, but inefficient, with both playes applying the myopic cutoff stategy. With the stakes high, playes ae so eage to play isky that thee exists a ange of beliefs whee both ae expeimenting. As long as no lump-sum aives, no new infomation is then made available, and the playes ae effectively feezing the poblem in its cuent state. This, howeve, they ae only willing to do if the cuent state is attactive fom a myopic pespective. If the stakes ae intemediate in size, thee is a continuum of equilibia, each of them chaacteized by a single belief at which both playes change thei actions. As the stakes gadually incease and we move fom the low to the intemediate case, at fist, given any initial belief, thee still exists an equilibium that achieves the efficient outcome. As stakes incease futhe, thee then appeas a ange of initial beliefs fo which no equilibium achieves efficiency. As we move fom high stakes down to intemediate stakes, thee at fist always exists an equilibium that involves one playe behaving myopically. To achieve this, the othe playe has to bea the entie load of expeimentation by himself when the uncetainty is geatest. As stakes gadually gow lowe, howeve, the othe playe will at some point no longe be willing to bea this buden, and the equilibium disappeas. Given pefect obsevability of actions and payoffs, any infomation that a playe ganes via expeimentation is a public good. In contast to the case whee all isky ams ae of the same type, howeve, the esulting fee-ide poblem does not cause leaning to cease pematuely. In fact, we find complete leaning in equilibium (meaning that beliefs convege to the tuth almost suely) fo intemediate and high stakes, which is pecisely the paamete ange whee the planne s solution calls fo complete leaning. The intuition is quite staightfowad. If playes hold common beliefs and thee is pefect negative coelation between the types of the isky ams, it can neve be the case that both playes ae simultaneously vey Muto and Välimäki (006) and Rosenbeg, Solan and Vieille (007) study stategic expeimentation with two-amed bandits whee the playes actions ae publicly obsevable, but thei payoffs ae pivate infomation. To simplify the analysis, these authos assume that the decision to stop playing isky is ievesible. In ou model, playes can feely switch back and foth between the two ams. 3

6 pessimistic about thei espective pospects; with stakes sufficiently high, this implies that at least one playe must be using the isky am at any time, and so leaning neve stops. Thus, wheneve society places a lot of emphasis on uncoveing the tuth, as one may ague is the case with medical eseach o the justice system, ou analysis would suggest an advesaial setup was able to achieve this goal. In this espect, ou wok is elated to Dewatipont and Tiole (1999) who, in a moal hazad setting beaing no esemblance to ous, pose the question whethe it is socially bette to adjudicate disputes though a centalized system of gatheing evidence, which they assimilate to the inquisitoial system of Civil Law counties, o whethe the inteests of justice may be bette seved in a decentalized, advesaial system, as it is found in the Common Law counties. They show that, in a centalized system, it is not possible to give adequate incentives to make sue the tuth is uncoveed, and conclude that the Common Law system of gatheing infomation was theefoe supeio. Ou model povides an altenative famewok to ascetain the effectiveness of infomation-gatheing pocesses in a stategic setting whee the paties expected isky payoffs ae pefectly negatively coelated acoss states of the wold. Chattejee and Evans (004) analyze an R&D ace with two fims and two pojects in which, like in ou setup, thee is common knowledge that exactly one of these pojects will bea fuit if pusued long enough, and actions and payoffs ae obsevable. Thei discete-time model diffes fom ous in seveal espects, chief of which is the payoff extenality implied by the fims choices. In ou model, thee is no payoff ivaly between playes stategic inteaction aises out of puely infomational concens. Moeove, Chattejee and Evans allow fims to change thei pojects at any time, so that it is possible fo them to exploe the same poject. Ou analysis, by contast, pesumes that pojects of opposite type have been ievocably assigned to playes at the stat of the expeimentation game. 3 The est of the pape is stuctued as follows. Section intoduces the model. Section 3 solves the planne s poblem. Section 4 sets up the non-coopeative game. Section 5 discusses long-un popeties of leaning in equilibium. Section 6 chaacteizes the Makov pefect equilibia of the non-coopeative game. Section 7 concludes. Poofs ae povided in the appendix. 3 In the concluding emaks, we biefly epot on an extension of ou model in which we patially elax this assumption by giving the playes a sequential once-and-fo-all choice of bandits pio to the expeimentation game. 4

7 The Model Thee ae two playes, 1 and, each of whom faces a two-amed bandit poblem in continuous time. Bandits ae of the exponential type studied in Kelle, Rady and Cipps (005). One am is safe in that it yields a known payoff flow of s; the othe am is isky in that it is eithe good o bad. If it is bad, it neve yields any payoff; if it is good, it yields a lumpsum payoff with pobability λdt when used ove a time of length dt. Let g dt denote the coesponding expected payoff incement; thus, g is the poduct of the aival ate λ and the aveage size of a lump-sum payoff. The time-invaiant constants λ > 0 and g > 0 ae common knowledge. It is also common knowledge that exactly one bandit s isky am is good. To have an inteesting poblem, we assume that the expected payoff of a good isky am exceeds that of the safe am, wheeas the safe am is bette than a bad isky am, i.e., g > s > 0. Each playe chooses actions {k t } t 0 such that k t {0, 1} is measuable with espect to the infomation available at time t, with k t = 1 indicating use of the isky am, and k t = 0 use of the safe am. The stategic link between the two playes actions is povided by the assumption that playes pefectly obseve each othe s actions and payoffs. Thus, as the bandits ae pefectly negatively coelated, any infomation that is ganeed about the quality of the isky am is a public good. At the outset of the game, playes have a common pio about which of the isky ams is good. Since the esults of each playe s expeimentation ae public, playes shae a common posteio at all times. We wite p t fo the playes posteio pobability assessment at time t that playe 1 s isky am is good. The posteio belief jumps to 1 if thee has been a beakthough on playe 1 s bandit, and to 0 if thee has been a beakthough on playe s bandit, whee in eithe case it will emain eve afte. If thee has been no beakthough on eithe bandit by time T given the playes actions {k 1,t } 0 t T and {k,t } 0 t T, Bayes ule yields p T = p 0 e λ T 0 k 1,t dt p 0 e λ T 0 k 1,t dt + (1 p 0 )e λ T 0 k,t dt, and so the posteio belief satisfies p t = (k 1,t k,t )λp t (1 p t ) at almost all t up to the fist beakthough on a isky am. We estict playes to stationay Makov stategies with the common posteio belief as the state vaiable. The pecise definition of the space of Makov stategies available to each playe equies some cae. Suppose fo example that playe 1 plays isky at all beliefs p 1 5

8 and safe othewise, while playe plays isky at all beliefs p < 1 and safe othewise. Then the above expession fo the time deivative of the posteio belief tanslates into p t = f(p t ) whee f(p) = { λp(1 p) fo p < 1, λp(1 p) fo p 1. Thee is no continuous function t p t fom [0, [ to [0, 1] with p 0 = 1 that is diffeentiable almost eveywhee and satisfies p t = f(p t ) at almost all t. 4 So the above stategies do not induce a well-defined law of motion fo beliefs, which means that thee ae histoies of the game afte which these stategies do not pin down the playes actions. 5 To avoid this poblem, we impose specific one-sided continuity equiements on the playes Makov stategies: each playe s action is ight-continuous with left limits with espect to the posteio pobability of that playe s isky am being the good one. Moe pecisely, we define a Makov stategy fo playe 1 to be a function k 1 : [0, 1] {0, 1} such that k 1 (0) = 0 and k 1 1 (1) is the disjoint union of the singleton {1} and a finite numbe of left-closed intevals [p i,p i [. Symmetically, a Makov stategy fo playe is a function k : [0, 1] {0, 1} such that k (1) = 0 and k 1 (1) is the disjoint union of the singleton {0} and a finite numbe of ight-closed intevals ]p j,p i ]. Fixing the playes actions at the boundaies of the unit inteval is innocuous because we ae simply imposing the dominant actions unde subjective cetainty. Moe impotantly, ou specification of Makov stategies ensues that fo any pai (k 1,k ) of such stategies, the diffeential equation ṗ = [k 1 (p) k (p)]λp(1 p) has a unique global solution fo any initial value in the unit inteval, which implies that the law of motion fo the posteio belief, the playes actions k n,t = k n (p t ) and thei payoff functions u n : [0, 1] IR ae all well-defined. The latte ae given by [ } ] u 1 (p) = E e {k t 1 (p t )p t g + [1 k 1 (p t )]s dt p 0 = p, 0 [ } ] u (p) = E e {k t (p t )(1 p t )g + [1 k (p t )]s dt p 0 = p, 0 4 Simila poblems have been encounteed in the decision-theoetic liteatue. To guaantee a well-defined law of motion, Pesman (1990) allows fo simultaneous use of both ams, i.e. fo expeimentation intensities k t [0,1]. 5 Replacing the diffeential equation fo posteio beliefs by a diffeential inclusion as in Filippov (1988) does not help. Following this appoach, one eplaces the function f by a coespondence F with F(p) = {f(p)} fo p 1 and F(1 ) [ λ 4, λ 4 ], the convex hull of the left and ight limits of f at p = 1. A solution to the diffeential inclusion p t F(p t ) with p 0 = 1 is then easily found in the constant function p t 1. Howeve, this solution is not meaningful in the context of ou model since it is not compatible with Bayes ule unde the given stategies: if p t = 1 fo all t, the action pofile must always be (k 1,k ) = (1,0), in which case Bayes ule would imply a downwad tend in beliefs conditional on no success on playe 1 s bandit. 6

9 whee > 0 is the playes common discount ate and the expectation is taken with espect to the law of motion of posteio beliefs induced by the stategy pai (k 1,k ). By standad esults, ou specification of Makov stategies futhe implies that the Bellman equations appeaing in subsequent sections ae necessay and sufficient fo optimality. A pofile of Makov stategies (k 1, k ) is called symmetic if k 1 (p) = k (1 p) fo all p [0, 1]. A Makov stategy k 1 fo playe 1 is called a cutoff stategy with cutoff ˆp 1 if k 1 1 (1) = [ˆp 1, 1]. Analogously, a Makov stategy k fo playe is a cutoff stategy with cutoff ˆp if k 1 (1) = [0, ˆp ]. If playes wee myopic, i.e. meely maximizing cuent payoffs, playe 1 would use the cutoff p m = s g and playe the cutoff 1 pm. If they wee fowadlooking but expeimenting in isolation, playe 1 would optimally use the single-agent cutoff computed in Kelle, Rady and Cipps (005), p s = < (+λ)g λs pm, and playe the cutoff 1 p. In both cases, the esulting stategy pofile would be symmetic. We will find it useful below to distinguish thee cases depending on the size of the stakes involved, i.e. on the value of infomation as measued by the atio g, and on the paametes s λ and that goven the speed of esolution of uncetainty and the playe s impatience, espectively. We speak of low stakes if g < +λ +λ ; intemediate stakes if g <, and s +λ +λ s high stakes if g. These cases ae easily distinguished by the positions of the cutoffs pm s and p : stakes ae low if and only if p > 1; intemediate if and only if p 1 < pm ; and high if and only if p m 1. 3 The Planne s Poblem In this section, we examine a benevolent utilitaian social planne s behavio in ou setup. Poceeding exactly as Kelle, Rady and Cipps (005), we can wite the Bellman equation fo the maximization of the aveage payoff fom the two bandits as [ u(p) = s + max {k 1 B 1 (p,u) c 1(p) (k 1,k ) {0,1} ] + k [ B (p,u) c (p) whee B 1 (p,u) = λ p[g+s u(p) (1 p)u (p)] measues the expected leaning benefit of playing isky am 1, B (p,u) = λ (1 p)[g+s u(p) + pu (p)] the expected leaning benefit of playing isky am, c 1 (p) = s pg the oppotunity cost of playing isky am 1, and c (p) = s (1 p)g the oppotunity cost of playing isky am. Thus, the planne s ]}, poblem is linea in both k 1 and k, and he is maximizing sepaately ove k 1 and k. If it is optimal to set k 1 = k = 0, the value function woks out as u(p) = s. If it is [ optimal to set k 1 = k = 1, the Bellman equation educes to u(p) = λ g+s u(p) ] + g (, and so u(p) = u 11 = 1 g + λ s). λ+ 7

10 ODE If it is optimal to set k 1 = 0 and k = 1, the Bellman equation amounts to the fist-ode λp(1 p)u (p) [ + λ(1 p)]u(p) = 1 {[ + λ(1 p)]s + ( + λ)(1 p)g}. This has the solution whee C is some constant of integation. u(p) = 1 +λ [s + (1 p)g] + Cp λ (1 p) λ, Finally, if it is optimal to set k 1 = 1 and k = 0, the Bellman equation is tantamount to the fist-ode ODE λp(1 p)u 1(p) + ( + λp)u 1 (p) = 1 {( + λp)s + ( + λ)pg}, which is solved by u(p) = 1 +λ (s + pg) + C (1 p) λ p λ. Note that wheneve k 1 = k, the value function is constant as the planne does not cae which am is good. Fo the same eason, the poblem is symmetic aound p = 1. All the planne caes about is the uncetainty that stands in the way of his ealizing the uppe bound on the value function, g+s. Hence, intuitively, the planne s value function will admit its global minimum at p = 1, whee the uncetainty is stakest. It is clea that (k 1,k ) = (1, 0) will be optimal in a neighbohood of p = 1, and (k 1,k ) = (0, 1) in a neighbohood of p = 0. What is optimal at beliefs aound p = 1 depends on which of the two possible plateaus s and u 11 is highe. This in tun depends on the size of the stakes involved. In fact, s > u 11 if and only if stakes ae low, i.e. g < +λ. This is the case s +λ we conside fist. Poposition 3.1 (Planne s solution fo low stakes) If g s < +λ +λ, and hence p > 1, it is optimal fo the planne to apply the single-agent cutoffs p and 1 p, espectively, that is, to set (k 1,k ) = (0, 1) on [0, 1 p ], k 1 = k = 0 on ]1 p,p [, and (k 1,k ) = (1, 0) on [p, 1]. The coesponding value function is { ( ) +λ ( ) } 1 s + (1 p)g + (s p p λ 1 p λ g) if p 1 p, 1 p p u(p) = s if 1 p p p, { ( ) +λ ( ) } 1 s + pg + (s p 1 p λ p λ g) if p p. 1 p p 8

11 Figue 1 illustates the esult. Thus, when the value of infomation, as measued by g, is so low that the single-agent cutoff s p exceeds 1, it is optimal fo the planne to let the playes behave as though they wee single playes solving two sepaate, completely unconnected, poblems. 6 g+s s 0 1 p 1 p 1 Figue 1: The planne s value function fo g s < +λ +λ. Conditional on thee not being a beakthough, the belief will evolve accoding to λp(1 p) if p < 1 p, ṗ = 0 if 1 p p p, λp(1 p) if p > p. Let us suppose isky am 1 is good. If the initial belief p 0 < 1 p, then the posteio belief will convege to 1 p with pobability 1 as thee cannot be a beakthough on isky am. If 1 p p 0 p, the belief will emain unchanged at p 0. If p 0 > p, the belief will convege eithe to 1 o to p. If t is the length of time needed fo the belief to each p conditional on thee not being a beakthough on isky am 1, the pobability that the belief will convege to p is e λt. By Bayes ule, we have 1 pt p t = 1 p 0 of a beakthough, and so e λt = 1 p 0 p 0 p p 0 e λt leaning will emain incomplete) with pobability 1 p 0 p 0 with the counte-pobability. Analogous esults hold when isky am is good. in the absence. The belief will theefoe convege to p (and 1 p p, and to 1 (and hence the tuth) 1 p 6 This would be diffeent if playing the isky am could also lead to bad news events that tiggeed downwad jumps in beliefs. If, stating fom p, such a jump wee lage enough to take the belief below 1 p, then letting playe 1 play isky at beliefs somewhat below p would aise aveage payoffs if the oppotunity cost of doing so wee offset by infomational gains aising fom subsequent expeimentation by playe. 9

12 stakes. Next, we tun to the case whee u 11 s, which is obtained fo intemediate and high Poposition 3. (Planne s solution fo intemediate and high stakes) If g +λ s and hence p 1, it is optimal fo the planne to apply the cutoffs p = (+λ)s (+λ)g+λs [p, 1] and 1 p, espectively, that is, to set (k 1,k ) = (0, 1) on [0, p[, k 1 = k = 1 on [ p, 1 p], and (k 1,k ) = (1, 0) on ]1 p, 1]. The coesponding value function is u(p) = { s + (1 p)g + [ pg ( g + λ { s) +λ s + pg + [ pg ) +λ s]( p λ +λ p ) +λ ( s]( 1 p λ p +λ p 1 p ( 1 p 1 p ) } λ ) } λ if p p, if p p 1 p, if p 1 p. +λ, Figue illustates this esult fo g > +λ, which is tantamount to u s +λ 11 > s and easily seen to imply p < p < 1. To undestand why the planne has each playe use the isky am on a smalle inteval of beliefs than in the espective single-agent optimum, conside the effect of playe 1 s action on the aveage payoff when playe is playing isky. If the planne is indiffeent between playe 1 s actions at the belief p, it must be the case that B 1 ( p,u) = c 1( p). By value matching at the level u 11 and smooth pasting, this educes to λ p[g + s u 11] = c 1 ( p); thus, the possibility of a jump in the sum of the two playes payoffs fom u 11 to g + s exactly compensates fo the oppotunity cost of playe 1 using the isky am. Fo a playe 1 expeimenting in isolation, the coesponding equation eads λ p [g s] = c 1 (p ); at the single-agent optimal cutoff, the possibility of a jump in the payoff fom s to g exactly compensates fo the oppotunity cost of playe 1 using the isky am. When u 11 > s, the jump fom s to g is lage than the one fom u 11 to g + s, and so we cannot have p = p. That p must be geate than p follows fom the fact that the oppotunity cost of using playe 1 s isky am is deceasing in p. by The dynamics of beliefs conditional on thee not being a beakthough ae now given λp(1 p) if p < p, ṗ = 0 if p p 1 p, λp(1 p) if p > 1 p. Wheneve the stakes ae intemediate o high, theefoe, the planne shuts down incemental leaning on [ p, 1 p]. Yet he still leans the tuth with pobability 1 in the long un because this inteval is absobing fo the posteio belief pocess in the absence of a beakthough, and once it is eached, the planne uses both isky ams until all uncetainty is esolved. 10

13 g+s u 11 s 0 p p 1 1 p 1 p 1 Figue : The planne s value function fo g s > +λ +λ. In summay, when stakes ae intemediate o high, efficiency calls fo complete leaning, i.e., almost sue convegence of the posteio belief p t to the tuth. When stakes ae low, howeve, efficient leaning can be incomplete. 4 The Stategic Poblem Ou solution concept is Makov pefect equilibium, with the playes stategies as defined in Section above. Again poceeding as in Kelle, Rady and Cipps (005), we see that the following Bellman equation chaacteizes playe 1 s best esponses against his opponent s stategy k : u 1 (p) = s + k (p)β 1 (p,u 1 ) + max k 1 {0,1} k 1[b 1 (p,u 1 ) c 1 (p)], whee c 1 (p) = s pg is the oppotunity cost playe 1 has to bea when he plays isky, b 1 (p,u 1 ) = λ p[g u 1(p) (1 p)u 1(p)] is the leaning benefit accuing to playe 1 when he plays isky, and β 1 (p,u 1 ) = λ (1 p)[s u 1(p) + pu 1(p)] is his leaning benefit fom playe s playing isky. 7 7 By standad esults, playe 1 s payoff function fom playing a best esponse against k is once continuously diffeentiable on any open inteval of beliefs whee playe s action is constant. At a belief whee k is discontinuous, u 1(p) must be undestood as the one-sided deivative of u 1 in the diection implied by the law of motion of beliefs. 11

14 Analogously, the Bellman equation fo playe is u (p) = s + k 1 (p)β (p,u ) + max k {0,1} k [b (p,u ) c (p)], whee c (p) = s (1 p)g is the oppotunity cost playe has to bea when he plays isky, b (p,u ) = λ (1 p)[g u (p) + pu (p)] is the leaning benefit accuing to playe when he plays isky, and β (p,u ) = λ p[s u (p) (1 p)u (p)] is his leaning benefit fom playe 1 s playing isky. It is staightfowad to obtain closed-fom solutions fo the payoff functions. If k 1 (p) = k (p) = 0, the playes payoffs ae u 1 (p) = u (p) = s. If k 1 (p) = k (p) = 1, the Bellman equations yield u 1 (p) = pg + λ λ+ (1 p)s and u (p) = u 1 (1 p). On any inteval whee k 1 (p) = 1 and k (p) = 0, u 1 and u satisfy the ODEs λp(1 p)u 1(p) + ( + λp)u 1 (p) = ( + λ)pg, λp(1 p)u (p) + ( + λp)u (p) = ( + λp)s, which have the solutions u 1 (p) = pg+c 1 (1 p) +λ λ p λ and u (p) = s+c (1 p) +λ λ p λ with constants of integation C 1 and C, espectively. Finally, on any inteval whee k 1 (p) = 0 and k (p) = 1, u 1 and u solve λp(1 p)u 1(p) [ + λ(1 p)]u 1 (p) = [ + λ(1 p)]s, λp(1 p)u (p) [ + λ(1 p)]u (p) = ( + λ)(1 p)g, which implies u 1 (p) = s + C 1 p +λ λ (1 p) λ and u (p) = (1 p)g + C p +λ λ (1 p) λ. Note that each of the above closed-fom solutions is the sum of one tem that expesses the expected payoff fom committing to a paticula action and anothe tem that captues the option value of being able to switch to the othe action. 5 Complete Leaning In this section, we shall show that wheneve the planne s solution leads to complete leaning, so will any Makov pefect equilibium of the expeimentation game. To this end, we fist establish a lowe bound on equilibium payoffs. Fom Kelle, Rady and Cipps (005), the optimal payoffs of playe 1 and, if they wee expeimenting in isolation and hence applying the cutoffs p and 1 p, espectively, would be s if p p, u 1(p) = ( ) +λ ( ) pg + (s p 1 p λ p λ g) if p p 1 p p 1

15 and u (p) = u 1(1 p). Since each playe in the expeimentation game always has the option to act as though he wee a single playe by just ignoing the additional signal he gets fom the othe playe, it is quite intuitive that he cannot possibly do wose with the othe playe aound than if he wee by himself. The following lemma confims this intuition. Lemma 5.1 The value function of the espective single-agent poblem constitutes a lowe bound on each playe s equilibium value function in any Makov pefect equilibium. Now, if g s > +λ +λ, then p < 1 < 1 p, so at any belief p, Lemma 5.1 implies u 1(p) > s o u (p) > s o both. Thus, thee cannot exist a p such that k 1 (p) = k (p) = 0 be mutually best esponses as this would mean u 1 (p) = u (p) = s. This poves the following poposition: Poposition 5. (Complete leaning) If Makov pefect equilibium. g s > +λ, leaning will be complete in any +λ Poposition 6.5 below will show that in the knife-edge case whee g = +λ, the unique s +λ Makov pefect equilibium also entails complete leaning. Wheneve efficiency calls fo complete leaning, theefoe, leaning will be complete in equilibium. This esult is in stak contast to the benchmak poblem of pefect positive coelation in Bolton and Hais (1999) and Kelle, Rady and Cipps (005), whee any MPE entails an inefficiently lage pobability of incomplete leaning. 6 Makov Pefect Equilibia Ou next aim is to the chaacteize the Makov pefect equilibia of the expeimentation game. The pofile of actions (k 1,k ) must be (0, 0), (0, 1), (1, 0) o (1, 1) at any belief. Fo g < +λ, the pofile (1, 1) cannot occu in equilibium since it would imply an aveage s +λ payoff of u 11 < s at the elevant belief, giving at least one playe a payoff below s, and hence below his single-agent optimum. Fo g > +λ, on the othe hand, the pofile (0, 0) cannot s +λ occu since it would imply incomplete leaning. We say that the tansition (k 1,k ) (k + 1,k ) (k + 1,k + ) occus at the belief ˆp ]0, 1[ if lim p ˆp k 1 (p) = k 1, (k 1 (ˆp),k (ˆp)) = (k + 1,k ), lim p ˆp k (p) = k +, and at least one of the sets {k 1,k + 1 } and {k,k + } contains moe than one element. Given ou definition of stategies, each equilibium has a finite numbe of tansitions. 13

16 We fist conside tansitions whee one playe s action stays fixed. Invoking the standad pinciples of value matching and smooth pasting, we obtain the following esult. Lemma 6.1 In any Makov pefect equilibium, a tansition (k 1, 0) (k + 1, 0) (k + 1, 0) can only occu at the belief p, (0,k ) (0,k ) (0,k + ) only at 1 p, (k 1, 1) (k + 1, 1) (k + 1, 1) only at p m, and (1,k ) (1,k ) (1,k + ) only at 1 p m. While it is intuitive that a playe would apply the single-agent cutoff ule against an opponent who plays safe and thus povides no infomation, it is supising that the myopic cutoff detemines equilibium behavio against an opponent who plays isky. Technically, this esult is due to the fact that along playe 1 s payoff function fo k 1 = k = 1, his leaning benefit fom playing isky vanishes: b 1 (p,u 1 ) = λ ( [g p pg + λ ) ( (1 p)s (1 p) g λ + λ )] λ + s = 0, and so k 1 = 1 is optimal against k = 1 if and only if c 1 (p) 0, that is, p p m. This is best undestood by ecalling the law of motion of beliefs in the absence of a success on eithe am, ṗ = (k 1 k )λp(1 p), which tells us that if both playes ae playing isky, the state vaiable does not budge until the fist success occus and all uncetainty is esolved. In othe wods, all a playe does by chiming in in his opponent s expeimentation is to keep the belief, his action and his continuation value constant and wait fo the esolution of uncetainty. But this can only be optimal if he eaps maximal cuent payoffs while waiting. So his playing the isky am must be myopically optimal. In the following lemma, we conside the tansitions whee both playes change action. Lemma 6. The following statements hold fo all Makov pefect equilibia. (i) The tansition (1, 0) (0, 0) (0, 1) can only occu if g +λ and only at beliefs in [1 s +λ p,p ]. (ii) The tansition (0, 1) (1, 1) (1, 0) can only occu if +λ g and only at beliefs in +λ s [1 p m,p m ]. The stuctue of Makov pefect equilibia depends on the elative position of the possible tansition points, which in tun depends on the stakes involved, i.e. on the atio g s. Fo expositional easons, we shall fist analyze the cases of low and high stakes. 6.1 Low Stakes Recall that the low-stakes case is defined by the inequality g s < +λ +λ. In this case, 1 pm < 1 p < 1 < p < p m. 14

17 Poposition 6.3 (Makov pefect equilibium fo low stakes) When g s < +λ +λ, the unique Makov pefect equilibium is symmetic and coincides with the planne s solution. That is, playe 1 plays isky on [p, 1] and safe on [0,p [, while playe plays isky on [0, 1 p ] and safe on ]1 p, 1]. The petaining value functions ae those of the espective single-agent poblems, u 1 and u. Figue 3 illustates this esult. 8 The playes aveage payoff function coincides with the planne s value function as stated in Poposition 3.1. g g+s u u 1 s 0 1 p 1 p 1 Figue 3: The equilibium value functions fo g s < +λ +λ. Why we should have efficiency in this case is intuitively quite clea, as the planne lets playes behave as though they wee single playes. As p > 1, thee is no spillove fom a playe behaving like a single agent on the othe playe s optimization poblem. Hence the latte s best esponse calls fo behaving like a single playe as well. Thus, thee is no conflict between social and pivate incentives. 9 The law of motion fo the belief and the pobability of the playes eventually finding out the tue state of the wold ae thus the same as in the planne s solution fo low stakes. 8 In this and all subsequent figues, the thick solid line depicts the value function of playe 1, the thin solid line that of playe, and the dotted line the playes aveage payoff function. 9 As aleady discussed in the intoduction, the uniqueness pat of Poposition 6.3 is obvious given the lowe bound on equilibium payoffs in Lemma 5.1 and the planne s solution in Poposition 3.1. The poof of uniqueness that we give in the appendix does not ely on knowledge of the planne s solution. This method of poof has the advantage that it caies ove to intemediate and high stakes. 15

18 6. High Stakes The high-stakes case is defined by the inequality g s. In this case, p < p m 1 1 pm < 1 p. Poposition 6.4 (Makov pefect equilibium fo high stakes) When g s, the unique Makov pefect equilibium is symmetic and has both playes behave myopically. That is, playe 1 plays isky on [p m, 1] and safe on [0,p m [, while playe plays isky on [0, 1 p m ] and safe on ]1 p m, 1]. The petaining value functions ae ( ) +λ ( ) s + λ (1 λ+ pm p λ 1 p λ )s if p p m, p m 1 p m u 1 (p) = pg + λ (1 p)s if λ+ pm p 1 p m, ( ) +λ ( ) pg + λ λ+ pm 1 p λ p λ s if p 1 p m p m 1 p m and u (p) = u 1 (1 p). When the stakes ae high, the unique equilibium calls fo both playes behaving myopically. This is best undestood by ecalling fom ou discussion above that individual optimality calls fo myopic behavio wheneve one s opponent is playing isky. When the stakes ae high, playes myopic cutoff beliefs ae moe pessimistic than p = 1, so the elevant intevals ovelap. Figue 4 illustates this esult. Playe 1 s value function has a kink at 1 p m, whee playe changes action. Symmetically, playe s value function has a kink at p m, whee playe 1 changes action. As a consequence, the aveage payoff function has a kink both at p m and at 1 p m. That it dips below the level u 11 close to these kinks is evidence of the inefficiency of equilibium. We will etun to this point in Section 6.4 below. Aguing exactly as afte Poposition 3., it is staightfowad to see that leaning will be complete, as pedicted by Poposition Intemediate Stakes This case is defined by the condition that +λ +λ g s <. In this case, p 1 < pm. When the stakes ae intemediate in size, equilibium is not unique; athe thee is a continuum of equilibia, as the following poposition shows. 16

19 g u u 1 g+s u 11 s 0 p m 1 1 p m 1 Figue 4: The equilibium value functions fo g s >. Poposition 6.5 (Makov pefect equilibia fo intemediate stakes) When +λ g s +λ <, thee is a continuum of Makov pefect equilibia. Each of them is chaacteized by a unique belief ˆp [max{1 p m,p }, min{p m, 1 p }] such that playe 1 plays isky if and only if p ˆp, and playe if and only if p ˆp. The petaining value functions ae given by s + [ˆpg ) +λ ( ) + λ λ+ u 1 (p) = (1 ˆp)s s]( p λ 1 p λ if p ˆp ˆp 1 ˆp ( ) +λ ( ) pg + λ (1 ˆp)s 1 p λ p λ if p ˆp λ+ 1 ˆp ˆp fo playe 1, and ( (1 p)g + λ λ+ u (p) = ˆps p ˆp ) +λ λ ( ) 1 p λ 1 ˆp s + [ (1 ˆp)g + λ λ+ ˆps s]( 1 p 1 ˆp ) +λ λ ( ) p λ ˆp if p ˆp if p ˆp fo playe. Amongst the continuum of equilibia chaacteized in Poposition 6.5, thee is a unique symmetic one, given by ˆp = 1. Figue 5 illustates this equilibium. Both playes value functions and thei aveage ae kinked at p = 1, whee both playes change action. At any belief except p = 1, the aveage payoff function is below the planne s solution; if the initial belief is p 0 = 1, howeve, the efficient aveage payoff u 11 is achieved. 17

20 g u u 1 g+s u 11 s Figue 5: The value functions in the unique symmetic equilibium fo +λ +λ < g s <. Fo abitay ˆp, the dynamics of beliefs in the absence of a beakthough ae given by λp(1 p) if p < ˆp, ṗ = 0 if p = ˆp, λp(1 p) if p > ˆp. As pedicted by Poposition 5., leaning is complete in all these equilibia. 6.4 Efficiency vs. Myopia As we have pointed out aleady, when the stakes ae low, playes do not intefee with each othe s optimization poblem and behave as though they wee all by themselves. We have seen that this kind of behavio is also efficient. If stakes ae high, howeve, we have seen that playes behave myopically. This implies that in the unique MPE, expeimentation is at efficient levels except on [ p, p m ] [1 p m, 1 p], the union of two non-empty and non-degeneate intevals, whee expeimentation is inefficiently low. Put diffeently, thee is a egion of beliefs whee one playe fee-ides on the othe playe s expeimentation, which is inefficient fom a social point of view. In the case of intemediate stakes, equilibium behavio changes gadually fom efficiency to myopia. Indeed, as is easily veified, if +λ < g +λ + (+λ) + λ, then the lowe +λ s (+λ) 4(+λ) +λ bound on the equilibium cutoff ˆp satisfies max{p, 1 p m } p. Now, if the playes initial belief is p 0 < p, the equilibium with ˆp = p achieves efficiency as the only beliefs that 18

21 ae eached with positive pobability unde the equilibium stategies ae given by the set {0, 1} [p 0, p], and the equilibium stategies pescibe the efficient actions at all of these beliefs. Similaly, fo p 0 > 1 p, efficiency is achieved by the equilibium with ˆp = 1 p. Finally, if p p 0 1 p, efficiency is achieved by the equilibium with ˆp = p 0, since this ensues that only beliefs in {0,p 0, 1} ae eached with positive pobability. If +λ (+λ) + (+λ) 4(+λ) + λ +λ < g s <, then p < p < 1 p m. Now, suppose p p 0 < 1 p m. Equilibium uniquely calls fo (k 1,k )(p) = (0, 1) fo all p 1 p m, wheeas efficiency would equie (k 1,k )(p) = (1, 1) wheneve p < p 1 p. Thus, equilibium implies inefficient play on the inteval ] p, 1 p m [ which is eached with positive pobability given the initial belief p 0. Combined with ou esults fo low and high stakes, these aguments establish the following poposition. Poposition 6.6 (Efficiency) If g +λ + (+λ) + λ, then fo each initial belief, s (+λ) 4(+λ) +λ thee exists a Makov pefect equilibium that achieves the efficient outcome. If g > +λ + s (+λ) (+λ) + λ, thee ae initial beliefs unde which the efficient outcome cannot be eached 4(+λ) +λ in equilibium. If 1 + +λ g s <, then pm 1 p. In this situation, setting ˆp = p m (ˆp = 1 p m ) yields an equilibium whee only playe 1 (playe ) behaves myopically, while the othe playe beas the entie buden of expeimentation by himself, something he is only willing to do povided the stakes involved exceed the theshold of 1+. In view of ou findings +λ fo low and high stakes, this establishes the following esult. Poposition 6.7 (Myopic behavio) If g 1 +, thee exists a Makov pefect s +λ equilibium whee at least one of the playes behaves myopically. If g < 1+, no playe s +λ behaves myopically in equilibium. Note that fo cetain paamete values, namely if 1+ +λ g s +λ (+λ) + (+λ) 4(+λ) + λ +λ, equilibia whee one playe behaves myopically co-exist with equilibia that achieve efficiency given the initial belief. 7 Concluding Remaks We have analyzed a game of stategic expeimentation in continuous time whee playes expected isky payoffs ae pefectly negatively coelated acoss states of the wold. We have 19

22 found that, in shap contast to the case of pefectly positive coelation, all the equilibia ae of the cutoff type, and that fo a lage subset of paametes, equilibium is unique. When the stakes ae low, equilibium behavio is efficient, wheeas fo high stakes playes behave myopically. In ode to ensue a well-defined state vaiable fo all initial values, we have esticted playes stategies to be continuous in the diection of moe optimistic beliefs. Although this estiction ules out moe tansitions than would be necessay to guaantee a well-defined solution to the law of motion fo evey initial belief, it tuns out to be innocuous in the sense that no futhe equilibia emege when we ule out only those tansitions that ae incompatible with a well-defined law of motion. 10 Futhemoe, we have esticted attention to what in the liteatue have been temed pue stategy equilibia (by Bolton and Hais, 1999 and 000) o simple equilibia (by Kelle, Rady and Cipps, 005, and Kelle and Rady, 007). Ou esults on efficiency and complete leaning ae obust to an extension of the stategy space whee playes ae allowed to choose expeimentation intensities fom the entie unit inteval. Ou esults on efficiency and complete leaning would not change eithe if, as in Kelle and Rady (007), a bad isky am also had a non-zeo aival ate of lump-sum payoffs, so that the fist success on a isky am no longe evealed the tue state of the wold. Ou analysis natually aises the question unde what cicumstances playes would choose to play a stategic expeimentation game with pefectly negatively, athe than positively, coelated bandits. To analyze this question, we can extend ou model by letting playes fist decide sequentially whethe they want to expeiment with isky am 1, whose pio pobability of being good is p 0, o with isky am, whose coesponding pobability is 1 p 0. They then play the stategic expeimentation game with eithe pefectly positively o negatively coelated bandits, as the case might be. Using the fact that in any equilibium of the expeimentation game, no playe can obtain a payoff highe than twice the planne s solution minus the single-agent solution, it is staightfowad to deive a condition on the model paametes unde which equilibium of the extended game uniquely pedicts that playes choose diffeent isky ams fo all pios p 0 in a neighbohood of 1.11 It is easy to find paamete combinations that satisfy this condition; fo instance, λ = and g s = 3 will do. Given and λ, moeove, the condition will always be fulfilled if the stakes g s ae 10 We have chosen the a pioi moe estictive couse because using the altenative method would have entailed the undesiable featue that a playe s stategy space depended on his opponent s stategy. The teatment of this case, which, as noted, leads to the exact same set of equilibia, is available fom the authos upon equest. 11 Details ae available fom the authos upon equest. 0

23 lage enough. This shows that fo high uncetainty (p 0 close to 1 ) and sufficiently high stakes, playes pefe going thei own sepaate ways on mutually exclusive hypotheses ove investigating the same hypothesis togethe even when, as we assume hee, thee ae pefect spilloves and no isk of peemption. While this extension of ou model meely allows fo one ievesible poject selection, we plan to exploe epeated selection in a vaiant of ou setup whee, akin to Chattejee and Evans (004), each playe has access to both isky ams and can choose between them at will. This equies each playe to solve a thee-amed bandit poblem with a safe am and two isky ams that ae known to be of opposite types. Finally, we plan to exploe the stategic expeimentation poblem with impefect coelation between bandit types, which necessaily involves a state space with moe than two elements and beliefs that evolve in a simplex of dimension highe than one. 1

24 Appendix Poof of Poposition 3.1 The policy (k 1,k ) implies a well-defined law of motion fo the posteio belief. The function u satisfies value matching and smooth pasting at p and 1 p, hence is of class C 1. It is stictly deceasing on [0,1 p ] and stictly inceasing on [p, 1]. Moeove, u = s + B c on [0, 1 p ], u = s on [1 p,p ], and u = s + B 1 c 1 on [p, 1] (we dop the aguments fo simplicity), which shows that u is indeed the planne s payoff function fom (k 1,k ). To show that u and this policy (k 1,k ) solve the planne s Bellman equation, and hence that (k 1,k ) is optimal, it is enough to establish that B 1 < c 1 and B > c on ]0, 1 p [, B 1 < c 1 and B < c on ]1 p,p [, and B 1 > c 1 and B < c on ]p, 1[. Conside this last inteval. Thee, u = s + B 1 c 1 and u > s (by monotonicity of u) immediately imply B 1 > c 1. Next, B = λ [g+s u] B 1 = λ [g+s u] u+s c 1 ; this is smalle than c if and only if u > u 11, which holds hee since u > s and s > u 11. The othe two intevals ae teated in a simila way. Poof of Poposition 3. It is staightfowad to check that p p 1 if g s +λ +λ. The est of the poof poceeds along the same lines as the pevious one and is theefoe omitted. Definitions and an Auxiliay Result Fo p [0, 1], we define w 1 (p) = pg + λ + λ (1 p)s and w (p) = (1 p)g + λ + λ ps = w 1(1 p). We ecall fom Section 4 that these ae the playes payoff functions when both ae playing isky. Futhemoe, we define the playes expected full-infomation payoffs: u 1 (p) = pg + (1 p)s and u (p) = (1 p)g + ps = u 1 (1 p). The following lemma will be useful in the poofs of Lemma 6. and Popositions Lemma A.1 At any belief whee the payoff function of playe n satisfies u n (p) = s + β n (p, u n ), the sign of b n (p,u n ) c n (p) coincides with the sign of w n (p) u n (p). Poof: We fist note that b n (p,u n ) = λ [u n(p) u n (p)] β n (p,u n ). As β n (p, u n ) = u n (p) s, this implies b n (p,u n ) c n (p) = λ [u n(p) u n (p)] u n (p) + s c n (p) = +λ [w n (p) u n (p)]. Poof of Lemma 5.1 Let u 1 be playe 1 s equilibium value function in some MPE with equilibium stategies (k 1,k ). Wite b 1 (p) = b 1(p,u 1 ), and β 1 (p) = β 1(p,u 1 ). Hencefoth, we shall suppess aguments wheneve

25 this is convenient. Since p is the single-agent cutoff belief fo playe 1, we have u 1 = s fo p p and u 1 = s + b 1 c 1 = pg + b 1 fo p > p. Thus, if p p, the claim obviously holds as s is a lowe bound on u 1. Now, let p > p. Then, noting that b 1 = u 1 pg, we have β 1 = λ [u 1 u 1 ] (u 1 gp). Thus, β1 > 0 if and only if u 1 < pg + λ +λ (1 p)s = w 1. Noting that w 1 (p ) = u 1 (p ) = s, w 1 (1) = u 1 (1) = g, and that w 1 is linea wheeas u 1 is stictly convex in p, we conclude that u 1 < w 1 and hence β1 > 0 on ]p,1[. As a consequence, we have u 1 = pg + b 1 gp + k β1 + b 1 on [p, 1]. Now, suppose u 1 < u 1 at some belief. Since s is a lowe bound on u 1, this implies existence of a belief stictly geate than p whee u 1 < u 1 and u 1 (u 1 ). This immediately yields b 1 > b 1 > c 1, so that we must have k 1 = 1 and u 1 = pg + k β 1 + b 1 at the belief in question. But now, [ ] λ u 1 u 1 pg + k β 1 + b 1 (pg + k β1 + b 1) = (1 k )(b 1 b 1) + k (u 1 u 1 ) > 0, a contadiction. An analogous agument applies fo playe s equilibium value function u. Poof of Lemma 6.1 At each of these tansitions, we must have value matching and smooth pasting fo the playe who changes his action. Fo example, suppose that thee is a tansition (0, 0) (1,0) (1,0) at the belief ˆp. Then the value function of playe 1 must satisfy u 1 (ˆp) = s, u 1 (ˆp) = 0 and λˆp(1 ˆp)u 1 (ˆp) + ( + λˆp)u 1 (ˆp) = (+λ)ˆpg by the ODE fo (k 1,k ) = (1, 0). Substituting fo u 1 (ˆp) and u 1 (ˆp) and solving s yields ˆp = (+λ)g λs = p. The othe tansitions ae dealt with in the same way. Poof of Lemma 6. Suppose the tansition (1, 0) (0, 0) (0, 1) occus at belief ˆp. This implies u 1 (ˆp) = u (ˆp) = s. Now, playe s value function solves the ODE fo k 1 = 0 and k = 1 to the ight of ˆp, which, by continuity of u, implies λˆp(1 ˆp)u (ˆp+) = [ + λ(1 ˆp)]s ( + λ)(1 ˆp)g, whee u (ˆp+) := lim p ˆp u (p), and so we find u (ˆp+) < 0 wheneve ˆp < 1 p. So we must have ˆp 1 p. Now, playe 1 s value function solves the ODE fo k 1 = 1 and k = 0 to the left of ˆp, which implies λˆp(1 ˆp)u 1 (ˆp ) = ( + λ)ˆpg ( + λˆp)s, whee u 1 (ˆp ) = lim p ˆp u 1 (p); so we have u 1 (ˆp ) > 0 wheneve ˆp > p. Thus, we must have ˆp [1 p,p ], which equies g s +λ +λ. This poves statement (i). Suppose now that the tansition (0, 1) (1, 1) (1, 0) occus at belief ˆp. This implies u 1 (ˆp) = w 1 (ˆp) and u (ˆp) = w (ˆp). To the ight of ˆp, playe value function solves the ODE fo k 1 = 1 and k = 0, which implies u (ˆp+) = + λˆp [ ] + λ(1 ˆp) s (1 ˆp)g. λˆp(1 ˆp) + λ Now, if ˆp < 1 p m, then u (ˆp+) < w (ˆp) and so u < w to the immediate ight of ˆp, implying by Lemma A.1 that k = 0 is not a best esponse to k 1 = 1 thee a contadiction. Thus, we must 3