International Journal of Industrial Organization

Transcription

1 Internatonal Journal of Industral Organzaton 3 203) 92 0 Contents lsts avalable at ScVerse ScenceDrect Internatonal Journal of Industral Organzaton journal homepage: Equlbrum analyss of dynamc models of mperfect competton Juan F. Escobar Center for Appled Economcs, Department of Industral Engneerng, Unversty of Chle, Republca 70, Santago, Chle artcle nfo abstract Artcle hstory: Receved 8 August 200 Receved n revsed form 25 October 202 Accepted 26 October 202 Avalable onlne 5 November 202 JEL classfcaton: C73 C6 C62 Keywords: Industry dynamcs Dynamc stochastc games Markov perfect equlbrum Motvated by recent developments n appled dynamc analyss, ths paper presents new suffcent condtons for the exstence of a Markov perfect equlbrum n dynamc stochastc games. The man results mply the exstence of a Markov perfect equlbrum provded the sets of actons are compact, the set of states s countable, the perod payoff functons are upper sem-contnuous n acton profles and lower sem-contnuous n actons taken by rval frms, and the transton functon depends contnuously on actons. Moreover, f for each frm a statc best-reply set s convex, the equlbrum can be taken n pure strateges. We present and dscuss suffcent condtons for the convexty of the best reples. In partcular, we ntroduce new suffcent condtons that ensure the dynamc programmng problem each frm faces has a convex soluton set, and deduce the exstence of a Markov perfect equlbrum for ths class of games. Our results expand and unfy the avalable modelng alternatves and apply to several models of nterest n ndustral organzaton, ncludng models of ndustry dynamcs. 202 Elsever B.V. All rghts reserved.. Introducton Ths paper consders nfnte horzon games n whch at each perod, after observng a payoff-relevant state varable, players choose actons smultaneously. The state of the game evolves stochastcally parameterzed by past hstory n a statonary Markov fashon. The settng ncludes a broad class of models, ncludng Ercson and Pakes' 995) model, as well as more general dynamc models of mperfect competton. We present a general exstence theorem for dynamc stochastc games and offer several applcatons to ndustral organzaton. A strct mplcaton from our man result, Theorem, s the followng. A dynamc stochastc game possesses a behavor strategy Markov perfect equlbrum f the sets of actons are compact, the set of states s countable, the perod payoff functons are upper sem-contnuous n acton profles and lower sem-contnuous n rvals' actons, and the probablty dstrbuton of the next state depends contnuously on the actons chosen. Moreover, f for each player a statc best-reply set s convex, the equlbrum can be takennpurestrateges. As n prevous work Doraszelsk and Satterthwate, 200; Horst, 2005), to obtan exstence n pure strateges, we need to mpose convexty restrctons on the dynamc game. Our result requres the game to have convex best reples, meanng that for all rvals' actons and all bounded) contnuaton functons, each frm's statc best-reply set s convex. Ths condton resembles and ndeed reduces to) the standard convexty restrcton mposed on the payoff functons n strategc-form games to ensure the exstence of Nash equlbrum. We state ndependent, suffcent E-mal address: jescobar@d.uchle.cl. condtons that ensure the convexty of the best reples. Our frst suffcent condton s the unqueness of the set of best reples, a condton requrng best-reply sets to be sngle-valued. Ths condton reduces to the convexty condton ntroduced by Doraszelsk and Satterthwate 200) n an ndustry dynamcs model. The second suffcent condton, satsfed by the so-called games wth concave reduced payoffs, ensures each player's maxmzaton problem s concave and so best reples are convex-valued. Although these two condtons do not cover all the games that have convex best reples, they sgnfcantly broaden the modelng alternatves that exstng results offer. Our man results have several applcatons; Secton 4 provdes a few. We analyze an ndustry dynamcs model smlar to that ntroduced by Ercson and Pakes 995). Doraszelsk and Satterthwate 200) have recently studed a verson of the Ercson Pakes model and ntroduced acondton,theunque nvestment choce UIC) condton, to guarantee equlbrum exstence. Under the UIC condton, best reples are sngle-valued and thus our convexty restrctons are met. Moreover, we provde a new alternatve condton for the exstence n the Ercson Pakes model and dscuss how ths new condton permts modelng alternatves uncovered by Doraszelsk and Satterthwate's 200) analyss. In partcular, our results allow for multdmensonal nvestment decsons and complementartes among frms' nvestments. We also study a Markov Cournot game n whch frms compete n quanttes, and at each round, a decson-controlled demand shock s realzed. We provde suffcent condtons ensurng equlbrum exstence. We show how restrctons on how rvals' actons affect payoffs and on how the transton functon depends on acton profles make current results unsatsfactory Amr, 996; Horst, 2005; Nowak, 2007). Notably, to ensure equlbrum exstence, we do not need to restrct the number /$ see front matter 202 Elsever B.V. All rghts reserved.

2 J.F. Escobar / Internatonal Journal of Industral Organzaton 3 203) of frms nor do we need to assume the transton functon s lnear n acton profles. We also consder a verson of the Markov Cournot game n whch frms have fxed costs, and show results ensurng the exstence of behavor strategy equlbrum. Fnally, we also apply our results to an ncomplete nformaton dynamc model extensvely studed and appled recently e.g. Bajar et al., 2007; Doraszelsk and Escobar, 200). Datng back to Shapley 953), several authors have studed the problem of equlbrum exstence n dynamc stochastc games. Among these, Mertens and Parthasarathy 987), Nowak and Raghavan 992), and Duffe et al. 994) consttute mportant contrbutons that nether generalze nor are generalzed by our results. Two strands of the lterature are more closely related to ths work. Frst, Horst 2005), Doraszelsk and Satterthwate 200), andnowak 2007) deal wth the pure-strategy equlbrum exstence problem. Some of these results cover state spaces uncovered by our results and prove not only exstence but also unqueness. Although our man result s formally unrelated to these authors', ths paper dentfes convexty condtons that expand and unfy avalable modelng alternatves. Indeed, a game satsfyng any of the convexty condtons those authors mpose has convex best reples as requred by our man result. Moreover, games such as those that Horst 2005) and Nowak 2007) consder are games wth concave reduced payoffs and so, accordng to Proposton 3, have convex best reples. Ths work contrbutes to ths lterature by dentfyng convexty restrctons that are sgnfcantly weaker than the condtons so far avalable. 2 These results also contrbute to the lterature on dynamc games wth countable state spaces. Federgruen 978) and Whtt 980) provde exstence results that are corollares to our man behavor strategy result, Corollary 2, n that they do not permt payoffs to be dscontnuous. In partcular, they do not deal wth the problem of pure strategy exstence, nor do they answer whether a nontrval class of models could satsfy a convexty condton as the one we mpose. The paper s organzed as follows. Secton 2 presents the model. Secton 3 presents and dscusses the man theorem. Secton 4 provdes a number of applcatons of our results. Secton 5 concludes. All proofs are n the appendx, except where the proof provdes mportant ntuton. 2. Setup In ths secton we ntroduce our dynamc game model and defne our equlbrum noton. Smlar to many studes n ndustral organzaton, we consder a dynamc stochastc game played by a fnte set of frms. In each round of play, there s a payoff-relevant state varable e.g., the dentty of the ncumbent frms). The state varable evolves stochastcally, and frms can nfluence ts evoluton through actons e.g., by enterng or extng the market). The goal of each frm s to maxmze the expected present value of ts stream of payoffs. 2.. Model There s a fnte set of frms denoted by I. At the outset of perod t=, frms are nformed about the ntal state of the game, s. Then they smultaneously pck ther actons a =a ) I. At the outset of perod t=2, frms are nformed of the new state of the game s 2 and then smultaneously pck ther actons a 2 =a 2 ) I. And so on for t 3. Bernhem and Ray 989) and Dutta and Sundaram 992) derve pure strategy results formally unrelated to ours. For a class of dynamc models, they restrct the strategy sets so that best reples are sngle valued and the games therefore satsfy the convexty restrctons requred by our analyss. 2 Whle Amr 996) and Curtat 996) restrct ther attenton to supermodular stochastc games, they do need to mpose convexty condtons that, as we explan n Secton 3.2, cannot be deemed as less strngent than ours. The state space s S. For each frm, the set of actons s A. In most applcatons, we wll assume A s contaned n R L, where L s a natural number, but allowng some more generalty wll be useful when studyng models of mperfect competton n whch frms have prvate nformaton see Secton 4.3). When frms make decsons at round t, they know the whole sequence of realzed states s,,s t, and past actons a,,a t. The evoluton of the state varable s Markovan n the sense that a t,s t )fully determnes the dstrbuton over the state n the next round s t+.the Markovan transton functon takes the form P½s tþ Bjða t ; s t ÞŠ ¼ QB; ð a t ; s t Þ,whereBpS. Gven realzed sequences of actons ða t Þ t and states ðs t Þ t, the total payoff for frm s the dscounted sum of perod payoffs X t¼ δ t π ða t ; s t Þ; where δ [0,[ s the dscount factor, and π a,s) s the per perod payoff functon. Ths dynamc stochastc game model s flexble and, ndeed, several models wdely used n the lterature ft nto ths framework. We wll dscuss applcatons and examples n detal n Secton 4. Throughout the paper, we wll mantan the followng assumptons. A S s a countable set. A2 For all, A s compact and contaned n a lnear metrc space. 3 A3 For all, π s a bounded functon. A4 The transton functon Q s setwse contnuous n a A Royden, 968, Chapter.4): for every BpS and s S, QB;a,s) s contnuous n a A. In applcatons, Assumpton A) s perhaps the most demandng one. Whle ths assumpton s usually made n ndustry dynamcs models Doraszelsk and Satterthwate, 200; Ercson and Pakes, 995), t rules out dynamc stochastc games n whch the state varable s contnuous. From Assumpton A3), we can defne π l and π u as, respectvely, the lower and upper bounds for the functon π, and denote π ¼ sup a A;s S π ða; sþ Markov perfect equlbra We now present the equlbrum noton wth whch we work. One may study subgame perfect equlbra of our dynamc model, but recent research has focused on Markov perfect equlbra. Markov perfect equlbra are a class of subgame perfect equlbrum strateges n whch players condton ther play only on payoff-relevant nformaton. 4 The dea s that, n a gven round, frms choose actons dependng on the current state, wth the purpose of maxmzng the sum of current and future expected dscounted payoffs. A Markov strategy for frm s a functon ā :S A mappng current states to actons. Thus, a Markov strategy defnes a dynamc game strategy n whch n each round t, frm chooses acton ā s t ), where s t s the state realzed n round t. A tuple of Markov strateges a s a Markov I perfect equlbrum f t s a subgame perfect equlbrum of the dynamc game. In a Markov perfect equlbrum, although frms condton ther play only on the current state, they may devate to arbtrary strateges condtonng on the whole transpred hstory. We wll also consder behavor Markov perfect equlbra,defned as subgame perfect equlbra n whch each frm uses a strategy ā :S ΔA ) that maps current states to a dstrbuton over actons. 3 A lnear metrc space s a vector space endowed wth a metrc. For example, A could be a compact subset of R L for some L. 4 Several arguments n favor of ths restrcton can be gven; see Maskn and Trole 200) for a partcularly nsghtful dscusson.

3 94 J.F. Escobar / Internatonal Journal of Industral Organzaton 3 203) The man result In ths secton, we present our man exstence result, Theorem. We then derve several suffcent condtons for Theorem to be applcable. 3.. Statement As n many dynamc models, dynamc programmng tools wll be useful for analyzng our setup. We thus defne Π a; s; v ¼ π ða; s Þþδ v ðs ÞQs ; ð a; sþ; where a A, s S, and v : S R are bounded functons. The number Π a,s;v ) s the total expected payoff for player, gven that the current state s s S, the current acton profle s a A, and the contnuaton payoff, as a functon of the next state s S,sv s ). Intutvely, fxng a state s and contnuaton value functons v, the functons I Π,s;v ), I, defne a statc game n whch frms' acton profles are a A.TheMarkov perfect equlbrum requrement, on the one hand, restrcts contnuaton value functons and, on the other hand, nduces Nash equlbrum behavor n the correspondng famly of statc games. To guarantee the exstence of a Markov perfect equlbrum, we wll mpose convexty and regularty restrctons on our dynamc game. The dynamc stochastc game s sad to have convex best repleshf for all, all s S, alla n A, and all bounded functon v : S best-reply set arg max Π x ;;a ; s; v x A π l δ ; π u δ the ð3:þ s convex. Ths condton says the statc optmzaton problem n whch each frm chooses an acton x A wth the purpose of maxmzng ts total expected payoffs has a convex soluton set, gven the profle played by ts rvals a,thecurrentstates, and the contnuaton value functon v. Imposng some contnuty restrctons on payoffs wll also be useful. Recall that a functon f : X R, wherex s a metrc space, s sad to be upper sem-contnuous f for all sequence x n x n X, lm sup no fx n ) fx), and s sad to be lower sem-contnuous f for all x n x n X, lm nf n fx n ) fx). The followng s our man exstence result. Theorem. The dynamc stochastc game possesses a Markov perfect equlbrum f t has convex best reples and for all, π a,s) s upper sem-contnuous n a A and lower sem-contnuous n a A. We provde a proof of ths result n Appendx A. We employ a fxed pont argument on the space of best reples and contnuaton values to fnd a soluton to the condtons mposed by Markov equlbra. We use upper and lower sem-contnuty 5 to ensure that best reples and contnuaton values exst and are suffcently contnuous n rvals' strateges. 6 Together wth the convexty of the best reples, these contnuty restrctons ensure equlbrum exstence usng Kakutan's fxed pont theorem. Secton 3.2 presents some easy-to-check condtons to apply Theorem. Before dscussng those condtons, we observe that when the game does not have convex best reples, our man result can stll be appled to deduce the exstence of behavor Markov perfect equlbra. Appendx B presents a proof. 5 A prevous verson of the paper relaxed the lower sem-contnuty assumpton n Theorem. 6 Observe that f π a,s) s upper sem-contnuous n a and lower sem-contnuous n a, then t s contnuous n a. Ths observaton mples that our result only allows π to be dscontnuous n a. We note ths observaton does not mply that exstence s obtaned when π a,s) s upper sem-contnuous n a and contnuous n a, because to ensure best reples are contnuous n other strateges some degree of jont contnuty n own and rvals' actons s needed. Corollary 2. The dynamc stochastc game possesses a behavor Markov perfect equlbrum f for all, π a,s) s upper sem-contnuous n a A and lower sem-contnuous n a A. Although computng behavor strategy equlbrum s dffcult f not mpossble) when the set A s not fnte, ths corollary can be appled to analytcally study dynamc olgopoly games n whch frms ncur fxed costs; see Secton Dscusson and suffcent condtons The game wll have convex best reples whenever the set of maxmzers 3.) s a sngleton; n ths case, we say the game has sngle-valued best reples. Games studed by Horst 2005) and Doraszelsk and Satterthwate 200), among others, have snglevalued best reples. Secton 4 presents the models extensvely employed n the appled IO lterature where ths knd of unqueness restrcton can be exploted. We wll now ntroduce a new class of dynamc stochastc games exhbtng convex best reples. To present these games and n the rest of ths secton), we wll restrct our man settng by assumng that the set of actons of each player, A, s a convex set contaned n R L, where L s a natural number. For a gven real-valued symmetrc square matrx M, we denote mevðm Þ ¼ maxfλλs j an egenvalue of Mg: We also assume that π a,s) and Qs ;a,s) are twce contnuously dfferentable wth respect to a A, and denote the Hessan matrces wth respect to a A by π a,s) and Q s ;a,s) respectvely. 7 We say that the game has concave reduced payoffs f for all, the functon π a,s) s concave n a, and for all a,s) A S ether π u π l maxf0; mevðq ðs ; a; sþþg mev π ða; sþ ¼ 0 or ths expresson s strctly postve and mev π δ ða; sþ π u π l max 0; mev Q : ð3:2þ s ; a; s mev π ða; sþ The followng result provdes a suffcent condton for a game to have convex best reples. Proposton 3. Suppose that the game has concave h reduced payoffs. Then, for all I, all a A, all s S, and all v : S π l ; π δ u, Π a,s;v ) δ s concave n a. In partcular, the game has convex best reples. Because the proof of ths result s smple and ntutve, we present the argument n the text. Assume frst π l =0. Observe that Π a; s; v ¼ π ða; sþþδ s Sv s Q s ; a; s.therefore,π s concave n a f for all x R L, x π ða; sþx þ δ v ðs Þx q ðs ; a; sþx 0: To prove ths property, observe that for any symmetrc matrx Aleskerov et al., 20, secton 9.4), x Mx mevðmþkk x 2 : 7 Observe that when Q s twce contnuously dfferentable n a, then our assumpton A4) s satsfed.

4 J.F. Escobar / Internatonal Journal of Industral Organzaton 3 203) Therefore, for all v :S [0,π u ] x π ða; sþx þ δ v s x Q s ; a; s x mev π ða; sþ kk x 2 þ δ π n o u δ max 0; x Q s ; a; s x mev π ða; sþ kk x 2 þ δ π n o u δ max 0; mev Q s ; a; s kk x 2 ¼ mev π ða; sþ þ δ π n o! u δ max 0; mev Q s ; a; s kk x 2 : Under the condtons of the proposton, ths expresson s less than or equal to 0. When π l 0, consder π ða; sþ ¼ π ða; sþ π l and apply the result above to the modfed payoffs. Ths step completes the proof. Proposton 3 provdes a condton under whch Π a,s;v )sa concave functon of a for all a A,alls S, andallv.togan ntuton, suppose frst that π a,s) sstrctlyconcavena.then, even f for some v the term s Sv s Q s ; a; s s hghly non-lnear, the sum of π a,s) andδ s Sv s Q s ; a; s can stll be aconcavefunctonfδ s small enough. More generally, Eq. 3.2) can be seen as makng explct a tenson between the dscount factor δ and the second dervatve wth respect to a of π a,s). Note that, as the followng result shows, n some models, restrctng attenton to games wth concave reduced payoffs mposes no restrcton on δ. Corollary 4. Suppose that the transton functon can be wrtten as Q s ; a; s ¼ XK α k ðaþf k s ; s k¼ where for all s, F k ;s) s a probablty dstrbuton over S, and for all K a A, k= α k a)= wth α k a) 0 for all k. Assume that for all k, α k s twce contnuously dfferentable as a functon of a A R L wth a Hessan matrx that equals 0. Then, the game has concave reduced payoffs and best reples are convex. Ths result smply follows by notng that under the assumptons on the transton n the corollary, mevq s ;a,s)) equals 0 for all,s, a, and s, and therefore the game has concave reduced payoffs and, from Proposton 3, best reples are convex. The mportance of ths corollary s that t provdes easy-to-check suffcent condtons ensurng the convexty of the best reples. In partcular, when the transton functon Q s a multlnear functon of a,,a I ), the game has concave reduced payoffs provded each π a,s) s a concave functon of a A. Although ths restrcton on Q may seem demandng n some applcatons, t provdes as much flexblty n perod payoffs as one can hope for.e., concavty of payoffs as functons of own actons) and mposes no restrcton on the dscount factors. One of the attractve features of games wth concave reduced payoffs s ther tractablty. Indeed, n games wth concave reduced payoffs, frst-order condtons, beng necessary and suffcent for optmalty, can be used to characterze equlbrum strateges. Ths observaton s mportant not only when analytcally dervng propertes of the equlbrum strateges, but also when numercally solvng for those strateges. The restrcton to games havng concave reduced payoffs relates to smlar condtons mposed n prevous work. Horst's 2005) Weak Interacton Condton 2005) s strctly more demandng than our suffcent condton; ths can be seen by notng that any stochastc game satsfyng condton 7) n Horst's 2005) paper also satsfes condton 3.2). 8 Indeed, Horst's 2005) assumpton addtonally makes reacton functons vrtually flat functons of others' strateges. More recently, Nowak 2007) works under the assumpton that π a,s) s concave n a, Qs ;a,s) s affne n a as n Corollary 4), and a strct dagonal domnance assumpton holds. 9 It s not hard to see that under Nowak's 2007) concavty condton, Q =0 and so, from Corollary 4, the game has concave reduced payoffs and convex best reples for all δ b. Amr 996) and Curtat 996) have studed supermodular stochastc games. These authors work under the assumpton that the payoffs and the transton are supermodular and satsfy a postve spllovers property. 0 Moreover, these works stll need to mpose some convexty restrctons. Consder, for example, Curtat's 996) strct dagonal domnance SDD) assumpton. To smplfy the exposton, assume frst that for all, A R. Then, the SDD condton can be expressed as follows: For all and all a,s) A S, 2 π ða;sþ I b0. Snce a a j π s supermodular, SDD mples that π s strctly concave n a. More generally, f A R L, L, SDD s related to the concavty of π a,s) na, but nether condton mples the other. Yet, the SDD condton on the transton restrcts the model dynamcs substantally. Indeed, n all the examples studed by Curtat 996), the transton s a lnear functon of the acton profle a A. 4. Applcatons Ths secton provdes a number of applcatons of our man results. Secton 4. studes a model smlar to Ercson and Pakes 995) and relates our suffcent condtons to those recently derved by Doraszelsk and Satterthwate 200). Secton 4.2 shows a dynamc verson of the textbook Cournot game wth stochastc demand. Secton 4.3 ensures equlbrum exstence n a dynamc model of ncomplete nformaton Bajar et al., 2007; Doraszelsk and Escobar, 200). Fnally, Secton 4.4 ensures exstence n behavor strateges n a Markov Cournot game wth fxed costs. 4.. Ercson Pakes ndustry dynamcs model We now study an ndustry dynamcs game n the sprt of Ercson and Pakes's 995) semnal model. Consder a fnte set I of frms. At each t, some of the frms are ncumbent; the others are entrant. The state of frm s s ¼ s ;;η S f0; g, where s reflects a demand or technology shock; η = f frm s an ncumbent, and η =0 f frm s an entrant. The state of the ndustry s s ¼ s I. The acton of frm s,x ) {0,} X, wth X R L þ, where = resp. =0) f frm changes resp. does not change) ts ncumbency/ entrance status and x s a vector of nvestment projects frm undertakes. In other words, f frm s an entrant resp. ncumbent) and =, then becomes an ncumbent resp. entrant). Snce the set {0,} s not convex, we allow frms to randomze. Let p [0,] be the probablty wth whch changes ts statues. Frm 's acton s therefore avectora =p,x ) [0,] X; we assume that X s convex and compact. 8 To see ths, assumng that π l =0, L =, and δ =δ for all, our condton 3.2) can be equvalently wrtten as δ δ π max f0;q sup ðs ;a;sþg for all and all s. Denotngby a A s π S j ða;sþ j π jða; sþ ¼ π ða;sþ a aj, condton 7 on assumpton 2.2 n Horst 2005) can be wrtten as π sup ða;sþ j π j a A j ða;sþ j þ δ δ π sup j Q jðs ;a;sþj þ π j a A j ða;sþ j δ a π jq sup ðs ;a;sþj b forall a A s π S j ða;sþ j and all s. It follows that the left-hand sde of my restrcton s strctly less than the left-hand sde of the restrcton above. 9 Ths assumpton makes reacton functons n the statc one-shot game a contracton. 0 A game has postve spllovers f payoff functons are nondecreasng n rvals' actons.

5 96 J.F. Escobar / Internatonal Journal of Industral Organzaton 3 203) 92 0 Gven a state s and an acton profle a ¼ a off for frm s gven by, the per-perod pay- I π ða; sþ ¼ η g ðþþ s ψ ðþp s þ η ψ ðþp s c x ; s : The frst term s the proftthatfrm obtans when competng n a spot market, g s),plusthescrapvalueatwhchfrm may be sold, ψ s), tmes the probablty of ext p when frm s ncumbent, η =. The second term s the set-up prce frm must pay to enter the market, ψ ðþ,tmesthe s probablty of entry, p,whenfrm s an entrant, η =0. The thrd term s the cost of nvestment x when the state s s. In appled work, one would restrct g s) to depend not on the whole vector s, but only on those s j for whch frm j s an ncumbent η j =. Analogously, the scrap and set-up values wll typcally depend only on the frm's own state s.frm 's dscount factor s δ. For a gven vector ¼ ð Þ I f0; g jj I of decsons on status changes and a profle x ¼ x of nvestment decsons, the state of the system I n the followng perod s dstrbuted accordng to Q r ;,x,s). It s relatvely easy to see that gven the vector of actons a=p,x), the next perod state s dstrbuted accordng to Qs ; ð a; sþ ¼ Q r f0;g jj I s ; ; x; s jj I j¼ p j j p j! j ; wherewedefne 0 0 =0. We assume that c x,s) andq r s ;,x,s) are twce contnuously dfferentable functons of x. Doraszelsk and Satterthwate 200) study a smlar model. They ntroduce the unque nvestment choce UIC) condton, a condton mplyng that the best-reply set 3.) s unque. It s therefore evdent that after ntroducng a UIC condton n our model, the stochastc game has convex best reples and so the exstence of equlbrum s a consequence of Theorem. Although the UIC condton may be appled to many varatons of the Ercson Pakes model, we provde a new condton that apples to mportant stuatons that Doraszelsk and Satterthwate's 200) result does not. Frst note that π ða; sþ ¼ c x ; s ; where c x,s) denotes the matrx of second dervatves wth respect to x. Now, the Hessan matrx of the transton functon, Q, can be expressed as h! ¼ p j j p j j Q ðs ; p; x; sþ 0 f0;g B Q r x s ; ; x; s 2 0 s ; ; x; s 2 Q r x x s ; ; x; s p p Q r x C A; where Q r r x s ;,x,s) resp. Q x x s ;,x,s)) denotes the column vector of dervatves resp. matrx of second dervatves) of Q r s ;,x,s) wth respect to the varable x. Denotng λ s ;p,x,s)=mevq s ;p,x,s)), t follows that the Ercson Pakes ndustry dynamcs model has an equlbrum provded mev c δ x ; s μ max 0; ð4:þ λ s ; p; x; s mev c x ; s For completeness, let me smplfy the model to offer a UIC condton n the sprt of Doraszelsk and Satterthwate 200). Suppose that the status change decson s payoff) rrelevant, that s, the only choce varable s x. Also suppose that X=[0,] and c x,s)=x. Then the UIC condton holds provded for all, Qs ;x,s)=a s ;x,s)η s,x )+b s ;x,s), where η s twce dfferentable, strctly ncreasng, and strctly concave n x. Under ths condton, Eq. 3.) s sngle valued. for all,p,x, and s, where μ =π u π l equals μ ¼ max g s þ ψ s c y ;;s s S wth η ¼;y X þ max ψ s þ c y ;;s : s S wth η ¼0;y X Although Eq. 4.) s not more general than the UIC condton a condton already shown to ft nto our general framework), ths new condton allows modelng alternatves uncovered by Doraszelsk and Satterthwate 200). Doraszelsk and Satterthwate's 200) analyss hnges on the undmensonalty of the nvestment decsons rulng out, for example, nvestment plans that can affect the demand and the cost structure ndependently, and the separable form of the transton rulng out several transtons exhbtng non-trval complementartes among the nvestment decsons. These and other modelng alternatves can be analyzed wth ths new alternatve condton. Condton 4.) nvolves the maxmum egenvalue of the matrx of second dervatves of mnus the cost functon. Intutvely, the condton says that c must be suffcently concave, gven the dscount factor δ and the transton n functon as captured by the non-lnear term s Smax 0; λ o s ; p; x; s ), so that all of ts egenvalues are negatve enough. Alternatvely, the frm must be suffcently mpatent gven the technology c and the transton functon Q. Condton4.) resonates well wth other exstence results n equlbrum theory, whch emphasze the mportance of rulng out ncreasng returns of producton to ensure equlbrum exstence Mas-Colell et al., 995, Proposton 7.BB.2). The novel aspect of condton 4.) s that, because of the dynamcs, convexty of the cost functons must be strengthened so that even when frms maxmze total payoffs, best reples are convex-valued. We therefore restrct attenton to models n whch returns to scale are suffcently decreasng. Also of nterest s the observaton that when Eq. 4.) holds, each frm's payoff s a concave functon of ts decson varables. Thus, frst-order condtons are necessary and suffcent for optmalty. The problem of numercally fndng equlbrum strateges s therefore effectvely reduced to the problem of solvng a potentally huge) system of frst-order condtons equaltes or varatonal nequaltes). Remarkably, under Eq. 4.), we can be confdent that a convergng method solvng frst-order condtons wll yeld an equlbrum of the model. In applcatons, checkng condton 4.) amounts to solvng I S nonlnear mnmzaton problems on [0,] X) I. In complcated models, one can solve such problems numercally before runnng the routnes to solve for the equlbra. Ths frst step s relatvely easy to mplement numercally because the I S mnmzaton problems are unrelated. If ths ntal step s successful, our model s well behaved n that t possesses an equlbrum and all the dynamc programmng problems nvolved wll be concave maxmzaton problems. The followng example presents a smple model n whch nvestment decsons are multdmensonal and returns to scale are decreasng; we observe that Doraszelsk and Satterthwate 200) results do not apply. Example 5. Suppose that frms nvest jontly n a project and the total nvestment determnes the common state of the ndustry s. Inother words, we now assume that s ¼ s j for all j, and that ths state s stochastcally determned by = I x,wherex [0,] 2.Thestates only determnes spot market profts so that the proft ofafrm competng n the spot market s gðs; ηþ, whereas scrap values and set-up prces are ψ ðþ¼ψ R s þ and ψ ðþ¼ s ψ R þ for all and all s. Each frm may carry out two types of nvestment projects so that x [0,] 2, and the cost functons take the form c x ; s ¼ 2 x 2 þ x 2 2. We assume that δ =δ for all. Ths model s a symmetrc one n whch frms jontly

6 J.F. Escobar / Internatonal Journal of Industral Organzaton 3 203) nvest n mprovng spot market profts for example, by advertsng or by makng the non-propretary) producton technology more effcent. We refer to the state s as the spot market condtons. The set of spot market condtons S s ordered and can be wrtten S ¼ ; ; S. Hgher states result n hgher profts so that gðs; ηþ s ncreasng n s. The evoluton of the spot market condtons s takes the form Q r ¼ α XI jj s ; x ; s ¼ x!f s ; s þ α XI jj ¼ x!!f 2 s ; s where αy) [0,] for all y [0, I ] 2,andF k ð js Þ s a probablty dstrbuton over S as n Corollary 4). The transtons are such that F moves the subsequent state up determnstcally n one step or stays n the same state f the current step s S ), whereas F2 moves the state down n one step or stays n the same state f the current step s ). Intutvely, the hgher the jont effort y= I x, the more lkely the next spot market condtons wll be favorable. We assume that α ) s a lnear functon and that the frst dmenson of the nvestments s more effectve:αðyþ ¼ þ α 2 y þ α 2 y 2 wth α 2 =α /2. If no frm nvests, the subsequent state s equally lkely to go up or down. Whether frms enter or ext the market s determned by q η ; p; η 0 B p :η ¼η 00 CBB :η η C p A: Therefore the transton takes the form 0 0 Q s B jcb ; p; x; s p A@ j:η j ¼η j j:η j η j j C p A Q r s ; x; s : Once we have an expresson for the transton functon, t s relatvely easy to show that 2 n o q max 0; λ s ; p; x; s ¼ 2 ffffffffffffffffffffffffffffffffff α 2 þ α2 2 ¼ p α and that mev c x,s))=. We also assume that g þ ψ þ ψ ¼, where g ¼ max s gs ðþ, meanng that the sum of spot market profts, scrap values, and set-up costs s at most whch s the maxmum nvestment cost a frm can ncur). We derve the followng suffcent condton for equlbrum exstence: δ p 2 ffffff 5 α þ : For example, f α =/00 so that a frm nvestng Δ>0 unts can ncrease the probablty of the hgh state n Δ per cent), then the condton above amounts to δ When frms make entry and ext decsons before nvestment decsons, and f when makng nvestment decsons frms observe the dentty of the market partcpants, then n the model above the 0 2 Note that Q takes the form P η η ; p F s ; s F2 s 0 α α 2 ; α 0 0 A and α h therefore mev Q s ; p; x; s ¼ P η η ; p F s ; s F2 s qffffffffffffffffffffffffffffffffff ; s α 2 þ α2 2. Notng that each F k ð ; s Þ puts postve weght on 2 states and summng up over subsequent states s S, the result follows. 3 Observe that because αy) must belong to [0,], α s bounded above by. 3jj I ffffff 5 ; exstence of Markov perfect equlbrum can be guaranteed usng Corollary 4 and Theorem, regardless of the dscount factor. In such a model, because the transton Q r s lnear n the nvestment decsons and entry and ext decsons are randomzed strateges, the game has concave reduced payoffs for all dscount factors. In some ndustres, advertsng decsons are lkely to take place after the dentty of the market partcpants s publcly known, and therefore a model of sequental decsons would seem more approprate Markov Cournot olgopoly We now consder a smple dynamc verson of the textbook Cournot game. A fnte set I of olgopolsts exsts. At each t, olgopolsts set quanttes a t [0,], I, smultaneously and ndependently. The nverse) demand functon takes the form P I a t,s t ), where the state, s t,belongs to a fnte set. No costs of producton exst. Thus, the perod payoff to frm s π ða; sþ ¼ a P a ; s : I The demand functon assumes the functonal form P I a,s)= s I a ). Players dscount future payoffs at a constant rate δ ]0,[. The set of states S s a countable subset of ]0, [. The evoluton of the state ðs t Þ t 0 s gven by the transton Q s ; a; s ¼ XK k¼ α k a I F k s ; s ; where K s a fnte number, α k s a quadratc functon of I a, and F k ;s) s a probablty dstrbuton on S. As prevously dscussed, we can nterpret ths transton as beng drawn n two steps: frst, we draw a lottery over the set {,,K} where the weghts are determned by the total producton I a ), then, gven the result k of the lottery, a draw from the dstrbuton F k ;s) s realzed and determnes the subsequent state. The assumpton that α k s a quadratc functon of I a mples that ts second dervatve s constant; let α k be the second dervatve of α k. We also assume that F k ;s) puts weght on some state s S for example, t may put weght on the state mmedately below or above s). It s relatvely easy to see that 4 n max 0; Q s ; a; s o XK k¼ jα k j: for all a [0,] I and all s S. The exstence of a Markov perfect equlbrum s guaranteed provded δ H 4L K k¼j α kjþ ; where H=max{s S} and L=mns S)>0). Note that the results by Curtat 996) do not apply to ths Cournot settng because he consders supermodular games satsfyng strong monotoncty restrctons. To ensure exstence we do not need to mpose condtons on the number of players, nor do we need to assume that α k ¼ 0 for all k. To apply Horst's 2005) and Nowak's 2007) results, regardless of the transton functon, we would need to mpose that I 2. 4 To see ths, note that Q s ; a; s ¼ K α k¼ kf k s ; s. Fx s, and let s k) be the state K X K wth weght gven F k ;s). Then max 0; Q s ; a; s jα k jf k s ; s ¼ j α k j s SXk¼ k¼ F k ðsk ð Þ; sþ ¼ XK j α k j: k¼

7 98 J.F. Escobar / Internatonal Journal of Industral Organzaton 3 203) 92 0 Moreover, results by Nowak 2007) can be appled only f we consdered a lnear transton. For example, consder a model of habt formaton, n whch K=2, 2, α I a ¼ α I a wth α > 0, and gven s S, F ;s) puts weght on a pont strctly greater than the pont n whch F 2 ;s) puts weght on. The dea s that the hgher the volume of sales I a, the hgher the probablty the next demand state s wll be hgh. Because α I a ) [0,] and α I a )+α 2 I a )=, t follows that jα k j for k=,2. Assumng that H/L=2, the suffcent jij 2 condton for equlbrum exstence s δ jij2 I 2 þ : When I =2, an equlbrum exsts provded δ 4 5, whereas when I =6, equlbrum exstence s guaranteed when δ Dynamc model wth ncomplete nformaton We now consder a model of dynamc nteracton wth prvate nformaton. The appled lterature extensvely employs smlar models; consult Bajar et al. 2007) for a recent applcaton. Consder a model smlar to that ntroduced n Secton 2, but now suppose that at the begnnng of each perod, each frm not only observes the publc state s t but also receves a prvate shock ν t RN.Theneachfrm pcks ts acton a t and obtans a perod proft π a,s,ν ). Prvate shocks are drawn ndependently accordng to a dstrbuton functon G ), I, and the transton functon takes the form Qs ;a,s). A pure strategy for a frm s a functon a : S R N A. However, to apply our general framework, we nterpret a strategy as a functon a : S A, where n o A ¼ a : R N A a s measurable : Functons n A are deemed dentcal f they are equal G almost sure. Gven functons a A and a publc state s S, defne π ða; sþ ¼ π a ν ; ; a I ν I ; s; ν G dν G I dν I ; and Q s ; a; s ¼ Q s ; a ν ; ; a I ν I ; s G dν G I dν I : We thus have defned a dynamc model that fts nto our dynamc stochastc game framework. To see the mportance of the prvate shocks when applyng our results, assume that A s fnte, G s absolutely contnuous wth respect to the Lebesgue measure, N = A, and the perod payoff functon takes the form π a; s; ν ¼ g ða; sþþ a ¼k ν;k : k A Now, endow A wth the dscrete topology and A wth the convergence n measure metrc. That s, gven measurable functons a ; b : R N A,defne d A d a ν ; b ν a ;;b ¼ þ d a ν ; b ν G dν : where d s the dscrete metrc over A.Underd A, A s compact. The transton Q ds ; a; s s contnuous and for all, thepayoff π ða; sþ s contnuous n a A. Prvate sgnals come crucally nto play when verfyng the convexty of the best reples. Indeed, t s not hard to see that the best-reply set of each frm s essentally) unque for any contnuaton value functon. 5 So, the game has sngle-valued best reples and the exstence of an equlbrum follows from Theorem Markov Cournot olgopoly wth fxed costs We fnally apply our results to a Markov Cournot game wth fxed costs. Fxed costs ntroduce dscontnutes that make all prevous results n the lterature unapplcable. A fnte set I of olgopolsts exsts that, at each t, set quanttes a [0,], I, smultaneously and ndependently. The nverse) demand functon takes the form P I a,s), where the state belongs to a countable set S. P I a,s) s a contnuous functon of I a. Frm 's cost functon, c a,s), s lower sem-contnuous n a. For example, suppose that each frm must ncur a fxed cost κ>0 to produce any strctly) postve quantty, and that margnal costs equal 0. Then, frm 's cost functon can be wrtten as c a ; s ¼ 0 f a ¼ 0; κ f a > 0: Ths cost functon s lower sem-contnuous at a =0. More generally, the presence of fxed costs that, by defnton, can be avoded f producton s suspended n a gven round) makes cost functons naturally lower sem-contnuous, but not contnuous, at a =0. Hence, the perod payoff to frm s! π ða; sþ ¼ a P a j ; s c a ; s : j I The transton Q s assumed setwse contnuous n a A. For example, the state varable s t+ may represent a demand shock that s contnuously modfed by current sales a t.becauseπ s upper sem-contnuous n a and lower sem-contnuous n a, the exstence of behavor strategy Markov perfect equlbrum follows from Corollary Concludng comments Ths paper offers results that guarantee the exstence of Markov perfect equlbra n a class of dynamc stochastc games. Dynamc models that can be solved analytcally are exceptonal; therefore researchers often need to resort to computatonal routnes to analyze ther models. Yet unless an equlbrum s guaranteed to exst, a non-convergng algorthm desgned to compute an equlbrum may fal ether because an equlbrum exsts and the algorthm s not sutable for ts computaton or, more dramatcally, because an equlbrum does not exst. The results n ths paper provde gudance on the nature of dynamc models that possess Markov perfect equlbra n pure and behavor strateges). In dong so, we expand and unfy several modelng alternatves Doraszelsk and Satterthwate, 200; Horst, 2005) and apply our results to several dynamc models of mperfect competton. We mpose restrctons on the fundamentals of the model, ensurng that each frm's optmzaton problem has a concave objectve functon. Ths property not only consttutes a suffcent condton for equlbrum exstence, but also makes avalable numercal algorthms more relable. The contnuty restrctons we mpose on the payoff functons lmt the applcablty of our results. As a consequence of these 5 Indeed, a frm s ndfferent between two actons wth probablty zero. 6 As n statc models, t s hard to ensure the exstence of pure) Markov perfect equlbra n models wth fxed costs.

8 J.F. Escobar / Internatonal Journal of Industral Organzaton 3 203) assumptons, our settng does not permt applcatons to aucton and prcng games. In fact, n those games, the possblty of equlbrum tes makes payoff functons not upper sem-contnuous. Unless one rules out equlbrum tes, our results are not applcable to aucton and prcng games. Ths observaton opens an mportant research avenue. Appendx A A.. Proof of Theorem In ths appendx, we prove Theorem. We begn by statng the followng key result Stokey and Lucas, 989, Theorem 9.2). Lemma A.. For each I, consder a functon ā :S A. Suppose that there s a tuple v I, where v : S R s bounded, such that for all and for all s S ) v ðþ¼max s π x ; a ðþ s ; s þ δ v s Q s ; x ; a ðþ s ; s jx A ðaþ and a ðþ s arg max π x ; a ðþ s ; s þ δ v s Q s ; x ; a ðþ s ; s jx A ): ða2þ Then, a s a Markov perfect equlbrum. I Ths result allows us to reduce the problem of fndng an equlbrum to the problem of solvng a system of functonal equatons. We wll therefore verfy the exstence of solutons to ths system of functonal equatons usng Kakutan's fxed pont theorem. Before presentng the detals of the proof of Theorem, we present some dynamc programmng results. A.2. Dynamc programmng results Consder the functonal equaton Vs ðþ¼sup x X πðx; sþþδ V s ν s ; x; s ; s S; ða3þ where X s a compact subset of a metrc space, πx,s) s the per-perod proft functon, δ [0,[, and ν ;x,s) s a probablty dstrbuton over S. In ths subsecton, we study the solutons to the functonal Eq. A3) and provde results concernng the exstence and contnuty of those solutons. Consder the followng assumptons. D) πx,s) s upper sem-contnuous n x X. D2) νs ;x,s) s setwse contnuous n x X: for all EpS, νs ;x,s) s contnuous n x X. The followng result guarantees the exstence of a bounded soluton for Eq. A3). Theorem A.2. Assume D) D2). Then, there exsts a sngle soluton V to Eq. A3). Moreover, V kπk = ð δþ. Further, there exsts a polcy functon x : S X. The proof of ths result s standard see Theorem 9.6 n Stokey and Lucas, 989). Now, let us study contnuty propertes for the only soluton to Eq. A3), vewng ths soluton as a functon of the transton functon ν and the per-perod payoff π. We can defne TVðÞ¼sup s x X πðx; sþþδ V s ν s ; x; s : ða4þ For each n N, consder a transton functon ν n and a per-perod payoff functon π n.foreachn, consder the operator T n defned as we dd n Eq. A4), but replacng π and ν wth π n and ν n respectvely. Let V n and V be the only bounded functons such thatt n V n ¼ V n and T V ¼ V. Addtonally, let the set-valued maps X n and X be defned by ) X n ðs; VÞ ¼ arg max π n ðx; sþþδ V s ν n s ; x; s x X and ) X ðs; VÞ ¼ argmax πðx; sþþδ V s ν s ; x; s : x X The followng result shows that the only soluton to Eq. A3) and the related polcy map depends contnuously on ν and π. Proposton A.3. For all n N, assume D) D2) for the problems defned by T n and T. Suppose that for all sequence x n x n X, E S, and s S, the sequence of real numbers ðν n ðe; x n ; sþþ n N converges to νe;x,s) Further suppose that for all s S. For all sequence x n x n X, lm sup n π n x n,s) πx,s); 2. For all x X, there exsts y n x n X, such that lm nf n π n y n, s) πx,s). Then, the followng statements hold. α) For all subsequence V nj V pontwse) and gven any selecton x nj ðþ X nj ; V nj convergng to x : S X, x ðþ s Xs; ð VÞ for all s S. b) For all s S, V n ðþ s V ðþ. s c) The polcy sets are closed maps of the per perod payoff and transton functons. Before provng ths proposton, we establsh a prelmnary lemma. Lemma A.4. Let ðp n Þ n N and P be probablty measures on S. Suppose that for all EpS, P n E) converges to PE). Fx α>0 and let V n :S [ α,α] be a sequence of functons pontwse convergng to V:S [ α,α]. Then, s SV n s Pn ðþ s s SV s Ps. Proof. Observe that V n s bounded by α and that s SαP n s ¼ α converges to s SαP s ¼ α. The result follows from Theorem 8 n Royden 968, Chapter ). Proof of Proposton A.3. Let us begn provng b). Consder any convergng subsequence V nk k N to V 0 such a subsequence always exsts). Fx s S. Consder any sequence x k x n X. Snce V nk ; V 0 are unformly bounded by the same constant, we can apply Lemma A.4 above to deduce that for any x k x, s SV nk s νnk s ; x k ; s V 0 s ν s ; x; s. So, defnng ψn ðx; sþ ¼ V n s νn s ; x; s and ψ 0 ðx; sþ ¼ s SV 0 s ν s ; x; s, we deduce that for all s, ψnk ð ; sþ converges to ψ 0,s) unformly on the compact set X. Fx now x X. Condton n Proposton A.3 permts us to deduce that for all x k x lmsup k π nk ðx k ; sþþδψ nk ðx k ; sþ πðx; sþþδψðx; sþ: ða5þ

9 00 J.F. Escobar / Internatonal Journal of Industral Organzaton 3 203) 92 0 Addtonally, there exsts a sequence y k such that lmnf k π nk ðy k ; sþþδψ nk ðy k ; sþ πðx; sþþδψðx; sþ: ða6þ To prove ths result, defne φ nk ðx; sþ ¼ π nk ðx; sþþδ s SV nk s νnk s ; x; s and φ0 ðx; sþ ¼ πðx; sþþδ V 0 s ν s ; x; s.fxη>0. From condton 2 n the proposton, there exsts ^x k x n X such that π nk ð ^x k ; sþ πðx; s Þ η. We further know that the functon x X δ 3 V nk s νnk s ; x; s converges contnuously to x X δ s SV 0 s ν s ; x; s. Consequently, for k bg enough δ V 0 s ν s ; x; s δ s SV nk s ν s ; ^x k ; s η. Therefore, 3 max x X φ n k ðx; sþ φ nk ð^x k ; sþ πðx; sþ η 3 þ δ V nk s πðx; sþ η 3 þ δ V 0 s ν s ; ^x k ; s ν s ; x; s η 3 : Takng y k to be a η/3-maxmzer of the maxmzaton problem above, we deduce that for all k bg enough φ nk ðy k ; sþ φðx; sþ η. Takng lmnf, nequalty A6) follows. Wth these prelmnary results, we are n poston to prove b). We wll prove that for any subsequence V nk V 0, where V 0 s some functon, TV 0 ¼ V 0. The result then follows from the unqueness property stated n Theorem A.2. Let x nk X nk s; V nk, s S. Snce X s compact, we assume wthout loss of generalty that x nk x eventually through a subsequence). Let x X 0 s; V 0 and consder yk as n Eq. A6). Then φðx; sþ lmnf k φ nk ðy k ; sþ lmnf k φ nk x nk ; s lmsup k φ nk x nk ; s φðx Þ: The frst nequalty s by constructon of the sequence y k. The second nequalty follows snce x nk X nk s; V nk. The thrd nequalty follows by defnton. The fourth nequalty holds by vrtue of Eq. A5). It follows that x X 0 ðs; V 0 Þ and that the sequence of nequaltes above s actually equaltes. Therefore, V 0 ðþ¼lm s k V nk ðþ¼ s lm k T nk V nk ðþ¼ s lm k φ nk x nk ; s ¼ φðx; sþ ¼ TV 0 ðþ, s provng the frst part of the proposton. Fnally, to see a), just apply the argument above to V nj V. Fnally, c) follows from a) and b). A.3. Provng Theorem π l δ ; π u δ For each, defne A has the set of functons a :S A and V as the set of functons v : S π l ; π δ u. Each of these sets s contaned n a δ vectoral space that, when endowed wth the product h topology, s a Hausdorff topologcal vector space. Snce A and are compact, Tychonoff's theorem Royden, 968, Chapter 9, Theorem 4) mples that A and V are compact. Addtonally, snce S s countable, A and V are metrc spaces Royden, 968, Chapter 8, Exercse 45). 7 We defne A¼ I A, and V¼ I V. We have therefore shown the followng result. Lemma A.5. AV s a convex compact subset of a lnear metrc space. Now, for I, consder the map Φ defned on AV by ða; vþ AV Φ ða; vþ ) ¼ a Aja ðþ s argmax Π x ; a ðþ s ; s; v forall s S : x A Ths map yelds the best Markov strateges ā for frm, gven arbtrary strateges for 's rvals and gven contnuaton values v. Observe 7 Observe that f S were not countable, the sets A and V need not be metrc spaces. The fact that these sets are metrc spaces s used n the proof of Lemma A.8. that the arguments v j j and a n the defnton of Φ appear only for consstency as wll be shown soon). It wll also be useful to nal down contnuaton values. To do that, defne the map T by ða; vþ AV T ða; vþ ¼ v V jv ðþ¼max s Π x ; a ðþ s ; s; v forall s S : x A Ths map yelds the solutons to the dynamc programmng problem faced by frm, gven the Markov strateges followed by 's rvals. Agan, the dependance of ths map on v j j I and a s just for consstency. Fnally defne Φ by Φa,v)= I Φ a,v)) I T a,v)). We state three prelmnary results. Lemma A.6. ΦðAVÞ AV. To see ths lemma, fx and note that for any ða; vþ AV, Φ ða; vþ A and T ða; vþ V. Ths mples that Φα; ð vþ AV. Lemma A.7. Φ s nonempty- and convex-valued. The proof of ths lemma s as follows. Snce the product of sets whch are nonempty- and convex-valued nherts these propertes, t s enough to prove that for each, Φ and T are nonempty- and convex-valued. Fx. Gven a :S A and a contnuaton value v : S R þ,foreachs the exstence of soluton for frm 's statc problem A2) s evdent; let a s) be such a soluton. By defnton, a Φ so that Φ s nonempty-valued. Moreover, Φ s convex-valued. Indeed, fx λ [0,] and consder ϕ,ϕ Φ a,v), where a; v AV. Then, for all s, ϕ ðþ; s ϕ ðþ argmax s x A Π x ; a ðþ s ; s; v.sncethe game has convex best reples, for all s, λϕ ðþþ s ð λþϕ ðþ s argmax x A Π x ; a ðþ s ; s; v. Ths observaton mples that λϕ + da)ϕ Φ a,v) and proves that Φ s convex-valued. Now, let us analyze T.Gvena :S A, Theorem A.2 n the Appendx mples the exstence of a sngle functon v V satsfyng the dynamc programmng condton A) for frm. Consequently, T, beng the set of all such solutons to Eq. A), s nonempty- and sngle-valued. The proof s complete. Lemma A.8. Φ has closed graph. Because AVsametrcspace, to prove ths lemma t s enough to prove that for any sequence ϕ n Φa n,v n )wthϕ n ϕ and a n, v n ) a,v), we have ϕ Φa,v). Ths result follows as an mmedate consequence of Proposton A.3 n the Appendx. We are now n a poston to provde a proof of Theorem. Lemma A.5 mples that AV s a compact convex set contaned n a metrc lnear space. From Lemmas A.6, A.7 and A.8, Φ : AV AV s nonemptyand convex-valued, and ts graph s closed. From Glcksberg's 952) generalzaton of Kakutan's fxed pont theorem, we deduce the exstence of a fxed pont ða; vþ AVof Φ:ā,v) Φā,v). It readly follows that, for each I and each s S, ā,v) satsfes condtons A) and A2). Lemma A. mples that ā s a Markov perfect equlbrum. Appendx B. Proof of Corollary 2 We wll defne a new game and apply Theorem to ths new game. Denote the set of probablty measures on A by P A.We endow the set P A wth the weak* topology; consult Chapter 5 n Alprants and Border 2006) for detals. Ths space s contaned