Investment and R&D in a Dynamic Equilibrium with Incomplete Information

Transcription

1 Investment and R&D in a Dynamic Equilibrium with Incomplete Information Carlos Daniel Santos y March 7, 27 Abstract In this paper I study industry behavior when rms can invest to accumulate both knowledge and physical capital. For this purpose I develop an algorithm to compute the (Markov) equilibrium for dynamic industry models which can be used in industries where the rm s information set is reduced. One of the problems with these models is that the existing algorithms require a solution to the full industry ( rm by rm) state space transition and when there are several state variables per rm (e.g. size and productivity) or when there is a large number of rms, the possible combinations increase dramatically making computations impossible. This constraints the analysis to very simple frameworks that cannot be applied to answer some economic questions requiring more complex settings. The algorithm here proposed does not su er from the curse of dimensionality and can also be applied to other elds in industrial organization, macroeconomics, growth or international economics. I show how introducing incomplete information and continuous states: (i) solves the existence problems of Ericson and Pakes (995) in a similar way to Doraszelski and Satterthwaite (25) and; (ii) reduces the dimensionality of the problem and its computational disadvantage. This is illustrated with a monopolistic competition framework where rms can invest in both physical capital and R&D and where they do not observe each others state. The individual states are continuous and the private information makes it an incomplete information problem. One extreme case of incomplete information used by Weintraub, Benkard and Van Roy (25) is when the rms only "know" the long run equilibrium distribution and act according to these beliefs. An interesting result from the simulations is that we no longer get neutrality to market size as in typical CES models. Keywords: Incomplete Information, Investment, Markov Equilibrium, R&D JEL Classi cation: C6, D2, D92, L, L22, O3 Introduction Industry behavior plays a crucial role in explaining the dynamic of an economy. Understanding individual rm s decisions is therefore fundamental to explain how an industry blossoms, matures and perishes. In this area, recent dynamic industry equilibrium models have been developed that London School of Economics and Center for Econonomic Performance, c.d.santos@lse.ac.uk y Acknowledgments: I would like to thank John Van Reenen and Philipp Schmidt-Dengler for excellent guidance. I would also like to thank Ulrich Doraszelski, Steve Redding and participants at the London School of Economics IO seminar for very usefull comments and suggestions.

2 explain these phenomena. We can observe even in very narrow industries that R&D performing rms coexist with non-r&d rms. Both productivity and size are very heterogeneous and entry and exit are positively correlated. One of the early models to address the question of selection was Jovanovic (982) where rms learned their productivity through time and this generated a rich pattern of entry and exit. In Hoppenhayn (992) the di erence is that instead of learning, the rms productivity is stochastic and this generates high correlation between entry and exit within industries in a stationary equilibrium. In both of these cases, there are no strategic interactions and the equilibrium is deterministic. The existence of a continuum of rms together with the law of large number and some mild conditions is what guarantees the existence of a stationary distribution (and a deterministic equilibrium). Ericson and Pakes (995) later introduced the concept of Markov Perfect Nash Equilibrium which accounts for all strategic interactions between the rms and where the equilibrium is Markovian on each rms individual state variables. This allows for very complex strategic interactions in the product market to be formulated, but since there is no analytical solution, numerical methods need to be used. However, the MPNE su ers from some problems, one was the non-existence of equilibrium that has been recently solved by Doraszelski and Satterthwaite (25), a second one is the curse of dimensionality and the computational burden. More recent algorithms like the one developed by Pakes and Mcguire (2) try to avoid this second problem. However, it is not likely that the concept can be applied to a broad range of situations. An alternative has been proposed by Weintraub, Benkard and Van Roy (25). They propose a new equilibrium concept, the "Oblivious Equilibrium". In this type of equilibrium rms disregard the current state of the industry and base their decisions upon the long run industry state. They also show that as the number of rms in the industry grow large, this converges to the MPNE. Obviously when the number of rms grow large we are back into a model similar to Hopenhayn and the equilibrium is deterministic. To better understand the curse of dimensionality problem, consider a complex model with several state variables per rm and/or large numbers of rms. Equilibria and policy rules are then impossible to compute since the size of the problem grows exponentially. For example, call s the industry state (i.e. if we de ne s it the state vector of rm i at time t, then the industry state is s = (s t ; :::s Nt )), nding the industry state transition, Q(s t+ js t ), for an industry with 5 rms and 2 discrete state variables would mean calculating a transition matrix. If one assumes the typical anonymity and symmetry (Pakes and Mcguire, 2) the problem will be greatly reduced but still intractable ( ). Once Q(s t+ js t ) is known, the problem then becomes a simple single agent problem and can be solved using available techniques (Judd, 998). For some time, computing dynamic industry equilibrium models was very di cult due to the low available computational power (Pakes and McGuire 994). The increase in computing power and the development of new algorithms (e.g. Pakes and McGuire 2) allowed the calculation of more complex dynamic problems. There has recently been an increased interest in the literature with some successful applications (Gowrishankaran 999, Pesendorfer and Schmidt-Dengler 25, Benkard 24, Ryan 25) and computation of more complex problems (Weintraub, Benkard and Van Roy 25). The estimation of these models is currently an area under research with some sucessfull applications (Pesendorfer and Schmidt-Dengler 23, Bajari, Benkard and Levin 25, Pakes, Ostrovsky and Berry 25, Macieira 26). I will construct a model that avoids the curse of dimensionality. The crucial assumption is that individual states are private information. This assumption gives two big advantages i) it reduces the dimensionality of the state space and ii) solves the non-existence in a similar way to Doraszelski 2

3 and Satterthwaite (25). The disadvantage is that I no longer have direct strategic interactions and in the typical Duopoly case this can be extremely costly. To understand how by reducing or increasing the information set, we reduce or increase the state space, take for example the extreme case where the rm does not observe anything about its rivals or the market. It is clear that trivially in this case we have a simple single agent problem to solve. The interaction between the rms and the market may vary from case to case. Whereas in some situations, direct interaction between rms are crucial like situations of duopoly or market leadership, in other situations the interaction may well be just between the rms and the market outcome and the strategic play between each individual rm plays a secondary role. It is in this second case that the model here presented is useful. The main advantage is that in these cases, we may well solve for complex models without su ering the curse of dimensionality so typical of the dynamic games. Beside the incomplete information assumption, there are other important assumptions like the representation of the aggregate industry state (as a mapping from individual states) and the imperfect signal conveyed by the aggregate state in order to guarantee that we cannot back out the individual states from the aggregate state. Given the beliefs about the evolution of the industry, which must be consistent with actual play, the problem becomes a simple single agent dynamic model. I will also argue that a bridge can be built between Weintraub, Benkard and Van Roy (992) and Ericson and Pakes (995) by modifying the rms information set. Whereas in E&P, individual states are public information, in WBVR the only thing which is common knowledge is the long run industry distribution. In my setting individual states are privately observed but the aggregate industry state (to be de ned later) is publicly observed. Since rms are not a continuum, the aggregate state is not deterministic and it will have a transition that generates rich aggregate time patterns. The algorithm to solve the model resembles a nested xed point where the inside loop computes the single agent problem and the outside loop calculates the equilibrium beliefs (see gure??). I use this algorithm to compute a rich, and at the same time manageable model of rm behavior which looks at optimal rm entry/exit decisions, capital investment and R&D. The model deals with optimal strategic investment and innovation decisions in a particular way. As with Ericson and Pakes (995), the strategic interaction arises from product market competition. The equilibrium outcome resulting from optimal behavior is characterized by optimal policy functions that depend on what else is happening in the industry. Some theoretical models exist that try to study the R&D decision in an industry framework like Vives (23) in a static setting or Klette and Kortum (24) in a dynamic setting. However, none of these has looked at the question of R&D and physical capital investment together and it is not a trivial issue to introduce it in the models. Besides, Vives (23) does not have heterogeneous rms and it cannot explain the "dual R&D behavior". The Klette and Kortum (24) framework does not allow for aggregate uncertainty and the equilibrium of the model is deterministic as in the case of Hoppenhayn. A problem with R&D, physical capital, entry and exit is already quite complex as there are 4 state variables and the number of rms can reach hundreds. The state is summarized by productivity, capital stock, R&D status and industry state (i.e. the state variables of all competitors, productivity, capital stock and R&D status). On the cost side there is irreversible investment, sunk costs of entering the industry and sunk costs of starting an R&D project. Current available algorithms would never be able to solve this whereas the algorithm here proposed computes it in 3

4 very short time. The main results show how demand elasticity, entry costs and market size a ects R&D performance, investment and the number of rms in the industry. The results show that in this model an increase in demand elasticity generate more R&D performance. The same is true for an increase in entry costs and market size. The most interesting feature is the fact that even tough demand has a Dixit-Stiglitz speci cation, I do not get the typical neutrality to market size. The reason for this relates with the fact that the aggregate industry state is a stochastic variable over which dynamic players have to integrate to calculate expected values. An increase in market size generates an increase in the number of rms but also an increase in the variance of the aggregate variable. Since the value of the rm is convex in the aggregate industry state, this will generate an increase in investment and R&D performance. This clearly contradicts models that are either static (Vives, 25) or have a deterministic equilibrium (Hopehnayn, 992; Weintraub, Benkard and Van Roy, 25). The rest of the paper is organized as follows. In section 2 I outline the model, section 3 explains the algorithm to compute the equilibrium, section 4 reviews some related literature, section 5 provides an application, section 6 contains the results, section 7 gives some extensions and nally section 8 concludes. 2 The model Each period there are N rms in the market (N t incumbents and Nt = N N t potential entrants). Incumbents choose actions that can be continuous a c it 2 Ac or discrete (exit, R&D start-up) a d it 2 f; g l, given their own state, s it 2 s i, and the industry state, s t = (s t ; :::; s Nt ). Throughout the analysis we will restrict to binary actions for a matter of simplicicty. For example, if rms choose to exit the industry they set a d it = and receive a "scrap" value. Potential (short lived) entrants may choose to pay a privately observed entry cost ( e i ) and enter the industry. Lets call it the strategies used by all rms, incumbents and potential entrants. Time is discrete and rms receive per period returns which depend on the state of the industry, current actions and shocks ((a it ; s t ) ). The timing is the following:. States (s t ) and shocks ( e i ) are observed, 2. Actions (a) are taken simultaneously (given the observed state), 3. Both stochastic and deterministic outcomes of actions are realized. New state is formed (s t+ ). The aggregate industry state, S t, can be de ned as a function of each incumbent s individual state but it is observed with some noise S t = S(s t ; :::; s Nt ) + " t. Assumption 2. There exists a function (S : s N! S 2 R) that maps the vector of rm s individual states (s t ) into an aggregate state (S t = S(s t ; s 2t ; :::; s Nt )+" t, where " t is iid distributed (" t ) and bounded support). Per period returns can be written as (a it ; s t ) = (a it ; s it ; S t ) The error " t is important to guarantee positive densities and to insure there is no perfectly informative state S t that would tell exactly (s t ; :::; s Nt ). This plays an important role for the existence of equilibrium. 4

5 Assumption 2.2 Individual states and actions are private information and Pr(s t js t ; :::S ) = Pr(s t js t ) Assumption 2.3 (No Spillovers) Conditional on current state and actions, own state evolves according to the transition function p(s it+ js it ; a it ) Assumption 2.2 implies that the only common information is the aggregate state. Moreover it says that all we can learn about the state of the industry s t is contained in S t and history (S t ; :::; S ) brings no more additional information. Proposition 2. Under assumption 2. to 2.3 industry state transition takes the form Q(S js) so that we can restrict to Markovian strategies i (s it ; S t ). Proof. From assumption 2.3 Pr(s t+ j t ; s t ; s t ; ::) = Pr(s t+ j t ; s t ) = Pr(s ;t+ j t ; s t )::: Pr(s N;t+ j Nt ; s Nt ) However, S t is observed instead of s t. Assumption 2.2 tells us that Pr(s t js t ; :::S ) = Pr(s t js t ) So that we have Pr(s t+ j t ; s t ) = Pr(s t+ js t ) Finally using assumption 2., all it is required to know is S t+ and knowledge about s t cannot be recovered as shown above, we get Pr(S t+ js t ; S t ; ::::) = Q(S t+ js t ) So while s t is a vector s t = (s t ; s 2t ; :::; s Nt ), S t is a scalar variable that maps individual rm s states into an aggregate industry state S t = g(s t ; s 2t ; :::; s Nt ) + " t. Example 2. CES demand: S t = P N = s it + " t Example 2.2 Logit demand: S t = P N k= es k;t + " t Corollary 2.2 Under assumptions,2, and 3, as N becomes large Q(S js) is approximately normally distributed with mean NE(s js) and variance NV (s js). Proof. The proof is straightforward using the central limit theorem. Given Proposition and Assumption 3, we can write the continuation value Z EfV i (s it+ ; S t+ )js it ; S t ; a it ; Qg = S t+ Z s i;t+ V i (s it+ ; S t+ )p(s it+ js it ; a it )Q(S t+ js t )ds i;t+ ds t+ : This continuation value depends on the beliefs about the transition of the aggregate state. These beliefs depend on the equilibrium strategies played by all players. Note that since rm i does not observe s jt ; 8j 6= i, it can only form an expectation on it s rivals actions conditional on the information available S t, j (S t ) = R s j j (s j ; S)g(s j js)ds j where g(s j js) is the probability density function of rm j s state conditional on S. We can then say that our assumption is playing the same role as mixed strategies or privately observed information in D&S. For this reason there is no need to introduce stochastic exit values since the rm can only attach probabilities to the outcome of exit (we could do the same about entry if the rms observe their state before deciding upon whether to enter or not). Note that this is the main problem with the original E&P framework which has been shown by D&S not to have 5

6 2. Actions and strategies For each state rms can take actions from some space a it 2 A:I restrict attention to Symmetric Markovian Pure Strategies 2. Strategies are a mapping from the set of states onto the action space : s S! A ( it (s it ; S t ) = ( c it (s it; S t ); d it (s it; S t ))) where the action space is f; g [; a A(s it ; S t ) = c ] if s it 6= s e f; g s it = s e Using symmetry we can drop the i subscript and imposing the markov restriction we can drop the t subscript: it (s it ; S t ) = (s it ; S t ): 2.2 The incumbent s problem Incumbent rm i has to solve an intertemporal problem taking into account the equilibrium evolution of the aggregate state, without knowing its competitors state (s jt ; j 6= i). V i (s it ; S t ; Q) = sup (s it ; S t ; a it ) + ( a it2a a d it) +a d it fefv i (s it+ ; S t+ )js it ; S t ; a it ; Qg Exit decisions are made according to d if EfVi (s (s it ; S t ; Q) = it+ ; S t+ )js it ; S t ; a it ; Qg otherwise and actions (conditional on not exiting) c (s it ; S t ; Q) = arg sup a c it 2A (s it ; S t ; a it ) + E [V i (s it+ ; S t+ )js it ; S t ; a it ; Q] 2.3 The entrant s problem Entrants have to decide on wether or not to enter the industry and pay an entry cost e i, given their "outside state" s e and the industry state S t. Upon entry their state is drawn from a distribution Pr(s js e ). There are N potential entrants each period and they are short lived, meaning that they do not take into account the option value of delaying entry. Given this, the entry problem can be described as. V e (s e ; S t ; Q) = sup a d it a d it 2f;g e i + E V i (s it+ ; S t+ )js e ; S t ; a d it; Q So the entering rule takes the simple form d (s e if e ; S t ; Q) = i EfV i (s it+ ; S t+ )js e ; S t ; Qg otherwise necessarilly an equilibrium in pure strategies. In my framework I just need to guarantee that given the aggregate state, rm i can never know with certainty the state of the industry, i.e. g(sjs) > for all (s; S). So that any aggregate state S t is never perfecly informative. This is done by the introduction of an iid shock " t in S t = g(s t)+" t. 2 Anonimity as de ned in E&P is imposed by assuming that rms do not observe each others state. 6

7 Assuming entry costs are independent and identically distributed amongst potential entrants with some distribution F ( e ), integrating out over the distribution of entering costs, the probability of rm i entering can be de ned as e (S t ) = F ( e (S t )) where e = R S t+ Rs i;t+ V i (s it+ ; S t+ ) Pr(s it+ js e )Q(S t+ js t )ds i;t+ ds t+ If there are Nt potential entrants in the market, the expected number of entrants when the industry state is S t is (S t ) = N t e (S t ) Given that the probability of one rm entering is e (S t ), the probability that j < Nt is a random variables with a binomial distribution and parameters (Nt ; e (S t ) ). rms enter 2.4 Equilibrium The equilibrium concept is one of Markov Perfect Equilibrium as de ned in Maskin and Tirole (2). Since I will look at a pure markovian strategies the rm can take actions in state a it 2 A(s; S) and the problem can be represented as where V (s it ; S t ; Q) = sup h(s; S; a; V ; Q) a2a(s;s) = h(s; S; a; V ; Q) 8 < (s it ; S t ; a it ) + ( a d it ) i +a d it : fefv (s if s it+; S t+ )js it ; S t ; a it ; Qgg it 6= s e a d it f e i + EfV (s it+ ; S t+ )js it ; S t ; a it ; Qgg if s it = s e De nition 2. The equilibrium is. Firms strategies conditional on beliefs about industry evolution ( it = (s it ; S t ; ~ Q)) optimize the value function V (s it ; S t ; Q) = h(s; S; (s; S; Q); V i ; Q) 8 2 ; s 2 s; S 2 S where E [V (s i;t+ ; S t+ )js it ; S t ; it ; Q] = R R s2s S2S V (s i;t+; S t+ ) Q(s ~ it+ ; S t+ js it ; S t )dsds and ~Q(s it+ ; S t+ js it ; S t ) = Q(S t+ js t )P (s it+ js it ; it (:jq)) 2. The transition matrix (Q (S t+ js it ; S t ; (s it ; S t jq))) resulting from using optimal strategies ( it ) de ned above is consistent with beliefs Q(s it+; S t+ js it ; S t ) The solution to the single agent dynamic programing conditional on Q provides optimal strategies (:jq) (a solution to this exist under Blackwell s regularity conditions). These strategies will then characterize the industry state transition Q(S t+ js t ; ) and the equilibrium is the xed point to a mapping from the beliefs used to obtain the strategies into this industry state transition (Q)(S t+ js t ) = Q (S t+ js t ; (:jq)) where rm s follow optimal strategies a (:). All we need is to prove that there is a xed point for the mapping (Q) : Q! Q 7

8 2.4. Existence Theorem 2.3 An Equilibrium Q exists. Proof. See the appendix. The solution to this problem is then a stationary stochastic Markov Equilibrium. 2.5 Discussion The model above brings some of the ideas developed in Doraszelski and Sattertwhaite (25) about the existence of equilibrium in the original Ericson and Pakes (995). Instead of introducing iid stochastic shocks for the discrete decisions, I introduce incomplete information in an extreme form that no rm knows their competitors individual state. By having this, rms have to attach probabilities to the outcomes which eliminates the discreteness that caused non-existence problems in the E&P framework. An important assumption is that the aggregate state is never perfectly informative as otherwise we would again face the original problem. By using the incomplete information I also considerably reduce the burden of computation and we no longer have the curse of dimensionality. The disadvantage is that I do not allow some types of strategic interactions where rms actions directly depend on competitors states. In a sense this is imposing more restrictions to the usual anonymity and symmetry assumptions (which are also fundamental to reduce the dimension of the state space). In the case of E&P, these assumptions of symmetry and anonymity are also a restriction on the evolution of the state space that allows them to characterize industry structures more compactly as a set of "counting measures". What it says is that s i captures all relevant rm heterogeneity and that given this state, rms are indistinguishable. As in the original Ericson and Pakes (995) article, interactions arise through market demand. By reducing the information set (Pr(S t+ js t ; S t ) = Q(S t+ js t )), we also reduce the importance of strategic interactions since rms no longer take into account some of the competitors state (either the whole state or some part of it). This resembles a monopolistic competition framework but instead with a Dynamic Stochastic Markov Equilibrium. The link between MPNE and monopolistic competition models is determined by Assumption 2. The problem with the Full Information MPNE (FIMPNE) is not to solve for Pr(s t+ js t ) in itself but the fact that this object can have a very high dimensionality that makes it impossible to be solved with current (and future) computational power (the industry state transition matrix could easily have millions of data points). As explained before, in industries with either many rms and/or big state spaces, the industry state, s t, will grow very rapidly making any solution to the problem impossible. Take for example an industry with 5 rms and 2 discrete (binary) state variables. This would mean calculating a transition matrix and even tough introducing anonymity and symmetry reduces the size of the problem, it will still be intractable ( ). Once Pr(s js) is known, the problem then becomes a simple single agent problem and currently available techniques can easily be used (Judd, 998). Therefore, it seems that restricting the information set might be a reasonable solution to circumvent the problem. Current empirical applications involve estimating Pr(s js) from the data as this does not require solving for it (Pakes Ostrovski and Berry 25 and Bajari Benkard and Levin 25), however as shown above if the industry state is large this requires thousands or millions of datapoints and it doesn t solve the dimensionality of the problem either. A second solution similar to mine, proposed by Weintraub, Benkard and Van Roy (25) is to compute the "oblivious equilibrium" de ned as an approximation to the true equilibrium. The main shortcomings of this 8

9 is that it performs badly in the presence of aggregate shocks and it is not exible to the inclusion of relevant information in the state space. Assumptions 2. and 2.2 are fundamental and whereas the rst is not problematic, the same cannot be said about the second 3. The rst is that the impact of the industry state on per period returns can be summarized in one function (S t = g(s t )). This is not very restrictive as most reduced form pro t functions satisfy it. In the example below this comes from the demand side. The second is that rm i does not observe s jt ; 8j 6= i. It plays a central role since it allows us to greatly simplify the problem by reducing the state space. This is crucial since it reduces the computation of Pr(s t+ js t ) to Q(S t js t ). Given this importance it is worth spending some time on its meaning. For example, imagine the state variable is price, this means that rms observe industry aggregate prices (e.g. published by some entity) but they do not observe rms individual prices because this is costly market research. One important part of the assumption is that rms cannot recover individual states either. For explanatory convenience I assume the information set is S t but other cases might be considered as well. Imagine that there is a clear market leader in the industry, publicly traded or with publicly available nancial information. This means that Assumption 2.2 is violated. Even in this case there is a way of implementing the methodology by enlarging the state space as I propose below. If there is a market leader we just need to increase the information set so that it includes the state of the leader. We now have to solve two di erent problems, one for the leader and one for all other rms but it is still a much simpler case than the FIMPNE. The state space becomes (s it ; S t ; s Lt ) where s Lt is the state of the leader. The algorithm is also exible to allow di erent demand structures. In the example below I use Dixit-Stiglitz demand as an illustration but as explained, one can also use logit or more complex demand systems. The su cient condition is that the resulting per period returns respect Assumption 2., i.e, the industry state a ects the rm through some aggregate index S t = g(s t ; :::; s Nt ). The problem left is how to recover the transition Q(S t+ js t ) and I will present an algorithm to do this using simulation in the next section. We don t need to keep track of every rm s state and it su ces to have knowledge of the aggregate industry state. While this solves the curse of dimensionality problem and the rm s problem greatly simpli es, in many IO studies with oligopolistic industries assumption 2.2 may be questionable and other frameworks need to be adopted to study those cases. However as I will show below, it still applies to a large number of cases. Provided the rm knows the evolution of the aggregate state, Q(S t+ js t ), the problem then becomes a single agent and we can estimate it similarly to Bajari, Benkard and Levin (25) and Pakes, Ostrovsky and Berry (25). Bajari et al (25) use observed actions to estimate directly the policy functions from the data and use them to simulate the state transitions. The main problem with their approach is that there mustn t be any unobserved (or non-recoverable) state variables, otherwise, policy functions will not be correctly estimated. Using a similar approach, Pakes et al (25) compute the state transition from the data and use this to get an estimate of the continuation (and entry) values. The main drawback of this approach is that it doesn t solve the curse of dimensionality problem as if we have an industry with many rms and many state variables, the state transition one needs to compute will have thousands or millions of points and will require immense data to be calculated so the methodology is only applicable in cases with 3 Assumption 3 ("no spillovers") is normal in the literature, but not innocent. It allows us to write down the transition for the individual state conditional on the rms actions independently of the other rms action/states. 9

10 small state spaces. Note that the model can easily incorporate aggregate industry shocks 4 whereas previous works could not (Weintraub, Benkard and Van Roy (25), Hoppenhayn (992)). 3 Algorithm The algorithm to compute the equilibrium described above is simpler and easier to run than the available algorithms (Pakes and McGuire 994, 2). Instead of computing the full information transition matrix Pr(s t+ js t ) we only need to compute Q(S t+ js t ) with a much smaller dimensionality. The algorithm works as follows (see gure ):. Set down the primitives of the model (s it ; S t ; a it ),,F ( e ),, P (s it+ js it ; a it ); ("). 2. Fix the beliefs for the evolution of the aggregate state, Q (S t+ js t ) and solve a single agent dynamic problem to get the policy and value functions ((s it ; S t ), V (s it ; S t )). 3. With the policy functions (), simulate sample paths for the industry by simulating draws from P (s it+ js it ; a it ). Recover Q (S t+ js t ) from these simulated paths. 4. Repeat this process until Q j+ (S t+ js t ) = Q j (S t+ js t ). The simulation of industry sample paths is crucial as it would be impossible to build Q(S t+ js t ) just using the policy functions since otherwise we would need to calculate Pr(s) and Pr(s js; S). To calculate this would mean we are back in the curse of dimensionality problem. 3. Stopping rule The algorithm stops when the di erence between Q j+ (S t+ js t ) and Q j (S t+ js t ) is small enough. I use a distance measure to calculate the di erence between j + and j simulated transition matrix and stop when this distance is smaller than a prede ned o. where jjq j+ (S js) jjq j+ (S js) Q j (S js)jj is de ned as Q j (S js)jj < o jjq j+ (S js) Q j (S js)jj = SX SX i= l= j Qj+ il Ns j+ l Q j il Ns j l j: Qj+ il Ns and Q j+ il = Q j+ (S = ijs = l), Ns j+ l is the number of simulation draws at j + that have reached state l and Ns is the total number of simulation draws ( Ns = P S l= Nsj l ). The error (o) cannot be set in nitesimally small since there are approximation and simulation errors. Approximation error arises from the use of numerical approximations to solve the problem. Simulation error is caused by the fact that we cannot simulate an in nite number of times at each state due to computational restrictions. 4 Below I explain how to do this.

11 3.2 Convergence Convergence of the algorithm is not guaranteed. The causes may be diverse: (i) Not all points in S are necessarily visited and the transition may be inaccurately estimated in each step (j) for non-visited or low visited states; or (ii) we need to discretize the state space and this may create discreteness in the mapping. To illustrate this, take the simple case where we want to compute the degenerate case where S is constant. Imagine that the algorithm behaves as in gure 2. It is easy to see that convergence will not be achieved and the algorithm will jump from state to state. The problem is that close to the xed point we are trying to nd, the slope is bigger than. This is because the algorithm fails to be a contraction, eventough conditional on Q(:j:) the value function is still a contraction. If we could transform the problem in a contraction we would guarantee convergence but in general the xed point for the equilibrium solution is not a contraction and we may face this problem. Fershtman and Pakes (25) and Pakes and Mcguire (2) face similar non-convergence problems and discuss them very brie y. However they ignore the fact that these non-convergence problems come from discreteness created from entry and exit. To insure that we nd the equilibrium, I use a weighting method to build the j + th iteration transition matrix. Q j+ (S js) = {Q j (S js) + ( {) Q j+ (S js) where { 2 [; ] and Q j+ (S js) is the j + simulated transition matrix. The new Q j+ is a weighted average of the Q j and the simulated transitions. A higher weight { insures convergence but at the cost of longer time and more iterations until convergence 5. This type of solution has been used in the literature (Judd 998, Fershtman and Pakes 25, Pakes and McGuire 2). 4 Related literature The literature on industry dynamics has been extensive with some of the rst models of rm behavior and industry evolution being Simon and Bonini (958), Mans eld (962) and Jovanovic (982). Jovanovic s (982) model describes the industry evolution in the case rms take time to learn their productivity. There is no stochastic environment in terms of productivity but there is a problem of information and learning so that rms will decide on exit/staying depending on what they observe and they learn about their true unknown productivity. He then solves a perfect foresight equilibrium with exit behavior and size distribution that matches some stylized facts. The distribution of rms is deterministic since there are no stochastic shocks (only lack of information) and in the limit there is no entry or exit. Variability for surviving (older) rms should be lower than for younger rms, also, considering the same cohort, small rms are less productive than big rms. Hopenhayn (992) proposes a di erent framework where productivity evolution is stochastic. As in Jovanovic, the only decision is about entry and exit but there is stochastic evolution rather than uncertainty about the productivity. There are no industry-wide shocks, which implies that aggregate output, employment, prices and the frequency distribution for productivity follow a deterministic path and this is crucial to generate a stable distribution. He shows that under certain conditions there is a stationary equilibrium with positive entry and exit. 5 In all simulations I have run, using a { = :7 insured covergence.

12 Ericson and Pakes (995) provide a model of industry dynamics where rms choose entry and exit but also investment that generates stochastic productivity shocks. The decisions are taken in a strategic environment where rms will build capacity according to what they expect their competitors to do. This generates a Markov -Perfect Nash Equilibrium (MPN). The industry is said to be in dynamic equilibrium when the process generating the change in industry structure is accurately re ected in the beliefs of each of the rms. In this case, the limiting structure is not deterministic as in Jovanovic (982) and Hopenhayn (992). Klette and Kortum (24) have a simple model with a closed form solution for the distribution of rms but rich enough to match a range of stylized facts about productivity, R&D, size and growth. It accounts for the persistence over time of rms R&D investment, the concentration of R&D among incumbent rms, and the link between R&D and patenting. It also explains why R&D as a fraction of revenues is strongly related to rm productivity yet largely unrelated to rm size or growth 6. The two models closest to mine are Ericson and Pakes (995) and Klette and Kortum (24) since these allow for endogenous R&D choices but whereas E&P is impossible to implement due to the curse of dimensionality, the K&K framework is too restrictive by not allowing the introduction of physical capital investment and having a deterministic rate of innovation. On the other hand the methodology proposed can t between Hopenhayn (992) and E&P since whereas I also have a dynamic equilibrium typical of E&P (re ected in the beliefs about its evolution), I still disregard strategic interactions at a rst stage like Hopenhayn. Some other di erences in the application below are. Inclusion of capital investment decisions together with R&D decisions. This is important since rms decision to do R&D can depend on their size and the interaction may be important 2. Introduction of sunk costs of R&D start-up 3. Allowing more general stochastic shocks then the unitary restriction imposed by E&P. 5 An application - Monopolistic competition As an illustration I use a model which can not be solved with the current methods. The setting is the following. Products are di erentiated and rms can a ect their own productivity by choosing to do R&D. R&D can only be done after paying a sunk cost (e.g. to build the R&D lab). Afterwards, rms choose levels of R&D expenditure that generate stochastic increases in productivity. Firms can also invest in capital with a deterministic outcome. 5. State and action space The state space is represented by four variables: Physical capital, productivity, R&D status and active/inactive status 6 The empirical literature in this area is extensive (Acs and Audretsch (987, 988), Dunne, Roberts and Samuelson (988), Cohen (995), Griliches (99, 998, 2), Klette and Griliches (996), Sutton (997), Caves (998), Doms and Dunne (998), Bartelsman and Dhrymes (998), Cabral and Mata (23), Mortensen and Lentz (25)). There are some main stylized facts about sales, growth, R&D, productivity dynamics and entry & exit summarized by Klette and Kortum (24) and Griliches (2). 2

13 s it = (K it ;! it ; R it ; it ) where K it 2 K;! it 2 ; R it 2 f; g; it 2 f; g where = means the rm is active and R = means the rm has paid the sunk cost to built the R&D lab. Both K and! are de ned in a discrete (approximated) space even though they are continuous variables. After entering the industry, rms can invest in physical capital, pay a sunk cost and start doing R&D (this is done only once and R&D can be done forever), choose optimal R&D level and nally decide on exit from the industry. it = (a it ; it ) = (I it ; R it ; l it ; it+ ) where I it 2 I; l it 2 l This generates a Markov law of motion for the state variables that depends on the actions s i;t+ = s(s it ; it ) As will be explained below, this law of motions will be stochastic for productivity and deterministic for all other state variables. Both productivity and capital will need to be discretized for the numerical calculations 5.2 Models primitives 5.2. Period returns Per period returns are a primitive of the model which must be speci ed (s it ; S t ; a it ). Demand Using the Dixit-Stiglitz monopolistic competition framework as speci ed by Klette and Griliches (996) and Melitz (2) 7. There are N t available goods, each supplied by a di erent rm so there are N t rms in the market. Consumers choose quantities of each good Q i to consume with quality i and pay P i with the following preferences X i ( i Q i )! ; ZA With U(:) di erentiable and quasi-concave and Z represents aggregate industry shifters. i represents consumer valuation of good i. Changes in i over time could come from two e ects: the quality embodied in the good changes (the actual product is changing) or the consumer s idiosyncratic preferences across varieties change. The aggregate price index is ~P = and demand becomes (see Appendix A.2): NX i=! i P i () Q i = ~ Y ~ P i P i (2) 7 As refered before the model may work with other demand structures. The adoption of Dixit-Stiglitz demand is a choice of convenience. 3

14 P ~Y~P Nt i= Where = PiQi is total industry de ated revenues. If the goods were perfect substitutes ~P ( is in nite), then there can be no variations in quality adjusted prices across rms and Pi i = P ~ Y for all rms. Then ~ i = Q P ~ i i for all rms. Production function The production technology uses both capital (K) and labor (L) with a given productivity factor (A) according to a cobb douglas P j Q i = A i L i K i (3) It is easy to show 8 that maximizing out for labor, ~ = P (Q)Q wl becomes, ~(! i ; K i ;!K; ; ) =! i K i ~Y h i (4)! j K j where h! i i = A i i and 2 [; ( ) ]. We call! productivity from here onwards, S =! j K j is the state of the industry and (; ) are the elasticity of substitution and capital coe cient, respectively. It is important to note that since in the short run, productivity and physical capital are xed, the only way to adjust production is through the labor market which is assumed to be perfectly exible. Productivity One characteristic of R&D expenditures is the uncertainty and its stochastic e ect on pro ts. This can arise from the revenue side (product innovation, i.e. either by developing a new product or improving the quality of an existing product R&D outcome changes the revenues and therefore the pro ts of the rm) or from the cost side (process innovation, i.e. by changing a current process of production, or improving the use of resources the R&D outcome will a ect the costs of the company and therefore its prices). In general, product and process innovation are very di cult to disentangle from each other unless one has rm level price data. It is because of this that I consider them to be indistinguishable in the model. This distinction would be important to model other type of phenomena, like for example dynamic pricing, as the e ects on R&D would be qualitatively di erent and the researcher would need to take this into consideration. This "internal" source of uncertainty distinguishes R&D investment from other rm s decisions like capital investment, labor hiring, entry and exit which have deterministic outcomes and where the only source of uncertainty a ecting them is "external" to the company (e.g. the environment, competition, demand). This distinction is important since the stochastic R&D decisions and productivity will determine the stochastic nature of the equilibrium (Markov). Whereas we can also generate a similar Markov Equilibrium by using exogenous stochastic shocks, in our model the stochastic equilibrium is endogenously determined. Cost function 8 See the Appendix A.3. P j 4

15 Investment cost Investment cost takes the traditional quadratic adjustment form (Hayashi 982). Non-linear adjustment costs that would allow us to generate more realistic investment patterns can also be used (Cooper and Haltiwanger 995 and 2). C K (I t ; K t ) = (I t) 2 K t + p k I t (5) There is irreversibility in the sense that there is no disinvestment, modeled as restricting I t 2 R +. R&D technology To start doing R&D, rms have to pay a sunk cost of $ (e.g. to build the R&D lab), after paying this, they choose R&D expenditure level that translates into (l) ideas and (j) stochastic innovations that nally generate increases in productivity!. Note that we have to introduce a binary state variable to track if the R&D sunk cost has been paid or not R 2 f; g. The speci cation of how R&D expenditures produce "innovations" is important since di erent speci cations can predict di erent outcomes. In this case we want "innovations" to be de ned broadly as they could be both increases in productivity (A i ) or quality ( i ). Following Klette and Kortum (23), I use an ideas generating function l = H(R&D). This is a deterministic function that given a certain amount of R&D expenditures (RD) produces l ideas. Innovations are then a stochastic event. After the ideas have been produced deterministically according to H(RD), j innovations happen stochastically with some probability q(jjl), with j l. I will use the following constant elasticity functional form for the ideas generating function l = ard Inverting we get the R&D cost function so that rms choose the number of ideas they want to produce (l) and this translates into a cost RD. RD(l;!) = cl (6) Entry cost Potential entrants are short lived and cannot delay entry. Upon entry, rms must pay a (privately observed) sunk entry fee of e i to get a draw of! with distribution Pr(! j = ) next period. The capital stock level upon entry is xed K = K and R =, i.e., rms enter the market with a capital stock of K and no R&D. Active rms take a value = and inactive rms =. Exit value exit value of. Every period the rm has the option of exiting the industry and collect a scrap State transition Productivity is stochastic and follows a Markov process whereas entry/exit, R&D start-up and Capital are deterministic decisions. where v is an exogenous Poisson process! it+ =! it + j it v it p(v) = exp( )v v! (7) 5

16 and j is a binomial process So that q(jjl;!) = q(jjl) = l p j ( p) l j (8) j P (! it+ j! it ; l it ; R it ) = P (! it + j it v it j! it ; l it ; R it ) =! it + R it q(j it j! it ; l it ) p(v it ) This speci cation is similar to Ericson and Pakes (995) but it allows increases (and decreases) in productivity to be more than unitary. This is important because even tough they argue that unitary process can be generalized by changing the time interval, there is inconsistency between the timing of the shocks and the timing of decisions (i.e. rms can act immediately upon a negative shock whereas in my case this shock may be big and so rms response late). So this framework allows for a more generic stochastic process where a rm can either "win the lottery" or "be killed". Given all this we get the per period return function = (! it ; K it ; R it ; it ; R it+ ; it+ ; I it ; l it ) = 8 < ~Y : it (! itk it) P j[! jtk jt] I 2 it K it p k I it cl it $(R it+ R it )R it+ ( it ) it+ e i + ( it+ ) it 9 = ; where we can de ne the aggregate industry state as the sum over all rms of a function of productivity and capital stock S t = P h. j! jt Kjti Using the demand speci ed above (2) there are two external variables that a ect company s revenues. One is market size ( Y ~ ) and the other is competitors quality adjusted price index ( P ~ ). Since individual quality adjusted prices are determined by productivity and physical capital (Pi = i = P (! i ; K i ; P ~ ), see appendix), the quality adjusted price index is a mapping from individual rms productivity and capital stock onto a pricing function so that we get the nal result for the aggregate state variable S t = X h! jt Kjti (9) j It is important to remember that as explained before, the rm adjusts production to maximize short run pro ts through the only exible input, labor. 5.3 Value Function The value function for the rm is where V (s it ; S t ; Q) = suph(s; S; a; V ; Q) a2a 6

17 = h(s; S; a; V ; Q) 8 < (s it ; S t ; a it ) + ( it+ ) if + : it+ fefv (s it+ ; S t+ )js it ; S t ; a it ; Qgg it = e i+ f e i + EfV (s it+ ; S t+ )j it = ; S t ; Qgg if it = = sup a2a V (s; S) = () (s; ; S) + E [V (s )js; a; S] where s = (!; K; R; ), a c = (K ; l) and a d = (R ; ) and the expectation E[:js; ] is taken over Pr(! j = )Q(S js) (and K = K and R = ) if = and P (! j!; l; R)Q(S js) if =. So the rms decide on next period capital investment (K ), R&D start-up (R ), R&D expenditures (l) and next period status, i.e. entry and exit ( ). Firms optimally choose their entry, exit, R&D and investment given the kn owledge of the evolution of the industry Q(S js). The main computational burden is to calculate this state transition that respects the equilibrium condition. For that I have written a Matlab code that uses the algorithm speci ed earlier 9. Entry Given the above speci cation, it is easy to understand that there will be entry whenever the expected value of entry is bigger then the entry cost. However one problem di cult to solve is how many rms will enter. The literature has adopted several approaches in this case. One is to allow only one entrant per period. Another is to assume that the number of rms is always xed and they enter and exit the market whenever that decision is optimal. In this case I will follow D&S and assume there is an entry cost distribution F ( e ) and each potential entrant observes his own entry cost. The entry cost distribution is uniform with parameters U[; ]As explained in section 2, the number of entrants is a random variable with a binomial distribution with parameters (Nt ; e (S t ) ) and mean (S t ), where e (S t ) = F ( e (S t )) and e (S t ) = R S t+ Rs i;t+ V i (s it+ ; S t+ ) Pr(s it+ js e )Q(S t+ js t )ds i;t+ ds t+ is the entry cost threshold. 6 Simulations In this section I will present some results for the equilibrium solution to the dynamic monopolistic equilibrium presented above. I simulate a sample path of rms and do some parameter changes. The results illustrate the behavior of the model and the importance of looking at dynamic rather than static equilibrium. 6. Baseline case 6.. Parameters The following parameters were used as the primitives to the model 9 The code is available from the author upon request. 7

18 Binomial process parameter ( p).6 Depreciation rate (). Poisson process parameter () 2.5 R&D price (c).25 Capital Share ().35 Discount factor ().9 Elasticity of Substitution () 6 Entry Cost min () 2 Market Size ( Y ~ ) 4 Entry Cost max ( ) 3 Physical Capital Adj. Cost ().5 Exit Value ().5 Price of Physical Capital (p K ) R&D Start-up Cost ($). Table - Baseline Parameters Some of these parameters are taken from the literature whereas others are chosen to be within reasonable values. Following the empirical literature on investment and physical capital I use a capital depreciation rate () of % as used by Bond and Soderbon (25). The capital coe cient in the production function, :35 is in line with the ones found by Olley and Pakes (992), Griliches and Mairesse (995) and others. The quadratic adjustment cost for capital investment is :5 as estimated by Ejarque and Cooper (2) using a method to correct for market power. The price of capital is normalized to. The elasticity of demand () re ects a mark-up of 7% within the values in the literature, normally between % and 4% (Martins, Scarpetta and Pilat 996, Berry, Levinshon and Pakes 995, Hubbard, Kashyap and Whited 995). The discount factor is 9% close to the value used by Pakes and Mcguire (994, 2). The scrap value () is set 25% below the minimum entry cost value so that entry followed by exit is never a pro table strategy. The sunk R&D cost ($) and the R&D price (c) were adjusted so that we get sensible R&D behavior (note that these can be scaled up or down as prices are relative and not absolute). The size of demand ( ~ Y ) and entry cost bounds were also adjusted to have a reasonable number of rms in the market (both of these will be changed in the sensitivity analysis part). Finally the parameters of the stochastic outcomes p and are from binomial and Poisson distributions with the corresponding interpretations Results To compute the equilibrium I discretize the state space for productivity (!), capital (K) and industry state (S). To calculate the transition matrix I use the computed policy functions and run 25 industry simulations for 4 time periods. Finally the statistics are computed from simulated data on an industry panel with time periods where the rst 25 periods are dropped. The time required to compute the equilibrium depends on the approximation, ", and the convergence weight, {. I have used an o = 3 and a convergence weight { = :7. This required around -2 outer loops which took approximately 6 minutes to run on a P4 2.8 Ghz, Gb RAM laptop. Figure 3 illustrates the solution to the problem, by representing the value and policy functions in the (!; K) space for a give industry state (S). The value function for R&D and no R&D rms are quite similar in shape with the value of R&D rms being up to % bigger, i.e., V (!;K;R=) max!;k V (!;K;R=) < :. For investment levels again di erences are not signi cant but R&D performers tend to invest slightly more. Investment for big low productivity rms is zero and it is highest for small high productivity rms. R&D start-ups have a U shape in the (!; K) space. One of the main problems with the tobin s Q framework has been the very high estimated adjustment costs. Ejarque and Cooper (995) argue that the problem is related with market power and propose a methodology that signi cantly lowers the estimates by introducting some curvature into the pro t function. 8