Neoclassical Growth Model / Cake Eating Problem

Transcription

1 Dynamic Optimization Institute for Advanced Studies Vienna, Austria by Gabriel S. Lee February 1-4, 2008 An Overview and Introduction to Dynamic Programming using the Neoclassical Growth Model and Cake Eating Problem Good references for this course are Stokey et al. (1991), Ljungqvist and Sargent (2000), Sargent (1987), and Adda and Cooper (2003). And of course, one can always go to the original source of.bellman (1957) for Dynamic Programming. This overview follows Chapters 1 and 2 of Stokey et al. (1991) closley: you might want to read these two chapters before starting the class. Neoclassical Growth Model / Cake Eating Problem We ll start this course with a brief introduction to the deterministic neoclassical growth model of Ramsey (1927) and Cass (1965), and "Cake eating" problem of Gale (1967). These simple models are the foundation for our current macroeconomics, especially for looking at business cycle issues. Cooley and Prescott (1995) is a good reference for how growth models can be used to study business cycles. 1. A Cake Eating Problem of Gale (1967) Suppose that I gave you a cake of size f today. You only live for T periods and the time is discrete. Further, you can only either consume or save (there is nothing else to eat to survive, or no borrowing from anyone else). The cake doesn t get spoil (i.e. no depreciation). At each point in time, t = 1; 2; 3; ::::T; you have to make a decision on the amount of consumption and saving. So, how would you determine the optimal amount of cake consumption (saving) at each point in time? To answer this question, we would need to know about the properties of your preference, your time discount factor, and initial/terminal conditions.

2 Suppose I tell you that your preference is stationary (not indexed by time) and there is a real value function, u (c t ) that represents the ow of utility from the consumption at each period, c t : Also, imagine that u (c t ) is twice di erentiable, strictly increasing and strictly concave, with limu 0 (c) = 1 and that limu 0 (c) = 0 c!0 c!1 (i.e. Inada conditions).represent lifetime utility by TX t 1 u (c t ) (1.1) t=1 where 0 1and is called the discount factor (e.g. time preference). Since that the cake does not depreciate, the evolution of the cake over time is governed by: f t+1 = f t c t (1.2) for t = 1; 2; ::T. Now, how would you nd the optimal path of consumption, fc t g T 1? We ll come back in solving this problem, but I want to now turn to the neoclassical optimal growth model to see how these two models are related. 2. Neoclassical Growth Model Ramsey (1927) proposed a question "How much of its income should a nation save". In this economy, there are many identical, in nitely lived households. In each period t there is a single good, y t, that is produced using two inputs: capital, k t, in place at the beginning of the period, and labor, n t. A production function relates output to inputs, y t = F (k t ; n t ) : In each period current output must be divided between current consumption, c t ; and gross investment, i t (sounds familiar to cake eating problem?). In Ramsey s problem, this representative agent was a ctitious planning authority The aim is to maximize this agent s utility function given his choice of consumption and investment that is bounded by his current production. 2

3 The nite horizon Ramsey problem can be written as follows: max fc t;k t+1 g T 1 s:t: TX t 1 u (c t ) (2.1) t=1 k t+1 = (1 ) k t + i t (2.2) c t + i t y t = F (k t ; n t ) ; t = 1; 2; ::T (2.3) c t ; k t+1 > 0; n t 2 (0; 1) ; k 0 given (2.4) where, is the depreciation rate for the capital. The utility function u is assumed to be strictly increasing, and F (k t ; n t ) will take the characteristics of the neoclassical production function: i) F (0; 0) = 0 (no free lunch), ii) concave (homogeneous of degree one: no increasing returns to scale) iii) twice continuously di erentiable and iv) strictly increasing in both of its arguments. i.e.f : R 2 +! R + with F (0; 0) = 0, F k (k; n) > 0; F n (k; n) > 0; F kk (k; n) < 0; F nn (k; n) < 0 8 k; n > 0;and the Inada conditions of limf k (k; 1) = 1; lim F k (k; 1) = 0: k!0 k!1 A few things to note here. By the strict monotonicity of u an optimal plan will satisfy the feasibility constraint c t + i t F (k t ; n t ) with equality. Moreover, since the agent does not value leisure (and labor is productive), an optimal path will be such that n t = 1 for any t. Using the law of motion for k t, one can rewrite the above problem dropping the leisure and investment variables as follows: max fc t;k t+1 g T 1 s:t: TX t 1 u (c t ) t=1 k t+1 + c t f (k t ) ; t = 1; 2; ::T (2.5) c t ; k t+1 > 0; k 0 given where, f (k t ) = F (k t ; 1) + (1 ) k t : Note that if we compare eqs. (1.2) and (2.5), these two problems seem quite similar. e.g.if f (k t ) = k t : Note that we could further simplify the Ramsey maximization problem above 3

4 (expressed only as a function of k alone) as max fc t;k t+1 g T 1 s:t: TX t 1 u (f (k t ) k t+1 ) (2.6) t=1 k t+1 > 0; k 0 given How should one solve this problem? There are two ways to solve this: sequential or recursive. We ll brie y talk about the sequential approach (The Kuhn-Tucker or the Lagrange method) and then concentrate on the Recurive Method (Dynamic Programming)! 3. Sequential Approach: The Kuhn-Tucker Method The rst order conditions for the stated problem above with t being the multiplier on (2.5) and t being the multiplier on the non-negative constraint on k t+1 : To be more precise, we also need a multiplier, say t ; for the non-negative constraint on c t, but we know that this constraint is not going to be 1 if c! 0 for all t = 1; 2; ::T: i.e.c t > 0 8t = 1; 2; ::T; t = 0 (think of this condition as an agent "hates starting to death"). 0 = t (c t t t ; t = 1; 2; ::T (3.1) 0 = t + t+1 f 0 (k t+1 ) + t+1 ; t = 1; 2; ::T 1 (3.2) 0 = T + T +1 (3.3) 0 = t (f (k t ) k t+1 c t ) ; t = 1; 2; ::T (3.4) 0 = t k t+1 ; t = 1; 2; ::T (3.5) Things to note. we t = t ; so the resource constraint is always binding: k t+1 +c t = f (k t ) : We assumed that f (0) = 0; so with c t > 0; we also need k t > 0 through period T. However, the agent will consume his entire capital stock (the amount of cake) in the last period of his life as any capital stock (cake) left reduces his lifetime untility (k T +1 = 0). That is, with equations (3.3) and (3.5), we have T k T +1 = 0: For an in nite horizon problem, the analogous to the terminal condition of T k T +1 = 0 is called the transversality condition: lim t t k t+1 = 0: t!1 i.e. the present value of the terminal capital stock must approach zero. 4

5 Now combining all f.o.c., we can characterize the solution (c t ) (c t+1) f 0 (k t+1 ) for any t+1 (c t) (c t+1) = f 0 (k t+1 ) t+1 k t+1 + c t = f (k t ) Equation (3.6) is known as the Euler equation: This is a necessary condition of optimality for any t: if it is violated, then the agent can do better by adjusting c t and c t+1. The lhs of (3.6) is the marginal rate of subsititution between consumption if two adjacent periods. The rhs (marginal rate of transformation) provides the compensation for an additional unit of savings, which increases the amount of goods available for t + 1. Essentially, to solve the optimal fc t g T 1 ; you would need to solve for T equations and T unknowns. If either T or the number of control variables become large, then the computational aspects becomes a bit cumbersom if not tedious. So, we now turn to the alternative method, which is called the Recurive method or the Solution of Functional Equation (Bellman equation). Exercise 1: Let u be given by a constant elasticity of substitution function u (c 0 ; c T ) = ( TX t=1 t 1 c t ) 1 ; 2 ( 1; 1]; and let f (k t ) = k t ; 2 (0; 1): Use the rst order conditions in (3.6) to show that the solution to the problem boils down to solving T non-linear equations in the T unknowns of (k 1; k T ): 4. Dynamic Programming Approach: Recursive Method The essence of dynamic programming approach is that the T (or in nite horizon) period sequential problem can be converted into a two-period problem with some appropriate rewriting of the objective function. That is, unlike in the sequential problem of solving many T (or in nite) equations with T unknowns, in dynamic programming, we are reducing large T (or in nite) dimensional problem into a sequence of one-dimensional problems! In the neoclassical growth model example, the only choice variable would be k t+1! 5

6 4.1. Finite T Horizon Set Up How do you set up the problem in (2.6) recursevely? We know that at T + 1, the optimal saving decision is to set to zero. i.e. k t+1 = 0: So, at T period problem, one solves V 1 (k T ) max k T +1 [u (f (k T ) k T +1 )] (4.1) = u (f (k T )) (4.2) where, V 1 (k T ) refers to the value of choosing consumption and capital optimally and the superscript denotes the number of periods remaining in the planning problem. Obviously, since everyone is dead at T + 1, the best the planner can do at time T is to let the V 1 (k T ) = u (f (k T )) :i:e:kt +1 = 0 solves the problem in (4.1). Note that we call V 1 (k T ) the value function and k T is known as the state variable. The planner s problem at T 1 is as follows: V 2 (k T 1 ) max u (f (kt 1 ) k T ) + V 1 (k T ) (4.3) k 0 T 1 B = (k T 1 ) kt {z} =G 2 (k T 1 ) C A + V 1 (k T ) where, kt = G2 (k T 1 ) solves problem (4.3). Here, the function G 2 is called the decision rule or policy function. And note that here, kt solves the rst order condition 0 = u 1 (f (k T 1 ) kt ) + V1 1 (kt ) Think of the problem in (4.3) as follows. Given some inherited stock of capital k T 1, the planner has to decide how much to let individuals consume and how much capital to take into period T. On the one hand, the planner knows that if it saves k T, then the representative individual will enjoy utility u(f(k T )) tomorrow. In other words, tomorrow s value of saving k T today is given by V 1 (k T ),which individuals discount by factor. The planner also knows that whatever is not saved will be consumed today, which provides the representative individual with utility u (f (k T 1 ) k T ) : We can keep going on with this logic. And consequently, 6

7 the period t problem is written as follows: V T +1 t (k t ) max u (f (kt ) k t+1 ) + V T t (k t+1 ) (4.4) k t B = (k t ) kt+1 {z} =G T +1 t (k t) C A + V T t (k t+1 ) where, kt+1 = G T +1 t (k t ) solves problem (4.4). So as you can see from above that dynamic programming has e ectively collapsed a single large problem involving T + 1 t choice variables into T + 1 t smaller problems, each involving one choice variable! Another way of looking at dynamic programming approach is to know that the state of the economy (in this set up) is 100% chracterized by the capital stock at t, k t ; and the exact time when the decision takes place doesn t really matter (if T is very large...): that is, the value function, in the neoclassical growth model, at t only depends on one variable, capital at the beginning of the period. And since the planner chooses k t+1 to optimize the value function, we can get rid of time subscripts altogether and denote the capital stock today by k and the capital stock tomorrow by k 0. This T + 1 t period problem can be characterized by today and tomorrow variables. The policy function k 0.= G (k) determines tomorrow s optimal capital level, k 0 ; for any possible today s capital level, k In nite Horizon Problem As T! 1 one might expect that V T +1 t (k t )! V (k t ) and G T +1 t (k t )! G(k t ): This is true but it takes some e ort to show it. We ll show this later as we progress with some mathematics. But for now, knowing that the economic environment is "stationary",. the problem for the in nite horizon will take the form V (k t ) max [u (f (k t ) k t+1 k t+1 ) + V (k t+1 )] (4.5) or V (k) max [u (f (k) k 0 ) + V (k 0 )] k 0 1 = (k) {z} k 0 =G(k) A + V (k 0 ) 7

8 The equation in (4.5) is known as Bellman Equation (or functional equation). The rst thing to notice here is that this is a much easier maximization problem than maximizing over infnite sequences: we only need to choose k 0. However, we can t really solve this maximization problem as we don t know what this V (k) function looks like. Plus, V (k) is in the maximization problem as well. So, how would you go about solving this problem in (4.5)? We will answer this question more formally later in this course, but for now, we need to assume something about V (k) and G (k) ; and then show you two "quick and dirty" ways to solve this problem: 1) Guess and Verify (due to the Contraction Mapping) and 2) The Envelop Theorem (sometimes it s called Benveniste and Scheinkman equation). But for now, suppose that the function V (k) was di erentiable and that the maximization value of k 0 = G (k) was interior, the the rst order and envelop conditions for (4.5) are respectively Guess and Verify u 0 (f (k) G (k)) = V 0 (G (k)) ; (4.6) V 0 (k) = f 0 (k) u 0 (f (k) G (k)) (4.7) For this method, we make an informed guess about the functional form of the value function and verify whether that guess is correct (this result is due to the contraction mapping theorem which we ll see later). If the value function is readly available, this method can save a lot computation. For certain classes of problems, when the period utility function u (f (k t ) k t+1 ) lies in the HARA (hyperbolic absolute risk aversion (or linear risk tolerance utility function)) class, which includes CRRA, CARA, and quadratic functions, the value function takes the same general functional form.thus, for example, if the period utility function is logarithmic, we can expect the value function will also be logarithmic. The important properties that one needs to have in mind is that both u (c t ) and f (k t ) have to be concave. But, with our assumptions on u (c t ) and f (k t ) ; we ll see that V (k) will also be concave. Once again, we ll get into some mathematics about these properties later. Now, suppose that u (c t ) = ln c t and f (k t ) = kt ; : 2 (0; 1) Then we can solve for the analytical form of G (k) in (4.5). Here is how: write the problem in (4.5) as V (k) max [ln (k k 0 ) + V (k 0 )] (4.8) k 0 8

9 Now, make an "educated" guess of the form of V (k) : We know since u (c t ) = ln c t, the value function V (k) will also take the logrithm form V (k) A + B ln k where A and B are constants (these must be determined). The rst order condition for k 0 is 1 k k 0 = B k 0 ) k 0 = Bk 1 + B Now, substituting above.k 0 into (4.8), we have A + B ln k = ln k Bk B A + B ln Bk 1 + B (4.9) : (4.10) Collecting terms on the rhs of (4.10) and matching them to the lhs of (4.10), we have two unknowns (A and B) and two equations to solve: A = A + B ln (B) (1 + B) ln (1 + B) B = + B: The solution to this system can be easily obtained: A = 1 ln (1 ) + ln () 1 1 B = 1 : Now, we can say concretly about our policy function G (k) = k 0 : Using equation (4.9) and B = 1 ; G (k) = k 0 = k : (4.11) Exercise 2: Take the example above (i.e. u (c t ) = ln c t and f (k t ) = k t ; 2 (0; 1)). Using the iteration method on the value function in (4.8), show that as t! 1; we obtain the policy function G (k) in (4.11). That is, rst let V 0 = 0, 9

10 and solve for k0 that would give you the maximum V 1 : Using the same procedure, you should be able to obtain P t 1 k 0 i=1 = P ()t t 1 k : i=0 ()t With the policy function in (4.11), we can also solve for the consumption decision rule from the resource constraint as G c (k) = c = k k = k (1 ) (4.12) That means that in this model, under our assumptions, consumption and investment are constant fractions of output. The fact that the decision rules do not depend on the capital stock is very unique to this particular example. The decision rule (4.11) is a law of motion for capital along the optimal path, as shown in Figure 4.1. The gure illustrates an important property of the neoclassical growth model: the stock of capital converges to its unique steady state whether we start from a low level of capital (such as k 0 on the Figure) or a high level of capital (such as e k 0 on the Figure). The steady state is de ned as a state that no longer changes, i.e. if you start at that level of capital, you will stay at that level of capital forever. That means that k t+1 = k t = k : We can use the policy rule (4.11) to solve for the steady state. In steady state, k = 1 1.It follows that steady state consumption is c = (1 ) k = (1 ) 1 and output is.y = k = 1 : Exercise 3: Habit Persistence Consider the problem of choosing a consumption sequence c t to maximize TX t 1 (ln c t + ln c t 1 ) ; 0 < < 1; > 0 t=1 s:t: k t+1 + c t k t ; 0 < < 1 c t ; k t+1 > 0; k 0 ; c 1 given where the current utility function ln c t + ln c t 1 is designed to represent habit persistence in consumption. Let V (k 0 ; c 1 ) be the value of the in nite sum of discounted utility function for a consumer who begins time 0 with captial stock k 0 and lagged consumption c 1. A) Formulate Bellman s functional equation in V (k; c 1 ) : B) Show that the optimal policy is of the form ln k t+1 = A + B ln k t : express A and B explicitly. 10

11 Figure 4.1: Convergence of capital stock to its Steady State ( xed point of G(k)) Envelop Theorem: Benveniste and Scheinkman (B-S) Under some conditions, Benveniste and Scheinkman (1979) established that the value function is di erentiable in dynamic programming problems. When the value function is di erentiable and concave, we can use rst order conditions to characterize the solution to the planner s problem. The advantage of using this method is that one is not required to know the form of the value function to get some results. From Sargent (1987) p. 21, the modi ed B-S formula (f (k) k0 which is essentially the same as the Envelop Theorem in (4.7). Here are the steps in using the B-S formula: (4.13) 1. Set up the Bellman equation with the value function in general form. V (k) max [u (f (k) k 0 ) + V (k 0 )] k 0 11

12 2. Perform the maximization problem on the r.h.s. of the Bellman equation. That is, derive the F.O.C. s. u 0 (f (k) k 0 ) = V 0 (k 0 ) ; 3. Use the B-S Formula to substitute out any terms in the value function. V 0 (k) = f 0 (k) u 0 (f (k) k 0 ) : That is, update the above equation by one period, V 0 (k 0 ) = f 0 (k 0 ) u 0 (f (k 0 ) k 00 ) : where is k 00 capital two periods from now. We can now substitute the value of V 0 (k 0 ) into our rrst order condition above to obtain u 0 (f (k) k 0 ) = f 0 (k 0 ) u 0 (f (k 0 ) k 00 ) (4.14) As a result, you will get an Euler equation, which may be useful for certain purposes, although it is essentially a second-order di erence equation in the state variables. The interpretation of this Euler equation (4.14) is straightforward: if the planner were to save one more unit of consumption today, it could increase production tomorrow by f 0 (k 0 ), which is valued at u 0 (f (k 0 ) k 00 ) and discounted at. The cost of doing so is the lost utility today, valued at u 0 (f (k) k 0 ). At the optimum, the marginal cost equals the marginal bene t.. We can use Euler equation (4.14) to characterize the steady state. Since the steady is de ned as a state where variables are constant, we can set k 00 = k 0 = k = k in (4.14) and get f 0 (k ) = 1 ) f 0 (k ) = 1 : or 1 + F 0 (k ; 1) = 1 i.e. one plus the marginal product of capital net of depreciation is equal to the inverse of the discount factor. We will see shortly that the interest rate in the neoclassical growth model is equal to the marginal product of capital net of depreciation, so that the gross interest rate is equal to the inverse of discount rate (1 + r = 1 ). In any event, the steady state stock of capital only depends on the discount factor and the depreciation rate. Since capital is constant in steady state, investment must equal the depreciation (i = k ), so consumption is equal to production net of depreciation (c = F (k ; 1) k). 12

13 5. Competitive Equilibrium Growth This section is from Martin Gervais (2006) lecture notes on "The Neoclassical Growth Model and Dynamic Programming". Here, I will simply assert that the there is a unique Pareto optimal allocation that is also the competitive equilibrium allocation in this model. While the most straight forward way to determine competitive equilibrium quantities in this dynamic model is to solve the planner s problem, to determine equilibrium prices we need some information from the solutions to the consumer s and rm s optimization problems. For the discussion of Pareto optimal allocations it did not matter who owned what in the economy, since the planner was allowed to freely redistribute endowments across agents. For a competitive equilibrium we need to take a stand on the ownership structure. We will assume that consumers own all factors of production (i.e. they own the capital stock at all times) and rent it out to rms. We also assume that households own the rms, i.e. are claimants of the rms pro ts. Finally, we will assume that all markets ( nal goods, capital and labour) are perfectly competitive, which means that households and rms take prices as given and beyond their control Consumer s Problem Individuals face a fully dynamic problem in this economy. They own the capital stock and hence have to decide how much labour services to supply at wage w t, how much capital services to supply at rental rate r t, how much to consume, and how much capital to accumulate for the future. However, since individuals do not value leisure, we can set their labour supply to 1. Given prices fw t ; r t g 1 t=0, individuals face the following (sequential markets) problem: max fc t;k t+1 g 1 t=0 s:t: 1X t u (c t ) (5.1) t=0 c t + k t+1 = w t + r t k t + (1 ) k t (5.2) c t ; k t+1 > 0; k 0 given Ignoring the nonnegativity constraints on capital (in equilibrium, prices will be such that the consumer will always choose k t+1 > 0), the rst-order conditions for 13

14 an optimum are u 0 (c t ) = [r t+1 + (1 )] u 0 (c t+1 ) u 0 (c t ) u 0 (c t+1 ) ) = [1 + (r t+1 )] (5.3) This is the standard intertemporal optimality condition: the inverse of the intertemporal marginal rate of substitution is equal to one plus the net rate of return on capital Firm s Problem Since we have assumed that the technology is constant returns to scale, we can assume without loss of generality that there is only one rm. Also, since rms do not own any assets, their problem is a static one: they only choose the inputs to use given prices. In other words the rm simply maximizes pro ts on a period by period basis, i.e. given (w t ; r t ), it solves max ff (n t ; k t ) n t;k t w t n t + r t k t g The rst order conditions simply state that factors are paid their marginal products: w t = F nt (n t ; k t ) (5.4) r t = F kt (n t ; k t ) (5.5) You should convince yourself that at these prices, rms make zero pro ts in equilibrium De nition of Competitive Equilibrium A competitive equilibrium is an allocation fc t ; n t ; k t g 1 t=0 and prices fw t; r t g 1 t=0 such that 1. Given prices fw t ; r t g 1 t=0, the allocation fc t; n t ; k t g 1 t=0 solves the consumers problem of maximizing (5.1) subject to the constraints (5.2); 14

15 2. Factors are paid their marginal products, i.e. w t and r t are given by (5.4) and (5.5) respectively; 3. Markets clear: c t + k t+1 = F (n t ; k t ), t = 0; 1; 5.4. Competitive Equilibrium Prices First notice that if we combine the optimality condition (5.5) from the rm s problem to that of the consumer s problem (5.3), we have u 0 (c t ) u 0 (c t+1 ) = [1 + (F kt (1; k t ) )] = f 0 (k t ) ; which is identical to the Euler equation we derived from the Planner s problem (equation (4.14)). In general, once we have solved for the Pareto e cient allocation, we can get prices to support that allocation as a competitive equilibrium simply by setting w t and r t according to (5.4) and (5.5). A nal note on competitive equilibria: when the consumer solves his/her optimization problem, he/she knows the whole sequence of pricesfw t ; r t g 1 t=0. That is, this is a rational expectations or perfect foresight equilibrium where each period the consumer makes forecasts of future prices and optimizes based on those forecasts, and in equilibrium the forecasts are correct. In an economy with uncertainty, a rational expectations equilibrium has the property that consumers and rms may make errors,but those errors are not systematic. 6. What to Expect in the Following Lectures? In the subsequent lectures, we will be preparing ourselves (more formally) to answer the following questions: Given our functional equation (Bellman equation) V (k) max [u (f (k) k 0 ) + V (k 0 )] k 0 Will V exist? Is V unique? Is V continuous? Is V continuously di erentiable? Is V increasing in k? Is V concave in k? Is the policy G increasing in k? Is G continuously di erentiable? 15

16 References [1] Adda J., and Cooper, (2003) Dynamic Economics: Quantitative Methods and Applications, MIT Press. [2] Bellman, R. (1957), Dynamic Programming. Princeton, NJ: Princeton University Press. [3] Cass, D. (1965) Optimum Economic Growth in an Aggregative Model of Capital Accumulation, Review of Economic Studies: [4] Cooley, T. F., and E. C. Prescott (1995) Economic Growth and Business Cycles, in T. F. Cooley ed., Frontiers of Business Cycle Research, Princeton University Press: [5] Gale, D,. (1967), A Geometric Duality Theorem with Economic Applications, Review of Economic Studies, 34, [6] Sargent, T. (1987), Dynamic Macroeconomic Theory, Harward University Press. [7] Sargent, T. J. and L. Ljungqvist (2000), Recursive Macroeconomic Theory, MIT Press. [8] Stokey, N., R. Lucas and E. Prescott, (1991), Recursive Methods for Economic Dynamics,Harvard University Press. 16