Dynamic optimization: a recursive approach. 1 A recursive (dynamic programming) approach to solving multi-period optimization problems:

Transcription

1 E 600 F 206 H # Dynamic optimization: a recursive approach A recursive (dynamic programming) approach to solving multi-period optimization problems: An example A T + period lived agent s value of life is given by T β t ln c t, β <, () where c t is the amount of consumption of a single homogenous good. This good is produced through a technology y t = k α t, α <, where y t is the amount of good produced on date t by using capital k t. The individual is born on date 0 with a capital stock k 0. Capital stock depreciates completely after use. Any capital for future dates can be accumulated by saving (a part of) current output. One unit of output invested in capital on date t yields a unit of capital on date t+. Thus, the individual s budget constraint is c t + k t+ k α t, t = 0,,..., T (2) The individual s problem is to imize () subject to the sequence of constraints given by (2). The solution { c t, kt+} T can be obtained by writing down the Lagrangian of the above problem and applying Kuhn-Tucker conditions. Another approach is to solve the problem backwards. This is the approach we follow below. We will assume that (2) always binds with equality.

2 Date T problem (here k T is already given) k T + ln kt α k }{{ T + }. c T The obvious solution is k T + = 0; c T = k α T (3) Therefore, the imized value of utility is ln kt α. Let us call this as the agent s value of remaining lifetime. Denote this by V T : R + R. Thus, V T (k T ) = α ln k T (4) Let us proceed backwards to T. Date T problem: Here k T is given and the agent s problem is to choose c T and leave some capital for date T, i.e., k T. The agent knows that leaving k T gets V T (k T ) for her remaining lifetime from T onwards. Thus, her imization problem can be written as k T ln kt α k T + βv T (k T ). (5) c T Use (4) in the above and take first-order conditions to obtain k T = αβ + αβ kα T ; c T = + αβ kα T (6) These optimal choices imply a value V T (k T ) for the agent on date T. This is simply obtained by substituting (6) in (5): V T (k T ) = ( + αβ) ln ( + αβ) + αβ ln β + α ( + αβ) ln k T Ω T (7) Notice that Ω T is a function of parameters. 2

3 By now the approach is clear; use V T to formulate date T 2 problem: k T ln kt α 2 k T + βω T + αβ ( + αβ) ln k T, (8) c T 2 the solution to which is given by (notice that the term βω T is irrelevant for the optimal choices) + αβ k T = αβ + αβ + (αβ) 2 kα T 2; c T 2 = Substituting (9) in (8) obtains + αβ + (αβ) 2 kα T 2 (9) V T 2 (k T 2 ) = Ω T 2 + α ( + αβ + (αβ) 2) ln k T 2. (0) Once again, Ω T 2 is a function of parameters α and β, but it is different from Ω T. Continuing in this manner we can verify that on any date t < T 2 + αβ (αβ) T t k t+ = αβ + αβ + (αβ) (αβ) T t kα t ; c t = which can be more compactly written as t (αβ)t k t+ = αβ (αβ) T t+ kα t ; c t = Similarly the value function on date t can be obtained as t+ (αβ)t V t (k t ) = Ω t + αβ ln k t αβ + αβ + (αβ) (αβ) T t kα t αβ (αβ) T t+ kα t () where Ω t is once again a function of parameters. We assign a subscript t to signify that this function varies over time. Now that we have obtained a general solution to the problem of imizing () subject to (2), let us have a summary of this solution approach: 3

4 In each period t, the dynamic programming approach summarizes all future period s utility by the value function V t+ (k t+ ) and reduces the problem to a one period problem for period t, as follows {ln c t + βv t+ (k t+ ) : c t + k t+ kt α } {c t,k t+ } The solutions to the above problems are policy functions c t = g ct (k t ) and k t = g kt (k t ) (See equations (3), (6), (9), and ()). These functions are the rules by which optimal c t and k t+ are chosen once the state variable k t is given. (Richard Bellman s) Principle of Optimality: If the choices { c t, kt+} T are optimal for the lifetime (given k 0 ), then for any given ks, the choices { c t, kt+} T must be optimal for the remaining lifetime t=s starting at t = s. The principle of optimality requires that the optimal choices solve the dynamic programming problem; i.e., at each t, c t and k t+ are given by the corresponding policy functions g ct and g kt. Another way to state the principle of optimality is that, for all t = 0, 2,...T, the choices c t and k t+ solve: V t (k t ) = {c t,k t+ } {ln c t + βv t+ (k t+ ) : c t + k t+ k α t } This equation is the Bellman equation for the current problem (that is, to imize () subject to (2)). 2 What if T When T =, there is no last period to start the recursive procedure. But it is useful to view an economy with T = as the limit of T. Take the limit T in (). The policy functions approach c t g c (k t ) = ( αβ) k α t, k t+ g k (k t ) = αβk α t. The policy functions depend on time only through the state variable k t ; i.e., they are time invariant (notice that the time subscripts on g c and 4

5 g k disappear). For this reason, we typically suppress the time subscript t and denote one-period future variables by adding a superscript. For example, k denotes k t+. Then the above policy functions are c g c (k) = ( αβ) k α, k g k (k) = αβk α. The value function is also time invariant. It can be shown that as T, Ω t Ω = [ ln ( αβ) + αβ ] ln αβ β αβ The Bellman equation becomes: V (k) = c,k {ln c + βv (k ) : c + k k α } 5