Dynamic Programming. Macro 3: Lecture 2. Mark Huggett 2. 2 Georgetown. September, 2016

Transcription

1 Macro 3: Lecture 2 Mark Huggett 2 2 Georgetown September, 2016

2 Three Maps: Review of Lecture 1 X = R 1 + and X grid = {x 1,..., x n } X where x i+1 > x i 1. T (v)(x) = max u(f (x) y) + βv(y) s.t. y Γ 1 (x) Γ 1 (x) = {y X : 0 y F (x)} 2. T (v)(x) = max u(f (x) y) + βv(y) s.t. y Γ2 (x) Γ 2 (x) = {y X grid : 0 y F (x)} 3. T next page Map 1: T is standard DP map Map 2: T is finite DP map Map 3: T is piecewise-linear DP map While Map 1 cannot quite be executed on a computer, both Map 2 and Map 3 can be at least approximately.

3 X grid = {x 1,..., x n } X α(x) (x x i )/(x i+1 x i ) for x i x x i+1 { T max u(f (x) y) + βv(y) s.t. y Γ1 (x) : x X (v)(x) = grid (1 α(x))t (v)(x i ) + α(x)t (v)(x i+1 ) : x X grid

4 One could prove the following: T maps B(X, R) into itself and is a contraction map. T maps B(Xgrid, R) into itself and is a contraction map. T maps P L(X, R) into itself and is a contraction map. Thus, one can apply the Contraction Mapping Theorem to get the existence of a unique fixed point for each map. Even though one might view both ( T, T ) as simply computational tools, they both can be analyzed at a theoretical level. At a minimum, this is potentially useful for discovering errors in ones code.

5 Homework 1-2: Message Cake Eating Problem: All methods (Map 2-3) poorly approximate the theoretical value function at low asset levels. Optimal Growth Problem: All methods (Map 2-3) seem to work well for a range of values of the state variable. Approximate convergence of all algorithms is rapid as β << 1. True optimal decision rule is well approximated (at least at grid points) by the computed decision rules in Maps 2-3. Map 3 produces a monotone decision rule. Could one prove that this must be true regardless of the precise choice of grid? If so, useful check on error(s) in your code.

6 Construct a New Map Based on Euler Eqn Note two things about the original problem: 1. one could prove that T maps increasing, concave and differentiable functions into the same function space. 2. could carry out the T map by using first-order conditions v j+1 (x) = T (v j )(x) = max u(f (x) y) + βv j (y) s.t. y Γ 1 (x) Sufficient condition for y j+1 (x) to solve RHS of T map: u (F (x) y j+1 (x)) βv j(y j+1 (x)) = when y j+1 (x) > 0 Useful Recursion: v j+1 (x) = u (F (x) y j+1 (x))f (x)

7 Algorithm Based on Euler Eqn 1. Guess v 1 (x) decreasing for x X grid. 2. Given v j, solve for y j+1(x) = y for x X grid. Use ( ) or ( ) ( ) u (F (x) y) βv j(y) and = if y > 0 ( ) G(x, y) u (F (x) y) βv j(y) 0 and = if y > 0 3. Set v j+1 (x) = u (F (x) y j+1 (x))f (x) for x X grid. 4. Repeat 2-3 until approx convergence. Get y(x) for x X grid. 5. Approximate v(x), where y is the computed decision rule, by interating on v j+1 (x) = u(f (x) y(x)) + βv j (y(x)).

8 Algorithm Based on Euler Eqn Comments 1. Algorithm uses Euler eqn to do maximization. Thus, CMT applies to this algorithm. Fixed point is w/in class of differentiable functions v instead of piecewise linear v. 2. To solve the non-linear eqn need to interpolate v j because we only calculate it at x X grid. Use linear interpolation... or a continuous interpolant consistent with v j decreasing. 3. Useful to graph G to see what solving the non-linear eqn involves... and see downside to some interpolants. 4. Instead of solving for choice given state, could pick y and solve for the state x. Method of endogenous grid points

9 Endogenous Grid Method 1. Specify Y grid = {y 1,..., y n }, where y i < y i+1, and specify arbitrary {v 1 (y i)} decreasing. 2. Solve for state x i, given choice y i Y grid using u (F (x i ) y i ) βv 1 (y i) and = if y i > 0. Get {(x i, y i )} n i=1. 3. Set v 2 (x i) = u (F (x i ) y i )F (x i ) for x i X grid, which is the ENDOGENOUS grid constructed in step Repeat 2-3 until approximate convergence. Interpolate at each iteration to get v 2 on Y grid rather than X grid. 5. Result: Set {(x i, y(x i ))} n i=1 = {(x i, y i )} n i=1 based on last iteration.

10 Endogenous Grid Method: Comments 1. Roughly put, the algorithm solves for x given y using F (x) y = (u ) 1 [βv j (y)]. 2. This is much faster than the previous method. Instead of solving a nonlinear equation by several trial calculations or by your favorite non-linear eqn solver, you solve the equation above by one function evaluation. 3. The method is the Euler equation method but with a faster implementation of solving the Euler equation. 4. To keep the Y grid fixed across iterations on the previous slide you need to interpolate v 2 to figure out values on Y grid. 5. Method can be extended to handle shocks and several control variables for some cases.

11 How to Approximate a Markov process Issue: Choose finite Markov chain to approx a VAR or an AR-1 process Motivation: VAR is an empirical summary of a time series process. Use a VAR (or AR-1) as an input to an economic model as a driving process. Specific Paper: Tauchen (1986) - simple method 1. EX 1: x F ( ) and E[x t ] = 0 iid process on R 1 2. EX 2: x t+1 = ρx t + ɛ t+1 and E[ɛ t+1 ] = 0 AR-1

12 Tauchen: EX 1 iid shocks 1. Set {x 1,..., x n } 1. (increasing) x i+1 > x i, i 2. (synmmetry) x 1 = x n 3. (equispaced) x i+1 x i = w > Set {p 1,..., p n } 1. p 1 = F ((x 1 + x 2 )/2) 2. p i = F ((x i + x i+1 )/2) F ((x i 1 + x i )/2) 3. p n = 1 F ((x n 1 + x n )/2)

13 Tauchen Method: Let F be defined on an interval [a, b]. Then the distribution function F n produced by an n-point Tauchen procedure on [a, b] converges weakly to F. Proof by picture. Comments: 1. Weak convergence is equivalent to lim n gdfn = gdf for all g continuous. Thus, in the conext of DP, the discretized process will in the limit get the expectation term in Bellman s equation right for continuous objectives. 2. While Tauchen is simple(r), quadrature methods will have smaller integration errors.. In some applications, may want discretized process to integrate well AND cover a wide range of shock values. Quadrature does former but not the latter.

14 EX 2: x t+1 = ρx t + ɛ t+1 and E[ɛ t+1 ] = 0, ɛ F 1. Set {x 1,..., x n } V ar(ɛ 2 ) = σ 2 ɛ and σ 2 σ 2 ɛ /(1 ρ) 2 1. (symmetry) x 1 = x n = 3σ 2. (equi spaced) x i+1 x i = w > Set p i,j 1. p i,1 = F (x 1 + w/2 ρx i ) 2. p i,j = F ((x i + x i+1 )/2) F ((x i 1 + x i )/2) for 1 < j < n 3. p i,n = 1 F (x n w/2 ρx i ) NOTE: p i,1 = P r{x t+1 = ρx t + ɛ t+1 x 1 + w/2 x t = x i } p i,1 = P r{ɛ : ɛ x 1 + w/2 ρx i } = F (x 1 + w/2 ρx i )

15 Numerical Quadrature Issue: Given (f, w), compute ( ) or ( ) by choosing N pairs (x k, w k ): f(x)dx = N k=1 f(x k)w k ( ) f(x)w(x)dx = N k=1 f(x k)w k ( ) f - function to be integrated w - weighing function (density) (x k, w k ) - quadrature points and weights Newton Quadrature: Given {x k }, choose {w k } in a smart way. Gaussian Quadrature: Choose both {x k } and {w k } in a smart way.

16 Example: Trapezoidal Rule [x 0,x n] f(x)dx h 2 [f(x 0) + 2f(x 1 ) + + 2f(x n 1 ) + f(x n )] h x k x k 1, k Graph it to be clear. The Trapezoidal Rule is like Newton quadrature since weights are chosen but not the quadrature points.

17 Claim: N point Guassian quadrature rule integrates polynomials of order 2N 1 exactly and weights satisfy w k > 0 and k w k = 1. Points and weights satisfy: (i) 1w(x)dx = k 1w k (ii) (iii) xw(x)dx = k x 2 w(x)dx = k x k w k x 2 k w k Idea: Know LHS of (i)-(iii) for specific w functions. Choose RHS variables to solve system of equations.

18 In practice, for many choices of densities w( ), the N quadrature points and weights have already been solved for you. You just need to look them up in textbook quadrature tables. Could choose low N value as N = 6 implies that you would integrate exactly all polynomials of degree 11. So if the only point of quadrature in applications is to integrate the expectation term in Bellman s equation accurately and you conjecture that v is smooth, then not many quadrature points are needed.

19 Application: Based on Gauss-Hermite f(x) exp( x2 )dx v(x) = max u(x a ) + βe[v(x )] s.t. 0 a x x = exp(e ) + a (1 + r) and e N(µ, σ 2 ) Apply quadrature to get (z (e µ)/( 2σ)). E[v(x )] =. 1 N v(a(1 + r) + exp( 2σz i + µ)w i π i=1 Some steps behind this are on the next slide.

20 Application: Warm up Step 1: E[f(y)] = 1 2πσ Step 2: x = (y µ) 2σ Step 3: Step 4: E[f(y)] = 1 2πσ E[f(y)] = 1 π (y µ)2 f(y) exp( )dy 2σ 2 and y = 2σx + µ and dy = 2σdx f( 2σx + µ)exp( x 2 ) 2σdx f( 2σx + µ)exp( x 2 )dx E[f(y)]. = 1 π N f( 2σx i + µ)w i i=1

21 E[v(exp(e ) + a (1 + r))] = 1 2π v() exp( (e µ) 2 2σ 2 )de z (e µ) 2σ z N(0, 1), e = 2σz + µ de /dz = 2σ E[v(exp(e ) + a (1 + r))] = v(a (1 + r) + exp( 2σz + µ)) exp( z 2 )dz 1 π E[v(exp(e ) + a (1 + r))] = 1 π N v(a (1 + r) + exp( 2σz i + µ))w i k=1

22 How to Simulate a Markov Process? Context: Suppose you have a computed decision rule a(a t, e t ) solving the income fluctuation problem. Earnings are Markov. Create draws {e t } T t=0 and {a(a t, e t )} T t=0 starting from a known initial state (a 0, e 0 ). Suppose e t takes discrete values in E = {1, 2, 3} with transition probabilities π i,j. 1. Create draws {u t } T t=1 on [0, 1] from a random number generator. 2. Suppose e 0 = i. Set e 1 = 1 if u 1 [0, π i1 ), e 1 = 2 if u 1 [π i1, π i1 + π i2 ), and e 1 = 3 if u 1 [π i1 + π i2, 1). 3. Repeat step 2 for t = 1, 2,.. using the intervals [0, π j1 ), [π j1, π j1 + π j2 ) and [π j1 + π j2, 1) if e t = j. 4. Calculate {a t } T t=0 using {a(a t, e t )} T t=0, based on the computed decision rule and the shock draw from steps 2-3

23 Psuedo Code 1. oldindex=1 2. y(1) = e(oldindex) 3. do it =2, nt u = randomnumber(0,1) index = 1 sum = pi(oldindex, index) do while (u.ge. sum) index = index + 1 sum = sum + pi(oldindex, index) end do y(it)=e(index) oldindex = index 4. end do