Discrete State Space Methods for Dynamic Economies

Transcription

1 Discrete State Space Methods for Dynamic Economies A Brief Introduction Craig Burnside Duke University September 2006 Craig Burnside (Duke University) Discrete State Space Methods September / 42

2 What are Discrete State Space Methods? Discrete state space methods are computational techniques used to obtain the solutions of dynamic economic models If the model in question assumes that the state variables are de ned over nite sets of points, the solutions are exact. Often the model s state-space is continuous, in which case the discrete state-space is an approximation to the continuous state-space Craig Burnside (Duke University) Discrete State Space Methods September / 42

3 Why use Discrete State Space Methods? Discrete state space methods preserve nonlinearities in situations where log-linear approximations may not be accurate, or their accuracy is unknown Properties are backed up by theoretical results on convergence. Intuitive, usually reliable, and applicable to a wide variety of problems One signi cant drawback: the curse of dimensionality. Computational burden (memory and speed) rises quickly with the number of state variables Craig Burnside (Duke University) Discrete State Space Methods September / 42

4 Outline of this Lecture Introduction to Gaussian quadrature A method used in numerical integration Useful here for developing a discrete state-space approximation to a continuous state-space Explain discrete state space methods using two illustrative examples One risky asset version of Lucas (1978) consumption-based asset pricing model Simple stochastic neoclassical growth model Discuss extensions, drawbacks & limitations Craig Burnside (Duke University) Discrete State Space Methods September / 42

5 What is Quadrature? Numerical methods used to approximate integrals by sums are formally referred to as quadrature methods. Euler equations relate functions of variables today to expectations of functions of variables at future dates (tomorrow) e.g. u 0 (c t ) = βe t (1 + r)u 0 (c t+1 ) When the state-space is discrete, this conditional expectation is computed as a sum over the conditional probability distribution. When the state-space is continuous, it involves evaluating an integral over the conditional density function. We use quadrature methods to approximate conditional expectations by sums, when they would be exactly represented by integrals Craig Burnside (Duke University) Discrete State Space Methods September / 42

6 Gaussian Quadrature Introduction Suppose we want to evaluate the integral R ψ(y)f (y)dy Y ψ is the kernel function f is some density function de ned over the set Y. Quadrature approximates the integral using a sum: N Z ψ(y i,n )w i,n ψ(y)f (y)dy, i=1 Y with the points (abscissas), fy i,n g N i=1, and weights, fw i,n g N i=1, being chosen according to some rule. Gaussian quadrature uses a set of rules related to the orthogonal polynomials corresponding to the density function f. Craig Burnside (Duke University) Discrete State Space Methods September / 42

7 Gaussian Quadrature Orthogonal Polynomials Gaussian quadrature with N (the number of points) arbitrarily large requires that all non-negative integer moments of y exist. This assumption guarantees the existence of a set of orthogonal polynomials, fφ N (y)gn =0 for the density f (y) determined according to the following rules φ N (y) = λ N0 + λ N1 y + λ N2 y 2 + λ NN y N, λ NN > 0 (1) Z Y 1 if N = M φ N (y)φ M (y)f (y)dy = 0 otherwise. (2) Easy to see that the 0-polynomial is φ 0 (y) = 1, so (1) and (2) imply that the others are mean zero and have unit variance. I.e. they form an orthonormal basis with respect to f. Craig Burnside (Duke University) Discrete State Space Methods September / 42

8 Gaussian Quadrature Orthogonal Polynomials for the Standard Normal Density For N = 1: so φ 1 (y) = y E (λ 10 + λ 11 y) = 0 =) λ 10 = 0 E (λ 10 + λ 11 y) 2 = 1 =) λ 2 11 = 1 For N = 2: E (λ 20 + λ 21 y + λ 22 y 2 ) = 0 =) λ 20 = λ 22 E (λ 20 + λ 21 y + λ 22 y 2 )y = 0 =) λ 21 = 0 E (λ 20 + λ 21 y + λ 22 y 2 ) 2 = 1 =) 2λ 2 22 = 1 so φ 2 (y) = (y 2 1)/ p 2 Then φ 3 (y) = y 2 3 y/ p φ N (y) = p p 1/Nφ N 1 (y)y (N 1)/NφN 2 (y) Craig Burnside (Duke University) Discrete State Space Methods September / 42

9 Gaussian Quadrature Abscissas and Weights The abscissas for an N-point quadrature rule are the roots of the Nth orthogonal polynomial. The weights for an N-point Gaussian quadrature rule are chosen so that if the kernel is a 2N 1th-or-lower ordered polynomial the quadrature approximation is exact. Although this represents 2N restrictions in N unknowns, only N of the restrictions are unique. For a standard normal example Abscissas Weights 1 f0g f1g 2 f 1, 1g 3 f p 3, 0, p 3g f 1 2, 1 2 g f 1 6, 2 3, 1 6 g Craig Burnside (Duke University) Discrete State Space Methods September / 42

10 Gaussian Quadrature Gaussian quadrature has several nice properties 1 Leads naturally to a discrete state-space interpretation of the approximation Craig Burnside (Duke University) Discrete State Space Methods September / 42

11 Gaussian Quadrature Gaussian quadrature has several nice properties 1 Leads naturally to a discrete state-space interpretation of the approximation 2 An alternative interpretation connects the method to a whole family of methods that approximate the kernel function, ψ, by an alternative function whose integral is easy to compute Craig Burnside (Duke University) Discrete State Space Methods September / 42

12 Gaussian Quadrature Gaussian quadrature has several nice properties 1 Leads naturally to a discrete state-space interpretation of the approximation 2 An alternative interpretation connects the method to a whole family of methods that approximate the kernel function, ψ, by an alternative function whose integral is easy to compute 3 A wealth of convergence results showing that the approximation improves as N gets large. Craig Burnside (Duke University) Discrete State Space Methods September / 42

13 Gaussian Quadrature Gaussian Quadrature Rules Lead to a Natural Discrete State Space Interpretation A discrete state-space interpretation is natural because it can be shown that the weights sum to 1 for any N, and all admissible functions f. Implies that the approximation N i=1 ψ(y i,n )w i,n can be interpreted as E [ψ (y)] when y has a discrete distribution over the set fy i,n g N i=1 with associated probabilities fw i,n g N i=1. Craig Burnside (Duke University) Discrete State Space Methods September / 42

14 Gaussian Quadrature Gaussian Quadrature has a Weighted Residual Method Interpretation Gaussian quadrature is equivalent to replacing ψ with a step function approximation, ψ N, whose integral is computed exactly There exist points fz i,n g N i=0 which satisfy z i 1,N < y i,n < z i,n such that Z zi,n w i,n = f (y)dy z i 1,N implying that N ψ (y i,n ) w i,n = i=1 = N i=1 Z Z zi,n ψ (y i,n ) f (y)dy z i 1,N N Y i=1 ψ (y i,n ) 1 (zi 1,N,z i,n )(y) f (y)dy. {z } ψ N (y ) Craig Burnside (Duke University) Discrete State Space Methods September / 42

15 Gaussian Quadrature Convergence In cases where Y is a compact set [a, b], f (y) is an arbitrary density on Y and ψ (y) is any function for which the Riemann-Stieltjes integral R ψ(y)f (y)dy exists, it follows that Y N lim N! i=1 Z ψ (y i,n ) w i,n = Y ψ (y) f (y) dy. In fact, this convergence is uniform. The CDF corresponding to the discrete distribution, fy i,n g N i=1, fw i,n g N i=1 converges pointwise to the CDF corresponding to f (y). Craig Burnside (Duke University) Discrete State Space Methods September / 42

16 The Lucas Model Setting up the Model Economy has N identical agents with instantaneous utility function U(C t ) = C 1 γ t 1 1 γ, C t is consumption at time t γ is the coe cient of relative risk aversion. Output is obtained from K assets which produce stochastic endowments of a single perishable consumption good. Agent s budget constraint is C t + K k=1 P kt S kt+1 K k=1 (P kt + D kt )S kt. S kt units of asset k at the beginning of time t D kt exogenous stochastic endowment per unit of the asset. P kt price of the kth asset at time t in units of consumption. Craig Burnside (Duke University) Discrete State Space Methods September / 42

17 The Lucas Model Optimization At time 0 the agent maximizes by choosing contingency plans for his budget constraint. E 0 t=0 β t U(C t ) 0 < β < 1. First order conditions for this problem are n o C t, fs kt+1 g K k=1 subject to t=0 P kt U 0 (C t ) = βe t U 0 (C t+1 )(P kt+1 + D kt+1 ), k = 1,..., K. Since the agents are identical if we assume that the total supply of each asset is N, then S kt = 1, 8k, t, for every agent, in equilibrium. Then the budget constraint implies C t = K k=1 D kt for all t. Craig Burnside (Duke University) Discrete State Space Methods September / 42

18 The Lucas Model Equilibrium We have a trivial solution for the equilibrium level of consumption in terms of the exogenous endowments: C t = K k=1 D kt. But we do not know how the prices of the assets, which are endogenous, depend on the values of the endowments. Our goal: characterize the function relating the endowments to the prices of the assets. Craig Burnside (Duke University) Discrete State Space Methods September / 42

19 Solving the Lucas Model The Case of a Single Shock When K = 1, C t = D t and P t U 0 (D t ) = βe t U 0 (D t+1 )(P t+1 + D t+1 ). Using the fact that U(C t ) = C 1 γ /(1 γ) we get P t D γ t t 1 = βe t D γ t+1 (P t+1 + D t+1 ). Express this in terms of the price-dividend ratio V t = P t /D t : or where X t+1 = D t+1 /D t. V t D 1 γ t = βe t D 1 γ t+1 (V t+1 + 1), V t = βe t X 1 γ t+1 (V t+1 + 1), Craig Burnside (Duke University) Discrete State Space Methods September / 42

20 Solving the Lucas Model Single shock is normal and iid (1) Let x t = ln(x t ) be distributed Niid(µ, σ 2 ) and de ne α = 1 γ. Altug and Labadie (1994, p.83): V t = θ if θ = β exp αµ θ α2 σ 2 < 1 This (constant) function of x t is the solution to the functional equation Z V (x t ) = β exp(αx t+1 ) [V (x t+1 ) + 1] f (x t+1 ) dx t+1. Drop the extra N in the notation from now on! Approximate the integral using an N-point Gaussian quadrature rule: V (x t ) = N β exp (αy i ) [V (y i ) + 1] w i (3) i=1 Craig Burnside (Duke University) Discrete State Space Methods September / 42

21 Solving the Lucas Model Single shock is normal and iid (2) Use an N-point Gaussian quadrature rule for f N(µ, σ 2 ): Abscissas Weights Standard normal ȳ i w i Normal(µ, σ 2 ) y i = µ + σȳ i w i = w i Solve (3) for each x t 2 fy i g N i=1 implying N linear equations in N unknowns N V (y j ) = β exp (αy i ) [V (y i ) + 1] w i, j = 1,..., N. i=1 and a trivial solution V = N i=1 β exp (αy i ) w i 1 N i=1 β exp (αy i ) w i. This is the exact solution for the discrete state space model where x t has a discrete distribution with Pr(x t = y i ) = w i. Craig Burnside (Duke University) Discrete State Space Methods September / 42

22 Solving the Lucas Model Single shock is normal, but persistent (1) Assume β i exp i=1 x t = µ(1 ρ) + ρx t 1 + ɛ t, where jρj < 1 and ɛ t Niid(0, σ 2 ). Exact solution in Burnside (1998) given by V t = α µ i if 1 2 α 2 (1 ρ) 2 ρ(1 ρ i ) 1 ρ i 2 ρ(1 ρi ) 1 ρ + ρ(1 ρi ) 1 ρ x t + ρ2 (1 ρ 2i ) 1 ρ 2 β exp αµ + 1 α 2 2 (1 ρ) 2 σ2 < 1. σ 2, + Not a useful solution because it s an in nite series of expressions and does not converge quickly. Craig Burnside (Duke University) Discrete State Space Methods September / 42

23 Solving the Lucas Model Single shock is normal, but persistent (2) Write the functional equation as Z V (x t ) = β exp(αx t+1 ) [V (x t+1 ) + 1] f (x t+1 jx t ) dx t+1. Tauchen and Hussey (1991) suggest the transformation Z V (x t ) = β exp(αx t+1 ) [V (x t+1 ) + 1] f (x t+1jx t ) f (x t+1 jµ) f (x t+1jµ) dx t+1. Base the N-point rule on the function f (x t+1 jµ): V (x t ) N β exp(αy i ) [V (y i ) + 1] f (y i jx t ) i=1 f (y i jµ) w i Solve for x t 2 fy i g N i=1 to get N linear equations in N unknowns: V (y j ) = N β exp(αy i ) [V (y i ) + 1] f (y i jy j ) i=1 f (y i jµ) w i. (4) Craig Burnside (Duke University) Discrete State Space Methods September / 42

24 Solving the Lucas Model The solution for a persistent discrete state space process Let x t be a simple rst-order Markov process, with a discrete state-space fy i g N i=1. Let Pr(x t+1 = y i jx t = y j ) = π ji. Exact representation of the Euler equation for this model is V (y j ) = N β exp(αy i ) [V (y i ) + 1] π ji. i=1 In this equation π ji replaces the term [f (y i jy j ) /f (y i jµ)] w i that appeared in (4) Notice that i π ji = 1 but i [f (y i jy j ) /f (y i jµ)] w i 6= 1. Craig Burnside (Duke University) Discrete State Space Methods September / 42

25 Solving the Lucas Model Single shock is normal, but persistent / Discrete state space interpretation Tauchen and Hussey (1991) suggest solving a modi ed version of (4): where V (y j ) = N β exp(αy i ) [V (y i ) + 1] π ji (5) i=1 π ji = f (y N i jy j ) w i f (y i jy j ) and s j = f (y i jµ) s j i=1 f (y i jµ) w i. By construction i π ji = 1. Also, because lim N! s j = 1 for all j so that the solutions to (4) and (5) converge to the same limit. The solution can be interpreted as the exact solution to a model with a discrete state space. Craig Burnside (Duke University) Discrete State Space Methods September / 42

26 Models with Endogenous State Variables In the asset pricing examples I ve given, the endowments are the only state variables and are exogenous In many macroeconomic models, at least one of the state variables is endogenous. In this case we don t know the law of motion of one of the variables whose state-space we wish to discretize. Craig Burnside (Duke University) Discrete State Space Methods September / 42

27 Models with Endogenous State Variables Step 1: Generate a Discrete State Space for the Exogenous Variables Assume you have an exogenous state variable x t = µ(1 ρ) + ρx t 1 + ɛ t, where jρj < 1 and ɛ t Niid(0, σ 2 ). Get the N point rule for a N(µ, σ 2 ). Approximate the law of motion of x t with a Markov process where and Pr(x t+1 = y i jx t = y j ) = π ji π ji = f (y N i jy j ) w i f (y i jy j ) and s j = f (y i jµ) s j i=1 f (y i jµ) w i. Easily extended to the case where x t is a vector of exogenous variables. Craig Burnside (Duke University) Discrete State Space Methods September / 42

28 Models with Endogenous State Variables Step 2: Set up a State-Space for the Endogenous Variables Here you set up a nite grid of points for the relevant endogenous state variables You cannot specify the law of motion of the endogenous state variables, since it is unknown. The grid of points must be set up without full knowledge regarding the nature of the exact solution. Craig Burnside (Duke University) Discrete State Space Methods September / 42

29 Models with Endogenous State Variables Step 3: Solve for the Optimal Policy Rule for the Endogenous Variables For each value of the endogenous state variable, and each value of the exogenous variables, solve for the optimal value (tomorrow) of the endogenous state variable Method 1: Numerical approaches to dynamic programming. Value function iteration Policy function iteration Method 2: Euler-equation based methods Guess a policy rule and use the Euler equation errors generated by it to re ne the guess iteratively Craig Burnside (Duke University) Discrete State Space Methods September / 42

30 The Stochastic Neoclassical Growth Model Model Setup Representative agent with lifetime expected utility U = E 0 t=0 β t ln(c t ) 0 < β < 1, Resource constraint is C t + K t+1 (1 δ)k t A t K θ t, 0 < δ < 1 Assume that a t = ln(a t ) has the law of motion a t = ρa t 1 + ɛ t, ɛ t Niid(0, σ 2 ) A social planner maximizes U by choosing contingency plans for C t and K t+1 subject to the resource constraint and K 0. Craig Burnside (Duke University) Discrete State Space Methods September / 42

31 The Stochastic Neoclassical Growth Model Optimality Conditions Substitute in the resource constraint and maximize i E 0 β t ln ha t Kt θ + (1 δ)k t K t+1. t=0 The Euler equation for this problem is 1 θa t+1 K θ 1 t+1 + (1 δ) A t Kt θ = βe t + (1 δ)k t K t+1 A t+1 Kt+1 θ + (1 δ)k. t+1 K t+2 We are looking for a solution K t+1 = h(k t, A t ) that solves (6). The nonlinearity prevents solving for h in closed form unless δ = 1 in which case K t+1 = βθa t K θ t. (6) Craig Burnside (Duke University) Discrete State Space Methods September / 42

32 Solving the Growth Model by Value Function Iteration My discussion follows Tauchen (1990) Let r(k, K 0 ; a) = ln he a K θ + (1 δ)k K 0i Start with the Bellman equation for the original model: Z V (K, a) = max r(k, K 0 ; a) + β V (K 0, a 0 )f a 0 ja da 0 K 0 2Γ(K,a)K C Γ(K, a) = n K 0 j0 K 0 e a K θ + (1 The set K C is the continuous state space for K. Under the assumption of normality, K C = [0, ). The next step is to solve an approximating problem. o δ)k Craig Burnside (Duke University) Discrete State Space Methods September / 42

33 Solving an Approximation to the Growth Model Discrete State Space for Technology Approximate the law of motion of a using a discrete state-space process as in the asset pricing model. Let a 2 A = fa i g N i=1 where a i = σȳ i and fȳ i g N i=1 is the set of quadrature points corresponding to an N-point rule for a standard normal. Let where Pr(a 0 = a i ja = a j ) = π ji = f (a i ja j ) f (a i j0) s j = N f (a i ja j ) i=1 f (a i j0) w i and f w i g N i=1 are the quadrature weights for an N-point rule for the standard normal. w i s j Craig Burnside (Duke University) Discrete State Space Methods September / 42

34 Solving an Approximation to the Growth Model Discrete State Space for the Capital Stock We will approximate the state space for capital with the discrete set K D K C with K D = fk m g M m=1. Notice that the maximally sustainable capital stock is now K = (e a N /δ) 1/(1 θ), but there are many ways to set up K D. A number of alternative rules for constructing K D are available. One is to locate a grid of evenly spaced points, K m in the set [0, K ]. Tauchen s (1990) rule: assume linear utility, compute the implied mean and standard deviation of K t implied, and then use an equal spaced grid in a 4σ K band around E (K ). My programs set up a grid in the logarithm of the capital stock based on the E (K ) and σ K implied by the log-linear approximation to the model. Craig Burnside (Duke University) Discrete State Space Methods September / 42

35 Solving an Approximation to the Growth Model Bellman s Equation You can imagine the following two approximations to the original model: Ṽ (K, a j ) = max r(k, K 0 ; a j ) + β K 0 2Γ(K,a j ) ˆV (K, a j ) = max r(k, K 0 ; a j ) + β K 0 2Γ D (K,a j ) N i=1 N i=1 Ṽ (K 0, a i )π ji, (7) ˆV (K 0, a i )π ji, (8) for j = 1,..., N, and with Γ D (K, a) = K D \ Γ(K, a) Since, in both problems, there is an upper bound on K, results in Bertsekas (1976) imply unique bounded solutions to (7) and (8). The solution to (8) can be made arbitrarily close to the solution to (7), as long as in the limit, as M!, with K 1 = 0, K M = K, K m > K m 1, lim M! sup m (K m K m 1 ) = 0. Craig Burnside (Duke University) Discrete State Space Methods September / 42

36 Solving an Approximation to the Growth Model Computation Start with an initial guess at the value function, ˆV 0. Iteratively compute, for m = 1, 2,..., M and j = 1, 2,..., N: ˆV S (K m, a j ) = max r(k m, K 0 ; a j ) + β K 0 2Γ D (K m,a j ) N i=1 Continue iterating until ˆV S (K m, a j ) ˆV S 1 (K m, a j ) < tolerance sup m,j ˆV S 1 (K 0, a i )π ji, Tauchen suggests relating the tolerance to the minimum value of the value function, inf m,j ˆV S (K m, a j ). Craig Burnside (Duke University) Discrete State Space Methods September / 42

37 Solving the Growth Model using the Euler Equation The Euler Equation Baxter, Crucini and Rouwenhorst (1990) use a discrete state-space method which directly approximates the decision rules. De ne g(k, a) = e a K θ + (1 g K (K, a) = θe a K θ δ. δ)k and Looking for a decision rule, h : K C R! K C that satis es the Euler equation: Z 1 g(k, a) h(k, a) = β g K [h(k, a), a 0 ] g [h(k, a), a 0 ] h [h(k, a), a 0 ] f a0 ja da 0. Start by setting up discrete state-spaces for K and a as above. (9) Craig Burnside (Duke University) Discrete State Space Methods September / 42

38 Solving the Growth Model using the Euler Equation The Algorithm Start with a candidate decision rule h S : K D A! K D The next decision rule is obtained by nding the function h S +1 : K D A! K D that best ts the Euler equation. For each m, j, equivalent to nding K 0 2 Γ D (K, a j ) to minimize N 1 g K (K g(k m, a j ) K 0 β 0, a i ) g(k 0, a i ) h S (K 0, a i ) π ji. i=1 The algorithm can be stopped if h S +1 (K, a) = h S (K, a) (often occurs in practice) maximum change in the decision rule eventually satis es some tolerance Little known about convergence properties. Craig Burnside (Duke University) Discrete State Space Methods September / 42

39 Extending a Solution to the Real Line A Step Function In the asset pricing example, the discrete state-space method gave us an approximate solution for any x 2 fy i g N i=1. Denote the solution for x = y i as v i = V (y i ). What do we do if we want a solution for any x 2 Y, the original state space? One solution is to de ne a function V (x) = N v j 1 (zj j=1 1,z j )(x). A disadvantage of this extension to any x 2 Y is that it is not continuous. Craig Burnside (Duke University) Discrete State Space Methods September / 42

40 Extending a Solution to the Real Line An Interpolated Spline Function In the asset pricing example we could de ne 8 >< v 1 if x y 1 V (x) = v j 1 + (x y j 1) (y >: j y j 1 ) (v j v j 1 ) if y j 1 < x y j v N if x > y N. A disadvantage of this extension to any x 2 Y is that it is not di erentiable in x. Craig Burnside (Duke University) Discrete State Space Methods September / 42

41 Extending a Solution to the Real Line The Nystrom Extension Substitute the solution to the discrete state-space model, fv i g N i=1 into the approximate Euler equation for any x, V (x) = N β exp(αy i ) (v i + 1) i=1 N i=1 f (y i jx ) f (y i jµ) w i f (y i jx ) f (y i jµ) w i This function is continuous and di erentiable in x, and coincides with the step function when x is one of the y i s.. Craig Burnside (Duke University) Discrete State Space Methods September / 42

42 Computational Burden In a univariate example we had to solve N linear equations in N unknowns. But suppose we had a multivariate example with M state variables, and suppose the state space for each of these was approximated using N points Implies solving N M equations. Computation time for solving linear equations is approximately proportional to the cube of the number of equations, or in our case N 3M. So computation rapidly becomes more di cult as the number of state variables or the number of points rises. This is the so-called curse of dimensionality. Craig Burnside (Duke University) Discrete State Space Methods September / 42

43 Concluding Remarks Three main advantages of discrete state-space methods. They are easy to implement and very intuitive. Numerous theoretical results regarding convergence. Reliable. Main disadvantage of discrete state-space methods. Computationally ine cient for relatively complicated problems with a large number of state variables. Craig Burnside (Duke University) Discrete State Space Methods September / 42

44 Software Simple software for my chapter in the Marimon and Scott book is available on the research page of my website, Craig Burnside (Duke University) Discrete State Space Methods September / 42