Discrete Math Camp Optimization Problem Graduate School of Economics, University of Tokyo April 2, 2016
Goal of day 4 Discrete We discuss methods both in discrete and continuous : Discrete : condition : condition Because of constraint, I will mainly focus on discrete case.
Discrete Why are s important? One s decision about today s action affects the tomorrow s utility through the tomorrow s state. The amount of consumption today determines the amount of saving and tomorrow s budget constraint. Then, the amount of consumption today affects the amount of consumption tomorrow. Thus, to analyze individuals decision-makings, we have to take into account the aspects. Indeed, many economic researches deal with the aspects of social phenomena: RBC model, search theory, growth theory (macroeconomics) general equilibrium, repeated games, auction theory (microeconomics) discrete choice model (econometrics)
Discrete 1.1. discrete
Motivation: optimal growth theory Discrete You will encounter the optimal growth theory in Macroeconomics I. The objective of the optimal growth theory is to derive the optimal growth rate which maximizes the utility of a representative household. From now on, I will mainly use this theory as an example of a problem.
Goal Discrete Euler equation Transversality condition
Finite horizon Discrete You have seen the following problem: max u(c 0 ) + βu(c 1 ). c 0,c 1,k 1 s.t. c 0 + k 1 = f (k 0 ), c 1 = f (k 1 ) where k 0 is given. You can solve this problem by using Lagrange multiplier method.
Euler equation Discrete u (c t ) = βf (k t+1 )u (c t+1 ) This represents that the benefit and cost of a marginal change in consumption and saving are equal at the optimum. u (c t ): marginal utility cost of increasing saving today. u (c t+1 ): marginal utility of consumption tomorrow. f (k t+1 ): marginal income gain tomorrow.
One conjecture Discrete Even if there exist more periods, Euler equation is just enough for the optimality condition. Is this really true? Answer Finite horizen model Yes. Infinite horizen model No. Transversality condition is needed.
Finite vs. infinite Discrete Finite horizen T β t u(c t ). t=0 One knows exactly the timing of his own death. Infinite horizen β t u(c t ). t=0 Even though the horizen is finite, if one does not know the timing of his own death, the situation can be formulated as an infinite horizen model. Suppose that with probability m, one will die at the end of each period. And denote the discount factor by β. Then, β = β (1 m).
Finite vs. infinite Discrete Which is plausible?
Finite vs. infinite Discrete It depends on what kind of aspect you want to focus on in your model. When you focus on the difference between ages, you should use finite horizen model. Otherwise, it may be better to use infinite horizen model.
Finite horizon Discrete Next, consider the following problem: max c t,k t+1 T β t u(c t ) t=0 s.t. k t+1 + c t = f (k t ) where k 0 is given, k t+1 0 for t = 0,.., T 1, and k T +1 = 0. Again, you can solve this problem by using Lagrange multiplier method. Euler equation is derived as FOC.
Transversality condition Discrete In the former problem, I put the assumption such that k T +1 = 0. This is because nothing should be saved in the last period. There exists no terminal point in the infinite horizon case. We modify the former problem. max c t,k t+1 T β t u(c t ) t=0 s.t. k t+1 + c t = f (k t ) where k 0 is given, and k t+1 0 for t = 0,.., T 1.
condition Discrete FOC for t = 0,..., T 1 is: u (c t ) = βf (k t+1 )u (c t+1 ) FOC for t = T is: β T u (c T ) = 0 or β T u (c T ) 0, k T +1 = 0. Because u (c T ) 0, these two cases can be combined into β T u (c T )k T +1 = 0.
Transversality condition Discrete Transversality condition means that nothing should be saved in the last period unless it is costless to do so (u (c T ) = 0). When the value of the terminal point k T +1 is not given, there may be many paths satisfying the Euler equation. If we do not put an adhoc assumption such that k T +1 = 0, we need transversality condition as well as Euler equation.
Discrete Infinite horizon Last, consider the following infinite horizon problem: max c t,k t+1 β t u(c t ) t=0 s.t. k t+1 + c t = y t + f (k t ) where k 0 is given and k t+1 0 for t = 0, 1,... Euler equation: For t = 0,..., Transversality condition: u (c t ) = βf (k t+1 )u (c t+1 ) lim T βt u (c T )k T +1 = 0.
Discrete No-Ponzi-game condition and transversality condition In the analysis of a consumer s behavior, we consider some constraint on debt to exclude a situation such that the consumer will never pay back his debt. To do so, we often put an assumption such that the present discounted value of debt at infinity must be nonnegative. This condition is called no-ponzi-game condition. But, this is often called transversality condition, too. However, this condition is different from transversality condition which has been discussed until now.
Discrete 1.2.
Discrete Motivation of We can characterize the solution by using Lagrange multiplier method even in the case of infinite horizon. However, we can no longer solve this problem and obtain the solution itself by using Lagrange multiplier method. This is due to infinite dimensionality of the problem: infinitely many FOCs. The way to solve this problem (not to characterize) is.
Exploiting recursive structure Discrete The previous problem has the recursive structure. That is, the structure of the choice problem that a decision maker faces is identical at every point in. Therefore, we expect the agent s decisions are the same whenever k t are the same.
Exploiting recursive structure Discrete Examples of recursive structure S = x + βx + β 2 x +... is equal to S = x + βs. Suppose that x t = x 0 + αt. Then, the sum is S(x 0 ) = x 0 + β(x 0 + α) + β 2 (x 0 + 2α) +... And this can be rewritten as: S(x 0 ) = x 0 + βs(x 0 + α). S(x 0 ) = x 0 + βs(x 1 ).
Bellman equation Discrete V (k 0 ) = = max u(c 0 ) + β c 0,k 1 max {c t,k t+1 } t=0 max β t u(c t ) t=0 {c t,k t+1 } t=1 Then, because of resursive structure, β t 1 u(c t ) V (k 0 ) = max c 0,k 1 u(c 0 ) + βv (k 1 ).
Bellman equation Discrete The problem in a recursive form is: V (k t ) = max c t,k t+1 u c (t) + βv (k t+1 ). s.t. k t+1 + c t = f (k t ) where k 0 is given, k t+1 0 for t = 0, 1,... The above equation is called bellman equation. Under certain conditions, solving bellman equation is equivalent to solving the original problem!
State variable vs. control variable Discrete What characterizes the decision structure is k t, which was pre-determined at t. We call this a state variable. On the other hand, what you choose at t is called a control variable. In this case, this is c t.
Another example Discrete McCall (1970) s search model: Two states of a worker: unemployed, employed. If unemployed, the worker receives a job offer whose wage is w. This offer is drawn randomly according to the probability distribution F (w). After the worker accepted the offer, the accepted wage continues forever. The worker is risk-neutral and discount factor is β.
Another example Discrete Bellman equation is: V (w) = max { w 1 β, β } V (w )df (w ). Here, the state variable tomorrow is not deterministic. This is stochastic although I do not cover the details about stochastic case.
General model Discrete Consider the following sequential problem: max {x t+1 } t=0 β t F (x t, x t+1 ) t=0 s.t. x t+1 Γ(x t ), t = 0, 1, 2,... given x 0 X. Here x t is called a state variable (vector) and take a value in some set X, and a correspondence Γ : X X describes the state transition. F : A R is the one-period return function where A := {(x, y) X X : y Γ(x)}. β (0, 1) is a discount factor.
General model Discrete The problem in a recursive form is: V (x t ) = max F (x t, x t+1 ) + βv (x t+1 ) x t+1 Γ(x t ) s.t. x t+1 Γ(x t ), t = 0, 1, 2,... given x 0 X. The above equation is called bellman equation. This is a functional equation (FE) in which unknown variables are functions.
Relationship between V and V Discrete What is the relationship between V (value function in the sequential problem) and V (solution to FE)? Remark V is unique. On the other hand, there may exist more than one solution to (FE).
Relationship between V and V Discrete Plan from x 0 is denoted by x = (x 0, x 1,...). Assumption 1 Γ(x) ϕ for any x X. Assumption 2 For any x 0 X and x Γ(x 0 ), lim n t=0 n β t F (x t, x t+1 ) = β t F (x t, x t+1 ) t=0 exists.
Relationship between V and V Discrete Proposition Suppose that Assumption 1 and 2 are satisfied. Then, V satisfies (FE).
Relationship between V and V Discrete Proposition Suppose that Assumption 1 and 2 are satisfied. Also, suppose that if V satisfies (FE) and lim t βt V (x t ) = 0 for any x 0 X and x Γ(x 0 ). Then, V = V.
How to solve Discrete There are three ways to solve Bellman equation. Guess and Verify Guess the functional form of V or policy function, and verify whether the guess is correct. Value Function Iteration Start from any initial guess of value function, and keep updating until it converges. Policy Function Iteration Start from any initial guess of policy function, and keep updating until it converges.
condition revisited Discrete Without solving the functional equation, we can characterize the solution of FE by deriving the optimality conditions (e.g. Euler condition).
FOC Discrete FOC is: V (x t ) = max F (x t, x t+1 ) + βv (x t+1 ) x t+1 Γ(x t) s.t. x t+1 Γ(x t ), t = 0, 1, 2,... given x 0 X. F (x i,t, x i,t+1 ) + β V (x i,t+1 ) = 0. x i,t+1 x i,t+1 Here, V x i,t+1 (x i,t+1 ) is unknown. Envelope theorem
Envelope theorem Discrete If V is differentiable, Thus, FOC can be rewritten as F x i,t+1 V (x x i,t) = F (x i,t x i,t, xi,t+1). i,t (x i,t, x i,t+1) + β F x i,t (x i,t, x i,t+1) = 0.
Euler equation Discrete Consider the following problem: max c t,k t+1 β t u(c t ) t=0 s.t. k t+1 + c t = f (k t ) where k 0 is given and k t+1 0 for t = 0, 1,... Then, FOC is: This is called Euler equation. u (c t ) = βf (k t+1 )u (c t+1 ).
Discrete 2. continuous
Typical problem in continuous Discrete We discussed the discrete version optimal growth theory. However, in macroeconomics I, you will encounter the continuous version optimal growth theory. and k 0 is given. max c(t),k(t) 0 e ρt u(c(t)) s.t. c(t) + k(t) = f (k(t)) δk(t), k(t) 0,
Typical problem in continuous Discrete and k(0) is given. max c(t),k(t) 0 v(k(t), c(t), t)dt s.t. k(t) = g(k(t), c(t), t), k(t) 0, How can we derive optimality conditions?
How to get FOC Discrete Construct a Hamiltonian function: Then, the FOC is: and H = v(k, c, t) + µ(t) g(k, c, t). H c = 0, H k = µ(t), lim µ(t)k(t) = 0. t The last condition is transversality condition.
Discrete In many economic applications, Thus, 0 v(k(t), c(t), t)dt = Current-value Hamiltonian 0 e ρt u(k(t), c(t), t)dt. H = e ρt u(k(t), c(t), t) + µ(t) g(k, c, t). Then, the shadow price µ(t) represents present-value of the capital stock k. We can restructure the problem in terms of current-value prices. where q(t) = e ρt µ(t). Ĥ = u(k, c, t) + q(t) g(k, c, t)
Current-value Hamiltonian Discrete Then, FOC is: and Ĥ k Ĥ c = 0, = ρq(t) q(t), lim t q(t)e ρt k(t) = 0.
Discrete Adda and Cooper (2003). Economics: Quantitative Methods and Applications. The MIT Press. Barro and Sala-i-Martin (2003). Economic Growth, Second Edition. The MIT Press. Mathematical Appendix. [introduction to continuous case] Kamihigashi (2008). Transversality Conditions and Economic Behavior, New Palgrave Dictionary of Economics, 2nd Edition, Vol.8. 384-387.[introduction to transversality condition] Stokey and Lucas, Jr. (1989). Recursive Methods in Economic s, Harvard Univeristy Press. [, difficult]