Heterogeneous Agent Models in Continuous Time Part I

Transcription

1 Heterogeneous Agent Models in Continuous Time Part I Benjamin Moll Princeton University of Chicago, 28 September 2015

2 What this mini-course is about Many interesting questions require thinking about distributions Why are income and wealth so unequally distributed? Is there a trade-off between inequality and economic growth? What are the forces that lead to the concentration of economic activity in a few very large firms? Modeling distributions is hard closed-form solutions are rare computations are challenging Goal: teach you some new methods that make progress on this solving heterogeneous agent model = solving PDEs main difference to existing continuos-time literature: handle models for which closed-form solutions do not exist based on joint work with Yves Achdou, SeHyoun Ahn, Jiequn Han, Greg Kaplan, Pierre-Louis Lions, Jean-Michel Lasry, Gianluca Violante, Tom Winberry 1

3 Solving het. agent model = solving PDEs More precisely: a system of two PDEs 1. Hamilton-Jacobi-Bellman equation for individual choices 2. Kolmogorov Forward equation for evolution of distribution Many well-developed methods for analyzing and solving these codes: Apparatus is very general: applies to any heterogeneous agent model with continuum of atomistic agents 1. heterogeneous households (Aiyagari, Bewley, Huggett,...) 2. heterogeneous producers (Hopenhayn,...) can be extended to handle aggregate shocks (Krusell-Smith,...) 2

4 Outline Lecture 1 1. Refresher: HJB equations 2. Textbook heterogeneous agent model 3. Tools Lecture 2 viscosity solutions numerical solution of HJB equations 1. Analysis and numerical solution of heterogeneous agent model 2. models with non-convexities (Skiba) 3. models with multiple assets (HANK) 4. perturbation methods for handling aggregate uncertainty (starring Tom Winberry) 3

5 Computational Advantages relative to Discrete Time 1. Borrowing constraints only show up in boundary conditions FOCs always hold with = 2. Tomorrow is today FOCs are static, compute by hand: c γ = v a (a, z) 3. Sparsity solving Bellman, distribution = inverting matrix but matrices very sparse ( tridiagonal ) reason: continuous time one step left or one step right 4. Two birds with one stone tight link between solving (HJB) and (KF) for distribution matrix in discrete (KF) is transpose of matrix in discrete (HJB) reason: diff. operator in (KF) is adjoint of operator in (HJB) 4

6 Real Payoff: extends to more general setups non-convexities multiple assets aggregate shocks focus of lecture 2 5

7 What you ll be able to do at end of this course Joint distribution of income and wealth in Aiyagari model Density g(a,z,t) Income, z Wealth, a 10 6

8 What you ll be able to do at end of this course Experiment: effect of one-time redistribution of wealth Density g(a,z,t) Income, z Wealth, a 10 7

9 What you ll be able to do at end of this course Video of convergence back to steady state 8

10 Review: HJB Equations 9

11 Hamilton-Jacobi-Bellman Equation: Some History (a) William Hamilton (b) Carl Jacobi (c) Richard Bellman Aside: why called dynamic programming? Bellman: Try thinking of some combination that will possibly give it a pejorative meaning. It s impossible. Thus, I thought dynamic programming was a good name. It was something not even a Congressman could object to. So I used it as an umbrella for my activities. 10

12 Hamilton-Jacobi-Bellman Equations Pretty much all deterministic optimal control problems in continuous time can be written as v (x 0 ) = max {α(t)} t 0 subject to the law of motion for the state 0 e ρt h (x (t), α (t)) dt ẋ (t) = f (x (t), α (t)) and α (t) A for t 0, x(0) = x 0 given. ρ 0: discount rate x X R m : state vector α A R n : control vector h : X A R: instantaneous return function 11

13 Example: Neoclassical Growth Model subject to v (k 0 ) = max {c(t)} t 0 0 e ρt u(c(t))dt k (t) = F (k(t)) δk(t) c(t) for t 0, k(0) = k 0 given. Here the state is x = k and the control α = c h(x, α) = u(α) f (x, α) = F (x) δx α 12

14 Generic HJB Equation How to analyze these optimal control problems? Here: cookbook approach Result: the value function of the generic optimal control problem satisfies the Hamilton-Jacobi-Bellman equation ρv(x) = max α A h(x, α) + v (x) f (x, α) In the case with more than one state variable m > 1, v (x) R m is the gradient of the value function. Sometimes people (especially mathematicians) also write ρv(x) = H(x, v (x)) H(x, p) := max α A mathematicians call H the Hamiltonian h(x, α) + p f (x, α). 13

15 Example: Neoclassical Growth Model cookbook implies: ρv(k) = max c u(c) + v (k)(f (k) δk c) Proceed by taking first-order conditions etc u (c) = v (k) Derivation from discrete time Bellman equation 14

16 Poisson Uncertainty Easy to extend this to stochastic case. Simplest case: two-state Poisson process Example: RBC Model. Production is Z t F (k t ) where Z t {Z 1, Z 2 } Poisson with intensities λ 1, λ 2 Result: HJB equation is ρv i (k) = max c u(c) + v i (k)[z if (k) δk c] + λ i [v j (k) v i (k)] for i = 1, 2, j i. Derivation similar as before. 15

17 A Textbook Heterogeneous Agent Model 16

18 Households are heterogeneous in their wealth a and income y, solve max c t E 0 e ρt u(c t )dt 0 da t = (y t + r t a t c t )dt s.t. y t {y 1, y 2 } Poisson with intensities λ 1, λ 2 a t a c t : consumption u: utility function, u > 0, u < 0. ρ: discount rate r t : interest rate a > : borrowing limit e.g. if a = 0, can only save later: carries over to y t = general diffusion process. 17

19 Stationary Equilibrium Bonds in zero net supply 0 =S(r) = ag 1 (a)da + ag 2 (a)da (EQ) ρv i (a) = max c u(c) + v i (a)(y i + ra c) + λ i (v j (a) v i (a)) (HJB) 0 = d da [s i(a)g i (a)] λ i g i (a) + λ j g j (a), (KF) s i (a) =y i + ra c i (a), c i (a) = (u ) 1 (v i (a)) The two PDEs (HJB) and (KF) together with (EQ) fully characterize stationary equilibrium Derivation of (HJB) (KF) 18

20 Transition Dynamics 0 =S(r) = ag 1 (a, t)da + ag 2 (a, t)da (EQ) ρv i (a, t) = max u(c) + a v i (a, t)(y i + r(t)a c) c + λ i (v j (a, t) v i (a, t)) + t v i (a, t), (HJB) t g i (a, t) = a [s i (a, t)g i (a, t)] λ i g i (a, t) + λ j g j (a, t), (KF) s i (a, t) =y i + r(t)a c i (a, t), c i (a, t) = (u ) 1 ( a v i (a, t)) Given initial condition g i,0 (a), the two PDEs (HJB) and (KF) together with (EQ) fully characterize equilibrium. Notation: for any function f, x f means f x 19

21 Viscosity Solutions: A Primer 20

22 Viscosity Solutions For our purposes, useful for two reasons: 1. problems with kinks, e.g. coming from non-convexities 2. problems with state constraints borrowing constraints computations with bounded domain Things to remember 1. viscosity solution no concave kinks (convex kinks are allowed) 2. constrained viscosity solution: boundary inequalities 3. uniqueness: HJB equations have unique viscosity solution 21

23 Viscosity Solutions: Definition Consider HJB for generic optimal control problem { ρv(x) = max h(x, α) + v (x)f (x, α) } (HJB) α A Next slide: definition of viscosity solution Basic idea: v may have kinks i.e. may not be differentiable replace v (x) at point where it does not exist (because of kink in v) with derivative of smooth function ϕ touching v two types of kinks: concave and convex two conditions concave kink: ϕ touches v from above convex kink: ϕ touches v from below Remark: definition allows for concave kinks. In a few slides: these never arise in maximization problems 22

24 Convex and Concave Kinks Convex Kink (Supersolution) Concave Kink (Subsolution) v φ φ v x x 23

25 Viscosity Solutions: Definition Definition: A viscosity solution of (HJB) is a continuous function v such that the following hold: 1. (Subsolution) If ϕ is any smooth function and if v ϕ has a local maximum at point x (v may have a concave kink), then ρv(x { ) max h(x, α) + ϕ (x )f (x, α) } α A 2. (Supersolution) If ϕ is any smooth function and if v ϕ has a local minimum at point x (v may have a convex kink), then ρv(x { ) max h(x, α) + ϕ (x )f (x, α) }. α A 24

26 A few remarks, terminology If v is differentiable at x, then local max or min of v ϕ implies v (x ) = ϕ (x ) sub- and supersolution conditions viscosity solution of (HJB) is just classical solution If a continuous function v satisfies condition 1 (but not necessarily 2) we say that it is a viscosity subsolution Conversely, if v satisfies condition 2 (but not necessarily 1), we say that it is a viscosity supersolution a continuous function v is a viscosity solution if it is both a viscosity subsolution and a viscosity supersolution. subsolution and supersolution come from 0 and 0 viscosity is in honor of the method of vanishing viscosity : add Brownian noise and 0 (movements in viscuous fluid) 25

27 Viscosity Solutions: Intuition Consider discrete time Bellman: v(x) = max α h(x, α) + βv(x ), x = f (x, α) Think about it as an operator T on function v (T v)(x) = max α Solution = fixed point: T v = v Intuitive property of T ( monotonicity ) h(x, α) + βv(f (x, α)) ϕ(x) v(x) x (T ϕ)(x) (T v)(x) x ( ) Intuition: if my continuation value is higher, I m better off Viscosity solution is exactly same idea Key idea: sidestep non-differentiability of v by using monotonicity Mathematicians call monotonicity in ( ) comparison principle 26

28 Viscosity Solutions: Heuristic Derivation Time periods of length. Consider HJB equation v(x t ) = max α h(x t, α) + (1 ρ )v(x t+ ) s.t. x t+ = f (x t, α) + x t Suppose v is not differentiable at x and has a convex kink problem: when taking 0, pick up derivative v solution: replace continuation value with smooth function ϕ Convex Kink (Supersolution) v φ x 27

29 Viscosity Solutions: Heuristic Derivation For now: consider ϕ such that ϕ(x ) = v(x ) local min of v ϕ and ϕ(x ) = v(x ) v(x) > ϕ(x), x x Then for x t = x, we have v(x t ) max α h(x t, α t ) + (1 ρ )ϕ(x t+ ) Subtract (1 ρ )ϕ(x t ) from both sides and use ϕ(x t ) = v(x t ) ρv(x t ) max α h(x t, α) + (1 ρ )(ϕ(x t+ ) ϕ(x t )) Dividing by and letting 0 yields the supersolution condition ρv(x t ) max α h(x t, α) + ϕ (x t )f (x t, α) 28

30 Viscosity Solutions: Heuristic Derivation Turns out this works for any ϕ such that v ϕ has local min at x define κ = v(x ) ϕ(x ). Then v(x ) = ϕ(x ) + κ and v(x t ) max α h(x t, α t ) + (1 ρ )(ϕ(x t+ ) + κ) subtract (1 ρ )(ϕ(x t ) + κ) from both sides ρv(x t ) max α h(x t, α) + (1 ρ )(ϕ(x t+ ) ϕ(x t )) rest is the same... Derivation of subsolution condition exactly symmetric 29

31 Viscosity + maximization no concave kinks Proposition The viscosity solution of the HJB equation corresponding to { ρv(x) max h(x, α) + v (x)f (x, α) } = 0 α A only admits convex (downward) kinks, but not concave (upward) kinks. Admissible Not Admissible v(x) v(x) x x think: not special to HJB, true for any v(x) = max y f (x, y) opposite would be true for minimization problem 30

32 Constrained Viscosity Soln: Boundary Inequalities How handle state constraints? borrowing constraints computations with bounded domain Example: growth model with state constraint v (k 0 ) = max {c(t)} t 0 0 e ρt u(c(t))dt k (t) = F (k(t)) δk(t) c(t) k(t) k min all t 0 s.t. purely pedagogical: constraint will never bind if k min < st.st. HJB equation ρv(k) = max c Key question: how impose k k min? { u(c) + v (k)(f (k) δk c) } (HJB) 31

33 Example: Growth Model with Constraint Result: if v is differentiable at k min, it needs to satisfy v (k min ) u (F (k min ) δk min ) (BI) Intuition: v (k min ) is such that if k(t) = k min then k(t) 0 if v is differentiable, the FOC still holds at the constraint u (c(k min )) = v (k min ) (FOC) for constraint not to be violated, need F (k min ) δk min c(k min ) 0 ( ) (FOC) and ( ) (BI). Next: rigorous derivation from constrained viscosity solution 32

34 Constrained Viscosity Soln: Rigorous Derivation Consider variant of generic maximization problem v(x) = max {α(t)} t 0 ẋ(t) = f (x(t), α(t)), x(t) x min all t 0 0 e ρt h(x(t), α(t))dt x(0) = x s.t. HJB equation { ρv(x) = max h(x, α) + v (x)f (x, α) } (HJB) α A Key question: how impose x x min? natural: v has to be a viscosity solution for x > x min but what about at x = x min? 33

35 Constrained Viscosity Soln: Rigorous Derivation Definition: a constrained viscosity solution of (HJB) is a continuous function v such that 1. v is a viscosity solution (i.e. both sub- and supersolution) for all x > x min 2. v is a supersolution at x = x min : if ϕ is any smooth function and if v ϕ has a local minimum at point x min, then { ρv(x min ) max h(xmin, α) + ϕ (x min )f (x min, α) } ( ) α A ( ) functions as boundary condition, or rather boundary inequality More can be said in case v is differentiable: next slide 34

36 Again intuitive: v (x min ) is such that ẋ(t) 0 if x(t) = x min 35 Constrained Viscosity Soln: Rigorous Derivation Use Hamiltonian formulation: Note: H(x, p) := max α A h(x, α) + pf (x, α) p H(x, v (x)) = f (x min, α (v (x min ))) = optimal drift Condition 2 becomes: if ϕ is any smooth function and if v ϕ has a local minimum at point x min, then ρv(x min ) H(x min, ϕ (x min )) Corollary: if v is differentiable at x min, then ( ) is equivalent to Proof p H(x min, v (x min )) = optimal drift at x min 0 ( )

37 Example: Growth Model with Bounded Domain Consider again HJB equation for growth model { ρv(k) = max u(c) + v (k)(f (k) δk c) } c For numerical solution, want to impose k min k(t) k max all t How can we ensure this? Answer: impose two boundary inequalities v (k min ) u (F (k min ) δk min ) v (k max ) u (F (k max ) δk max ) 36

38 Uniqueness of Viscosity Solution Theorem 1: HJB equation has unique viscosity solution Theorem 2: viscosity solution equals value function, i.e. solution to sequence problem due to Crandall and Lions (1983), Viscosity Solutions of Hamilton-Jacobi Equations My intuition for Theorem 1 in one-dimensional case ODE has unique solution given one boundary condition two boundary inequalities = one boundary condition but much more powerful: generalizes to n dimensions, kinks etc 37

39 Some general, somewhat philosophical thoughts MAT 101 way ( first-order ODE needs one boundary condition ) is not the right way to think about HJB equations these equations have very special structure which one should exploit when analyzing and solving them Particularly true for computations (next) Important: all results/algorithms apply to problems with more than one state variable, i.e. it doesn t matter whether you solve ODEs or PDEs 38

40 Numerical Solution of HJB Equations 39

41 Finite Difference Methods See Explain using neoclassical growth model, easily generalized to heterogeneous agent models Functional forms ρv(k) = max c u(c) + v (k)(f (k) δk c) Use finite difference method Two MATLAB codes u(c) = c1 σ, F (k) = kα 1 σ

42 Barles-Souganidis There is a well-developed theory for numerical solution of HJB equation using finite difference methods Key paper: Barles and Souganidis (1991), Convergence of approximation schemes for fully nonlinear second order equations Result: finite difference scheme converges to unique viscosity solution under three conditions 1. monotonicity 2. consistency 3. stability Good reference: Tourin (2013), An Introduction to Finite Difference Methods for PDEs in Finance. 41

43 Finite Difference Approximations to v (k i ) Approximate v(k) at I discrete points in the state space, k i, i = 1,..., I. Denote distance between grid points by k. Shorthand notation v i = v(k i ) Need to approximate v (k i ). Three different possibilities: v (k i ) v i v i 1 k v (k i ) v i+1 v i k v (k i ) v i+1 v i 1 2 k = v i,b = v i,f = v i,c backward difference forward difference central difference 42

44 Finite Difference Approximations to v (k i ) Backward Forward! #! #%$!! # $ Central # $ # #%$ " 43

45 Finite Difference Approximation FD approximation to HJB is ρv i = u(c i ) + v i [F (k i) δk i c i ] ( ) where c i = (u ) 1 (v i ), and v i is one of backward, forward, central FD approximations. Two complications: 1. which FD approximation to use? Upwind scheme 2. ( ) is extremely non-linear, need to solve iteratively: explicit vs. implicit method My strategy for next few slides: first: what works afterwards: why it works (Barles-Souganidis) 44

46 Which FD Approximation? Which of these you use is extremely important Best solution: use so-called upwind scheme. Rough idea: forward difference whenever drift of state variable positive backward difference whenever drift of state variable negative In our example: define s i,f = F (k i ) δk i (u ) 1 (v i,f ), s i,b = F (k i ) δk i (u ) 1 (v i,b ) Approximate derivative as follows v i = v i,f 1 {s i,f >0} + v i,b 1 {s i,b <0} + v i 1 {s i,f <0<s i,b } where 1 { } is indicator function, and v i = u (F (k i ) δk i ). Where does v i term come from? Answer: since v is concave, v i,f < v i,b (see figure) s i,f < s i,b if s i,f < 0 < s i,b, set s i = 0 v (k i ) = u (F (k i ) δk i ), i.e. we re at a steady state. 45

47 Sparsity Discretized HJB equation is ρv i = u(c i ) + v i+1 v i k s + i,f + v i v i 1 s k i,b Notation: for any x, x + = max{x, 0} and x = min{x, 0} Can write this in matrix notation ρv = u + Av where A is I I (I= no of grid points) and looks like... 46

48 Visualization of A (output of spy(a) in Matlab) nz =

49 The matrix A FD method approximates process for k with discrete Poisson process, A summarizes Poisson intensities entries in row i: s i,b k }{{} inflow i 1 0 s i,b k s+ i,f k }{{} outflow i 0 s + i,f }{{} k inflow i+1 0 v i 1 v i v i+1 negative diagonals, positive off-diagonals, rows sum to zero: tridiagonal matrix, very sparse A depends on v (nonlinear problem). Next: iterative method... 48

50 Iterative Method Idea: Solve FOC for given v n, update v n+1 according to v n+1 i v n i + ρv n i = u(c n i ) + (v n ) (k i )(F (k i ) δk i c n i ) Algorithm: Guess v 0 i, i = 1,..., I and for n = 0, 1, 2,... follow 1. Compute (v n ) (k i ) using FD approx. on previous slide. 2. Compute c n from c n i = (u ) 1 [(v n ) (k i )] 3. Find v n+1 from ( ). 4. If v n+1 is close enough to v n : stop. Otherwise, go to step 1. ( ) See Important parameter: = step size, cannot be too large ( CFL condition ). Pretty inefficient: I need 5,990 iterations (though quite fast) 49

51 Efficiency: Implicit Method Efficiency can be improved by using an implicit method v n+1 i v n i + ρv n+1 i = u(ci n ) + (v n+1 i Each step n involves solving a linear system of the form 1 (v n+1 v n ) + ρv n+1 = u + A n v n+1 ( (ρ + 1 )I A ) n v n+1 = u + 1 v n but A n is super sparse super fast ) (k i )[F (k i ) δk i c n i ] See In general: implicit method preferable over explicit method 1. stable regardless of step size 2. need much fewer iterations 3. can handle many more grid points 50

52 Implicit Method: Practical Consideration In Matlab, need to explicitly construct A as sparse to take advantage of speed gains Code has part that looks as follows X = -min(mub,0)/dk; Y = -max(muf,0)/dk + min(mub,0)/dk; Z = max(muf,0)/dk; Constructing full matrix slow for i=2:i-1 A(i,i-1) = X(i); A(i,i) = Y(i); A(i,i+1) = Z(i); end A(1,1)=Y(1); A(1,2) = Z(1); A(I,I)=Y(I); A(I,I-1) = X(I); Constructing sparse matrix fast A =spdiags(y,0,i,i)+spdiags(x(2:i),-1,i,i)+spdiags([0;z(1:i-1)],1,i,i); 51

53 Why this works? Barles-Souganidis Here: version with one state variable, but generalizes Can write any HJB equation with one state variable as 0 = G(k, v(k), v (k), v (k)) (G) Corresponding FD scheme 0 = S ( k, k i, v i ; v i 1, v i+1 ) (S) Growth model G(k, v(k), v (k), v (k)) = ρv(k) max c u(c) + v (k)(f (k) δk c) S ( k, k i, v i ; v i 1, v i+1 ) = ρv i u(c i ) v i+1 v i (F (k i ) δk i c i ) + k v i v i 1 (F (k i ) δk i c i ) k 52

54 Why this works? Barles-Souganidis 1. Monotonicity: the numerical scheme is monotone, that is S is non-increasing in both v i 1 and v i+1 2. Consistency: the numerical scheme is consistent, that is for every smooth function v with bounded derivatives S ( k, k i, v(k i ); v(k i 1 ), v(k i+1 )) G(v(k), v (k), v (k)) as k 0 and k i k. 3. Stability: the numerical scheme is stable, that is for every k > 0, it has a solution v i, i = 1,.., I which is uniformly bounded independently of k. 53

55 Why this works? Barles-Souganidis Theorem (Barles-Souganidis) If the scheme satisfies the monotonicity, consistency and stability conditions 1 to 3, then as k 0 its solution v i, i = 1,..., I converges locally uniformly to the unique viscosity solution of (G) Note: convergence here has nothing to do with iterative algorithm converging to fixed point Instead: convergence of v i as k 0. More momentarily. 54

56 Intuition for Monotonicity Write (S) as ρv i = S( k, k i, v i ; v i 1, v i+1 ) For example, in growth model S( k, k i, v i ; v i 1, v i+1 ) = u(c i ) + v i+1 v i (F (k i ) δk i c i ) + k + v i v i 1 (F (k i ) δk i c i ) k Monotonicity: S in v i 1, v i+1 ( S in v i 1, v i+1 ) Intuition: if my continuation value at i 1 or i + 1 is larger, I must be at least as well off (i.e. v i on LHS must be at least as high) 55

57 Checking the Monotonicity Condition in Growth Model Recall upwind scheme: S ( k, k i, v i ; v i 1, v i+1 ) = ρv i u(c i ) v i+1 v i (F (k i ) δk i c i ) + k v i v i 1 (F (k i ) δk i c i ) k Can check: satisfies monotonicity: S is indeed non-increasing in both v i 1 and v i+1 c i depends on v i s but doesn t affect monotonicity due to envelope condition 56

58 Meaning of Convergence Convergence is about k 0. What, then, is content of theorem? have a system of I non-linear equations S ( k, k, v i ; v i 1, v i+1 ) = 0 need to solve it somehow Theorem guarantees that solution (for given k) converges to solution of the HJB equation (G) as k. Why does iterative scheme work? Two interpretations: 1. Newton method for solving system of non-linear equations (S) 2. Iterative scheme solve (HJB) backward in time v n+1 i v n i + ρvi n in effect sets v(k, T ) = initial guess and solves ρv(k, t) = max c = u(c n i ) + (v n ) (k i )(F (k i ) δk i c n i ) u(c) + k v(k, t)(f (k) δk c) + t v(k, t) backwards in time. v(k) = lim t v(k, t). 57

59 Relation to Kushner-Dupuis Markov-Chain Approx There s another common method for solving HJB equation: Markov Chain Approximation Method Kushner and Dupuis (2001) Numerical Methods for Stochastic Control Problems in Continuous Time effectively: convert to discrete time, use value fn iteration FD method not so different: also converts things to Markov Chain ρv = u + Av Connection between FD and MCAC see Bonnans and Zidani (2003), Consistency of Generalized Finite Difference Schemes for the Stochastic HJB Equation also shows how to exploit insights from MCAC to find FD scheme satisfying Barles-Souganidis conditions Another source of useful notes/codes: Frédéric Bonnans website 58

60 Appendix 59

61 Derivation from Discrete-time Bellman Back Time periods of length discount factor β( ) = e ρ Note that lim 0 β( ) = 1 and lim β( ) = 0. Discrete-time Bellman equation: v(k t ) = max c t u(c t ) + e ρ v(k t+ ) s.t. k t+ = [F (k t ) δk t c t ] + k t 60

62 Derivation from Discrete-time Bellman For small (will take 0), e ρ 1 ρ v(k t ) = max c t u(c t ) + (1 ρ )v(k t+ ) Subtract (1 ρ )v(k t ) from both sides ρ v(k t ) = max c t u(c t ) + (1 ρ)[v(k t+ ) v(k t )] Divide by and manipulate last term ρv(k t ) = max u(c t ) + (1 ρ) v(k t+ ) v(k t ) c t k t+ k t k t+ k t Take 0 ρv(k t ) = max c t u(c t ) + v (k t ) k t 61

63 Derivation of Poisson KF Equation Back Work with CDF (in wealth dimension) G i (a, t) = Pr(ã t a, z t = z i ) Over time period of length, wealth evolves as ã t+ = ã t + s i (ã t ) income switches from z i to z j with probability λ i Fraction of people with wealth below a evolves as Pr(ã t+ a, z t+ = z i ) = (1 λ i ) Pr(ã t a s i (a), z t = z i ) + λ j Pr(ã t a s j (a), z t = z j ) Intuition: if have wealth < a s i (a) at t, have wealth < a at t + 62

64 Derivation of Poisson KF Equation Subtracting G i (a, t) from both sides and dividing by G i (a, t + ) G i (a, t) Taking the limit as 0 = G i(a s i (a), t) G i (a, t) λ i G i (a s i (a), t) + λ j G j (a s j (a), t) t G i (a, t) = s i (a) a G i (a, t) λ i G i (a, t) + λ j G j (a, t) where we have used that G i (a s i (a), t) G(a, t) G(a x, t) G(a, t) lim = lim s i (a) 0 x 0 x = s i (a) a G i (a, t) Differentiate w.r.t. a and use g i (a, t) = a G i (a, t) (KF). 63

65 Proof of State Constraint Boundary Inequality Back Recall condition 2: if ϕ is any smooth function and if v ϕ has a local minimum at point x min, then ρv(x min ) H(x min, ϕ (x min )) If v is differentiable at x min, local min of v ϕ (v ϕ) (x min ) 0 ρv(x min ) H(x min, p) all p v (x min ) (1) Note: (1) true only at boundary x = x min interior: have v (x) = ϕ (x) ρv(x) H(x, p), p=v (x) By continuity of v and v Combining (1) and (2) ρv(x min ) = H(x min, v (x min )) (2) H(x min, v (x min )) H(x min, p) all p v (x min ) Equivalently p H(x min, v (x min )) 0. 64