Mean Field Consumption-Accumulation Optimization with HAR

Transcription

1 Mean Field Consumption-Accumulation Optimization with HARA Utility School of Mathematics and Statistics Carleton University, Ottawa Workshop on Mean Field Games and Related Topics 2 Padova, September 4-7, 2013

2 Mean field games and stochastic growth Some background Mean field games: Competitive decision with a large number of agents The fast growing literature for MFGs Stochastic growth theory: Consumption and investment optimization Optimal control of a whole sector of an economy More generally: Nash games of N producers The pioneering work (Brock and Mirman, J. Econ. Theory, 1972); a nice survey (Olson and Roy, 2006)

3 Mean field games and stochastic growth Our plan Endogenize the mean field effect into the production dynamics Thus generalize the conventional modeling Decentralized optimization, closed-form solution Illustrate long time nonlinear behaviour The success of mean field games so far relies on the fact: Agents can rationally anticipate the mean field behaviour Can the above be assumed without questioning? The model here shows new challenges. Very different from ergodic stochastic control

4 Mean field games and stochastic growth Note from the computational aspect Except the LQG/LQEG cases (see e.g. the work of Bardi; Bensoussan et al; Caines, Huang and Malhamé; Li and Zhang; Tembine at al.;...), closed form solutions in MFGs are rare. Guéant, Lasry and Lions (2011) obtained closed form solutions in a human capital optimization model by considering certain linear control classes. We look for closed-form solutions in a new class of models. Stepping on this, we can address deeper issues.

5 Classical stochastic one-sector growth model: Review The one-sector economy at stage t involves two basic quantities: κ t : the capital stock (used for investment) c t : consumption The next stage output y t+1 : y t+1 = f (κ t, r t ), t = 0, 1,..., f (, ): called the production function r t : random disturbance y 0 : the initial output, given If the output remaining after investment is all consumed, one has the constraint κ t + c t = y t

6 Objective: maximize the utility functional E ρ t ν(c t ), t=0 ν(c t ): utility from consumption, usually concave on [0, ) Issues of interest: sustainability, extinction (like whale hunting), business cycles and other oscillatory behaviors, etc. The model is closely related to the theory of optimal savings and portfolio optimization which usually consider linear production functions. Brock and Mirman (J. Econ. Theory, 1972) pioneered stochastic growth theory.

7 Notation in the mean field model Keep track of the notation (for the main part): ut: i control (investment) Xt i : state (production output) N: number of players in the game ct: i consumption V i (x, t): value function G(p, W ), g: growth coefficient in production W : white noise p: aggregate investment γ: HARA utility exponent

8 Mean field production dynamics of N agents X i t : output (or wealth) of agent i, 1 i N u i t [0, X i t ]: investment (so no borrowing) c i t = X i t u i t: amount for consumption u (N) t = (1/N) N j=1 uj t: aggregate investment level The next stage output, measured by the unit of capital, is Xt+1 i = G(u (N) t, Wt i )ut, i t 0, (3.1) Regard u (N) t unit. as a quantity measured according to a macroscopic

9 Motivation for the mean field production dynamics Some literature addresses negative externalities. For example, Barro and Sala-I-Martin (Rev. Econ. Stud., 1992) considered the so-called congestion effect y = Ak(G/K) α, α (0, 1) y: output of a private producer; k: its input; K: aggregate input of n producers; G: total resource provided by the government. For similar models, see (Liu and Turnovsky, J. Pub. Econ., 2005)

10 The utility functional The utility functional is T J i (u i, u i ) = E ρ t v(xt i ut), i t=0 ρ (0, 1]: the discount factor c i t = X i t u i t: consumption, u i = (, u i 1, u i+1, ) We take the HARA utility v(z) = 1 γ zγ, z 0, γ (0, 1).

11 Assumptions (A1) (i) Each sequence {Wt i, t Z + } consists of i.i.d. random variables with support D W and distribution function F W. The N sequences {Wt i, t Z + }, i = 1,..., N are i.i.d. (ii) The initial states {X0 i, 1 i N} are i.i.d. positive random variables with distribution F X0 and mean m 0, which are also independent of the N noise sequences. (A2) (i) The function G: [0, ) D W [0, ) is continuous; (ii) for a fixed w D W, G(z, w) is a decreasing function of z [0, ). (A3) (iii) EG(0, W ) < and EG(p, W ) > 0 for each p [0, ). (A2) implies congestion effect: when the aggregate investment level increases, the production becomes less efficient.

12 Example Example. Suppose G(z, w) = are parameters. αw 1+δz η, where α > 0, δ > 0, η > 0 The congestion effect is indicated by the monotonic decreasing property of G

13 Further interpretation of (A2): Diminishing return of the sector (although no notable d.r. to individual scale) Illustration: Suppose the production structure G(z, W ) = g(z)w, g decreasing. Let X (N) t = 1 N N i=1 X i t. Then X (N) t = g(u (N) t = g(u (N) t )u (N) t )u (N) t t ) EW0 1 + g(u(n) N N i=1 EW small fluctuation u i t(w i t EW i t )

14 To do (in the Nash certainty equivalence approach) Step 1 Approximate the aggregate investment (u (N) t a sequence (p 0,..., p T 1 ). Step 2 Solve an optimal control problem with a representative agent Obtain control ût i = α t ˆX t i (linear feedback). Determine closed loop state ˆX t i. Step 3 Consistency requirement ) T t=0 1 by lim (1/N) N ût i = lim (1/N) N α t ˆX t i = α t E ˆX t i = p t. N N i=1 i=1 Step 4 Show ε N -Nash property for (û 1 t,..., û N t ).

15 Step 1: mean field limit Now agent i considers the optimal control problem with dynamics where u i t [0, X i t ]. X i t+1 = G(p t, W i t )u i t, t 0, (3.2) The utility functional is now written as J i (u i, (p t ) T 1 0, 0) = E T ρ t v(xt i ut), i (3.3) t=0

16 Step 2: optimal control Further denote the utility functional with initial time t J i (u i, (p l ) t T 1, t) = E T ρ s t v(xs i us), i which is affected only by (p l ) T t 1. The value function is defined as V i (x, t) = sup {u i s }T s=t s=t E[ J i (u i, (p l ) T t 1, t) Xt i = x]. We have the dynamic programming equation [ V i (x, t) = max v(x ui ) + ρev i (G(p t, Wt i )u i, t + 1) ], 0 u i x where t = 0, 1,..., T 1.

17 Step 2 We look for a solution V i (x, t) = 1 γ Dγ 1 t x γ. (3.4) Denote Φ(z) = ρeg γ (z, W ) and ϕ(z) = Φ 1 γ 1 (z). Theorem (i) The value function V i (x, t) takes the form (3.4), where D t = ϕ(p t)d t ϕ(p t )D t+1, t T 1, D T = 1. (3.5) (ii) The optimal control has the feedback form u i t = X i t 1 + ϕ(p t )D t+1, t T 1, u i T = 0. (3.6)

18 Step 3: consistency We have the closed-loop X i t = G(p t, W i t ) X i t 1 + ϕ(p t )D t+1. By symmetry, lim N Eu (N) t = Eu i t =: Λ t (p 0,..., p T 1 ). Define the operator Λ (an example to follow) (p 0,..., p T 1 ) Λ(p 0,..., p T 1 ):=(Λ 0,..., Λ T 1 )(p 0,..., p T 1 ). assumed mean field actual mean field of closed-loop Theorem Λ has a fixed point in a rectangle region. By this fixed point, the initially assumed mean field is regenerated by the agents taking individual optimal responses.

19 Step 3: consistency Example: We take T = 2. Let (p 0, p 1 ) be given. Then Λ 0 (p 0, p 1 ) = (1 + ϕ 1)EX i ϕ 1 + ϕ 1 ϕ 0, Λ 1 (p 0, p 1 ) = EG(p 0, W i 0 )EX i ϕ 1 + ϕ 1 ϕ 0. where ϕ t = ϕ(p t ) and we recall ϕ(z) = Φ 1 γ 1 (z).

20 Step 4: ε-nash Theorem The set of NCE based strategies {ǔ i t, 0 t T, 1 i N} is an ε N -Nash equilibrium, i.e., for any i {1,..., N}, sup J i (u i, ǔ i ) ε N J i (ǔ i, ǔ i ) sup J i (u i, ǔ i ), (3.7) u i u i where u i is a centralized strategy and 0 ε N 0 as N. I did not achieve ε N = O(1/ N) since no Lipschitz condition is assumed on G (so no usual linear inequality for error estimate and then Gronwall s lemma) The performance estimate only uses certain dominated convergence argument

21 Discussion One might think the generalization to infinite horizon is straightforward. Unfortunately this is not the case. We can see rich collective behavior by developing out-of-equilibrium analysis.

22 The model Consider the model Let the utility functional be X i t+1 = G(p, W i t )u i t, t 0. (4.1) J [0, ) = 1 γ E ρ t (Xt i ut) i γ. (4.2) i=0 Proposition Assume X i 0 > 0. Then sup J [0, ) < if and only if (H) ρeg γ (p, W ) < 1. Suppose by luck p is already an equilibrium state (i.e., optimal responses of many agents exactly replicate it). Will use this model to diagnose dynamic properties of the mean field system under self-optimizing behavior.

23 Assume (H). The stationary dynamic programming equation V i (x) = max u i [ v(x u i ) + ρev i (G(p, W )u i ) ]. We try a solution of the form The optimal control is V i (x) = 1 γ Dγ 1 x γ, D > 0. where D > 0 satisfies u i t = D = X i t 1 + ϕ(p)d, (4.3) ϕ(p)d 1 + ϕ(p)d, (4.4) ϕ(p) = [ρeg γ (p, W )] 1 γ 1 > 1 due to (H).

24 The NCE approach gives 1 + ϕ(p)d = ϕ(p), EG(p, W ) = 1 + ϕ(p)d (so = ϕ(p)) (4.5) EX0 i = p(1 + ϕ(p)d) where ϕ(p) = [ρeg γ (p, W )] 1 γ 1. The first equation is due to optimal reponse/control (recall D = ϕ(p)d 1+ϕ(p)D ) The second and third equations are the consistency condition on p The second equation means the mean of the aggregate investment is preserved The third equation means the initial state mean is right

25 question We repeat 1 + ϕ(p)d = ϕ(p), EG(p, W ) = 1 + ϕ(p)d (so = ϕ(p)) (4.6) EX0 i = p(1 + ϕ(p)d) Suppose the above NCE based u i is applied by an infinite population which has its initial mean EX0 i just slightly different from the right one as above. Can the right mean be restored when time goes to infinity? Or at least, this system can tolerate the small error?

26 Definition The pair (p, D) is called a relaxed stationary mean field (RSMF) solution of (4.1)-(4.2) if { 1 + ϕ(p)d = ϕ(p) (4.7) EG(p, W ) = 1 + ϕ(p)d (so = ϕ(p)) holds and satisfies D > 0, p 0 and ρeg γ (p, W ) < 1. Recall ϕ(p) = [ρeg γ (p, W )] 1 γ 1 Here the pair (p, D) is so called since the restriction on the initial mean is removed.

27 We give an existence result for a class of models with multiplicative noise, i.e., G(p, W ) = g(p)w (4.8) for some function g > 0 on [0, ), W 0 and EW = 1. Theorem Assume (i) 0 < ρ < 1, (ii) g is continuous and strictly decreases on [0, ), (iii) ρe(w γ )g(0) 1, and ρe(w γ )g( ) < 1 where g( ) = lim p g(p). Then there exists a unique RSMF solution.

28 The mean field dynamics with RMFS The closed-loop state equation is given by We obtain the relation Xt+1 i = G(u(N) t, W i ϕ(ˆp) t )X i t ut+1 i = G(u(N) t, W i ϕ(ˆp) t ), 1 i N. u i t, t 0. (4.9) The mean of û i t should coincide with p t, and hence we derive the mean field dynamics p t+1 = p teg(p t, W ), t 0, (4.10) ϕ(ˆp) where p 0 = Eu i 0.

29 Stable equilibrium For cases 1)-3), we take g(p) = 1 Case 1). C = 1.2, so p t+1 = ρe(w γ ) C. 1+(C 1)p 3 1.2pt 1+0.2p 3 t =: Q C (p t ), Q C (ˆp) = 0.5 (slope). Case 2). If C = 2, so p t+1 = 2p t, Q 1+pt 3 C (ˆp) = p p Figure : The iteration of Q C (p) and stable equilibria. Left: C = 1.2; if 0 < p 0 < 1, the iteration converges to 1 monotonically. Right: C = 2.

30 Limit cycle Case 3). Take C = 4 (i.e., now more sensitive to the MF) to obtain p t+1 = 4p t 1 + 3pt 3 =: Q C (p t ), Q C (ˆp) = 1.25, which has a limit cycle determined by the two points on the graph p p Figure : The iteration of Q C (p), C = 4. Left: the limit cycle. Right: the convergence to the limit cycle with an initial value p 0 = 0.3.

31 Chaotic mean field dynamics Define g(p) = h(p) ρew for the production function (see (4.8)), where γ EW = 1. (See graph of h below) The sharp decrease near p = 1 models a pollution effect : when the aggregate investment level approaches a certain reference level (here equal to 1), the production becomes very inefficient p

32 Chaotic mean field dynamics p t+1 = p t h(p t ) =: Q(p t ) p p Figure : Initial condition p 0 = 0.4. Left: 8 iterates; right: 100 iterates. See (Li and Yorke, 1975) for definition of chaos. Also see (J. Benhabib ed., Cycles and Chaos in Economic Equil., Princeton Univ. Press, 1992)

33 Discussion The RSMF solution can be used as a classifier of the model. When the RSMF solution detects a stable equilibrium, one may propose an asymptotically constant mean field and do consistent MF approximations even if the initial state mean is not the right one (Huang, CDC 13) p t 0.6 D t t Figure : Infinite horizon mean field approx. with transient behavior.

34 Discussion When oscillatory behavior is diagnosed, is there any hope to develop consistent MF approximations from a certain class of functions? No answer known. This weird situation is similar to the El Farol Bar Problem (initially examined by W.B. Arthur, Amer. Econ. Rev., 1994)

35 Continuous time (CT) modeling A mean field version of the AK model dkt i = (A(k (N) t )kt i µkt i ct)dt i σktdw i t i where (µk i tdt + σk i tdw i t ) models stochastic depreciation. c i t is treated as the control variable Note: The above mean field variant is not a strict CT counterpart of the previous discrete time model

36 The utility functional [ T J i (0, k0, i c( )) = E where we take the HARA utility for z 0. We further take 0 e ρt U(c i t)dt + e ρt S(k i T ) ], U(z) = zγ, 0 < γ < 1. (4.11) γ S(x) = ηx γ for a small η > 0. We need this small η to regularize the control near T (well known trick in portfolio selection literature). γ

37 The limiting problem Approximate A(k (N) t ) by A t := A( k t ) (becoming a deterministic function of t). The dynamics of agent i are now approximated by The cost is rewritten as [ T J i (0, k0, i c i ) = E dk i t = (A t k i t µk i t c i t)dt σk i tdw i t, (4.12) 0 e ρt U(c i t)dt + e ρt S(k i T ) ], which is affected only by c i t once A t, t [0, T ] is fixed.

38 Optimal control of the limiting problem The optimal control law has the explicit form where ĉ i t = k i t η 1 1 γ e T t λ sds + T t e τ t λ t := λ( k t ) = ρ + γµ γa( k t ) 1 γ. λ sds dτ + σ2 γ 2.

39 Replicating k t By averaging we obtain new d k t dk i t = (A( k t )k i t µk i t ĉ i t)dt σk i tdw i t, = (A( k t ) µ For a given k 0, we obtain 1 η 1 1 γ e T t λ( k s )ds + T t e τ t λ( k s )ds dτ k new = Γ( k ), t [0, T ] ) k t new dt The consistent mean field approximation reduces to find a fixed point of Γ.

40 Concluding remarks For the continuous time case, No chaos in this case Limit cycles are still possible A close counterpart of the discrete time model is of interest (Introduce continuous time multiplicative noise?) Mean field effect depending on distribution instead of just mean (empirical first order moment)