Hamilton-Jacobi-Bellman Equation of an Optimal Consumption Problem

Transcription

1 Hamilton-Jacobi-Bellman Equation of an Optimal Consumption Problem Shuenn-Jyi Sheu Institute of Mathematics, Academia Sinica WSAF, CityU HK June 29-July 3, 2009

2 1. Introduction X c,π t is the wealth with the consumption policy (c, π). π t is a trading strategy, c t X c,π t is the rate of consumption. We consider the optimal consumption problem (1.1) sup (c,π) A E[ 0 e ρt1 γ (c tx c,π t ) γ dt], A is a family of admissible strategies. ρ is the discount factor.

3 U(x) = 1 γ (x)γ is the HARA utility with parameter γ, γ < 1, γ 0. Purpose: find an optimal strategy. We consider a model for the market (factor model). Some economic factors affect the returns and volatilities of the stocks. Dynamic programming approach is used. The Hamilton-Jacobi-Bellman (HJB) equation is derived.

4 We study the solution of HJB equation. A general existence result for the solutions of HJB equation will be proved from the existence of a pair of sub/super-solution (of HJB equation). We construct a suitable pair of sub/supersolution. We give the verification theorem. The policy constructed from the solution is shown to be optimal. We show that the solutions have different behaviors for γ > 0 and γ < 0.

5 References H.Hata and S.J.Sheu (2009), Hamilton-Jacobi-Bellman equation for an optimal consumption problem, preprint. H.Hata and S.J.Sheu (2009), An optimal consumption and investment problem with linear Gaussian model, preprint.

6 2. A Brief History There are two kinds of investment problem that we can find many discussions in the literature. Optimal consumption problem discussed in this paper. Optimization problem of expected utility of final wealth, (2.1) sup π Â E[ 1 γ (Xπ T) γ ],

7 We show an interesting relation of these two problems following a recent result of Hata-Sheu. Fleming-Hernandez (2005) is one of few examples discussing such relation. We start a brief review of the studies of these two problems in the literature. In Merton(1969), the following problem is discussed, sup E[ T 0 e ρt U(C(t))dt + B(X(T ), T )]. C(t) is the consumption rate, X(t) is the wealth process.

8 U, B are utility functions. R.C. Merton (1969), Lifetime portfolio selection under uncertainty: the continuous time case, The Review of Economics and Statistics, Vol 51, ( with 1475 citations). In Cox-Huang (1986), a similar problem to Merton (1969) is considered for general complete markets. J. C. Cox and C.F. Huang (1989), Optimal consumption and portfolio policies when the asset prices follow a diffusion process, J. Economic Theory, Vol 49, 33-83(with 690 citations). In Pliska(1986), the following problem is considered for

9 the complete market with general diffusion model, sup π E[U(X π (T ))]. S. Pliska(1986), A stochastic calculus model of continuous trading: optimal portfolios, Math. Operation Research Vol.11, (with 218 citations) The developments in Pliska(1986) and Cox-Huang (1989) start the use of stochastic calculus, martingale representation theorem and duality argument. Using their approach, the optimal solution can be explicitly

10 calculated without solving PDE. This is very interesting and is also a reason for receiving so many citations. Their method is different from the approach in Merton (1969). In Merton (1969), the solution is obtained by solving the HJB (Hamilton-Jacobi-Bellman) equation. HJB is a PDE (partial differential equation). For a simple model, the equation can be solved explicitly. In general, it is difficult to calculate the solution.

11 The idea in Pliska, Cox-Huang can not be applied to incomplete markets. Previous studies have suggested some possible directions for research: Obtain an explicit solution by generalizing the idea of Pliska and Cox-Huang. Study HJB equation. In practice, it is also important to solve the equation numerically, if an analytical solution is not possible.

12 In addition to these, there are also a huge number of possible applications can suggest many interesting problems. The following show some models which have analytical solution, Wachter(2002),Chacko-Viceira(2005),Jun Liu (2007) An initial attempt to use HJB equation in more complicated models is first proposed in Fleming (1995). The idea is to reformulate the investment problem as a stochastic control problem. Then the dynamic programming approach can be used to derive the HJB

13 equation. Solving HJB equation will provide a candidate of optimal investment strategy. This suggests several interesting questions for the solutions of such HJB equations. Study the regularity, growth conditions of the solutions. Obtain suitable estimates for the solutions. This is needed when we want to prove the candidate of optimal portfolio derived from a solution is indeed optimal (verification theorem).

14 To show how this works, Fleming-Sheu(1999) study a simple model different from that of Merton(1969) and provides a detailed analysis. The model can be briefly described as follows There is one stock and one banking account that an investor can trade. The interest rate for the banking account is constant r > 0. The price of stock is given by P t = exp(l t ),

15 dl t = c(µt + α 0 L t )dt + σdw t. It turns out that the model in Fleming-Sheu (1999), after reformulation, becomes a special case of factor model. plays the role of factor. y(t) = L t µt (α 0 µ c ) Using this approach, the risk-sensitive portfolio optimization problem for more general factor models have been considered in a list of papers: Fleming-Sheu (1999a, b), Fleming-Sheu(2002), Kuroda-Nagai(2002), Nagai-Peng(2002), Nagai(2003),

16 Kaise-Sheu(2004), Hata-Sekine(2005), Bilecki-Pliska- Sheu(2005) An useful theorem about the structure of the solutions of HJB equation is given in Kaise-Sheu(2006). The studies mentioned above have interesting applications to the minimization of down-side risk probabilities, (2.2) min P ( log Xπ T T k). There is a duality relation between (2.2) and the risk sensitive portfolio optimization problem (2.1) with a particular risk sensitive parameter γ = γ(k) < 0.

17 The following is a list of papers, Hata-Nagai-Sheu(2009), Hata-Sheu(2008), Hata(2008), Nagai(2008, 2009) The result is interesting because of the following reasons. The problem (2.2) is not a conventional optimization problem and a direct solution is not available. The result shows that we can solve (2.2) using a solution of (2.1). HARA utility appears naturally from (2.2). Although

18 (2.2) seems to not have relation with utilty function at the first look. For this connection, see also works of Pham (2003) on the maximization problem of up-side chance probabilities, max P ( log Xπ T T c). We remark that a recent work of Follmer-Schachermayer (2008) seems to also relate to our study, In this talk, we will discuss another application of the study of the risk sensitive portfolio optimization problem

19 (2.1) to the consumption problem in (1.1). This application seems to not be expected from the results in the literature. We are motivated by the three recent papers, Fleming- Hernandez(2003, 2005), Fleming-Pang(2004). Fleming-Pang (2004) considers a model that the stock price is geometric Brownian motion and interest rate of the banking account is random and is an ergodic 1-d diffusion process. We also use an approach similar to Fleming-Pang (2004).

20 We show that a solution of HJB for (2.1) can be used to construct a supersolution for the HJB of (1.1). Then a nice solution of the HJB for (1.1) can be obtained. From this, (1.1) can be solved. We develop some useful ideas for general factor models with multiple stocks. An attempt to use the duality argument similar to that in Pliska(1986)and Cox-Huang(1989) is proposed in Castaneda-Hernandez (2005)for general factor models. A solution is given in the case of HARA utility. However, the result for general utilities is not satisfactory.

21 References Analytical Solution J.A. Wachter(2002), Portfolio and consumption decisions under mean-reverting ruturns: an exact solution for complete markets, J. Financ. Quant. Anal. 37, Chacko, G. and Viceira, L.M.(2005), Dynamic consumption and portfolio choice with stochastic volatility in incomplete markets, Rev. Financ. Stud. 18, Jun Liu (2007), Portfolio selection in stochastic enviroments, Rev. Financ. Stud. 20, Risk Sensitive Portfolio Optimization

22 Fleming, W.H.(1995),Optimal investment models and risk-sensitive stochastic control. Mathematical Finance (Davis M, et al,ed), Spring-Verlag, Berlin. W.H. Fleming and S. J. Sheu (1999), optimal long term growth rate of expected utility of wealth, Ann. Appl. Probab., Vol 9, W.H. Fleming and S.J. Sheu (1999), Risk sensitive control and an optimal investment model, Math. Finance 10, W.H. Fleming and S.J. Sheu (2002), Risk sensitive control and an optimal investment model II, Ann. Appl. Probab. 12, K. Kuroda and H. Nagai (2002), risk sensitive portfolio optimization

23 on infinite time horizon, Stoch. Stoch. Report, 73, H. Nagai and S. Peng (2002), Risk sensitive portfolio optimization with partial information on infinite time horizon, Ann. Appl. Probab. 12, H. Nagai (2003), Optimal strategies for risk-sensitive portfolio optimization problems for general models, SIAM J. Cont. Optim. 41, H. Kaise and S.J. Sheu (2004), Risk sensitive optimal investment: solutions of the dynamical programming equation. In Mathematics of Finance, Contemp. Math. 351, H. Hata and J. Sekine (2005), Solving long term optimal

24 investment problems with Cox-Ingersol-Ross interest rates, Advance in Mathematical Economics 8, T.R. Bielecki, S. Pliska and S.J. Sheu (2005), Risk-sensitive portfolio management with Cox-Ingersol-Ross interest rates:hjb equation, SIAM J. Cont. Optim. 44, H. Kaise and S. J. Sheu (2006), On the structure of solutions of ergodic type Bellman equations related to risk-sensitive control, Ann. Probab. 34, Down-side Risk probability H. Hata, H. Nagai and S.J. Sheu (2009), Asymptotics of probability minimizing a down-side risk, to appear in Ann. Appl. Probab.

25 H. Hata and S.J. Sheu (2008), Down-side risk probability minimization problem for a multidimensional model with stochastic volatility, preprint. H. Hata (2008), Down-side risk large deviations control problem with Cox-Ingersoll-Ross interest rates, preprint. H. Nagai (2008), Asymptotics of the probability minimizing a down-side risk under partial information, preprint. H. Nagai (2009), Down-side risk minimization as large deviation control, preprint. H. Follmer and W. Schachermayer (2008), Asymptotic arbitrage and large deviations, preprint.

26 Optimal Consumption W.H. Fleming and D. Hernandez-Hernandez (2003), An optimal consumption model with stochastic volatility, Finance Stochastics, 7, W.H. Fleming and T. Pang (2004), An application of stochastic control theory to financial economics, SIAM J. Control Optim., 43, W.H. Fleming and D. Hernandez-Hernandez (2005), The tradeoff between consumption and investment in incomplete markets, Appl. Math. Optim., 52, N. Castaneda, D. Hernandez-Hernandez (2005), Optimal

27 consumption-investment problems in incomplete markets with stochastic coefficients, SIAM J. Cont. Optim., 44,

28 3. Factor Model There are N risky assets and a banking account. S i (t) is the price of i-th asset, i = 1, 2,, N. ds i (t) = S i (t)(µ i (y(t))dt + σ (i) P (y(t)) db(t)), The banking account has interest rate r(y(t)). y(t) = (y 1 (t),, y m (t)) is the factor process dy(t) = b(y(t))dt + σ F (y(t))db(t).

29 The investment strategy is given by π(t) = (π 1 (t), π 2 (t),, π N (t)), π i (t) is the proportion of wealth in i-th asset. N 1 π i (t) i=1 is the proportion of wealth in banking account. The dynamics of the wealth is given by dx π (t) = X π (t)( N i=1 π i (t) ds i(t) S i (t) + (1 N i=1 π i (t)) ds 0(t) S 0 (t) ).

30 Here dx π (t) = dx π (t) = This can be solved, (3.1) X π (t)(( i π i(t)(µ i (y(t))dt + σ (i) P (y(t)) db(t)) +(1 N i=1 π i(t))r(y(t))dt). X π (t)( i π i(t) µ i (y(t)) + r(y(t)))dt +π i (t)σ (i) P (y(t)) db(t)). µ i (y) = µ i (y) r(y). X π (t) = x exp( t 0 π i(s)σ (i) P (y(s)) db(s) 1 t 2 0 σ P (y(s)) π(s) 2 ds + t 0 ( i π i(s) µ i (y(s)) + r(y(s)))ds), σ P (y) is the matrix with columns σ (i) P.

31 When there is a consumption, the dynamics for the wealth process becomes dx π,c (t) = X π,c (t){( i π i(t) µ i (y(t)) + r(y(t)) c(t))dt +π i (t)σ (i) P db(t)}. Here c(t) is the consumption rate at time t. (3.2) X π,c (t) = x exp( t 0 π i(s)σ (i) P (y(s)) db(s) 1 2 t 0 σ P (y(s)) π(s) 2 ds + t 0 ( i π i(s) µ i (y(s)) + r(y(s))) c(s))ds), We assume the following conditions. A1) µ i, r, σ P, σ F are bounded smooth with bounded

32 derivatives of any order. A2) There are constants c 1, c 2 > 0 such c 1 σ P σ P(y) c 2, c 1 σ F σ F(y) c 2, y R n. A3) b has bounded derivatives. There are k 1, k 2 > 0 such that y b(y) k 1 y 2 + k 2, y R n.

33 4. Optimization of Expected Utility of Final Wealth Let T be fixed and 0 t T. Define (4.1) φ(t, y, x) = sup π E[(X π (T )) γ ], 0 < γ < 1, φ(t, y, x) = inf π E[(Xπ (T )) γ ], γ < 0, when X π (t) = x, y(t) = y. Then φ(t, y, x) = x γ exp(w (t, y)).

34 To derive an equation for W (t, y), let X π t = x, we write T (X π (T )) γ = ζt,tx π γ exp( t l(y(t), π(t))dt), l(y, π) = 1 2 γ(1 γ) σ P(y) π 2 + γπ µ(y) + γr(y), ζt,t π = exp(γ T t π i (t)σ (i) P (y(s)) db(s) 1 2 γ2 T 0 σ P(y(s)) π(t) 2 ds). Under some conditions, ζt,t π is a probability density. Then T E[(XT) π γ ] = x γ E P π [exp( t l(y(t), π(t))dt)],

35 dp π dp Ft T = ζt,t. π Under P π, y(t) satisfies the equation, (4.2) dy(s) = (b(y(s))+γσ F (y(s))σ P (y(s)) π(s))ds+ i σ F (y(s))db π (s), B π (s) = B(s) B(t) γ s is a Brownian motion under P π. Therefore, t σ P (y(u)) π(u))du, (4.3) exp(w (t, y)) = sup π T E P π t,y [exp( t l(y(s), π(s))ds)]

36 By a standard argument, the stochastic control problem with criterion (4.3) and the dynamics (4.2) for the controlled process has HJB equation given in the following. dw (t,y) sup π { dt aij (y)d ij W (t, y) aij (y)d i W (t, y)d j W (t, y) +(b(y) + γσ F (y)σ P (y) π)dw (t, y) + l(y, π)} = 0. After simplification, (4.4) dw (t, y) dt aij (y)d ij W (t, y) + b (γ) E (y, W (t, y), DW (t, y)) = 0, where a ij (y) = a ij F (y) = σ Fσ F(y),

37 b (γ) E (y, w, p) = m i=1 b (γ) i (y)p i m ij=1 a ij γ (y)p i p j + V (γ) (y), b (γ) (y) = b(y) + γ 1 γ σ FσPa 1 P µ(y), a γ = a F + γ 1 γ σ FσPa 1 P σ PσF, V (γ) γ (y) = a 1 P µ(y) + γr(y) 2(1 γ) µ(y) a F = σ F σf, a P = σ P σp. By assuming the separation of variables, W (t, y) Λ(T t) + W (y), T,

38 we arrive at the equation, (4.5) 1 2 aij (y)d ij W (y) + b (γ) E (y, W (y), DW (y)) = Λ This is the HJB equation to optimize the long term growth rate, (4.6) Λ = sup π (4.7) Λ = inf π lim sup T lim inf T 1 T log E[(Xπ (T )) γ ], 0 < γ < 1, 1 T log E[(Xπ (T )) γ ], γ < 0, A solution of (4.5) (given by a pair (Λ, W )) gives a

39 candidate of the optimal strategy for the problem (4.6) or (4.7). (4.8) u (y) = 1 1 γ a 1 P (y)( µ(y) + σ P(y)σ F(y) W (y)), (4.9) π (t) = u (y(t)). Theorem 4.1. (Theorem 7 in Kaise-Sheu(2004)). Assume (A1) (A3). Then for any γ < 1, γ 0, there is Λ (γ) such that for Λ Λ (γ), (4.5) has a smooth solution.

40 For Λ = Λ (γ), (4.3) has a unique solution Wγ up to the addition of constants. The following properties hold. (a) For any δ, β > 0, there is a constant c β,δ such that Wγ (y) δ y β + c β,δ, DWγ (y) δ y β + c β,δ. Λ (γ) is the value for the problem (4.6) or (4.7). (c) πt defined by (4.9) is an optimal portfolio.

41 5. Optimal Consumption Problem We consider the consumption problem, V C (x, y) = sup π,c E[ 0 exp( ρt) 1 γ (c(t)xπ,c (t)) γ dt]. For a consumption policy (π, c), X π,c (t) = x exp( t 0 π i(s)σ (i) P (y(s)) db(s) 1 2 t 0 σ P (y(s)) π(s) 2 ds + t 0 ( i π i(s) µ i (y(s)) + r(y(s))) c(s))ds),

42 As above, we have V C (x, y) = 1 γ xγ exp(w C (y)), and W (y) = W C (y) satisfies the equation, (5.1) 1 2 aij (y)d ij W (y) + b (γ) C (y, W (y), DW (y)) = 0, b (γ) C w (y, w, p) = (1 γ) exp( 1 γ ) ρ + m + 1 m 2 ij=1 aij γ (y)p i p j + V (γ) (y). To study (5.1), we introduce an useful concept. i=1 b(γ) i (y)p i

43 Smooth functions W, W are called a pair of sub/supersolution of (5.1) if (a) For all y, we have (b) W W. 1 2 aij (y)d ij W (y) + b (γ) C (y, W (y) + DW (y)) 0, 1 2 aij (y)d ij W (y) + b (γ) C (y, W (y) + DW (y)) 0. We can state our main results.

44 Theorem 5.1. Assume W, W is a pair of sub/supersolution of (5.1). Then there is a solution W of (5.1) such that W W W. Theorem 5.2. Assume (A1) (A3). Then for any γ < 1, γ 0, there is a solution W (γ) C of (5.1) satisfying the following properties.. (a) Let 0 < γ < 1. For any δ, β > 0, there is a constant c β,δ such that W (γ) C (y) δ y β + c β,δ,

45 DW (γ) C (y) δ y β + c β,δ. (b) Let γ < 0. There are c 1 < c 2 depending on γ such that for all y we have c 1 W C (y) c 2, c 1 DW C (y) c 2. (c) c t, π t defined by the following is an optimal portfolio. π (t) = u (y(t)),

46 u (y) = 1 1 γ a 1 P (y)( µ(y) + σ P(y)σF(y) W (γ) C (y)), c (t) = exp( 1 1 γ W (γ) C (y(t))). µ(y) = µ(y) r(y)1

47 6. Proof of Theorem 5.1 There are two steps to prove Theorem 5.1. The first step is to consider the following boundary value problem. For each R > 0 and a smooth function ψ, (6.1) { 1 2 aij D ij W (y) + b (γ) C (y, W (y), DW (y)) = 0, y < R, W (y) = ψ(y), y = R.

48 Smooth functions W, W is called a pair of sub/supersolution of (5.1) if (a) For all y < R, we have 1 2 aij (y)d ij W (y) + b (γ) C (y, W (y) + DW (y)) 0, 1 2 aij (y)d ij W (y) + b (γ) C (y, W (y) + DW (y)) 0. (b) W (y) W (y), y R. (c) W (y) ψ(y) W (y), y = R

49 Theorem 6.1. Assume W, W is a pair of sub/supersolution of (6.1). Then there is a solution W of (6.1) such that W W W. In addition to Theorem 6.1, we also need a comparison theorem. Lemma 6.2. Assume smooth functions W 1, W 2 satisfy the following properties (a) For any y < R, 1 2 aij (y)d ij W 1 (y) + b (γ) C (y, W 1(y) + DW 1 (y)) 0,

50 1 2 aij (y)d ij W 2 (y) + b (γ) C (y, W 2(y) + DW 2 (y)) 0. (b) For any y with y = R, W 1 (y) W 2 (y). Then W 1 (y) W 2 (y), y R. Proof of Theorem 5.1. We shall use Theorem 6.1 and Lemma 6.2 to prove Theorem 5.2. (W, W ) is a pair of sub/supersolution of (5.1). For each R, consider (6.1) with ψ = W.

51 By Theorem 6.1, there is a solution of (6.1), denoted by W R. By Lemma 6.2, W R is unique. We now fix R 0 > 0. R 0 < R 1 < R 2. Compare W R1 (y), W R2 (y), y R 0. Take R = R 2 in (6.1). W R2 satisfies 1 2 aij (y)d ij W R2 (y) + b (γ) C (y, W R 2 (y) + DW R2 (y)) = 0, y R 2

52 1 2 aij (y)d ij W (y) + b (γ) C (y, W (y) + DW (y)) 0, y R 2 and W (y) = W R2 (y), y = R 2. By Lemma 6.2, we have W (y) W R2 (y), y R 2. In particular, W R1 (y) = W (y) W R2 (y), y = R 1. We now take R = R 1 in (6.1).

53 Apply Lemma 6.2 to W 1 = W R1, W 2 = W R2. We have W R1 (y) W R2 (y), y R 1. In particular, for R 0 < R 1 < R 2 we have W R1 (y) W R2 (y), y R 0. y R 0, W R (y) is non decreasing in R > R 0. W R (y), R > R 0 is bounded by W (y). Then W R (y) has limit W (y) as R. W ( ) is a solution of (5.1) satisfying W W W.

54 Lemma 6.2 follows by a simple comparison argument. For the proof of Theorem 6.1, we need the following result giving apriori estimates for the solution of (6.1). Using these apriori estimates and following a continuity argument in the theory of PDE, we can prove the existence of the solution of (6.1) claimed in Theorem 6.1. Theorem 6.3. R > 0, 0 < τ < 1 and ψ is a smooth function. Let W τ be the solution of the equation (6.2) { 1 2 aij (y)d ij W τ (y) + b (τγ) C (y, W τ(y), DW τ (y)) = 0, y < R, W τ,γ (y) = τψ(y), y = R.

55 W is a supersolution of (6.1) (or (6.2) with τ = 1). Then exp(w τ (y)) τ exp(w (y)) + (1 τ)f 0 (y), y R. f 0 satisfies the equation { 1 2 aij (y)d ij f 0 (y) + b(y) Df 0 (y) ρf 0 (y) + 1 = 0, y R, f 0 (y) = 1, y = R f 0 (y) has the expression, f 0 (y) = 1 ρ + (1 1 ρ )E y[exp( ρθ)], θ = inf{t > 0; Y (t) = R}.

56 Moreover, W τ (y) log(max{ ρ 1 γ, 1}) sup { ψ(y) }. y =R For the proof of Theorem 6.3, we consider V τ (x, y) = 1 τγ xτγ exp(w τ (y)).

57 We have the equation, [ 1 2 tr(ad yyv τ ) + b c D y V τ + sup τγ x τγ c>0,π τγ ρv τ +xπ σ P σf D xyv x2 π σ P σp D xxv τ +xd x V τ {r + π (µ r1 c}] = 0. We consider ˆV τ (x, y) = 1 τγ (xτγ exp(w τ (y)) f 0 (y)).

58 We have the equation, (6.3) 1 2 tr(ad ˆV [ yy τ ) + b 1 D y ˆVτ + sup c>0,π τγ (cτγ x τγ 1) ρ ˆV τ +xπ σ P σf D ˆV xy τ x2 π σ P σ ] P D ˆV xx τ +xd x ˆVτ {r + π (µ r1 c} = 0. Theorem 6.3 will follow by applying comparison argument to (6.3).

59 The Proof of Theorem 5.2 From Theorem 5.1, to get a solution of (5.1) we need to construct a pair of sub/supersolution of (5.1). In the following, we consider 0 < γ < 1. For a subsolution W, we take a constant function W = K where K is a large positive constant.

60 For a supersolution W, we take W (y) = W γ (y) + δ(1 + y 2 ) β + C. Here we take small positive δ, β and large C. For such choice, by Theorem 4.1, W is a nonnegative function. By Theorem 5.1, we get a solution W of (5.1) satisfying W W W. We easily deduce a upper bound of W (y) given in Theorem 5.2(a).

61 Using an idea in Kaise-Sheu (2004) and the upper estimate of W, we can get an upper estimate of DW (y) as shown in Theorem 5.2 (a). Using the estimate in Theorem 5.2(a), we can prove the verification theorem stated in Theorem 5.2(c). The analysis for the case γ < 0 will be similar.