HJB equations. Seminar in Stochastic Modelling in Economics and Finance January 10, 2011

Transcription

1 Department of Probability and Mathematical Statistics Faculty of Mathematics and Physics, Charles University in Prague Seminar in Stochastic Modelling in Economics and Finance January 10, 2011

2 Outline

3 1 Outline

4 2 Economic model At each t 0 an agent owns a capital X t by investing in two assets a riskfree bond that pays interest rate r, db t = rb t dt. a risky asset with s (geometric Brownian motion + jumps) ( ) ds t = S(t ) αdt + σdw t + zñ(dt, dz). At each t 0 an agent controls the number of stocks t in his portfolio, and possibly consumption C t. 1

5 3 Admissible strategies Definition A strategy ( t, t 0) is admissible if 1 it is predictable, 2 the portfolio (X t, t 0) is self-financing, i.e. dx t = t ds t + (X t t S t )db t. We denote the set of admissible strategies by A(x) for a given capital X 0 = x. We can substitute for ds t and db t and obtain ( ) dx t = t S t (α r)dt + σdw t + zñ(dt, dz) + rx t dt. 1

6 4 Objective of investment The objective of an agent is to maximize his utility from investments by using admissible strategies t ( t, C t ). His aim is 1 to maximize his consumption over infinite horizon sup ( t,c t) A(x) 0 where β is a discount factor e βt E U(C t )dt, (1.1) 2 to maximize his terminal utility of terminal wealth in a given horizon T sup t A(x) E U(X T ), (1.2) 3 combination of the first two { sup ( t,c t) A(x) E U(X T ) + T 0 e βt E U(C t )dt }. (1.3)

7 5 Lévy process - definition Definition ( ) Let Ω, F, {F t } t 0, P be a filtered probability space. An adapted process L t is called a Lévy process if it is continuous in probability and has stationary, independent increments. Theorem Let L t be a Lévy process. Then L t has the decomposition L t = b + σw t + zñ(t, dz) + zn(t, dz), 0 t <. z 1 z >1 (1.4) where b R, σ 0, (Ñ) N is a (compensated) Poisson random measure with a Lévy measure ν, all adapted to filtration {F t } t 0.

8 6 Lévy process - Itô formula Theorem (Itô formula) Suppose L t R is an Lévy process of the form dl t = bdt + σdw t + 1 zñ(t, dz). Let f C 1,2 (R + R) and define Y t = f (t, L t ). Then Y t is again an Lévy process and dy t = f t (t, L t )dt + f x (t, L t ) (bdt + σdw t ) f xx(t, L t )σ 2 dt (f (t, L t + z) f (t, L t )) Ñ(dt, dz) (f (t, L t + z) f (t, L t ) f x (t, L t )z) ν(dz)dt.

9 7 Lévy process - Generator Definition Suppose f : R 2 R. Then the generator A of process L t (from the previous theorem) is defined as where L s = x. Theorem 1 Af (s, x) = lim t 0+ t E [f (s + t, L s+t) f (s, x)], Suppose f C 1,2 (R + R). Then Af (s, x) exists and Af (s, x) = f t (s, x) + f x (s, x)b f xx(s, x)σ (f (s, x + z) f (s, x) f x (s, x)z) ν(dz).

10 8 Outline

11 State process Y t = Y (u) t is a stochastic process (on filtered probability space) with s dy t = b(y t, u t )dt + σ t (Y t, u t )dw t + γ(y t, u t, z)ñ(dt, dz), Y 0 = y R k where b : R k U R k, σ : R k U R k m, γ : R k U R k R k l are given functions (time homogenous), W is Wiener process (on the given probability space), Ñ compensated Poisson random measure and U R p given set. u(t) = u(t, ω) : R + Ω U R is predictable control process and Y t = Y (u) t jump-diffusion. is a controlled 9

12 10 Performance criterion For a fixed T (possibly T = ) we define [ ] T J (u) (y) = E f (Y t, u t )dt + g(y T ), 0 where f : S U R, g : R k R are given continuous functions, S is called solvency region. Definition Control u is admissible, denote u A if the state process has a unique, strong solution for all x S and [ ] T E f (Y t, u t )dt + g(y T ) <. 0

13 11 Value function Our goal is to find the value function v and an optimal control u A such that v(x) = J (u ) = sup J (u) (x). u A We consider Markov controls u(t) = u(y t ), then Av(y) = + k b i (y, u(y))v xi (y) + i=1 k i=1 R k ( σσ T ) (y, u(y))v ij x i x j (y) i,j=1 [v(y + γ j (y, u(y), z j )) v(y) v(y)γ j (y, u(y), z j )] ν j (dz j )dt.

14 Revision If we start the state process from any t [0, T h] it holds [ ] t+h v(y t ) E f (Y s, u(y s ))ds + v(y (u) t+h ) with equality for u = u. We know that E t v(y (u) t+h ) = v(y t) + t t+h t A (u) v(y s )ds and by substitution into (2.1) we obtain [ t+h ( ) ] 0 E f (Y s, u(y s )) + A (u) v(y s ) ds or in differential t 0 f (y, u(y)) + A (u) v(y), for any u and equality holds for u = u. (2.1) 12

15 13 HJB for optimal control of jump diffusion Lemma (Verification lemma) Let ṽ C 1,2 satisfies the following 1 lim t T ṽ(y t ) = g(y T ) 2 For any u A(x) f (y, u(y)) + A (u) ṽ(y) 0. 3 There is ũ A(y) such that f (y, ũ(y)) + A (ũ) ṽ(y) = 0. Then ũ = u. and ṽ(y) = v(y) = J (u ) (y), for any y S.

16 14 Remarks Verification theorem holds also for random time T however with additional requirements. Example T = inf {t > 0, Y t / S} The Hamilton-Jacobi-Bellman equation provides only sufficient for an optimum, but not necessary, which is provided by Pontryagin Maximum.

17 15 Outline

18 16 Investor s question We refer back to the motivation example. An investor puts his money into risky S t and riskless B t asset. His portfolio X t evolves ( dx t = t S t (α r)dt + σdw t + )+rx zñ(dt, dz) t dt c t X t dt. and he wants to maximize utility from his consumption sup ( t,c t) A(x) 0 1 e βt E U(C t )dt, (3.1) Investor knows that his utility is given by the power utility function, i.e. U(x) = x 1 p, p > 0, p 1, 1 p = log(x), p = 1.

19 17 Change of notation New processes θ t = ts t X t time t, c t = Ct X t is the proportion of capital invested in risky asset at denotes the consumption proportion. s of investor s portfolio: dx t =θ t X t ( (α r)dt + σdw t + + rx t dt c t X t dt. with X (0) = x, θ t F t, c t F t. 1 zñ(dt, dz) ) (3.2)

20 18 Computation of generator We would like to apply the verificatin lemma on the controlled process Y t = (t, X t ) T, with Y 0 = (0, x) T. Generator of v(y t ) A (u) v(y) = v t + ((α r)θ + r c) xv x σ2 θ 2 x 2 v xx + Consumption 1 ( v(t, x + xθz) 1 p v(t, x) θzv x ) ν(dz). f (y, u(y)) = e βt U(cx).

21 19 PDE We guess the form of the value function, v(t, x) = Ke βt x 1 p A (u) v(y) = Ke βt x 1 p [ β + ((α r)θ + r c) (1 p) 1 2 σ2 θ 2 p(1 p) ( + (1 + θz) 1 p 1 θz(1 p) ) ] ν(dz) 1 = Ke βt x 1 p [ β + (r c) (1 p) + h(θ)]. A (u) v(y) + f (y, u(y)) [ ] = Ke βt x 1 p β + (r c)(1 p) + h(θ) + c1 p (3.3). K(1 p)

22 20 We apply the verification theorem. We demand { } A (u) v(y) + f (y, u(y)) = 0. sup u A We differentiate formula (3.3) with respect to c and θ. Optimal proportion 0 = Λ(θ) = (α r) σ 2 θp + Optimal consumption 0 = (1 p) + c p K 1 ( 1 (1 + θz) p ) zν(dz). c = (K(1 p)) 1/p

23 21 Constant K Finally we substitute θ and c into equation (3.3) and demand equality to zero 0 = A (u ) v(y) + f (y, u (y)) 0 = Ke βt x 1 p [ β + r(1 p) + h(θ ) K 1/p (1 p) 1/p+1 + (K(1 p)) 1/p] = Ke βt x 1 p [ β + r(1 p) + h(θ ) p (K(1 p)) 1/p] nontrivial solution is K = 1 1 p [β r(1 p) h(θ )p] p.

24 22 Theorem (Optimal Proportion and Consumption) Assume the portfolio (3.2) and the objective. Let and Then where θ is the optimal proportion, c = (K(1 p)) 1/p Λ(θ ) = 0 β r(1 p) h(θ ) > 0. v(0, x) = Kx 1 p is the value function, K = 1 1 p [β r(1 p) h(θ )p] p. (3.4)

25 23 Merton proportion and consumption Merton investment proportion Merton consumption proportion c 0 = A(p) = θ 0 = α r pσ 2, β r(1 p) p Let all the no-jump variables be indexed by 0. 1 (α r) 2 1 p 2 σ 2 p.

26 23 Merton proportion and consumption Merton investment proportion Merton consumption proportion c 0 = A(p) = θ 0 = α r pσ 2, β r(1 p) p Let all the no-jump variables be indexed by 0.??? What is the effect of jumps on optimal values? 1 (α r) 2 1 p 2 σ 2 p.

27 24 Optimal (jumps included) proportion and consumption Optimal proportion θ solves the equation Λ(θ ) = (α r) σ 2 θp + Optimal consumption 1 c = (K(1 p)) 1/p for a constant K given by equation (3.4). ( 1 (1 + θz) p ) zν(dz) = 0.

28 25 Function Λ We know that for Λ 0 (θ) solves the Merton and can see that 1 Λ(0) = α r, Λ(θ) is a decreasing function of θ. 2 Function (1 (1 + θz) p ) z is positive for z ( 1/θ, ). We conclude that Λ(θ) < Λ 0 (θ). Corollary θ θ 0, v v 0, c c 0, p > 1, c c 0, 0 < p < 1.

29 26 Merton cont. S x 1 (money units in S t ) the Merton line (ν = 0) (x 1 = θ0 x 2) 1 θ0 risk decreasing jumps risk increasing jumps x 2 (money units in B t ) S

30 27 Approximation of small jumps Let us suppose that the measure ν has light tails (jumps are small in absolute value). We can use the taylor expansion and after the substitution into Λ 1 (1 + θz) p = pzθ + o(z 2 ) θ 1 p α r σ z2 ν(dz), i.e. for smaller jumps we can approximate Lévy process by a Brownian motion with volatility σ z2 ν(dz).

31 28 Investor s question II An investor wants to maximize his utility from the terminal wealth Optimal strategy sup E U(X T ) (3.5) t A(x) It is optimal to put constant proportion θ of his money into the risky asset, same as in the previous.

32 29 Outline

33 Maximum - intro Alternative approach for solving optimal control. In the deterministic case introduced by Russian mathematician Lev Pontryagin. State process X t = X (u) t with s dx t = b(t, X t, u t )dt+σ t (t, X t, u t )dw t + Objective [ ] T J (u) = E f (t, X t, u t )dt + g(x T ), 0 R γ(t, X t, u t, z)ñ(dt, dz). for T < deterministic, f continuous, g concave. We want to find an admissible policy u A such that J (u ) = sup J (u). u A 30

34 31 Hamiltonian We define a function, called Hamiltonian by H : [0, T ] R U R R R R H(t, x, u, p, q, r) = b(t, X t, u t )p + σ t (t, X t, u t )q + γ(t, X t, u t, z)r(t, z)ν(dz), (4.1) R where R is the set of functions r : [0, T ] R R such that the integral (4.1) converges. p, q, r satisfies the corresponding adjoint backward stochastic differential equation dp t = H x (t, x, u, p, q, r)dt + qdw t + r(t, z)ñ(dt, dz), p T = g (X T ). R (4.2)

35 32 Let u, u A and let Xt = X (u ) t, X t = X (u) t be the corresponding state processes. We know that u is optimal if J (u ) J (u), u A, and after substitution [ ] T J (u ) J (u) = E (f (t, Xt, ut ) f (t, X t, u t )) dt + g(xt ) g(x T ) Assumption 0 We assume that the integrals in the following derivation are finite.

36 33 Terminal wealth I Since g is concave E [g(xt ) g(x T )] E [(XT X T ) g (XT )] = E [(XT X T ) p (T )] [ T = E (Xt X t ) dp t T 0 T 0 T 0 0 (σ(t, X t, u t ) σ(t, X t, u t )) q t dt p t d (X t X t ) (γ(t, X t, u t, z) γ(t, X t, u t, z)) r (t, z)ν(dz)dt ].

37 34 Terminal wealth II We substitute into p t and X t and not rewrite m gales with zero expected value [ T = E (Xt X t ) H x (t, Xt, ut, pt, qt, r (t,.))dt T 0 T 0 T 0 0 p t (b(t, X t, u t ) b(t, X t, u t )) dt (σ(t, X t, u t ) σ(t, X t, u t )) q t dt (γ(t, X t, u t, z) γ(t, X t, u t, z)) r (t, z)ν(dz)dt ].

38 35 Consumption By the definition of H [ ] T E (f (t, Xt, ut ) f (t, X t, u t )) dt 0 [ T = E (H(t, Xt, ut, pt, qt, r (t,.)) 0 H(t, X t, u t, p t, q t, r (t,.))) dt T 0 T 0 T 0 p t (b(t, X t, u t ) b(t, X t, u t )) dt (σ(t, X t, u t ) σ(t, X t, u t )) q t dt (γ(t, X t, u t, z) γ(t, X t, u t, z)) r (t, z)ν(dz)dt ].

39 36 Terminal wealth + Consumption [ T J (u ) J (u) E (H(t, Xt, ut, pt, qt, r (t,.)) 0 H(t, X t, u t, p t, q t, r (t,.))) dt T 0 (X t X t ) H x (t, X t, u t, p t, q t, r (t,.))dt and if we find such that J (u ) J (u) 0 (4.3) we know that u is the optimal control. ]

40 37 Theorem Theorem (Sufficient maximum ) Let u A with corresponding solution X = X (u ) and suppose there exists a solution (p t, q t, r (t, z)) of the corresponding adjoint equation. Moreover, suppose that H(t, Xt, ut, pt, qt, r (t,.)) = sup H(t, Xt, u, pt, qt, r (t,.)), t [0, T ], u U and H (x) = max u U H(t, x, u, p t, q t, r (t,.)) (4.4) exists and is a concave function of x, t [0, T ] (Arrow ). Then u is optimal control.

41 38 Remarks to theorem Condition (4.4) is guaranteed by concavity of the function H(t, x, u, p t, q t, r (t,.)) in (x, u), t [0, T ]. To finish the proof, denote and h(t, x, u) = H(t, x, u, p t, q t, r (t,.)) h (t, x) = max h(t, x, u) u A (4.3) holds if 0 h (t, x ) h(t, x, u) (x x)h (t, x ) h (t, x ) h (t, x) (x x)h (t, x ) 0. because h is concave in x for t [0, T ].

42 39 For the we define criterion [ ] T s J (u) (s, x) = E f (t + s, X t, u t )dt + g(x T s ), Theorem 0 v(s, x) = sup J (u) (s, x). u A Assume v C 1,3 and that there exists an optimal control ut and corresponding state process Xt for the maximum. Define p t = v x (t, X t ), q t = σ(t, X t, u t )v xx (t, X t ), r(t, z) = v x (t, X t + γ(t, X t, u t, z)) v x (t, X t ). Then p t, q t, r(t,.) solve the adjoint equation (4.2).

43 40 R. Cont and P. Tankov. Financial modelling with jump processes. Chapman & Hall/CRC Financial Mathematics Series., K. Janeček. Advanced topics in financial mathematics. Study material, MFF UK, B. Øksendal and A. Sulem. Applied stochastic control of jump diffusions. 2nd ed. Universitext. Berlin: Springer., 2007.

44 41... Thank you for attention