Optimal control of stochastic delay equations and time-advanced backward stochastic differential equations

Transcription

1 Dep. of Mah./CMA Universiy of Oslo Pre Mahemaics No 16 ISSN Sepember 21 Opimal conrol of sochasic delay eqaions and ime-advanced backward sochasic differenial eqaions Bern Øksendal 1),2) Agnès Slem 3) Tsheng Zhang 4),1) evised in November 21 MSC (21): 93EXX, 93E2, 6H1, 6H15, 6H2, 6J75, 49J55, 356 Key words: Opimal conrol, sochasic delay eqaions, Lévy processes, maximm principles, Hamilonian, adjoin processes, ime-advanced BSDEs. Absrac We sdy opimal conrol problems for (ime-) delayed sochasic differenial eqaions wih jmps. We esablish sfficien and necessary (Ponryagin ype) maximm principles for an opimal conrol of sch sysems. The associaed adjoin processes are shown o saisfy a (ime-) advanced backward sochasic differenial eqaion (AB- SDE). Several resls on exisence and niqeness of sch ABSDEs are shown. The resls are illsraed by an applicaion o opimal consmpion from a cash flow wih delay. 1 Inrodcion Le B() = B(, ω) be a Brownian moion and Ñ(d, dz) := N(d, dz) ν(dz)d, where ν is he Lévy measre of he jmp measre N(, ), be an independen compensaed Poisson random measre on a filered probabiliy space (Ω, F, {F } T, P ). 1 Cener of Mahemaics for Applicaions (CMA), Universiy of Oslo, Box 153 Blindern, N-316 Oslo, Norway. oksendal@mah.io.no 1 The research leading o hese resls has received fnding from he Eropean esearch Concil nder he Eropean Commniy s Sevenh Framework Programme (FP7/27-213) / EC gran agreemen no [ Norwegian School of Economics and Bsiness Adminisraion (NHH), Helleveien 3, N-545 Bergen, Norway. 3 INIA Paris-ocqencor, Domaine de Volcea, ocqencor, BP 15, Le Chesnay Cedex, 78153, France. agnes.slem@inria.fr 4 School of Mahemaics, Universiy of Mancheser, Oxford oad, Mancheser M139PL, Unied Kingdom. sheng.zhang@mancheser.ac.k 1

2 where We consider a conrolled sochasic delay eqaion of he form dx() = b(, X(), Y (), A(), (), ω)d + σ(, X(), Y (), A(), (), ω)db() + θ(, X(), Y (), A(), (), z, ω)ñ(d, dz) ; [, T (1.1) X() = x () ; [ δ,, (1.2) Y () = X( δ), A() = and δ >, ρ and T > are given consans. Here and δ b :[, T U Ω σ :[, T U Ω θ : [, T U Ω e ρ( r) X(r)dr, (1.3) are given fncions sch ha, for all, b(, x, y, a,, ), σ(, x, y, a,, ) and θ(, x, y, a,, z, ) are F -measrable for all x, y, a, U and z := \{}. The fncion x () is assmed o be coninos, deerminisic. Le E F ; [, T be a given sbfilraion of {F } [,T, represening he informaion available o he conroller who decides he vale of () a ime. For example, we cold have E = F ( c) + for some given c >. Le U be a given se of admissible conrol vales () ; [, T and le A E be a given family of admissible conrol processes ( ), inclded in he se of càdlàg, E-adaped and U-valed processes () ; [, T sch ha (1.1)-(1.2) has a niqe solion X( ) L 2 (λ P ) where λ denoes he Lebesge measre on [, T. The performance fncional is assmed o have he form J() = E f(, X(), Y (), A(), (), ω)d + g(x(t ), ω) ; A E (1.4) where f = f(, x, y, a,, ω) : [, T U Ω and g = g(x, ω) : Ω are given C 1 fncions w.r.. (x, y, a, ) sch ha 2 E[ { f(, X(), A(), ()) + f (, X(), Y (), A(), ()) x i }d + g(x(t )) + g (X(T )) 2 < for x i = x, y, a and. Here, and in he following, we sppress he ω, for noaional simpliciy. The problem we consider in his paper is he following: Find Φ(x ) and A E sch ha Φ(x ) := sp A E J() = J( ). (1.5) 2

3 Any conrol A E saisfying (1.5) is called an opimal conrol. Varians of his problem have been sdied in several papers. Sochasic conrol of delay sysems is a challenging research area, becase delay sysems have, in general, an infiniedimensional nare. Hence, he naral general approach o hem is infinie-dimensional. For his kind of approach in he conex of conrol problems we refer o [1, 7, 8, 9 in he sochasic Brownian case. To he bes of or knowledge, despie he saemen of a resl in [19, his kind of approach was no developed for delay sysems driven by a Lévy noise. Noneheless, in some cases sill very ineresing for he applicaions, i happens ha sysems wih delay can be redced o finie-dimensional sysems, since he informaion we need from heir dynamics can be represened by a finie-dimensional variable evolving in erms of iself. In sch a conex, he crcial poin is o ndersand when his finie dimensional redcion of he problem is possible and/or o find condiions ensring ha. There are some papers dealing wih his sbjec in he sochasic Brownian case: we refer o [1, 6, 12, 13, 15. The paper [3 represens an exension of [13 o he case when he eqaion is driven by a Lévy noise. We also menion he paper [5, where cerain conrol problems of sochasic fncional differenial eqaions are sdied by means of he Girsanov ransformaion. This approach, however, does no work if here is a delay in he noise componens. Or approach in he crren paper is differen from all he above. Noe ha he presence of he erms Y () and A() in (1.1) makes he problem non-markovian and we canno se a (finie dimensional) dynamic programming approach. However, we will show ha i is possible o obain a (Ponryagin ype) maximm principle for he problem. To his end, we define he Hamilonian by H : [, T U Ω H(, x, y, a,, p, q, r( ), ω) = H(, x, y, a,, p, q, r( )) = f(, x, y, a, ) + b(, x, y, a, )p + σ(, x, y, a, )q + θ(, x, y, a,, z)r(z)ν(dz); (1.6) where is he se of fncions r : sch ha he las erm in (1.6) converges. We assme ha b, σ and θ are C 1 fncions wih respec o (x, y, a, ) and ha [ { T 2 2 b E (, X(), Y (), A(), ()) x i + σ (, X(), Y (), A(), ()) x i } 2 + θ (, X(), Y (), A(), (), z) x i ν(dz) d < (1.7) for x i = x, y, a and. 3

4 Associaed o H we define he adjoin processes p(), q(), r(, z) ; [, T, z, by he following backward sochasic differenial eqaion (BSDE): dp() = E[µ() F d + q()db() + r(, z)ñ(d, dz) ; [, T (1.8) p(t ) = g (X(T )), where µ() = H (, X(), Y (), A(), (), p(), q(), r(, )) x H y ( + δ, X( + δ), Y ( + δ), A( + δ), ( + δ), p( + δ), q( + δ), r( + δ, ))χ [,T δ() e ρ ( +δ H a (s, X(s), Y (s), A(s), (s), p(s), q(s), r(s, ))e ρs χ [,T (s)ds ). (1.9) Noe ha his BSDE is anicipaive, or ime-advanced in he sense ha he driver µ() conains fre vales of X(s), (s), p(s), q(s), r(s, ) ; s + δ. In he case when here are no jmps and no inegral erm in (1.9), anicipaive BSDEs (ABSDEs for shor) have been sdied by [18, who prove exisence and niqeness of sch eqaions nder cerain condiions. They also relae a class of linear ABSDEs o a class of linear sochasic delay conrol problems where here is no delay in he noise coefficiens. Ths, in or paper we exend his relaion o general nonlinear conrol problems and general nonlinear ABSDEs by means of he maximm principle, where we hrogho he sdy inclde he possibiliy of delays also in he noise coefficiens, as well as he possibiliy of jmps. 2 A sfficien maximm principle In his secion we esablish a maximm principe of sfficien ype, i.e. we show ha -nder some assmpions- maximizing he Hamilonian leads o an opimal conrol. Theorem 2.1 (Sfficien maximm principle) Le û A E wih corresponding sae processes ˆX(), Ŷ (), Â() and adjoin processes ˆp(), ˆq(), ˆr(, z), assmed o saisfy he AB- SDE (1.8)-(1.9). Sppose he following hold: (i) The fncions x g(x) and are concave, for each [, T, a.s. (x, y, a, ) H(, x, y, a,, ˆp(), ˆq(), ˆr(, )) (2.1) 4

5 (ii) for all A E. ( ) E {ˆp() 2 σ 2 () + θ 2 (, z)ν(dz) ( )} +X 2 () ˆq 2 () + ˆr 2 (, z)ν(dz) d < (2.2) (iii) for all [, T, a.s. [ max E H(, ˆX(), ˆX( δ), Â(), v, ˆp(), ˆq(), ˆr(, )) E v U [ = E H(, ˆX(), ˆX( δ), Â(), û(), ˆp(), ˆq(), ˆr(, )) E Then û() is an opimal conrol for he problem (1.5). (2.3) Proof. Choose A E and consider J() J(û) = I 1 + I 2 (2.4) where I 1 = E {f(, X(), Y (), A(), ()) f(, ˆX(), Ŷ (), Â(), û())}d (2.5) By he definiion of H and concaviy of H we have I 1 = E {H(, X(), Y (), A(), (), ˆp(), ˆq(), ˆr(, )) I 2 = E[g(X(T )) g( ˆX(T )). (2.6) H(, ˆX(), Ŷ (), Â(), û(), ˆp(), ˆq(), ˆr(, )) (b(, X(), Y (), A(), ()) b(, ˆX(), Ŷ (), Â(), û()))ˆp() (σ(, X(), Y (), A(), ()) σ(, ˆX(), Ŷ (), Â(), û()))ˆq() (θ(, X(), Y (), A(), (), z) θ(, ˆX(), Ŷ (), Â(), û(), z))ˆr(, z)ν(dz)}d [ E { Ĥ x ()(X() ˆX()) + Ĥ Ĥ ()(Y () Ŷ ()) + ()(A() Â()) y a + H ()(() û()) (b() ˆb())ˆp() (σ() ˆσ())ˆq() (θ(, z) ˆθ(, z))ˆr(, z)ν(dz)}d, (2.7) 5

6 where we have sed he abbreviaed noaion Since g is concave we have, by (2.2), Ĥ H () = (, ˆX(), Ŷ (), Â(), û(), ˆp(), ˆq(), ˆr(, )), x x b() = b(, X(), Y (), A(), ()), ˆb() = b(, ˆX(), Ŷ (), Â(), û() ec. I 2 E[g ( ˆX(T ))(X(T ) ˆX(T )) = E[ˆp(T )(X(T ) ˆX(T )) = E ˆp()(dX() d ˆX()) + (X() ˆX())dˆp() + (σ() ˆσ())ˆq()d + (θ(, z) ˆθ(, z))ˆr(, z)ν(dz)d = E (b() ˆb())ˆp()d + (X() ˆX())E[µ() F d + (σ() ˆσ())ˆq()d + (θ(, z) ˆθ(, z))ˆr(, z)ν(dz)d. (2.8) Combining (2.4)-(2.8) we ge, sing ha X() = ˆX() = x () for all [ δ,, { H J() J(û) E x ()(X() ˆX()) + H ()(Y () Ŷ ()) y } + Ĥ Ĥ ()(A() Â()) + ()(() û()) + µ()(x() ˆX()) d a [ { } T +δ Ĥ Ĥ = E ( δ) + δ x y ()χ [,T () + µ( δ) (Y () Ŷ ())d Ĥ T Ĥ + ()(A() Â())d + ()(() û())d. (2.9) a Using inegraion by pars and sbsiing r = δ, we ge s Ĥ Ĥ (s)(a(s) Â(s))ds = a a (s) e ρ(s r) (X(r) ˆX(r))drds s δ ( T ) r+δ Ĥ = r a (s)e ρs χ [,T (s)ds e ρr (X(r) ˆX(r))dr ( T +δ ) Ĥ = a (s)e ρs χ [,T (s)ds e ρ( δ) (X( δ) ˆX( δ))d. (2.1) δ δ 6

7 Combining his wih (2.9) and sing (1.9) we obain [ { T +δ Ĥ Ĥ J() J(û) ( δ) + δ x y ()χ [,T () ( ) } Ĥ + δ a (s)e ρs χ [,T (s)ds e ρ( δ) + µ( δ) (Y () Ŷ ())d T Ĥ + ()(() û())d [ T Ĥ = E ()(() û())d [ = E E [ = E E [ Ĥ ()(() û()) E [ Ĥ () E d (() û())d The las ineqaliy holds becase v = û() maximizes E[H(, ˆX(), Ŷ (), Â(), v, ˆp(), ˆq(), ˆr(, ) E for each [, T. This proves ha û is an opimal conrol.. 3 A necessary maximm principle A drawback wih he sfficien maximm principle in Secion 2 is he condiion of concaviy, which does no always hold in he applicaions. In his secion we will prove a resl going in he oher direcion. More precisely, we will prove he eqivalence beween being a direcional criical poin for J() and a criical poin for he condiional Hamilonian. To his end, we need o make he following assmpions: A 1 For all A E and all bonded β A E here exiss ε > sch ha + sβ A E for all s ( ε, ε). A 2 For all [, T and all bonded E -measrable random variables α he conrol process β() defined by β() = αχ [,T () ; [, T (3.1) belongs o A E. A 3 For all bonded β A E he derivaive process exiss and belongs o L 2 (λ P ). ξ() := d ds X+sβ () s= (3.2) 7

8 I follows from (1.1) ha { b b b dξ() = ()ξ() + ()ξ( δ) + x y a () e ρ( r) ξ(r)dr + b } δ ()β() d { σ σ σ + ()ξ() + ()ξ( δ) + x y a () e ρ( r) ξ(r)dr + σ } δ ()β() db() { θ θ + (, z)ξ() + (, z)ξ( δ) x y + θ a () e ρ( r) ξ(r)dr + θ } ()β() Ñ(d, dz) (3.3) δ where we for simpliciy of noaion have p b b () = (, X(), X( δ), A(), ()) ec... x x and we have sed ha d ds Y +sβ () s= = d ds X+sβ ( δ) s= = ξ( δ) (3.4) and d ds A+sβ () s= = d ( ) e ρ( r) X +sβ (r)dr s= ds Noe ha = δ δ e ρ( r) d ds X+sβ (r) s= d = δ e ρ( r) ξ(r)dr. (3.5) ξ() = for [ δ,. (3.6) Theorem 3.1 (Necessary maximm principle) Sppose û A E wih corresponding solions ˆX() of (1.1)-(1.2) and ˆp(), ˆq(), ˆr(, z) of (1.7)-(1.8) and corresponding derivaive process ˆξ() given by (3.2). Assme ha [ { T ( σ ) 2 ( ) 2 σ E ˆp 2 () ()ˆξ 2 () + ()ξ 2 ( δ) x y ( ) 2 ( σ ) 2 ( ) 2 σ + () e ρ( r) ˆξ(r)dr + () a δ { ( ) 2 ( ) 2 θ θ + (, z)ˆξ 2 () + (, z)ˆξ 2 ( δ) x y ( ) 2 ( θ ) 2 ( ) } 2 θ + (, z) e ρ( r) ˆξ(r)dr + (, z)} ν(dz) d a δ { } + ˆξ 2 () ˆq 2 () + ˆr 2 (, z)ν(dz) d <. (3.7) 8

9 Then he following are eqivalen: (i) d ds J(û + sβ) s== for all bonded β A E. (ii) E [ H (, ˆX(), Ŷ (), Â(),, ˆp(), ˆq(), ˆr(, )) E = a.s. for all [, T. =û() Proof. For simpliciy of noaion we wrie û =, ˆX = X, ˆp = p, ˆq = q and ˆr = r in he following. Sppose (i) holds. Then = d ds J( + sβ) s= = d ds E f(, X +sβ (), Y +sβ (), A +sβ (), () + sβ())d + g(x +sβ (T )) s= { f f f = E ()ξ() + ()ξ( δ) + x y a () e ρ( r) ξ(r)d + f } δ ()β() d + g (X(T ))ξ(t ) { } H b σ = E () ()p() x x x ()q() θ (, z)r(, z)ν(dz) ξ()d x { } H b σ + () ()p() y y y ()q() θ (, z)r(, z)ν(dz) ξ( δ)d y { } ( H b σ + () ()p() a a a ()q() θ ) (, z)r(, z)ν(dz) e ρ( r) ξ(r)dr d a δ f + ()β()d + g (X(T ))ξ(t ). (3.8) 9

10 By (3.3) E[g (X(T ))ξ(t ) = E[p(T )ξ(t ) = E p()dξ() + { σ q() x σ + ()ξ() + y { θ θ + r(, z) (, z)ξ() + x y + θ } ()β() ν(dz)d { b = E p() x ()ξ( δ) + σ a () δ ξ()dp() θ (, z)ξ( δ) + (, z) a e ρ( r) ξ(r)dr + σ δ } ()β() d e ρ( r) ξ(r)dr b b ()ξ() + ()ξ( δ) + y a () e ρ( r) ξ(r)dr + b } δ ()β() d ξ()e[µ() F d { σ σ q() ()ξ() + x y { θ θ r(, z) (, z)ξ() + x y } ν(dz)d + θ (, z)β() σ ()ξ( δ) + a () e ρ( r) ξ(r)dr + σ δ θ (, z)ξ( δ) + (, z) a δ } ()β() e ρ( r) ξ(r)dr d (3.9) 1

11 Combining (3.8) and (3.9) we ge { } H T = E ξ() () + µ() d + ξ( δ) H x y ()d ( ) H T + e ρ( r) ξ(r)dr δ a ()d + H ()β()d { H H H = E ξ() () () x x y ( + δ)χ [,T δ() ( +δ )} e ρ H T a (s)e ρs χ [,T (s)ds d + ξ( δ) H y ()d ( s ) H T + e ρ(s ) ξ()d s δ a (s)ds + H ()β()d { = E ξ() H ( +δ y ( + δ)χ [,T δ() e ρ H a (s)e ρs χ [,T (s)ds )} d + ξ( δ) H y ()d ( +δ ) +e ρ H T H a (s)e ρs χ [,T (s)ds ξ()d + ()β()d H = E ()β()d, (3.1) where we again have sed inegraion by pars. If we apply (3.1) o β() = α(ω)χ [s,t () where α(ω) bonded and E -measrable, s, we ge H E ()d α =. Differeniaing wih respec o s we obain [ H E (s)α =. Since his holds for all s and all α we conclde ha [ H E ( ) E =. s This shows ha (i) (ii). Conversely, since every bonded β A E can be approximaed by linear combinaions of conrols β of he form (3.1), we can prove ha (ii) (i) by reversing he above argmen. 11

12 4 Time-advanced BSDEs wih jmps We now sdy ime-advanced backward sochasic differenial eqaions driven boh by Brownian moion B() and compensaed Poisson random measres Ñ(d, dz). 4.1 Framework Given a posiive consan δ, denoe by D([, δ, ) he space of all càdlàg pahs from [, δ ino. For a pah X( ) : +, X will denoe he fncion defined by X (s) = X( + s) for s [, δ. P H = L 2 (ν). Consider he L 2 spaces V 1 := L 2 ([, δ, ds) and V 2 := L 2 ([, δ H, ds). Le F : + V 1 V 1 H H V 2 Ω be a predicable fncion. Inrodce he following Lipschiz condiion: There exiss a consan C sch ha F (, p 1, p 2, p, q 1, q 2, q, r 1, r 2, r, ω) F (, p 1, p 2, p, q 1, q 2, q, r 1, r 2, r, ω) C( p 1 p 1 + p 2 p 2 + p p V1 + q 1 q 1 + q 2 q 2 + q q V1 + r 1 r 1 H + r 2 r 2 H + r r V2. (4.1) 4.2 Firs exisence and niqeness heorem We firs consider he following ime-advanced backward sochasic differenial eqaion in he nknown F adaped processes (p(), q(), r(, z)): dp() = E[F (, p(), p( + δ)χ [,T δ (), p χ [,T δ (), q(), q( + δ)χ [,T δ (), q χ [,T δ (), r(), r( + δ)χ [,T δ (), r χ [,T δ () ) F d + q()db() + r(, z)ñ(d, dz) ; [, T (4.2) p(t ) = G, (4.3) where G is a given F T -measrable random variable. Noe ha he ime-advanced BSDE (1.8)-(1.9) for he adjoin processes of he Hamilonian is of his form. For his ype of ime-advanced BSDEs we have he following resl: Theorem 4.1 Assme ha E[G 2 < and ha condiion (4.1) is saisfied. Then he BSDE (4.2)-(4.3) has a niqe solion p(), q(), r(, z)) sch ha { } E p 2 () + q 2 () + r 2 (, z)ν(dz) d <. (4.4) Moreover, he solion can be fond by indcively solving a seqence of BSDEs backwards as follows: 12

13 Sep : In he inerval [T δ, T we le p(), q() and r(, z) be defined as he solion of he classical BSDE dp() = F (, p(),,, q(),,, r(, z),, ) d + q()db() + r(, z)ñ(d, dz) ; [T δ, T (4.5) p(t ) = G. (4.6) Sep k ; k 1: If he vales of (p(), q(), r(, z)) have been fond for [T kδ, T (k 1)δ, hen if [T (k + 1)δ, T kδ he vales of p( + δ), p, q( + δ), q, r( + δ, z) and r are known and hence he BSDE dp() = E[F (, p(), p( + δ), p, q(), q( + δ), q, r(), r( + δ), r ) F d + q()db() + r(, z)ñ(d, dz) ; [T (k + 1)δ, T kδ (4.7) p(t kδ) = he vale fond in Sep k 1 (4.8) has a niqe solion in [T (k + 1)δ, T kδ. We proceed like his nil k is sch ha T (k + 1)δ < T kδ and hen we solve he corresponding BSDE on he inerval [, T kδ. Proof. The proof follows direcly from he above indcive procedre. The esimae (4.4) is a conseqence of known esimaes for classical BSDEs. 4.3 Second exisence and niqeness heorem Nex, we consider he following backward sochasic differenial eqaion in he nknown F -adaped processes (p(), q(), r(, x)): dp() = E[F (, p(), p( + δ), p, q(), q( + δ), q, r(), r( + δ), r ) F d + q()db + r(, z)ñ(d, dz), ; [, T (4.9) p() = G(), [T, T + δ. (4.1) where G is a given coninos F -adaped sochasic process. Theorem 4.2 Assme E[sp T T +δ G() 2 < and ha he condiion (4.1) is saisfied. Then he backward sochasic differenial eqaion (4.9) admis a niqe solion (p(), q(), r(, z)) sch ha E[ {p 2 () + q 2 () + r 2 (, z)ν(dz)}d <. Proof. 13

14 Sep 1 Assme F is independen of p 1, p 2 and p. Se q () :=, r (, x) =. For n 1, define (p n (), q n (), r n (, x)) o be he niqe solion o he following backward sochasic differenial eqaion eqaion: dp n () = E[F (, q n 1 (), q n 1 ( + δ), q n 1, r n 1 (, ), r n 1 ( + δ, ), r n 1 ( )) F d + q n ()db + r n (, z)ñ(d, dz), [, T (4.11) p n () = G() [T, T + δ. I is a conseqence of he maringale represenaion heorem ha he above eqaion admis a niqe solion, see, e.g. [22, [17. We exend q n, r n o [, T + δ by seing q n (s) =, r n (s, z) = for T s T + δ. We are going o show ha (p n (), q n (), r n (, x)) forms a Cachy seqence. By Iô s formla, we have = p n+1 (T ) p n (T ) 2 = p n+1 () p n () (p n+1 (s) p n (s))(e[f (s, q n (s), q n (s + δ), q n s, r n (s, ), r n (s + δ, ), r n s ( )) F s E[F (s, q n 1 (s), q n 1 (s + δ), qs n 1, r n 1 (s, ), r n 1 (s + δ, ), rs n 1 ( ))) F s ds + r n+1 (s, z) r n (s, z) 2 ds ν(dz) + q n+1 (s) q n (s) 2 ds (p n+1 (s) p n (s))(q n+1 (s) q n (s))db s { r n+1 (s, z) r n (s, z) 2 + 2(p n+1 (s ) p n (s ))(r n+1 (s, z) r n (s, z))}ñ(ds, dz) (4.12) 14

15 earranging erms, in view of (4.1), we ge E[ p n+1 () p n () 2 + E r n+1 (s, z) r n (s, z) 2 dsν(dz) + E q n+1 (s) q n (s) 2 ds 2E (p n+1 (s) p n (s))(e[f (s, q n (s), q n (s + δ), r n (s, ), r n (s + δ, )) F (s, q n 1 (s), q n 1 (s + δ), r n 1 (s, ), r n 1 (s + δ, )) F s ) ds C ε E p n+1 (s) p n (s) 2 ds + εe q n (s) q n 1 (s) 2 ds s+δ + εe q n (s + δ) q n 1 (s + δ) 2 ds + εe ( q n () q n 1 () 2 d)ds s + εe r n (s) r n 1 (s) 2 Hds ( s+δ ) + εe r n (s + δ) r n 1 (s + δ) 2 Hds + εe r n () r n 1 () 2 Hd ds s (4.13) Noe ha E q n (s + δ) q n 1 (s + δ) 2 ds E q n (s) q n 1 (s) 2 ds. (4.14) Inerchanging he order of inegraion, ( s+δ E s δe ) +δ q n () q n 1 () 2 d ds = E q n () q n 1 () 2 d( ds δ q n (s) q n 1 (s) 2 ds. (4.15) Similar ineqaliies hold also for r n r n 1. I follows from (4.13) ha E[ p n+1 () p n () 2 + E r n+1 (s, z) r n (s, z) 2 dsν(dz) + E q n+1 (s) q n (s) 2 ds C ε E p n+1 (s) p n (s) 2 ds + (2 + M)εE q n (s) q n 1 (s) 2 ds + 3εE r n (s) r n 1 (s) 2 Hds. (4.16) 15

16 Choose ε > sfficienly small so ha E[ p n+1 () p n () 2 + E r n+1 (s, z) r n (s, z) 2 ds ν(dz) + E q n+1 (s) q n (s) 2 ds C ε E p n+1 (s) p n (s) 2 ds + 12 E q n (s) q n 1 (s) 2 ds E r n (s) r n 1 (s) 2 Hds. (4.17) This implies ha d ( ) e Cε E p n+1 (s) p n (s) 2 ds d + e Cε E r n+1 (s, z) r n (s, z) 2 ds ν(dz) + e Cε E q n+1 (s) q n (s) 2 ds 1 2 ecε E q n (s) q n 1 (s) 2 ds ecε E r n (s) r n 1 (s) 2 Hds. (4.18) Inegraing he las ineqaliy we ge E p n+1 (s) p n (s) 2 ds + d e Cε E q n+1 (s) q n (s) 2 ds + d e Cε E r n+1 (s, z) r n (s, z) 2 dsν(dz) 1 d e Cε E q n (s) q n 1 (s) 2 ds + 1 d e Cε E r n (s) r n 1 (s) Hds (4.19) In pariclar, d e Cε E 1 2 d e Cε E This yields d e Cε E ( ) n 1 C 2 r n+1 (s, z) r n (s, z) 2 dsν(dz) + d e Cε E q n (s) q n 1 (s) 2 ds + 1 d e Cε E 2 q n+1 (s) q n (s) 2 ds r n (s) r n 1 (s) 2 Hds (4.2) r n+1 (s, z) r n (s, z) 2 dsν(dz) + d e Cε E q n+1 (s) q n (s) 2 ds 16 (4.21)

17 for some consan C. I follows from (4.19) ha E (4.16) and ((4.19) frher gives E p n+1 (s) p n (s) 2 ds ( ) n 1 C. (4.22) 2 r n+1 (s, z) r n (s, z) 2 dsν(dz) + E q n+1 (s) q n (s) 2 ds ( ) n 1 CnC ε. 2 (4.23) In view of (4.16), (4.19) and (4.2), we conclde ha here exis progressively measrable processes (p(), q(), r(, z)) sch ha lim n E[ pn () p() 2 =, lim n lim n lim n E[ p n () p() 2 d =, E[ q n () q() 2 d =, E[ r n (, z) r(, z) 2 ν(dz)d =. Leing n in (4.11) we see ha (p(), q(), r(, z)) saisfies p() + + E[F (s, q(s), q(s + δ), q s, r(s, ), r(s + δ, ), r s ( )) F s ds q(s)db s + r(s, z)ñ(ds, dz) = g(t ) (4.24) i.e., (p(), q(), r(, z)) is a solion. Uniqeness follows easily from he Io s formla, a similar calclaion of dedcing (4.12) and (4.13), and Gronwall s Lemma. Sep 2. General case. Le p () =. For n 1, define (p n (), q n (), r n (, z)) o be he niqe solion o he following BSDE: dp n () = E[F (, p n 1 (), p n 1 ( + δ), p n 1, q n (), q n ( + δ), q n, r n (, ), r n ( + δ, ), r n ( )) F d + q n ()db + r n (, z)ñ(d, dz), (4.25) p n () = G(); [T, T + δ. The exisence of (p n (), q n (), r n (, z)) is proved in Sep 1. By he same argmens leading 17

18 o (4.16), we dedce ha E[ p n+1 () p n () E E q n+1 (s) q n (s) 2 ds CE r n+1 (s, z) r n (s, z) 2 dsν(dz) p n+1 (s) p n (s) 2 ds + 12 E p n (s) p n 1 (s) 2 ds This implies ha d ( ) e C E p n+1 (s) p n (s) 2 ds 1 d 2 ec E p n (s) p n 1 (s) 2 ds (4.26) (4.27) Inegraing (4.27) from o T we ge E e CT p n+1 (s) p n (s) 2 ds 1 2 de C( ) E p n (s) p n 1 (s) 2 ds de[ p n (s) p n 1 (s) 2 ds. (4.28) Ieraing he above ineqaliy we obain ha E[ p n+1 (s) p n (s) 2 ds ecnt T n n! Using above ineqaliy and a similar argmen as in Sep 1, i can be shown ha (p n (), q n (), r n (, z)) converges o some limi (p(), q(), r(, z)), which is he niqe solion of eqaion (4.9). Theorem 4.3 Assme E [ sp T T +δ G() 2α < for some α > 1 and ha he following condiion hold: F (, p 1, p 2, p, q 1, q 2, q, r 1, r 2, r) F (, p 1, p 2, p, q 1, q 2, q, r 1, r 2, r) C( p 1 p 1 + p 2 p 2 + sp p(s) p(s) + q 1 q 1 + q 2 q 2 + q q V1 s δ + r 1 r 1 H + r 2 r 2 H + r r V2 ). (4.29) Then he BSDE (4.9) admis a niqe solion (p(), q(), r(, z)) sch ha Proof. [ E sp T p() 2α + {q 2 () + r 2 (, z)ν(dz)}d <. 18

19 Sep 1. Assme F is independen of p 1, p 2 and p. In his case condiion (4.29) redces o assmpion (4.1). By he Sep 1 in he proof of Theorem 4.2, here is a niqe solion (p(), q(), r(, z)) o eqaion (4.9). Sep 2. General case. Le p () =. For n 1, define (p n (), q n (), r n (, z)) o be he niqe solion o he following BSDE: dp n () =E[F (, p n 1 (), p n 1 ( + δ), p n 1, q n (), q n ( + δ), q n, r n (, ), r n ( + δ, ), r n ( )) F d + q n ()db + r n (, z)ñ(d, dz), (4.3) p n () = G(), [T, T + δ. By Sep 1, (p n (), q n (), r n (, z)) exiss. We are going o show ha (p n (), q n (), r n (, z)) forms a Cachy seqence. Using Iô s formla, we have p n+1 () p n () 2 + = 2 (p n+1 (s) p n (s)) r n+1 (s, z) r n (s, z) 2 dsν(dz) + q n+1 (s) q n (s) 2 ds [E[F (s, p n (s), p n (s + δ), p n s, q n+1 (s), q n+1 (s + δ), qs n+1, r n+1 (s, ), r n+1 (s + δ, ), rs n+1 ( )) F (s, p n 1 (s), p n 1 (s + δ), p n 1, q n (s), q n (s + δ), qs n, r n (s, ), r n (s + δ, ), rs n ( )) F s ds 2 s (p n+1 (s) p n (s))(q n+1 (s) q n (s))db s [ r n+1 (s, z) r n (s, z) 2 + 2(p n+1 (s ) p n (s ))(r n+1 (s, z) r n (s, z))ñ(ds, dz) (4.31) 19

20 Take condiional expecaion wih respec o F, ake he spremm over he inerval [, T and se he condiion (4.29) o ge sp p n+1 () p n () 2 + sp E q n+1 (s) q n (s) 2 ds F T T + sp E r n+1 (s, z) r n (s, z) 2 dsν(dz) F T C ε sp E p n+1 (s) p n (s) 2 ds F T + C 1 ε sp E T + C 2 ε sp E T + C 3 ε sp E T + C 4 ε sp E T p n (s) p n 1 (s) 2 ds F E[ sp p n (v) p n 1 (v) 2 F s ds F q n+1 (s) q n (s) 2 ds F r n+1 (s, z) r n (s, z) 2 ds ν(dz) F Choosing ε > sch ha C 3 ε < 1 and C 4 ε < 1 i follows from (4.32) ha sp p n+1 () p n () 2 C ε sp T T + (C 1 + C 2 )ε sp E T Noe ha E p n+1 (s) p n (s) 2 ds F and E E p n+1 (s) p n (s) 2 ds F E[ sp p n (v) p n 1 (v) 2 F s ds F E[ sp are righ-coninos maringales on [, T wih erminal random variables p n (s) 2 ds and E[ sp p n (v) p n 1 (v) 2 F s ds. Ths for α > 1, we have (4.32) (4.33) p n (v) p n 1 (v) 2 F s ds F p n+1 (s) [( ) α [( ) α E sp E p n+1 (s) p n (s) 2 ds F c α E p n+1 (s) p n (s) 2 ds T c T,α E sp p n+1 (v) p n (v) 2α ds, (4.34) 2

21 and [( E sp E T c T,α E c T,α E ) α E[ sp p n (v) p n 1 (v) 2 F s ds F E[ sp p n (v) p n 1 (v) 2α F s ds sp p n (v) p n 1 (v) 2α ds, (4.35) (4.33), (4.34) and (4.35) yield ha for α > 1, [ E sp p n+1 () p n () 2α C 1,α E sp T + C 2,α E sp p n (v) p n 1 (v) 2α ds p n+1 (v) p n (v) 2α ds (4.36) P (4.36) implies ha g n () = E sp p n (s) p n 1 (s) 2α s T d d (ec 1,α g n+1 ()) e C 1,α C 2,α g n () (4.37) Inegraing (4.37) from o T we ge g n+1 () c 2,α e C1,α(s ) g n (s)ds C 2,α e C 1,αT Ieraing he above ineqaliy we obain ha E sp p n+1 (s) p n (s) 2α d ecnt T n s T n! g n (s)ds. (4.38) Using above ineqaliy and a similar argmen as in sep 1, we can show ha (p n (), q n (), r n (, z)) converges o some limi (p(), q(), r(, z)), which is he niqe solion of eqaion (4.9). Finally we presen a resl when he coefficien f is independen of z and r. [ Theorem 4.4 Assme E sp G() 2 < and F saisfies T T +δ F (, y 1, y 2, p) F (, ȳ 1, ȳ 2, p) C( y 1 ȳ 1 + y 2 ȳ 2 + sp p(s) p(s) ). (4.39) s δ Then he backward sochasic differenial eqaion (4.9) admis a niqe solion. 21

22 Proof. Le p () =. For n 1, define (p n (), q n (), r n (, z)) o be he niqe solion o he following BSDE: dp n () = E[F (, p n 1 (), p n 1 ( + δ), p n 1 ) F d + q n ()db + r n (, z)ñ(d, dz), (4.4) p n () = G() [T, T + δ. We will show ha (p n (), q n (), r n (, z)) forms a Cachy seqence. Sbracing p n from p n+1 and aking condiional expecaion wih respec o F we ge p n+1 () p n () = E[ (E[F (s, p n (s), p n (s + δ), p n s ) F s E[F (s, p n 1 (s), p n 1 (s + δ), p n 1 s ) F s )ds F (4.41) Take he spremm over he inerval [, T and se he assmpion (4.39) o ge sp p n+1 () p n () 2 C sp T T ( ) 2 E p n (s) p n 1 (s) ds F ( ) 2 + C sp E E[ sp p n (v) p n 1 (v) F s ds F (4.42) T By he Maringale Ineqaliy, we have [ ( ) 2 E sp E p n (s) p n 1 (s) ds F ce T c T E sp p n (v) p n 1 (v) 2 ds [ ( ) 2 p n (s) p n 1 (s) ds, (4.43) and [ ( E sp E T c T E ) 2 E[ sp p n (v) p n 1 (v) F s ds F E[ sp p n (v) p n 1 (v) 2 F s ds, (4.44) Taking expecaion on boh sides of (4.42) gives [ [ E sp p n+1 () p n () 2 C E sp p n (v) p n 1 (v) 2 ds T (4.45) I follows easily from here ha (p n (), q n (), r n (, z)) converges o some limi (p(), q(), r(, z)), which is he niqe solion of eqaion (4.9). 22

23 5 Example 5.1 Opimal consmpion from a cash flow wih delay Le α(), β() and γ(, z) be given bonded adaped processes, α() deerminisic. Assme ha γ 2 (, z)ν(dz) <. Consider a cash flow X () wih a dynamics dx () = X ( δ) [ α()d + β()db() + γ(, z)ñ(d, dz) ; [, T (5.1) X () = x () > ; [ δ,, (5.2) where x () is a given bonded deerminisic fncion. Sppose ha a ime [, T we consme a he rae c(), a càdlàg adaped process. Then he dynamics of he corresponding ne cash flow X() = X c () is dx() = [X( δ)α() c()d + X( δ)β()db() + X( δ) γ(, z)ñ(d, dz) ; [, T (5.3) X() = x () ; [ δ,. (5.4) Le U 1 (, c, ω) : [, T + Ω be a given sochasic iliy fncion saisfying he following condiions U 1 (, c, ω) is F -adaped for each c, lim c c U 1 (, c, ω) is C 1, U 1 (, c, ω) >, c c U 1 (, c, ω) is sricly decreasing c U 1 (, c, ω) = for all, ω [, T Ω. (5.5) c P v (, ω) = U 1 (,, ω) and define c if v v (, ω) ( ) I(, v, ω) = 1 U1 (,, ω) (v) if v < v (, ω) c (5.6) Sppose we wan o find he consmpion rae ĉ() sch ha J(ĉ) = sp{j(c) ; c A} (5.7) where J(c) = E U 1 (, c(), ω)d + kx(t ). 23

24 Here k > is consan and A is he family of all càdlàg, F -adaped processes c() sch ha E[ X(T ) <. In his case he Hamilonian given by (1.6) ges he form H(, x, y, a,, p, q, r( ), ω) = U 1 (, c, ω) + (α()y c)p + yβ()q + y γ(, z)r(z)ν(dz). (5.8) Maximizing H wih respec o c gives he following firs order condiion for an opimal ĉ(): U 1 (, ĉ(), ω) = p(). (5.9) c The ime-advanced BSDE for p(), q(), r(, z) is, by (1.8)-(1.9) { } dp() = E[ α()p( + δ) + β()q( + δ) + γ(, z)r( + δ, z)ν(dz) χ [,T δ () F d + q()db() + r(, z)ñ(d, dz) ; [, T (5.1) p(t ) = k. (5.11) Since k is deerminisic, we can choose q = r = and (5.1)-(5.11) becomes dp() = α()p( + δ)χ [,T δ ()d ; < T (5.12) To solve his we inrodce p() = k for [T δ, T + δ. (5.13) h() := p(t ) ; [ δ, T. Then for [, T, and dh() = dp(t ) = α(t )p(t + δ)d = α(t )p(t ( δ))d = α(t )h( δ)d (5.14) h() = p(t ) = k for [ δ,. (5.15) This deermines h() indcively on each inerval [jδ, (j + 1)δ ; j = 1, 2,..., as follows: If h(s) is known on [(j 1)δ, jδ, hen h() = h(jδ) + We have proved h (s)ds = h(jδ) + jδ α(t s)h(s δ)ds ; j [jδ, (j + 1)δ. (5.16) 24

25 Proposiion 5.1 The opimal consmpion rae ĉ δ () for he problem (5.3)-(5.4), (5.7) is given by ĉ δ () = I(, h δ (T ), ω), (5.17) where h δ ( ) = h( ) is deermined by (5.15)-(5.16). emark 5.2 Assme ha α() = α > for all [, T. Then we see by indcion on (5.16) ha δ 1 < δ 2 h δ1 () > h δ2 () for all (, T and hence, perhaps sprisingly, δ 1 < δ 2 ĉ δ1 () < ĉ δ2 () for all [, T ). Ths he opimal consmpion rae increases if he delay increases. The explanaion for his may be ha he delay pospones he negaive effec on he growh of he cash flow cased by he consmpion. Acknowledgmens. commens. We wan o hank Joscha Diehl and Marin Schweizer for helpfl eferences [1 Chojnowska-Michalik A., epresenaion heorem for general sochasic delay eqaions, Bll. Acad. Polon. Sci. Ser. Sci. Mah. Asronom. Phys., 26, 7, pp , [2 Da Prao G., Zabczyk J., Sochasic Eqaions in Infinie Dimensions, Encyclopedia of Mahemaics and is Applicaions, Cambridge Universiy Press, Cambridge (UK), [3 David D., Opimal conrol of sochasic delayed sysems wih jmps, preprin, 28. [4 Diekmann O., Van Gils S.A., Verdyn Lnel S.M., Walher H.O., Delay Eqaions. Fncional-, Complex-, and Nonlinear Analysis, Springer-Verlag, New York, [5 El Karoi, N., Hamadène, S., BSDEs and risk-sensiive conrol, zero-sm and nonzerosm game problems of sochasic fncional differenial eqaions, Sochasic Processes and heir Applicaions, 17, pp , 23. [6 Elsanosi I., Øksendal B., Slem A., Some solvable sochasic conrol problems wih delay, Sochasics and Sochasics epors, 71, pp 69 89, 2. [7 Federico S., A sochasic conrol problem wih delay arising in a pension fnd model, o appear in Finance and Sochasics,

26 [8 Gozzi F., Marinelli C., Sochasic opimal conrol of delay eqaions arising in adverising models, Da Prao (ed.) e al., Sochasic parial differenial eqaions and applicaions VII - Papers of he 7h meeing, Levico Terme, Ialy, Janary 5 1, 24. Boca aon, FL: Chapman & Hall/CC. Lecre Noes in Pre and Applied Mahemaics, 245, pp , 24. [9 Gozzi F., Marinelli C., Savin S., On conrolled linear diffsions wih delay in a model of opimal adverising nder ncerainy wih memory effecs, Jornal of Opimizaion, Theory and Applicaions, o appear. [1 Kolmanovski, V.B., Shaikhe, L.E. Conrol of Sysems wih Afereffec, American Mahemaical Sociey [11 Ikeda, N., Waanabe, S., Sochasic Differenial Eqaions and Diffsion Processes, Second Ediion, Norh-Holland/Kodansha [12 Larssen B., Dynamic programming in sochasic conrol of sysems wih delay, Sochasics and Sochasics epors, 74, 3-4, pp , 22. [13 Larssen B., isebro N.H., When are HJB-eqaions for conrol problems wih sochasic delay eqaions finie-dimensional?, Sochasic Analysis and Applicaions, 21, 3, pp , 23. [14 Øksendal B., Slem A., A maximm principle for opimal conrol of sochasic sysems wih delay wih applicaions o finance, Opimal Conrol and PDE, Essays in Honor of Alain Bensossan, eds J.L. Menaldi, E. ofman and A. Slem, IOS Press, Amserdam, pp , 21. [15 Øksendal B., Slem A., Applied Sochasic Conrol of Jmp Diffsions. Second Ediion, Springer, 27. [16 Øksendal B., Zhang T., Opimal conrol wih parial informaion for sochasic Volerra eqaions, Inernaional Jornal of Sochasic Analysis (o appear). [17 Pardox E. and Peng S., Adaped solion of a backward sochasic differenial eqaion, Sysems Conrol Leer 14, 55-61, 199. [18 Peng, S., Yang, Z., Anicipaed backward sochasic differenial eqaions, The Annals of Probabiliy, 37,3, pp , 29. [19 Pesza S., Zabczyk J., Sochasic Parial Differenial Eqaions wih Lévy Noise, Encyclopedia of Mahemaics and is Applicaions, Vol. 113, Cambridge Universiy Press, Cambridge (UK), 28. [2 Proer P.E., Sochasic Inegraion and Differenial Eqaions, 2nd ediion, Springer- Verlag, Berlin-Heidelberg-New York,

27 [21 eiss M., iedle M., Van Gaans O., Delay differenial eqaions driven by Lévy processes: saionariy and Feller s properies, Sochasic Processes and heir Applicaions, Vol.116, pp , 26. [22 Si., On solions of backward sochasic differenial eqaions wih jmps and applicaions, Sochasic Processes and heir Applicaions 66, ,