Opimal Conrol versus Sochasic Targe problems: An quivalence Resul Bruno Bouchard CRMAD, Universié Paris Dauphine and CRST-NSA bouchard@ceremade.dauphine.fr Ngoc Minh Dang CRMAD, Universié Paris Dauphine and CA Cheuvreux dang@ceremade.dauphine.fr January 17, 2011 Absrac Wihin a general absrac framework, we show ha any opimal conrol problem in sandard form can be ranslaed ino a sochasic arge problem as defined in [17], whenever he underlying filered probabiliy space admis a suiable maringale represenaion propery. This provides a unified way of reaing hese wo classes of sochasic conrol problems. As an illusraion, we show, wihin a jump diffusion framework, how he Hamilon-Jacobi-Bellman equaions associaed o an opimal conrol problem in sandard form can be easily rerieved from he parial differenial equaions associaed o is sochasic arge counerpar. Key words: Sochasic arge, sochasic conrol, viscosiy soluions. Mahemaical subjec classificaions: Primary 49-00; secondary 49L25. 1 Inroducion In heir simples form, sochasic arge problems can be formulaed as follows. Given a conrolled process Z,x,y ν = (X,x, ν Y,x,y), ν associaed o he iniial condiion Z,x,y() ν = (x, y) R d R a ime, find he se S() of iniial condiions (x, y) such ha Ψ(X,x(T ν ), Y,x,y(T ν )) 0 P a.s. for some conrol ν U, where Ψ is a given real valued Borel measurable map and U is a prescribed se of conrols. When y Y ν,x,y(t ) and y Ψ(, y) are non-decreasing, for all ν U, hen he se S() can be idenified o {(x, y) R d R : y v(, x)} where v(, x) := inf{y R : ν U s.. Ψ(Z ν,x,y(t )) 0 P a.s.}, whenever he above infimum is achieved. Such problems can be viewed as a naural generalizaion of he so-called super-hedging problem in mahemaical finance. In his case, Y,x,y ν is inerpreed as he wealh process associaed o a given invesmen policy ν, X,x ν as sock prices or facors (ha can possibly be influenced by he rading sraegy) and Ψ akes he form Ψ(x, y) = y g(x) where g is viewed as he payoff funcion of an uropean opion. Then, Ψ(X,x(T ν ), Y,x,y(T ν )) 0 means Y ν,x,y(t ) g(x,x(t ν )), i.e. he value of he hedging porfolio is greaer a ime T han he payoff g(x,x(t ν )) of he uropean claim. The value funcion v(, x) hen coincides wih he super-hedging price of he opion, see e.g. [13] for references on mahemaical finance. Moivaed by he sudy of super-hedging problems under Gamma consrains, Soner and Touzi [15] were he firs o propose a direc reamen of a paricular class of sochasic arge 1
problems. I relies on a Geomeric Dynamic Programming Principle (GDP) which essenially assers ha S() = {(x, y) R d R : ν U s.. Z,x,y(θ) ν S(θ) a.s.} for all [, T ]-valued sopping ime θ. The main observaion of Soner and Touzi is ha i acually allows one o provide a direc characerizaion of he associaed value funcion v as a viscosiy soluion of a non-linear parabolic parial differenial equaion. This approach was furher exploied in [6] and [18], in he conex of super-hedging problems in mahemaical finance. A general version of he GDP was hen proved in [17], where he auhors also used his mehodology o provide a new probabilisic represenaion of a class of geomeric flows. The link wih PDs in a general Markovian framework for Brownian diffusion processes was esablished in [16], and exended o jump diffusion processes in [1] whose main moivaion was o apply his approach o provision managemen in insurance. Finally, an exension o pah dependen consrains was proposed in [8]. This approach urned ou o be very powerful o sudy a large family of non-sandard sochasic conrol problems in which a arge has o be reached wih probabiliy one a a given ime horizon T. However, i was limied o his case, up o he paper [5] who showed how he a.s. consrain Ψ(Z,x,y(T ν )) 0 can indeed be relaxed in momen consrains of he form [ Ψ(Z,x,y(T ν )) ] p, where he real number p is a given hreshold, ypically non-posiive. This relaxed version was called sochasic arge problem wih conrolled loss in [5]. The resul of [5] (exended by [14] o jump diffusion processes) opened he door o a wide range of new applicaions. In paricular in mahemaical finance, in which a P a.s. consrain would ypically lead o degenerae resuls, i.e. v or v much oo large in comparison o wha can be observed in pracice, while he above relaxaion provides meaningful resuls. A good illusraion of such a siuaion in given in [3] which discusses he pricing of financial book liquidaion conracs. See also he forhcoming paper [9] on he problem of P&L maching in opion hedging or opimal invesmen problems. In view of all he poenial pracical applicaions of he echnology originally proposed by Soner and Touzi in [15], and given he fac ha he heory is now well-esablished, i seems naural o consider his recenly developed class of (non-sandard) sochasic conrol problems as a par of he general well-known ool box in opimal conrol. However, i seems a-priori ha sochasic arge problems and opimal conrol problems in sandard form (i.e. expeced cos minimizaion problems) have o be discussed separaely as hey rely on differen dynamic programming principles. This shor noe can be viewed as a eacher s noe ha explains why hey can acually be reaed in a unified framework. More precisely: any opimal conrol problem in sandard form admis a (simple and naural) represenaion in erms of a sochasic arge problem. In he following, we firs discuss his equivalence resul in a raher general absrac framework. In he nex secion, we show hrough a simple example how he Hamilon-Jacobi-Bellman equaions of an opimal conrol problem in sandard form can be easily recovered from he PDs associaed o he corresponding sochasic arge problem. We will no provide in his shor noe he proof of he PD characerizaion for sochasic arge problems. We refer o [5] and [14] for he complee argumens. Noaions: We denoe by x i he i-h componen of a vecor x R d, which will always be viewed as a column vecor, wih ransposed vecor x, and uclidean norm x. The se M d is he collecion of d-dimensional square marices A wih coordinaes A ij, and norm A defined by viewing A as an elemen of R d d. Given a smooh funcion ϕ : (, x) R + R d R, we denoe by ϕ is derivaive wih respec o is firs variable, we wrie Dϕ and D 2 ϕ for he Jacobian and Hessian marix wih respec o x. The se of coninuous funcion C 0 (B) on a 2
Borel subse B R + R d is endowed wih he opology of uniform convergence on compac ses. Any inequaliy or inclusion involving random variables has o be aken in he a.s. sense. 2 The equivalence resul Le T be a finie ime horizon, given a general probabiliy space (Ω, F, P) endowed wih a filraion F = {F } T saisfying he usual condiions. We assume ha F 0 is rivial. Le us consider an opimal conrol problem defined as follows. Firs, given a se U of deerminisic funcions from R + o R κ, κ 1, we define Ũ = {ν : F-predicable process s.. ν(, ω) U for P-almos every ω Ω}. The conrolled erminal cos is a map ν Ũ Gν L 0 (Ω, F T, P). Wihou loss of generaliy, we can resric o a subse of conrols U Ũ such ha { } U ν Ũ : Gν L 1 (Ω, F T, P). Given (, ν) [0, T ] U, we can hen define he condiional opimal expeced cos: where V ν := ess inf µ U(,ν) [Gµ F ], (2.1) U(, ν) := {µ U : µ = ν on [0, ] P a.s.}. Our main observaion is ha he opimal conrol problem (2.1) can be inerpreed as a sochasic arge problem involving an addiional conrolled process chosen in a suiable family of maringales. Moreover, exisence in one problem is equivalen o exisence in he oher. Lemma 2.1. [Sochasic arge represenaion] Le M be a any family of maringales such ha Then, for each (, ν) [0, T ] U: where G := {G ν, ν U} {M T, M M}. (2.2) V ν = Y ν, (2.3) Y ν := essinf { Y L 1 (Ω, F, P) (M, µ) M U(, ν) s.. Y + M T M G µ}. (2.4) Moreover, here exiss ˆµ U(, ν) such ha [Gˆµ ] = F if and only if here exiss ( M, µ) M U(, ν) such ha V ν In his case, one can choose µ = ˆµ, and M saisfies (2.5) Y ν + M T M G µ. (2.6) Y ν + M T M = G µ. (2.7) 3
Le us make some remarks before o provide he shor proof. Remark 2.2. I is clear ha a family M saisfying (2.2) always exiss. In paricular, one can ake M = M := {( [G ν F ]) 0, ν U}. When he filraion is generaed by a d- dimensional Brownian moion W, hen he maringale represenaion heorem allows one o rewrie any elemen M of M in he form M = P α 0,M0 := M 0 + 0 α s dw s where α belongs o A loc, he se of R d -valued locally square inegrable predicable processes such ha P0,0 α is a maringale. This allows choosing M as {P0,p α, (p, α) R A loc}. When G L 2 (Ω, F T, P), hen we can replace A loc by he se A of R d -valued square inegrable predicable processes. A similar reducion can be obained when he filraion is generaed by Lévy processes. We shall see below in Secion 3 ha such classes of families M allow us o conver a Markovian opimal conrol problem in sandard form ino a Markovian sochasic arge problem for which a PD characerizaion can be derived. A similar idea was already used in [5] o conver sochasic arge problems under conrolled loss, i.e. wih a consrain in expecaion, ino regular sochasic arge problems, i.e. associaed o a P a.s.-consrain, see heir Secion 3. Remark 2.3. A similar represenaion resul was obained in [2] where i is shown ha a cerain class of opimal swiching problems can be ranslaed ino sochasic arge problems for jump diffusion processes. In his paper, he auhor also inroduces an addiional conrolled maringale par bu he idenificaion of he wo conrol problems is made hrough heir associaed PDs (and a suiable comparison heorem) and no by pure probabilisic argumens. Moreover, an addiional randomizaion of he swiching policy is inroduced in order o remove his iniial conrol process. Remark 2.4. I is well-known ha, under mild assumpions (see [11]), he map [0, T ] V ν can be aggregaed by a càdlàd process V ν which is a submaringale for each ν U, and ha a conrol ˆµ is opimal for (2.1) if and only if V ˆµ is a maringale on [, T ]. This is indeed a consequence of he dynamic programming principle which can be (a leas formally) saed as V ν = essinf{ [ V µ θ F ], µ U(, ν)} for all sopping ime θ wih values in [, T ]. I is clear from he proof below ha he addiional conrolled process (Y ν + M s M ) s whose T -value appears in (2.7) hen coincides wih his maringale. Proof of Lemma 2.1 a. We firs prove (2.3). To see ha Y ν V ν, fix Y L 1 (Ω, F, P) and (M, µ) M U(, ν) such ha Y + M T M G µ. Then, by aking] he condiional expecaion on boh sides, we obain Y [G µ F ] essinf{ [G µ F, µ U(, ν)} = V ν, which implies ha Y ν V ν. On he oher hand, (2.2) implies ha, for each µ U(, ν), here exiss M µ M such ha G µ = [G µ F ] + M µ T M µ. In paricular, [Gµ F ] + M µ T M µ [G µ F ] Y ν for all µ U(, ν), and herefore V ν Y ν. G µ. This shows ha b. We now assume ha ˆµ U(, ν) is such ha (2.5) holds. Since V ν = Y ν, his leads o Y ν = [ Gˆµ ] [ ] F. Moreover, (2.2) implies ha here exiss M M such ha Gˆµ F + M T M = Gˆµ. Hence, (2.6) and (2.7) hold wih µ = ˆµ. Conversely, if ( M, µ) M U(, ν) is such ha (2.6) holds, hen he ideniy (2.3) implies ha V ν = Y ν [G µ F ] V ν. Hence, (2.5) holds wih ˆµ = µ. 3 xample In his secion, we show how he Hamilon-Jacobi-Bellman equaions associaed o an opimal conrol problem in sandard form can be deduced from is sochasic arge formulaion. We 4
resric o a classical case where he filraion is generaed by a d-dimensional Brownian moion W and a -marked ineger-valued righ-coninuous poin process N(de, d) wih predicable (P, F)-inensiy kernel m(de)d such ha m() < and supp(m) =, where supp denoes he suppor and is a Borel subse of R d wih Borel ribe. We denoe by Ñ(de, d) = N(de, d) m(de)d he associaed compensaed random measure, see e.g. [10] for deails on random jump measures. The se of conrols U is now defined as he collecion of square inegrable predicable K-valued processes, for some K R d. Given ν U and (, x) [0, T ] R d, we define X,x ν as he unique srong soluion on [, T ] of X ν = x + µ(x ν (r), ν r )dr + σ(x ν (r), ν r )dw r + β(x ν (r ), ν r, e)n(de, dr),(3.1) where (µ, σ, β) : (x, u, e) R d K R d M d R d are measurable and are assumed o be such ha here exiss L > 0 for which µ(x, u) µ(x, u) + σ(x, u) σ(x, u) + ( β(x, u, e) β(x, u, e) 2 m(de) ) 1 2 L x x µ(x, u) + σ(x, u) + esssup β(x, u, e ) L(1 + x + u ), (3.2) e for all x, x R d and u K. Given a coninuous map g wih linear growh (for sake of simpliciy), we hen define he opimal conrol problem in sandard form: v(, x) := inf ν U [ g ( X ν,x(t ) )]. Remark 3.1. Conrary o he above secion, we do no fix here he pah of he conrol ν on [0, ]. This is due o he fac ha X ν,x depends on ν only on (, T ], since he probabiliy of having a jump a ime is equal o 0. Moreover, we can always reduce in he above definiion o conrols ν in U ha are independen of F, see Remark 5.2 in [7]. Then, i follows from sandard argumens, see [12] or [19] and he recen paper [7], ha he lower-semiconinuous envelope v and he upper-semiconinuous envelope v of v defined as v (, x) := saisfy: lim inf (,x ) (,x), <T v(, x ) and v (, x) := lim sup v(, x ), (, x) [0, T ] R d (,x ) (,x), <T Theorem 3.2. Assume ha v is locally bounded. Then, v is a viscosiy supersoluion of ϕ + H (, Dϕ, D 2 ϕ, ϕ) = 0 on [0, T ) R d (ϕ g)1 H (,Dϕ,D 2 ϕ,ϕ)< (T, ) = 0 on R d, (3.3) where H is he upper-semiconinuous envelope of he lower-semiconinuous map wih H : [0, T ] R d R d M d C 0 ([0, T ] R d ) R (, x, q, A, f) sup u K I[f](, x, u) := ( I[f](, x, u) µ(x, u) q 1 2 Tr[σσ (x, u)a] (f(, x + β(x, u, e)) f(, x)) m(de). 5 ),
Moreover, v is a viscosiy subsoluion of ϕ + H(, Dϕ, D 2 ϕ, ϕ) = 0 on [0, T ) R d (ϕ g)(t, ) = 0 on R d. (3.4) On he oher hand, i follows from (3.2) and he linear growh condiion on g ha g ( X,x(T ν ) ) L 2 (Ω, F T, P) for all ν U. The maringale represenaion heorem hen implies ha (2.2) holds for he family of maringales M defined as { } M := R + αs dw s + γ s (e)ñ(de, ds), (α, γ) A Γ 0 0 where A denoes he se of square inegrable R d -valued predicable processes, and Γ is he se of P measurable maps γ : Ω [0, T ] R such ha [ T ] γ s (e) 2 m(de)ds <, 0 wih P defined as he σ-algebra of F-predicable subses of Ω [0, T ]. Hence, we deduce from Lemma 2.1 ha where v(, x) = inf { p R : (ν, α, γ) U A Γ s.. P α,γ,p (T ) g ( X ν,x(t ) )}, P α,γ,p := p + αs dw s + γ s (e)ñ(de, ds). We herefore rerieve a sochasic arge problem in he form sudied in [14]. If v is locally bounded, we can hen apply he main resul of [14] (see [5] for he case of Brownian diffusion models) o deduce ha v is a viscosiy supersoluion of ϕ + F 0,0(, Dϕ, D 2 ϕ, ϕ) = 0 on [0, T ) R d (ϕ g)1 F 0,0 (,Dϕ,D 2 ϕ,ϕ)< (T, ) = 0 on R d, (3.5) where F is he upper-semiconinuous envelope of he map F defined for (ε, η,, x, q, A, f) [0, 1] [ 1, 1] [0, T ] R d R d M d C 0 ([0, T ] R d ) by ( F ε,η (, x, q, A, f) := b(e)m(de) µ(x, u) q 1 ) 2 Tr[σσ (x, u)a], sup (u,a,b) N ε,η(,x,q,f) wih N ε,η (, x, q, f) defined as he se of elemens (u, a, b) K R d L 2 (,, m) such ha a σ(x, u) q ε and b(e) f(, x + β(x, u, e)) + f(, x) η for m-a.e e. Similarly, Theorem 2.1 in [14] saes ha v is a viscosiy subsoluion of ϕ + F 0,0 (, Dϕ, D 2 ϕ, ϕ) = 0 on [0, T ) R d where F is he lower-semiconinuous envelope of F. (ϕ g)(t, ) = 0 on R d, (3.6) To rerieve he resul of Theorem 3.2, i suffices o show he following. 6
Proposiion 3.3. F 0,0 H and F 0,0 H. Proof. Firs noe ha u K implies ha (u, σ(x, u) q, f(, x + β(x, u, )) f(, x) + η) N 0,η (, x, q, f). This implies ha F 0,η H. Since ε 0 F ε, is non-decreasing, i follows ha F 0,0 H. On he oher hand, fix (, x) [0, T ) R d and (q, A, f) R d M d C 0 ([0, T ] R d ), and consider a sequence (ε n, η n, n, x n, q n, A n, f n ) n ha converges o (0, 0,, x, q, A, f) such ha lim n F ε n,η n ( n, x n, q n, A n, f n ) = F 0,0(, x, q, A, f). Then, by definiion of N εn,η n ( n, x n, q n, f n ), we have F 0,0(, x, q, A, f) = lim F ε n n,η n ( n, x n, q n, A n, f n ) ( ( lim η n m() + sup I[f n ]( n, x n, u) µ(x n, u) q n 1 )) n u K 2 Tr[σσ (x n, u)a n ] H (, x, q, A, f). Remark 3.4. I is clear ha he same ideas apply o various classes of opimal conrol problems: singular conrol, opimal sopping, impulse conrol, problem involving sae consrains, ec. We refer o [3] and [8] for he sudy of sochasic arge problems wih conrols including bounded variaion processes or sopping imes. The case of sae consrains is discussed in [8], see also [4]. References [1] B. Bouchard. Sochasic Targe wih Mixed diffusion processes. Sochasic Processes and heir Applicaions, 101, 273-302, 2002. [2] B. Bouchard. A sochasic arge formulaion for opimal swiching problems in finie horizon. Sochasics, 81(2), 171-197, 2009. [3] B. Bouchard and N. M. Dang. Generalized sochasic arge problems for pricing and parial hedging under loss consrains - Applicaion in opimal book liquidaion. Preprin Ceremade, Universiy Paris-Dauphine, 2010. [4] B. Bouchard, R. lie, and C. Imber. Opimal Conrol under Sochasic Targe Consrains. SIAM Journal on Conrol and Opimizaion, 48(5), 3501-3531, 2010. [5] B. Bouchard, R. lie, and N. Touzi. Sochasic arge problems wih conrolled loss. SIAM Journal on Conrol and Opimizaion, 48(5), 3123 3150, 2009. [6] B. Bouchard and N. Touzi. xplici soluion of he mulivariae super-replicaion problem under ransacion coss. Annals of Applied Probabiliy, 10, 685-708, 2000. [7] B. Bouchard and N. Touzi. Weak Dynamic Programming Principle for Viscosiy Soluions. Preprin Ceremade, Universiy Paris-Dauphine, 2009. 7
[8] B. Bouchard and T. N. Vu. The American version of he geomeric dynamic programming principle, Applicaion o he pricing of american opions under consrains. Applied Mahemaics and Opimizaion, 61(2), 235 265, 2010. [9] B. Bouchard and T. N. Vu. A PD formulaion for P&L Maching Problems. In preparaion. [10] P. Brémaud. Poin Processes and Queues - Maringale Dynamics. Springer-Verlag, New- York, 1981. [11] N. l Karoui. Les aspecs probabilises du conrôle sochasique. cole d é de Probabiliés de Sain Flour IX, Lecure Noes in Mahemaics 876, Springer Verlag, 1979. [12] W. H. Fleming and H. M. Soner. Conrolled Markov Processes and Viscosiy Soluions. Second diion, Springer, 2006. [13] I. Karazas and S.. Shreve. Mehods of Mahemaical Finance. Springer Verlag, 1998. [14] L. Moreau. Sochasic Targe Problems wih Conrolled Loss in jump diffusion models. Preprin Ceremade, Universiy Paris-Dauphine, 2010. [15] H. M. Soner and N. Touzi. Super-replicaion under Gamma consrain. SIAM Journal on Conrol and Opimizaion, 39, 73-96, 2000. [16] H. M. Soner and N. Touzi. Sochasic arge problems, dynamic programming and viscosiy soluions. SIAM Journal on Conrol and Opimizaion, 41, 404 424, 2002. [17] H. M. Soner and N. Touzi. Dynamic programming for sochasic arge problems and geomeric flows. Journal of he uropean Mahemaical Sociey, 4, 201 236, 2002. [18] N. Touzi. Direc characerizaion of he value of super-replicaion under sochasic volailiy and porfolio consrains. Sochasic Processes and heir Applicaions, 88, 305-328, 2000. [19] J. Yong and X. Zhou. Sochasic Conrols: Hamilonian Sysems and HJB quaions. Springer, New York, 1999. 8