A Quasi-Sure Approach to the Control of Non-Markovian Stochastic Dierential Equations

Transcription

1 A Quasi-Sure Approach o he Conrol of Non-Markovian Sochasic Dierenial Equaions Marcel Nuz Firs version: June 16, This version: March 18, Absrac We sudy sochasic dierenial equaions (SDEs) whose drif and diusion coeciens are pah-dependen and conrolled. We consruc a value process on he canonical pah space, considered simulaneously under a family of singular measures, raher han he usual family of processes indexed by he conrols. This value process is characerized by a second order backward SDE, which can be seen as a non- Markovian analogue of he Hamilon-Jacobi-Bellman parial dierenial equaion. Moreover, our value process yields a generalizaion of he G-expecaion o he conex of SDEs. Keywords Sochasic opimal conrol, non-markovian SDE, second order BSDE, G-expecaion, random G-expecaion, volailiy uncerainy, risk measure AMS 2000 Subjec Classicaions 93E20, 49L20, 60H10, 60G44, 91B30 Acknowledgemens Financial suppor by Swiss Naional Science Foundaion Gran PDFM /1 is graefully acknowledged. The auhor hanks Shige Peng, Mee Soner, Nizar Touzi and Jianfeng Zhang for simulaing discussions and wo anonymous referees for helpful commens. 1 Inroducion We consider a conrolled sochasic dierenial equaion (SDE) of he form X = x + 0 μ(r, X, ν r ) dr + 0 σ(r, X, ν r ) dw r, 0 T, (1.1) where ν is an adaped conrol process, W is a Brownian moion and he Lipschiz-coninuous coeciens μ(r, X, ν r ) and σ(r, X, ν r ) may depend on Dep. of Mahemaics, Columbia Universiy, New York, mnuz@mah.columbia.edu 1

2 he pas rajecory {X s, 0 s r} of he soluion. Denoing by X ν he soluion corresponding o ν, we are ineresed in he sochasic opimal conrol problem V 0 := sup E[ξ(X ν )], (1.2) ν where ξ is a given funcional. The sandard approach o such a non-markovian conrol problem (cf. [9, 10]) is o consider for each conrol ν he associaed value process J ν = ess sup E[ξ(X ν ) F ], (1.3) ν: ν=ν on [0,] where (F ) is he given lraion. The dependence on ν reecs he presence of a forward componen in he opimizaion problem. The siuaion is quie dieren in Markovian opimal conrol (cf. [12]), where one uses a single value funcion which depends on cerain sae variables bu no on a conrol. This is essenial o describe he value funcion by a dierenial equaion, such as he Hamilon-Jacobi-Bellman PDE, which is he main meri of he dynamic programming approach. I is worh noing ha his equaion is always backward in ime. An analogous descripion for (1.3) via backward SDEs (BSDEs, cf. [20]) is available for cerain popular problems such as uiliy maximizaion wih power or exponenial uiliy funcions (e.g., [13, 18]) or drif conrol (e.g., [10]). However, his relies on a very paricular algebraic srucure which allows for a separaion of J ν ino a backward par independen of ν and a forward par depending on ν. In his paper, we consider he problem (1.2) on he canonical space by recasing i as V 0 = sup E P ν [ξ(b)], (1.4) ν where P ν is he disribuion of X ν and B is he canonical process, and we describe is dynamic value by a single value process V = {V (ω)}. Formally, V corresponds o a value funcion in he Markovian sense if we see he whole rajecory of he conrolled sysem as a sae variable. Even hough (1.2) has feaures of coupled forward-backward ype, he value process is dened in a purely backward manner: one may say ha by consrucing V on he whole canonical space, we essenially calculae he value for all possible oucomes of he forward par. An imporan ingredien in he same vein is ha V is dened quasi-surely under he family of muually singular measures {P ν }. Raher han forming a family of processes as in (1.3), he necessary informaion is sored in a single process which is dened on a large par of he probabiliy space; indeed, he process V seen under P ν should be hough of as an analogue of J ν. Clearly, his is a necessary sep o obain a (second order) backward SDE. We remark ha [22] considered he same conrol problem (1.2) and also made a connecion o nonlinear expecaions. However, in [22], he value process was considered only under he given probabiliy measure. 2

3 We rs consider a fairly regular funcional ξ and dene V (ω) as a condiional version of (1.4). Applying and advancing ideas from [28] and [17], regular condiional probabiliy disribuions are used o dene V (ω) for every ω and prove a pahwise dynamic programming principle (Theorem 3.2). In a second sep, we enlarge he class of funcionals ξ o an L 1 -ype space and prove ha he value process admis a (quasi-sure) càdlàg modicaion (Theorem 5.1). We also show ha he value process falls ino he class of sublinear expecaions sudied in [19]. Indeed, if ξ is considered as a random variable on he canonical space, he mapping ξ V can be seen as a generalizaion of he G-expecaion [23, 24], which, by [7], corresponds o he case μ 0 and σ(r, X, ν r ) = ν r, where he SDE (1.1) degeneraes o a sochasic inegral. Moreover, V can be seen as a varian of he random G-expecaion [17]; cf. Remark 6.5. Finally, we characerize V by a second order backward SDE (2BSDE) in he spiri of [19]; cf. Theorem 6.4. The second order is clearly necessary since he Hamilon-Jacobi-Bellman PDE for he Markovian case is fully nonlinear, while ordinary BSDEs correspond o semilinear equaions. 2BSDEs were inroduced in [5], and in [27] for he non-markovian case. We refer o [27] for he precise relaion beween 2BSDEs in he quasi-sure formulaion and fully nonlinear parabolic PDEs. We remark ha our approach is quie dieren from he (backward) sochasic parial dierenial equaions sudied in [21] for a similar conrol problem (mainly for unconrolled volailiy) and in [14, 15, 2, 3, 4] for so-called pahwise sochasic conrol problems. The relaion o he pahdependen PDEs, inroduced very recenly in [25], is ye o be explored. The remainder of he paper is organized as follows. In Secion 2 we deail he conrolled SDE and is condiional versions, dene he value process for he case when ξ is uniformly coninuous and esablish is regulariy. The pahwise dynamic programming principle is proved in Secion 3. In Secion 4 we exend he value process o a more general class of funcionals ξ and sae is quasi-sure represenaion. The càdlàg modicaion is consruced in Secion 5. In he concluding Secion 6, we provide he Hamilon-Jacobi- Bellman 2BSDE and inerpre he value process as a varian of he random G-expecaion. 2 Consrucion of he Value Funcion In his secion, we rs inroduce he seing and noaion. Then, we dene he value funcion V (ω) for a uniformly coninuous reward funcional ξ and examine he regulariy of V. 3

4 2.1 Noaion We x a consan T > 0 and le Ω := C([0, T ];R d ) be he canonical space of coninuous pahs equipped wih he uniform norm ω T := sup 0 s T ω s, where is he Euclidean norm. We denoe by B he canonical process B (ω) = ω, by P 0 he Wiener measure, and by F = {F } 0 T he (raw) lraion generaed by B. Unless oherwise saed, probabilisic noions requiring a lraion (such as adapedness) refer o F. For any probabiliy measure P on Ω and any (, ω) [0, T ] Ω, we can consruc he corresponding regular condiional probabiliy disribuion P ω ; cf. [29, Theorem 1.3.4]. We recall ha P ω is a probabiliy kernel on F F T ; i.e., P ω is a probabiliy measure on (Ω, F T ) for xed ω and ω P ω (A) is F -measurable for each A F T. Moreover, he expecaion under P ω is he condiional expecaion under P : E P ω [ξ] = E P [ξ F ](ω) P -a.s. whenever ξ is F T -measurable and bounded. Finally, P ω he se of pahs ha coincide wih ω up o, is concenraed on P ω { ω Ω : ω = ω on [0, ] } = 1. (2.1) While P ω is no dened uniquely by hese properies, we choose and x one version for each riple (, ω, P ). Le [0, T ]. We denoe by Ω := {ω C([, T ]; R d ) : ω = 0} he shifed canonical space of pahs saring a he origin. For ω Ω, he shifed pah ω Ω is dened by ωr := ω r ω for r T, so ha Ω = {ω : ω Ω}. Moreover, we denoe by P0 he Wiener measure on Ω and by F = {Fr} r T he (raw) lraion generaed by B, which can be idenied wih he canonical process on Ω. Given wo pahs ω and ω, heir concaenaion a is he (coninuous) pah dened by (ω ω) r := ω r 1 [0,) (r) + (ω + ω r)1 [,T ] (r), 0 r T. Given an F T -measurable random variable ξ on Ω and ω Ω, we dene he condiioned random variable ξ,ω on Ω by ξ,ω ( ω) := ξ(ω ω), ω Ω. Noe ha ξ,ω ( ω) = ξ,ω ( ω ); in paricular, ξ,ω can also be seen as a random variable on Ω. Then ω ξ,ω ( ω) is FT -measurable and moreover, ξ,ω depends only on he resricion of ω o [0, ]. We noe ha for an Fprogressively measurable process {X r, r [s, T ]}, he condiioned process {Xr,ω, r [, T ]} is F -progressively measurable. If P is a probabiliy on Ω, he measure P,ω on FT dened by P,ω (A) := P ω (ω A), A F T, where ω A := {ω ω : ω A}, 4

5 is again a probabiliy by (2.1). We hen have E P,ω [ξ,ω ] = E P ω [ξ] = E P [ξ F ](ω) P -a.s. Analogous noaion will be used when ξ is a random variable on Ω s and ω Ω s, where 0 s T. We denoe by Ω s := {ω [s,] : ω Ω s } he resricion of Ω s o [s, ], equipped wih ω [s,] := sup r [s,] ω r. Noe ha Ω s can be idenied wih {ω Ω s : ω r = ω for r [, T ]}. The meaning of Ω is analogous. 2.2 The Conrolled SDE Le U be a nonempy Borel subse of R m for some m N. We consider wo given funcions μ : [0, T ] Ω U R d and σ : [0, T ] Ω U R d d, he drif and diusion coeciens, such ha (, ω) μ(, X(ω), ν (ω)) and (, ω) σ(, X(ω), ν (ω)) are progressively measurable for any coninuous adaped process X and any U-valued progressively measurable process ν. In paricular, μ(, ω, u) and σ(, ω, u) depend only on he pas rajecory {ω r, r [0, ]}, for any u U. Moreover, we assume ha here exiss a consan K > 0 such ha μ(, ω, u) μ(, ω, u) + σ(, ω, u) σ(, ω, u) K ω ω (2.2) for all (, ω, ω, u) [0, T ] Ω Ω U. We denoe by U he se of all U-valued progressively measurable processes ν such ha T 0 μ(r, X, ν r ) dr < and T 0 σ(r, X, ν r ) 2 dr < (2.3) hold pah-by-pah for any coninuous adaped process X. Given ν U, he sochasic dierenial equaion X = x + 0 μ(r, X, ν r ) dr + 0 σ(r, X, ν r ) db r, 0 T under P 0 has a P 0 -a.s. unique srong soluion for any iniial condiion x R d, which we denoe by X(0, x, ν). We shall denoe by he disribuion of X(0, x, ν) on Ω and by P (0, x, ν) := P 0 X(0, x, ν) 1 (2.4) P (0, x, ν) := P 0 ( X(0, x, ν) 0) 1 he disribuion of X(0, x, ν) 0 X(0, x, ν) x; i.e., he soluion which is ranslaed o sar a he origin. Noe ha P (0, x, ν) is concenraed on Ω 0 and can herefore be seen as a probabiliy measure on Ω 0. We shall work under he following nondegeneracy condiion. 5

6 Assumpion 2.1. Throughou his paper, we assume ha F X P 0 F for all X = X(0, x, ν), (2.5) where F X P 0 is he P 0 -augmenaion of he lraion generaed by X and (x, ν) varies over R d U. One can consruc siuaions where Assumpion 2.1 fails. For example, if x = 0, σ 1 and μ(r, X, ν r ) = ν r, hen (2.5) fails for a suiable choice of ν; see, e.g., [11]. The following is a posiive resul which covers many applicaions. Remark 2.2. Le σ be sricly posiive denie and assume ha μ(r, X, ν r ) is a progressively measurable funcional of X and σ(r, X, ν r ). Then (2.5) holds rue. Noe ha he laer assumpion is saised in paricular when μ is unconrolled; i.e., μ(r, ω, u) = μ(r, ω). Proof. Le X = X(0, x, ν). As he quadraic variaion of X, he process σσ (r, X, ν r ) dr is adaped o he lraion generaed by X. In view of our assumpions, i follows ha M := σ(r, X, ν r ) db r = X x μ(r, X, ν ) dr has he same propery. Hence B = σ(r, X, ν r ) 1 dm r is again adaped o he lraion generaed by X. Remark 2.3. For some applicaions, in paricular when he SDE is of geomeric form, requiring (2.5) o hold for all x R d is oo srong. One can insead x he iniial condiion x hroughou he paper, hen i suces o require (2.5) only for ha x. Nex, we inroduce for xed [0, T ] an SDE on [, T ] Ω induced by μ and σ. Of course, he second argumen of μ and σ requires a pah on [0, T ], so ha i is necessary o specify a hisory for he SDE on [0, ]. This role is played by an arbirary pah η Ω. Given η, we dene he condiioned coeciens μ,η : [0, T ] Ω U R d, σ,η : [0, T ] Ω U R d d, μ,η (r, ω, u) := μ(r, η ω, u), σ,η (r, ω, u) := σ(r, η ω, u). (More precisely, hese funcions are dened also when ω is a pah no necessarily saring a he origin, bu clearly heir value a (r, ω, u) depends only 6

7 on ω.) We observe ha he Lipschiz condiion (2.2) is inheried; indeed, μ,η (r, ω, u) μ,η (r, ω, u) + σ,η (r, ω, u) σ,η (r, ω, u) K η ω η ω [,r] = K ω ω [,r] 2K ω ω [,r]. We denoe by U he se of all F -progressively measurable, U-valued processes ν such ha T μ(r, X, ν r ) dr < and T σ(r, X, ν r ) 2 dr < for any coninuous F -adaped process X = {X r, r [0, T ]}. For ν U, he SDE X s = η + s μ,η (r, X, ν r ) dr+ s σ,η (r, X, ν r ) db r, s T under P 0 (2.6) has a unique soluion X(, η, ν) on [, T ]. Similarly as above, we dene P (, η, ν) := P 0 ( X(, η, ν) ) 1 o be he disribuion of X(, η, ν) X(, η, ν) η on Ω. Noe ha his is consisen wih he noaion P (0, x, ν) if x is seen as a consan pah. 2.3 The Value Funcion We can now dene he value funcion for he case when he reward funcional ξ is an elemen of UC b (Ω), he space of bounded uniformly coninuous funcions on Ω. Deniion 2.4. Given [0, T ] and ξ UC b (Ω), we dene he value funcion V (ω) = V (ξ; ω) = sup ν U E P (,ω,ν) [ξ,ω ], (, ω) [0, T ] Ω. (2.7) The funcion ξ is xed hroughou Secions 2 and 3 and hence ofen suppressed in he noaion. In view of he double dependence on ω in (2.7), he measurabiliy of V is no obvious. We have he following regulariy resul. Proposiion 2.5. Le [0, T ] and ξ UC b (Ω). Then V UC b (Ω ) and in paricular V is F -measurable. More precisely, V (ω) V (ω ) ρ( ω ω ) for all ω, ω Ω wih a modulus of coninuiy ρ depending only on ξ, he Lipschiz consan K and he ime horizon T. 7

8 The rs source of regulariy for V is our assumpion ha ξ is uniformly coninuous; he second one is he Lipschiz propery of he SDE. Before saing he proof of he proposiion, we examine he laer aspec in deail. Lemma 2.6. Le ψ UC b (Ω ). There exiss a modulus of coninuiy ρ K,T,ψ, depending only on K, T and he minimal modulus of coninuiy of ψ, such ha E P (,ω,ν) [ψ] E P (,ω,ν) [ψ] ρ K,T,ψ ( ω ω ) for all [0, T ], ν U and ω, ω Ω. Proof. We se E[ ] := E P 0 [ ] o alleviae he noaion. Le ω, ω Ω, se X := X(, ω, ν) and X := X(, ω, ν), and recall ha X = X X = X ω and similarly X = X ω. (i) We begin wih a sandard SDE esimae. Le X = M + A and X = M + A be he semimaringale decomposiions and τ T be a sopping ime such ha M, M, A, A are bounded on [, τ]. Then Iô's formula and he Lipschiz propery (2.2) of σ yield ha E[ M τ M τ 2 ] E τ K 2 E K 2 E σ(r, ω X, ν r ) σ(r, ω τ τ ω X ω X 2 r dr X, νr ) 2 dr ( ω ω + X X [,r] ) 2 dr T 2K 2 T ω ω 2 + 2K 2 E [ X X 2 [,r τ]] dr. Hence, Doob's maximal inequaliy implies ha E [ M M 2 [,τ]] 4E[ M τ M τ 2 ] T 8K 2 T ω ω 2 + 8K 2 E [ X X 2 [,r τ]] dr. Moreover, using he Lipschiz propery (2.2) of μ, we also have ha A s A s s K s μ(r, ω X, ν r ) μ(r, ω ( ω ω + X X [,r] ) dr for all s T and hen Jensen's inequaliy yields ha E [ A A 2 [,τ]] 2K 2 T 2 ω ω 2 + 2K 2 T T X, νr ) dr E [ X X 2 [,r τ]] dr. 8

9 Hence, we have shown ha E [ X X 2 ] T [,τ] C0 ω ω 2 + C 0 E [ X X 2 [,r τ]] dr, where C 0 depends only on K and T, and we conclude by Gronwall's lemma ha E [ X X 2 [,τ]] C ω ω 2, C := C 0 e C 0T. By he coninuiy of heir sample pahs, here exiss a localizing sequence (τ n ) n 1 of sopping imes such ha M, M, A, A are bounded on [, τ n ] for each n. Therefore, monoone convergence and he previous inequaliy yield ha E [ X X 2 [,T ]] C ω ω 2. (2.8) (ii) Le ρ be he minimal (nondecreasing) modulus of coninuiy for ψ, ρ(z) := sup { ψ( ω) ψ( ω ) : ω, ω Ω, ω ω [,T ] z }, and le ρ be he concave hull of ρ. Then ρ is a bounded coninuous funcion saisfying ρ(0) = 0 and ρ ρ. Le P := P (, ω, ν) and P := P (, ω, ν), hen P and P are he disribuions of X and X, respecively; herefore, E P [ψ] E P [ψ] = E[ψ(X ) ψ( X )] E [ ρ ( X X [,T ] )]. (2.9) Moreover, Jensen's inequaliy and (2.8) yield ha E [ ρ ( X X [,T ] )] ρ ( E [ X X [,T ] ]) ρ ( E [ X X 2 ] 1/2 ) [,T ] ρ ( ) C ω ω for every n. In view of (2.9), we have E P [ψ] E P [ψ] ρ( C ω ω ); i.e., he resul holds for ρ K,T,ψ (z) := ρ( Cz). Afer hese preparaions, we can prove he coninuiy of V. Proof of Proposiion 2.5. To disenangle he double dependence on ω in (2.7), we rs consider he funcion (η, ω) E P (,η,ν) [ξ,ω ], (η, ω) Ω Ω. Since ξ UC b (Ω), here exiss a modulus of coninuiy ρ (ξ) for ξ; i.e., ξ(ω) ξ(ω ) ρ (ξ) ( ω ω T ), ω, ω Ω. Therefore, we have for all ω Ω ha ξ,ω ( ω) ξ,ω ( ω) = ξ(ω ω) ξ(ω ω) ρ (ξ) ( ω ω ω ω T ) = ρ (ξ) ( ω ω ). (2.10) 9

10 For xed η Ω, i follows ha E P (,η,ν) [ξ,ω ] E P (,η,ν) [ξ,ω ] ρ (ξ) ( ω ω ). (2.11) Fix ω Ω and le ψ ω := ξ,ω. Then ρ (ξ) yields a modulus of coninuiy for ψ ω ; in paricular, his modulus of coninuiy is uniform in ω. Thus Lemma 2.6 implies ha he mapping η E P (,η,ν) [ψ ω ] admis a modulus of coninuiy ρ T,K,ξ depending only on T, K, ξ. In view of (2.11), we conclude ha E P (,η,ν) [ξ,ω ] E P (,η,ν) [ξ,ω ] ρ (ξ) ( ω ω ) + ρ T,K,ξ ( η η ) for all η, η, ω, ω Ω and in paricular ha E P (,ω,ν) [ξ,ω ] E P (,ω,ν) [ξ,ω ] ρ( ω ω ), ρ := ρ (ξ) + ρ T,K,ξ (2.12) for all ω, ω Ω. Passing o he supremum over ν U, we obain ha V (ω) V (ω ) ρ( ω ω ) for all ω, ω Ω, which was he claim. 3 Pahwise Dynamic Programming In his secion, we provide a pahwise dynamic programming principle which is fundamenal for he subsequen secions. As we are working in he weak formulaion (1.4), he argumens used here are similar o, e.g., [26], while [22] gives a relaed consrucion in he srong formulaion (i.e., working only under P 0 ). We assume in his secion ha he following condiional version of Assumpion 2.1 holds rue; however, we shall see laer (Lemma 4.4) ha his exended assumpion holds auomaically ouside cerain nullses. Assumpion 3.1. Throughou Secion 3, we assume ha for all (, η, ν) [0, T ] Ω U. F X P 0 F for X := X(, η, ν), (3.1) The main resul of his secion is he following dynamic programming principle. We shall also provide more general, quasi-sure versions of his resul laer (he nal form being Theorem 5.2). Theorem 3.2. Le 0 s T, ξ UC b (Ω) and se V r ( ) = V r (ξ; ). Then V s (ω) = sup E P (s,ω,ν)[ (V ) s,ω] for all ω Ω. (3.2) ν U s 10

11 We remark ha in view of Proposiion 2.5, we may see ξ V r (ξ; ) as a mapping UC b (Ω) UC b (Ω) and recas (3.2) as he semigroup propery V s = V s V on UC b (Ω) for all 0 s T. (3.3) Some auxiliary resuls are needed for he proof of Theorem 3.2, which is saed a he end of his secion. We sar wih he (well known) observaion ha condiioning he soluion of an SDE yields he soluion of a suiably condiioned SDE. Lemma 3.3. Le 0 s T, ν U s and ω Ω. If X := X(s, ω, ν), hen X,ω = X (, ω s X(ω), ν,ω ) P 0-a.s. for all ω Ω s. Proof. Le ω Ω s. Using he deniion and he ow propery of X, we have X r = ω s + = X + r s r μ s, ω (u, X, ν u ) du + r μ(u, ω s X, νu ) du + σ s, ω (u, X, ν u ) db s u s r σ(u, ω s X, νu ) db s u P s 0 -a.s. for all r [, T ]. Hence, using ha (P0 s),ω = P0 by he P 0 s -independence of he incremens of B s, X,ω r = X,ω + r μ(u, ω X,ω s, νu,ω ) du+ r σ(u, ω s X,ω, ν,ω u ) db u P 0-a.s. (3.4) Since X is adaped, we have X,ω ( ) = X(ω ) = X(ω) on [s, ] and in paricular ω s X,ω = ω s X(ω) X,ω = η X,ω, for η := ω s X(ω). Therefore, recalling ha X s = ω s, (3.4) can be saed as X,ω r = η + r μ,η (u, X,ω, ν,ω ) du + r σ,η (u, X,ω, ν,ω ) db u P 0-a.s.; i.e., X,ω solves he SDE (2.6) for he parameers (, η, ν,ω ). Now he resul follows by he uniqueness of he soluion o his SDE. Given [0, T ] and ω Ω, we dene P(, ω) = { P (, ω, ν) : ν U }. (3.5) These ses have he following invariance propery. 11

12 Lemma 3.4. Le 0 s T and ω Ω. If P P(s, ω), hen P,ω P(, ω s ω) for P -a.e. ω Ω s. Proof. Since P P(s, ω), we have P = P (s, ω, ν) for some ν U s ; i.e., seing X := X(s, ω, ν), P is he disribuion of X s = s μ(r, ω s X, ν r ) dr + s σ(r, ω s X, ν r ) db s r under P s 0. We se ˆμ r := μ(r, ω s X, ν r ) and ˆσ r := σ(r, ω s X, ν r ) and see he above as he inegral s ˆμ r dr + s ˆσ r dbr s raher han an SDE. As in [26, Lemma 2.2], he nondegeneracy assumpion (3.1) implies he exisence of a progressively measurable ransformaion β ν : Ω s Ω s (depending on s, ω, ν) such ha β ν (X s ) = B s P s 0 -a.s. (3.6) Furhermore, a raher edious calculaion as in he proof of [26, Lemma 4.1] shows ha ( 1 P,ω = P0 ˆμ r,βν(ω) dr + ˆσ r,βν(ω) dbr) for P -a.e. ω Ω s. s s Noe ha, abbreviaing ˇω := β ν (ω), we have ˆμ,βν(ω) r = μ ( r, ω s X,ˇω, ν,ˇω r ) = μ ( r, ω s X(ˇω) X,ˇω, ν,ˇω r and similarly for ˆσ,βν(ω). Hence, we deduce by Lemma 3.3 ha In view of (3.6), we have P,ω = P (, ω s X(ˇω), ν,ˇω ) for P -a.e. ω Ω s. X s (β ν (B s )) = B s P -a.s.; (3.7) i.e., X(ˇω) s = X s (ˇω) = ω for P -a.e. ω Ω s, and we conclude ha P,ω = P (, ω s ω, ν,ˇω ) for P -a.e. ω Ω s. (3.8) In paricular, P,ω P(, ω s ω). Lemma 3.5 (Pasing). Le 0 s T, ω Ω, ν U s and se P := P (s, ω, ν), X := X(s, ω, ν). Le (E i ) 0 i N be a nie F s -measurable pariion of Ω s, ν i U for 1 i N and dene ν U s by ν(ω) := 1 [s,) ν(ω) + 1 [,T ] [ν(ω)1 E 0(X(ω) s ) + Then P := P (s, ω, ν) saises P = P on F s and N i=1 ) ] ν i (ω )1 E i(x(ω) s ). P,ω = P (, ω s ω, ν i ) for P -a.e. ω E i, 1 i N. 12

13 Proof. As ν = ν on [s, ), we have X(s, ω, ν) = X on [s, ] and in paricular P = P on F s. Le 1 i N, we show ha P,ω = P (, ω s ω, ν i ) for P -a.e. ω E i. Recall from (3.8) ha P,ω = P (, ω s ω, ν,β ν(ω) ) for P -a.e. ω Ω s, where β ν is dened as in (3.6). Since boh sides of his equaliy depend only on he resricion of ω o [s, ] and X(s, ω, ν) = X on [s, ], we also have ha P,ω = P (, ω s ω, ν,ˇω ) for P -a.e. ω Ω s, (3.9) where ˇω = β ν (ω) is dened as below (3.6). Noe ha by (3.7), ω E i implies X(ˇω) s E i under P. (More precisely, if A Ω s is a se such ha A E i P -a.s., hen {X(ˇω) s : ω A} E i P -a.s.) In fac, since X is adaped and E i F s, we even have ha X(ˇω ω) s E i for all ω Ω, for P -a.e. ω E i. By he deniion of ν, we conclude ha ν,ˇω ( ω) = ν(ˇω ω) = ν i ((ˇω ω) ) = ν i ( ω), ω Ω, for P -a.e. ω E i. In view of (3.9), his yields he claim. Remark 3.6. In [26] and [17], i was possible o use a pasing of measures as follows: in he noaion of Lemma 3.5, i was possible o specify measures P i on Ω, corresponding o cerain admissible conrols, and use he pasing ˆP (A) := P (A E 0 ) + N i=1 EP [ P i (A,ω )1 E i(ω) ] o obain a measure in Ω s which again corresponded o some admissible conrol and saised ˆP,ω = P i for ˆP -a.e. ω E i, (3.10) which was hen used in he proof of he dynamic programming principle. This is no possible in our SDE-driven seing. Indeed, suppose ha ˆP is of he form P (s, ω, ν) for some ω and ν, hen we see from (3.8) ha, when σ is general, ˆP,ω will depend explicily on ω, which conradics (3.10). Therefore, he subsequen proof uses an argumen where (3.10) holds only a one specic ω E i ; on he res of E i, we conne ourselves o conrolling he error. We can now show he dynamic programming principle. Apar from he dierence remarked above, he basic paern of he proof is he same as in [26, Proposiion 4.7]. Proof of Theorem 3.2. Using he noaion (3.5), our claim (3.2) can be saed as sup E P [ ξ s, ω] = sup E P [ (V ) s, ω] for all ω Ω. (3.11) P P(s, ω) P P(s, ω) 13

14 (i) We rs show he inequaliy in (3.11). Fix ω Ω and P P(s, ω). Lemma 3.4 shows ha P,ω P(, ω s ω) for P -a.e. ω Ω s and hence ha E P,ω [ (ξ s, ω ),ω] = E P,ω [ ξ, ω sω ] sup E P [ ξ, ω ] sω P P(, ω sω) = V ( ω s ω) = V s, ω (ω) for P -a.e. ω Ω s. Since V is measurable by Proposiion 2.5, we can ake P (dω)-expecaions on boh sides o obain ha E P [ ξ s, ω] [ = E P E P [,ω (ξ s, ω ),ω]] E P [ V s, ω ]. We ake he supremum over P P(s, ω) on boh sides and obain he claim. (ii) We now show he inequaliy in (3.11). Fix ω Ω, ν U s and le P = P (s, ω, ν). We x ε > 0 and consruc a counable cover of he sae space as follows. Le ˆω Ω s. By he deniion of V ( ω s ˆω), here exiss ν (ˆω) U such ha P (ˆω) := P (, ω s ˆω, ν (ˆω) ) saises V ( ω s ˆω) E P (ˆω) [ξ, ω s ˆω ] + ε. (3.12) Le B(ε, ˆω) Ω s denoe he open [s,] -ball of radius ε around ˆω. Since (Ω s, [s,] ) is a separable (quasi-)meric space and herefore Lindelöf, here exiss a sequence (ˆω i ) i 1 in Ω s such ha he balls B i := B(ε, ˆω i ) form a cover of Ω s. As an [s,] -open se, each B i is F s -measurable and hence E 1 := B 1, E i+1 := B i+1 (E 1 E i ), i 1 denes a pariion (E i ) i 1 of Ω s. Replacing E i by ( E i {ˆω i } ) {ˆω j : j 1, j = i} if necessary, we may assume ha ˆω i E i for i 1. We se ν i := ν (ˆωi) and P i := P (, ω s ˆω i, ν i ). Nex, we pase he conrols ν i. Fix N N and le A N := E 1 E N, hen {A c N, E1,..., E N } is a nie pariion of Ω s. Le X := X(s, ω, ν), dene ν(ω) := 1 [s,) ν(ω) + 1 [,T ] [ν(ω)1 A c N (X(ω) s ) + N i=1 ] ν i (ω )1 E i(x(ω) s ) and le P := P (s, ω, ν). Then, by Lemma 3.5, we have P = P on F s and P,ω = P (, ω s ω, ν i ) for all ω E i, for some subse E i E i of full measure P. Le us assume for he momen ha ˆω i E i for 1 i N, (3.13) 14

15 hen we may conclude ha P,ˆωi = P i for 1 i N. (3.14) Recall from Proposiion 2.5 ha V admis a modulus of coninuiy ρ (V). Moreover, we obain similarly as in (2.10) ha here exiss a modulus of coninuiy ρ (ξ) such ha ξ, ω sω ξ, ω sω ρ (ξ) ( ω ω [s,] ). Le ω E i Ω s for some 1 i N, hen ω ˆω i [s,] < ε. Togeher wih (3.12) and (3.14), we obain ha V s, ω (ω) V s, ω (ˆω i ) + ρ (V) (ε) Recall from (2.12) ha he mapping E P i [ξ, ω s ˆωi ] + ε + ρ (V) (ε) = E P,ˆωi [ξ, ω s ˆωi ] + ε + ρ (V) (ε). (3.15) ω E P (,ω,ν i) [ξ,ω ] is uniformly coninuous wih a modulus ρ independen of i and N. Since ω E i, i follows ha E P,ˆωi [ξ, ω s ˆωi ] E P,ω [ξ, ω sω ] = E P (, ω s ˆωi,ν i) [ξ, ω s ˆωi ] E P (, ω sω,νi) [ξ, ω sω ] ρ(ε) for P -a.e. ω E i. (3.16) Seing ρ(ε) := ρ(ε) + ε + ρ (V) (ε) and noing ha E P,ω [ξ, ω sω ] = E P,ω [(ξ s, ω ),ω ] = E P [ξ s, ω F s ](ω), he inequaliies (3.15) and (3.16) imply ha V s, ω (ω) E P [ ξ s, ω F s ] (ω) + ρ(ε) (3.17) for P -a.e. (and hus P -a.e.) ω E i. This holds for all 1 i N. As P = P on F s, aking P -expecaions yields E P [V s, ω 1 AN ] E P N [ξ s, ω 1 AN ] + ρ(ε), (3.18) where we wrie P N = P o recall he dependence on N. Since A N Ω s, we have P N (A c N ) = P (Ac N ) 0 as N. In view of E P N [ξ s, ω 1 AN ] = E P N [ξ s, ω ] E P N [ξ s, ω 1 A c N ] E P N [ξ s, ω ] + ξ P N (A c N), 15

16 we conclude from (3.18) ha E P [V s, ω ] lim sup E P N [ξ s, ω ] + ρ(ε) N sup E P [ξ s, ω ] + ρ(ε). P P(s, ω) Since P P(s, ω) was arbirary, leing ε 0 complees he proof of (3.11). I remains o argue ha our assumpion (3.13) does no enail a loss of generaliy. Indeed, assume ha ˆω i / E i for some i. Then here are wo possible cases. The case P (E i ) = 0 is easily seen o be harmless; recall ha he measure P was xed hroughou he proof. In he case P (E i ) > 0, we also have P ( E i ) > 0 and in paricular E i =. Thus we can replace ˆω i by an arbirary elemen of Ei (which can be chosen independenly of N). Using he coninuiy of he value funcion (Proposiion 2.5) and of he reward funcion (2.12), we see ha he above argumens sill apply if we add an addiional modulus of coninuiy in (3.15). 4 Exension of he Value Funcion In his secion, we exend he value funcion ξ V (ξ; ) o an L 1 -ype space of random variables ξ, in he spiri of, e.g., [8]. While he consrucion of V in he previous secion required a precise analysis ω by ω, we can now move owards a more probabilisic presenaion. In paricular, we shall ofen wrie V (ξ) for he random variable ω V (ξ; ω). For reasons explained in Remark 4.2 below, we x from now on an iniial condiion x R d and le P x := {P (0, x, ν) : ν U} be he corresponding se of measures a ime s = 0. Given a random variable ψ on Ω, we wrie ψ x as a shorhand for ψ 0,x ψ(x 0 ). We also wrie V x (ξ) for (V (ξ)) x. Given p [1, ), we dene L p P x o be he space of F T -measurable random variables X saisfying X L p Px := sup P P x X L p (P ) <, where X p L p (P ) := EP [ X p ]. More precisely, we idenify funcions which are equal P x -quasi-surely, so ha L p P x becomes a Banach space. (Two funcions are equal P x -quasi-surely, P x -q.s. for shor, if hey are equal P -a.s. for all P P x.) Furhermore, given [0, T ], L p P x (F ) is dened as he L p -closure of UC b (Ω ) L p P Px x. Since any L p P x -convergen sequence has a P x -q.s. convergen subsequence, any elemen of L p P x (F ) has an F -measurable represenaive. For breviy, we shall ofen wrie L p P x for L p P x (F T ). 16

17 Remark 4.1. The space L p P x can be described as follows. We say ha ξ L p P x is P x -quasi uniformly coninuous if ξ has a represenaive ξ wih he propery ha for all ε > 0 here exiss an open se G Ω such ha P (G) < ε for all P P and such ha he resricion ξ Ω G is uniformly coninuous. Then L p P x consiss of all ξ L p P x such ha ξ is P x -quasi uniformly coninuous and lim n ξ1 { ξ n} L p = 0. Moreover, If P x is weakly relaively compac, hen L p P x Px conains all bounded coninuous funcions on Ω. The proof is he same as in [17, Proposiion 5.2], which, in urn, followed an argumen of [7]. Before exending he value funcion o L 1 P x, le us explain why we are working under a xed iniial condiion x R d. Remark 4.2. There is no fundamenal obsrucion o wriing he heory wihou xing he iniial condiion x; in fac, mos of he resuls would be more elegan if saed using P insead of P x, where P is he se of all disribuions of he form (2.4), wih arbirary iniial condiion. However, he se P is very large and herefore he corresponding space L 1 P is very small, which is undesirable for he domain of our exended value funcion. As an illusraion, consider a random variable of he form ξ(ω) := f(ω 0 ) on Ω, where f : R d R is a measurable funcion. Then ξ L 1 P = sup x R d f(x) ; i.e., ξ is in L 1 P only when f is uniformly bounded. As a second issue in he same vein, i follows from he Arzelà-Ascoli heorem ha he se P is never weakly relaively compac. The laer propery, which is saised by P x for example when μ and σ are bounded, is someimes useful in he conex of quasi-sure analysis. Lemma 4.3. Le p [1, ). The mapping V x V x (ξ) V x (ψ) L p Px As a consequence, V x ξ ψ L p Px on UC b (Ω) is 1-Lipschiz, for all ξ, ψ UC b (Ω). uniquely exends o a Lipschiz-coninuous mapping V x : L p P x (F T ) L p P x (F ). Proof. The argumen is sandard and included only for compleeness. Noe ha ξ ψ p is again in UC b (Ω). The deniion of V x and Jensen's inequaliy imply ha V x (ξ) V x (ψ) p V x ( ξ ψ ) p V x ( ξ ψ p ). Therefore, V x (ξ) V x (ψ) L p Px sup P P x E P [ V x ( ξ ψ p ) ] 1/p = sup P P x E P [ ξ ψ p ] 1/p, where he equaliy is due o (3.2) applied wih s = 0. (For he case s = 0, he addiional Assumpion 3.1 was no used in he previous secion.) Recalling from Proposiion 2.5 ha V x maps UC b (Ω) o UC b (Ω ), i follows ha he exension maps L p P x o L p P x (F ). 17

18 4.1 Quasi-Sure Properies of he Exension In his secion, we provide some auxiliary resuls of echnical naure. The rs one will (quasi-surely) allow us o appeal o he resuls in he previous secion wihou imposing Assumpion 3.1. This is desirable since we would like o end up wih quasi-sure heorems whose saemens do no involve regular condiional probabiliy disribuions. Lemma 4.4. Assumpion 2.1 implies ha Assumpion 3.1 holds for P x - quasi-every η Ω saisfying η 0 = x. For he proof of his lemma, we shall use he following resul. Lemma 4.5. Le Y and Z be coninuous adaped processes, [0, T ] and le P be a probabiliy measure on Ω. Then F Y P F Z implies ha F Y,ω P,ω F Z,ω for P -a.e. ω Ω. Proof. The assumpion implies ha here exiss a progressively measurable ransformaion β : Ω Ω such ha Z = β(y ) P -a.s. For P -a.e. ω Ω, i follows ha Z(ω ) = β(y (ω )) P,ω -a.s., which, in urn, yields he resul. Proof of Lemma 4.4. Le X := X(, η, ν) wih η 0 = x, we have o show ha P 0 FX F whenever η 0 is ouside some P x -polar se. Hence we shall x an arbirary ˆP P x and show ha he resul holds on a se of full measure ˆP. Le ˆP P x, hen ˆP = P (0, x, ˆν) for some ˆν U and ˆP is concenraed on he image of ˆX 0, where ˆX := X(0, x, ˆν). Tha is, recalling ha η 0 = ˆX 0 = x, we may assume ha η = ˆX(ω) for some ω Ω. Le ν( ω) := 1 [0,)ˆν( ω) + 1 [,T ] ν( ω ). (4.1) Then X := X(0, x, ν) saises X = ˆX on [0, ] and hence we may assume ha η = X(ω) on [0, ]. Using Lemma 3.3 and (4.1), we deduce ha X,ω = X (, x X(ω), ν,ω) = X(, η, ν) = X. Since F F X P 0 by Assumpion 2.1, we conclude ha F P 0 F X,ω P 0 = F X P 0 by using Lemma 4.5 wih Z being he canonical process. The nex wo resuls show ha (for μ 0 and σ posiive denie) he mapping ξ V x (ξ) on L 1 P x falls ino he general class of sublinear expecaions considered in [19], whose echniques we shall apply in he subsequen 18

19 secion. More precisely, he wo lemmas below yield he validiy of is main condiion [19, Assumpion 4.1]. The following propery is known as sabiliy under pasing and well known o be imporan in non-markovian conrol. I should no be confused wih he pasing discussed in Remark 3.6, where he considered measures correspond o dieren poins in ime. Lemma 4.6. Le τ be an F-sopping ime and le Λ F τ. Le P, P 1, P 2 P x saisfy P 1 = P 2 = P on F τ. Then P (A) := E P [ P 1 (A F τ )1 Λ + P 2 ] (A F τ )1 Λ c, A FT denes an elemen of P x. Proof. I follows from he deniion of he condiional expecaion ha P is a probabiliy measure which is characerized by he properies { P = P on F τ and P τ(ω),ω (P 1 ) τ(ω),ω for P -a.e. ω Λ, = (P 2 ) τ(ω),ω for P -a.e. ω Λ c (4.2). Le ν, ν 1, ν 2 U be such ha P = P (0, x, ν) and P i = P (0, x, ν i ) for i = 1, 2. Moreover, le X := X(0, x, ν), dene ν U by ν r (ω) := 1 [0,τ(X(ω) 0 ))(r)ν r (ω) [ ] + 1 [τ(x(ω) 0 ),T ](r) νr 1 (ω)1 Λ (X(ω) 0 ) + νr 2 (ω)1 Λ c(x(ω) 0 ) and le P := P (0, x, ν) P x. We show ha P saises he hree properies from (4.2). Indeed, ν = ν on [0, τ(x 0 )) implies ha P = P on F τ. Moreover, as in (3.8), P τ(ω),ω = P ( τ(ω), x 0 ω, ν τ(x(ω)0 ),ˇω ) for P -a.e. ω Ω. Similarly as below (3.9), we also have ha ν τ(x(ω)0 ),ˇω = ν 1(ˇω τ(x(ω) 0 ) ) = (ν 1 ) τ(x(ω)0 ),ˇω for P -a.e. ω Λ. Therefore, P τ(ω),ω = P ( τ(ω), x 0 ω, (ν 1 ) τ(x(ω)0 ),ˇω ) = (P 1 ) τ(ω),ω for P -a.e. ω Λ. An analogous argumen esablishes he hird propery from (4.2) and we conclude ha P = P P x. The second propery is he quasi-sure represenaion of V x (ξ) on L 1 P x, a resul which will be generalized in Theorem 5.2 below. 19

20 Lemma 4.7. Le [0, T ] and ξ L 1 P x. Then V x (ξ) = ess sup P E P [ξ x F ] P -a.s. for all P P x, (4.3) P P x(f,p ) where P x (F, P ) := {P P x : P = P on F }. Proof. Recall ha Lemma 4.4 allows us o appeal o he resuls of Secion 3. (i) We rs prove he inequaliy for ξ UC b (Ω). Fix P P x. We use Sep (ii) of he proof of Theorem 3.2, in paricular (3.17), for he special case s = 0 and obain ha for given ε > 0 and N 1 here exiss a measure P N P x (F, P ) such ha V x (ω) E P N [ξ x F ](ω) + ρ(ε) for P -a.s. ω E 1 E N, where V x (ω) := V x (ξ; ω). Since i 1 Ei = Ω 0 = Ω P -a.s., we deduce ha V x (ω) sup E P N [ξ x F ](ω) + ρ(ε) for P -a.s. ω Ω. N 1 The claim follows by leing ε 0. (ii) Nex, we show he inequaliy in (4.3) for ξ UC b (Ω). Fix P, P P x and recall ha (P ),ω P(, x 0 ω) for P -a.s. ω Ω by Lemma 3.4. Therefore, (3.2) applied wih s := and := T yields ha V x (ω) = V (x 0 ω) E (P ),ω [ξ,x 0ω ] = E (P ),ω [(ξ x ),ω ] = E P [ξ x F s ](ω) P -a.s. on F. If P P x (F, P ), hen P = P on F and he inequaliy holds also P -a.s. The claim follows as P P x (F, P ) was arbirary. (iii) So far, we have proved he resul for ξ UC b (Ω). The general case ξ L 1 P x can be derived by an approximaion argumen exploiing he sabiliy under pasing (Lemma 4.6). We omi he deails since he proof is exacly he same as in [17, Theorem 5.4]. 5 Pah Regulariy for he Value Process In his secion, we consruc a càdlàg P x -modicaion for V x (ξ); ha is, a càdlàg process Y x such ha Y x = V x (ξ) P x -q.s. for all [0, T ]. (Recall ha he iniial condiion x R d has been xed.) To his end, we exend he raw lraion F as in [19]: we le F + = {F + } 0 T be he minimal righconinuous lraion conaining F and we augmen F + by he collecion N Px of (P x, F T )-polar ses o obain he lraion G = {G } 0 T, G := F + N Px. We noe ha G depends on x R d since N Px does, bu for breviy, we shall no indicae his in he noaion. In fac, he dependence on x is no crucial: 20

21 we could also work wih F +, a he expense of obaining a modicaion which is P x -q.s. equal o a càdlàg process raher han being càdlàg iself. We recall ha in he quasi-sure seing, value processes similar o he one under consideraion do no admi càdlàg modicaions in general; indeed, while he righ limi exiss quasi-surely, i need no be a modicaion (cf. [19]). Boh he regulariy of ξ L 1 P x and he regulariy induced by he SDE are crucial for he following resul. Theorem 5.1. Le ξ L 1 P x. There exiss a (P x -q.s. unique) G-adaped càdlàg P x -modicaion E x (ξ) = {E x (ξ)} [0,T ] of {V x (ξ)} [0,T ]. Moreover, E x (ξ) = for all [0, T ]. ess sup P E P [ξ x G ] P -a.s. for all P P x, (5.1) P P x(g,p ) Proof. In view of Lemmaa 4.6 and 4.7, we obain exacly as in [19, Proposiion 4.5] ha here exiss a P x -q.s. unique G-adaped càdlàg process E x (ξ) saisfying (5.1) and E x (ξ) = V+(ξ) x := lim Vr x (ξ) P x -q.s. for all 0 < T. (5.2) r The observaion made here is ha (4.3) implies ha V x (ξ) is a (P, F)- supermaringale for all P P x, so ha one can use he sandard modi- caion argumen for supermaringales under each P. This argumen, cf. [6, Theorem VI.2], also yields ha E P [E x (ξ) F + ] V x (ξ) P -a.s. and in paricular E P [E x (ξ)] E P [V x (ξ)] for all P P x. Hence, i remains o show ha E x (ξ) V x (ξ) P x -q.s. (5.3) for [0, T ), which is he par ha is known o fail in a more general seing. We give he proof in several seps. (i) We rs show ha E x maps UC b (Ω) o L 1 P x (F ), and in fac even o UC b (Ω ) if a suiable represenaive is chosen. Le ξ UC b (Ω), r (, T ] and se Vr x := Vr x (ξ). By Proposiion 2.5, here exiss a modulus of coninuiy ρ independen of r such ha V x r (ω) V x r (ω ) ρ( ω ω r ). Hence he P x -q.s. limi from (5.2) saises V + (ω) V + (ω ) ρ( ω ω r ) for all r (, T ] Q, P x -q.s. Since E x (ξ) = V + (ξ), aking he limi r yields ha E x (ξ)(ω) E x (ξ)(ω ) ρ( ω ω ) P x -q.s. 21

22 By a varian of Tieze's exension heorem, cf. [16], his implies ha E x (ξ) coincides P x -q.s. wih an elemen of UC b (Ω ). In paricular, E x (ξ) L 1 P x (F ). (ii) Nex, we show ha E x is Lipschiz-coninuous. Le ξ, ψ L 1 P x and n. Using (5.2), Faou's lemma and Lemma 4.3, we obain ha E x (ξ) E x (ψ) L 1 Px = limn V x n (ξ) V x n (ψ) L 1 Px = sup E P [ lim n V x n (ξ) V x n (ψ) ] P P x sup lim inf E P [ V x P P x n n (ξ) V x n (ψ) ] ξ ψ L 1 Px. (iii) Le ξ L 1 P x. Then here exis ξ n UC b (Ω) such ha ξ n ξ in L 1 P x and in hus E x (ξ n ) E x (ξ) in L 1 P x by Sep (ii). Since E x (ξ n ) L 1 P x (F ) by Sep (i) and since L 1 P x (F ) is closed in L 1 P x, we conclude ha E x (ξ) L 1 P x (F ) for all ξ L 1 P x. (iv) Le ξ L 1 P x. Since V x is he ideniy on L 1 P x (F ), Sep (iii) implies ha E x (ξ) = V x (E x (ξ)). Moreover, he represenaions (4.3) and (5.1) yield V x (E x (ξ)) = We conclude ha (5.3) holds rue. ess sup P E P [E x (ξ) F ] P P x(f,p ) ess sup P E P [ E P [ξ x G ] ] F P P x(f,p ) = ess sup P E P [ξ x F ] P P x(f,p ) = V x (ξ) P -a.s. for all P P x. Since E x (ξ) is a càdlàg process, is value E x τ (ξ) a a sopping ime τ is well dened. The following resul saes he quasi-sure represenaion of E x τ (ξ) and he quasi-sure version of he dynamic programming principle in is nal form. Theorem 5.2. Le 0 ρ τ T be G-sopping imes and ξ L 1 P x. Then E x ρ (ξ) = and in paricular E x ρ (ξ) = ess sup P E P [Eτ x (ξ) G ρ ] P -a.s. for all P P x (5.4) P P x(g ρ,p ) ess sup P E P [ξ x G ρ ] P -a.s. for all P P x. P P x(g ρ,p ) Moreover, here exiss for each P P x a sequence P n P x (G ρ, P ) such ha wih a P -a.s. increasing limi. E x ρ (ξ) = lim n EPn [ξ x G ρ ] 22 P -a.s.

23 Proof. In view of Lemmaa 4.6 and 4.7, he resul is derived exacly as in [19, Theorem 4.9]. As in (3.3), he relaion (5.4) can be seen as a semigroup propery E x ρ (ξ) = E x ρ (E x τ (ξ)), a leas when E x τ (ξ) is in he domain L 1 P x of E x ρ. The laer is guaraneed by Lemma 4.3 when τ is a deerminisic ime. However, one canno expec E x τ (ξ) o be quasi uniformly coninuous (cf. Remark 4.1) for a general sopping ime, for which reason we prefer o express he righ hand side as in (5.4). 6 Hamilon-Jacobi-Bellman 2BSDE In his secion, we characerize he value process E x (ξ) as he soluion of a 2BSDE. To his end, we rs examine he properies of B under a xed P P x. The following resul is in he spiri of [28, Secion 8]. Proposiion 6.1. Le x R d, ν U and P := P (0, x, ν). There exiss a progressively measurable ransformaion β : Ω Ω (depending on x, ν) such ha W := β(b) is a P -Brownian moion and F P = F W P. (6.1) Moreover, B is he P -a.s. unique srong soluion of he SDE B = 0 μ(, x + B, ν (W )) d + 0 σ(, x + B, ν (W )) dw under P. Proof. Le X := X(0, x, ν). As in Lemma 3.4, Assumpion 2.1 implies he he exisence of a progressively measurable ransformaion β : Ω Ω such ha β(x 0 ) = B P 0 -a.s. (6.2) Le W := β(b). Then (B, X 0 ) P0 = (β(x 0 ), X 0 ) P0 = (β(b), B) P = (W, B) P ; i.e., he disribuion of (B, X 0 ) under P 0 coincides wih he disribuion of (W, B) under P. In paricular, W is a P -Brownian moion. Moreover, we have F P X0 0 = F β(x0 ) P 0 by Assumpion 2.1 and herefore F BP = F β(b)p, which is (6.1). Noe ha X 0 = X 0 (B) = μ(, x + X 0, ν (B)) d + σ(, x + X 0, ν (B)) db 23

24 under P 0. Le Y be he (unique, srong) soluion of he analogous SDE Y = μ(, x + Y, ν (W )) d + σ(, x + Y, ν (W )) dw under P. Using he deniion of P and (6.2), we have ha (Y, W ) P = (X 0 (W ), W ) P = (X 0 (B), B) P0 = (X 0, β(x 0 )) P0 = (B, β(b)) P = (B, W ) P. In view of (6.1), i follows ha Y = B holds P -a.s. In he sequel, we denoe by M B,P he local maringale par in he canonical semimaringale decomposiion of B under P. Corollary 6.2. Le P P x. Then he lraion F P is righ-coninuous. If, in addiion, σ is inverible, hen (M B,P, P ) has he predicable represenaion propery. The laer saemen means ha any righ-coninuous (F P, P )-local maringale N has a represenaion N = N 0 + Z dm B,P under P, for some F P -predicable process Z. Proof. We have seen in Proposiion 6.1 ha F P is generaed by a Brownian moion W, hence righ-coninuous, and ha M B,P = 0 ˆσ dw for ˆσ := σ(, x + B, ν (W )), where ν U. By changing ˆσ on a d P -nullse, we may assume ha ˆσ is F P -predicable. Using he Brownian represenaion heorem and W = ˆσ 1 dm B,P, we deduce ha M B,P has he represenaion propery. The following formulaion of 2BSDE is, of course, inspired by [27]. Deniion 6.3. Le ξ L 1 P x and consider a pair (Y, Z) of processes wih values in R R d such ha Y is càdlàg G-adaped while Z is G-predicable and T 0 Z s 2 d B s < P x -q.s. Then (Y, Z) is called a soluion of he 2BSDE (6.3) if here exiss a family (K P ) P Px of F P -adaped increasing processes saisfying E P [ KT P ] < such ha Y = ξ T Z s dm B,P s + K P T K P, 0 T, P -a.s. for all P P x (6.3) and such ha he following minimaliy condiion holds for all 0 T : ess inf P [ P P x(g,p ) EP K P T K P ] G = 0 P -a.s. for all P Px. (6.4) 24

25 Moreover, a càdlàg process Y is said o be of class (D,P x ) if he family {Y τ } τ is uniformly inegrable under P for all P P x, where τ runs hrough all G-sopping imes. The following is our main resul. Theorem 6.4. Assume ha σ is inverible and le ξ L 1 P x. (i) There exiss a (d P x -q.s. unique) G-predicable process Z ξ such ha Z ξ = ( d B, B P ) 1 d E x (ξ), B P P -a.s. for all P P x. (6.5) (ii) The pair (E x (ξ), Z ξ ) is he minimal soluion of he 2BSDE (6.3); i.e., if (Y, Z) is anoher soluion, hen E x (ξ) Y P x -q.s. (iii) If (Y, Z) is a soluion of (6.3) such ha Y is of class (D,P x ), hen (Y, Z) = (E x (ξ), Z ξ ). In paricular, if ξ L p P x for some p > 1, hen (E x (ξ), Z ξ ) is he unique soluion of (6.3) in he class (D,P x ). Proof. Given wo processes which are (càdlàg) semimaringales under all P P x, heir quadraic covariaion can be dened P x -q.s. by using he inegraion-by-pars formula and Bicheler's pahwise sochasic inegraion [1, Theorem 7.14]; herefore, he righ hand side of (6.5) can be used as a deniion of Z ξ. The deails of he argumen are as in [19, Proposiion 4.10]. Le P P x. By Proposiion 6.1, B is an Iô process under P ; in paricular, we have B, S P = M B,P, S P P -a.s. for any P -semimaringale S. The Doob-Meyer heorem under P and Corollary 6.2 hen yield he decomposiion E x (ξ) = E0 x (ξ) + Z ξ dm B,P K P P -a.s. and we obain (ii) and (iii) by following he argumens in [19, Theorem 4.15]. If ξ L p P x for some p (1, ), hen E x (ξ) is of class (D,P x ) as a consequence of Jensen's inequaliy (cf. [19, Lemma 4.14]). Therefore, he las asserion follows from he above. We conclude by inerpreing he canonical process B, seen under he se of scenarios P x, as a model for drif and volailiy uncerainy in he Knighian sense. Remark 6.5. Consider he se-valued process D (ω) := {( μ(, ω, u), σ(, ω, u) ) : u U } R d R d d. In view of Proposiion 6.1, each P P x can be seen as a scenario in which he drif and he volailiy (of B) ake values in D, P -a.s. Then, he upper expecaion E x (ξ) is he corresponding wors-case expecaion (see [19] for a connecion o superhedging in nance). Noe ha D is a random process alhough he coeciens of our conrolled SDE are non-random. Indeed, 25

26 he pah-dependence of he SDE ranslaes o an ω-dependence in he weak formulaion ha we are considering. In paricular, for μ 0, we have consruced a sublinear expecaion similar o he random G-expecaion of [17]. While he laer is dened by specifying a se-valued process like D in he rs place, we have sared here from a conrolled SDE under P 0. I seems ha he presen consrucion is somewha less echnical ha he one in [17]; in paricular, we did no work wih he process ˆa = d B /d which played an imporan role in [26] and [17]. However, i seems ha he Lipschiz condiions on μ and σ are essenial, while [17] merely used a noion of uniform coninuiy. References [1] K. Bicheler. Sochasic inegraion and L p -heory of semimaringales. Ann. Probab., 9(1):4989, [2] R. Buckdahn and J. Ma. Sochasic viscosiy soluions for nonlinear sochasic parial dierenial equaions. I. Sochasic Process. Appl., 93(2):181204, [3] R. Buckdahn and J. Ma. Sochasic viscosiy soluions for nonlinear sochasic parial dierenial equaions. II. Sochasic Process. Appl., 93(2):205228, [4] R. Buckdahn and J. Ma. Pahwise sochasic conrol problems and sochasic HJB equaions. SIAM J. Conrol Opim., 45(6): , [5] P. Cheridio, H. M. Soner, N. Touzi, and N. Vicoir. Second-order backward sochasic dierenial equaions and fully nonlinear parabolic PDEs. Comm. Pure Appl. Mah., 60(7): , [6] C. Dellacherie and P. A. Meyer. Probabiliies and Poenial B. Norh Holland, Amserdam, [7] L. Denis, M. Hu, and S. Peng. Funcion spaces and capaciy relaed o a sublinear expecaion: applicaion o G-Brownian moion pahs. Poenial Anal., 34(2):139161, [8] L. Denis and C. Marini. A heoreical framework for he pricing of coningen claims in he presence of model uncerainy. Ann. Appl. Probab., 16(2): , [9] N. El Karoui. Les aspecs probabilises du conrôle sochasique. In Ecole d'éé de probabiliées de Sain-Flour, volume 876 of Lecure Noes in Mah., pages 73238, Springer, Berlin, [10] R. J. Ellio. Sochasic Calculus and Applicaions. Springer, New York, [11] J. Feldman and M. Smorodinsky. Simple examples of non-generaing Girsanov processes. In Séminaire de Probabiliés XXXI, volume 1655 of Lecure Noes in Mah., pages Springer, Berlin, [12] W. H. Fleming and H. M. Soner. Conrolled Markov Processes and Viscosiy Soluions. Springer, New York, 2nd ediion, [13] Y. Hu, P. Imkeller, and M. Müller. Uiliy maximizaion in incomplee markes. Ann. Appl. Probab., 15(3): , [14] P.-L. Lions and P. Souganidis. Fully nonlinear sochasic parial dierenial equaions. C. R. Acad. Sci. Paris Sér. I Mah., 326(9): , [15] P.-L. Lions and P. Souganidis. Fully nonlinear sochasic parial dierenial equaions: non-smooh equaions and applicaions. C. R. Acad. Sci. Paris Sér. I Mah., 327(8):735741,

27 [16] M. Mandelkern. On he uniform coninuiy of Tieze exensions. Arch. Mah., 55(4):387388, [17] M. Nuz. Random G-expecaions. Preprin arxiv: v1, [18] M. Nuz. The Bellman equaion for power uiliy maximizaion wih semimaringales. Ann. Appl. Probab., 22(1):363406, [19] M. Nuz and H. M. Soner. Superhedging and dynamic risk measures under volailiy uncerainy. Preprin arxiv: v1, [20] E. Pardoux and S. Peng. Adaped soluion of a backward sochasic dierenial equaion. Sysems Conrol Le., 14(1):5561, [21] S. Peng. Sochasic Hamilon-Jacobi-Bellman equaions. SIAM J. Conrol Opim., 30(2):284304, [22] S. Peng. Filraion consisen nonlinear expecaions and evaluaions of coningen claims. Aca Mah. Appl. Sin. Engl. Ser., 20(2):191214, [23] S. Peng. G-expecaion, G-Brownian moion and relaed sochasic calculus of Iô ype. In Sochasic Analysis and Applicaions, volume 2 of Abel Symp., pages , Springer, Berlin, [24] S. Peng. Muli-dimensional G-Brownian moion and relaed sochasic calculus under G-expecaion. Sochasic Process. Appl., 118(12): , [25] S. Peng. Noe on viscosiy soluion of pah-dependen PDE and G-maringales. Preprin arxiv: v1, [26] H. M. Soner, N. Touzi, and J. Zhang. Dual formulaion of second order arge problems. To appear in Ann. Appl. Probab., [27] H. M. Soner, N. Touzi, and J. Zhang. Wellposedness of second order backward SDEs. To appear in Probab. Theory Relaed Fields, [28] H. M. Soner, N. Touzi, and J. Zhang. Quasi-sure sochasic analysis hrough aggregaion. Elecron. J. Probab., 16(2): , [29] D. Sroock and S. R. S. Varadhan. Mulidimensional Diusion Processes. Springer, New York,