Verification Theorems for Stochastic Optimal Control Problems via a Time Dependent Fukushima - Dirichlet Decomposition

Transcription

1 Verificaion Theorems for Sochasic Opimal Conrol Problems via a Time Dependen Fukushima - Dirichle Decomposiion Fauso Gozzi, Francesco Russo To cie his version: Fauso Gozzi, Francesco Russo. Verificaion Theorems for Sochasic Opimal Conrol Problems via a Time Dependen Fukushima - Dirichle Decomposiion. Preprin LAGA-Paris To appear: Sochasic Processes and Their Applicaions. 34 pages <hal > HAL Id: hal hps://hal.archives-ouveres.fr/hal Submied on 13 Apr 2006 HAL is a muli-disciplinary open access archive for he deposi and disseminaion of scienific research documens, wheher hey are published or no. The documens may come from eaching and research insiuions in France or abroad, or from public or privae research ceners. L archive ouvere pluridisciplinaire HAL, es desinée au dépô e à la diffusion de documens scienifiques de niveau recherche, publiés ou non, émanan des éablissemens d enseignemen e de recherche français ou érangers, des laboraoires publics ou privés.

2 Verificaion Theorems for Sochasic Opimal Conrol Problems via a Time Dependen Fukushima - Dirichle Decomposiion Fauso GOZZI Diparimeno di Scienze Economiche e Aziendali Facolà di Economia LUISS - Guido Carli Viale Pola 12, I Roma, Ialy Francesco RUSSO Universié Paris 13 Insiu Galilée, Mahémaiques 99, av. JB Clémen, F Villeaneuse, France ccsd , version 1-13 Apr 2006 A.M.S. Subjec Classificaion: Primary: 35R15, 49L10, 60H05, 93B52, 93E20. Secondary: 35J60, 47D03, 47D07, 49L25, 60J35, 93B06, 93B50. Key words: Hamilon-Jacobi-Bellman (HJB) equaions, sochasic calculus via regularizaion, Fukushima-Dirichle decomposiion, sochasic opimal conrol, verificaion heorems. Absrac. This paper is devoed o presen a mehod of proving verificaion heorems for sochasic opimal conrol of finie dimensional diffusion processes wihou conrol in he diffusion erm. The value funcion is assumed o be coninuous in ime and once differeniable in he space variable (C 0,1 ) insead of once differeniable in ime and wice in space (C 1,2 ), like in he classical resuls. The resuls are obained using a ime dependen Fukushima - Dirichle decomposiion proved in a companion paper by he same auhors using sochasic calculus via regularizaion. Applicaions, examples and comparison wih oher similar resuls are also given. 1 Inroducion In his paper we wan o presen a mehod o ge verificaion heorems for sochasic opimal conrol problems of finie dimensional diffusion processes wihou conrol in he diffusion erm. The mehod is based on a generalized Fukushima - Dirichle decomposiion proved in he companion paper [22]. Since his Fukushima - Dirichle decomposiion holds for funcions u : [0, T] R n R ha are C 0 in ime and C 1 in space (C 0,1 in symbols), our verificaion heorem has he advanage of requiring less regulariy of he value funcion V han he classical ones which need C 1 regulariy in ime and C 2 in space of V (C 1,2 in symbols), see e.g. [13, pp. 140, 163, 172]. There are also oher verificaion heorems ha works in cases when he value funcion is nonsmooh: e.g. i is possible o prove a verificaion heorem in he case when V is only coninuous (see [23], [27], [39, Secion 5.2]) in he framework of viscosiy soluions. However all hese resuls applied o our cases are weaker han ours, for reasons ha are clarified in Secion 8. Since he mehod is a bi complex and ariculaed we presen firs in nex Secion 2 he saemen of our verificaion Theorems 2.7 and 2.8 in a model case wih simplified 1

3 assumpions (which subsanially yield nondegeneracy of he diffusion coefficien). In he same secion we also pu Subsecion 2.1 where we give some commens on he heorem, is applicabiliy and is relaionship wih oher similar resuls. Below we pass o he body of he paper giving firs some noaions in Secion 3 and hen presening he general saemens (including also possible degeneracy of he diffusion coefficien) and heir proof in Secion 4. Secion 5 is devoed o necessary condiions and opimal feedbacks. Secions 6 and 7 conain applicaions of our echnique o more specific classes of problems where oher echniques are more difficul o use. The firs is a case of exi ime problem where he HJB equaion is nondegenerae bu C 1,2 regulariy is no known o hold due o he lack of regulariy of he coefficiens; he second is a case where he HJB equaion is degenerae parabolic. Finally in Secion 8 we compare our resul wih oher verificaion echniques. 2 The saemen of he verificaion heorems in a model case To clarify our resuls we describe briefly and informally below he framework and he saemen of he verificaion heorem ha we are going o prove in a model case. The precise saemens and proofs are given in Secion 4; hen in Secions 6, 7 applicaions o more specific (and somehow more difficul) classes of problems are given. We decided his srucure since i is difficul o provide a single general resul: we can say ha we inroduce a echnique, based on he Fukushima-Dirichle decomposiion and on is represenaion given in [22], ha can be adaped wih some work o differen seings each ime wih a differen adapaion. ( ) Firs we ake a given sochasic basis Ω, (F s ) s 0, P ha saisfies he so-called usual condiions, a finie dimensional Hilber space A = R n (he sae space), a finie dimensional Hilber space E = R m (he noise space), a se U R k (he conrol space). We fix hen a erminal ime T [0, + ] (he horizon of he problem, ha can be finie or infinie bu is fixed), an iniial ime and sae (, x) [0, T] A (which will vary as usual in he dynamic programming approach). The sae equaion is (recall ha T = [, T] R) dy(s) = [F 0 (s, y (s)) + F 1 (s, y (s),z (s))] ds + B (s, y (s))dw(s), s T, y () = x, (1) where he following holds (for a marix B by B we mean i,j b ij and, given E, F finie dimensional spaces, by L(E, F) we mean he se of linear operaors from E o F). Hypohesis z : T Ω U (he conrol process) is measurable, locally inegrable in for a.e. ω Ω, adaped o he filraion (F s ) s 0 ; 2. F 0 : T 0 R n R n, F 1 : T 0 R n U R n and B : T 0 R n L(E, R n ) are coninuous. 3. There exiss C > 0 such ha T 0, x 1, x 2 R n, F 0 (, x 1 ) F 0 (, x 2 ),x 1 x 2 + B (, x 1 ) B (, x 2 ) 2 C x 1 x 2 2, F 0 (, x),x + B (, x) 2 C [1 + x 2] and here exiss a consan K such ha, T 0, x R n, z U, F 1 (, x 1, z) F 1 (, x 2, z) K x 1 x 2 F 1 (, x, z),x K (1 + x + z ). 2

4 4. For every x R n 0 [ F 0 (, x) + B (, x) 2] d < +. We call Z ad () he se of admissible conrol sraegies when he iniial ime and sae is (, x) defined as { } z : T Ω U, z measurable, adaped o (F Z ad () = s ) s 0, locally inegrable in s for a.e. ω Ω and call y (s;, x, z) he sae process associaed wih a given z Z ad (); his is he srong soluion of he equaion (1) which exiss and is unique for any given z hanks o Theorem 1.2 in [25, p.2]. Consider now he case T < +. We ry o minimize he funcional { } T J (, x; z) = E l (s, y (s;, x, z),z (s))ds + φ(y (T;, x, z)) (2) over all conrol processes z Z ad (). On he coefficiens we assume he following. Hypohesis 2.2 l : [0, T] R n U R and φ : R n R are coninuous and such ha for each z Z ad () he funcion (s, ω) l (s, y (s, ω),z (s, ω)) is lower semiinegrable in [, T] Ω (recall ha a real valued funcion f is lower semiinegrable in a measure space (M, µ) if M f dµ < +, where f sands for he negaive par of f i.e f = ( f) 0). Remark 2.3 The above requiremen implies ha J (, x; z) is well defined and > for each z Z ad (). I is assumed expliciely o cover a more general class of problems. This is obvious under addiional assumpions, e.g. if l and φ are bounded below. However some ineresing problems arising in economics conain funcions l like lnz and in his case he lower semiinegrabiliy of (s, ω) lnz (s, ω) for each z Z ad () follows by ad hoc argumens. These are problems wih sae consrains ha are no expliciely reaed in his paper bu we wan o se our framework so o be able o rea such class of problems. The value funcion is defined as V (, x) = inf J (, x; z) (3) z Z ad () and a conrol z Z ad () is opimal a (, x) if V (, x) = J (, x; z ). The curren value Hamilonian is defined, for (, x, p, z) [0, T] R n R n U as H CV (, x, p; z) = F 0 (, x),p + F 1 (, x, z),p + l (, x, z) and he (minimum value) Hamilonian as H (, x, p) = inf z U H CV (, x, p; z). Since he firs erm of H CV (, x, p; z) does no depend on he conrol z we will usually define H 0 CV (, x, p; z) = F 1 (, x, z),p + l (, x, z) and H 0 (, x, p) = inf z U H0 CV (, x, p; z), (4) 3

5 so we can wrie and H CV (, x, p; z) =: F 0 (, x),p + H 0 CV (, x, p; z) (5) H (, x, p) =: F 0 (, x),p + H 0 (, x, p). (6) When T < + he Hamilon-Jacobi-Bellman (HJB) equaion for he value funcion is a semilinear parabolic PDE ((, x) [0, T] R n ) v (, x) = 1 2 Tr[B (, x) xx v(, x)b (, x)] + F 0 (, x), x v (, x) + H 0 (, x, x v (, x)), wih he final condiion v (T, x) = φ(x), x R n. (8) To ensure finieness and coninuiy of he Hamilonian we need also o add he following assumpion. Hypohesis 2.4 The funcions l, F 1 and he se U are such ha he value funcion is always finie and he Hamilonian H 0 (, x, p) is well defined, finie and coninuous for every (, x, p) [0, T] R n R n. Remark 2.5 The above Hypohesis 2.4 is saisfied e.g. when l, φ are coninuous and U is compac. Anoher possibiliy is o ake U unbounded, F 1 sublinear, l (, x, z) = g (x) + h (z) wih g coninuous and bounded, h coninuous and such ha (7) h(z) / z + as z + ; his case was sudied e.g in [6, 18, 21] even in he infinie dimensional case (see on his he Secions 6, 7). The saemen of he classical verificaion heorem for his model problem when T < + is he following, see Definiion 4.5 for he definiion of sric soluion of equaion (7)-(8). Theorem 2.6 Assume ha Hypoheses 2.1, 2.2, 2.4 hold rue. Le v C 1,2 ([0, T] R n ) be a polynomially growing sric soluion of he HJB equaion (7)-(8) on [0, T] R n. Then he wo following properies hold rue. (i) v V on [0, T] R n ; (ii) fix (, x) [0, T] R n ; if z Z ad () is such ha, calling y (s) = y (s;, x, z), H 0 (s, y (s), x v (s, y (s))) = H 0 CV (s, y (s), x v (s, y (s)),z (s)), P a.s., for a.e. s [, T], hen z is opimal a (, x) and v (, x) = V (, x). This heorem saes a sufficien opimaliy condiion and is proof is based on Iô s formula, see e.g. [39, p.268]. In he classical conex also necessary condiions and exisence of opimal feedback can be proved, see again [39, p.268]. The saemen of our Verificaion Theorem in he model problem described above is very similar. We give i in he case when he following nondegeneracy hypohesis hold B 1 (, x)f 1 (, x, z) is bounded on [0, T] R n U, (9) leaving he discussion for he general case o Secion 4 (srong soluions of he HJB equaion (7)-(8) are defined in Definiion 4.6). 4

6 Theorem 2.7 Assume ha Hypoheses 2.1, 2.2, 2.4 and (9) hold rue. Le v C 0,1 ([0, T] R n ) be a polynomially growing srong soluion of he HJB equaion (7)-(8) on [0, T] R n. Then (i) and (ii) of Theorem 2.6 above hold. In fac we will go furher: proving a resul in he case when (9) does no hold; showing ha in our seing we can obain a necessary condiion and he exisence of opimal feedback conrols on he same line of he classical resuls, see Secion 5. Moreover in a paper in preparaion we will show ha also some cases where he drif is a disribuion can be reaed and also cases where he soluion of HJB enjoys weaker regulariy. When T = +, F 0, F 1 and B do no depend on ime, we consider he problem of minimizing { + } J (, x; z) = E e λ l 1 (y (s;, x, z),z (s))ds, where l (, x, z) = e λ l 1 (x, z), wih λ > 0, l 1 coninuous and bounded. In his case he value funcion V is defined as in (3) bu is dependence on becomes rivial as V (, x) = e λ V (0, x) for every and x. Then we se = 0 and call V 0 (x) = V (0, x). The HJB equaion for V 0 (x) becomes an ellipic PDE (x R n ) λv (x) = 1 2 Tr[B (x) xx v(x)b (x)] + F 0 (, x), x v (, x) + H 1 (x, x v (, x)), (10) where H 1 (x, p) = inf { F 1 (x, z),p + l 1 (x, z)} =: inf z U z U H1 CV (x, p; z). (11) In his ellipic case he saemen of our verificaion heorem is he following (for he definiion of srong soluion of he HJB equaion (10) see Definiion 4.14). Theorem 2.8 Assume ha Hypoheses 2.1 and (9) hold. Assume also ha l 1 is coninuous and bounded and ha Hypohesis 2.4 hold rue wih H 1 in place of H 0. If v is a bounded srong soluion of he HJB equaion (10) and v C 1 (R n ) hen v V 0. Moreover fix x R n ; if z Z ad (0) is such ha, calling y (s) = y (s; 0, x, z), H 1 (y (s), x v (s, y (s))) = H 1 CV (y (s), xv (s, y (s)), z (s)), for a.e. s 0, P a.s., hen z is opimal a (0, x) and v (x) = V 0 (x). 2.1 Oher Verificaion Theorems for sochasic opimal conrol problems We recall ha in he lieraure on sochasic opimal conrol oher verificaion heorems for no C 1,2 funcions were proved. Roughly speaking we can say ha each of hem (and he echnique of proof, oo) is sricly conneced wih he concep of weak soluion of he HJB equaion ha is considered. We have in fac he following. (Srong soluions). In [18, 21] (see also [7, 11, 17]) in a seing similar o ours bu in infinie dimension, a verificaion heorem is proved assuming ha here exiss a srong soluion v of HJB which is C 1 in space. In fac he mehod oulined here goes in he 5

7 same direcion, bu improves and generalizes in he finie dimensional case he ideas conained here. Such improvemen is no a sraighforward one, as we need o use compleely differen ools o ge our resuls and we ge more powerful heorems. We refer o Secions 6 and 7 for examples and o Secion 8 for explanaions. (Viscosiy soluions). In [27, 39, 23] a verificaion echnique for viscosiy soluions is inroduced and sudied. Such echnique adaps o he case when v is only coninuous and so is very general and applicable o cases when he conrol eners in he diffusion coefficiens B. However in he case of our ineres, i.e. when v is C 1 in space, such echnique gives weaker resuls as i requires more assumpions on he coefficiens F 0, F 1, B and on he candidae opimal sraegy; see Secion 8 for explanaions. (Mild soluions). In [14, Theorem 7.2] a verificaion heorem is given when he HJB equaion admis mild soluions which are C 1 in space (see Definiion 6.13 or Remark 7.2) and when he soluion can be represened using he soluion of a suiable backward SDE. The resuls available wih his echnique are limied o infinie dimensional cases where mild soluions exis and Girsanov heorem can be applied in a suiable way. Such resricions preven he use of his echnique (a leas o he presen sae of he ar) e.g. in he cases described in Secions 6 and 7, see Secion 8 for explanaions. Moreover we wan o sress he fac ha he Fukushima-Dirichle decomposiion we go in [22] is in fac sronger han wha we need o prove he verificaion heorems above. In fac for such purpose i would be enough o prove a Dynkin-ype formula (roughly speaking an Iô formula afer expecaion) which is in general easier. We decided o sae and prove such Fukushima-Dirichle decomposiion since i can help o deal wih sochasic conrol problems where he crierion o minimize does no conain expecaion (e.g. pahwise opimaliy and opimaliy in probabiliy). In such conex he HJB equaion becomes a sochasic PDE and o ge a verificaion heorem, Dynkin-ype formulae canno be used. See e.g. [10, 28, 29, 33] on his subjec. To sum up we hink ha he ineres of our verificaion resuls is he fac ha we obain hem using only he fac ha he soluion v of HJB belongs o C 0,1 and no necessarily o C 1,2 obaining beer resuls han oher echniques. hey can be applied also o problems wih pahwise opimaliy and opimaliy in probabiliy. Remark 2.9 As a firs sep of our work, we consider here he case of a sochasic opimal conrol problem in finie dimension wih no sae consrains. We are aware of he fac ha in many cases, he HJB equaion associaed wih he conrol problem admis a C 1,2 soluion, so ha our mehod does no give a real advanage in his case. However in some cases, mainly degenerae ones (see Secions 6 and 7) i is known ha he soluion of HJB is C 0 (or C α wih Hölder exponen α > 0) in ime and C 1 in space bu C 1,2 regulariy is no known a his sage. In paricular we show an explici example where he value funcion is C 1 bu canno be C 2 in space; we hink ha such echnique could be exended also in cases where HJB equaion is inended in a generalized sense; in a paper in preparaion, we invesigae a verificaion heorem relaed o an SDE whose drif is he derivaive of a coninuous funcion herefore a Schwarz disribuion, see for insance [12]. Tha sae equaion models a sochasic paricle moving in a random (irregular) medium; 6

8 we consider problems wihou sae consrains; again we hink ha some sae consrains problems can be reaed wih our approach bu we do no handle hem here: we provide an example wih exi ime which usually presens similar difficulies; we hink ha our mehod can be exended o he infinie dimensional case, where C 1,2 regulariy resuls for he soluion of HJB equaion are much less known while C 1 regulariy can be found in a variey of cases (see e.g. [6, 18, 21, 7]). We do no perform his exension here leaving i for furher work. Remark 2.10 Similar ideas as in his work have been expressed wih a differen formalism in [8]. There he auhors have implemened a generalized Iô formula in he Krylov spiri for proving an exisence and uniqueness resul for a generalized soluion of Bellman equaion for conrolled processes, in a non-degeneracy siuaion. 3 Noaions Throughou his paper we will denoe by (Ω, F, P) a given sochasic basis, where F sands for a given filraion (F s ) s 0 saisfying he usual condiions. Given a finie dimensional real Hilber space E, W will denoe a cylindrical Brownian moion wih values in E and adaped o (F s ) s 0. Given 0 T + and seing T = [, T] R he symbol C F (T Ω; E), will denoe he space of all coninuous processes adaped o he filraion F wih values in E. This is a Fréche space if endowed wih he opology of he uniform convergence in probabiliy (u.c.p. from now on). To be more precise his means ha, given a sequence (X n ) C F (T Ω; E) and X C F (T Ω; E) we have if and only if for every ε > 0, 1 T lim sup n + s [, 1] X n X P ( X n s X s E > ε) = 0. Given a random ime τ and a process (X s ) s T, we denoe by X τ he sopped process defined by Xs τ = X s τ. The space of all processes in [, T], adaped o F and square inegrable wih values in E is denoed by L 2 F (, T; E). Sn will denoe he space of all symmeric marices of dimension n. Le k N. As usual C k (R n ) is he space of all funcions : R n R ha are coninuous ogeher wih heir derivaives up o he order k. This is a Fréche space equipped wih he seminorms sup u (x) R + sup x u (x) R n + sup xx u (x) R n n +... (12) x K x K x K for every compac se K R n. This space will be denoed simply by C k when no confusion may arise. If K ia compac subse of R n hen C k (K) is a Banach space wih he norm (12). The symbol C k b (Rn ) will denoe he Banach space of all funcions from R n o R ha are coninuous and bounded ogeher wih heir derivaives up o he order k. This space is endowed wih he usual sup norm. Passing o parabolic spaces we denoe by C 0 (T R n ) he space of all funcions u : T R n R, (s, x) u (s, x) 7

9 ha are coninuous. This space is a Fréche space equipped wih he seminorms sup u (s, x) R (s,x) [, 1] K for every 1 > 0 and every compac se K R n ). Moreover we will denoe by C 1,2 (T R n ) (respecively C 0,1 (T R n )), he space of all funcions u : T R n R, (s, x) u (s, x) ha are coninuous ogeher wih heir derivaives u, x u, xx u (respecively x u). This space is a Fréche space equipped wih he seminorms (respecively sup u (s, x) R + sup s u (s, x) R n (s,x) [, 1] K (s,x) [, 1] K + sup x u (s, x) R n + sup xx u (s, x) R n n (s,x) [, 1] K (s,x) [, 1] K sup u (s, x) R + sup x u (s, x) R n ) (s,x) [, 1] K (s,x) [, 1] K for every 1 > 0 and every compac se K R n. This space will be denoed simply by C 1,2 (respecively C 0,1 ) when no confusion may arise. Similarly, for α, β [0, 1] one defines C α,1+β (T R n ) (or simply C α,1+β ) as he subspace of C 0,1 (T R n ) of funcions u : T R n R such ha are u (, x) is α Hölder coninuous and x u (s, ) is β Hölder coninuous (wih he agreemen ha 0-Hölder coninuiy means jus coninuiy). If such properies hold jus locally hen such space is denoed by C α,1+β loc (T R n ). Similarly, given an open subse of O of R n one can define C α,1+β (T O) and C α,1+β loc (T O). If K is a compac subse of T R n we define C k (K) or C α,1+β (K) as above. Similarly o Cb k (Rn ) we define he Banach spaces Cb 0 (T R n ) C 1,2 b (T R n ), C α,1+β b (T R n ), C 0,1 b (T R n ). 4 Proof of he verificaion heorems in he model case In his secion we give he precise saemen and he proof of he verificaion Theorems 2.7 and 2.8 for he model problem described in Secion 2. Then in Secions 6 and 7 we will consider wo families of problems o which our echnique apply. Consider firs he parabolic case fixing he horizon T < +. Under Hypoheses 2.1, 2.2 and 2.4 we seek o minimize he cos J (, x; z) given in (2) over all admissible z Z ad () where he sae equaion is given by (1). We define he operaor L 0 : D (L 0 ) C 0 ([0, T] R n ) C 0 ([0, T] R n ), D (L 0 ) = C 1,2 ([0, T] R n ), L 0 v (, x) = v (, x) + F 0 (, x), x v (, x) Tr [B (, x) xx v (, x)b (, x)]. The HJB equaion associaed wih he problem (1) - (2) can hen be wrien as where H 0 is given in (4). L 0 v (, x) + H 0 (, x, x v (, x)) = 0, v (T, x) = φ(x), (13) 8

10 Now we wan o apply he represenaion resul proved in [22] Secion 4, which we recall below for he reader s convenience. Firs we consider he following Cauchy problem for h C 0 ([0, T] R n ). wih he following definiions of soluion. L 0 u (s, x) = h (s, x), u (T, x) = φ(x), (14) Definiion 4.1 We say ha u C 0 ([0, T] R n ) is a sric soluion o he backward Cauchy problem (14) if u D (L 0 ) and (14) holds. Definiion 4.2 We say ha u C 0 ([0, T] R n ) is a srong soluion o he backward Cauchy problem (14) if here exiss a sequence (u n ) D (L 0 ) and wo sequences (φ n ) C 0 (R n ), (h n ) C 0 ([0, T] R n ), such ha 1. For every n N u n is a sric soluion of he problem L 0 u n (, x) = h n (, x), u n (T, x) = φ n (x). 2. The following limis hold u n u in C 0 ([0, T] R n ), h n h in C 0 ([0, T] R n ), φ n φ in C 0 (R n ). The represenaion resul is he following (Corollaries 4.6 and 4.8 of [22]). Theorem 4.3 Le b 1 : [0, T] R n Ω R n, be a coninuous progressively measurable field (coninuous in (s, x)) and b : [0, T] R n R n, σ : [0, T] R n L(R m, R n ), be coninuous funcions. Le u C 0,1 ([0, T] R n ) be a srong soluion of he Cauchy problem (14). Fix [0, T], x R n and le (S s ) be a soluion o he SDE Then u (s, S s ) = u (, S ) + + ds s = b 1 (s, S s )ds + σ (s, S s )dw s, S = x. s 0 s h (r, S r )dr + s x u (r, S r )σ (r, S r )dw r. x u (r, S r ), b 1 (r, S r ) b (r, S r ) dr provided ha: eiher we can choose he approximaing sequence (u n ) of Definiion 4.2 so ha for every 0 s T lim n + s or he funcion x u n (r, S r ) x u (r, S r ),b 1 (r, S r ) b (r, S r ) dr = 0, u.c.p., (15) (, x, ω) σ 1 (, x)[b 1 (r, x, ω) b (r, x)], (where σ 1 sands for he pseudo-inverse of σ), is well defined and bounded on [0, T] R n Ω. 9

11 Remark 4.4 If lim xu n = x u, in C 0 ([0, T] R n ), n + hen Assumpion (15) is verified. This means ha he resul of Theorem 4.3 above applies if we know ha u is a srong soluion in a more resricive sense, i.e. subsiuing he poin 2 of Definiion 4.2 wih u n u in C 0 ([0, T] R n ), x u n x u in C 0 ([0, T] R n ), h n h in C 0 ([0, T] R n ), φ n φ in C 0 (R n ). This is a paricular case of our seing and i is he one used e.g in [18, 21] o ge he verificaion resul. We can say ha in hese papers a resul like Theorem 4.3 is proved under he assumpion ha u is a srong soluion in his more resricive sense. I is worh o noe ha in such simplified seing he proof of Theorem 4.3 follows simply by using sandard convergence argumens. In paricular here one does no need o use he Fukushima-Dirichle decomposiion presened in Secion 3 of [22]. So, from he mehodological poin of view here is a serious difference wih he resul of Theorem 4.3, see Secion 8 for commens. To apply Theorem 4.3 o our case we need firs o adap he noion of srong soluion and o rewrie he main assumpions. Le us give he following definiions. Definiion 4.5 We say ha v C 0 ([0, T] R n ) is a sric soluion of he HJB equaion (13) if v D (L 0 ) and (13) holds on [0, T] R n. Definiion 4.6 A funcion v C 0,1 ([0, T] R n ) is a srong soluion of he HJB equaion (13) if, seing h (, x) = H 0 (, x, x v (, x)), v is a srong soluion of he backward Cauchy problem L 0 v (, x) = h (, x), v (T, x) = φ(x), in he sense of Definiion 4.2. We now presen our verificaion heorem in he exended version, as announced in Secion 2. We need he following assumpion. Hypohesis 4.7 There exiss a funcion v C 0,1 ([0, T] R n ) which is a srong soluion of he HJB equaion (13) in he sense of Definiion 4.6 and is polynomially growing wih is space derivaive in he variable x. Moreover: (i) eiher we can choose he approximaing sequence (v n ) of Definiion 4.6 so ha for every 0 s T and for every admissible conrol z Z ad () s lim n + (ii) or he funcion x v n (r, y (r)) x v (r, y (r)),f 1 (r, y (r), z (r)) dr = 0, u.c.p., (, x, z) B 1 (, x) F 1 (, x, z), where B 1 sands for he pseudo-inverse of B, is well defined and bounded on [0, T] R n U. Remark 4.8 All he resuls below sill hold rue wih suiable modificaions if we assume ha: 10

12 he srong soluion v belongs o C 0 ([0, T] R n ) C 0,1 ([ε, T] R n ) for every small ε > 0; for some β (0, 1) he map (, x) β x v (, x) belongs o C 0 ([0, T] R n ). The proof of Theorem 4.9 in his case is a sraighforward generalizaion of he one presened here: we do no give i here o avoid echnicaliies since here we deal wih a model problem. In Secion 6 we will rea a case wih such difficuly. See also Remark 4.10 of [22] on his. We give now here he precise saemen of Theorem 2.7. Theorem 4.9 Assume ha Hypoheses 2.1, 2.2, 2.4 and 4.7 hold. Le H 0 CV, H0 be as in (4)-(5). Le v C 0,1 ([0, T] R n ) be a srong soluion of (13) and fix (, x) [0, T] R n which is polynomially growing wih is space derivaive in he variable x. Then (i) v V on [0, T] R n. (ii) If z is an admissible conrol a (, x) ha saisfies (seing y (s) = y (s;, x, z)) H 0 (s, y (s), x v (s, y (s))) = H 0 CV (s, y (s), x v (s, y (s));z (s)), (16) for a.e. s [, T], P almos surely, hen z is opimal a (, x) and v (, x) = V (, x). The proof of his heorem follows by he following fundamenal ideniy ha we sae as a lemma. Lemma 4.10 Assume ha Hypoheses 2.1, 2.2, 2.4 and 4.7 hold. Le v C 0,1 ([0, T] R n ) be a srong soluion of (13). Then, for every (, x) [0, T] R n and z Z ad () such ha J (, x; z) < + he following ideniy holds J (, x; z) = v (, x) [ +E H 0 (s, y(s), x v (s, y (s))) + HCV 0 (s, y(s), xv (s, y (s));z(s)) ] ds (17) where y(s) def = y (s;, x, z) is he soluion of (1) associaed wih he conrol z. Proof. For he sake of compleeness we firs show how he proof goes in he case when v C 1,2 ([0, T] R n ) and is a sric soluion of (13). We use Iô s formula applied o he funcion v (, x) by obaining, for every (, x) [0, T] R n, v (T, y (T)) v (, y ()) = x v (s, y (s)), B (s, y (s))dw (s) F 0 (s, y (s)), x v (s, y (s)) ds (18) F 1 (s, y (s),z (s)), x v (s, y (s)) ds s v (s, y (s))ds Tr [B (s, y (s)) xx v (s, y (s))b (s, y (s))] ds, 11

13 (which is exacly he decomposiion of Proposiion 2.4 of [22]) so, by aking expecaion of boh sides (his is finie since we have he polynomial growh assumpion) we ge he so-called Dynkin formula Ev (T, y (T)) = Ev (, y ())+E Now by (13) we ge, for every s 0, L 0 v (s, y (s))ds+e L 0 v (s, y (s)) = H 0 (s, y (s), x v (s, y (s))), F 1 (s, y (s),z (s)), x v (s, y (s)) ds. which yields Ev (T, y (T)) = Ev (, y ()) [ +E H 0 (s, y (s), x v (s, y (s))) + F 1 (s, y (s),z (s)), x v (s, y (s)) ] ds. Now since we assumed ha J (, x; z) < + and since Hypohesis 2.2 implies J (, x; z) > (his also follows observing ha he las line is no smaller han J (, x; z) and ha he firs line is finie by he polynomial growh of v), we add J (, x; z) o boh erms, use ha v (T, y (T)) = φ(y (T)) and ha y () = x o ge which is he claim (17). J (, x; z) = v (, x) + E H 0 (s, y (s), x v (s, y (s)))ds +E [ F 1 (s, y (s),z (s)), x v (s, y (s)) + l (s, y (s),z (s))] ds We now show how he proof goes when v C 0,1 ([0, T] R n ). The difference is due o he fac ha he erm I := s v (s, y (s))ds Tr [B (s, y (s)) xx v (s, y (s))b (s, y (s))] ds in he hird and fourh line of equaion (18) is no well defined now. However by Hypohesis 4.7 we know ha v is a srong soluion (in he sense of Definiion 4.2) of L 0 v (, x) = h 0 (, x), v (T, x) = φ(x), where we have se h 0 (, x) = H 0 (, x, x v (, x)). Then applying now Theorem 4.3 for he operaor L 0 (seing b = F 0, σ = B and b 1 = F 0 + F 1 ) we ge ha v (T, y (T)) = v (, y ()) + + x v (s, y (s)),b (s, y (s))dw (s) [ H 0 (s, y (s), x v (s, y (s))) + F 1 (s, y (s), z (s)), x v (s, y (s)) ] ds. Now, aking expecaion, adding and subracing J (, x; z) (which is a.s. finie using he same argumen for he smooh case), using ha v (T, y (T)) = φ(y (T)) and ha y () = x we ge [ ] T E l (s, y (s),z (s))ds + φ(y (T)) = v (, x) 12

14 [ +E H 0 (s, y (s), x v (s, y (s))) + HCV 0 (s, y (s), xv (s, y (s));z (s)) ] ds and he claim again follows. Proof of Theorem 4.9. By he definiion of H 0 and H 0 CV, for every (, x) [0, T] Rn, z Z ad () and for a.e. s T, he following inequaliy holds P-almos surely (seing y (s) = y (s;, x, z)) H 0 (s, y (s), x v (s, y (s))) + H 0 CV (s, y (s), xv (s, y (s)) ; z (s)) 0, (19) and hen by he fundamenal ideniy (17) i follows v (, x) J (, x; z) for every admissible z. This gives v V and so par (i) of he Theorem. Now consider an admissible conrol z such ha, for a.e. s [, T], P a.s. H 0 (s, y (s), x v (s, y (s))) = H 0 CV (s, y (s), xv (s, y (s));z (s)). (20) By he fundamenal ideniy (17) we have v (, x) = J (, x; z), which implies opimaliy and v (, x) = V (, x). Remark 4.11 To ge he mehodological novely in he above proof i is crucial o noe he following. Since we know ha u is he limi of classical soluions, he firs idea o approach he above proof (and ha has been used e.g. in he papers [18, 21]), is probably o prove he fundamenal ideniy for he approximaing soluions u n and hen pass o he limi for n. However if one ries o do his one needs o use he uniform convergence of he space derivaives x u n o x u. This is no needed wih our mehod, see also on his Remark 4.4. Le us go now o he proof of Verificaion Theorem 2.8 for he infinie horizon case. Se T = +, assume ha F 0, F 1 and B do no depend on ime and ha l (, x, z) = e λ l 1 (x, z) (λ > 0), φ = 0; hen we can se = 0 (since he dependence of he value funcion V on becomes rivial in his case) and he HJB equaion for V 0 becomes an ellipic PDE. We define he operaor L 0 : D (L 0 ) C 0 (R n ) C 0 (R n ), D (L 0 ) = C 2 (R n ), L 0 u (x) = F 0 (x), x u (x) Tr [B (x) xx u (x) B (x)] and we rewrie he HJB equaion (10) as λu (x) + L 0 u (x) + H 1 (x, x u (x)) = 0, x R n. (21) where H 1 is defined in (11). We now define srong soluions for such equaions. Consider firs, for h C 0 (R n ) he inhomogenous ellipic problem λu (x) + L 0 u (x) + h (x) = 0, x R n. (22) Definiion 4.12 We say ha u is a sric soluion o he ellipic problem (22) if u D (L 0 ) and (22) holds. 13

15 Definiion 4.13 We say ha u is a srong soluion o he ellipic problem (22) if here exiss a sequence (u n ) D (L 0 ) and a sequence (h n ) C 0 (R n ), such ha 1. For every n N u n is a sric soluion of he problem λu n (x) L 0 u n (x) = h n (x), x R n. 2. The following limis hold u n u in C 0 (R n ), h n h in C 0 (R n ). For funcions u C 1 (R n ) ha are srong soluions of he Cauchy problem (22) he resul of Theorem 4.3 sill holds: i is indeed a simpler case. Now we go o srong soluions of he HJB equaion (21). Definiion 4.14 A funcion v C 1 (R n ) is a srong soluion of he HJB equaion (21) if, seing h 0 (x) = H 1 (x, x v (x)), v is a srong soluion of he linear problem in he sense of Definiion Assume he following. λv (x) + L 0 v (x) = h 0 (x), Hypohesis 4.15 There exiss a funcion v C 1 (R n ) which is a srong soluion of he HJB equaion (21) in he sense of Definiion Moreover: (i) eiher we can choose he approximaing sequence (v n ) of Definiion 4.14 so ha for every s 0 and for every admissible conrol z Z ad (0) s lim n + 0 (ii) or he funcion x v n (y (r)) x v (y (r)),f 1 (y (r), z (r)) dr = 0, u.c.p., (x, z) B 1 (x)f 1 (x, z), where B 1 sands for he pseudo-inverse of B, is well defined and bounded on R n U. The precise saemen of he verificaion heorem is he following. Theorem 4.16 Assume ha Hypoheses 2.1, and 4.15 hold. Assume also ha l 1 is coninuous and bounded and ha Hypohesis 2.4 hold rue wih H 1 in place of H 0. If v is a bounded srong soluion of he HJB equaion (21) and v C 1 (R n ) hen v V 0 on R n. Morerover if z is an admissible conrol a (0, x) ha saisfies (seing y (s) = y (s; 0, x, z)) H 1 (y (s), x v (y (s))) = H 1 CV (y (s), x v (y (s));z (s)) for a.e. s [0, + ), P almos surely, hen z is opimal and v (x) = V 0 (x). Proof. The proof goes along he same lines as he finie horizon case. Firs we observe ha he boundedness of he daum l 1 implies ha also V 0 is bounded and ha for every x R n, z Z ad (0) he funcional J (0, x; z) is finie. Then we prove he following fundamenal ideniy J (0, x; z) = v (x) + E + 0 e λs [ H 1 (y(s), x v (y (s))) + H 1 CV (y(s), xv (y (s));z(s)) ] ds. 14

16 To do i we apply Theorem 4.3 for he operaor L 0 = + L 0 (so b = F 0, σ = B and b 1 = F 0 + F 1 ) bu aking = 0 and replacing v (s, x) by e λs v (x) for s 0. Observe firs ha, if v C 2 (R n ) hen he funcion w (, x) = e λ v (x) solves, for 0, x R n he equaion w (, x) + L 0 w (, x) = e λ H 1 (x, x v (x)). (23) Moreover if v C 1 (R n ) is a srong soluion of (21), also w (, x) is a srong soluion of (23) and saisfies he assumpions of Theorem 4.3. So we ge for every T 1 > 0 e λt1 v (y (T 1 )) = v (y (0)) e λs x v (y (s)),b (y (s))dw (s) 1 + e λs [ H 1 (s, y (s), x v (s, y (s))) + F 1 (s, y (s),z (s)), x v (s, y (s)) ] ds. 0 Now, aking expecaion, adding and subracing E 1 0 e λs l 1 (y (s),z (s))ds (which is a.s. finie by he boundedness of l 1 ) and using ha y (0) = x we ge [ ] T1 E e λs l 1 (y (s),z (s))ds + e λt1 v (y (T 1 )) 0 = v (x) + E 1 0 e λs [ H 1 (y (s), x v (y (s))) + H 1 CV (y (s), x v (y (s));z (s)) ] ds and he claim again follows aking he limi for T 1 + and using he boundedness of v. The res of he proof is exacly he same as in he finie horizon case. Remark 4.17 I mus be noed ha he assumpion on he boundedness of l 1 and v is made for simpliciy of exposiion. Wha is really needed in his infinie horizon case is ha J is well defined for each z Z ad (0) and ha lim T1 + e λt1 v (y (T 1 )) = 0 for every z Z ad (0). This allows o pass o he limi in he laer formula. In some cases his can be checked direcly, in oher cases much weaker condiions can be imposed (e.g. sublineariy of l 1 and esimaes on he soluion y as T 1 + ). 5 Necessary Condiions and Opimal Feedback Conrols Here we wan simply o show ha under addiional assumpions (mainly abou exisence and/or uniqueness of he maximum of he Hamilonian and of he soluion of he closed loop equaion) we can ge necessary condiions and opimal feedback conrols. This is a consequence of he verificaion Theorems 4.9 and 4.16 which has some imporance for applicaions. For he sake of breviy we give he resuls only for he finie horizon case observing ha compleely analogous resuls hold rue for he infinie horizon case. We sar by making a remark abou he so-called weak formulaion of a sochasic conrol problem. Remark 5.1 The seing inroduced in Secion 2 corresponds o he so-called srong formulaion of a sochasic conrol problem. For cerain purposes (namely he exisence of opimal feedbacks) i is convenien o consider he weak formulaion, leing he sochasic basis vary. In such formulaion one considers as he se Z ad () of admissible conrols as he se of 5-uples (Ω, F, P, W, z) such ha (Ω, F, P) is a complee filered probabiliy space wih he filraion F saisfying he usual condiions, 15

17 W is an E valued m-dimensional Brownian moion on [, T], he process z is measurable, (F s )-adaped and a.s. locally inegrable. We will use he noaion (Ω, F, P, W, z) Z ad () and we will call his conrol sraegies weakly admissible (weakly opimal when hey are opimal). When no ambiguiy arises we will leave aside he probabiliy space and he whie noise (regarding i as fixed) and simply consider z Z ad (). We will specify when he weak concep of admissible conrol is needed. The same will be done in Secions 6 and 8. One can see e.g. [39, p.64], [13, pp.141, 160] for commens on hese formulaions. We sar recalling a case when he sufficien condiion of he verificaion Theorem 4.9 becomes also necessary. Proposiion 5.2 Assume ha he hypoheses of Theorem 4.9 hold rue. We also assume ha he value funcion V is a srong soluion of he HJB equaion. Then an admissible conrol z Z ad () is opimal a (, x) [0, T] R n if and only if (16) holds wih V in place of v. Proof. The fundamenal ideniy in his case gives J (, x; z) = V (, x) [ +E H 0 (s, y (s), x V (s, y (s))) HCV 0 (s, y (s), x V (s, y (s));z (s)) ] ds and his immediaely gives he claim. Remark 5.3 One case where he above Proposiion 5.2 can be applied is when: i is known ha V is he unique viscosiy soluion of he HJB equaion; i is known ha srong soluions are also viscosiy soluions. This happens e.g. in he example of Secion 7. We now pass o he exisence of feedbacks. Firs we recall he definiion of (weakly) admissible feedback map. Definiion 5.4 A measurable map G : [0, T] R n U is a (weakly) admissible feedback map if for every (, x) [0, T] R n he closed loop equaion (s [, T]) y(s) = x + s F 1 (r, y (r), G(r, y (r)))dr + s F 0 (r, y (r)) dr + s B (r, y (r))dw(r), (24) admis a (weak) soluion y = y G,,x for every (, x) [0, T] R n and he conrol sraegy z G,,x (s) = G(s, y G,,x (s)) is (weakly) admissible. If we have a (weakly) admissible feedback map G hen he conrol sraegy z G,,x (s) = G(s, y G,,x (s)) is by definiion (weakly) admissible and so, given G, we have, for every (, x) [0, T] R n an admissible (weak) couple (z G,,x, y G,,x ). Our goal is o find an admissible feedback map G such ha, for every (, x) [0, T] R n, he sraegy z G,,x is (weakly) opimal. Such G will be called a (weakly) opimal feedback map. 16

18 Proposiion 5.5 Assume ha he Hypoheses of Theorem 4.9 hold rue. Assume also ha: (i) he maximum of he Hamilonian (4) exiss and i is possible o define a measurable map G 0 : [0, T] R n R n U such ha G 0 (, x, p) argmin z H CV (, x, p; z) for every (, x, p) [0, T] R n R n ; (ii) he map G(, x) = G 0 (, x, x v (, x)) is a (weakly) admissible feedback map. Then for every (, x) [0, T] R n, z G,,x is a (weakly) opimal conrol, y G,,x is he corresponding (weakly) opimal sae and v = V. Finally if arg min z U H CV (, x, v x (, x); z) is always a singleon and he closed loop equaion (24) admis a unique (weak) soluion, hen he opimal conrol is (weakly) unique. Proof. This is an obvious consequence of he verificaion Theorem 4.9. Remark 5.6 We observe ha if we have exisence of he (weak) soluion of he closed loop equaion (24) for every srong soluion of he HJB equaion (13), hen we auomaically have ha he srong soluion is unique. Remark 5.7 We finally observe ha Proposiions 5.2 and 5.5 holds also in he infinie horizon case wih obvious changes. We will use he infinie horizon version of Proposiion 5.5 in he sudy of he one dimensional example in Secion 7. 6 Applicaion 1: a class of exi ime conrol problems wih non degenerae diffusion Here we show ha our echnique for proving verificaion heorems works for a family of sochasic opimal conrol problems wih exi ime and nondegenerae diffusion where he soluions of he associaed HJB equaion are no known o be C 1, The problem Le firs R n be he sae space, R n be he space of noises and U (a given subse of a Polish space) be he conrol space. Le T be a fixed finie horizon, (Ω, F, P) be a given sochasic basis (where F sands for a given filraion (F s ) s [0,T] saisfying he usual condiions), W be a cylindrical Brownian moion wih values in R n adaped o (F s ) s [0,T]. Consider a sochasic conrolled sysem in R n wih fixed finie horizon T and iniial ime [0, T) governed by he sae equaion { dy (s) = [F0 (s, y (s)) + F 1 (s, y (s),z (s))] ds + B (y (s))dw(s), s [, T] (25) y () = x. We consider an open bounded domain O R n wih uniformly C 2 boundary (see e.g. [30, pp.2-3] for he definiion). The iniial daum x belongs o O and we call τ O he firs exi ime of he process y from his open se, i.e. τ O (ω) = inf {s > : y (s; ω) O c }. Moreover F 0 : [0, T] Ō Rn, F 1 : [0, T] Ō U Rn, B : [0, T] Ō L(Rn ; R n ), (Ō sands for he closure of he se O) saisfy 17

19 Hypohesis 6.1 F 0 and F 1 are coninuous and bounded. Hypohesis 6.2 B is ime independen, uniformly coninuous and bounded and nondegenerae i.e. here exiss λ 0 > 0 such ha λ 1 0 ξ n [ B (x)b T (x) ] ξ i,j iξ j λ 0 ξ (, x) [0, T] Ō, ξ Rn. i,j=1 Remark 6.3 Firs noe ha he above Hypohesis 6.2 implies ha B is inverible wih bounded inverse, so we fall in he Hypohesis 4.7-(ii). Moreover i is possible o exend our resuls o he case when B is ime dependen, provided suiable regulariy condiions are saisfied. Such condiions are needed o apply he resuls of semigroup heory in he subsecion below and subsanially require ha he domain of he operaor L ()u = 1 2 Tr[B (, x)b (, x) xx u] be consan and he coefficien B be Hölder coninuous in ime, see on his [3, 4] and [30, Secion 3.1.2]. We leave aside hese assumpions o keep a more simplified seing where, anyway, C 1,2 regulariy does no hold in general. Abou he conrol sraegy z : [, T] Ω U we assume ha i belongs o a given se Z ad () of sochasic processes defined on [, T] Ω wih values in a fixed Polish space U. More precisely we will assume he following. Hypohesis 6.4 The conrol space U is a Polish space, while Z ad () is he space of all measurable processes z : [, T] Ω U adaped o he filraion F. Remark 6.5 The above seing corresponds o he so-called srong formulaion of a sochasic conrol problem. In his problem, since he sae equaion admis only weak soluions in general (see nex proposiion) we need o consider he weak formulaion, leing he sochasic basis o vary, see Remark 5.1. To avoid heavy noaion we will keep he same symbols as in he srong formulaion, as we did in Secion 5. The following Proposiion can be proved in he same way as in [37], even if he drif is random. In fac Theorem herein makes use of Girsanov ransformaion o eliminae he drif; he same can be done here. Then we can apply Corollary and Theorem of [37]. Proposiion 6.6 Assume ha Hypoheses 6.1, 6.2 hold. Then, for all z Z ad (), equaion (25) has a weak soluion y( ;, x, z) CF 0 (, T; X). This soluion is unique in he sense of probabiliy law. Remark 6.7 In he case of dimension 1, no coninuiy on he diffusion coefficiens is required, see [37] Exercise a page

20 We now consider he following sochasic opimal conrol problem wih exi ime. Minimize he cos funcional J (, x; z) = [ τo T E l (s, y (s;, x, z),z (s))ds (26) +I {τo<t }ψ (τ O, y (τ O ;, x, z)) + I {τo T }φ(y (T;, x, z)) ], over all conrols z Z ad (). Here y ( ;, x, z) is he soluion of he equaion (25) and we assume ha l, ψ, φ saisfy Hypohesis 6.8 l C 0 ( [0, T] Ō), φ C 0 ( Ō ) ψ C 1+β 2,1+β ([0, T] O) (for some β > 0), ψ (T, x) = φ(x) on O. Remark 6.9 The above hypohesis is needed o apply he resuls of semigroup heory in he subsecion below. In paricular, under Hypohesis 6.8 he operaor Lu = 1 2 Tr[B (x)b (x) xx u] (wih 0 Dirichle boundary condiions) generaes an analyic semigroup and he associaed boundary value problem in O is well posed (see [30, Ch.5]). The value funcion of his problem is defined as V (, x) = inf {J (, x; z) : z Z ad ()} (27) and a conrol z Z ad () and such ha V (, x) = J(, x; z ) is said o be opimal wih respec o he iniial ime and sae (, x). The corresponding HJB equaion is where wih v (, x) Tr [B (x) xxv(, x)b (x)] = F 0 (, x), x v (, x) + H 0 (, x, x v (, x)), [0, T], x O, v (T, x) = φ(x), x O, v (, x) = ψ (, x), [0, T], x O, being he curren value Hamilonian. (28) H 0 (, x, p) = inf z U H0 CV (, x, p; z), (29) H 0 CV (, x, p; z) = F 1 (, x, z),p + l (, x, z), Proposiion 6.10 Under Hypoheses 6.1, 6.2, 6.4, 6.8, he Hamilonian H 0 (, x, p) defined in (29) is coninuous in [0, T] Ō Rn. Moreover here exiss a consan C > 0 such ha H 0 (, x, p) H 0 (, x, q) C p q H 0 (, x, p) C (1 + p ) (30) Proof. I is enough o apply he definiion of he Hamilonian and use he coninuiy and he boundedness of F 1 and l. 19

21 6.2 Srong soluions of he HJB equaion The HJB equaion (28) above is a semilinear parabolic equaion wih coninuous coefficiens. Since he second order erm is nondegenerae (Hypohesis 6.2) one expecs inerior regulariy resuls for he soluion even if he boundary daa are merely coninuous, as i is under our assumpions. In paricular he following resul, aken from [9], Theorems 9.1 and 9.2, apply. Theorem 6.11 Under Hypoheses 6.1, 6.2, 6.4, 6.8, here exiss a viscosiy soluion u of (28) and u belongs o he space C 1+α 2,1+α loc ([0, T) O) C ( 0 [0, T] Ō) for some α (0, 1). We noe also ha i is no known if he soluion is classical in he inerior. This is known, using heorems of [5], if he daa are supposed o be Hölder coninuous in (, x), which we do no assume here. Moreover even in he linear case, if he coefficiens are only coninuous, soluions are only W 2,p, so hey are also C 1,α. An example in his direcion is in [16, Ch.4]. This means ha we are exacly in he case where i makes sense o apply our echnique o prove verificaion heorems: no classical soluion bu soluions (in a generalized sense) wih a leas C 0,1 regulariy. Once his is clear, we need o check if our soluions (ha exiss a leas in he viscosiy sense hanks o he above Theorem 6.11) are also srong soluions in he sense of Definiion 4.6 (suiably modified o ake care of he boundary daum ψ). Such a resul is no available in he lieraure in his form bu i can be easily deduced using he resuls on analyic semigroups conained e.g. in [30]. We explain here below how his can be done in he case when ψ = 0 (he general case can be reaed wih he ideas explained in [30, Secion 5.1.2]). In his case he operaor L : D (L) C 0 ( Ō ) C 0 ( Ō ), { D (L) = η p 1 W 2,p loc (O) : η, Lη ( C0 Ō ) }, η O = 0, (Lη)(x) = 1 2 Tr[B (x) xxη(x)b (x)], generaes an analyic semigroup { e L, 0 }, [30, p.97, Corollary ]. Moreover, given any iniial daum φ C ( 0 Ō ) and any funcion H 0 coninuous in [0, T] Ō Rn and saisfying (30) we can apply a modificaion of he argumen used o prove Proposiion in [30, p.281] o ge exisence and uniqueness of a soluion u of he inegral equaion ( ) u (, x) = e (T )L φ (x) (31) + ( e (T s)l [ F 0 (s, ), x u (s, ) + H 0 (s,, x u (s, )) ]) (x) ds, which can be considered as an inegral form of he PDE (28) when ψ = 0 (in he general case one needs o lif he funcion ψ ino he equaion obaining an exra erm in (31), see on his [30, Remark , p.195]) wrien using he variaion of consans formula. More precisely we have he following definiions and resuls. Definiion 6.12 We say ha a funcion w belongs o he space Σ ( 1,α [0, T] Ō) if w C ( 0 [0, T] Ō), w is Fréche differeniable in x O, x w C ( 0 [0, T ε] Ō) for every ε (0, T), and sup (T ) α x w (, x) < +. (,x) [0,T) Ō ( Definiion 6.13 A funcion u Σ 1, 1 2 [0, T] Ō ) ha saisfies he inegral equaion (31) for every (, x) [0, T] Ō is called a mild soluion of he HJB equaion (28). 20

22 Theorem 6.14 Assume ha B C ( 0 Ō ), F 0 C ( 0 [0, T] Ō), φ C ( 0 Ō ) and ψ = 0. Moreover le H 0 be coninuous ( in [0, T] Ō Rn and saisfies (30). Then here exiss a unique mild soluion u Σ 1, 1 2 [0, T] Ō ) of he HJB equaion (28). Proof. The proof is a sandard applicaion of he conracion mapping principle, see [30, Proposiion 7.3.4] and also [6, 18]. Now we wan o show ha such a mild soluion is a srong soluion in he sense ha i can be seen as he limi of smooh soluions. We rewrie here he definiion of srong soluion for our case since i is slighly weaker han he one used in Secion 4 due o he possible singulariy of he spaial gradien a = 0, see Remark 4.8. Definiion 6.15 Le φ C ( 0 Ō ) ( and ψ C 0 ([0, T] O). A funcion u Σ 1, 1 2 [0, T] Ō ) is a srong soluion of he equaion (28) if i saisfies he boundary and final condiions u (T, x) = φ(x), x O, u (, x) = ψ (, x), [0, T],s O, and if, for every ε > 0, i is a srong soluion (in he sense of Definiion 4.2) of he linear parabolic problem w + Lw = h, [0, T], x O, w (T ε, x) = u (T ε, x), x O, where h = [ F 0 (, ), x u (, ) + H 0 (,, x u (, )) ], i.e. if, for every ε > 0 here exis sequences u ε n, hε n, φε n such ha 1. for every n u ε n is a sric soluion (i.e. i belong o ( C1,2 [0, T ε] Ō) and saisfy he equaliies on [0, T ε] Ō) of he approximaing problem v + Lv = h ε n, v (T ε, x) = φε n (x) ; 2. u ε n converges o u uniformly in [0, T ε] Ō, as n + ; 3. h ε n converges o h uniformly in [0, T ε] Ō, as n + ; 4. φ ε n u (T ε, ) converges o zero uniformly in Ō, as n +. Given he above Definiion 6.15 we can apply direcly Proposiion (see also Theorem ) of [30] o ge he following. Theorem 6.16 Under he same assumpions of Theorem 6.14 he mild soluion of (28) is also srong. Proof. I is enough o apply Proposiion and Theorem of [30] o he nonhomogenous linear parabolic problem v + Lv = h, v (T, x) = u (T ε, x), where we se h (, x) = F 0 (, x), x u (, x) + H 0 (, x, x u (, x)). In fac for every ε > 0 such funcion h belongs o he space C 0 ( [0, T ε] ; C 0 ( Ō )) = C 0 ( [0, T ε] Ō). 21

23 Remark 6.17 In he case when ψ is no 0 he above resuls sill hold rue using he same echniques shown in [30], Theorem , and We do no do i here for simpliciy of exposiion. Remark 6.18 Suppose ha he operaor L can be wrien in divergence form and ha he coefficiens are only Borel measurable and no necessarily coninuous; suppose moreover ha he diffusion coefficiens are lower and upper bounded by a consan. Then, he semigroup has a densiy wih respec o he Lebesgue measure and i fulfills he classical Aronson esimaes, see for insance [1, 36]. Fukushima - Dirichle decomposiion for mild or weak soluions o equaions of ype (28) were reaed by [2, 26, 35]. Using such kind of resuls or possible generalizaions in he spiri of [22], one could esablish verificaion heorems relaed o opimal conrol problems even in ha framework. 6.3 The verificaion heorem Now we prove a verificaion heorem for he opimal conrol problem above (25) - (26). The proof is a modificaion of he proof given in Secion 4. The main differences are due o 1. a singulariy of he firs derivaive and so a differen definiion of srong soluion; 2. he consrain on he se O and so he presence of boundary daa in x (exi ime). The saemen of he verificaion heorem in his case is he following ( Theorem 6.19 Assume ha Hypoheses 6.1, 6.2, 6.4, 6.8, hold. Le v Σ 1, 1 2 [0, T] Ō ) be a srong soluion of (28). Then (i) v V on [0, T] Ō. (ii) If z is an admissible conrol a (, x) [0, T] O ha saisfies (seing y (s) = y (s;, x, z)), P-a.s., H 0 (s, y (s), x v (s, y (s))) = H 0 CV (s, y (s), xv (s, y (s));z (s)), for a.e. s [, T τ O ], hen z is opimal a (, x) and v (, x) = V (, x). The proof of his heorem follows as usual by he following fundamenal ideniy ha we sae as a lemma. ( Lemma 6.20 Assume ha Hypoheses 6.1, 6.2, 6.4, 6.8, hold. Le v Σ 1,1 2 [0, T] Ō ) be a srong soluion of (28). Then, for every (, x) [0, T] Ō and z Z ad () he following ideniy holds J (, x; z) = v (, x) τo T [ +E H 0 (s, y(s), x v (s, y (s))) + HCV 0 (s, y(s), x v (s, y (s));z(s)) ] ds, (32) where y(s) def = y (s;, x, z) is he soluion of (25) associaed wih he conrol z. 22