Sufficient Optimality Condition for a Risk-Sensitive Control Problem for Backward Stochastic Differential Equations and an Application

Transcription

1 Jornal of Nmerical Mathematics and Stochastics, 9(1) : 48-6, 17 JNM@S Eclidean Press, LLC Online: ISSN Sfficient Optimality Condition for a Risk-Sensitive Control Problem for Backward Stochastic Differential Eqations and an Application A. CHALA Laboratory of Applied Mathematics, University of Mohamed Khider, POBox 145, Biskra 7, Algeria, adelchala@yahoo.fr Abstract. We stdy the risk-sensitive optimal control by sing the extending part of the same problem with a backward stochastic differential eqation as has been reported in []. We also establish sfficient optimality conditions, by means of the convexity propriety of pertaining fnctions. he control domain is assmed to be convex, and the generator coefficient of the associated system is allowed to depend on the control variable. An example is provided to illstrate or main reslt for a risk-sensitive control problem nder linear stochastic dynamics with an exponential qadratic cost fnction. Key words : Backward Stochastic Differential Eqations, Risk-Sensitive, Sfficient Optimality Conditions, Variational Principle, Logarithmic ransformation. AMS Sbject Classifications : 93E, 6H3, 6H1, 91B8 1. Introdction In this paper, we investigate sfficient optimality conditions for the system driven by a backward stochastic differential eqation in a risk sensitive model for the performance fnctional. In particlar, we aim at a certain extension of an initial work, reported by Chala in [], where a necessary optimality condition, of the Pontryagin s maximm principle type, for risk-sensitive performance fnctionals, had been established. In [] the problem was in fact solved by sing an approach developed by Djehiche et al. [1]. For more details, the interested reader is referred to the papers [, 1], and to references therein. *his work is partially spported by the Algerian Biskra CNEPRU project No. CL3UN

2 49 A. CHALA In a risk-sensitive control problem, the system is governed by the nonlinear backward stochastic differential eqation dy v t ft,y v t,z v t,v t dt z v t dw t, y v a. he criterion to be minimized, with an initial risk-sensitive fnctional cost, can be defined as follows J v E e y v ft,y v t,z v t,v tdt. A control is called optimal if it solves J inf v U J v. A stochastic maximm principle (SMP in short) for risk-sensitive optimal control problems for Markov diffsion processes, with an exponential integral performance fnctional was obtained in [6] by relating the SMP to the dynamic programming principle (DPP in short). Specifically, the athors of [6] have sed the first order adjoint process as the gradient of the vale fnction of the control problem. Sch a relationship holds, however, only when the vale fnction is smooth (see Assmption (B4) in [6]). Moreover, by tilizing the smoothness assmption, the two papers [8] and [9], have sed the approach noted above, bt extended to jmp processes and to a linear system with an application to finance. he existence of an optimal soltion to the posing problem, of risk-sensitive control, is investigated in this paper, and some sfficient optimality conditions for the pertaining model are established. A few sentences are in order here abot paper []. In this paper, we have reformlated the same problem in terms of an agmented state process and a terminal payoff problem. An intermediate stochastic maximm principle is then obtained by applying the SMP of [11], heorem 3.1) for loss fnctionals withot a rnning cost. hen, we transformed the intermediate first order adjoint processes to a simpler form by assming convexity for the controls. Necessary optimality conditions were then established by sing the logarithmic transform introdced in [3]. In this respect, the method of Lim and Zho [6], shows in fact that it sffices to se a generic sqare-integral martingale to transform the pair p 1,q 1 into the adjoint process p 1 t,, where the process p 1 t is still a sqare-integrable martingale, which wold mean that p 1 t p 1, and is eqal to the constant E p 1. Bt this generic martingale needs not be related to the adjoint process pt, as in [6]. Instead, it will be part of the adjoint eqation associated with the risk-sensitive SMP (see theorem 3., bellow, or heorem 3., page 49 in []). he rest of this paper is organized as follows. In Section, we give the precise formlation of the problem and introdce the risk-sensitive model, together with the varios assmptions, as in [], sed throghot the paper. In section 3, we give only the important reslts and stdy or system of backward SDE. he necessary optimality condition for the backward differential eqation, with risk-sensitive performance cost, is also given here. Or main reslt : the sfficient optimality conditions for the risk-sensitive control problem, nder an additional hypothesis, is provided in section 4. In sections 5, we finish the paper by giving an application to a qadratic stochastic linear control problem. In the conclsion and otlook, of section 6, we give a discssion of ftre research challenges, and otline the relationship between this paper and other related works.

3 Sfficient Optimality Condition for a Risk-Sensitive Control Problem for BSDEs 5. Formlation of the Problem Let,F,F W t t,,p be a probability space satisfying the sal conditions, in which a one dimensional Brownian motion W W t : t is defined. We assme that F W t t, is defined by t, F W t W r ; for any r,t N, wheren denotes the totality of P nll sets. Let M,;R denotes the set of one dimensional jointly measrable random processes t,t,, which satisfy: i : E t dt, ii : t is F W t t, measrable, for any t,. We denote similarly by S,;R the set of continos one dimensional random processes which satisfy: i : E sp t, ii : t is F W t t, measrable, for any t,. t We also let be a strictly positive real nmber and U to be a nonempty sbset of R. Definition.1. An admissible control v is a process with vales in U sch that E vt dt. WedenotebyUthe set of all admissible controls. he set of all admissible control shold be convex. For any v U, we consider the following backward stochastic differential eqation system dy v t gt,y v t,z v t,v t dt z v t dw t, 1 y v a, where g :, U, and the terminal condition a R is a random variable F -measrable. We define the criterion to be minimized, with initial risk-sensitive performance cost, as follows J v E e y v ft,y v t,z v t,v tdt, where is the risk-sensitive index, and :, f :, U. he control problem is to minimize the fnctional J over U, if U is an optimal control, that is J inf J v. 3 v U Assmption.1. Assme that gt,, M,;R, and that there exists c, sch that gt,y 1,z 1 t,y,z c y 1 y z 1 z. Proposition [7].1. For any given admissible control v., we let assmption.1 to hold. hen the backward SDE 1 has a niqe soltion. A control that solves the problem 1,,3 is called optimal, and that applies to the

4 51 A. CHALA case where the necessary optimality conditions for risk-sensitive model as given in []. Or goal here is to establish the risk-sensitive sfficient conditions of optimality in the posing stochastic control problem. For that we need the assmption that follows. Assmption.. g, f and are continosly differentiable with respect to y,z,v. Under the above assmption, for every v U eqation 1 has a niqe strong soltion, and the cost fnction J is well defined from U into. For more details, the reader can be referred to the famos paper [11], and also to Yong s book [1]. 3. Risk-Sensitive Stochastic Maximm Principle of Backward ype Control his section is basically taken from the paper []. We start by introdcing an axiliary state process t v, which is soltion of the following forward SDE: d t v ft,y t v v,v t dt, v. he backward type control problem 1,,3 is eqivalent to inf v U E e y v, sbject to d v t ft,y v t,z v t,v t dt, dy v t gt,y v t,z v t,v t dt z v t dw t, v, y v a. Recall that A : e y ft,y t,z t, tdt. 5 We can pt Θ y ft,yt, t dt, the risk-sensitive loss fnctional is given by Θ : 1 loge exp y ft,yt, t dt 1 logeexpθ. When the risk-sensitive index is small, the loss fnctional Θ can be expanded as EΘ VarΘ O, where, VarΘ denotes the variance of Θ.If, the variance of Θ, as a measre of risk, improves the performance Θ, in which case the optimizer is called risk seeker. Bt, when, the variance of Θ worsens the performance Θ, in which case the optimizer is called risk averse. he risk-netral loss fnctional EΘ can be conceived as a limit of risk-sensitive fnctional Θ when. For more details, the reader can conslt the papers [1, 3, 1]. Notation 3.1. he following notations shall be sed throghot the paper. For every g, f 4

5 Sfficient Optimality Condition for a Risk-Sensitive Control Problem for BSDEs 5 respectively, we define t t,y t,z t, t, t t,y t,z t,v t t,y t,z t, t, t t, y, z, where v t in an admissible control from U. If we assme that assmptions.1-. hold, then the adjoint eqation can be fond by sing the stochastic maximm principle for risk-netral of forward-backward type control, [11], with agmented state dynamics,y,z. here exist niqe F t adapted pairs of processes p 1,q 1,p,q, which solve the following system of the compact-backward stochastic differential eqation: dpt dp 1 t dp t f y t g y t p 1 t p t dt q 1 t dw t, H zt 6 with E p 1 A, p y y A, i 1 sp p i t q1 t dt. t We sppose here that H is the netral Hamiltonian associated with the optimal state dynamics,y,z, and that the pair of adjoint process pt, qt 6, is given by: H t,y t,z t, t, pt : ftp 1 t gtp t. he following theorem is called the stochastic maximm principle for risk-netral forward-backward type control from. heorem 3.1. Assme that assmptions.1-. hold. If,y,z is an optimal soltion of the risk-netral control problem 4, then there are two pairs of F t adapted processes p 1,q 1, and p,q that satisfy 6, sch that H t, for all U, almost every t,, and P almost srely, where H t : H t, t,y t,z t,v, pt H t, t,y t,z t, t, pt. Proof. he proof of (7) can be fond in [11]. 7 In this section, we will state, withot proof, the necessary conditions of optimality for the

6 53 A. CHALA system driven by a backward stochastic differential eqation with a risk sensitive performance type. For this end, let s smmarize some of lemmas that are needed later. Lemma [] 3.1. V solves the following linear backward SDE dv t ltv tdw t, V A. 8 W Hence, the process defined on,f, F t by L t, where V t V t,,p exp t lsdws t ls ds : L t, t. 9 is a niformly bonded F t martingale. Frthermore, we may define the adjoint eqation adopted to this kind of problem by invoking the next lemma. Lemma [] 3.. he risk-sensitive adjoint eqation satisfied by p, q and V,l becomes d p t H y tdt H z tdw t, dv t ltv tdw t, V A, p y y. he soltion p, q,v,l of the system 1 is niqe, sch that E where sp pt sp V t qt lt dt, 11 t t 1 H t,y t, p q,v,l gt zlt p t ft. 1 heorem 3.. (Risk-sensitive stochastic maximm principle) Assme that assmption.1 holds. If y t, t is an optimal soltion of the risk-sensitive control problem 1,, 3, then there exist two pairs of F t -adapted processes V,l, p,q that satisfy 1 11 sch that H t, 13 for all U, almost every t and P-almost srely, where the Hamiltonian H associated with 4, is given by H t, t,y t, pt, t V t H t,y t, p t q t,v t,lt, t, and H is the risk-sensitive Hamiltonian given by 1.

7 Sfficient Optimality Condition for a Risk-Sensitive Control Problem for BSDEs Sfficient Optimality Conditions for a Risk-Sensitive Performance Cost In this section, we stdy when the necessary optimality conditions 13 become sfficient. For any v U, wedenotebyy v t,z v t the soltion of eqation 1 controlled by v, to state the reslt that follows. heorem 4.1. (Sfficient optimality conditions) Assme that the fnctions, and y v t,z v t H t,y v t,z v t, p 1 t, p t,v t,lt,vt are convex, and that for any v U, y v aisan one dimensional F measrable random variable sch that E a. hen, is an optimal soltion of the control problem 1,, 3, if it satisfies 13. Proof. Let be an arbitrary element of U ( candidate to be optimal). For any U, wehave J t J t E e y 1 E e y E e y 1 y y y 1 y. Becase is convex, we can write J t J t E e y 1 E e y y y y 1 y. It follows from 6, and5, that p 1 A,p y y A, and then we have J t J t Ep 1 1 Ep y 1 y. Applying Itô s formla to p 1 t 1 t t,andp ty 1 t y t lead to Ep 1 1 E p1 tf t f tdt, 14 and Ep y 1 y n E fy tp 1 t gy tp ty 1 t y t dt E fz tp 1 t gz tp tz 1 t z t dt E p1 g t g tdt. By sbstitting the above two formlas in 14, weget

8 55 A. CHALA J t J t E H t,yt,p 1 t,p t, t H t,y t,z t,p 1 t,p t, t dt E Hyt,yt,z t,p 1 t,p t, ty 1 t y t dt t E t Hz t,yt,p 1 t,p t, tz t z t dt. 15 Since H is convex with respect to y t, t, then by considering the gradient of H, evalated at y t,v, and the necessary optimality conditions 13 and 15, we arrive at H t,y t,p 1 t,p t, t H t,y t,p 1 t,p t, t H y t,yt,p 1 t,p t, ty t y t H z t,yt,p 1 t,p t, tz t z t H t,yt,z t,p 1 t,p t, t t t, or eqivalently at E Ht,yt,z t,p 1 t,p t, t t tdt E H t,yt,p 1 t,p t, t H t,y t,p 1 t,p t, t dt E Hy t,yt,p 1 t,p t, ty t y t dt E Hz t,yt,p 1 t,p t, tz t z t dt. hen from 15, and also according to the necessary optimality conditions 7, we observe that J t J t E H t,yt,p 1 t,p t, t t tdt. Which means that J t J t. Here the proof completes. 5. Application to the Qadratic Risk-Sensitive Linear Control Problem he basic secrities consist of two assets; one of them is risky means that the money

9 Sfficient Optimality Condition for a Risk-Sensitive Control Problem for BSDEs 56 market is bond with price P governed by the eqation dp t P t r t dt, where r t is the short rate. he price process P for the stock is modeled by the linear stochastic differential eqation dp t P t b t dt t dw t, where W is a standard Brownian motion on, defined on a probability space,f,p. We frther assme that the rate r is a bonded nonnegative process, the scalar of the stock rate b is a bonded process. is called the volatility, a bonded process, and is called risk premim. Here we note that b t r t t t,whereis the scalar whose every component is 1. Let s consider an investor whose actions cannot affect the market prices and who can decide at time t, what amont t of the wealth V t to invest in this stock. Here, we say a strategy is self-financing. he starting price of the claim is an initial endowment to garantee a. Following El Karoi et al. [4], a hedging strategy against a is feasible self-financing strategy y, sch that y a, where a is a positive sqare-integrable contingent claim, then or hedging strategy y, against a, sch that dy t dy v t ry v t v t dt v t dw t, is sch that the market vale y is the fair price and the pper price of the claim. he investor wants to minimize the fnctional cost of a risk-sensitive type of the following model 1 E exp vt y t dt 1 y. Next, we provide a concrete example of risk-sensitive backward stochastic linear qadratic (LQ) problem, and give the explicit optimal portfolio and validate or major theoretical reslts in theorem 4.1 (Risk-sensitive sfficient optimality conditions ). Consider the following qadratic risk-sensitive linear control problem 1 inf E exp vt y t dt 1 y, v U sbject to : dy v t ry v t v t dt v t dw t, y v a. 16 Recall from 5, that A : exp 1 vt y t dt 1 y. Instantly, we give the Hamiltonian H defined by H t,y t,v t, p t,lt ry t v t z t lt p t 1 v t y t. Clearly H t,y t,v t, p t,lt p t v t, and maximizing the Hamiltonian yields t p t. hen, the optimal state dynamics is given by dy t ry t p tdt v t dw t, y a Let y t be a soltion of 18 associated with optimal portfolio t. hen, there exists a

10 57 A. CHALA niqe adapted process p t of the following SDE system (called adjoint eqation), according to eqation 1. Let s define this eqations as d p t H y tdt H z tdw t, p y, 19 where H y r p t y t, H z lt p t, dw t ltdt dw t, A 1 exp vt y t dt 1 y. At this point, we need only to prove that is an optimal portfolio. heorem 5.1. Sppose that the portfolio satisfies 17, where p t satisfies 19. henis the niqe optimal portfolio of the above backward stochastic differential eqation of qadratic linear control problem 16. Proof. From the definition of the fnctional cost J,wehave J J v E exp 1 y t dt 1 y 1 E exp vt y v t dt 1 y. By applying Ito s formla, sing the explicit forms of the adjoint eqation 19,and following the same steps of the proof as in theorem 4.1, we can obtain that J J v.his means that the portfolio is the optimal process to or system of qadratic linear control with a the risk-sensitive performance fnctional. he system governed by the eqations 18 and 19 is a flly copled forward backward stochastic differential eqation. It is very hard to find the explicit soltion to the system 18 and 19. o this end we mst follow the next method. For any smooth deterministic fnctions t, t, we can write the soltion of p t as p t ty t t. 1 By applying the Itô s formla to 1, we find that d p t ty t dt tdy t tdt t t tr y t tt t dt t t dw t. Frthermore, the adjoint eqation 19, can be rewritten as follows d p t try t rt y t dt lt p tdw t, p y, According to the proof of Lemma 3. in [], it is very important to observe that

11 Sfficient Optimality Condition for a Risk-Sensitive Control Problem for BSDEs 58 dw t ltdt dw t,and p t ty t t, asin1. hen the above adjoint eqation implies that d p t 1 l tt 1y t l ttdt ltty t tdw t, p y, 3 By identifying with 3, weget t t rt r l tt 1 tt t l tt. hen we can dedce that the above eqation is the Riccati eqation to t, and is given by t t r l tt 1,.. he second step in identifying the coefficients, wold be to list the ordinary differential eqation t t l tt,. 5 he optimal portfolio 17 can be written as t ty t t, 6 where the deterministic fnctions t, and t have the soltion given by 4, and 5 respectively. heorem 5.. Assme that the pair t,t are the niqe soltion to 4 and 5. hen the optimal portfolio of the problem 16 has the state feedback form Conclsion and Otlook his paper reports on one main reslt, theorem 4.1, which establishes the sfficient optimality conditions for backward stochastic differential eqation system of risk sensitive performance. A reslt obtained sing an almost similar scheme as in Khallot and Chala [5]. In this paper, a detailed proof can be fond of the explicit soltion to the Riccati eqation and to the ordinary differential eqation. he last paper can be considered as an extension of the backward differential eqation into flly copled forward backward SDE. he main tools in the proof are tightness and se of the risk netral maximm principle (theorem 3.1), in addition to sing the reslt of El Karoi et al. [3]. he present work relies, moreover, on the paper of Chala []. Its proof is based on the convexity conditions of the Hamiltonian fnction, and the initial term of the performance fnction. It shold be noted that the risk sensitive control problems stdied by Lim and Zho in [6] are different from ors. Or reslts can be compared with maximm principle obtained by Khallot et al. [5], whose reslts shall be discssed in ftre new joint paper. In this paper we will generalize the last reslt to the flly copled 4

12 59 A. CHALA forward backward stochastic differential eqation, which is motivated by an optimal portfolio choice problem in the financial market. Specifically, the motivation is by the model of control cash flow of a firm or project. For example, one can set the model of pricing and management of an insrance contract, a conterpart withot mean field term, as in [1]. A problem to be thoroghly addressed in or ftre paper, where the dynamical system is governed by a flly copled stochastic differential eqation of mean field type. Remarkably, the maximm principle of risk-netral control obtained by Shi and W [8] is qite similar to or theorem 3.1, bt their adjoint eqation and maximm conditions heavily depend on the risk sensitive parameter. Acknowledgments he athor is gratefl to an anonymos referee for his critical reading of the original typescript. References [1] B. Djehiche, H. embine, and R.empone, A stochastic maximm principle for risk-sensitive mean-field type control, IEEE ransactions on Atomatic Control 61, ( 15), [] A. Chala, Pontryagin s risk-sensitive stochastic maximm principle for backward stochastic differential eqations with application, Blletin of the Brazilian Mathematical Society, New Series 48(3), ( 17), [3] N. El-Karoi, and S. Hamadène, BSDEs and risk-sensitive control, zero-sm and nonzero-sm game problems of stochastic fnctional differential eqations, Stochastic Processes and their Applications 17(1), (3), [4] N. El-Karoi, S. Peng, and N.C. Qenez, Backward stochastic differential eqation in finance, Mathematical Finance 7(1), (1997), [5] R. Khallot, and A.Chala, A risk-sensitive stochastic maximm principle for flly copled forward-backward stochastic differential eqations with applications, Sbmitted for pblication. [6] A. E. B. Lim, and X. Zho, A new risk-sensitive maximm principle, IEEE ransactions on Atomatic Control 57, ( 5), [7] E. Pardox, and S. Peng, Adapted soltion of a backward stochastic differential eqation, Systems and Control Letters 14(1 ), (199), [8] J. Shi, and Z. W, A risk-sensitive stochastic maximm principle for optimal control of jmp diffsions and its applications, Acta Mathematica Scientia 31(), (11), [9] J. Shi, and Z. W, Maximm principle for risk-sensitive stochastic optimal control problem and applications to finance, Stochastic Analysis and Applications 3(6), (1),

13 Sfficient Optimality Condition for a Risk-Sensitive Control Problem for BSDEs 6 [1] H. embine, Q. Zh, and. Basar, Risk-sensitive mean-field games, IEEE ransactions on Atomatic Control 59(4), (14), [11] J. Yong, Optimality variational principle for controlled forward-backward stochastic differential eqations with mixed initial-terminal conditions, SIAM Jornal on Control and Optimization 48(6), (1), [1] J. Yong, Optimality variational principle for controlled forward-backward stochastic differential eqations with mixed initial-terminal conditions, SIAM Jornal on Control and Optimization 48(6), (1), [13] J. Yong and X. Zho, Stochastic Controls: Hamiltonian Systems and HJB Eqations (Stochastic Modelling and Applied Probability), Springer-Verlag, New York, Article history: Sbmitted March, 3, 17; Revised December, 9, 17; Accepted December, 1, 17.