Dual control Monte-Carlo method for tight bounds of value function in regime switching utility maximization

Transcription

1 Dual conrol Mone-Carlo mehod for igh bounds of value funcion in regime swiching uiliy maximizaion Jingang Ma, Wenyuan Li and Harry Zheng Absrac In his paper we sudy he dual conrol approach for he opimal asse allocaion problem in a coninuous-ime regime-swiching marke. We find he lower and upper bounds of he value funcion ha is a soluion o a sysem of fully coupled nonlinear parial differenial equaions. These bounds can be ighened wih addiional conrols o he dual process. We sugges a Mone-Carlo algorihm for compuing he igh lower and upper bounds and show he mehod is effecive wih a variey of uiliy funcions, including power, non-hara and Yaari uiliies. The laer wo uiliies are beyond he scope of any curren mehods available in finding he value funcion. MSC : 49L, 9C46, 65C5 Keywords: Porfolio opimizaion, regime swiching, dual conrol, non-hara uiliy, Yaari uiliy, igh lower and upper bounds, Mone-Carlo mehod Inroducion Sochasic conrol mehods are widely used in solving dynamic porfolio opimizaion problems in finance, among many oher applicaions. The key idea is o apply he dynamic programming principle o he opimal value funcion and show ha i saisfies a parial differenial equaion (PDE), called he Hamilon-Jacobi-Bellman (HJB) equaion. If one can find a classical soluion o he HJB equaion, hen one may verify i is he value funcion by he maringale principle of opimaliy, and find a feedback opimal conrol as a by-produc. Oherwise, one may show he value funcion is a unique viscosiy soluion o he HJB equaion and find i numerically; see he excellen exposiion of sochasic conrol and is applicaions in Pham (9). Since he HJB equaion is a fully nonlinear PDE, is solvabiliy crucially depends on he erminal condiion. In a Black-Scholes complee marke model wih a power or logarihmic uiliy, we know here is a closed-form classical soluion o he HJB equaion. Bian e al. School of Economic Mahemaics and Collaboraive Innovaion Cener of Financial Securiy, Souhwesern Universiy of Finance and Economics, Chengdu, 63, China ( mj@swufe.edu.cn). The work was suppored by Naional Naural Science Foundaion of China (Gran No. 6733) and Program for New Cenury Excellen Talens in Universiy (China Gran No. NCET--9). School of Economic Mahemaics, Souhwesern Universiy of Finance and Economics, Chengdu, 63, China ( Wylfm@.swufe.edu.cn). Corresponding auhor. Deparmen of Mahemaics, Imperial College, London SW7 BZ, UK ( h.zheng@imperial.ac.uk).

2 () and Bian and Zheng (5) use he dual conrol mehod o show here is a classical soluion o he HJB equaion for a broad class of uiliy funcions and give a represenaion of he soluion o he HJB equaion in erms of ha of he dual HJB equaion. For consrained marke models one may have o use numerical mehods o solve he HJB equaions; see for example, Huang e al. () for a combined fixed poin and policy ieraions mehod and Reisinger and Forsyh (6) for a piecewise consan policy approximaions mehod. In his paper we discuss he mehod of finding an approximae soluion o he HJB equaion arising from regime swiching (RS) uiliy maximizaion problems. The RS model is popular in financial daa modelling and analysis as i allows parameers of asse price dynamics o depend on a finie sae Markov chain process (MCP). I has been shown in he lieraure ha MCP is effecive in providing informaion of he marke environmen. For example, Hamilon (989) inroduces a RS model for nonsaionary ime series and business cycles. Hardy () applies a wo-regime model o provide a good fi o monhly sock marke reurns. The RS model provides good flexibiliy for characerizing macro marke uncerainies while preserves analyic racabiliy for underlying asse price dynamics. There has been acive research in porfolio opimizaion wih RS models. For example, Zhang e al. (5) and Yin e al. (6) sudy he rading rules in a RS marke. Zhou and Yin (3) and Çelikyur and Özekici (7) invesigae he mean-variance porfolio opimizaion in a discree-ime RS model. Çanakoğlu and Özekici () discuss he HARA uiliy maximizaion in a coninuous-ime RS model. Honda (3), Sass and Haussmann (4), and Rieder and Bäuerle (5) solve porfolio opimizaion problems wih parial informaion and regime-swiching drif processes. For a regime swiching sochasic conrol problem, Bäuerle and Rieder (4) and Fu e al. (4) show ha he value funcion saisfies an HJB sysem of fully coupled nonlinear PDEs and prove a verificaion heorem. Furhermore, for a power or logarihmic uiliy funcion, hey reduce he HJB equaion o a sysem of linear ODEs which are hen solved wih marix exponenials. For general uiliy funcions, i seems highly unlikely, if no impossible, ha one can solve he sysem of HJB equaions analyically. There are some grea effors in he lieraure in designing numerical schemes. Huang e al. () sudy numerical mehods for pricing American opions under regime swiching using policy ieraion o find he soluion of he discreized HJB variaional equaions. I is compuaionally expensive o apply he mehods of Huang e al. () o RS uiliy maximizaion as a each grid poin one needs o solve an opimizaion problem over some unknown value funcion. Similarly, he sysem is no clearly ellipic, so ha a monoone scheme in Forsyh and Labahn (8) may be ou of reach. Fu e al. (4) inroduce a funcional operaor o generae a sequence of value funcions and show ha he opimal value funcion is he limi of his sequence. Each funcion in he sequence solves an auxiliary porfolio opimizaion problem in a single-regime marke and is a soluion o he HJB equaion. The mehod in Fu e al. (4) is concepually appealing bu pracically ineffecive in finding a soluion o he regime swiching HJB equaion for general uiliy funcions, due o he curse of dimensionaliy; see Remark. for furher discussions on his poin. In his paper we discuss how o derive igh lower and upper bounds of he soluion o he regime swiching HJB equaion. Inspired by he work of Bian e al. () and Bian and Zheng (5), we exend he dual conrol approach o regime-swiching models. The dual value funcion can be easily compued by he Mone Carlo mehod as i is an opion pricing problem. The dual value funcion no only provides an upper bound for he primal value funcion, bu also suggess a good feasible conrol ha can be used o find a lower bound for he primal value funcion. These bounds can be improved, ha is, he

3 gap of hese bounds can be reduced, wih addiional conrols o he dual process. The suggesed mehod can be applied o general uiliy funcions, including ones no necessarily differeniable and/or sricly concave. The bounds are very igh (less han % of relaive error in mos of our numerical ess) and may be used as an approximaion o he value funcion. An addiional benefi in compuing he igh lower bound is ha we derive a feedback conrol which may be used as an approximaion o he opimal conrol. To he bes of our knowledge, his is he firs ime a Mone Carlo algorihm is suggesed o find he igh lower and upper bounds of he soluion o he regime swiching HJB equaion for general uiliy funcions. Our numerical resuls show he mehod is reliable, effecive and accurae for power, non-hara, and Yaari uiliies. The laer wo uiliies are beyond he scope of any curren mehods available in finding he value funcion. The remaining pars of he paper is arranged as follows. Secion inroduces he opimal asse allocaion problem in regime-swiching markes. Secion 3 presens he dual conrol mehod for finding he igh lower and upper bounds and saes he main resul (Theorem 3.) and he Mone Carlo algorihms. Secion 4 gives exensive numerical ess for a variey of uiliy funcions, including power, non-hara and Yaari uiliies. Secion 5 concludes he paper. The appendix gives he proofs of he soluion (8) o he sochasic differenial equaion (7) and Theorem 3., and some ables of numerical ess. Opimal asse allocaion in regime-swiching models In his secion, we formulae he sochasic conrol problem in a regime swiching diffusion model. Consider a fixed ime horizon [, T ]. Le (Ω, F, P ) be a complee probabiliy space, W a sandard Brownian moion, α a coninuous ime finie sae observable Markov chain process (MCP), which are independen of each oher, and le {F } be he naural filraion generaed by W and α compleed wih all P -null ses. We idenify he sae space of {α } as a finie se of uni vecors E := {e, e,..., e d } where e i R d is a column vecors wih one in he ih posiion and zeros elsewhere, i =,..., d. Denoe by Q = (q ij ) d d he generaor of he Markov chain {α } wih q ij for i j and d j= q ij = for each i D := {,..., d}. The MCP α has a semi-maringale represenaion α = α + Q α v dv + M, T, () where Q is he ranspose of Q, M is a purely disconinuous square-inegrable maringale wih iniial value zero; see Ellio e al. (994). Assume he financial marke consiss of one risk-free bond and one risky sock. The bond and sock price processes B and S are assumed o follow he sochasic differenial equaions (SDEs) db = r B d, ds = S (µ d + σ dw ), T, where r = rα, µ = µα, σ = σα, and r = (r,..., r d ) is a vecor of risk-free ineres raes wih r i being he rae in regime i, and µ = (µ,..., µ d ) and σ = (σ,..., σ d ) are vecors of reurn and volailiy raes of he risky asse. Assume all raes are posiive consans. Denoe by θ := (θ,..., θ d ) he vecor of marke prices of risk wih θ i = (µ i r i )/σ i for i D. 3

4 Le X be he wealh process of a porfolio comprising he bond B and he sock S. The wealh process X saisfies he SDE dx = X (r d + π σ (θ d + dw )), T, () where π is a progressively measurable conrol process. π represens he proporion of wealh X invesed in risky asse S and θ = θα is he marke price of risk a ime. The uiliy maximizaion problem is defined by sup E[U(X T )] subjec o (), (3) π where U is a uiliy funcion ha is coninuous, increasing, and concave on [, ). Sochasic conrol is a sandard mehod one may use o solve problem (3) (he oher mehod is convex dualiy maringale mehod; see Karazas and Shreve (998)). To do so, we define a value funcion V i (, x) := sup π Π E,x,i [U(X T )], i D, (4) where E,x,i is he condiional expecaion operaor given X = x and α = e i (abbreviaed by α = i) for i D, and Π := {π s, s [, T ]} is he se of all admissible conrol sraegies over [, T ]. I is proved in Fu e al. (4) and Bäuerle and Rieder (4) ha for a coninuous, sricly increasing, and sricly concave uiliy funcion U, he opimal value funcion V i, for i D, saisfies he following sysem of HJB equaions V i + r ix V i x θ i ( ) Vi / V i x d x + q ij V j =, i D, (5) wih boundary condiion V i (T, x) = U(x) for i D. Verificaion resuls are also given in Bäuerle and Rieder (4) and Fu e al. (4). Remark.. When U is a power (or logarihmic) uiliy, one may wrie V i (, x) = U(x)f i () (or V i (, x) = U(x)+f i ()) o reduce (5) o a sysem of linear ODEs wih unknown funcions f i, i D, which can be solved wih marix exponenials (see Bäuerle and Rieder (4) and Fu e al. (4)). When U is a general uiliy, i is difficul o solve (5) as i is a sysem of fully nonlinear PDEs and, unlike power uiliy, here is no clear candidae soluion. Fu e al. (4) define a funcional operaor M and a sequence of funcions {H n } by H n+ = MH n and show ha H n converges o he value funcion V as n ends o. The funcion H can be compued wih he uiliy U. I also reduces a sysem of fully coupled nonlinear PDEs o a sysem of decoupled nonlinear PDEs. Concepually, his is an excellen algorihm o find he value funcion, bu i is impracical for implemenaion for he following reasons: o find H n+, one has o solve a sochasic conrol problem using H n which can only be derived numerically. As such, curse of dimensionaliy quickly arises in he algorihm of Fu e al. (4). Remark.. We assume in his paper ha he Markov chain α is observable. If α is no observable, for example, when here is no regime swiching in he volailiy process σ, i.e., σ =... = σ d, hen a filering problem arises for an invesor who observes only he sock prices. Such models wih parial informaion and regime-swiching in µ have been sudied j= 4

5 in he lieraure. Honda (3) inroduces he filered probabiliy as a new sae variable, derives an HJB equaion wih wo sae variables, and reduces he HJB equaion o one sae variable for power uiliy. Rieder and Bäuerle (5) use a similar approach and derive he represenaion of he value funcion for power and logarihmic uiliy. Sass and Haussmann (4) derive an explici represenaion of he opimal rading sraegy using HMM (hidden Markov model) filering resuls and Malliavin calculus. Ellio e al. (8) discuss he differences of he models wih consan and wih swiching volailiy and explain why filers have o be used in he firs case while he Markov chain is observable in heory in he second case. When here is no regime swiching, Bian e al. () and Bian and Zheng (5) apply he dual conrol mehod o show ha here is a classical soluion o he HJB equaion and he opimal value funcion is he conjugae funcion of he opimal dual value funcion. Inspired by hese works, we will inroduce in he following secions an efficien Mone Carlo mehod based on he dual conrol framework o find he lower and upper bounds of he value funcion for general uiliies and show ha hese bounds are igh and can be improved wih increased number of he dual conrol variables. The usefulness of he algorihm is illusraed by numerical ess of power, non-hara, and Yaari uiliies. 3 Dual-conrol Mone-Carlo mehod In his secion, we assume ha he uiliy funcion U is coninuous, increasing, and concave (bu no necessarily sricly increasing and concave), and U() =. The dual funcion of U is defined by Ũ(y) = sup(u(x) xy), (6) x for y. The funcion Ũ(y) is a coninuous, decreasing and convex funcion on [, ) and saisfies Ũ( ) = (see e.g., Bian and Zheng (5)). Define a dual process Y dy = Y ( r d θ dw + CdM ), (7) where C is a consan row vecor in R d, W is a Brownian moion, and M is he maringale defined in (). The soluion o (7) a ime T, wih iniial condiion Y = y, can be wrien as where A,T = T Y T = y exp(a,t ), (8) ( r u + CQ α u + ) T θ u du θ u dw u + ln( + C(α s α s )). (9) <s T Tha Y T defined as in (8) for all T > saisfies he SDE (7) can be shown by he Iô formula for semi-maringales. For convenience we provide a proof in Appendix A. The dual value funcion is defined by Ṽ i (, y) = E,y,i [Ũ(Y T )], i D. () 5

6 Since Y saisfies a linear SDE (7) and Ũ is a decreasing convex funcion, we know Ṽi(, y) is a decreasing convex funcion for y > and fixed and i. Denoe by W i (, x) he dual funcion of Ṽi(, y) for fixed and i, given by The nex heorem is he main resul of he paper. W i (, x) = inf y> (Ṽi(, y) + xy). () Theorem 3.. Le W i (, x) be given by () and S and S be ses of vecors C saisfying C i < / for i D. Then he opimal value funcion V i (, x) defined in (4) saisfies V i (, x) inf C S W i (, x). () Furhermore, suppose Ṽi(, y) given by () is wice coninuously differeniable and sricly convex for y > and fixed and i. Le y = y (, x, i, C), wrien as y for noaional simpliciy, be he soluion of Ṽi(, y) + x =. (3) y Le he feedback conrol π i (, x) be defined by π i (, x) = θ i y σ i x y Ṽi(, y ) (4) for [, T ] and x >. Le X be he unique srong soluion of SDE () wih conrol process π = πα (, X ). Le Wi (, x) be defined by Then he opimal value funcion V i (, x) saisfies W i (, x) := E,x,i [U( X T )]. (5) V i (, x) sup C S Wi (, x). (6) Proof. The proof is given in Appendix B. Remark 3.. Clearly, if S S, hen V i (, x) inf C S W i (, x) inf C S W i (, x). Using S insead of S gives a igher upper bound bu i may be more expensive in compuaion. The same applies o he lower bound. Remark 3.. Nex we give some explanaion o he choice of conrol π i (, x) for he igh lower bound for V i (, x). In heory, his is easy as any feasible conrol π and is corresponding wealh process X provide a value E[U(X T )], which is a lower bound of V i (, x). Our objecive is o find a lower bound ha is as large as possible. When here is no regime swiching, Bian e al. () and Bian and Zheng (5) show ha V i (, x) = W i (, x) wih C = and he opimal conrol is given by π i (, x) = θ i x W i(, x) σ i x W x i (, x), (7) 6

7 which can be equivalenly compued using he dual value funcion Ṽi(, y) and is derivaives. This suggess srongly o use W i (, x) o consruc a conrol which may provide a good lower bound for V i (, x), even hough we only have he relaion V i (, x) W i (, x) in he curren regime-swiching framework. Assume Ṽi(, y) is wice coninuously differeniable and sricly convex for y > and fixed and i and y = y is he soluion of equaion (3) (a sufficien condiion ha ensures he exisence of a unique soluion o equaion (3) is lim y Ṽi(,y) y = and lim y Ṽi(,y) y = ). Then we have W i (, x) = Ṽi(, y ) + xy. Noe ha y is coninuously differeniable for x > and fixed and i by he implici funcion heorem. Some simple calculus, using (3), shows ha x W i(, x) = y, y Ṽi(, y ) x y + =, x W i(, x) = / y Ṽi(, y ). Noe ha x y, oherwise, one would have =, a conradicion. Subsiuing he derivaives above ino (7), we ge a candidae conrol π i (, x) in (4) for he lower bound. Remark 3.3. The dual conrol vecor C in (7) mus saisfy + C(α α ) > for [, T ]; see (9). A sufficien condiion o ensure his is C i < / for i D. We may also use a general progressively measurable process (C ) in (7), provided (9) is well defined, which indicaes one may poenially ge even igher lower and upper bounds if C is chosen o be a process, no jus a consan vecor. When here is no regime swiching, C can be se equal o zero. The Mone-Carlo mehod can be used o find he igh lower and upper bounds. To compue he igh lower bound, one key sep is o find a soluion of (3). We assume ha he uiliy U is sricly concave and saisfies Inada s condiion (U () = and U ( ) = ), which implies ha Ũ is coninuously differeniable and Ũ() = and Ũ( ) =. From (), he pahwise differeniaion mehod gives Ṽi(, y) y = y E,y,i [ ] Y T Ũ (Y T ), (8) which is for y close o and for y close o. Choosing a sufficienly small y and a sufficienly large y such ha he expression Ṽi(,y) y + x has opposie signs for x >, we can hen use he bisecion mehod o find he soluion o equaion (3). The Markov chain process α can be generaed in a sandard procedure as follows. Assume he MCP is a sae i. Generae wo independen sandard uniform variables ζ and ζ, define τ i = ln ζ. q ii where q ii := j i q ij is he inensiy rae of MCP jumping from sae i o some oher sae (no decided ye). Then τ i is he firs jump ime of MCP from sae i. To decide which sae i jumps o, divide inerval [, ] by d subinervals, wih lengh of q ij /q ii for j i. If ζ is realized in he j-h subinerval, he MCP has jumped o sae j a ime τ i. Repea hese seps o generae a sample pah of MCP α on he inerval [, T ]. The dual conrol variable C may be chosen randomly by specifying a disribuion for C which ensures C i < / and hen generaing samples C by simulaion, or deerminisically 7

8 by specifying paricular values such as fixed grid poins on a d-dimensional hypercube saisfying C i < /. Nex we describe he Mone-Carlo mehod for compuing he igh lower and upper bounds a ime =. The igh lower and upper bounds a oher imes can be compued similarly. Assume X = x, α = i and he dual funcion Ũ in (6) are known. Mone-Carlo mehod for compuing he igh upper bound: Sep : Sample d independen uniform variables C i in [.4,.4], which are componens of a vecor C. Sep : Generae M sample pahs of Brownian moion W and MCP α, which are used o compue Y T wih Y = y and he average derivaives: Ṽi(, y) y y M Y T Ũ (Y T ). M l= Sep 3: Use he bisecion mehod o solve equaion (3) and ge he soluion y y. Sep 4: Compue he upper bound W i (, x) Ṽi(, y ) + xy. Sep 5: Repea Seps o 4 N imes and hen compue he igh upper bound inf C S W i (, x). Mone-Carlo mehod for compuing he igh lower bound: Sep : Sample d independen uniform variables C i in [.4,.4], which are componens of a vecor C. Sep : Generae M sample pahs of Brownian moion W and MCP α, which are used o find he conrol process π in (4) and he wealh process X in (). Sep 3: Compue he lower bound W i (, x) M U( M X T ). l= Sep 4: sup C S Repea Seps o 3 N imes and hen compue he igh lower bound Wi (, x). Remark 3.4. I is much more ime consuming o compue he igh lower bound han o he igh upper bound. The reason is ha one has o generae sample pahs of he wealh process X and conrol process π, which requires solving equaion (3) a all grid poins of ime, no jus a = as in he case of compuing he igh upper bound. However, here is one excepion when U is a power uiliy. In his case, he compuaion ime for he igh lower bound is much shorer han ha for he igh upper bound, and he igh lower bound wih C = coincides wih he primal value; see he example in he nex secion. 8

9 4 Case sudies for a variey of uiliy funcions 4. Power uiliy Consider a power uiliy funcion: The dual funcion is given by U(x) = xp, < p <. (9) p Ũ(y) = p p y p p. Subsiuing (8), Y T = y exp(a,t ), ino (), we have Ṽ i (, y) = p p y p p β i (), () where β i () = E,i [exp( p p A,T )]. Since Ṽi(, y) is C and sricly convex a y > and Ṽi(,) y = and Ṽi(, ) y =. The condiions of Theorem 3. are saisfied and here is a unique soluion o equaion (3). I follows from definiion () ha W i (, x) = Ṽi(, y i ) + xy i, where y i = ( x ) p β i () is he soluion of (3) wih Ṽi(, y) given in (). The igh upper bound on he primal value is given by { } p inf W i (, x) = inf C C p (y i ) p p β i () + xyi and can be compued wih he Mone-Carlo mehod. To calculae he igh lower bound on he primal value, we need firs o calculae he conrol π i (, x). A direc compuaion, using (4), gives ha π i (, x) = θ i σ i p. The wealh process becomes ( d X = X r d + θ ) p (θ d + dw ). The soluion is given by [ T ( X T = x exp r s + p ) ( p) θ s ds + θ ] s p dw s. Since here is a closed-form expression for X T, we can generae X T direcly wihou having o generae a sample pah of X firs. Noe also ha he conrol process π is independen of he dual conrol vecor C, which implies he lower bound W i (, x) is already he igh 9

10 lower bound. These wo facs make he compuaion of he lower bound for power uiliy very fas, which is no he case for general uiliies. The lower bound in (5) is given by W i (, x) = [ T p xp E,i [exp = [ T p xp E,i [exp ( p( p) pr s + ( p) θ s ( pr s + p ( p) θ s ) ds + ) ds pθ ]] s p dw s ]]. () The expecaion afer he firs equaliy is over boh Brownian moion W and MCP α and he expecaion afer he second equaliy is over MCP α only. We can verify direcly ha expression () saisfies he HJB equaion (5) and he erminal condiion W i (T, x) = x p /p. Therefore, he lower bound W i (, x) equals he primal value funcion V i (, x). In he nex numerical example we use he dual-conrol Mone-Carlo mehod o calculae he lower and upper bounds for power uiliy (9) wih p = /. The iniial wealh a ime = is x =, he invesmen period T =, and he number of simulaions is M = M = 7. Since he erminal wealh X T has a closed-form expression, here is no need o use he Euler mehod o discreize SDE () o generae random variable X T, which resuls in he compuaion of he lower bound being fas. Example 4.. We consider -sae Markov chain process wih generaing marix ( ) a a Q =, () b b where a, b are posiive consans. To show he robusness of he algorihm, we have chosen 5 samples of a, b from he uniform disribuion on inerval [.,.], which means he ransiion of one sae o anoher can be slow (average once every years) or fas (average wice a year) or anyhing in beween. The saes are a growh economy (sae ) and a recession economy (sae ). The riskless ineres raes, reurn, and volailiy raes of risky asse are given by r = (.5,.), µ = (.3,.7), σ = (.,.3). (3) The comparisons are carried ou for he cases of C = and addiional conrol vecor C (sampling N = 5 imes for compuing he igh upper bound and N = imes for compuing he igh lower bound). The reason for choosing N = is ha he exac lower bound wih C = coincides wih he primal value and here is no need o use he addiional conrol C in he compuaion of he lower bound. In he es, he benchmark value is he primal value explicily given by (see Bäuerle and Rieder (4) and Fu e al. (4)) V (, x, i) = a(, i) xp, for [, T ], x, i D, p where a(, i) is he ih componen of vecor a() = exp[ (Λ Q)(T )], is a d vecor wih all componens, and Λ is a d d diagonal marix wih diagonal elemens λ i = θ p i p pr i for i D. The numerics in Table 8 (see Appendix C) show ha he dual-conrol Mone-Carlo mehod generaes correc and igh lower and upper bounds on he primal value and he use of conrol vecor C can decrease furher he gap beween he lower and upper bounds. We have also lised saisics in Table 8, including he average compuaion ime in seconds,

11 he mean and he sandard deviaion of absolue and relaive difference beween he lower and upper bounds. The improved upper bound wih addiional dual conrol variable C comes wih increased compuaion ime. This is a rade-off of accuracy and speed. In Table, we give he mean and sandard deviaion of he absolue and relaive difference beween he lower and upper bounds for power uiliy wih many randomly sampled parameers-ses: samples of a, b from he uniform disribuion on inerval [.,.], r, r on [.,.8], µ, µ on [.3,.], σ, σ on [.,.6]. I is clear ha he gap beween he igh lower and upper bounds is very small, especially when he dual conrol C is used. This shows ha he algorihm is reliable and accurae. Table : Mean and sd of he absolue and relaive difference beween he lower and upper bounds for power uiliy in Example 4. wih many randomly sampled parameers-ses. No Conrol C Wih Conrol C mean diff.39.9 sd diff..5 mean rel-diff (%) sd rel-diff (%) A non-hara uiliy Consider he following non-hara uiliy funcion from Bian and Zheng (5): U(x) = 3 H(x) 3 + H(x) + xh(x) (4) for x >, where H(x) = ( + + 4x) /. Bian and Zheng (5) show ha U is coninuously differeniable, sricly increasing and sricly concave, saisfying U() =, U( ) =, U () = and U ( ) =. Furhermore, he relaive risk aversion coefficien of U is given by R(x) = xu (x) U (x) = ( + 4 ), + 4x which shows ha U is no a HARA uiliy and represens an invesor who will increase he percenage of wealh invesed in he risky asse as wealh increases. To find he dual funcion Ũ(y) we may use he definiion (6) o compue he maximum of U(x) xy over x >. Some simple calculus shows ha U (x) = H(x), so he maximum is achieved a a poin x saisfying H(x ) = y, which gives x = y + y 4. Subsiuing x ino U(x) xy, we conclude ha he dual funcion is given by Insering (8) ino (), we ge Ũ(y) = 3 y 3 + y. Ṽ i (, y) = E,y,i [Ũ(Y T )] = 3 y 3 D,i + y F,i, (5) where D,i = E,i [exp( 3A,T )] and F,i = E,i [exp( A,T )]. I is easy o check ha he condiions of Theorem 3. are saisfied and here is a unique soluion y o he equaion

12 (3). From definiion () we obain he upper bound where W i (, x) = 3 (y ) 3 D,i + (y ) F,i + xy, y = ( ) F,i + F,i x + 4xD,i is he soluion of (3) wih Ṽi(, y) given by (5). The igh upper bound inf C S W i (, x) can be compued wih he Mone-Carlo mehod. Now we compue he conrol π i (, x). A direc compuaion yields, using (4) and (5), π i (, x) = θ i [ (y ) 4 D,i + x ], i D. σ i x Here we have used he relaion (y ) 4 D,i + (y ) F,i = x. The Euler discreizaion for he wealh process () wih sep size is given by X + = ( + r ) X + X π(, X )σ (θ + W ), (6) wih X = x, where π(, X ) = ( π (, X ),..., π d (, X ) ) α. Using he Mone-Carlo mehod, we can find he igh lower bound sup C S Wi (, x) = sup C S E,x,i [U( X T )]. In he following numerical examples we use he dual-conrol Mone-Carlo mehod o calculae he lower and upper bounds for non-hara uiliy (4). The iniial wealh a ime = is x =, he invesmen period T =, and he number of simulaions is M = M = 6. Since here is no closed form expression for he erminal wealh X T, we use (6) wih sepsize =. o generae sample pahs of he wealh process X o ge X T, which makes he compaion of he lower bound more expensive. Example 4.. We compare he lower and upper bounds generaed by he dual-conrol Mone-Carlo wih he exac primal values when here is no regime-swiching. The primal value funcion has he following explici form (see Bian and Zheng (5)): V (, x) = 3 ( (y ) e (r+θ )(T ) + xy ), where (y ) = ( ) e (r+θ )(T ) + e x (r+θ )(T ) + 4xe 3(r+θ )(T ) and θ = (µ r)/σ. By choosing (r, µ, σ) = (.5,.3,.) and (.,.7,.3), we can find wo exac (benchmark) values of he primal problem. Now we se r, µ, σ as in (3) and he generaing marix Q as in () wih a = b =.5. Since he probabiliy of a jump of MCP on he inerval [, T ] is very small (less han.5), we expec o see he lower and upper bounds, saring from iniial saes and, are close o he exac values for (r, µ, σ) chosen above. Table shows ha his is indeed he case.

13 Table : Comparisons of lower and upper bounds wih he exac (benchmark) value for Example 4. (non-hara uiliy). α Benchmark LB UB diff rel-diff (%) Table 3: Lower bound (LB) and upper bound (UB) wih wo-sae regime-swiching for Example 4.3 (non-hara uiliy). No conrol C Wih conrol C α LB UB diff rel-diff (%) LB UB diff rel-diff (%) Example 4.3. We now consider a regime swiching model wih generaing marix Q given by () and a = b =.5 and oher parameers given by (3). There is no closed-form formula for he exac value. The comparisons are carried ou for he cases of C = and addiional conrol vecor C (sampling N = 5 imes for compuing he igh upper bound and N = 5 imes for compuing he igh lower bound). The numerics in Table 3 show ha he use of conrol vecor C significanly decreases he difference beween he lower and upper bounds. Using he opimal conrol vecor C for compuing he igh lower bound in Table 3, we draw D graphs for sample pahs of MCP α, feedback conrol π, opimal wealh process X (Figure ), 3D graphs for feedback conrol π (Figure ), and disribuions of he erminal wealh X T (Figure 3). Example 4.4. This numerical es is o show he robusness of he dual conrol Mone Carlo mehod. The seup and daa used are he same as hose in Example 4.. The comparisons are carried ou for he cases of C = and addiional conrol vecor C (N = 5 and N = ). The numerics in Table 9 (see Appendix C) show ha he use of he conrol vecor C significanly decreases he difference beween he lower and upper bounds. In Table 4, we give he mean and sandard deviaion of he absolue and relaive difference beween he lower and upper bounds for non-hara uiliy wih many randomly sampled parameers-ses: samples of a, b from he uniform disribuion on inerval [.,.], r, r on [.,.8], µ, µ on [.3,.], σ, σ on [.,.6]. I is clear ha he gap beween he igh lower and upper bounds is very small, especially when he dual conrol C is used. This shows ha he algorihm is reliable and accurae. Example 4.5. We consider 3-sae MCP wih generaing marix.5a a.5a Q = b b b,.5c c.5c where a, b, c are posiive consans. This srucure of he generaing marix is o indicae ha a sae is more likely o move o is adjacen sae hen o a sae farher away. The saes can be a growh economy (sae ), an average economy (sae ) and a recession 3

14 α α (, X) 4 π i (, X) 4 π i X.5 X.5 Figure : Sample pahs of MCP, π(, X ), X wih iniial wealh x = (Example 4.3 (non-hara uiliy)). The lef hree figures has he same sample wih iniial regime sae α = and opimal conrol vecor C = ( ,.3858). The righ hree figures has he same sample wih iniial regime sae α = and opimal conrol vecor C = (.3798,.8443). Table 4: Mean and sd of he absolue and relaive difference beween he lower and upper bounds for non-hara uiliy in Example 4.4 wih many randomly sampled parameers-ses. No Conrol C Wih Conrol C mean diff.4.84 sd diff mean rel-diff (%) sd rel-diff (%) economy (sae 3). The jump inensiies a, b, c are chosen from he uniform disribuion on inerval [.,.]. The riskless ineres raes, reurn and volailiy raes of risky asse are given by r = (.6,.4,.), µ = (.,.,.7), σ = (.5,.,.3). 4

15 π * i (,x) wih α = π * i (,x) wih α = π * i (,x) π * i (,x) x x.5 Figure : 3D graphs of he opimal conrol π (, x) (Example 4.3 (non-hara uiliy)). The lef figure is for iniial regime sae α = and opimal conrol vecor C = ( ,.3858) for compuing he igh lower bound. The righ figure is for iniial regime sae α = and opimal conrol vecor C = (.3798,.8443) for compuing he igh lower bound. The comparisons are carried ou for he cases of C = and addiional conrol vecor C (N = 5 and N = ). The numerics in Table (see Appendix C) show ha he use of conrol vecor C significanly decreases he difference beween he lower and upper bounds. 4.3 Yaari uiliy Consider he uiliy funcion: U(x) = x H, (7) where H is a posiive consan. Since he opimal erminal wealh X T is disribued as a Bernoulli random variable similar o he opimal porfolio choice in Yaari dual heory (see Yaari (987)) we call he uiliy defined in (7) a Yaari uiliy. The dual funcion is given by Ũ(y) = H( y) +, where x + = max(x, ). Insering (8) ino (), we ge he dual value funcion Ṽ i (, y) = E,i [ H( y exp(a,t )) +], where he expecaion is over Brownian moion W and MCP α. Given α, A,T is a normal variable wih mean m,t and variance σ,t, given by T ( m,t = r u + CQ α u + ) θ u du + σ,t = T θ udu. <s T ln( + C(α s α s )), 5

16 .6 Disribuion of erminal wealh wih α =. Disribuion of erminal wealh wih α = Frequency.3 Frequency X T X T Figure 3: Disribuion of erminal wealh process for non-hara uiliy (Example 4.3). The dual value funcion Ṽi(, y) is simply a European pu opion price, for a given MCP α, and can be simplified furher as ( ) ( ln y m,t Ṽ i (, y) = E,i [HΦ Hy exp m,t + ) ( )] ln y m,t σ,t σ,t Φ σ,t, σ,t (8) where he expecaion is over MCP α only and Φ is he cumulaive disribuion funcion of a sandard normal variable. The derivaives of Ṽi(, y) are given by ( Ṽi(, y) = E,i [H exp m,t + ) ( )] ln y m,t y σ,t Φ σ,t, (9) σ,t ( ln y m,t Ṽ i (, y) y = E,i [ ( H exp m,t + ) yσ,t σ,t ϕ Using definiion (), we obain he upper bound W i (, x) = Ṽi(, y ) + xy, σ,t σ,t σ,t where y is he soluion of (3) wih Ṽi(,y) y given by (9), i.e., ( E,i [H exp m,t + ) ( )] ln y m,t σ,t Φ σ,t + x =. (3) There is no explici formula for he soluion o equaion (3). The bisecion mehod can be applied o solve equaion (3). In addiion, he lef-hand side of (3) is increasing wih y, hus he equaion (3) has a unique roo if and only if he lef-hand side of (3) is less han zero when y goes o zero and greaer han when y goes o posiive infiniy. This gives a condiion o ensure equaion (3) has a unique roo, i.e., x < E,i [H exp ( m,t + σ,t )]. )]. (3) If (3) is no saisfied, hen Ṽi(, y) + xy is an increasing funcion of y >, which implies he minimum in he definiion of W i (, x), see (), is achieved a y =. Therefore, W i (, x) = H is an obvious upper bound as U(x) = x H H for all x >. 6

17 The igh upper bound inf C S W i (, x) can be compued wih he Mone-Carlo mehod. The conrol π i (, x) can be compued by he following formula: (i) When condiion (3) holds, we use (4) o compue π i (, x) and ge π i (, x) = θ i y [ ( i H σ i x E,i y exp m,t + ) ( ln y m,t σ,t σ,t ϕ σ,t σ,t )]. (ii) When consrain (3) is no saisfied, he bes sraegy is o inves all he money ino he risk-free asse, i.e., π i (, x) =. We can hen generae sample pahs of he wealh process X wih he Euler mehod as in (6) and find he igh lower bound sup C S Wi (, x) wih he Mone Carlo mehod. In he following numerical examples we use he dual-conrol Mone-Carlo mehod o calculae he lower and upper bounds for Yaari uiliy. The daa used are: he iniial wealh x = a ime =, he invesmen period T = and he hreshold level H =. Since here is no closed form expression for erminal wealh X T, we use (6) wih sepsize =. o generae sample pahs of he wealh process X o ge X T. There is a furher complicaion in compuing he lower bound as we have o find he soluion y of equaion (3) numerically. We use he bisecion mehod wih he error olerance 6. I is ime consuming o compue he lower bound, so we only choose M = M = 5 simulaions. Example 4.6. We compare he lower and upper bounds generaed by he dual-conrol Mone-Carlo wih he exac primal values when here is no regime-swiching. The primal value funcion has he following explici form (see Bian and Zheng (5)): V (, x) = { HΦ ( Φ ( x H er(t )) + θ T ), x < He r(t ), H, x He r(t ), where θ = (µ r)/σ is a consan. The oher daa used are he same as hose in Example 4.. The numerical resuls in Table 5 show ha he lower and upper bounds coincide wih he exac value ignoring he Mone-Carlo simulaion errors. Table 5: Comparisons of lower and upper bounds wih he exac (benchmark) value for Example 4.6 (Yaari uiliy). α Benchmark LB UB diff rel-diff (%) Example 4.7. The seup and daa used in his numerical es are he same as hose in Example 4.3. The comparisons are carried ou for he cases of C = and addiional conrol vecor C (N = and N = ). The numerics in Table 6 show ha he use of he conrol vecor C significanly decreases he difference beween he lower and upper bounds. Using he opimal conrol vecor C for compuing he igh lower bound in Table 6, we plo 3D graphs for he opimal conrol π (, x) (Figure 4) and disribuions of he erminal wealh X T (Figure 5). 7

18 Table 6: Lower bound (LB) and upper bound (UB) wih wo-sae regime-swiching for Example 4.7 (Yaari uiliy). No conrol C Wih conrol C α LB UB diff rel-diff (%) LB UB diff rel-diff (%) π * i (,x) wih α = π * i (,x) wih α = π * i (,x) 4 π * i (,x) x x Figure 4: 3D graphs of he opimal conrol π (, x) (Example 4.7 (Yaari uiliy)). The lef figure is for iniial regime sae α = and opimal conrol vecor C = (.746,.636) for compuing he igh lower bound. The righ figure is for iniial regime sae α = and opimal conrol vecor C = (.93885,.3356) for compuing he igh lower bound. Example 4.8. The seup and daa used in his numerical es are he same as hose in Example 4.. The comparisons are carried ou for he cases of C = and addiional conrol vecor C (N = and N = 5). The numerics in Table (see Appendix C) show ha he use of he conrol vecor C significanly decreases he difference beween he lower and upper bounds. In Table 7, we give he mean and sandard deviaion of he absolue and relaive difference beween he lower and upper bounds for Yaari uiliy wih many randomly sampled parameers-ses: samples of a, b from he uniform disribuion on inerval [.,.], r, r on [.,.8], µ, µ on [.3,.], σ, σ on [.,.6]. I is clear ha he gap beween he igh lower and upper bounds is small, bu he effec of he dual conrol C is less pronounced han in he cases for power and non-hara uiliies. The algorihm is sill reliable and accurae even for non-differeniable and non-sricly-concave uiliies. 5 Conclusions In his paper, we sudy he dual conrol approach for he opimal asse allocaion problem in a coninuous-ime regime-swiching marke. We find he igh lower and upper bounds of he value funcion ha is a soluion o he HJB equaion, a sysem of fully coupled nonlinear 8

19 .4 Disribuion of erminal wealh wih α =.8 Disribuion of erminal wealh wih α = Frequency..5 Frequency X T 3 4 X T Figure 5: Disribuion of erminal wealh process for Yaari uiliy (Example 4.7). Table 7: Mean and sd of he absolue and relaive difference beween he lower and upper bounds for Yaari uiliy in Example 4.8 wih many randomly sampled parameers-ses. No Conrol C Wih Conrol C mean diff sd diff.9. mean rel-diff (%) sd rel-diff (%) parial differenial equaions. We sugges a Mone-Carlo algorihm for compuing hese igh lower and upper bounds and show he mehod is reliable and accurae wih a number of numerical ess for power, non-hara and Yaari uiliy funcions. We can herefore find he approximae value funcion and is corresponding conrol sraegies numerically for general uiliy funcions in a regime swiching Black-Scholes model. Acknowledgemens The auhors are very graeful o he anonymous reviewers whose consrucive and deailed commens and suggesions have helped o improve he paper of wo previous versions. References Bäuerle, N. and Rieder, U. (4). Porfolio opimizaion wih Markov-modulaed sock prices and ineres raes, IEEE Transacions on Auomaic Conrol, 9, Bian, B., Miao, S. and Zheng, H. (). Smooh value funcions for a class of nonsmooh uiliy maximizaion problems, SIAM Journal of Financial Mahemaics,, Bian, B. and Zheng, H. (5). Turnpike propery and convergence rae for an invesmen model wih general uiliy funcions, Journal of Economic Dynamics and Conrol, 5,

20 Çanakoğlu, E. and Özekici, S. (). HARA froniers of opimal porfolios in sochasic markes, European Journal of Operaional Research,, Çelikyur, U. and Özekici, S. (7). Muliperiod porfolio opimizaion models in sochasic markes using he mean-variance approach, European Journal of Operaional Research, 79, 86. Ellio, R., Aggoun, L. and Moore, J. (994). Hidden Markov Models: Esimaion and Conrol, Springer. Ellio, R., Krishnamurhy, V. and Sass, J. (8). Momen based regression algorihms for drif and volailiy esimaion in coninuous-ime Markov swiching models, Economerics Journal,, Forsyh, P.A. and Labahn, G. (8). Numerical mehods for conrolled Hamilon-Jacobi- Bellman PDEs in finance, Journal of Compuaional Finance,, 44. Fu, J., Wei, J. and Yang, H. (4). Porfolio opimizaion in a regime-swiching marke wih derivaives, European Journal of Operaional Research, 33, Glasserman, P. (4). Mone Carlo Mehods in Financial Engineering, Springer. Hamilon, J.D. (989). A new approach o he economic analysis of nonsaionary ime series and he business cycle, Ecomomerica, 57, Hardy, M.R. (). A regime-swiching model for long-erm sock reurns, Norh American Acuarial Journal, 5, Honda, T. (3). Opimal porfolio choice for unobservable and regime-swiching mean reurns, Journal of Economic Dynamics & Conrol, 8, Huang, Y., Forsyh, P.A. and Labahn, G. (). Combined fixed poin and policy ieraion for HJB equaions in finance, SIAM Journal on Numerical Analysis, 5, Huang, Y., Forsyh, P.A. and Labahn, G. (). Mehods for pricing American opions under regime swiching, SIAM Journal on Scienific Compuing, 33, Karazas, I. and Shreve, S.E. (998). Mehods of Mahemaical Finance, Springer. Pham, H. (9). Coninuous-Time Sochasic Conrol and Opimizaion wih Financial Applicaions, Springer. Proer, P.E. (5). Sochasic Inegraion and Differenial Equaions, Second Ediion, Springer. Reisinger, C. and Forsyh, P.A. (6). Piecewise consan policy approximaions o Hamilon-Jacobi-Bellman equaions, Applied Numerical Mahemaics, 3, Rieder U. and Bäuerle, N. (5). Porfolio opimizaion wih unobservable Markovmodulaed drif process, Journal of Applied Probabiliy, 43, Sass, J. and Haussmann, U.G. (4). Opimizing he erminal wealh under parial informaion: The drif process as a coninuous ime Markov chain, Finance and Sochasics, 8,

21 Yaari, M.E. (987). The dual heory of choice under risk, Economerica, 55, Yao, D.D., Zhang, Q. and Zhou, X.Y. (6). A regime-swiching model for European opions, in Sochasic Processes, Opimizaion, and Conrol Theory: Applicaions in Financial Engineering, Queueing Neworks, and Manufacuring Sysems, (H.M. Yan, G. Yin, and Q. Zhang Eds.), Springer, pp Yin, G., Zhang, Q., Liu, F., Liu, R.H., and Cheng, Y. (6). Sock liquidaion via sochasic approximaion using NASDAQ daily and inra-day daa, Mahemaical Finance, 6, Zhang, Q., Yin, G. and Liu, R.H. (5). A near-opimal selling rule for a wo-ime-scale marke model, SIAM Journal of Muliscale Modeling & Simulaion, 4, Zhou X.Y. and Yin, G. (3). Markowiz s mean-variance porfolio selecion wih regime swiching: a coninuous-ime model, SIAM Journal on Conrol and Opimizaion, 4, Appendix A: Proof of (8) We verify ha Y T defined in (8) is he soluion of SDE (7) a ime T. Define s ( Xs c = r u + CQ α u + ) s θ u du θ u dw u, Xs j = ln( + C(α u α u )), <u s for s T. Then X c s is a coninuous process and X j s is a pure jump process. Define X s = X c s + X j s and Y s = f(x s ) for s T wih f(x) = y exp(x). Using Iô formula for semi-maringale processes, see Proer (5), we have for < s T, where [X c, X c ] s = dy s = Y s dx c s + Y sd[x c, X c ] s + Y s Y s, s is he pure jump par of Y a s. Since we have θ udu is he quadraic variaion of X c a s and Y s Y s Y s = Y s exp (ln( + C(α s α s ))) = Y s ( + C(α s α s )), Y s Y s = Y s C(α s α s ). Subsiuing [X c, X c ] s and Y s Y s ino dy s, also noing (), we derive SDE (7). Therefore, Y T is he soluion of (7) a ime T.

22 Appendix B: Proof of Theorem 3. Using Io s formula, we have d(x Y ) = X Y ( θ + π σ )dw + X Y CdM, which implies ha he process XY is a local-maringale and herefore a super-maringale due o X Y being non-negaive. This, ogeher wih (6), gives he following relaion E,x,i [U(X T )] E,y,i [Ũ(Y T )] + xy. (3) Since he lef side of (3) is independen of y whereas he righ side of (3) is independen of π, we have, see (4), V i (, x) W i (, x), (33) where W i (, x) is defined by (). Noe ha W i (, x) depends on C. If S is a se of vecors C, hen we mus have () from (33), which gives he igh upper bound. The igh lower bound (6) is obvious as he process π = πα (, X ), T, is an admissible conrol process by he assumpion of he heorem and V i (, x) is he opimal value funcion. Appendix C: Tables 8, 9,, We lis he ables for Examples 4., 4.4, 4.5, and 4.8 wih random choices of generaor marix Q. Table 8: Numerical resuls for Example 4. (power uiliy) wih random choices of generaor marix Q for wo-sae regime swiching. No Conrol C Wih Conrol C a, b α Benchmark LB UB LB UB ave ime diff rel-diff (%) diff rel-diff (%) mean sd

23 Table 9: Numerical resuls for Example 4.4 (non-hara uiliy) wih random choices of generaor marix Q for wo-sae regime swiching. No Conrol C Wih Conrol C a, b α LB UB LB UB ave ime diff rel-diff (%) diff rel-diff (%) mean sd Table : Numerical resuls for Example 4.5 (non-hara uiliy) wih random choices of generaor marix Q for hree-sae regime swiching. No Conrol C Wih Conrol C a, b, c α LB UB LB UB ave ime diff rel-diff (%) diff rel-diff (%) mean sd

24 Table : Numerical resuls for Example 4.8 (Yaari uiliy) wih random choices of generaor marix Q for wo-sae regime swiching. No Conrol C Wih Conrol C a, b α LB UB LB UB ave ime diff rel-diff (%) diff rel-diff (%) mean sd