Numerical Methods for Controlled Hamilton-Jacobi-Bellman PDEs in Finance

Transcription

1 Numercal Methods for Controlled Hamlton-Jacob-Bellman PDEs n Fnance P.A. Forsyth, G. Labahn October 12, 2007 Abstract Many nonlnear opton prcng problems can be formulated as optmal control problems, leadng to Hamlton-Jacob-Bellman (HJB) or Hamlton-Jacob-Bellman-Isaacs (HJBI) equatons. We show that such formulatons are very convenent for developng monotone dscretzaton methods whch ensure convergence to the fnancally relevant soluton, whch n ths case s the vscosty soluton. In addton, for the HJB type equatons, we can guarantee convergence of a Newton-type (Polcy) teraton scheme for the nonlnear dscretzed algebrac equatons. However, n some cases, the Newton-type teraton cannot be guaranteed to converge (for example, the HJBI case), or can be very costly (for example for jump processes). In ths case, we can use a pecewse constant control approxmaton. Whle we use a very general approach, we also nclude numercal examples for the specfc nterestng case of opton prcng wth unequal borrowng/lendng costs and stock borrowng fees. Keywords: Opton prcng, stochastc control, nonlnear HJB PDE 1 Introducton There are a number of fnancal models whch result n nonlnear partal dfferental equatons (PDEs). Examples where such nonlnear PDEs arse nclude transacton cost/uncertan volatlty models [28, 4, 38], passport optons [3, 43], unequal borrowng/lendng costs [13], large nvestor effects [2], rsk control n rensurance [32], prcng optons and nsurance n ncomplete markets usng an nstantaneous Sharpe rato [51, 31, 11], and optmal consumpton [12, 15]. A recent survey artcle on the theoretcal aspects of ths topc s gven n [35]. In many of these cases, the fnancal prcng problems can also be naturally posed as optmal control problems, leadng to nonlnear Hamlton-Jacob-Bellman (HJB) PDEs, partal ntegro dfferental equatons (PIDEs) or Hamlton-Jacob-Bellman-Isaacs (HJBI) equatons. Davd R. Cherton School of Computer Scence, Unversty of Waterloo, Waterloo ON, Canada N2L 3G1 e-mal: paforsyt@uwaterloo.ca Davd R. Cherton School of Computer Scence, Unversty of Waterloo, Waterloo ON, Canada N2L 3G1 glabahn@uwaterloo.ca 1

2 A common approach found n the lterature s to analytcally determne the optmal control, and then substtute ths control back nto the PDE. Unfortunately, ths method leads to PDEs whch are hghly nonlnear and where t s extremely dffcult to desgn numercal schemes whch are guaranteed to converge. In ths paper we consder the dscrete optmal control problem drectly. Our objectve s to provde a general procedure for numercal soluton of sngle factor optmal control problems n opton prcng. We focus on dscretzaton methods whch are uncondtonally stable, and for whch convergence to the fnancally relevant soluton s guaranteed. We place partcular emphass on the nteracton of the dscretzaton technque wth the method used to solve the nonlnear dscretzed algebrac equatons. Along the way we look at two specfc examples whch are nterestng n ther own rght: unequal borrowng/lendng rates and stock borrowng fees. There are many techncal ssues that need to be addressed when solvng optmal control problems drectly. For example, snce we have nonlnear PDEs, the solutons are not necessarly unque. For our problems we need to ensure that our dscretzaton methods converge to the fnancally relevant soluton, whch n ths case s the vscosty soluton [18]. As demonstrated n [38], seemngly reasonable dscretzaton methods can converge to non-vscosty solutons. We show that an optmal control formulaton s n fact qute convenent for verfyng monotoncty, l stablty and consstency of our dscrete schemes. Usng the basc results of [10, 5], ths ensures that our numercal solutons convergence to the vscosty soluton. In terms of exstng soluton methods, there are two basc threads of lterature concernng controlled HJB equatons. One classc approach s based on a Markov chan approxmaton (see for example [27]). In fnancal terms, ths approach s smlar to the usual bnomal lattce, whch s equvalent to an explct fnte dfference method. However, these methods are well-known to suffer from tmestep lmtatons due to stablty consderatons. A more recent approach s based on numercal methods whch ensure convergence to the vscosty soluton of the HJB equaton. Uncondtonally monotone mplct methods are descrbed n [8]. Ths leads to a nonlnear set of dscretzed equatons whch must be solved at each tmestep. It s common n the PDE lterature [8] to suggest relaxaton type methods for soluton of the nonlnear algebrac equatons at each tmestep. However convergence of relaxaton methods can be very slow for fne grds. On the other hand f we requre a monotone scheme, then the dscrete equatons can be related to the dscrete equatons whch occur n nfnte horzon controlled Markov chans. If we solve a dscrete verson of the control problem, then n some cases we can obtan guarantees on the convergence of Newton-type (Polcy) teraton schemes. Nevertheless, there are cases where the nonlnear dscrete equatons are qute dffcult to solve, or the convergence of the teraton s very slow. A case n pont, whch we wll not pursue n ths paper, would be the PIDE case. It may be qute dffcult to solve a local control problem wth a control parameter n the ntegral operator. An alternatve possblty, s to approxmate the acton of the control as pecewse constant n tme [25]. A smple case of ths whch s commonplace n fnance s the approxmaton of an Amercan opton by a Bermudan opton, wth exercse at the end of each tmestep. We use ths same dea for other types of controls. Ths gves a method whch has no tmestep lmtatons due to stablty, and does not requre the soluton of any nonlnear teratons at each tmestep. In ths case the controls must be dscretzed, and an addtonal PDE must be solved (at each tmestep) for each dscrete control. As such, ths approach reduces a sngle 2

3 complex nonlnear problem to a set of lnear problems, wth a nonlnear updatng rule at the end of each tmestep. The man results of ths paper are summarzed as follows: We show that many nonlnear opton prcng problems can be posed as optmal control problems, n partcular unequal borrowng/lendng rates and stock borrowng fees. If the control s handled mplctly, then the control formulaton allows us to easly check the condtons requred to ensure convergence of the dscretzaton to the vscosty soluton. The control formulaton leads to natural Newton-lke teraton schemes for the nonlnear algebrac equatons, whch arse from an mplct treatment of the control. The control problem can also be reformulated as pecewse constant (n tme) to avod solvng nonlnear algebrac equatons, at the expense of solvng a number of lnear problems at each tmestep. A combnaton of the mplct control and pecewse constant control can be used to obtan robust and effcent methods. We nclude numercal examples llustratng these deas for a model wth unequal borrowng/lendng rates and stock borrowng fees. We remark that, whle t s standard n the PDE lterature to use a combnaton of forward and backward dfferencng to ensure monotoncty (and hence mplyng that the error n the space-lke drecton s only frst order), n practcal fnancal applcatons, t s usually possble to use central dfferencng at most nodes, and stll obtan a monotone scheme. Our numercal examples llustrate that our schemes effectvely have second order convergence n most cases. 2 Prelmnares Let V (S, t) be the value of a contngent clam wrtten on asset S whch follows the stochastc process ds = µs dt + σs dz, (2.1) where µ s the drft rate, σ s volatlty, and dz s the ncrement of a Wener process. There are a number of fnancal stuatons where the value of a contngent clam s determned by solvng an optmal control problem. Consder for example, the uncertan volatlty model developed n [4, 30]. Ths provdes a prcng mechansm for cases where volatlty s uncertan, but les wthn a band, σ [σ mn, σ max ]. In ths case, the PDE whch s used to determne the value of a contngent clam s determned by the two extremal volatltes. For a short poston the optmal control problem s gven by q 2 V t + sup 1 S 2 Q ˆQ 2 V SS + SV S rv = 0 (2.2) 3

4 where Q = (q 1 ) and ˆQ = (σ mn, σ max ) and r s the borrowng/lendng rate. Replacng the sup by an nf gves the correspondng prcng equaton for a long poston. A PDE of precsely the same form s obtaned for the Leland model of transacton costs [28]. A second example of an optmal control problem s the passport opton on a tradng account [3, 43]. In ths case the holder of a passport opton s enttled to go long or short an underlyng asset wth value S. At the expry of the contract, the opton holder can receve the accumulated gan on the account W or walk away f W < 0. The Black-Scholes analyss nvolves constructon of a worst case hedgng portfolo whch gves an optmal control problem. If we set V = SU, where V (S, t) s the value of the short poston contract, then the far market prce for U s gven by U t + sup Q ˆQ + σ2 S 2 2 (x q 1) 2 U xx + ( ) (r γ r c )q 1 (r γ r t )x U x γu = 0, (2.3) wth Q = (q 1 ) and ˆQ = ([ 1, 1]). Here γ s the dvdend rate, r c s the cost of carry, r t s the nterest rate on the tradng account and x = W S wth < x <. We note that n ths case the control ranges over a contnuous nterval. However, when the payoff s convex then the controls become ˆQ = ( 1, +1). A thrd, more recent, example of an optmal control problem occurs n prcng certan nsurance contracts n ncomplete markets. One possble method for prcng such contracts s based on an nstantaneous Sharpe rato [31, 11]. In [51], an example s dscussed for the case of hedgng wth an mperfectly correlated asset. Suppose the asset underlyng the contract follows the process (2.1). However, t s not possble to trade n S, rather only n the asset S, whch follows ds = µ S dt + σ S dz wth dz dz = ρ dt. (2.4) For a short poston, the prcng PDE s then ( V t + sup r + q 1 λσ ) 1 ρ 2 SV S + σ2 S 2 Q ˆQ 2 V SS rv = 0, (2.5) where Q = (q 1 ) and ˆQ = ( 1, +1). Here λ s the nstantaneous Sharpe rato and r = µ (µ r) σρ σ. (2.6) Replacng the sup by an nf n equaton (2.5) gves the prce for a long poston. Whle the prevous three prcng PDEs are well known, we also nclude the followng examples of optmal control problems: 4

5 Example 2.1 : Unequal Borrowng/Lendng Rates Consder the case where the cash borrowng rate (gven by r b ) and the lendng rate (gven by r l, wth r b r l ) are not necessarly equal, a model dscussed, for example, n [13, 2]. The prce of an opton V s then gven by the nonlnear PDE (a bref dervaton s gven n Appendx A): where Short Poston: V t + σ2 S 2 2 V SS + ρ(v SV S )(SV S V ) = 0 Long Poston: V t + σ2 S 2 2 V SS + ρ(sv S V )(SV S V ) = 0, (2.7) ρ(x) = rl f x 0 r b f x < 0. (2.8) Notce that for the short poston the nonlnear problem can be posed as σ 2 S 2 V t + sup Q ˆQ 2 V SS + q 1 (SV S V ) = 0, (2.9) where Q = (q 1 ) and ˆQ = (r l, r b ). As before, the prce for a long poston opton s gven by replacng the sup by an nf n equaton (2.9). For a vanlla put, the bank account B = V SV S s always postve. Thus q 1 = r l n equaton (2.9) and the equaton becomes lnear. Smlarly, for a vanlla call the bank account B = V SV S < 0, so that q 1 = r b n equaton (2.9) and the PDE agan becomes lnear. At frst sght, the formulaton (2.9) appears to be unnecessarly complcated compared wth equaton (2.7). However, we wll demonstrate that usng an optmal stochastc control formulaton results n consderable smplfcaton of the analyss of the numercal algorthm used to solve problem (2.9). As well, the control formulaton permts use of a pecewse constant polcy algorthm for numercal soluton of control problem (2.9), a method whch s very straghtforward to mplement. Example 2.2 : Stock Borrowng Fees The borrowng/lendng model n Example 2.1 can be extended to nclude stock borrowng fees. Such fees are effectvely pad to stock lenders when a hedger shorts a stock. The stock borrowng process s descrbed n [21]. In essence, the holder of a short poston wll not receve the rate r l on the proceeds of the short sale, but rather r l r f, where r f s the stock borrowng fee. Typcally, r f can be about 40 bps (.4%) [52]. The nonlnear prcng PDE n the case where t s assumed that retal customers do not receve any nterest on the proceeds of a short sale (.e. r f = r l ), s derved n [13]. The prcng equaton s (for a dervaton of the opton prcng PDE for the general case where r f r l see Appendx A): Short Poston: V t + σ2 S 2 Long Poston: V t + σ2 S 2 2 V SS + H(V S ) [ρ(v SV S )(SV S V )] + H( V S ) [(r l r f )SV S ρ(v )V ] = 0 2 V SS + H( V S ) [ρ(sv S V )(SV S V )] + H(V S ) [(r l r f )SV S ρ( V )V ] = 0, (2.10) 5

6 where ρ(x) s defned n equaton (2.8), and H(y) = 1 f y 0 0 f y < 0 For a short poston we can pose our nonlnear PDE problem as the control problem σ 2 S 2 V t + sup Q ˆQ 2 V SS + q 3 q 1 (SV S V ) + (1 q 3 )[(r l r f )SV S q 2 V ] = 0, (2.11) where Q = (q 1, q 2, q 3 ) and ˆQ = (r l, r b, r l, r b, 0, 1). The prcng equaton for the long poston agan only nvolves replacng sup by nf. Remark 2.1 (Arbtrage Bands) The long and short prces n Examples 2.1 and 2.2 can be consdered to form an arbtrage band [13]. Any market prce hgher than the short prce or lower than the long prce represents an arbtrage opportunty. No arbtrage opportuntes exst for market prces whch le between the long prce and the short prce. Example 2.3 : Amercan Optons Let V be the payoff of an Amercan opton. The prce of an Amercan opton can be wrtten as σ mn V 2 S 2 t 2 V SS + rsv S rv, V V = 0. (2.12) We can also wrte ths as a penalzed control problem: σ 2 S 2 V t + sup 2 V SS + rsv S rv + µ (V V ) = 0, (2.13) ɛ µ 0,1 for ɛ 1. For a dscusson of the penalzed method from an analytcal pont of vew, see [1]. A numercal algorthm based on the penalty method s descrbed n [22]. Example 2.4 : Amercan Optons and Stock Borrowng Fees An nterestng case arses when the Stock Borrowng model n Example 2.2 wth a long poston, s combned wth Amercan early exercse. Ths gves rse to the problem σ mn V 2 S 2 t nf Q ˆQ 2 V SS + q 3 q 1 (SV S V ) +(1 q 3 )[(r l r f )SV S q 2 V ], V V = 0, wth Q = (q 1, q 2, q 3 ), ˆQ = (rl, r b, r l, r b, 0, 1) and where V s the opton payoff. formulate ths as a penalty problem σ 2 S 2 V t + sup nf µ 0,1 Q ˆQ 2 V SS + q 3 q 1 (SV S V )+(1 q 3 )[(r l r f )SV S q 2 V ] + µ (V V ) ɛ 6. (2.14) We can = 0. (2.15)

7 Note the nterestng feature of the sup nf n equaton (2.15). Ths type of problem s commonly referred to as a stochastc game. The PDE n ths case s referred to as the Hamlton-Jacob- Bellman-Isaacs equaton (HJBI). In our case, t s obvous that we can nterchange the nf sup n equaton (2.15), so that the Isaacs condton s satsfed, and we can expect a unque value. All of the above examples can be descrbed as HJB or HJBI equatons. If we assume the underlyng process s a jump process, we would end up wth a controlled partal ntegro dfferental equaton (PIDE). We wll not dscuss the PIDE case further n ths paper, leavng ths case for future work. We wll also not specfcally dscuss sngular or mpulse control problems n ths paper [35]. However, sngular controls can be formulated as a penalzed problem [19]. It s then straghtforward to use the methods descrbed n ths paper to solve the penalzed formulaton of a sngular control. Penalty methods can also be used for mpulse control. 3 General Form for the Example Problems All the methods descrbed n ths paper handle problems such Examples along wth passport optons, uncertan volatlty models, and many other problems n fnance. For concreteness, we wll make use of Examples 2.1 and 2.2 from the prevous secton. As s typcally the case wth fnance problems, we solve backwards n tme from the expry date of the contract t = T to t = 0 by use of the varable τ = T t. Wth a slght abuse of notaton, we now let V = V (S, τ) n the remander of the paper. Set L Q V a(s, τ, Q)V SS + b(s, τ, Q)V S c(s, τ, Q)V, (3.1) where the control parameter Q s n general a vector, that s, Q = (q 1, q 2,...). We wrte our problems n the general form V τ = sup Q ˆQ L Q V + d(s, τ, Q), (3.2) or V τ = nf Q ˆQ L Q V + d(s, τ, Q). (3.3) Here we nclude the d(s, τ, Q) term n equaton (3.2) snce t would be necessary for Amercan optons. As an example note that the coeffcents for equaton (3.2) wth Examples 2.1 and 2.2 are a(s, τ, Q) = σ2 S 2 b(s, τ, Q)) = c(s, τ, Q) = 2 Sq 1 Example 2.1 S(q 3 q 1 + (1 q 3 )(r l r f )) Example 2.2 q 1 Example 2.1 q 3 q 1 + (1 q 3 )q 2 Example 2.2 d(s, τ, Q) = 0. (3.4) 7

8 In the case of Amercan optons wth payoff V, the a(s, τ, Q) and b(s, τ, Q)) reman the same whle the other coeffcents become q 1 + µ c(s, τ, Q) = ɛ Example 2.1 q 3 q 1 + (1 q 3 )q 2 + µ ɛ Example 2.2 d(s, τ, Q) = µ ɛ V (3.5) wth the new set of controls now ncludng the addton of parameter µ 0, 1. We wll assume n the followng that a(s, τ, Q) 0, c(s, τ, Q) 0. In a fnancal context ths corresponds to non-negatve nterest rates and volatltes. In general t s useful for us to explctly separate the penalty term n equaton (3.6) from the non-penalty terms. To be more specfc, we assume that e(s, τ, Q) c(s, τ, Q) = ĉ(s, τ, Q) + ɛ d((s, τ, Q) = ˆd(S, e(s, τ, Q)f(S, τ) τ, Q) + ɛ and where ĉ(s, τ, Q), e(s, τ, Q), f(s, τ, Q) are all nonnegatve. If we have an addtonal set of controls P ˆP, and defne ; ɛ 1 (3.6) L Q,P V a(s, τ, Q, P )V SS + b(s, τ, Q, P )V S c(s, τ, Q, P )V, (3.7) then, wth d = d(s, τ, Q, P ), the HJBI case becomes V τ = sup nf L Q,P V + d(s, τ, Q, P ) Q ˆQ P ˆP. (3.8) For brevty n the followng, we wll only focus on the case wth the sup n equaton (3.2). All the results n the followng sectons hold for the nf case as well. We wll pont out the specal problems that arse when consderng the HJBI case (3.8). 3.1 Boundary Condtons At τ = 0, we set V (S, 0) to the specfed contract payoff. As S 0, we assume a(s, τ, Q) = 0 and b(s, τ, Q) 0 (3.9) so that equaton (3.2) reduces to the problem V τ = max Q ˆQ b(0, τ, Q)V S c(0, τ, Q)V + d(0, τ, Q). (3.10) In order to ensure that classcal solutons exst for the uncontrolled problem, we should have the addtonal condton [34] lm ( b(s, τ, Q) a S(S, τ, Q) ) 0 (3.11) S 0 8

9 so that no boundary condton (other than equaton (3.10)) s requred at S = 0. For the CIR model, the nonunqueness of the classcal soluton when condton (3.11) s not satsfed s dscussed n [23]. As S, we normally use fnancal reasonng to determne the asymptotc form of the soluton. A typcal assumpton s that V SS 0 [50], so that V B(τ)S + C(τ); S. (3.12) We make the approxmaton that the optmal control Q s ndependent of tme and S as S, so that Q can be determned from the payoff as S. Ths then leads to a set of ODEs to solve for B(τ), C(τ) [50], wth B(0), C(0) determned from the contract payoff. We wll assume n the followng that the asymptotc form s known. For computatonal purposes, we solve problem (3.2) on V (S max, τ) = B(τ)S max + C(τ) (3.13) 0 τ T and 0 S S max, (3.14) wth condton (3.10) mposed at S = 0, and the condton (3.13) wth B, C known functons mposed at S = S max. As ponted out n [7], we can expect any errors ncurred by mposng approxmate boundary condtons at S = S max to be small n areas of nterest f S max s suffcently large. Assumpton 3.1 (Propertes of the HJB and HJBI PDE.) We make the assumpton that the coeffcents a, b, c, d are contnuous functons of (S, τ, Q), wth a 0, and c 0 and that a, b, ĉ, e, ˆd, f (equaton (3.6)) are bounded on 0 S S max. Snce we restrct ourselves to a fnte computatonal doman 0 S S max, we avod dffcultes assocated wth coeffcents that grow wth S as S. We also assume that the set of admssble controls ˆQ (for the HJB case) and ˆQ, ˆP (for the HJBI case) are compact (.e. a closed, bounded nterval). It follows from [16, 9] that solutons to equaton (3.4) along wth the boundary condtons (3.10) and (3.13) satsfy the strong comparson property, n the case that the penalty terms are zero. From [1], we know that the penalzed equaton s also a good approxmaton to the vscosty soluton. Comparson results for the HJBI equaton (under more general condtons than dscussed n ths paper) are gven n [29]. Consequently, n all cases, we make the assumpton that the strong comparson property holds, so that a unque vscosty soluton exsts for equatons (3.2), (3.3), and (3.8). Remark 3.1 (Interpretaton of the Strong Comparson Property) As noted n [17], n a fnancal context, the strong comparson property smply states that f W (S, τ) and V (S, τ) are two contngent clams, wth W (S, 0) V (S, 0), then W (S, τ) V (S, τ) for all τ. We wll verfy that the schemes developed n ths paper satsfy a dscrete verson of the comparson prncple. 9

10 4 Dscretzaton In ths secton, we wll ntroduce the basc dscretzaton for the PDE n the general form (3.2), and ntroduce the matrx notaton to be used n the remander of the ths paper. We wll dscuss the concept of a postve coeffcent dscretzaton, whch wll ensure convergence to the vscosty soluton. In addton, the postve coeffcent property wll allow us to prove convergence of teratve schemes for solvng the nonlnear dscretzed algebrac equatons. Defne a grd S 0, S 1,..., S p wth S p = S max, and let V n be a dscrete approxmaton to V (S, τ n ). Let V n = [V0 n,..., V p n ], and let (L Q h V n ) denote the dscrete form of the dfferental operator (3.4) at node (S, τ n ). The operator (3.4) can be dscretzed usng forward, backward or central dfferencng n the S drecton to gve (L Q h V ) = α (Q)V 1 + β (Q)V+1 (α (Q) + β (Q) + c (Q))V.(4.1) Here α, β are defned n Appendx C. It s mportant that central, forward or backward dscretzatons be used to ensure that (4.3) s a postve coeffcent dscretzaton. To be more precse, ths condton s Condton 4.1 Postve Coeffcent Condton α (Q) 0, β (Q) 0, c (Q) 0. = 0,.., p 1 ; Q ˆQ. (4.2) We wll assume that all models have c (Q) 0. Consequently, we choose central, forward or backward dfferencng at each node to ensure that α (Q), β (Q) 0. Note that dfferent nodes can have dfferent dscretzaton schemes. If we use forward and backward dfferencng, then the equaton (C.3) guarantees a postve coeffcent method. However, snce ths dscretzaton s only frst order correct, t s desrable to use central dfferencng as much as possble (and yet stll obtan a postve coeffcent method). Ths s dscussed n detal n [49]. Equaton (3.2) can now be dscretzed usng fully mplct tmesteppng (θ = 0) or Crank- Ncolson (θ = 1/2) along wth the dscretzaton (4.1) to gve V τ V n = (1 θ) sup Q ˆQ (L Q h V ) + d + θ sup Q n ˆQ (L Qn h V n ) + d n. (4.3) These dscrete equatons are hghly nonlnear n general. We refer to methods whch use an mplct tmesteppng method where the control s handled mplctly as an mplct control method n the followng. 4.1 Order of Approxmaton Set ( S) max = max(s +1 S ) and ( S) mn = mn(s +1 S ) and suppose φ(s, τ) s a smooth test functon wth bounded dervatves of all orders wth respect to (S, τ). If φ n = φ(s, τ n ), then usng Taylor seres expansons (and the dscretzaton descrbed 10

11 n Appendx C) verfes that (L Q h φ)n (L Q φ) n = O(( S) max ). (4.4) For φ a smooth test functon, usng equatons (4.4), (B.4), (and Taylor seres expansons) also gves the order of our dscretzaton as [ (φ τ ) sup L Q φ φ n φ + d (1 θ) sup (L Q Q ˆQ τ Q ˆQ h φ ) + d ] θ sup (L Qn Q n ˆQ h φ ) + d n (φ τ ) φ φ n τ + sup L Q φ + d (1 θ) Q ˆQ = O( τ) + O(( S) max ) + θ sup L Q φ + d Q ˆQ (L Q h φ ) + d θ (L Q h φn ) n + d n (L Q h φn ) n + d n = O( τ) + O(( S) max ). (4.5) The last step follows snce the coeffcents of the PDE are assumed contnuous functons of tme. Remark 4.1 (Second Order Error) We have expanded the Taylor seres n equaton (4.5) about the pont (S, τ ). If we expand about the pont (S, τ /2 ) (where τ /2 = (τ + τ n )/2 ) and assume that the PDE coeffcents have bounded second dervatves wth respect to tme, then for θ = 1/2, the tme truncaton error s O(( τ) 2 ). As well, f we assume that the grd n the S drecton s slowly varyng, and that central weghtng s used, then the error n the S drecton wll be O(( S) 2 max). In general, of course, these assumptons may not be justfed. However, n many cases n practce, we observe close to second order convergence at most nodes of nterest f we use Crank-Ncolson weghtng. We requre our dscretzaton to satsfy α, β 0 and so requre a combnaton of forward/backward/central dfferencng choces. Of course we would lke to use central dfferencng as much as possble, rather than forward/backward dfferencng (whch are only frst order correct) (see the algorthm descrbed n [20]). However ths does mply that the dscretzaton n Appendx C s formally only frst order accurate n ( S) max due to the possblty of usng forward/backward dfferencng at some nodes, as well as the unequally spaced grd. In practce, forward/backward dfferencng s usually only requred at a small number of nodes, and usually the grd sze s changed smoothly near regons of nterest. The example computatons wll show near quadratc convergence as the mesh sze s reduced. From a practcal standpont, there are essentally two mportant cases. 11

12 4.2 Q Independent Dscretzaton In some cases, we can preselect central, forward or backward dfferencng ndependent of the optmal control Q, whch ensures that the postve coeffcent condton (4.2) s satsfed. In ths stuaton, the determnaton of the optmal control Q, for gven V, V+1, V 1 s usually straghtforward. As a result, we would expect that teratve soluton of the nonlnear equatons (4.3) s at least feasble. The followng method s used to preselect the dscretzaton method at each node [51]. We process each node n turn, frst testng to see f central dfferencng satsfes (4.2), for any Q ˆQ. If ths s the case, then we use central dfferencng at ths node, and proceed on to the next node. If central dfferencng does not ensure a postve coeffcent dscretzaton, then forward and backward dfferencng are tested. We remark that for the problems n Examples 2.1 and 2.2, as long as r l r f 0, then one of central or forward dfferencng wll satsfy the postve coeffcent condton, for an arbtrary choce of grd, for any Q ˆQ. In some cases, Q ndependent dscretzaton may not be possble for an arbtrary grd, but can be acheved for small enough node spacng. Usually, the problem nodes are few n number, and located near S 0, that s, where the dffuson term s small. In ths case, we can often take an arbtrary grd, and nsert a relatvely small number of nodes, whch wll guarantee that Q ndependent dscretzaton wll satsfy (4.2). An example of ths node nserton algorthm s gven n [51]. 4.3 Q Dependent Dscretzaton Unfortunately there are some stuatons where no matter how fne the grd, t may not be possble to preselect the type of dscretzaton at each node whch wll ensure that the postve coeffcent condton (4.2) s satsfed at each node for any Q ˆQ. Ths s the case for passport optons when there are non-convex payoffs [37, 49]. In ths case, the dscretzaton at node (central, forward or backward) wll depend on Q. Of course, the optmal value of Q wll now depend on the dscretzaton. In addton, for gven V, V+1, V 1, determnaton of the optmal value for Q may not be straghtforward. Ths follows snce the dscretzed equatons are contnuous functons of Q f forward and backward dfferencng only are used for the frst order terms, but the dscrete equatons wll not, n general, be contnuous functons of Q f central weghtng s used as much as possble. Ths ssue s dscussed n detal n [49]. In the followng, we wll not requre that the dscrete equatons be a contnuous functon of the control, to allow for the case descrbed n [49]. 4.4 Matrx Form of the Dscrete Equatons It wll be convenent to use matrx notaton for equatons (4.3), coupled wth boundary condtons (3.10) and (3.13). Let the boundary condtons at S = S max and tme τ n be gven by F n p = B(τ n )S max + C(τ n ), (4.6) 12

13 where S p = S max and B(0), C(0) determned from the payoff. Set We can wrte the dscrete operator (L Q h V n ) as V n = [V n 0, V n 1,..., V n p ] and Q = [Q 0, Q 1,..., Q p ] (4.7) (L Q h V n ) = [A(Q)V n ] = [ α n (Q)V 1 n + β n (Q)V+1 n (α n (Q) + β n (Q) + c n (Q))V n ] ; < p. (4.8) The frst and last rows of A are modfed as needed to handle the boundary condtons. The boundary condton at S = 0 (equaton (3.10)) s enforced by settng α = 0, and usng forward dfferencng for the frst order term at = 0. For notatonal consstency, ths s consstent wth the above f we defne V 1 n = 0. Let F n = [0,..., 0, Fp n ]. The boundary condton at = p s enforced by settng the last row of A to be dentcally zero. Wth a slght abuse of notaton, we denote ths last row as (A n (Q)) p 0. In the followng, t wll be understood that equatons of type (4.8) hold only for < p, wth (A n (Q)) p 0. Let D n (Q) be the vector wth entres [D(Q)] n = d n (Q), < p 0, = p. Remark 4.2 (Matrx Supremum Notatonal Conventon) In the followng, we wll denote [A (Q)V + D (Q) ] (4.9) by sup Q ˆQ A (Q )V + D (Q ) (4.10) [A where Q arg sup (Q)V + D (Q) ]. (4.11) Q ˆQ If the local objectve functon s a contnuous functon of Q, then, snce ˆQ s compact, the supremum s smply the maxmum value, and Q s the pont where a maxmum s attaned. If the local objectve functon s dscontnuous, we nterpret A (Q ) as the approprate lmtng value of [A (Q)] whch generates the supremum, at the lmt pont Q. A specfc example of an algorthm for computng ths lmt pont s gven for the case of maxmzng the usage of central weghtng as much as possble n [49]. Note that Q s not necessarly unque. The dscrete equatons (4.3) can be wrtten as [ I (1 θ) τa (Q ) ] V = [I + θ τa n (A n )] V n + (1 θ) τd (Q ) +θ τd n (Q n ) + (F F n ), [A where Q arg sup (Q)V + D (Q) ] Q ˆQ = 0,..., p 1. (4.12) 13

14 Here the term (F F n ) enforces the boundary condton at S = S p. Recall that Crank Ncolson (θ = 1/2) or fully mplct (θ = 0) tmesteppng s used. It wll be convenent to defne the followng ( τ) max = max n (τ τ n ) and ( τ) mn = mn n (τ τ n ) where we assume that there are mesh sze/tmestep parameters h mn, h max such that ( S) max = C 1 h max, ( τ) max = C 2 h max, ( S) mn = C 3 h mn, ( τ) mn = C 4 h mn (4.13) wth C 1, C 2, C 3, C 4 postve constants ndependent of h. We can then wrte the dscrete equatons (4.3) or (4.12) at each node n the followng form G (h max, V, V +1, V 1, V n, V n +1, V n 1) = 0 where V V n (1 θ) τ G F τ F n sup Q ˆQ ( ) A (Q )V + D (Q ) ( ) θ sup A n (Q n )V n + D n (Q n ) Q n ˆQ. (4.14) To avod longwnded notaton, we shall occasonally wrte G (h max, V, Vj where Vj j s the set of values Vj, j = 1,..., p. V n j j, V n j ) G (h max, V, V +1, V 1, V n, V n +1, V n 1), 5 Convergence to the Vscosty Soluton (4.15), j = 1,..., p, j, and Vj n s the set of values In [38], examples were gven n whch seemngly reasonable dscretzatons of nonlnear opton prcng PDEs were unstable or converged to the ncorrect soluton. It s mportant to ensure that we can generate dscretzatons whch are guaranteed to converge to the vscosty soluton [5, 18]. Assumng that equaton (3.2) satsfes the strong comparson property [6, 9, 16], then, from [10, 5], a numercal scheme converges to the vscosty soluton f the method s consstent, stable (n the l norm) and monotone. To be precse, we defne these terms. Defnton 5.1 (Stablty) Dscretzaton (4.14) s stable f V C 5, (5.1) for 0 n N, T = N τ, for ( τ) mn 0, ( S) mn 0, where C 5 s ndependent of ( τ) mn, ( S) mn. 14

15 Defnton 5.2 (Consstency) Scheme (4.14) s consstent f, for any smooth functon φ, wth φ n = φ(s, τ n ), we have ( ) lm φτ sup L Q φ + d G h max 0 (h max, φ, φ +1, φ 1, φn, φ n +1, φ n 1) = 0. Q ˆQ For the general case where the operator s degenerate, a more complcated defnton of consstency s requred n order to handle boundary data [5]. In our case, the degeneracy occurs at S 0, and boundary condton (3.10) s smply the lmt of equaton (3.2) as S 0. As such ths problem does not arse. The most nterestng requrement s monotoncty. Defnton 5.3 (Monotoncty) The dscrete scheme (4.14) s monotone f for all ɛ l j 0 and G (h max, V, V j + ɛ j Stablty and consstency are easly establshed. j, V n j + ɛ n j ) G (h max, V, Vj (5.2) j, V n j ). Lemma 5.1 (Stablty) If the dscretzaton (4.14) satsfes the postve coeffcent condton (4.2), and boundary condtons are mposed at S = 0 and S = S max, as n equaton (3.10) and (3.13), then the scheme (4.12) satsfes (for S max fxed, and recallng the defntons of ˆd, f n equaton (3.6)) (5.3) V n max( V 0 + C 6, C 7, C 8 ) (5.4) where C 6 = T max,n ˆd n, C 7 = max,n F n, and C 8 = max,n f n provded that τ θ (α n + β n + c n ) 1 ;. (5.5) Proof. For the fully mplct case (θ = 0), the dscrete equatons are, for < p, V = V n τ ( α + β + ĉ + e ) V ɛ + τα V 1 + τβ V+1 + τ ˆd + e τ f ɛ and Vp = Fp when = p. To avod notatonal clutter, we have suppressed the Q dependence n equatons (5.6). It wll be understood that the coeffcents are the lmtng values at the optmal Q. From equaton (5.6), we obtan V ( 1 + τ(α + β + ĉ + e ) ) V n + V τ(α + β ) ɛ 15 (5.6) + e τf + τ ˆd. (5.7) ɛ

16 If V = Vj, j < p, then equaton (5.7) gves V ( 1 + τĉ j + τ e j ) V n + e ɛ j τfj ɛ + τ ˆd j, (5.8) or, lettng f max = max j f j and ˆd max = max j ˆd j, we obtan V max( V n, fmax) + τ ˆd max. (5.9) If j = p then V = Vp and so equaton Vp = Fp gves Combnng equatons (5.9) and (5.10) gves V = F p. (5.10) V max( V n, fmax, Fp ) + τ ˆd max, (5.11) whch then results n equaton (5.4). A smlar seres of steps for θ > 0 shows that the dscretzaton s stable provded condton (5.5) holds. Lemma 5.2 (Consstency) If the dscrete equaton coeffcents are as gven n Appendx C, then the dscrete scheme (4.14) s consstent as defned n Defnton 5.2. Proof. Ths follows from equaton (4.5). The fact that a dscretzaton of a control problem whch satsfes the postve coeffcent condton (4.2) results n a monotone scheme was noted n [8]. Ths result holds for both Q dependent and Q ndependent dscretzatons (see Sectons 4.2 and 4.3). It s nstructve to nclude a proof of ths result, snce t llustrates the mportance of maxmzng/mnmzng the dscretzed equatons. Lemma 5.3 (Monotoncty) If the dscretzaton (4.14) satsfes the postve coeffcent condton (4.2), boundary condtons are mposed at S = 0 and S = S max, as n equaton (3.10) and (3.13), and the stablty condton (5.5) s satsfed, then dscretzaton (4.14) s monotone as defned n Defnton 5.3. Proof. Consder the fully mplct case (θ = 0 n equaton (4.14)). For = p, the Lemma s trvally true. For < p, we wrte equaton (4.14) out n component form G (h, V, V +1, V 1, V n = V τ V n ) + nf (α Q ˆQ (Q) + β (Q) + c (Q))V α (Q)V 1 β (Q)V +1 d (Q). (5.12) 16

17 For ɛ 0, we have G (h, V, V +1 + ɛ, V 1, V n ) G (h, V, V +1, V 1, V n ) (α (Q) + β (Q) + c (Q))V α (Q)V 1 β (Q)V +1 β (Q) = nf Q ˆQ (Q)ɛ d nf (α Q ˆQ (Q ) + β (Q ) + c (Q ))V α (Q )V 1 β (Q )V +1 d (Q ) sup β Q ˆQ (Q)ɛ = ɛ nf β Q ˆQ (Q) 0, (5.13) whch follows from equaton (B.2) and the fact that β (Q) 0. Smlarly (θ = 0), G (h, V, V+1, V 1 + ɛ, V n ) G (h, V, V+1, V 1, V n ) 0. (5.14) It s obvous from equaton (5.12) that (θ = 0) G (h, V, V +1, V 1, V n + ɛ) G (h, V, V +1, V 1, V n ) 0. (5.15) Fnally, for the general case wth θ 0, a smlar argument verfes that property (5.3) holds, as long as the stablty condton τ θ ( α n (Q) + β n (Q) + c n (Q) ) 1 ;, Q ˆQ, (5.16) s satsfed. Remark 5.1 (Extenson to Other Cases) Usng propertes (B.3), (B.6), we can replace the sup n equaton (5.13) by an nf, or a sup nf (wth two control varables Q, P as n equaton (3.8)) and the dscretzaton s monotone for these cases as well. Theorem 5.1 (Convergence to the Vscosty Soluton) Provded that the orgnal HJB satsfes Assumpton 3.1 and dscretzaton (4.12) satsfes all the condtons requred for Lemmas 5.1, 5.2, 5.3, then scheme (4.12) converges to the vscosty soluton of equaton (3.2). Proof. Ths follows drectly from the results n [10, 5]. It s also useful to note that [I (1 θ)a n (Q n )] s an M-matrx [47]. Remark 5.2 (Propertes of M-Matrces) An M-matrx B has the propertes that B 1 0 and dag(b 1 ) > 0. Lemma 5.4 (M-matrx) If the postve coeffcent condton (4.1) s satsfed, and boundary condtons (3.10,3.13) are mposed at S = 0, S max, then [I (1 θ) τa n ] s an M-matrx. Proof. Condton (4.1) mples that α n, βn, cn n equaton (4.8) are non-negatve. Hence [I (1 θ) τa n ] has postve dagonals, non-postve offdagonals, and s dagonally domnant, so t s an M-matrx [47]. 17

18 5.1 Dscrete Comparson Property It s nterestng to verfy that the dscrete equatons satsfy a dscrete verson of the Comparson Property (see Remark 3.1). Consder any two contngent clams W (S, τ), V (S, τ). If V (S, 0) W (S, 0), then by no arbtrage V (S, τ) W (S, τ). It s clearly desrable that dscrete solutons of the prcng PDEs also have these dscrete arbtrage nequaltes. Theorem 5.2 (Dscrete Arbtrage Inequalty) Suppose (a) the dscretzaton (4.8) satsfes the postve coeffcent condton (4.2), (b) boundary condtons are mposed at S = 0 and S = S max, as n equaton (3.10) and (3.13), wth boundary condton vector F n = [0,..., F n p ], (c) fully mplct tmesteppng s used. If W n and V n are two dscrete solutons to equaton (4.12), wth V n W n, wth boundary condton vectors F V F W, then V W. Proof. In the case of fully mplct tmesteppng, equaton (4.12) becomes V = V n + τ sup Q ˆQ A (Q )V + D (Q ) W = W n + τ sup Q ˆQ + (F V F n V ) (5.17) A (Q )W + D (Q ) + (F W F W n ) (5.18) Subtractng equaton (5.18) from equaton (5.17), and usng equaton (B.2), gves (V W ) = (V n W n ) + τ sup A (Q )V + D (Q ) Q ˆQ τ sup A (Q )W + D (Q ) Q ˆQ + (F V F n V ) (F W F n W ) (V n W n ) + (F V FV n ) (F W F W n ) + τ nf Q ˆQ A (Q)(V W ). (5.19) Let Q arg nf Q ˆQ A (Q)(V W ), so that equaton (5.19) becomes [I τa ( Q)](V W ) (V n W n ) + (F V F n V ) (F W F n W ). (5.20) By assumpton (V n W n ) + (F V FV n ) (FW F W n ) 0 (recall that F V, F W are dentcally zero except at = p where (F V ) n p = (V n ) p, (F W ) n p = (W n ) p ). Snce [I τa ( Q)] s an M matrx (from Lemma 5.4), we have that (V W ) 0. (5.21) 18

19 6 Soluton of Algebrac Dscrete Equatons Although we have establshed that dscretzaton (4.12) s consstent, stable and monotone, t s not obvous that ths s a practcal scheme, snce the mplct tmesteppng method requres soluton of hghly nonlnear algebrac equatons at each tmestep. In ths secton we gve two methods for solvng these algebrac equatons - one a relaxaton scheme and the second a Newton-lke (Polcy) teraton. 6.1 A Relaxaton Scheme Wrtng out equaton (4.12) n component form gves (for each < p) V = (1 θ) τ sup Q ˆQ α (Q)V 1 (Q)V+1 (α (Q) + β (Q) + c (Q))V + d (Q) + g n (6.1) ] where g n = V n +θ τ [A n V n + D n. Rearrangng equaton (6.1) and notng that α, β, c are all nonnegatve, we obtan V = sup Q ˆQ (1 θ) τ α (Q)V 1 + β (Q)V+1 + d (Q) (1 + (1 θ) τ)(α (Q) + β (Q) + c (Q)) g n + (1 + (1 θ) τ)(α (Q) + β (Q) + c (Q)). (6.2) Let ˆV k+1 be the (k + 1) estmate for V. Equaton (6.2) can then be used as a bass for the relaxaton scheme ˆV k+1 = sup Q ˆQ (1 θ) τ α (Q) ˆV 1 k + β (Q) ˆV +1 k + d (Q) (1 + (1 θ) τ)(α (Q) + β (Q) + c (Q)) g n + (1 + (1 θ) τ)(ᾱ k + β k + ck ). (6.3) Ths leads us to a constructve proof for the exstence of a unque soluton for the dscretzed equatons. Theorem 6.1 (Convergence of Relaxaton) Suppose that (a) the dscretzaton (4.8) satsfes the postve coeffcent condton (4.2), (b) boundary condtons are mposed at S = 0 and S = S max, as n equaton (3.10) and (3.13). 19

20 Then a unque soluton of the nonlnear equatons (6.1) exsts. Furthermore, the teraton scheme (6.3) s globally convergent for any ntal estmate. Proof. Wrtng equaton (6.3) for teraton k, and usng equaton (B.4) gves ˆV k+1 ˆV k γ ˆV k ˆV k 1 (1 θ τ)[α (Q) + β (Q)] γ = max sup Q ˆQ 1 + (1 θ τ)[α (Q) + β (Q) + c (Q)]. (6.4) Snce α n (Q), βn (Q), cn (Q) are nonnegatve for all Q ˆQ, we have that γ < 1. Thus the scheme (6.3) s a contracton and converges to the unque soluton of the dscretzed algebrac equatons. Remark 6.1 (Exstence of soluton: HJBI case) The above argument can be repeated f we replace the sup n equaton (6.3) by an nf or a sup nf. Hence, n all cases (HJB, or HJBI), the scheme (6.3) s a contracton. Although the soluton V s unque, the control may not be unque. Unfortunately, ths relaxaton scheme s not very useful n practce. To see ths consder the trval case where Q s constant. In ths stuaton, scheme (6.3) s smply a relaxaton method for the soluton of a dscretzed parabolc PDE. Recallng the defnton of the dscretzaton parameter h mn n equaton (4.13), ths mples that the error reducton n each teraton of scheme (6.3) s γ O(h mn ) (6.5) whch s very poor as h mn 0. Remark 6.2 (Markov Chans) Consder equaton (6.2) and, for smplcty, let θ = 0. wrte V = sup P, 1 V 1 + P,+1 V +1 + U Q ˆQ Then where P, 1 = τα ω, P,+1 = τβ and U ω = gn + τd (6.6) ω wth ω = (1 + τ)(α + β + c ). Snce 0 P,j 1 and j P,j < 1, we can dentfy the P,j as dscounted rsk neutral transton probabltes. Hence, at each tmestep, we can consder equaton (6.6) as the soluton of an nfnte horzon controlled Markov chan [27]. We wll refer to equaton (6.6) as the Markov chan form of the dscretzed equatons. Note that the Markov chan form necessarly puts some terms nvolvng the control parameter n the denomnator of the probabltes P,j. Ths can cause some complcatons, and varous methods have been suggested to remedy ths problem [27]. However, ths rearrangement s somewhat unnatural from the PDE pont of vew. 20

21 Remark 6.3 (Value Iteraton) We can vew the teraton (6.3) as smlar to the famlar value teraton n stochastc control [27]. In ths context, the problem s usually formulated as a dscrete Markov chan, as n Remark Polcy Iteraton It would seem desrable to have a scheme whch converged n one teraton f Q s constant. Ths leads us to the followng teratve scheme. Polcy Iteraton Let (V ) 0 = V n Let ˆV k = (V ) k For k = 0, 1, 2,... untl convergence EndFor Solve [ I (1 θ) τa (Q k )] ˆV k+1 = [I + θ τa n (Q n )] V n + (F F n ) If (k > 0) and max + (1 θ) τd (Q k ) + θ τd n [ Q k arg sup A (Q) ˆV k + D (Q)] Q ˆQ k+1 ˆV ˆV k ( max scale, ˆV k+1 ) < tolerance then qut (6.7) The term scale n scheme (6.7) s used to ensure that unrealstc levels of accuracy are not requred when the value s very small. Typcally, scale = 1 for optons prced n dollars. Some manpulaton of algorthm (6.7) results n [ ] I (1 θ) τa (Q k ) ( ˆV k+1 ˆV [ k ) = (1 θ) τ (A (Q k ) ˆV k + D (Q k )) (A (Q k 1 ) ˆV ] k + D (Q k 1 )). (6.8) We can also wrte equaton (6.8) as [ ] I (1 θ) τa (Q k ) ( ˆV k+1 ˆV k ) = R k, (6.9) where the resdual R vector s R k = ˆV k V n [ ( (1 θ) τ A (Q k ) ˆV ) ] k + D (Q k ) + H n (6.10) 21

22 wth H n = θ τ ( A n (Q n )V n + D n) + (F F n ). (6.11) In order to prove the convergence of Algorthm (6.7), we frst need an ntermedate result. Lemma 6.1 (Sgn of RHS of Equaton (6.8)) If A (Q k ) ˆV k s gven by equaton (4.8), wth the control parameter determned by [ Q k arg sup A (Q) ˆV k + D (Q)], (6.12) Q ˆQ then every element of the rght hand sde of equaton (6.8) s nonnegatve, that s, [ (A (Q k ) ˆV k + D (Q k )) (A (Q k 1 ) ˆV ] k + D (Q k 1 )) 0. (6.13) Proof. Recall that Q k s selected so that A (Q k ) ˆV k + D (Q k ) = sup Q ˆQ A (Q) ˆV k + D (Q). (6.14) for gven ˆV k. Hence, any other choce of coeffcents, for example A (Q k 1 ) ˆV k + D (Q k 1 ) (6.15) cannot exceed equaton (6.14). It s now easy to show that teraton (6.7) always converges. Theorem 6.2 (Convergence of Iteraton (6.7)) Provded that the condtons requred for Lemmas 6.1 and 5.4 are satsfed, then the nonlnear teraton (6.7) converges to the unque soluton of equaton (4.12) for any ntal terate ˆV 0. Moreover, the terates converge monotoncally. Proof. Gven Lemmas 6.1 and 5.4, the proof of ths result s smlar to the proof of convergence gven n [38]. We gve a bref outlne of the steps n ths proof, and refer readers to [38] for detals. A straghtforward maxmum analyss of scheme (6.7) can be used to bound ˆV k ndependent of teraton k. From Lemma 6.1, we have that the rght hand sde of equaton (6.8) s nonnegatve. Notng that [ I (1 θ) τa (Q k ) ] s an M-matrx (from Lemma 5.4) and hence [ I (1 θ) τa (Q k ) ] 1 0, t s easly seen that the terates form a bounded non-decreasng sequence. In addton, f ˆV k+1 = ˆV k the resdual s zero. Hence the teraton converges to a soluton. It follows from the M-matrx property of [ I (1 θ) τa (Q k ) ] that the soluton s unque. The above proof can be repeated wth the sup replaced by nf n equaton (6.7). 22

23 Remark 6.4 (Q Dependent Dscretzatons) Note that we obtan convergence for the case of Q dependent dscretzatons, even f the dscrete equatons, regarded as a functon of the control Q, are dscontnuous. Ths s dscussed n [49]. Remark 6.5 (Polcy Iteraton) Iteraton (6.7) s essentally the well known polcy teraton n stochastc control [42]. It dffers slghtly n that we do not use the Markov chan rearrangement of the dscrete equatons, as n equaton (6.6). Hence, the teraton sequence wll be dfferent than the classcal polcy teraton (a dfferent local control problem s solved at each node), but the convergence result s the same. Snce we do not rearrange the dscrete equatons nto the Markov chan form, we do not have the dffcultes assocated wth control parameters appearng n the denomnator of the dscrete equatons, as dscussed n Remark 6.2. Remark 6.6 (Equvalence of Iteraton (6.7) and Newton Iteraton) Suppose that that there s a sngle control at each node Q, and that the sup control s unconstraned. Then, from equaton (6.10), assumng that the dscrete equatons are dfferentable, we have But R k ˆV k j = Rk Q k Q ( ) ˆV + δ j k j (1 θ) τa j (Q k ). (6.16) R k Q k = 0 (6.17) snce Q k s locally optmal. Hence the teraton [ I (1 θ) τa k] ( ˆV k+1 ˆV k ) = R k, (6.18) whch s equvalent to teraton (6.7), s a Newton teraton. Of course, n general the coeffcents may not be dfferentable, and the control parameters are constraned. Nevertheless, as dscussed n [40, 39, 42], we may vew teraton (6.7) as a Newton-lke teraton (quadratc convergence when close to soluton). Remark 6.7 (Polcy Iteraton: HJBI Equaton) For the case of the HJBI equaton (problems wth a sup nf, equaton (3.8)), t s not clear when teraton (6.7) can be expected to converge. The convergence argument breaks down n ths case, snce we cannot expect Lemma 6.1 to hold. However, as dscussed n [36], we can also nterpret polcy teraton as a form of Newton-lke teraton, for the case of a fnte set of controls. In ths case, we can expect convergence, even for the stochastc game case, f the ntal estmate s suffcently close to the soluton. 7 Pecewse Constant Polces The relaxaton scheme (also known as value teraton) (6.3) from the prevous secton s globally convergent to the unque soluton of the dscretzed equatons for both HJB and HJBI equatons. However, the convergence rate becomes unacceptably slow as the grd sze s reduced. 23

24 The polcy teraton scheme (6.7) s globally convergent but only for HJB equatons. Snce ths method can be regarded as a Newton-lke teraton, convergence wll typcally be very rapd f the ntal estmate s suffcently close to the soluton. In typcal opton prcng problems, where we have the soluton from the prevous tmestep as the ntal guess, convergence generally occurs n 2 3 teratons f sx dgt accuracy s specfed. Unfortunately, there are examples where the convergence rates can be slow. In [42], an example wth dscrete controls s constructed whereby the teraton (begnnng from the zero state), takes R 1 steps, where R s the number of states (whch would correspond to nodes n our case). In some cases, t may also be a nontrval problem to solve the local control problem (6.12). Ths may be especally dffcult f jump processes are modelled, whch results n a controlled partal ntegrodfferental equaton (PIDE) [24]. In addton, the polcy teraton scheme does not guarantee global convergence of (6.7) for HJBI equatons. Indeed there are pathologcal cases where polcy teraton does not converge for these problems (c.f. [48]). Ths has led to the development of several varants of Newton teraton whch attempt to ensure global convergence for these problems [45, 14, 46]. In ths secton we consder an alternate tmesteppng method, one whch s guaranteed to converge to the vscosty soluton, does not have tmestep szes lnked to the mesh sze (whch precludes explct methods), and does not requre soluton of nonlnear equatons at each step. 7.1 An Informal Approach The basc dea behnd the pecewse constant polcy approxmaton s ntutvely appealng. Suppose an agent s allowed to make changes n the control only at dscrete forward tmes t, = 1,..., L. We wll also assume that the agent can choose from only a fnte number of controls, that s, all possble control choces can be enumerated Q m, m = 1,..., m max (for example m max = 2 n Example 2.1 and m max = 8 n Example double f we are lookng at Amercan optons). In the case that the control varables are contnuous, we approxmate the control by a fnte set of pecewse constant polces. Let τ = T t and V m be the soluton to (V m ) τ = L Qm V m + d(q m ), (7.1) where L Qm denotes the operator (3.1) for a fxed value of Q m. In other words, V m s the soluton to the optmal control problem wth the trval constant polcy Q m. At t = T, τ = 0, we set V m (S, 0) = Opton Payoff ; m. (7.2) Now suppose the agent s at t = t L, the last decson tme before the contract expry at t = T. In order to determne the optmal polcy at τ = T t L = τ L, the agent examnes all possble choces of the the polcy, and chooses the polcy whch maxmzes the value of the contract. Ths s smply done by solvng (V m ) τ = L Qm V m + d m wth d m = d(q m, S, τ), (7.3) from τ = 0 to τ = τ L for all m = 1,..., m max. The optmal value s then determned smply from V opt (S, τ L ) = max m V m(s, τ L ). (7.4) 24

25 We then set V m (S, τ L + ɛ) = V opt (S, τ L ) ; ɛ > 0, ɛ 1 ; m, (7.5) and repeat the above procedure at τ = τ L 1, and so on. If the tmes between decson dates t are small, and we have used a large enough sample of the polcy space Q, then ths should be a good approxmaton to the orgnal control problem (3.2). 7.2 A Formal Approach More precsely, consder the followng algorthm. tmesteppng. For smplcty, we consder only fully mplct Pecewse Constant Polcy Tmesteppng V 0 = Opton Payoff For n = 0,..., // Tmestep Loop V n,m = V n ; = 1,..., p m = 1,..., m max For m = 1,..., m max Solve (I τa (Q m ))V /2 m = V n m + τd (Q m ) EndFor V = max V /2 j,j ; = 1,..., p 1 Vp = Fp EndFor // End Tmestep Loop (7.6) Note that we have used a slghtly dfferent tme dscretzaton here compared wth equaton (4.12), wth the boundary condton updated explctly. We wll now verfy that that ths scheme satsfes the suffcent condtons for convergence. Lemma 7.1 (Stablty of Scheme (7.6)) If the dscretzaton (4.8) satsfes the same condtons as for Lemma 5.1, then the same stablty result (Lemma 5.1) holds for pecewse constant polcy tmesteppng. Proof. Ths follows usng the same maxmum analyss as used n the proof of Lemma 5.1. Showng consstency s a more challengng problem. In order to determne f the consstency condton (5.2) s satsfed, we need to elmnate Vm /2 from equaton (7.6). Let V /2 m = H m (V n ) [ ] 1 [ ] = I τa (Q m ) V n + τd (Q m ). (7.7) 25