Numerical Methods for Hamilton-Jacobi-Bellman Equations

Transcription

1 Unversty of Wsconsn Mlwaukee UWM Dgtal Commons Theses and Dssertatons May 27 Numercal Methods for Hamlton-Jacob-Bellman Equatons Constantn Gref Unversty of Wsconsn-Mlwaukee Follow ths and addtonal works at: Part of the Mathematcs Commons Recommended Ctaton Gref, Constantn, "Numercal Methods for Hamlton-Jacob-Bellman Equatons" (27). Theses and Dssertatons Ths Thess s brought to you for free and open access by UWM Dgtal Commons. It has been accepted for ncluson n Theses and Dssertatons by an authorzed admnstrator of UWM Dgtal Commons. For more nformaton, please contact krstnw@uwm.edu.

2 Numercal Methods for Hamlton-Jacob-Bellman Equatons by Constantn Gref A Thess Submtted n Partal Fulfllment of the Requrements for the Degree of Master of Scence n Mathematcs at The Unversty of Wsconsn - Mlwaukee May 27

3 Abstract Numercal Methods for Hamlton-Jacob-Bellman equaton by Constantn Gref The Unversty of Wsconsn - Mlwaukee, 27 Under the Supervson of Professor Bruce A. Wade In ths work we consdered HJB equatons, that arse from stochastc optmal control problems wth a fnte tme nterval. If the dffuson s allowed to become degenerate, the soluton cannot be understood n the classcal sense. Therefore one needs the noton of vscosty solutons. Wth some stablty and consstency assumptons, monotone methods provde the convergence to the vscosty soluton. In ths thess we looked at monotone fnte dfference methods, sem lagragan methods and fnte element methods for sotropc dffuson. In the last chapter we ntroduce the vanshng moment method, a method not based on monotoncty.

4 Copyrght by Constantn Gref, 27 All Rghts Reserved

5 TABLE OF CONTENTS Hamlton-Jacob-Bellman Equatons. Outlne HJB n Optmal Control Problems Dynamc Programmng Prncple Fully Nonlnear Second Order PDEs Challenges Monotone Methods Nonmonotone Methods Dfferent Types of HJB equatons Tme-dependent Case Tme-ndependent Case Infnte Tme-horzon Case Maxmsng Problem Examples Vscosty Soluton and the Barles-Sougands Convergence Argument 2 2. Vscosty Soluton Motvaton Defnton Fnal Value Problem and Comparson The Barles-Sougands Convergence Argument Usng Howard s Algorthm Idea Problem statement Howard s Algorthm Computatonal Implementaton Applcaton Fnte Dfference Methods 27

6 4.. Problem Statement Well-Posedness Approxmaton n Space Approxmaton of Kushner-Dupus Approxmaton of Bonnans and Zdan Fully Dscrete Scheme Fast Algorthm for 2 Dmensons Applcaton Sem-Lagrangan Schemes Idea Problem Statement Well-Posedness Defnton of SL-Schemes Collocaton Method Analyss Specfc SL Schemes Lnear Interpolaton SL Schemes Stochastc Control Applcaton Fnte Element Method for Isotropc Dffuson Idea Problem Statement Defnton of the Numercal Method Numercal Scheme Numercal Method Soluton Algorthm Analyss Well-Posedness Consstency propertes Super- and Subsoluton Unform Convergence Method of Artfcal Dffuson Applcaton Vanshng Moment Method Idea v

7 7.2 General Framework Fnte Element Method n 2d Parabolc Case A Notatons 59 A. Sobolev Spaces A.2 Norms A.3 Inner products A.4 Vectors, Matrces and Functons Bblography 6 v

8 . Hamlton-Jacob-Bellman Equatons In ths thess, we are searchng for the numercal soluton of a class of second-order fully nonlnear partal dfferental equatons (PDE), namely the Hamlton-Jacob-Bellman (HJB) equatons. These PDE are named after Sr Wllam Rowan Hamlton, Carl Gustav Jacob and Rchard Bellman. The equaton s a result of the theory of dynamc programmng whch was poneered by Bellman. In contnuous tme, the result can be seen as an extenson of earler work n classcal physcs on the Hamlton-Jacob equaton. The HJB equatons we consder arse from optmal control models for stochastc processes... Outlne In ths Chapter we brefly descrbe how HJB equatons arse from stochastc optmal control problems. Then n Chapter 2 we wll ntroduce the concept of vscosty solutons and we wll look at the Barles- Sougands Argument, whch guarantees us the convergence to the vscosty soluton for monotone schemes. In Chapter 3 we wll explan Howard s Algorthm, whch s ncluded n many methods solvng the HJB. In Chapter 4 we wll look at monotone fnte dfference methods. In Chapter 5, we wll look at Sem-Lagrangan Schemes, whch also get the convergence through the monotoncty argument. In Chapter 6 we wll look at monotone fnte element methods for sotropc problems. In Chapter 7 we wll look at a very dfferent concept, whch s ths of the vanshng moment method.

9 . Hamlton-Jacob-Bellman Equatons.2. HJB n Optmal Control Problems Optmal control problems descrbe the tme evoluton of a state vector X : t Ω R d related to a control process λ : t Λ, where Λ s the set of admssble control values. If not other stated, Λ wll be a compact metrc space. If we say λ( ) Λ, then we mean a measurable functon wth λ t Λ almost everywhere. The state vector X satsfes a gven stochastc dfferental equaton (SDE), whose drft µ R d and dffuson matrx σ R d p are functons whch depend on the control λ( ) Λ: dx t = µ λt (t, X t )dt + σ λt (t, X t )dw t for t >, X = x, (.) where W t s a gven p-dmensonal Wener process. The task s to mnmze a gven cost functonal J λ (t, x) R wth (λ, t, x) Λ [, T ) Ω, dependent on functons f λt (t, x) R and g(t, x) R. Then we can defne the value functon as [ τ u(t, x) := nf J λ (t, x),where J λ (t, x) := E λ Λ t ] f λs (s, X s )ds + g(τ, X τ ), (.2) where τ = nf{s t (s, X s ) / (, T ) Ω} s the fnal tme of our problem when the state X t leaves the open doman set Ω T = (, T ) Ω. The control problem (.2) wth the value functon u leads to the followng HJB equaton: t u + nf λ Λ [Lλ u + f λ ] = n Ω T, u = g on Ω T = {T } Ω (, T ) Ω, (.3) where the lnear operator L λ s defned by: L λ v := Tr[a λ D 2 v] + µ λ v, v H 2 (Ω), λ Λ, wth a λ := 2 σ(σ) R d R d, the symmetrc postve semdefnte dffuson coeffcent matrx and 2

10 . Hamlton-Jacob-Bellman Equatons D 2 v denotes the Hessan matrx after x. Equaton (.3) s now solved by the value functon we just defned n (.2). Example.2..(Control problem wth explct soluton) If the drft s gven by µ λ (t, X t ) = c X t + c 2 λ t, wth c and c 2 constants, the dffuson s also just a constant σ λ (t, X t ) = σ and the cost functon s gven by f λ (t, X t ) = r(t)λ2 2 + l(t)x2 t 2, where r(t) > and l(t) > are functons just dependend on tme. Then the HJB equaton s gven by t u + nf λ Λ [ (c x + c 2 λ) x u σ xx u + r(t)λ2 2 + l(t)x2 ] = 2 Through a basc calculaton by dervaton after λ, we see that the unque exact soluton at tme t s gven by λ t = c 2 x u r(t), whch leads to the followng HJB equaton wthout an nf( ) operator ( t u = c x + c2 2 xu 2r(t) ) x u + σ xx u l(t)x2 2 Ths s just an example, n ths Thess we have the focus on cases, where t can t be solved analytcally. We wll cte the Theorem n []. Theorem.2.2.(Krylov) If the followng hold: The control set Λ s compact, Ω s bounded Ω s of class C 3 (roughly speakng, the boundary s locally the graph of a C 3 functon), The functons a λ, µ λ, f λ are n C( Ω T Λ) wth ther t partal dervatve and frst and second x partal dervatves for all λ Λ, h C 3 ([, T ] R d ), 3

11 . Hamlton-Jacob-Bellman Equatons and furthermore, there exsts γ > such that for every (t, x) Ω T and λ Λ, L λ s unformly ellptc,.e. for a λ (t, x) holds d a λ j(t, x)ξ ξ j γ ξ 2 for all ξ R d.,j= Then the HJB equaton has a unque classcal soluton u C( Ω T ) wth contnuous t partal dervatve and contnuous frst and second x partal dervatves. Remark.2.3. If we allow the HJB equaton to become degenerate, a unque classcal soluton s not guaranteed anymore. Ths s why we need the concept of vscosty solutons. Under sutable assumptons, whch does not nclude unformly ellptcy, the HJB equaton (.3) has a unque, bounded, Hölder contnuous, vscosty soluton u..2.. Dynamc Programmng Prncple The HJB equaton s a result of the dynamc programmng prncple of Bellman, whch allows us to splt the value functon. Theorem.2.4. For every t [t, τ] and y Ω wth X t = y, we have { u(t, y) = nf E λ Λ [ t t ] } f λs (s, X s )ds + u(t, X t ) (.4) By knowng ths, we can apply the Dynamc Programmng Prncple. Means, nstead of lookng for u(, x ), we can go backwards. Therefore we start wth u(t, x) = g(t, x) for (t, x) Ω T and { [ t k+ ]} then nductvely, by knowng u(t k+, ), we get u(t k, x) = nf E f λs (s, X s )ds + u(t k+, X tk+ ), λ Λ t k [ t k+ ] where most numercal methods then approxmate E f λs (s, X s )ds t L λ k u + t f λ. Theorem.2.5. Assume that the value functon s n u C( Ω T ) and u = g on Ω T. Then wth the dynamc programmng prncple for u, we get u to be a soluton of the belongng HJB equaton. t k 4

12 . Hamlton-Jacob-Bellman Equatons.3. Fully Nonlnear Second Order PDEs The second order PDE (.3) s fully nonlnear, because the dependence on D 2 u s not lnear, snce the Hessan s ncluded n an nf( ), or often sup( ), operator n the PDE. The HJB equaton would be lnear, f the control set Λ was a sngleton. To solve fully nonlnear PDEs, there are classcal and weak soluton concepts and theores. It s well known that for a class of fully nonlnear second order PDEs, a C 2,α a pror estmate s provded by the celebrated Evans-Krylov Theorem [4]. The nonlnearty of the hghest order dervatve n (.3) makes t mpossble to use a weak soluton concept based on the ntegraton by parts approach, lke we would do for lnear, quas-lnear or semlnear PDEs. So n fact, there was no weak soluton concept for fully nonlnear PDEs untl Crandall and Lons [7] ntroduced the noton of vscosty solutons for frst order fully nonlnear PDEs. Then ther noton and theory were quckly extended to second order fully nonlnear PDEs, lke we use n ths work..3.. Challenges For solvng PDEs numercally there are three man classes, or none of the below. Methods based on drectly approxmatng dervatves by dfference quotents. Methods based on varatonal prncples and approxmatng nfnte-dmensonal spaces by fntedmensonal spaces. Methods based on fnte bass expansons and approxmatng PDEs at samplng ponts. Unfortunately none of those methods work rght away for fully nonlnear second order PDEs. A nave applcaton of the frst and thrd class can already lead to very bad results. And lke mentoned before, the second class can not even be formulated due to the nonlnearty. 5

13 .4. Monotone Methods. Hamlton-Jacob-Bellman Equatons In [] Barles and Sougands provded a general convergence theory for a broad class of possbly degenerate fully nonlnear ellptc and parabolc PDEs. In specfc, that the underlyng PDE satsfes a certan maxmums prncple and that the FDM s monotone, consstent and stable n the sense that the sequence of approxmatons {u h } h remans bounded n the maxmums norm, then t can be shown that u u h L for stepsze h. Some of the frst computatonal methods for HJB equatons n stochastc control are based on approxmatng the underlyng SDE by a dscrete Markov chan. Later t became clear that under some assumptons these are equvalent to monotone fnte dfference methods [3]. The computatonal practce of monotone methods has lagged behnd ther theoretcal development, especally for strongly ansotropc problems. Two man problems are the lower order convergence rate n the frst place and the necessary choce of wde stencl wdth. In fact, to acheve monotoncty for strongly ansotropc problems, compact stencls cannot offer consstence and monotoncty. But ncreasng the stencl wdth ncreases the truncaton error. In the degenerate case, there are examples, where no fnte stencl can yeld a monotone dscretzaton, so t needs to ncrease when the grd s refned. Bonnans and Zdan showed n [3] the number of condtons needed for the dffuson coeffcent. Bonnans gave n [2] an fast algorthm for computng monotone schemes n 2-dm wth fnte stencl and a consstency error dependng on the stencl wdth. Debraband and Jakobsen gave n [8] a sem-lagrangan framework n whch the stencl wdth also contnuously ncreases as the mesh s refned. One advantage of these methods s, that the monontcty s guaranteed for h. The argument of Barles and Sougands can not be appled for fnte element schemes rght away, snce t s made for fnte dfference schemes. But Ian Smears and Jensen were stll able [2] to create a monotone fnte element scheme for possble degenerate sotropc HJB equatons. It was shown that ths method converges to the vscosty soluton n the L norm. 6

14 .5. Nonmonotone Methods. Hamlton-Jacob-Bellman Equatons To guarantee monotoncty and consstency one needs wde stencls, whch causes hgh truncaton errors and therefore reduces the accuracy. In fact, monotone schemes are n practce behnd ther theoretcal development. Therefore many authors proposed dfferent nonmonotone methods n order to avod the stencl restrctons. One of them s the vanshng moment method, whch nvolves fourth order perturbatons to the PDE. Non of the non-monotone methods currently offers a satsfactory convergence analyss. Nevertheless, some methods have offered good computatonal results..6. Dfferent Types of HJB equatons HJB equatons can have dfferent forms, lke t v + H = or t v H = or H =, where H s the Hamltonan, n our case (.3) H = nf λ Λ [Lλ v + f λ ]. To show that dfferent types of the HJB equaton arse from the same control problem, we wll use the followng lemma. Lemma.6.. For a compact set A and contnuous functon F : A R, we have sup a A [F (a)] = nf [ F (a)] a A Proof. nf [ F (a)] F (a) for all a A = nf a A s a upper boundary,.e. nf [ F (a)] sup[f (a)]. a A a A a A [ F (a)] F (a) for all a A = nf [ F (a)] And smlarly, sup[f (a)] F (a) for all a A = sup[f (a)] F (a) for all a A = t s a a A a A lower boundary,.e. sup[f (a)] nf [ F (a)] = sup[f (a)] nf [ F (a)]. a A a A a A a A a A 7

15 . Hamlton-Jacob-Bellman Equatons.6.. Tme-dependent Case If we consder the mnmsng stochastc control problem lke before, where we defned the value functon as u(t, x) := nf λ Λ J λ (t, x), and wth Ω = R d, then we get the HJB equaton wth termnal condton: t u + nf λ Λ [Lλ u + f λ ] = n Ω T, u(t, x) = g(t, x) wth x Ω, If we change the value functon to v(t, x) := u(t, x), then we get the followng HJB t v + sup[l λ v f λ ] = n Ω T, λ Λ u(t, x) = g(t, x) wth x Ω, If we substtute the tme by defnng ũ(t, x) := u(t t, x), then we get the followng HJB wth ntal condton t ũ nf λ Λ [Lλ ũ + f λ ] = n Ω T, ũ(, x) = g(t, x) wth x Ω, whch s equvalent to t ũ + sup[ L λ ũ f λ ] = n Ω T, λ Λ ũ(, x) = gt, x) wth x Ω, If we do both at the same tme w(t, x) := u(τ t, x), then we get t w + sup[l λ w f λ ] = n Ω T, λ Λ w(, x) = g(t, x) wth x Ω, 8

16 . Hamlton-Jacob-Bellman Equatons whch s equvalent to t w sup[l λ w f λ ] = n Ω T, λ Λ w(, x) = g(t, x) wth x Ω Tme-ndependent Case In ths case, the HJB equaton s ellptc and we have nf λ Λ [Lλ v + f λ ] = n Ω. (.5) Or f we defne v(x) := u(x), we get sup[l λ u f λ ] = n Ω. (.6) λ Λ.6.3. Infnte Tme-horzon Case Case 3 s, when we have a nfnte tme-horzon, then f we consder the value functon: [ u(x) = nf E λ Λ ] f λs (s, X s )e γs ds subject to dx t = µ λt (t, X t )dt + σ λt (t, X t )dw t for t >, (.7) X = x, we get the HJB equaton γu nf λ Λ [Lλ u + f λ ] = n Ω, u = on Ω, (.8) Remark.6.2. If we have γ =, t s the ellptc case (.5) of the HJB equaton. 9

17 . Hamlton-Jacob-Bellman Equatons.6.4. Maxmsng Problem If we are dealng wth a maxmsng problem we get the HJB [ τ u(t, x) = sup E ] f λs (s, X s )ds + h(τ, X τ ), (.9) λ Λ t t u + sup[l λ u + f λ ] = n Ω T, λ Λ u = g on Ω T, (.) Remark.6.3. It s possble to rewrte methods for tme-dependent HJBs to tme-ndependent or nfnte tme nterval HJBs, snce the approxmaton of the Hamltonan s the challengng task..7. Examples Example.7.. Let s consder the stochastc control problem wth value functon u(t, x) = mn λ E[ τ t whch means we want to leave the doman Ω T as soon as possble to mnmze ths ntegral. Then wth Λ = {, }, f λ =, σ λ =, µ λ = λ, the HJB equaton to ths s gven by ds], t u(t, x) sup{ λ u(t, x) } = for t (, ) and x (, ) λ Λ u = on {} (, ) (, ) {, }. Wth unque vscosty soluton u(t, x) = mn( t, x ). See Fgure (2.2). Example.7.2. If we consder the HJB equaton wth f λ (t, x) = sn(x ) sn(x 2 )[( + 2β 2 )(2 t) ] 2(2 t) cos(x ) cos(x 2 ) sn(x + x 2 ) cos(x + x 2 ), c λ (t, x) = µ λ (t, x) =, σ λ (t, x) = ( sn(x + x 2 ) β ) 2 wth β 2 =.. In ths Case the HJB eqauton s lnear and the sn(x + x 2 ) β

18 . Hamlton-Jacob-Bellman Equatons 2 sn(x ) sn(x 2 ) x x 2 3 Fgure..: u(t, x, x 2 ) = (2 t) sn(x ) sn(x 2 ), plot for t = soluton of ths s u(t, x) = (2 t) sn(x ) sn(x 2 ). (.) See Fgure (.)

19 2. Vscosty Soluton and the Barles-Sougands Convergence Argument Defnton 2...(degenerate ellptc) An operator F : R d R R d S n (R) R s called degenerate ellptc on Ω f for all x Ω, r R P, Q S n (R) wth P Q and y R d we have F (x, r, y, P ) F (x, r, y, Q). We call an operator t +F degenerate parabolc, f F (, t,,, ) s degenerate ellptc for all t (, T ). Defnton 2..2.(proper) An operator F : R d R R d S n (R) R s called proper on Ω f for all x Ω, r, l R wth r l, P S n (R) and y R d we have F (x, r, y, P ) F (x, l, y, P ). We call an operator t + F proper, f F (, t,,, ) s proper for all t (, T ). Recall the HJB operator t u + sup[ L λ u f λ ]. (2.) λ Λ It can be proven, that ths operator s n fact degenerate parabolc and proper. 2.. Vscosty Soluton Wthout further requrements, general fully nonlnear PDEs second order, lke our HJB equaton, do not necessarly have a classcal soluton. Because of the nonlnearty on the hghest order dervatve 2

20 2. Vscosty Soluton and the Barles-Sougands Convergence Argument Fgure 2..: vscosty soluton,.e. ϕ t (x ) + F [ϕ(x )], rght: vs. supersoluton,.e. ϕ t (x ) + F [ϕ(x )] n (2.2), we can also not extend a weak soluton concept based on the ntegraton by parts approach for fully nonlnear PDEs. So n general there s no varatonal/weak formulaton for fully nonlnear PDEs. In 983 Crandall and Lons [7] ntroduced the noton of vscosty solutons and establshed ther theory for the Hamlton-Jacob equatons of frst order. For the defnton of frst order, we refer to Chapter 7. The noton and theory of vscosty solutons to fully nonlnear second order got extended by Jensen, who establshed the unqueness of solutons and by Ish, who proved the exstence of solutons. Vscosty solutons are a mathematcal concept to select the value functon u from the possbly nfnte set of weak solutons for the HJB equaton Motvaton To motvate the noton of vscosty solutons, suppose for a moment that F s degenerate ellptc and u s a C 2 -functon satsfyng F (D 2 u(x), u(x), u(x), x) (resp. F [u(x)] ). Suppose further that ϕ s also a C 2 -functon satsfyng u ϕ (resp. u ϕ) and u ϕ has a local maxmum (resp. mnmum) at x Ω R d, wthout loss of generalzaton the maxmum s allocated at zero, so u(x ) = ϕ(x ). Then elementary calculus tells us that u(x ) = ϕ(x ) and D 2 u(x ) D 2 ϕ(x ) (resp. D 2 u(x ) D 2 ϕ(x )). So we get F (D 2 ϕ(x ), ϕ(x ), ϕ(x ), x ) F (D 2 u(x ), u(x ), u(x ), x ) (resp. F [ϕ(x )] F [u(x )] ). Consder Fgure (2.) 3

21 2. Vscosty Soluton and the Barles-Sougands Convergence Argument Defnton Wth the observaton above, we gve the defnton of a vscosty soluton for t u + F (x, t, u, u, D 2 u) = on Ω T (2.2) Defnton 2...(contnuous case) Consder F : R d d R d R R d R s a contnuous non-lnear functon. () A functon u C (Ω) s called a vscosty subsoluton of (2.2) f, for every C 2 functon ϕ such that u ϕ has a local maxmum at x Ω, there holds ϕ t + F (D 2 ϕ(x ), ϕ(x ), ϕ(x ), x ). () A functon u C (Ω) s called a vscosty supersoluton of (2.2) f, for every C 2 functon ϕ(x) such that u ϕ has a local mnmum at x Ω, there holds t ϕ + F (D 2 φ(x ), φ(x ), φ(x ), x ). A functon u C (Ω) s called a vscosty soluton of (2.2) f t s both, a vscosty subsoluton and a vscosty supersoluton. Ths defnton can be generalzed to the case when both F and u are just bounded functons, whch can be easly done usng the lower and upper sem contnuous envelopes of F and u. Defnton 2..2.(sem contnuous envelope) For u B( Ω) we defne the lower sem-contnuous envelope as u (x) = lm nf y x and smlarly the upper sem-contnuous envelope as u(x), for x Ω u (x) = lm sup u(x), for x Ω y x Defnton Consder F : R d d R d R R d R and u : Ω R are bounded functons. () u s called a vscosty subsoluton of (2.2), f for every C 2 functon ϕ such that u ϕ has a local maxmum at x Ω, there holds t ϕ + F (D 2 ϕ(x ), ϕ(x ), ϕ(x ), x ). () u s called a vscosty supersoluton of (2.2), f for every C 2 functon ϕ(x) such that u ϕ has a local mnmum at x Ω, there holds t ϕ + F (D 2 ϕ(x ), ϕ(x ), ϕ(x ), x ). 4

22 2. Vscosty Soluton and the Barles-Sougands Convergence Argument Then u s called a vscosty soluton of (2.2) f t s both, a vscosty subsoluton and a vscosty supersoluton. Geometrcally speakng, u s a vscosty soluton f, for every C 2 functon ϕ that touches u from above at x, there holds F [ϕ(x )] and f ϕ touches the graph of u from below at x, there holds F [ϕ(x )]. Example If we consder the followng PDE t u(t, x) + u(t, x) = wth t (, ] and x (, ) u = on {} (, ) (, ) {, }. (2.3) Then wth Λ = {, }, f λ =, σ λ =, µ λ = λ, we get the HJB equaton t u(t, x) sup{ λ u(t, x) } = for t (, ] and x (, ) λ Λ u = on {} (, ) (, ) {, }. (2.4) Then the unque vscosty soluton for ths equaton s gven by u(t, x) = mn( t, x ). (2.5) Theorem Let u C( Ω T ) C 2, (Ω T ). Then u s a vscosty soluton of (2.2) f and only f u s a classcal pontwse soluton of (2.2) Fnal Value Problem and Comparson In vscosty soluton theory for second order fully nonlnear equatons the comparson prncple gves us the unqueness of a vscosty soluton. Generally speakng, the comparson prncple asserts that f F s ellptc, u, v are, respectvely, a vscosty subsoluton and a vscosty supersoluton,.e., f t u + F [u] and t u + F [v] and u v on Ω T, then u v n all of Ω T. Clearly such a result leads to unqueness of vscosty solutons, namely, f t u + F [u] = t v + F [v] n Ω and u = v on Ω T, then u = v. Indeed, f u, v are two vscosty solutons wth u = v on Ω T, then 5

23 2. Vscosty Soluton and the Barles-Sougands Convergence Argument x t.6.8 Fgure 2.2.: mn( t, x ) t u + F [u], t v + F [v], and u v, mplyng u v n Ω T. Swtchng the roles of u and v we obtan u v and so u = v. For that, we wll consder (2.2) wth boundary: t u + F (x, t, u, u, D 2 u) = n Ω T u = g on Ω T (2.6) Defnton An upper sem-contnous functon u USC( Ω T ) s a vscosty subsoluton of (2.6) f u s a vscosty subsoluton of the (2.2) n the sense of the above defnton and u g on Ω T. A lower sem-contnous functon u LSC( Ω T ) s a vscosty supersoluton of (2.6) f u s a vscosty supersoluton of (2.2) n the sense of defnton and u g on Ω T. A functon u C( Ω T ) s a vscosty soluton f both holds true. Now we wll cte some theorems n [9]. Theorem Let Ω R d be open and bounded. Let F C([, T ] Ω R R d S n (R)) be contnuous, proper, and degenerate ellptc wth the same functon w. If u s a subsoluton of (2.6) and v s a supersoluton of (2.6) then u v on [, T ) Ω. 6

24 2. Vscosty Soluton and the Barles-Sougands Convergence Argument Assumpton We assume that there exsts C such that for all λ Λ, x, y R d and t, x [, T ] µ λ (t, x) µ λ (s, y) C( x y + t s ) σ λ (t, x) σ λ (s, y) C( x y + t s ) µ λ (t, x) C( + x ) (2.7) σ λ (t, x) C( + x ) Assumpton f λ (t, x) f λ (s, y) C( x y + t s ) f λ (t, x) C( + x ) (2.8) g λ (t, x) C( + x ) Theorem 2... Gven assumptons (2..8) and (2..9) then there s at most one vscosty soluton to the HJB fnal value problem. Theorem 2... Provded that the value functon s unformly contnous up to the boundary. u s a vscosty soluton of the HJB equaton wth no boundary and f furthermore u = g on the boundary, then u s a vscosty soluton of the HJB wth boundary The Barles-Sougands Convergence Argument Barles and Sougands showed n [] the convergence of a wde class of approxmaton schemes to the soluton of fully nonlnear second order degenerate ellptc or degenerate parabolc PDE s. They proved that any monotone, stable and consstent scheme converges to the unque vscosty soluton, provded that there exsts a comparson prncple, whch s the case n our settng. 7

25 2. Vscosty Soluton and the Barles-Sougands Convergence Argument Agan, consder the fully nonlnear operator t u + F (D 2 u, u, u) = n R d u(, x) = u (x) n R d, (2.9) whle F s contnuous n all of t s arguments and degenerate ellptc. Defnton We say (2.9) satsfes the strong comparson prncple for a bounded soluton, f for all bounded functons u USC and v LSC t holds: u s a vscosty subsoluton v s a vscosty supersoluton the boundary condton holds n the vscosty sense max{ t u F [u], u u } on {} R d mn{ t u F [u], u u } on {} R d, then we have u v on [, T ] R d. Let s consder the general numercal scheme K(h, t, x, u h (t, x), [u h ] t,x ) = for (t, x) G h \ {t = }, where h = ( t, x), G h = t{,,..., n d } xz d, [u h ] t,x stands for the value of u h at other ponts than (t, x). () Monotoncty. K(h, t, x, r, a) K(h, t, x, r, b) whenever a b, where ths monotoncty assumpton can be weakened. We only need t to hold approxmately, wth an error that vanshes to as h goes to zero. 8

26 2. Vscosty Soluton and the Barles-Sougands Convergence Argument () Solvablty and Stablty. For arbtrary h >, there exsts a soluton u h B( Ω T ) to K[u h ] =, x Ω (2.) also, there also exsts a constant C > such that u h L C. () Consstency. For all x Ω and ϕ C there holds K(h, s, y, ϕ(y) + ξ, ϕ + ξ) lm sup t ϕ + F (D 2 ϕ(x), ϕ(x), ϕ(x), x) (h,t s,y x,ξ) h K(h, s, y, ϕ(y) + ξ, ϕ + ξ) lm nf t ϕ + F (D 2 ϕ(x), ϕ(x), ϕ(x), x) (h,t s,y x,ξ) h Remark If F s not contnuous n all of ts arguments, then F has to be replaced by ts upper and and lower sem-contnuous envelopes F and F, respectvely. To approxmate a degenerate parabolc PDE t u + F [u] =, n our case F [u] := Hu wth Hu := sup λ (L λ u f λ ), we consder a not more specfed sequence of numercal schemes n the -th refnement level K [u ](s k, x l ) =, wth solutons u, where {x l } l s the set of grd ponts and {s k } k the set of tme nodes. Under a stablty condton, one can defne the upper and lower envelope of the sequence by u (t, x) := u (t, x) := sup (s k,xl ) N (t,x) lm sup u (s k, x l ) nf lm nf u (s k (s k,xl ), x l ). N (t,x) We obvously get then u u. Then n [?] t s proven, that f u w has a strct local maxmum, for smooth w, also u I w has a strct local maxmum for a nearby pont (s k, yl ), where I s a nodal 9

27 2. Vscosty Soluton and the Barles-Sougands Convergence Argument nterpolaton operator. A certan monotoncty assumpton mples = K [u ](s k, x l ) K [Iw](s k, x l ), together wth the consstency condton K [I w](s k, x l ) t w(t, x) + Hw(t, x) we get t w(t, x) + Hw(t, x). Therefore u s a subsoluton. Smlar argument leads to u beng a supersoluton. Fnally wth a comparson prncple subsolutons are bounded from above by supersolutons u u, whch gves convergence. Theorem Assume that the problem (2.9) satsfes the strong comparson prncple for bounded functons. Assume further that the scheme satsfes the consstency, monotoncty and stablty propertes then t s Soluton u h converges locally unformly to the unque vscosty soluton of (2.9). Remark For example n [3] Oberman descrbed why monotone schemes are necessary and gave an example of a scheme whch s stable, but nonmonotone and nonconvergent. 2

28 3. Usng Howard s Algorthm 3... Idea The HJB equaton lke n (.3) s naturally related to lnear nondvergence form equatons wth dscontnuous coeffcents. The relaton between these lnear and nonlnear problems s that nondvergence form lnear operators can be vewed as lnearsatons of the fully nonlnear operator. Howard s Algorthm can be nterpreted as a Newton method for a nonlnear operator equaton. Another name s polcy teraton and t s ncluded n several numercal methods Problem statement Here we consder the followng fully non-lnear HJB equaton. { t u + nf L λ u + f λ} = n Ω T λ Λ u = g on Ω T (3.) wth L λ v = Tr[a λ D 2 v] + µ λ v (3.2) defned lke n Chapter. 3.. Howard s Algorthm [ τ ] We consder the value functonal J λ (t, x) := E f λs (s, X s )ds + h(τ, X τ ). For the optmal control λ t and tme t, the value functonal s the value functon u(t, x) = J λ (t, x). 2

29 3. Usng Howard s Algorthm Lemma 3... Let J λ (t, x) be the value functonal. Then J λ s also the soluton of the boundary value problem correspondng to the arbtrary but fxed control law λ Λ: t J λ + L λ J λ + f λ =, n Ω T wth boundary data J λ (t, x) = g(t, x), for (t, x) Ω T (3.3) Wth ths nformaton we can defne a successve approxmaton algorthm. The sequence of control laws s gven by λ k+ = arg mn λ Λ {Lλ (t, X t )J λ k + f λ (t, X t )} (3.4) Wth that, we get L λk+ (t, X t )J λk + f λk+ (t, X t ) L λk (t, X t )J λk + f λk (t, X t ). Now let J λk+ be defned as the soluton of (3.3) correspondng to the new control law λ k+ : t J λk+ + L λk+ J λk+ + f λk+ =, on Ω T wth boundary data J λk+ (t, x) = g(t, x), for (t, x) Ω T (3.5) If we contnue defnng the sequences of the control laws λ k and ther belongng value functonals J λk lke above, then we get the followng Lemma and Theorem. Lemma The sequence {J λk } k N satsfes: J λk+ J λk (3.6) Theorem λ k together wth J λk converge to the optmal feedback control law λ and the value functon u(x, t) of our optmal control problem : Ths gves us the followng Algorthm lm k λk = λ lm k J λk (t, x) = u(t, x) 22

30 3. Usng Howard s Algorthm Algorthm For k =, Choose ntal control law λ Λ. Get the functonal J λk (t, x) by solvng the boundary value problem for the already known control law λ k : t J λk + L λk J λk + f λk =, on Ω T wth boundary J λk (t, x) = g(t, x), for (t, x) Ω T Compute the control law λ k+, by solvng: λ k+ = arg max λ Λ {f λ (t, X t ) + L λ (t, X t )J k } k = k + and go back to the second step 3.2. Computatonal Implementaton For usng our successve Algorthm, we stll need to solve the PDE (3.5) and the Optmzaton problem (3.4). We can takle (3.5) wth standard methods for solvng lnear parabolc PDEs, lke the heat equaton. Then ths can be solved by fnte dfference schemes wth upwnd dfferences for the frst order dervatves and mxed dervatves for the second order dervatves. Now we just have a fnte grd, so also the optmzaton problem (3.4) just needs to be solved for every grd pont. Remark Many full methods have Howard s algorthm ncluded, lke for example the fnte element method we wll look at n Chapter Applcaton For the -dmensonal applcatons I dd use the followng scheme. Frst we approxmate the PDE (3.5) to u(t n+, x ) u(t n, x t ) t + µ λ D ± u(t n, x ) + σ λ u(t n, x + ) 2u(t n, x ) + u(t n, x ) ( x) 2 + f λ, (3.7) 23

31 3. Usng Howard s Algorthm where D ± stands for the upwnd operator related to the drft µ D ± ϕ(t n, x ) = ϕ(t n, x ) ϕ(t n, x ) x D ± ϕ(t n, x ) = ϕ(t n, x + ) ϕ(t n, x ) x, f µ >, f µ < Then we defned ths as F (y), where y = u(t n, x ). Then we appled the Newton method y k+ = y k F (y k) F (y k ) to get the soluton F (y) =. We appled ths method to the followng examples Example For the HJB t u + sup{λ xx u + ( x 2 )} = n (, ) (, ) λ Λ u(t, x) = for (t, x) {} [, ] (, ) {, }, (3.8) wth Λ = {,.5, }. Then the soluton of ths PDE s u(t, x) = ( t)( x 2 ). The numercal results are n Fgure (3.) Example For the HJB t u + sup{ λx xx u + λ x u π cos(πt)25x 2 } = n t (, ) (, ) λ Λ u(t, x) = sn(πt)(5x) 2 for (t, x) {} [, ] (, ) {, }, (3.9) wth Λ = {,.5, }. Then the soluton of ths PDE s u(t, x) = sn(πt)(5x) 2. The numercal results for h = are n Fgure (3.2) 24

32 3. Usng Howard s Algorthm.5.5 x t ( t) ( x 2 ).5.5 x t.6.8 Fgure 3..: 25

33 3. Usng Howard s Algorthm 2.5 x t sn(π t) (5 x) x t Fgure 3.2.: top: numercal soluton, botton: exact soluton 26

34 4. Fnte Dfference Methods Here we consder two dfferent fnte dfference approxmatons n space and then the θ method for the approxmaton n tme. For smplcty we take h = ( t, x) and consder the unform grd G h = t{,, 2,..., K} xz d and G + h = G h \ {t = } 4... Problem Statement Here we consder the followng fully non-lnear dffuson equatons { t u + sup L λ u c λ u f λ} = n Ω T = R d (, T ] λ Λ (4.) u(, x) = u (x) on R d wth L λ [u](t, x) = Tr(a λ (t, x)d 2 u(t, x)) + µ λ (t, x) u(t, x) (4.2) defned lke n Chapter Well-Posedness We wll use the followng assumptons on the ntal value problem (4.) Assumpton 4... For any λ Λ, a λ,β = 2 σλ σ λ for some d p matrx σ λ. There s a constant K ndependent of λ such that u + σ λ + µ λ + c λ + f λ K 27

35 4. Fnte Dfference Methods Ths assumpton ensures that we get a well-posedness bounded Lpschtz contnuous (resp. to the value x Ω) value functon, whch satsfes the comparson prncple. Proposton If assumpton (5..) holds. Then there exsts a unque soluton u of the ntal value problem (4.) and a constant C only dependng on T and K from the assumpton such that we have u C. Furthermore, f u and u 2 are sub- and supersolutons of (5.) satsfng u (, ) u 2 (, ), then t holds u u Approxmaton n Space To get the approxmaton n space we approxmate L λ by a fnte dfference operator L λ h L λ h ψ(t, x) = η S C λ h (t, x, η)(ψ(t, x + η x) ψ(t, x)) for (t, x) G h, (4.3) where the stencl S s a fnte subset of Z d \ {} and where Ch λ (t, x, η) for all η S, (t, x) G+ h, h = ( t, x) >, λ Λ, (4.4) whch gves us a dfference approxmaton of postve type, whch s a suffcent assumpton for monotoncty n the statonary case Approxmaton of Kushner-Dupus We denote by {e } d the standard bass for Rd and we defne d L λ hψ(t, x) = = ( a λ (x, t) ψ(x, t) + [ j a +,λ j (x, t) + j d = ψ(x, t) a,λ j (x, t) j ψ(x, t) ]) [µ +,λ δ + ψ(x, t) µ,λ δ ψ(x, t)], (4.5) 28

36 4. Fnte Dfference Methods where µ + = max{µ, }, µ = mn{µ, } and δ + ψ(x, t) := ψ(x + e x, t) ψ(x, t), x δ ψ(x, t) := ψ(x, t) ψ(x e x, t), x ψ(x, t) := ψ(x + e x, t) 2ψ(x, t) + ψ(x e x, t) x 2, + ( jv(x, t) := ψ(x + e 2 x 2 x + e j x, t) + 2ψ(x, t) + ψ(x e x e j x, t) ) ( ψ(x + e 2 x 2 x, t) + ψ(x e x, t) + ψ(x + e j x, t) + ψ(x e j x, t) ), ( jψ(x, t) := ψ(x + e 2 x 2 x e j x, t) + 2ψ(x, t) + ψ(x e x + e j x, t) ) + ( ψ(x + e 2 x 2 x, t) + ψ(x e x, t) + ψ(x + e j x, t) + ψ(x e j x, t) ). Ths approxmaton s of postve type f and only f a s dagonal domnant,.e. a λ (t, x) j a λ j(t, x) n Ω T, λ Λ, =, 2,..., d. (4.6) [3]. One can prove now, that ths scheme s monotone n the statonary case. For the proof we refer to Approxmaton of Bonnans and Zdan We assume a fnte stencl S and a set of postve coeffcents āη : η S} R + such that a λ (t, x) = η S ā λ η(t, x)η η n Ω T, λ Λ, (4.7) whch also ensures the approxmaton to be of postve type. The approxmaton of Bonnans and Zdan s then gven by L λ h ψ = η S ā λ η η ψ + d = [µ +,λ δ + µ,λ δ ]ψ, (4.8) 29

37 4. Fnte Dfference Methods where η s an approxmaton of Tr[ηη D 2 ] η ψ(x, t) = ψ(x + η x, t) 2ψ(x, t) + ψ(x η x, t) η 2 x 2 Ths approxmaton s of postve type per defnton, and so monotone n the statonary case. For both approxmatons there s a constant C > such that for every ψ C 4 (R d ) and (t, x) G + h L λ ψ L λ h ψ C( µλ D 2 ψ x + a λ D 4 ψ x 2 ) 4.2. Fully Dscrete Scheme For θ [, ], we set the fully dscrete scheme then as u(t, x) = u(t t, x) ( θ) t sup{ L λ h u cλ u f λ }(t t, x) λ θ t sup{ L λ h u cλ u f λ }(t, x) n G + h λ (4.9) Under assumpton (4.4) ths ths fully dscrete scheme s monotone f also the followng CFL condton holds Assumpton t( θ)( c λ (t, x) + η Ch λ (t, xη)), tθ(c λ (t, x) + η Ch λ (t, xη)), 4.3. Fast Algorthm for 2 Dmensons In [2] Bonnans proposed an algorthm for computng monotone dscretsatons of two-dmensonal HJB problems wth fnte stencls, wth a consstency error dependng on the stencl wdth. Ths s acheved by approxmatng the dffuson coeffcent a by another coeffcent ã for whch a monotone 3

38 4. Fnte Dfference Methods u sol u num wth N = and θ = u num wth N = 2 and θ = u num wth N = 3 and θ = u num wth N = and θ =.5 u num wth N = 2 and θ =.5 u num wth N = 3 and θ =.5 u num wth N = and θ = u num wth N = 2 and θ = u num wth N = 3 and θ = x Fgure 4..: Numercal solutons and the exact soluton u(t, x) = mn( t, x ), plotted for t = dscretsaton s avalable on a user-specfed stencl. To compute the coeffcents, one needs to solve a lnear programmng problem at each pont of the grd. They also show that ths can be solved n O(p), f p s the stencl sze. The method uses the Stern-Brocot tree and on the related fllng of the set of postve semdefnte matrces. Convergence s then acheved by ncreasng the stencl sze along wth mesh refnement Applcaton Example Here we assumed the HJB n Example (2..4) and appled the Kushner-Dupus Scheme on t, see Fgure (4.). The plot s for t =. N stands for the number of grd ponts,.e. x, x 2,..., x N. 3

39 4. Fnte Dfference Methods 2 y 2 2 x 2 (2 π) sn(x) sn(y) y 2 2 x 2 Fgure 4.2.: Numercal solutons and the exact soluton u(t, x, x 2 ) = (2 t) sn(x ) sn(x 2 ), plotted for t = π Example If we consder the HJB equaton wth f λ (t, x) = sn(x ) sn(x 2 )[( + 2β 2 )(2 t) ] 2(2 t) cos(x ) cos(x 2 ) sn(x + x 2 ) cos(x + x 2 ), c λ (t, x) = µ λ (t, x) =, σ λ (t, x) = ( sn(x + x 2 ) 2 β sn(x + x 2 ) β soluton of ths s ) wth β 2 =.. In ths Case the HJB equaton s lnear and the u(t, x) = (2 t) sn(x ) sn(x 2 ). (4.) See Fgure (4.2), where I appled the Kushner-Dupus Scheme for stepsze π. 32

40 5. Sem-Lagrangan Schemes 5... Idea The results n ths Chapter are based on the work of K. Debrabant and E. R. Jakobsen [8]. Sem- Lagrangan schemes are a type dfference-nterpolaton schemes and arse as tme-dscretzatons of the followng sem-dscrete equaton { } t u nf L λ k [I xu](t, x) + f λ (t, x), λ Λ where I s a monotone nterpolaton operator of the grd and L λ k s a monotone dfference approxmaton of the operator L λ. One advantage of these methods s the guaranteed monotoncty of the dscretsatons, wth consstency acheved for the step-sze h. But to acheve ths the stencl sze wdth contnually ncreases as the mesh s refned. Here we treat HJB equatons especally posed on the entre space R d, rather than on bounded domans. Usng a boundary, the formula used needs to be modfed for ponts close to the boundary, for example by a one-sded asymmetrc formula Problem Statement Here we consder the followng fully non-lnear dffuson equatons { t u nf L λ u + c λ u + f λ} = n Ω T = (, T ) R d λ Λ (5.) u(, x) = u (x) n R d 33

41 5. Sem-Lagrangan Schemes wth L λ [u](t, x) = Tr[a λ (t, x)d 2 u(t, x)] + µ λ (t, x) u(t, x), (5.2) where the functons are defned lke n Chapter. By settng c λ =, we get the HJB equaton n Chapter Well-Posedness We wll use the followng assumptons on the ntal value problem (5.) Assumpton 5... For any λ Λ, a λ = 2 σλ σ λ for some d p matrx σ λ. There s a constant K ndependent of λ such that u + σ λ + µ λ + c λ + f λ K Ths assumpton ensures that we get a well-posedness bounded Lpschtz contnuous (resp. to the value x Ω) value functon, whch satsfes the comparson prncple. Proposton If assumpton (5..) holds. Then there exsts a unque soluton u of the ntal value problem (5.) and a constant C only dependng on T and K from the assumpton such that we have u C. Furthermore, f u and u 2 are sub- and supersolutons of (5.) satsfng u (, ) u 2 (, ), then t holds u u 2. The norms are defned n the last Chapter. 5.. Defnton of SL-Schemes Let G t, x be a not necessarly structured famly of grds wth G = G t, x = {(t n, x )} n N, N = {t n } n N X x, 34

42 5. Sem-Lagrangan Schemes for t, x >. Here = t < t <... < t n < t n+ <... < T satsfy max n t n t, where t n = t n t n and X x = {x } N s the set of vertces of nodes for a non-degenerate polyhedral subdvson T x = {Tj x } j N of R d. For some ρ (, ) the polyhedrons T j = Tj x satsfy nt(t j T ) j =, j N T j = R d, ρ x sup{damb Tj } sup{damt j } x, j N j N where dam( ) s the dameter of the set and B Tj the greatest ball contaned n T j. Let s say the matrx σ = (σ, σ 2,..., σ P ) wth σ m s the m-th column of σ, ψ s a smooth functon and k >. In the second equaton we replace ψ by ts nterpolant Iψ on the grd G. We get the approxmaton 2 Tr[σσ D 2 ψ(x)] = P m= P m= ψ(x + kσ m ) 2ψ(x) + ψ(x kσ m ) 2 k 2 + O(k 2 ) (Iψ)(x + kσ m ) 2(Iψ)(x) + (Iψ)(x kσ m ) 2 k 2 µ ψ(x) = ψ(x + k 2 µ) 2ψ(x) + ψ(x + k 2 µ) 2 k 2 + O(k 2 ) (Iψ)(x + kµ) 2(Iψ)(x) + (Iψ)(x kµ) 2 k 2 These approxmatons are postve monotone. If we also consder the nterpolaton to be monotone, we get a fully monotone dscretzaton. The general fnte dfference operator has the form L λ k [ψ](t, x) = M = 2 where for smooth functons ψ holds ψ(t, x + y λ,+ k, (t, x)) 2ψ(t, x) + ψ(t, x + yλ, k, (t, x)) k 2, (5.3) L λ k [ψ] Lλ [ψ] C( Dψ D 4 ψ )k 2 (5.4) Then we get the fnal scheme by 35

43 5. Sem-Lagrangan Schemes δ tn U n { = nf L λ, λ Λ k [IŪ.θ,n ] n +θ U = h(x ) n X x, + c λ,n +θ Ū θ,n } + f λ,n +θ n G (5.5) where θ [, ] and U n δ t ψ(t, x) := ψ(t,x) ψ(t t,x) t = U(t n, x ), f λ,n +θ = f λ (t n + tθt n, x ) wth (t n, x ) G, and ψ. θ,n := ( θ)ψ. n + θψ. n Example 5... For θ = we get the Explct Euler. U(t n, x ) U(t n, x ) t = nf λ { } L λ k [IU(t n, )](t n, x ) + c λ (t n, x )U(t n, x ) + f λ (t n, x ) If we choose θ = we get mplct Euler and for θ = 2 we get the mdpont rule Collocaton Method We can also nterpret the scheme (5.5) as a collocaton method for a dervatve free equaton. If W x (Q T ) = {v : v s a functon on Q T satsfyng v = T v n Q T } denotes the nterpolant space, equaton (5.5) can be stated n a equvalent way: Fnd U W x (Q T ) solvng δ tn U n { = nf L λ k [Ū θ,n ] n +θ λ Λ + c λ,n +θ Ū θ,n } + f λ,n +θ n G (5.6) 5.2. Analyss In ths secton, we gve assumptons under whch the SL scheme (5.5) s monotone and consstent, and we also present L -stablty, exstence, unqueness, and convergence results for these schemes. 36

44 5. Sem-Lagrangan Schemes Assumpton For the operator L λ k we wll assume that M = [y λ,+ k, M = M = y λ,+ k, M = [ 3 j=y λ,+ k, [ 4 j=y λ,+ k, [y λ,+ k, + y λ, k, + y λ, k, ] = 2k2 µ λ + O(k 4 ), y λ, k, ] = k2 σ λ σ λ + O(k 4 ), + 3 j=y λ, k, ] = O(k4 ), + 4 j=y λ, k, ] = O(k4 ), (5.7) where stands for the Outer-product. And for the nterpolaton operator I we wll assume that there are K, r N such that (Iψ) ψ K D p ψ x p (5.8) for all p r and smooth functons ψ. Further we wll assume that there s a non-negatve bass of functons {w j (x)} j such that (Iψ)(x) = j ψ(x j )w j (x), w (x j ) = δ j, and w j (x) for all, j N, (5.9) where δ j stands for the Kronecker Delta. Lemma Assume that all three assumptons n (5.2.) hold, Then we get The consstency error of the scheme (5.5) s bounded by 2θ ( ψ ll t + C t 2 ( ψ ll + ψ lll + ψ ll + D 2 ψ ll ) 2 + D r x r ψ k 2 + ( ψ D 4 ψ )k 2) The scheme (5.5) s monotone f the followng CFL condton holds [ M ] ( θ) t k 2 cλ,n +θ and θ tc λ,n +θ for all λ, n and. (5.) 37

45 5. Sem-Lagrangan Schemes Theorem If we assume that the Assumptons (5..) and (5.2.) and (5.) holds, then There exsts a unque bounded soluton U h of (5.5). If 2θ t sup λ c λ,+ : U n e 2 sup λ, cλ,+ t n [ h + t n sup f λ ], λ then the soluton U h of (5.5) s L stable U h converges unformly to the soluton u of (5.) for t, k, xr k 2. Remark When solvng PDEs on bounded domans, the SL schemes may exceed the doman and therefore needs to be modfed. For Drchlet condtons, the scheme must be modfed or the boundary condtons must be extrapolated. For Homogenous Neumann condtons, the scheme can be mplemented exactly by extendng n the normal drecton. If the boundary has no regular ponts, no boundary condton may be mposed Specfc SL Schemes The approxmaton of Falcone, µ λ Dψ Iψ(x + hµλ ) Iψ(x) h correspondng to our L λ k for yλ,± k = k 2 µ λ and k = h. The approxmaton of Crandall-Lons, 2 Tr[σλ σ λ D 2 ψ] p Iψ(x + kσj λ) 2Iψ(x) + Iψ(x kσλ j ) j= 2k 2 38

46 5. Sem-Lagrangan Schemes correspondng to our L λ k for yλ,± k = ±kσ λ J and M = p. If we combne the frst two approxmatons, we get 2 Tr[σλ σ λ D 2 ψ] + µ λ Dψ p Iψ(x + kσj λ) 2Iψ(x) + Iψ(x kσλ j ) 2k 2 + Iψ(x + hµλ ) Iψ(x) (5.) k 2 j= The approxmaton of Camll-Falcone, 2 Tr[σλ σ λ D 2 ψ] + µ λ ψ p Iψ(x + hσj λ + h p µλ ) 2Iψ(x) + Iψ(x hσj λ + h p µλ ) j= 2h And last but not least a modfed more effcent verson of the last one, p 2 Tr[σλ σ λ D 2 ψ] + µ λ Iψ(x + kσj λ Dψ ) 2Iψ(x) + Iψ(x kσλ j ) 2k 2 j= 5.4. Lnear Interpolaton SL Schemes + Iψ(x + kσλ p + k 2 µ λ ) 2Iψ(x) + Iψ(x kσ λ p + k 2 µ λ ) 2k 2 A natural choce to keep our scheme (5.5) monotone, s to use lnear or mult-lnear nterpolaton. In ths case we call the scheme (5.5) the LISL scheme. If we apply the results of Secton 6.3. to ths specal case we get Corollary Let s assume that Assumptons (5..) and (5.2.) hold, then The LISL scheme s monotone f the CFL condtons (5.) hold. The consstency error of the LISL scheme s O( 2θ t + t 2 + k 2 + x2 ), and hence t s frst k 2 order accurate when k = O( x 2 ) and t = O( x) for θ 2 or t = O( x 2 ) for θ = 2. If 2θ t sup c λ,+ and (5.) hold, then there exsts a unque bounded and L -stable λ solutuon U h of the LISL scheme convergng unformly to the soluton u of (5.) as t, k, x k. 39

47 5. Sem-Lagrangan Schemes So ths scheme s at most frst order accurate, has wde and ncreasng stencl and a good CFL condton Stochastc Control Let s assume that B s a sngleton such that the equaton smplfes to the HJB equaton for a optmal stochastc control problem. Then we can apply the dynamc programmng prncple. We wll assume that Assumpton (5..) holds and that c λ (t, x) = and all coeffcents are ndependent of tme t. Then we know that the vscosty soluton u of (5.) s [ T u(t t, x) = mn E λ( ) Λ t ] f λs (X s )ds + g(x T ) (5.2) constraned to the SDE X t = x and dx s = σ λs (X s )dw s + µ λs ds for s > t (5.3) Now we wll wrte a SL scheme lke n the collocaton form (5.6). Let {t =, t,..., t M = T } be the dscrete tme steps and consder the dscretzaton of (5.2), A M Λ s an approprate subset of pecewse constant controls and k n = (p + ) t n. ũ(t t m, x) = mn λ A M E [ M k=m f λ k ( X k ) t k+ + g( X ] M ) (5.4) X m = x, X n = X n + σ λn ( X n )k n ξ n + µ λn ( X n )k 2 nη n, n > m, where ξ = (ξ n,,..., ξ n,p ) and η n are ndependent sequences of dentcally dstrbuted random varables wth Pr ( ) (ξ n,,..., ξ n,p, η n ) = ±e j = 2(p + ) Pr ( ) (ξ n,,..., ξ n,p, η n ) = e p+ = (p + ), for j {, 2,..., p} 4

48 5. Sem-Lagrangan Schemes Now we get [ ] ũ(t t m, x) = mn E t m+ f λ (x) + t m+ L λ k λ A m+ [ũ](t t m+, x) + ũ(t t m+, x), (5.5) M where we used L λ k lke n Chapter 5.4 the thrd approxmaton Applcaton We appled the above scheme wth an equdstant step sze t, where we got the followng scheme then for k = (p + ) t, = t, t,..., T and the pecwse constant control space A M [ ] ũ(t j, x) = mn tf λ (x) + tl λ k [ũ](t j, x) + ũ(t j, x) λ A M Example Here we appled the scheme to the HJB equaton t u nf λ [λ xxu + ( x 2 )] =, u = on [, ) {, } {} (, ), wth exact soluton u(t, x) = t( x 2 ). In Fgure (5.2) we see the soluton wth stepsze t = 4 and x = and n Fgure (5.) wth stepsze t =.2 and x =. Example Here we appled the scheme for x R 2 wth the dstance functon as a soluton u(t, x, x 2 ) = mn(t, x, x 2 ) to the HJB equaton, where the dffuson s zero, µ λ = (λ, λ) and the cost functon s gven by f λ =, Λ = {, } and Ω T = (, ) (, ) 2. We can see the numercal result at tme T = for t = x = 25 n Fgure (5.3). Example Here we appled the scheme for (t, x, x 2 ) Ω T = (, ) (, ) 2 wth the soluton u(t, x, x 2 ) = exp(t) exp( x 2 ) exp( x 2 2 ) to the HJB equaton, where the drft and the cost functon are ( 4 2 ) both zero and the dffuson s gven by σ λ =. We can see the numercal result at the 4 2 termnal tme T = for t = x = 4 n Fgure (5.4) and for t = x = 2 n Fgure (5.5). 4

49 5. Sem-Lagrangan Schemes.5.5 x t.6.8 t ( x 2 ).5.5 x t.6.8 Fgure 5..: top: numercal soluton wth t = 4 and x =, botton: exact soluton.5.5 x t.6.8 t ( x 2 ).5.5 x t.6.8 Fgure 5.2.: top: numercal soluton wth t =.2 and x =, botton: exact soluton 42