DETERMINISTIC APPROXIMATION FOR STOCHASTIC CONTROL PROBLEMS

Transcription

1 DETERMINISTIC APPROXIMATION FOR STOCHASTIC CONTROL PROBLEMS R.Sh.Lipter*, W.J.Ruggaldier**, M.Takar*** *Departmet of Electrical Egieerig-Sytem Tel Aviv Uiverity Ramat Aviv, Tel Aviv, ISRAEL **Dipartimeto di Matematica Pura ed Applicata, Uiveritá di Padova, Via Belzoi Padova, ITALY ***Departmet of Applied Mathematic ad Statitic State Uiverity of New York at Stoy Brook Stoy Brook,N.Y ,USA Abtract We coider a cla of tochatic cotrol problem where ucertaity i due to drivig oie of geeral ature a well a to rapidly fluctuatig procee affectig the drift. We how that, whe the oie iteity i mall ad the fluctuatio become fat, the tochatic problem ca be approximated by a determiitic oe. We alo how that the optimal cotrol of the determiitic problem i aymptotically optimal for the tochatic problem. Key Word : Stochatic ad determiitic cotrol, Stochatic differetial equatio, Weak covergece, Aymptotic optimality. AMS Subject Claificatio: 93E2, 93C15, 6B1, 6F17, 6G44, 49J15, 49K4, 49M45 Ackowledgemet : Thi work wa partially upported by GNAFA / CNR of the Italia Natioal Reearch Coucil allowig a viit of the firt ad lat author at the Uiverity of Padova. The work of the lat author wa alo upported by Natioal Sciece Foudatio Grat DMS 9312 ad NATO Scietific Exchage Grat CRG

2 1.Itroductio There are oly few tochatic cotrol problem that ca be olved i cloed form. A lot of effort ha therefore bee put ito developig approximatio techique for uch problem. Oe approach i thi directio i to coider, itead of the origial model, a model where the uderlyig procee are replaced by impler oe. Thi approach make it poible to cotruct early optimal cotrol for the origial model, baed o the olutio to the impler model. Thi impler model may ivolve uderlyig procee that are diffuio ( diffuio approximatio ), but it may alo imply be a determiitic model ( fluid approximatio ). A geeral tool, epecially for diffuio approximatio, are techique of weak covergece of radom procee ([1],[3],[6],[15]) combied with a averagig priciple ([5]). Thi methodology i actively ued i variou practical problem of egieerig, maufacturig, queueig, ivetory ad other ad i tudied e.g., i [9],[1],[11],[12],[7],[8],[13]. The uderlyig idea of thi methodology i actually rather imple, but the mathematic required for it implemetatio i i geeral quite ophiticated. Although there exit ome geeral approache (ee e.g. [9]), i each particular cae the rigorou verificatio of the covergece of the cotrolled ytem require pecific techical tool ad idea. I the preet paper we apply fluid approximatio techique to a rather geeral tochatic cotrol model with covex cotrol cot fuctio. I thi model the cotrolled proce X i decribed by a tochatic differetial equatio with repect to a geeral (ot ecearily cotiuou) martigale M. The cotrol affect the drift of X; thi drift i furthermore affected by a rapidly fluctuatig exogeou proce ξ. To implemet the approximatio approach, we embed the give model ito a family of imilar model, parametrized by a mall parameter >. We coider the cae whe the iteity of the radom oie diturbace M become mall with while the cotamiatig proce ξ fluctuate with icreaig peed. For uch a cae the limitig model become determiitic ad it i poible to obtai aymptotically (a ) optimal cotrol for the prelimit model by uig the optimal cotrol of the limitig determiitic ytem. Although we coider explicitly oly the cae whe the cotrolled tate proce X ca be completely oberved, everthele our reult hold i the ame form whe the tate i oly partially oberved. I a more formal way, we have a family of cotrolled tochatic ytem, parametrized by a mall (poitive) parameter, ( ), with dyamic dx t = [ a(x t, ξ t/ ) + b(x t )u (t) ] dt + dm t (1.1) ad iitial coditio X. Here X = (Xt ) i the cotrolled tate (or igal) proce, ξ = (ξ t ) i the cotamiatio proce affectig the drift of X, while M = (Mt ) i a proce repreetig the oie i the ytem. The radom fuctio u = (u (t)) i the cotrol that affect the drift of X i a liear way ad atifie the uual requiremet for admiibility (ee Defiitio 2.1 below). Give a fiite horizo T >, with each cotrol u we aociate the cot { } T J (u ) = E [p(xt ) + q(u (t))]dt + r(xt ), (1.2) 2

3 where p(x), q(u) ad r(x) are oegative fuctio o the real lie referred to a holdig cot, cotrol cot, ad termial cot fuctio repectively. The objective i to fid V = if u J (u ). (1.3) ad a optimal (miimizig) cotrol. For practical purpoe oe may jut a well be itereted i fidig a early optimal cotrol or, a will be the cae here, a aymptotically (a ) optimal cotrol. To decribe the limitig cotrol model, we aume that the followig ergodic propertie hold : P lim X = x, x R (1.4) 1 a(x) = P lim t t P lim up a(x, ξ )d ; x R (1.5) M t =. (1.6) I the ext ectio we formulate coditio uder which (1.4)-(1.6) are valid. The dyamic of the limitig ytem i give by the followig ordiary differetial equatio dx(t) = [a(x(t)) + b(x(t))u(t)] dt ; x() = x. (1.7) Here x(t) i a (determiitic) cotrolled proce ad u(t) i a (determiitic) cotrol. Defie ad j(u) := [p(x(t)) + q(u(t))] dt + r(x(t )) (1.8) v := if u j(u), (1.9) where the ifimum i take over all (determiitic) meaurable fuctio o [, T ]. Our mai reult are the followig two theorem. Theorem 1.1. The followig relatio hold lim V = v. Theorem 1.2. Let u (t), t T, be a optimal determiitic cotrol for (1.7)-(1.9). The u (t) i aymptotically optimal for (1.1)-(1.3) i the ee that lim J (u ) V =. Remark 1. If for the limit model there exit a feedback cotrol u (t) = u (t, x (t)), where x (t) i the cotrolled proce defied by the differetial equatio (1.7) with u(t) = u (t), ad the fuctio u (t, x) i Lipchitz cotiuou i x uiformly i t [, T ], 3

4 the the tatemet of Theorem 1.2 remai true with u (t, X t ) replacig u (t), i.e. the feedback cotrol u (t, X t ) i aymptotically optimal. Remark 2. The reult obtaied here for the oe dimeioal cotrol problem ca be exteded to a -dimeioal problem. The motivatio to coider jut the calar cae i to preet the mai idea i the implet form. The mai cotributio of thi paper i twofold : from a more theoretical poit of view we obtai a tability reult for the optimal cotrol of a determiitic ytem i the ee that thi cotrol i aymptotically optimal for a large cla of tochatic cotrol problem of a rather complicated ature. From a practical poit of view our reult allow oe to compute a aymptotically optimal cotrol for a variety of problem uder quite geeral coditio, where a direct approach would be impoible. The proof coit of two part carried out i Sectio 3 ad 4 : firt we how that v i a aymptotically lower boud for the optimal cot fuctio V. The we how that the determiitic optimal cotrol of the limitig problem ca be applied to the prelimit model, yieldig aymptotically optimal cot. Reult of more techical ature, iteretig i their ow right, are moved to appedice (Sectio 5,6, ad 7). 2. Mai aumptio ad otatio For implicity we aume (, 1]. For each let SB := (Ω, F, F = (F t ) t, P ) be a fixed tochatic bai, where (Ω, F, P ) i a complete probability pace ad F i a filtratio atifyig the uual aumptio (ee [2]). The iitial value X of the tate proce i F - meaurable, while (ξ t ), (M t ) are F -adapted. Defiitio 2.1. The cotrol proce u = (u (t)) t i aid to be admiible if it i F -adapted ad u (t) dt <, P a.. (2.1) Throughout the paper we make the followig aumptio : (A.1) The cotrol cot fuctio q(u) i oegative covex atifyig q(u) c u 1+γ ; c, γ >. (A.2) The cot fuctio p(x) ad r(x) are cotiuou oegative atifyig p(x), r(x) c 1 (1 + x γ 1 ), ; c 1, γ 1 >. (A.3) There exit x R ad poitive cotat c 2, γ 2 uch that (i) P lim X = x (ii) E X 2 < c 2, where i the mallet iteger uch that γ 1 <. (A.4) The fuctio a(x, y) i meaurable i (x, y) ad atifie the liear growth ad Lipchitz coditio i x (uiformly i y), i.e., there exit l > uch that (i) a(x, y) l(1 + x ) ; x, y R (ii) a(x, y) a(x, y) l x x ; x, x, y R. (A.5) The fuctio b(x) i bouded ad Lipchitz, i.e., (i) b(x) l (ii) b(x ) b(x ) l x x ; x, x R. 4

5 (A.6) The radom proce ξ = (ξ t ) t i ergodic, amely there exit a probability meaure λ(dy) o R uch that for ay bouded ad meaurable fuctio g(y) 1 t P lim g(ξ ) d = g(y) λ(dy). t t (A.7) The proce M = (Mt ) t i a quare itegrable martigale with path i the Skorokhod pace D[, ) whoe predictable quadratic variatio M t atifie (i) M t = m d with bouded deity m. The latter mea that there exit a cotat c 3 uch that (ii) m t c 3 ; t T P a.. The jump M := M lim v Mv are bouded, i.e., there exit a cotat L > uch that (iii) Mt L ; t T, (, 1]. Notice that by aumptio (A.4) ad (A.5) equatio (1.1) ha a uique trog olutio X for every admiible cotrol u. We hall refer to X a the tate proce aociated with u. The oly requiremet for the cotamiatio proce ξ i it ergodicity; o tatioarity of ξ or idepedece from other procee i required. We furthermore remark that our reult remai valid if M i ay proce with path i D atifyig i) up Mt P, T > (ee derivatio below (4.5) ad (6.3)), ii) up E up Mt 2 <, 1 (ee Sectio 7). I thi more geeral cae, a rigorou repreetatio of the dyamic of the ytem hould be made i the itegral form below rather tha i the differetial form (1.1) X t = X + [ a(x, ξ / ) + b(x )u () ] d + M t. Fially otice that our aumptio o the cot fuctio are quite atural ad repreet a miimal et of aumptio for the problem to be meaigful : (A.1) guaratee that we tay withi the claical cotrol problem rather tha havig alo to deal with igular cotrol (e.g., ee [14]), while (A.2) i the uual polyomial growth coditio aumptio. 3. Aymptotic lower boud for the optimal cot fuctio Let v ad V be the optimal cot fuctio, correpodig to the determiitic ad the origial cotrol problem repectively (ee (1.7)-(1.9) ad (1.1)-(1.3)). The aim of thi ectio i to prove the followig theorem. Theorem 3.1. Let the aumptio of Sectio 2 be atified, the lim if V v Proof: We may limit ourelve to the cae whe lim if J (u ) = β <. Take a ubequece k, (k ) uch that lim k J k (u k ) = lim if J (u ). The for k large eough J k (u k ) 2β (3.1) 5 R

6 (for otatioal coveiece we hall aume that (3.1) hold for all k). From (3.1) ad (1.2) it follow that E q(u k t ) dt < 2β. (3.2) Let X k be the tate proce aociated with u k. Give (3.2), we may apply Theorem 6.1 to coclude that the equece (X k, U k, U k ), k 1 i relatively compact, where U k t = u k ()d ad U k t = u k () d. Let (X k, U k, U k ) be a weakly covergig ubequece with limit (X, U, U ). The, by Theorem 6.1, we have X t = x + U(t) = [a(x ) + b(x )u()] d, u()d, (3.3) where x beig the limit of X (ee aumptio (A.3)), a(x) i defied i (1.5), ad b(x) i the ame a i (1.1). Sice lim if J (u ) = lim J k(u k), (3.4) k where ( k) i ay ubequece of ( k ), we ue (3.4) with ( k) correpodig to the weakly covergig equece (X k, U k, U k ). The by Theorem 5.1 ad 6.1 we get { } J k(u T k) E [p(x t ) + q(u(t))] dt + r(x T ). (3.5) lim k From (3.4) ad (3.5) we derive lim if J v. (3.6) If a optimal cotrol exit, the the tatemet of the theorem i a coequece of (3.6). Otherwie we approximate the optimal value fuctio by the cot aociated with δ-optimal cotrol. 4. Proof of Theorem 1.1 ad 1.2 It follow from Theorem 3.1 that the lower limit of the optimal cot i bouded from below by the optimal cot correpodig to the determiitic model (1.7)-(1.9). The exitece of a optimal cotrol u for problem (1.7)-(1.9) ca be how by tadard argumet (ee the Remark at the ed of Sectio 6 or the proof of Theorem III.4.1 i [4]). Notice alo that aumptio (A.1) implie u (t) 1+γ dt <. (4.1) 6

7 Next let x (t) be the (determiitic) olutio of (1.7) correpodig to the cotrol u (t) ad let X, = (X, t ) be the (tochatic) tate proce, aociated with the cotrol u t u (t) via (1.1). We firt how that P lim Let up Uig (1.1) ad (1.7), we get the iequality t X x + + up [a(x, [ a(x, X, t x (t) =. (4.2) t := X, t x (t). (4.3), ξ / ) a(x, ) a(x ()) + b(x, ) b(x ()) u () ] d )] d + up Mt. By the Lipchitziaity of a(x) ad b(x) (ee aumptio (A.4) ad (A.5)) it follow that { } t X x + up [a(x,,ξ / ) a(x, )] d + up Mt Therefore, by the Growall-Bellma iequality { up t X x + up } + up Mt + l [a(x, exp (1 + u () ) d., ξ / ) a(x, ( l )] d+ [1 + u () ] d ). (4.4) Now, by aumptio (A.2) we have P lim X x = ; furthermore, uig a imilar argumet a i the proof of (6.8) below, we get P lim up [a(x,, ξ / ) a(x, )] d =. (4.5) Fially, by aumptio (A.7) ad by Problem i [15] P lim up M t =. Thu, (4.2) hold. A a coequece of (4.2) we have P lim p(x, t ) = p(x (t)), t [, T ], P lim r(x, T ) = r(x (T )). (4.6) Next we eed to prove that the familie p(x, t ) of fuctio o [, T ] Ω ad of radom variable r(x, T ) are uiformly itegrable with repect to the meaure dt dp ad dp 7

8 o [, T ] Ω ad Ω repectively. To thi ed it i ufficiet to how that there exit a cotat c > uch that E [p(x, t )] 2 c, E [r(x, T )]2 c. (4.7) By aumptio (A.2) we have p(x), r(x) c 1 (1 + x γ 1 ). Let be the mallet iteger uch that γ 1 <. Evidetly, (4.7) hold if there exit a cotat c uch that Uig (1.1) a well a aumptio (A.4) ad (A.5), we get up t X, X + l E up X, t 2 c. (4.8) ( 1 + up τ The Growall-Bellma iequality implie From (7.1) we have Xτ, ) d + l up X, e { X lt + lt + l T u (t) dt + up Mt. T u (t) dt + up Mt T }. (4.9) E up Mt 2 cot. (4.1) Iequality (4.8) i therefore a coequece of (4.9),(4.1) ad Aumptio (A.3) By virtue of (4.8) ad Theorem 5.4 i [1] { } T lim J (u ) = lim E [p(x, t ) + q(u (t))] dt + r(x, T ) = [p(x (t)) + q(u (t))] dt + r(x (T )) = v. Sice V J (u ) we have lim up V v. Thi iequality ad Theorem 3.1 imply Theorem 1.1 ad Theorem Relative compactee of (U, U ) 1. Let q(u) be the cotrol cot fuctio from (1.2). Aume up E 1 q(u (t))dt <. (5.1) Recall that U (t) = u ()d ad deote it total variatio i the time iterval [, t] by U t = u () d. (5.2) The proce U t, t T ha path i a ubet of C [,T ] of cotiuou icreaig fuctio C + [,T ]. Alo, ρ will be ued for deigatig of the uiform metric i C [,T ]. 8

9 Theorem 5.1. Let aumptio (A.1) ad (5.1) be atified. The the family of radom procee (U, U ) = (U (t), U t ), 1 i relatively compact i the metric pace ( C [,T ] C + [,T ], ρ ρ). If (U k, U k ) i ay weakly covergig equece with limit (U, U ), the there exit a meaurable proce (u(t)) uch that 1. E u(t) 1+γ dt < ; 2. for ay t T ad P -a.. 3. U(t) = lim if k E u()d, U t = q(u k (t)) E u() d; q(u(t))dt. (5.3) Proof : Sice C + [,T ] i cloed i C [,T ] i the metric ρ, by virtue of Prokhorov theorem (ee e.g. [1]) oly tighte of the family i C [,T ] C + [,T ] ha to be checked. Due to Theorem 8.2 ad 15.2 i [1], we verify two coditio: lim c lim δ lim up lim up P (up U t > c) = P ( up t, T : t δ U t U ) > ν = ν > (5.4) ad the ame coditio for U. Coditio (A.1) ad (5.1) imply up E 1 u (t) 1+γ dt <. (5.5) Thereby, coditio (5.4) are verified by Hölder iequality. Namely up U t = U T T γ/(1+γ)( 1/(1+γ) u (t) dt) (1+γ) (5.6) ad for ay radom t, T : t δ U t U δ γ/(1+γ) ( u (t) (1+γ) dt) 1/(1+γ). (5.7) We coclude by uig Chebyhev iequality. The validity of the coditio of the type (5.4) for U i proved aalogouly. Let W (t) be ay radom proce with path from C [,T ] ad let I = { i = it 2, i =, 1,..., 2 }, 1 be ubdiviio of the time iterval [, T ]. Put w (t) = W i W i 1 i i 1, i 1 t < i. (5.8) 9

10 It i kow (ee [16]) that uder the aumptio up E w (t) 2 dt < (5.9) the proce W (t) i abolutely cotiuou (with repect to Lebegue meaure Λ(dt) = dt), i.e. there exit a meaurable proce w(t) uch that for ay t T ad P -a.. ad, what i more, W (t) = w()d, E w(t) 2 dt < (5.1) w(t, ω) = lim w (t, ω), Λ P a.. (5.11) The ame proof how that uder the aumptio: for ome γ > up E w (t) 1+γ dt < (5.12) we have that (5.1) with E w(t) 1+γ dt < ad (5.11) hold. Let W (t) U(t) ad correpodigly u (t) w (t). Therefore, tatemet 1. ad 2. take place if for γ the ame a i (A.1) up E u (t) 1+γ dt <. (5.13) To thi ed, defiig u k (t) i the ame way a w (t), but with W (t) U k (t), we fid E (U k ) = u k (t) 1+γ dt = 2 i=1 i 2 i 1 u k (t)dt 2 1+γ 2 2. O the other had, due to Jee iequality ad aumptio (A.1) 2 i=1 i 2 i 1 u k (t)dt 2 1+γ c 2 i=1 By virtue of the weak covergece of U k i 2 u k (t) 1+γ dt = i 1 2 u k (t) 1+γ dt q(u k (t))dt. (5.14) ad aumptio (5.1), for ay N 1 we get E mi [ N, E (U) ] = lim k E mi [ N, E (U k ) ] up E 1 u (t) 1+γ dt <. By the mootoe covergece Theorem up E E (U) < ad o, oticig that E (U) = u (t) 1+γ dt, we coclude that (5.13) hold. 1

11 To prove tatemet 3. of the Theorem, itroduce E,q (U k ) = Sice by Jee iequality q(u k (t))dt = q(u k (t))dt = 2 i=1 q 2 i=1 ( i 2 i 1 u k (t)dt) q ( i 2 i 1 u k (t)dt) q(u k (t))dt we derive tatemet 3. by Fatou lemma ad by (5.11), reformulated for u (t): lim if k E q(u k (t))dt lim if = lim if = lim if E lim lim E mi [ N, E,q (U k ) ] N k lim E mi [ N, E,q (U) ] N E E,q (U) = lim if lim if E q(u (t))dt = E q(u (t))dt q(u(t))dt. 6. Relative compacte of ( X, U, U ) 1. Let X = (Xt ) t be defied a i (1.1) ad U t i (5.2). We coider the triple (X, U, U ) = (Xt, U (t) t, U ) t ) with value i D [,T ] C [,T ] C + [,T ], where D [,T ] i Skorokhod pace. Theorem 6.1. Let the aumptio of Sectio 2 ad (5.1) be atified. The the family (X, U, U ), 1 i relatively compact i the metric pace ( D [,T ] C [,T ] C + [,T ], ρ ρ ρ). If (X k, U k, U k ) i ay weakly covergig equece with limit (X, U, U ), the the tatemet of Theorem 5.1 hold ad X t = x + [ a(x ) + b(x )u() ] d, t T, (6.1) where a(x) i defied a i (1.5) ad u() i the proce from Theorem 5.1. For ay cotiuou oegative fuctio p(x) ad r(x), lim if k { E Proof : Parallel to X t } { p(x k t )dt + r(x T } k T ) E p(x t )dt + r(x T ). (6.2) X, t = X + itroduce a proce X, t defied by (compare to (1.1)) [ a(x,, ξ / ) + b(x, )u () ] d (6.3) 11

12 Due to (1.1), (6.3) ad aumptio (A.4), (A.5), the proce Yt atifie the iequality: Y t l Y d[ + U ] + up T M, ad o by the Growall-Bellma iequality we get YT l up M exp{l[t + U T ]}. T lim lim up P c up t T = up t X X, By virtue of aumptio (A.7) ad Problem i [15], up M t, i probability ad U T atifie (5.4). Coequetly YT, i probability ad by Theorem 4.1, Ch.1 i [1] the reult of the Theorem remai true if it tatemet will be proved oly for the triple (X,, U, U ). By virtue of (5.4), it i ufficiet to verify oly the followig two coditio (ee Theorem 8.2 ad 15.2 i [1]): ( ) X, t > c = lim lim up P δ ( up X, t t, T : t δ X, > ν It follow from (6.3) ad aumptio (A.4) ad (A.5) that for ay t T up t X, X + l [1 + up τ ad o, uig Growall-Bellma iequality, we get ) Xτ, d + U T up X, e lt ( X + l U ) T. T =, ν >. (6.4) Evidetly, the firt coditio i (6.4) hold by the proof Theorem 5.1 ad by aumptio (A.3.i). For ay t < δ we ca apply aumptio (A.4) ad (A.5) to write X, t X, l t [1 + up Xτ, ] dτ + l ( U t U ) t τ T lδ[1 + up Xτ, ] + l ( U t U ) t. τ T Therefore, the validity of the ecod coditio i (6.4) follow from the proof of Theorem 5.1 ad from the firt coditio i (6.4) which ha already bee proved. Let (X k,, U k, U k ), k 1 be a weakly covergig equece with limit (X, U, U ). Deote by Q the ditributio of the limit (X, U, U ), i.e. Q i a probability 12

13 meaure o C [,T ] C [,T ] C + [,T ]. For ay elemet (X, U, U ) from C [,T ] C [,T ] C + [,T ] put Φ t (X, U, U ) := X t x a(x ) d b(x ) du(), (6.5) where the fuctio a(x) i defied by (1.5) ad x i the ame a i aumptio (A.3.i). The ecod tatemet of the Theorem hold if up Φ t (X, U, U ) = Q a... (6.6) To prove the validity of (6.6), we how that the fuctioal up Φ t (X, U, U ) i cotiuou i the product-metric ρ 3 = ρ ρ ρ. Let (X, U, U ) ad (X, U, U ), 1 be elemet of C [,T ] C [,T ] C + [,T ] uch that lim ρ 3 ((X, U, U ), (X, U, U ) =. We how that lim up Φ t (X, U, U ) = up Φ t (X, U, U ). ito accout (6.5), we get L := up Φ t (X, U, U ) up Φ t (X, U, U ) up Φ t (X, U, U ) Φ t (X, U, U ) 2 up Xt Xt + + a(x ) a(x ) d b(x ) b(x ) d U + up b(x ) d[u () U ()]. Takig Uig the Lipchitziaity of the fuctio a(x) (it i iherited from a(x, y), ee (A.4.ii)) ad b(x), we obtai the followig upper boud for L : where L ρ(x, X ) { 2 + lt + l U T + lρ( U, U ) } + L b, L b := up b(x ) d[u () U ()]. The quatity L b ca be evaluated from above i the followig way (below [α] tad for the iteger part of α) L b up b(x [N] N ) d[u () U ()] + l up X X [ ( U T + U ) T ]. 1 N Therefore, lim up L 2l U T up 1 N X X, for N, i.e. 13

14 up Φ t (X, U, U ) i cotiuou fuctioal. Uig thi fact, the equality ( ) ( Q Φ t (X, U, U ) ν = lim P k up up ) Φ t (X k,, U k, U k ) ν, ν >, i implied by the weak covergece metioed above, ad the etimate up Φ t (X k,, U k, U k ) X k x + up [a(x k, ξ /k ) a(x k )]d, (6.7) we ca coclude that (6.6) hold, if the rigth had ide of (6.7) goe to zero i probability a k. Takig ito accout aumptio (A.3.i), for the validity of (6.6) oly P lim k up [a(x k, ξ /k ) a(x k )]d = (6.8) ha to be checked. Evidetly, for a piecewie cotat fuctio uch that φ(t) = φ ( ) i for i t < i+1 P lim [ a(φ(), ξ /k ) a(φ()) ] d =, t T (6.9) k hold. Notice alo that (6.9) remai true whe replacig a(x, z) ad a(x) by a ± (x, z) ad a ± (x), where e + = max[, e] ad e = mi[, e]. The, by the Problem i [15] we get P lim [ a(φ(), ξ /k ) a(φ()) ] d =, (6.1) up k Approximate ow the proce (X k t ) by a equece X k,m, = (X k,m, t ), 1, m 1, where X k,m, t = j= j 1 m I ( j 1 m X k i/ < j ), m i t < i + 1. The proce X k,m, ha piecewie cotat path ad o the et { up X k t c } the umber of it path i fiite ad doe ot deped o k. Therefore, uig (6.1), we ee that for ay c >, m 1, 1 ad puttig ξ k = ξ /k ( P lim k I up ) X k t c up [ a(x k,m,, ξ k ) a(x k,m, ) ] d =. (6.11) O the other had, takig ito accout the weak covergece of (X k t ) which implie lim k lim up c P ( up X k t > c ) =, for the validity of (6.8) it remai to how that P lim m, lim k a(x k, ξ k ) a(x k,m,, ξ k ) d =, 14

15 P lim lim m, k a(x k ) a(x k,m, ) d =. Takig ito accout the Lipchitziaity of the fuctio a(x, y) (ee aumptio (A.4)), which i alo iherited by the fuctio a(x), it i ufficiet to how P lim m, lim k X k X k,m, d =. (6.12) To thi ed put X k, t Obviouly X k, X k = X k X k,m, [t] X k,m,, where [α] i the iteger part of α. The X k X k, 1 m. Coequetly + X k, X k,m,. X k X k,m, d T m + X k X k [] d T m + T up X k t X k., : t 1/ Therefore, for ay ν > ( P X k X k,m, d > ν ) P ( up X k t X k > ν, : t 1/ T 1 m (6.13) A a reult, (6.12) follow from weak covergece of (X k t ) which implie the covergece to zero of the right had ide of (6.13). It remai to prove (6.2). Due to the weak covergece of (X k t ), k 1, we fid { } { lim if E p(x k t )dt + r(x T k T ) ( lim E N p(x k t ) ) } dt + N r(x k T ) k k { ( = E N p(xt ) ) dt + N r(x T )} N 1 ad coclude by uig the mootoe covergece theorem. 2. Remark The method of proof of Theorem 6.1 ca be adapted to the followig determiitic problem. Let u (t), 1, be a equece of meaurable fuctio atifyig up u (t) 1+γ dt <, γ >. For each coider the differetial equatio d x (t) dt with the iitial coditio x () = x. Put U (t) = = a(x (t)) + b(x )u (t) u () d, U t = 15 u () d, ).

16 By the ame techique a i the proof of Theorem 6.1, oe ca how that the family (x (t), U (t), U t ), 1 i uiformly bouded ad equicotiuou. The by the Arzelá-Acoli theorem thi family i relatively compact ad there exit a ubequece (x k (t), U k (t), U k t ) covergig uiformly to a limit (x (t), U (t), U t ), with abolutely cotiuou U (t), i.e. there exit a meaurable fuctio u (t) uch that U (t) = u ()d. Furthermore, x (t) i the uique olutio of the differetial equatio d x (t) dt with the iitial coditio x () = x. = a(x (t)) + b(x )u (t) 7. Upper boud for E up M t 2 I thi ectio we prove, uder aumptio (A.7), that for ay > 1 ad T > up 1 E up M t 2 <. (7.1) I the cae of E M T 2 <, we ca apply Doob iequality (ee e.g. [15]) to obtai E up Thu, it uffice to how that M t 2 ( ) 2 2 E MT up E MT 2 <. (7.2) 1 We hall ue the otatio k, N t, ad V t to deote a geeric poitive cotat depedig o (c 3, L, ), a local martigale, ad a o decreaig proce (with path i D [, ) ) repectively, where N t ad V t are adapted to the filtratio F (all thee object might be differet i differet formula). To check the validity of (7.2), we hall how that (Mt ) 2 admit the repreetatio (M t ) 2 = k [1 + (M ) 2 ]d + N t V t. (7.3) From (7.3) the deired reult follow immediately. I fact, by Ito formula we fid e kt (M t ) 2 = 1 e kt e k dn e k dv e k dn. (7.4) The Ito itegral e k dn i a local martigale. Deote it localizig equece of toppig time by (τ j ) j 1, i.e. for ay t >, E τ j e k dn =, j 1. Thereby, from (7.4) it follow E e k(t τ j) (M T τ j ) 2 1, j 1 16

17 ad o we coclude by uig Fatou Lemma. Thu, oly (7.3) ha to be proved. By Ito formula (M t ) 2 = 2 + t (M )2 1 dm + (2 1) (M )2 2 d M,c [ (M ) 2 (M )2 2(M )2 1 M ], (7.5) where M,c t i the predictable quadratic variatio of the cotiuou part of the martigale M t. The repreetatio (7.5) i othig but with the local martigale ad the o decreaig proce B t = (2 1) N t = 2 (M t ) 2 = N t + B t (7.6) (M )2 2 d M,c + t (M ) 2 1 dm, (7.7) [ (M ) 2 (M ) 2 2(M ) 2 1 M ]. Deote by µ (dt, dz) the meaure of jump of the martigale Mt compeator. Sice (R = R \ {}) [ (M ) 2 (M ) 2 2(M ) 2 1 M ] t = ad the proce N t = R R R [ (M + z) 2 (M ) 2 2(M ) 2 1 z ] µ (d, dz) [ (M + z) 2 (M ) 2 2(M ) 2 1 z ] µ (d, dz) [ (M + z) 2 (M ) 2 2(M ) 2 1 z ] ν (d, dz) (7.8) ad by ν (dt, dz) it i a local martigale too, we arrive to a ew decompoitio of the type (7.6) with local martigale N t = 2 + R (M ) 2 1 dm [ (M + z) 2 (M ) 2 2(M ) 2 1] [µ ν ](d, dz) (7.9) 17

18 ad o decreaig proce B t = (2 1) + R (M )2 2 d M,c [ (M + z) 2 (M ) 2 2(M ) 2 1 z ] ν (d, dz). (7.1) Uig the fact that M L, we get ν (d, dz) = I( z L)ν (d, dz). Therefore, by virtue of Taylor expaio for the fuctio f(x) = x 2 ad Hölder iequality oe ca fid a cotat k uch that db t (2 1)(Mt ) 2 2 d M,c t + k (M ) 2 2 (1 + z 2 )ν (dt, dz). z L Recall that the quadratic variatio [M, M ] t of M t i defied a [M, M ] t = M,c t + ( M ) 2 t = M,c t + z 2 ν (dt, dz). Coequetly, takig ito accout that x x 2 we obtai db t 2[(2 1) + k](mt ) 2 2 d[m, M ] t 2[(2 1) + k](1 + Mt ) 2 d[m, M ] t. Defie a o decreaig proce V t = 2[(2 1) + k] The, for (M t ) 2 we have the followig decompoitio: (M t ) 2 = N t + 2[(2 1) + k] R (1 + M ) 2 d[m, M ] B t. (1 + M ) 2 d[m, M ] V t, (7.11) where the local martigale N t i defied i (7.9). Sice [M, M ] t M t martigale we arrive to a ew repreetatio for (Mt ) 2 : i a local (M t ) 2 = N t + 2[(2 1) + k] (1 + M ) 2 d M V t (7.12) with the ame o decreaig proce V t ad a ew local martigale N t Due to aumptio (A.7) we have (for 1) d M t c 3dt, i.e. [ V t = 2[(2 1) + k] (1 + M ) 2 c 3 d i a o decreaig proce. Thu, (7.3) i implied by (7.12) ad (7.13). (1 + M ) 2 d M ] (7.13) 18

19 Referece [1] P.BILLINGSLEY,Covergece of Probability Meaure, Joh Wiley, New York, [2] C.DELLACHERIE, Capacité et Proceu Stochatique, Spriger, Berli, [3] S.N.ETHIER AND T.G.KURTZ, Markov Procee : Characterizatio ad Covergece, Joh Wiley, New York, [4] W.H.FLEMING AND R.W.RISHEL, Determiitic ad Stochatic Optimal Cotrol, Spriger, Berli, [5] M.I.FREIDLIN AND A.D.WENTZELL, Radom Perturbatio of Dyamical Sytem, Spriger, New York, [6] J.JACOD AND A.N.SHIRYAYEV, Limit Theorem for Stochatic Procee, Spriger, Berli-New York, [7] E.V.KRICHAGINA AND M.I.TAKSAR, Diffuio Approximatio for GI/G/1 Cotrolled Queue, QUESTA, 12 (1992), pp [8] E.V.KRICHAGINA, S.X.C.LOU,S.P.SETHI AND M.I.TAKSAR, Productio Cotrol i a Failure-Proe Maufacturig Sytem : Diffuio Approximatio ad Aymptotic Optimality, Aal Appl.Probab., 3 (1993), pp [9] H.J.KUSHNER, Approximatio ad Weak Covergece Method for Radom Procee, MIT-Pre, Cambridge, [1] H.J.KUSHNER AND W.J.RUNGGALDIER, Nearly Optimal State Feedback Cotrol for Stochatic Sytem With Widebad Noie Diturbace,SIAM J.Cotrol ad Optimiz., 25 (1987), pp [11] H.J.KUSHNER AND K.M.RAMACHANDRAN, Optimal ad Approximately Optimal Cotrol Policie for Queue i Heavy Traffic, SIAM J.Cotrol ad Optimiz., 27 (1989), pp [12] H.J.KUSHNER AND L.F.MARTINS, Routig ad Sigular Cotrol for Queueig Network i Heavy Traffic, SIAM J.Cotrol ad Optimiz., 28 (199), pp [13] J.LEHOCZKY, S.SETHI, M.SONER AND M.TAKSAR, A Aymptotic Aalyi of Hierarchical Cotrol of Maufacturig Sytem, Math. of Oper. Re., 16 (1991), pp [14] J.LEHOCZKY AND S.SHREVE, Abolutely Cotiuou ad Sigular Stochatic Cotrol, Stochatic, 17 (1986), pp [15] R.S.LIPTSER AND A.N.SHIRYAYEV, Theory of Martigale, Kluwer Academic Publ., Dordrecht, [16] A.D.WENTZELL, Additive Fuctioal of a Multidimeioal Wieer Proce, Soviet Mathematic 2 (1961), pp