Probabiliy, Geomery and Inegrable Sysems MSRI Publicaions Volume 55, 28 Homogenizaion of random Hamilon Jacobi Bellman Equaions S. R. SRINIVASA VARADHAN ABSTRACT. We consider nonlinear parabolic equaions of Hamilon Jacobi Bellman ype. The Lagrangian is assumed o be convex, bu wih a spaial dependence which is saionary and random. Rescaling in space and ime produces a similar equaion wih a rapidly varying spaial dependence and a small viscosiy erm. Moivaed by corresponding resuls for he linear ellipic equaion wih small viscosiy, we seek o find he limiing behavior of he soluion of he Cauchy (final value) problem in erms of a homogenized problem, described by a convex funcion of he gradien of he soluion. The main idea is o use he principle of dynamic programming o wrie a variaional formula for he soluion in erms of soluions of linear problems. We hen show ha asympoically i is enough o resric he opimizaion o a subclass, one for which he asympoic behavior can be fully analyzed. The paper oulines hese seps and refers o he recenly published work of Kosygina, Rezakhanlou and he auhor for full deails. Homogenizaion is a heory abou approximaing soluions of a differenial equaion wih rapidly varying coefficiens by a soluion of a consan coefficien differenial equaion of a similar naure. The simples example of is kind is he soluion u " of he equaion u " D x 2 a u " xx " I u".; x/ D f.x/ on Œ;. The funcion a. / is assumed o be uniformly posiive, coninuous and periodic of period. The limi u of u " exiss and solves he equaion u D Na 2 u xxi u.; x/ D f.x/ where Na is he harmonic mean dx Na D : a.x/ 397
398 S. R. SRINIVASA VARADHAN Alhough his is a resul abou soluions of PDE s i can be viewed as a limi heorem in probabiliy. If we consider he Markov process x./ wih generaor 2 a.x/d2 x saring from a ime, as! he limiing disribuion of y./ D x./ p Gaussian wih mean and variance Na. The acual variance of y./ is E a.x.s// ds : The resul on he convergence of u " o u is seen o follow from an ergodic heorem of he ype lim a.x.s// ds D Na:! From he heory of Markov processes one can see an ergodic heorem of his ype wih Na D a.x/.x/ dx; where.x/ is he normalized invarian measure on Œ; wih end poins idenified. This is seen o be dx.x/ D a.x/ a.x/ ; so ha dx Na D a.x/.x/ dx D : a.x/ We can consider he siuaion where a.x/ D a.x;!/ is a random process, saionary wih respec o ranslaions in x. We can formally consider a probabiliy space. ; ; P/, and an ergodic acion x of on. We also have a funcion a.!/ saisfying < c a.!/ C <. The saionary process a.x;!/ is given by a.x;!/ D a. x!/. Now he soluion u " of is u ".; x;!/ D 2 a.x;!/u" xx.; x;!/i u".; x;!/ D f.x/ can be shown o converge again, in probabiliy, o he nonrandom soluion u of u.; x/ D Na 2 u xx.; x/i u ".; x/ D f.x/ wih Na D a.!/ dp :
HOMOGENIATION OF RANDOM HAMILTON JACOBI BELLMAN EQUATIONS 399 This is also an ergodic heorem for a.!.s// ds; bu he acual Markov process!./ for which he ergodic heorem is proved is one ha akes values in wih generaor L D 2 a.!/d2 ; where D is he generaor of he ranslaion group x on. The invarian measure is seen o be dq D Na a.!/ dp; where Na D a.x/ dp : We will ry o adap his ype of approach o some nonlinear problems of Hamilon-Jacobi Bellman ype. One par of he work ha we ouline here was done joinly wih Elena Kosygina and Fraydoun Rezakhanlou and has appeared in prin [Kosygina e al. 26], while anoher par, carried ou wih Kosygina, has been submied for publicaion. The problems we wish o consider are of he form u " C " x 2 u" C H " ; ru" ;! D I u.t; x/ D f.x/ for Œ; T d. Here f is a coninuous funcion wih a mos linear growh.. ; ; P/ is a probabiliy space on which d acs ergodically as measure preserving ransformaions x. H.; p;!/ is a funcion on d which is a convex funcion of p for every! and H.x; p;!/ D H.; p; x!/. I saisfies some bounds and some addiional regulariy. The problem is o prove ha u "! u as "!, where u is a soluion of u C H.ru/ D I u.t; x/ D f.x/ for some convex funcion H.p/ of p and deermine i. The analysis consiss of several seps. We migh as well assume T D and concenrae on u ".; ;!/. Firs we noe ha, by rescaling, he problem can be reduced o he behavior of lim! u.; ;!/; where u is he soluion in Œ; d, of u s C 2u C H.x; ru;!/i u.; x/ D f x :
4 S. R. SRINIVASA VARADHAN The second sep is o use he principle of dynamic programming o wrie a variaional formula for u.s; x;!/. Denoe by L. x!; q/ he convex dual L.x; q;!/ D sup p hp; qi H.x; p;!/ Le b.s; x/ be a funcion b W Œ; d! d. Le B denoe he space all such bounded funcions. For each b 2 B, we consider he linear equaion vs b C 2 vb C hb.s; x/; rv b x i L. x!; b.s; x// D ; v.; x/ D f I hen he soluion u.s; x/ is sup b v b.s; x/. If we denoe by Q b he Markov process wih generaor L b s D 2 C hb.s; x/; ri saring from.; /, hen v b.; ;!/ D E Qb f. x./ / L.x.s/; b.s; x.s//;!/ ds and u D sup v b b2b The hird sep is o consider a subclass of B of he form b.; x/ D c. x!/ wih c W! d chosen from a reasonable class C. The soluion v b wih his choice of b.; x/ D b.x/ D c. x!/ will be denoed by v c. We will show ha for our choice of C, he limi lim! vc.; ;!/ D g.c/ will exis for every c 2 C. I hen follows ha lim inf! u.; / sup g.c/: c2c Given c here is a Markov process Q c;! on saring from! wih generaor A c D 2 C hc.!/; ri: Here r is he infiniesimal generaor of he d acion f x g and D r r. This process can be consruced by solving dx./ D c. x./!/ d C ˇ./I x./ D Then one lifs i o by defining!./ D x./!. Such a process wih generaor A c could have an invarian densiy P c and i could (alhough i is unlikely) be muually absoluely coninuous wih respec o P, having densiy c. c will be a weak soluion of 2 c D r c. / c:
HOMOGENIATION OF RANDOM HAMILTON JACOBI BELLMAN EQUATIONS 4 We can hen expec g.c/ D f c.!/ dp c L!; c.!/ dp c : In general he exisence of such a for a given c is nearly impossible o prove. On he oher hand for a given finding a c is easy. For insance, will do. More generally one can have c D r 2 c D r 2 C c ; so long as r c D. So pairs.c; / such ha 2 c D r c. / c exis. Our class C will be hose for which exiss. I is no hard o show, using he ergodiciy of f x g acion, ha is unique for a given c when i exiss and he Markov process wih generaor A c is ergodic wih dp c D cdp as invarian measure. We will denoe by C he class of pairs.c; / saisfying he above relaion. So we have a lower bound where lim inf! I.m/ D uc.; / sup m2 d Œf.m/ inf c; W.c; /2C R c dpdm I.m/ L.c.!/;!/ dp Now we urn o proving upper bounds. Fix 2 d. If we had a nice es funcion W.x;!/ such ha for almos all! jw.x;!/ h; xij o.jxj/ and 2W C H.x; rw;!/ Then, by convex dualiy wih QW D W.x;!/.s /, we have QW s C 2 QW C hb.s; x/; r QW i L.b.s; x/;!/ : If H./ is defined as H./ D inff W W exissg
42 S. R. SRINIVASA VARADHAN hen under some conrol on he growh of L, i is no hard o deduce ha wih f.x/ D h; xi, lim sup! u.; ;!/ H./ If we can prove ha H./ D sup m Œh; mi I.m/; we are done. We would mach he upper and lower bounds. We reduce his o a minmax equals maxmin heorem. sup m Œh; mi I.m/ D sup hc.!/; i L.c.!/;!/ dp.c; /2C D sup inf hc.!/; i C A c W L.c.!/;!/ dp.c; / W D inf hc.!/; i C A c W L.c.!/;!/ dp W.c; / D inf W 2 W C H. C rw;!/ dp D inf W sup! D H./: 2 W C H. C rw;!/ dp While W may no exis, rw will exis. We can inegrae on d, hen ergodic heorem will yield an esimae of he form W.x/ D o.jxj/ and h; xi C W.x/ will work as a es funcion. There are some echnical deails on he issues of growh and regulariy. The deails have appeared in [Kosygina e al. 26] along wih addiional references. Similar resuls on he homogenizaion of random Hamilon Jacobi Bellman equaions have been obained by Lions and Souganidis [25], using differen mehods. Now we examine he ime dependen case. If we replace d acion by dc acion wih.; x/ denoing ime and space, hen he saionary processes H and L are space ime processes. The lower bound works more or less in he same manner. In addiion o r we now have D he derivaive in he ime direcion. The!./ process is he space-ime process. Is consrucion for a given c is slighly differen. We sar wih b.; x/ D c. ;x!/ and consruc a diffusion on d corresponding o he ime dependen generaor A c s D 2 C hb.s; x/; ri
HOMOGENIATION OF RANDOM HAMILTON JACOBI BELLMAN EQUATIONS 43 and hen lif i by!.s/ D s;x.s/!. The invarian densiies are soluions of D C 2 D r c : The lower bound works he same way. Bu for obaining he upper bound, a es funcion W has o be consruced ha saisfies W C 2W C H.; x; rw;!/ H./ In he ime independen case here was a lower bound on he growh of he convex funcion H ha provided esimaes on rw. Here one has o work much harder in order o conrol in some manner W. The deails will appear in [Kosygina and Varadhan 28]. References [Kosygina and Varadhan 28] E. Kosygina and S. R. S. Varadhan, Homogenizaion of Hamilon Jacobi Bellman equaions wih respec o ime-space shifs in a saionary ergodic medium, Comm. Pure Appl. Mah. 6:6 (28). [Kosygina e al. 26] E. Kosygina, F. Rezakhanlou, and S. R. S. Varadhan, Sochasic homogenizaion of Hamilon Jacobi Bellman equaions, Comm. Pure Appl. Mah. 59: (26), 489 52. [Lions and Souganidis 25] P.-L. Lions and P. E. Souganidis, Homogenizaion of viscous Hamilon Jacobi equaions in saionary ergodic media, Comm. Parial Differenial Equaions 3:-3 (25), 335 375. S. R. SRINIVASA VARADHAN COURANT INSTITUTE NEW YORK UNIVERSITY 25 MERCER STREET NEW YORK, NY 2 UNITED STATES varadhan@cims.nyu.edu