1. THE CALCULUS OF VARIATIONS

Transcription

1 1. HE CALCULUS OF VARIAIONS During he cold wr (s I her he sory), couple of nmeless superpowers invesed gre del of reserch monies ino solving opimizion prolems involving rjecories nmely, ino finding he opiml wy o lunch missiles ino your opponens living rooms nd schools. Ech of hese superpowers independenly developed sregy for solving hese prolems. his redundncy llows us o ckle simple dynmic opimizion prolems wih he dynmic progrmming pproch dvnced y Bellmn nd ohers or lernively wih he mximum principle of Ponrygin nd his crew. One of my fvorie quoes is on Mr. Ponrygin s recion o lerning ou he Bellmn sregy in less delice erms, he uhor of his line likened Ponrygin s recion o person hndling ded r. Being le o ell his sory is one of my fvorie prs of eing le o lk ou he pproches o dynmic prolems, s well s eing le o ell my friends nd fmily h I prcice rocke science in my spre ime. While he mximum principle nd dynmic progrmming hve heir own srengh nd limiions in solving hese prolems, I hve menioned lredy heir common wekness: s hey re wrien, hey work well only in simple vriionl prolems. he clculus of vriions ws pioneered y such noles s he Bernoulli Brohers, heir suden Euler, nd some of heir drinking uddies. Originlly someone ws posed wih he sk of designing curved werslide for seveneenhcenury werprk: he prolem ws o pick he shpe h go people from poin A o poin B s quickly s possile, ccelered only y grviy. his is known s he rchisochrone prolem. he soluion is no srigh line i is slide shped like cycloid, priculr funcion h is iniilly firly seep, geing shllower. he generl vriionl prolem is h we would like o pick ph for prmerized vrile o mximize he inegrl of some given funcion of h vrile. he simple cse (s I deem i) presens his inegrl: J( x)= f( x(),«( x ), ) d

2 he chllenge is o find n exremum ( mximum or minimum) of Jx ( ) wih respec o some funcion x (); Jx ( ) is ofen clled he cos funcion of he conrol x (). he noion x «( ) denoes he firs pril derivive of x () wih respec o ime. here is n ddiionl requiremen for he funcion x (): we re given he endpoins x ( ) nd x ( ). In he rchisochrone prolem, hese were wo poins in spce nd we were sked o find he ph eween hem h hd he les ime cos (ime is money when you re werprk). his form works for ny dynmic prolem in which one mus consider oh he effecs of some vrile s well s chnging i for insnce, hink of firm h ries o mximize profis, which re funcion of cpil nd lor, over numer of ime periods. I is lso fced wih cos of djusing is workforce or cpil rining nd lyoff coss for lor, insllion nd removl coss for mchinery. In he sence of such djusmen coss, he profi funcion over yer migh look like: ( )= [ ( ) ] Π k, l p y k ( ), l( ) w l() r k () d where y is producion funcion from kpil nd lor, nd p is he price of oupu. he second nd hird erms in he inegrnd re he coss of hiring hese inpus. Of course, we re old he moun of ech h we sred he yer wih, nd our consulns hve dvised us wh our rges should e he end. Now consider djusmen coss, which my look like: [ ] ( )= ( ) ( ) Π k, l p y k ( ), l ( ) w l () r k () c «l(), k «() d his is very rel prolem fcing firms ouside simple sic models of producion. he hing wihin rckes fis ino he form specified for he simple cse, nd we mus use he clculus of vriions (in one form or noher) o solve i. he soluion o he simple cse sisfies he Euler-Lgrnge equion; ll vriionl prolems re solved wih some generlizion of his. Here is n lmos heorem for he simple cse: provided we don hve differeniiliy prolems, in order for x () = rg mx J( x)= rg mx f( x ( ), x «( ), d ), sujec o iniil nd erminl condiions on x, i is necessry h he funcion lso sisfy he equion:

3 f d f = x d x«h s he punchline, he sis ool of he clculus of vriions! How is his derived, he ineresed reder my sk? (he unineresed reder my choose o skip his lmos proof.) he generl ide is h we ke he differenil of he cos funcion nd se i equl o zero: f δj x δ δ δ δ x x f x xd f x x d f ( )= + x x d = + = ««««he ide conveyed y δx is h we hve vried he funcion x lile i hroughou, hough we were required o hold down he endpoins. For no ppren reson, some very clever person ried inegring he ls erm in his equion y prs. He found h: = = f x x d f x x d f δ xd ««δ δ «d x«= his goes ck ino he differenil of J: = = f δj x δ δ δ x x d d f f ( )= x d d x x x + = ««he ls erm requires us o evlue he differenil δx is endpoins. Bu when we were vrying he funcion x, we were required o keep he iniil nd erminl vlues unchnged nd his mens h he ls erm equls zero! hen he pproprie condiion for opimliy is f d f δj( x)= δx d x d x = «One wy o solve his, clerly, is o se he erm in rckes equl o zero for ll [, ]. he firs fundmenl lemm of he clculus of vriions ells us h his is he only wy o chieve h condiion for ll dmissile choices of δx h is, for ll he possile wys o vry he funcion x, keeping i differenile enough nd keeping is endpoins unchnged. QED! (More or less.) Knowing how he simple resul is derived llows us o derive our own opimliy condiions when he prolem doesn fi ino he form of he simple cse. However, le s look couple of exmples firs.

4 Exmple: he mximum principle. Le s reurn o he generl opiml conrol prolem. Recll h in he generl cse, we were sked o solve (his funcion f is differen from he funcion f in he vriionl prolem): ( ) mx u f x( ), u( ), d sujec o: x«() g x(), u(), My inclinion is o re his like plin old consrined opimizion prolem excep wih consrin for ech ime period. he Lgrnge muliplier for ime is represened y λ( ). My inuiion is o define his Lgrngin funcion: = ( ) ( )= ( ) + [ ( ) ] L x (), u (), f x (), u (), d λ() g x (), u (), x «( ) d [ ] = ( )+ [ ( ) ] f x(), u(), λ() g x(), u(), x«( ) d he second line ppers o e in form similr o he simple cse. Since x () nd u () nd heir ime derivives re he only vriles showing up in he inegrnd, I wrie ou he Euler-Lgrnge equions for opimliy: ( f u)+ λ( ) ( g u)= ( f x)+ λ() ( g x)+ ( dλ d)= Wih sligh rerrngemen of he second condiion, hese ecome: f u+ λ( ) g u = f x+ λ() g x= λ«() I is lef s n exercise o he reder o flip ck secion or wo nd verify h hese re excly he sme opimliy condiions required y he Ponrygin mximum principle. Exmple: djusmen coss. Le s go ck o he cse of our firm h hd o mximize profis, sujec o coss of djusing heir cpil nd lor socks: [ ] ( )= ( ) ( ) Π k, l p y k ( ), l ( ) w l () r k () c «l(), k «() d he necessry condiions o opimize his prolem re: p yk d c k ( (), l() ) («l (), «() ) r + = k () d k «() p yk w d c k ( (), l() ) («l (), «() ) + = () d «l l () he firm will generlly no se he mrginl revenue produc of fcor equl o he fcor price in he presence of djusmen coss. his will e he cse only if he djusmen cos is he sme ll imes; wih decresing reurns o scle i will e he cse only if he firm is mking no chnges in cpil nd lor.

5 Exmple: squndering n inherince. A =, our friend gins ccess o his rus fund, sses vlued A. He wns o live off of his for he res of his life wihou working. Le c () denoe his consumpion ime. he cn -ke-iwih-you heorem sys h no erminl sses or des re permied: A ( )=. His lifeime uiliy is represened s: ( )= ( ) β Uc e uc () d he chnge in his sses wih ime cn e descried y: A «() = ρa () c () where ρ is he insnneous ineres re. Susiuing his ck ino his uiliy funcion, we ge h lifeime uiliy depends on he chosen ph A () nd A. «() ( ) β Uc ( )= e uρ A () A «() d his is vriion on he Solow growh model. he rue Euler-Lgrnge equion here ers some similriy o wh mcroeconomiss imprecisely descrie s Euler equions : β d β ρe u c d e ( ())+ [ u ( c () )]= ρ β u c() u c ()«( c) ( ) ( ) ( ) = hose re nice resuls, hough generlly his pproch hs no dvnge over he Bellmn nd Ponrygin sregies. Wh, however, hppens when good prolems go d when he simple Euler-Lgrnge formul fils us, or does no go fr enough? Here re some vriions on h equion. Vriion 1. A simple rewriing of he equion h my or my no provide more insigh ino he simple cse: f d f = x d x«+ + f = xx x f xx x f f ««««««x«x his is essenilly wh I did in order o derive he implicion of he Euler equion of he inherince squndering exmple: proceeding o olly differenie he derivive wih respec o A. «()

6 Vriion. he cos funcion my conin higher-order derivives of x, wih he inerpreion h here is some cos o djusing he sock nd ddiionl coss o chnging he re of djusmen: ( d n )= ( d d d n ) d d n n dx dx d x dx dx d x J x,,, K,, f x,,, K,, d hen we require iniil nd erminl condiions for x s well s for he firs n-1 ime derivives. he generlizion of he Euler-Lgrnge equion is: n f d f ( ) + d f ( ) + ( n d f K 1) x d d d ( ) = dx dx n n d x d n d d Exmple: inherince squndering wih he persisence of memory. Le s revisi his prolem, dding wis: our idle rich friend grows ccusomed o his sndrd of living, nd so here is some psychic cos of chnging his consumpion eween periods. I model his hi-forming uiliy funcion s: ( )= ( ) β Uc e uc (),«( c) d he udge consrin in his prolem gives us he chnge in consumpion: c () = ρ A () A «() c «() = ρ A «() A ««() We would hve o mke some sor of ssumpions or inferences ou he iniil nd erminl vlues of c «( ), u hen lifeime uiliy cn e wrien s funcion of he ime ph of sses, s well s is firs wo derivives: β Uc ( )= e uρa () A «(), ρa «() A ««() d ( ) he opimliy condiion is: β u d u u d u e e e e c + β d c β β ρ ρ = c «d c «Expnding his is relly messy, which is one of mny good reson no o work wih hi-forming preferences. Vriion 3. he cos funcion my e inegred over region in Ω R k, nd x is funcion of k prmeers h I cll,, K, : 1 k ( d n )= ( d d 1 k d1 d dk 1 k) 1 k n dx dx d x dx dx dx J x,,, K,, K f x(,, K, );,, K, ;,, K, d d Kd Ω wih n iniil nd erminl condiion for x. he necessry condiion for his prolem is:

7 f d f x d dx d ( ) d d f dx ( d ) 1 1 d f K dk dx ( d k ) = Exmple: inherince squndering wih your spouse. As is oo ofen he cse, our chelor friend hs kicked ll his es his nd gone nd goen himself mrried. Wih discouning included in he funcion u (, i ), his lifeime is once gin given y: U1( c)= u1( c1(), ) d And his spouse hs similr uiliy. Becuse of some rgining rrngemen, hey wn o mximize he produc of heir lifeime uiliies. his is known s he Nsh socil welfre funcion: 1 ( )= ( ) ( ) Nc (), cs ( ) u c (), u cs ( ), sdsd

8 HINK SMALL WHEN HINKING VARIAIONALLY n he clssicl vriionl prolems is o find funcion x: R o solve prolem of he form: mx f( x( ), x«( ), ) d s.. x( ) =, x() 1 = Admissile soluions o he prolem, s fr s we re concerned, re in suse of C. Unless non-exisen or corner, he soluion x * will sisfy he Euler- Lgrnge equion: f () d f () = ƒ [, 1] * * x () d x«() he derivion of his formul comes s follows. Firs, we define loss funcion for priculr conrol x; h is, JC R defined y: ( ) = ( ) J x : f x( ),«( x ), d : hen we consider some funcions which re wee iny perurions δx of he funcion x. he chnge in he loss is: f () f () δj( x; δx) : = ( δx ()) d+ ( δx «( )) d x () x «( ) Nurlly, his loss will e zero for sionry x *. In order o ge rid of h nsy erm δ«x, we re going o hve o rememer o use some inegrion-y-prs rick: = f () d f () f () δj( x; δx) : = δx () d δx () d δx () ( ) x () ( ) + d x «( ) x «( ) Bu if we re only considering perurions which leve he endpoins unchnged he only sensile ones we ll hve h ( δx() ) = nd ( δx() ) = =, so =1 h he ls erm disppers. A his poin we ll invoke he firs fundmenl lemm of vriionl clculus, which siclly sys: ( g ())( δy ()) d= δy () smll { g () ƒ } And so we hve he desired resul, f () d f () = * * x () d x«() Unless we wn o prove he lemm, h s more or less QED. = 1 Bu his isn he wy h Euler did i. Bck in his dy, infiniesimls were ll he rge. Ldies in Pris frequenly wore hem on heir hs. A h ime, we would

9 hve hough ou picking some relly, relly ig ineger N (infiniely lrge!) nd hen se = 1 N. he ide of n inegrl ws inerchngele wih h of sum: = N + f x x d f x x x ( (),«( ),) =,, = If we wn o exremize his prolem, we would seek sequence of { x } o solve he prolem. We don even need o ssume nyhing ou he disnce eween x nd x + nmely, i doesn even hve o e coninuous, or coninuously differenile. Believe i or no, his is n dvnge when working wih so-clled swooh prolems nd oher cses in which he soluion requires somehing like chering. So how does one find he sequence of { x } o mximize he funcion? Essenilly, jus find he sionry x on heir own, which should sisfy: = N + f x x x =,, x = f + f f = x x«x«f f 1 f x x«x«= f f : = x x«h s jus so slick, i s hrd o elieve h i cully plces fewer resricions on se of dmissile conrols. I lso kes ou 9 seconds o derive on he ck of n envelope, when you cn recll he formul. We hve sor of glossed over wheher ll x eing sionry is necessry o mximize he prolem, u i urns ou o e lmos necessry. Prolem: Using he usul mehod, find he necessry condiion for conrol x () o mximize he prolem: [ o, 1] ( ) f x(), x( s),«( x ),«( x s) ds d hen ry finding he necessry condiions using Euler s infiniesiml mehod. Prolem: Using he oh mehods, find he soluion o he prolem: min ( 1+ x ) 1+ («1) x d wih x = x1 =. In ech cse, verify h his is in fc soluion.

10 n he clssicl vriionl prolems is o find funcion x: R o solve prolem of he form: mx f( x( ), x«( ), ) d s.. x( ) =, x() 1 = Admissile soluions o he prolem, s fr s we re concerned, re in suse of C. Unless non-exisen or corner, he soluion x * will sisfy he Euler- Lgrnge equion: f () d f () = ƒ [, 1] * * x () d x«() he derivion of his formul comes s follows. Firs, we define loss funcion for priculr conrol x; h is, JC R defined y: ( ) = ( ) J x : f x( ),«( x ), d : hen we consider some funcions which re wee iny perurions δx of he funcion x. he chnge in he loss is: f () f () δj( x; δx) : = ( δx ()) d+ ( δx «( )) d x () x «( ) Nurlly, his loss will e zero for sionry x *. In order o ge rid of h nsy erm δ«x, we re going o hve o rememer o use some inegrion-y-prs rick: = f () d f () f () δj( x; δx) : = δx () d δx () d δx () ( ) x () ( ) + d x «( ) x «( ) Bu if we re only considering perurions which leve he endpoins unchnged he only sensile ones we ll hve h ( δx() ) = nd ( δx() ) = =, so =1 h he ls erm disppers. A his poin we ll invoke he firs fundmenl lemm of vriionl clculus, which siclly sys: ( g ())( δy ()) d= δy () smll { g () ƒ } And so we hve he desired resul, f () d f () = * * x () d x«() Unless we wn o prove he lemm, h s more or less QED. = 1 Bu his isn he wy h Euler did i. Bck in his dy, infiniesimls were ll he rge. Ldies in Pris frequenly wore hem on heir hs. A h ime, we would

11 hve hough ou picking some relly, relly ig ineger N (infiniely lrge!) nd hen se = 1 N. he ide of n inegrl ws inerchngele wih h of sum: = N + f x x d f x x x ( (),«( ),) =,, = If we wn o exremize his prolem, we would seek sequence of { x } o solve he prolem. We don even need o ssume nyhing ou he disnce eween x nd x + nmely, i doesn even hve o e coninuous, or coninuously differenile. Believe i or no, his is n dvnge when working wih so-clled swooh prolems nd oher cses in which he soluion requires somehing like chering. So how does one find he sequence of { x } o mximize he funcion? Essenilly, jus find he sionry x on heir own, which should sisfy: = N + f x x x =,, x = f + f f = x x«x«f f 1 f x x«x«= f f : = x x«h s jus so slick, i s hrd o elieve h i cully plces fewer resricions on se of dmissile conrols. I lso kes ou 9 seconds o derive on he ck of n envelope, when you cn recll he formul. We hve sor of glossed over wheher ll x eing sionry is necessry o mximize he prolem, u i urns ou o e lmos necessry. Prolem: Using he usul mehod, find he necessry condiion for conrol x () o mximize he prolem: [ o, 1] ( ) f x(), x( s),«( x ),«( x s) ds d hen ry finding he necessry condiions using Euler s infiniesiml mehod. Prolem: Using he oh mehods, find he soluion o he prolem: min ( 1+ x ) 1+ («1) x d wih x = x1 =. In ech cse, verify h his is in fc soluion.