Opmal Conrol Why Use I - verss calcls of varaons, opmal conrol More generaly More convenen wh consrans (e.g., can p consrans on he dervaves More nsghs no problem (a leas more apparen han hrogh calcls of varaons General Problem - ma f (,, ( d s.. '(,, (,,, ( fed; ( free Sae Varables - governed by frs order dfferenal eqaons (sae eqaons Conrol Varables - decson varables; assmed o be pecewse connos (connos ecep for fne nmber of pons ha aren'; affec objecve drecly and hrogh sae varables Sae Eqaon - dfferenal eqaon ha governs change n a sae varable; also called ranson eqaon; shold have one sae eqaon for each sae varable Fncons - f and g are connosly dfferenable fncons of hree ndependen argmens, none of whch s a dervave; n fac, here are no dervaves n opmal conrol ecep for he lef sde of he sae eqaons Smples Problem - ypcal calcls of varaon problem looks lke: ma f (,, '( d s.. ( Ths can be ransformed no opmal conrol problem by sng ma '( : f (,, ( d s.. '(, ( Transons - hs one was easy, b n general he hardes par s choosng he whch varables are sae and conrol varables Solvng - Hamlonan - H (,, (, ( f (,,,, (smlar o a lagrangan Properes of Opmal Solon - hese are necessary condons ( f g Opmaly Condon ( '( ( f g Mlpler Eqaon (3 ' ( Sae Eqaon (4 ( Bondary Condon (5 ( Bondary Condon (n hs case s free (6 H for ma ( for mn Second Order Condon Economc Inerpreaon - ( s margnal valaon of he assocaed sae varable a me ( becase s free; n mamzaon problem, we'd choose so we can' gan by changng so margnal benef of s zero ( of 9
General Procedre - [ Use ( o fnd ha mamzes H (same condon for mnmzaon problem [ Solve from [ as a fncon of,, and λ: (,, [3 Sbse o of ( and (3 whch gves s a sysem of wo dfferenal eqaons for and λ (,, (,, g(,, (,, ' '(,, (,, Eqvalen o Eler and Transversaly - Proof: look a ma f (,, '( d s.. ( Already showed eqvalen opmal conrol problem (se '( H f (,, From (: f λ d Dfferenae boh sdes wr : λ '( d From (: '( ( f g (becase d d Combne hese o ge Eler Eqaon: d d ' From (5: (... ha's he ransversaly condon: f ' From (6: H... same as he Legendre condon f 5 Eample - ma ( d s.. ', ( (,, f g (,, H f (,,,, ( Opmal solon properes: ( ( '( ( f g ( (3 ' ( (4 ( and (5 (5 (6 H (reqred for ma... good Solve ( for : of 9 ' '
Sb no ( and (3 for sysem of dfferenal eqaons: 3 '( 3 3 '( From here we se Trck 4 from he DffEq hando: r r General Solon: Ae Ae r r ( r a A ( r a A ( Be Be (7 B and (8 B b 3 b a b Coeffcen Mar: 3 3 a b 3 r 3 Characersc Eqaon: ( 3 ( 3 4 3 3 r r (egenvales r 3 r r 3 r 3 and r 3 9 4 4 4 Eqaons & 4 Unknowns - now se (4, (5, (7, (8 o solve for A, A, B, B Several Varables Each sae varable ( needs a sae eqaon: '( g ( Need one mlpler for each sae eqaon: '( Each conrol varable yelds one opmaly condon: # conrol varables cold be >,, or < # of sae varables General Problem - sae varables ( and conrol varables (j f (, (, (, (, ( d j '( g (,, (, (, ( (, ( (, ( ma s..,, (sae eqaons fed; free (end condons Hamlonan - H (,, (, (, ( f g g Properes of Opmal Solon - ( g g, j, j j j ( H f g g '(,, (3 '( g (, (, (, (, (,, (4 fed end condons from above (5 (,, j j 3 of 9
End Pon Condons Fed End Pons - modfy orgnal problem for boh end pons fed: ma Dervng Necessary Condons - Le J * be ma vale J * P f (, *, * d f (,, ( d s.. '(,, ( (, ( ± *', b realze *' g(, *, *,, all fed : J * [ f (, *, *, *, * *' Wha o ge rde of he *' n he las erm so solve ha wh negraon by pars: b a b b a UdV UV VdU... le U λ and dv *' d du λ' d and V * *' d * * a * ' d Plg ha back n: [ f (, *, *, *, * ' * J * d * * Pck a comparson pah [ f,,,, ' Snce J * s ma vale: J J* J Sb formlas: J ( d J {[ f (,,,, ' [ f (, *, *, *, * ' * } d * * Snce J s a feasble pah, ms have he same endpons as J * erms osde negral dsappear: * * Defne: Sb no L(,,, f (,,,, ' J : J [ L(,,, L(, *, *, d Ths s a dfference of he same fncon so we can se Taylor seres o appromae (noe ha and λ are fed: L(,,, L(, *, *, L (, *, *, ( * L (, *, *, ( * h.o.. where L f g ' and L f g d 4 of 9
Drop he hgher order erms o appromae J J : [( f g '( * ( f g ( * d Snce hs condon holds for arbrary, le ' ( f g (mlpler eqaon Now we have J ( f g ( * d Snce * can be >,, or <, we ms have f g (opmaly condon So we end p wh he same necessary condons we had on p., ecep condon (5 In general condon (5 wll be ( f s fed ( f s free Salvage Vale - back o orgnal problem and add salvage vale o objecve fncon (.e., vale ha's only based on endng condons and ; noe f boh and are fed, hs salvage vale s rrelevan becase wold be consan ma f (,, ( d φ(, s.. '(,, (,, ( fed, ( free Dervng Necessary Condons - 3 sep solon Sep - arbrarly choose fnal sae (... we js covered how o solve ha n prevos secon Le V ( f (, *, * d be ma vale of negral gven P * ± *', b realze *' g(, *, * : V * [ f (, *, *, *, * *' Wha o ge rde of he *' n he las erm so solve ha wh negraon by pars js lke we dd las me so we end p wh: ('s on he mddle of he prevos page Le [ f (, *, *, *, * ' * V * d * * (, *, * f g so now V * [ H (, *, * ' * d * * H Sep - show ( s margnal vale of ( Look a - look a change n V * wh respec o : d d* d* d d* d ' H H H H ' * d ( d d d d d d λ was chosen arbrarly so 's ndependen of ; also me s ndependen of hose dervaves are zero ( d 5 of 9
( H d* ' d d* d d H We know from necessary condons ha ( H f g ' and H he negral goes away and we have V * (... whch means he change n he mamm vale ( V * cased by a change n he sarng condon ( s (....e., margnal vale of ( Look a any (, - js break p he nerval; s nal pon for second negral; same as n prevos proof V * ( f (, *, * d f (, *, * d f (, *, * d Look a - look a change n V * wh respec o : d d* d* d d* d ' H H H H ' * d ( d d d d d d λ was chosen arbrarly so 's ndependen of ; also me s ndependen of hose dervaves are zero d* d* ( H ' H d ( d d We know from necessary condons ha H f g ' and H he negral goes away and we have (... whch means he change n he mamm vale ( V * cased by a change n he endng condon ( s ( `....e., margnal vale of ( Sep 3 - consder ( beng free; f we ge o choose ( obvosly we'd wan o mamze V * so we end p wh (... hs gnores he salvage vale hogh To consder he salvage vale le W f,, ( d φ(, ( Now defne W* ( V* φ(, s ma vale for gven ( We wan o mamze W * wr ( : W* φ(, φ(, ( whch means he margnal benef of salvage margnal cos of salvage Noe: f here s no salvage erm φ so hs becomes he same as he condon we had before ( 6 of 9
Eample - monopoly bldng prodc over me and sellng a me : r ma e P( [ c c revene (salvage e cos over me Endpon Consran - back o orgnal problem and add consran o endpon ma f (,, ( d s.. '(,, ( and K (,, ( fed, ( free Dervng Necessary Condons - Use same mehodology as prevos dervaon o ge V *( Now search for opmal ( sasfyng consran K ( Lagrangan: L V* ( p( K( KT Condons: L V* ( p( K'(, wh eqaly f ( > We know ( > and ( ( p( K'( L K(, wh eqaly f p >... so we ge cases: p (a f no bndng ( K ( >, hen p ( (b f bndng, hen K ( and ( p( K'( When solvng a problem, we sally have o look a boh cases and pck he one ha works (one probably won' have a solon; f boh have a solon, hen plg he solon no he objecve o see whch s beer r d Margnal vale of consran a Margnal vale of consran Change n consran a Free Horzon - back o orgnal problem and le be free ma f (,, ( d s.. '(,, (, (, ( fed, free Dervng Necessary Condons - V * (, H (, *, * ' * d ( ( Use same mehodology o ge [ Now opmze V * (, wr... need Lebnz Rle becase s n negral d* d* H (, *(, *(, ( λ'( *( ( H ' H d d d Noe erms ha cancel '( 7 of 9
Also a opmm we know H ' and H so negral erm goes away ( f g H (, * (, *(, ( Opmal Adversng Eample - a( adversng ependre A( goodwll (accmlaed adversng ependre Deprecaon of goodwll: A' ( a( δa( Sales: q ( f ( p(, A( Revene: p ( q( Revene ne of prodcon cos: R( p(, A( p( q( C( q( Insananeos prof: R( p(, A( a( Problem: ma e r [ R( p(, A( a( a(, p( d s.. A' ( a( δa(, a (, p (, A ( A fed Conrol Varables - a ( and p ( Sae Varable - A ( r H e ( R a ( a δa Observaons - (a prce eners negrand (objecve, b no sae r R (a e eqaon we can se sac opmzaon p p Le ( A R( p*( A, A (b r e p* ( A arg ma R( p, A (.e., p* s vale ha a p ( r R mamzes R for a gven A ' e δ (b a doesn' ener s own frs order condon so we can' solve he radonal way Look a problem wh only conrol varable r H e ( ( A a ( a δa r ( e a r ( A ( ' e δ Sysem of Dfferenal Eqaons: A' a δa ( A ' e r δ... can' go frher wh DffEq who more deals on ( A Can sll solve problem hogh: ( e r lm ( and ' re r Sb e r r ( A r no (: ' e δe r r ( A Se hese eqal: ' re e δe r 8 of 9
( A ( A Cancel e -r : r δ r δ A* s consan over me and we wan o ge o as soon as possble (jmp o mmedaely, no gradally A' a δa a* δa* Solon: If A < A* hen wan o jmp o A * so a* ( A* A If A > A* hen a *( nl A ( A*, hen a* ( δa Smmary - ma f (,, d s.. sae eqaon: '(,, sar me: fed end me: fed (nless saed oherwse sar condon: ( fed end condon: ( free (nless saed oherwse H (,,, f g Necessary Condons - (a ' (,, Sae eqaon g (b ' Mlpler (cosae, alary, adjon eqaon g (c Opmaly condon (d Transversaly condons: ( fed and ( fed ( ( fed and ( free ( ( free and ( fed H ( f g (v Endpon Consran: K ( Cases - solve boh; f here s a conradcon n one, 's no a solon; f here s no conradcon, ha case s a solon (cold have more han one or none Case - Bndng ( K ( ( pk'( Case - No bndng ( K ( > p so ( (v Salvage Vale: add φ (, o objecve wh fed and ( free (, ( φ 9 of 9