Notes on Optimal Control

Transcription

1 F.L. Lews Moncef-O Donnell Endowed Cha Head Conols & Sensos Gop Aomaon & Robocs Reseach Inse ARRI he Unvesy of eas a Alngon Noes on Opmal Conol Sppoed by : NSF - PAUL WERBOS ARO RANDY ZACHERY Dagna abe al avalable onlne a hp://arri.a.ed/acs

2 Me Ahans PhD fom Beeley n 96 BOOKS Opmal Conol McGaw Hll 966 Sysems Newos and Compaon: Basc Conceps McGaw Hll 972 Mlvaable Mehods McGaw Hll 974 hd Pesden of he IEEE Conol Sysems Socey 972 o 974 Moe han 40 Phd sdens- how many?

3 Awads Amecan Aomac Conol Concl's Donald P. Ecman Awad "fo osandng conbons o he feld of aomac conol" fs ecpen of he awad 964 Amecan Socey fo Engneeng Edcaon's Fedec E. eman Awad as "he osandng yong eleccal engneeng edcao" fs ecpen 969 Amecan Conol Concl's Edcaon Awad fo "osandng conbons and dsngshed leadeshp n aomac conol edcaon" second ecpen 980 Fellow of he IEEE 973 Fellow of he AAAS 977 Dsngshed Membe of he IEEE Conol Sysems Socey 983 IEEE Conol Sysems Socey's 993 H.W. Bode Pze ncldng he delvey of he Bode Plenay Lece a he 993 IEEE Confeence on Decson and Conol Amecan Aomac Conol Concl's Rchad E. Bellman Conol Heage Awad 995 "In Recognon of a Dsngshed Caee n Aomac Conol; As a Leade and Champon of Innovave Reseach; As a Conbo o Fndamenal Knowledge n Opmal Adapve Robs Decenalzed and Dsbed Conol; and as a Meno o hs Sdens" Ahans & Falb Opmal Conol McGaw Hll 966 Fs boo on OC?

4 . Use Neal Newos as Fncon Appomaos n Adapve Conol 2. Opmal Adapve Conol gves new adapve conol algohms

5 wo-laye feedfowad sac neal newo NN σ. W 2 σ. σ. y 2 σ. y 2 3 σ. n σ. y m nps σ. L ops Smmaon eqs hdden laye Ma eqs y σ K σ w n j v j j v 0 w 0 y W σ Have he nvesal appomaon popey Ovecome Baon s fndamenal accacy lmaon of -laye NN

6 Neal Newo Robo Conolle Unvesal Appomaon Popey Feedfowad Loop.. q d Feedbac lneazaon Nonlnea Inne Loop q d e [Λ I] K v ^ f τ Robo Sysem q Robs Conol em v PD acng Loop NN nvesal bass popey means no egesson ma s needed Eenson of Adapve Conol o Nonlnea-n he paamees sysems Poblem- Nonlnea n he NN weghs so ha sandad poof echnqes do no wo

7 Eenson of Adapve Conol o nonlnea-n paamees sysems No egesson ma needed heoem NN Wegh nng fo Sably Le he desed ajecoy q d and s devaves be bonded. Le he nal acng eo be whn a cean allowable se U. Le Z M be a nown ppe bond on he Fobens nom of he nnown deal weghs Z. ae he conol np as NLIP τ Wˆ σ ˆ K v wh v K Z Z. v Z F M Le wegh nng be povded by Wˆ F ˆ σ F ˆ' σ ˆ κf Ea Jacoban ems fom NLIP newo ˆ Wˆ G ˆ' σ W κg ˆ ˆ wh any consan maces F F > 0 G G > 0 and scala nng paamee κ > 0. Inalze he wegh esmaes as W ˆ 0 ˆ andom. hen he fleed acng eo and NN wegh esmaes Wˆ ˆ ae nfomly lmaely bonded. Moeove abaly small acng eo may be acheved by selecng lage conol gans K. v Bacpop ems- Webos obsfyng emse-mod sgma mod ec.

8 Fleble & baoy Sysems Bacseppng.. q d Add an ea feedbac loop wo NN needed Use passvy o show sably Nonlnea FB Lneazaon Loop NN# q d q. d q d e e. e [Λ I] K q F ^ q. q d η e Robo /K B K η Sysem Robs Conol em F ^ v 2 NN#2 q acng Loop Bacseppng Loop Neal newo bacseppng conolle fo Fleble-Jon obo am Advanages ove adonal Bacseppng- no egesson fncons needed

9 Acao Nonlneaes - q d Deadzone saaon baclash NN n Feedfowad Loop- Deadzone Compensaon Esmae of Nonlnea Fncon lle obseve NN Deadzone Pecompensao τ f I q d - e [Λ Τ Ι] K v - v w II D τ Mechancal Sysem q Cc: Aco: Wˆ Wˆ σ U w W σ ' ˆ U Sσ ' U U Wˆ σ U U w ˆ W S Wˆ 2 Wˆ Wˆ Acs le a 2-laye NN Wh enhanced bacpop nng!

10 Opmaly n Bologcal Sysems Cell Homeosass he ndvdal cell s a comple feedbac conol sysem. I pmps ons acoss he cell membane o manan homeosas and has only lmed enegy o do so. Pemeably conol of he cell membane Cellla Meabolsm hp://

11 Opmaly n Conol Sysems Desgn R. Kalman 960 Roce Ob Injecon Dynamcs w 2 v μ F w snφ 2 m wv F v cosφ m m Fm Objecves Ge o ob n mnmm me Use mnmm fel hp://mcosa.sm.bms./e-lbay/lanch/dnep_geo.pdf

12 Adapve Conol s geneally no Opmal Opmal Conol s off-lne and needs o now he sysem dynamcs o solve desgn eqs. We wan ONLINE ADAPIE OPIMAL Conol

13 f d R Q d 0 H f 0 0 mn mn 0 * * f g R h * 2 * d d g gr d d Q f d d * * 4 * Sysem Cos Hamlonan Opmal cos Opmal conol HJB eqaon Connos-me Opmal Conol Bellman mn mn 0 f Fo a gven conol he cos sasfes hs eq. In LQR hs s a Lyapnov eq In LQR hs s a Rcca eq

14 Lnea sysem qadac cos Sysem: Uly: A B Q R; R > 0 Q 0 he cos s qadac Opmal conol sae feedbac: R BP L dτ P HJB eqaon s he algebac Rcca eqaon ARE: 0 PA A P Q PBR B P Fll sysem dynamcs ms be nown Off-lne solon

15 C Polcy Ieaon o avod solvng HJB eqaon Uly Q R Cos fo any gven 0 f H Lyapnov eqaon Ieave solon Pc sablzng nal conol Fnd cos 0 f h h 0 0 Updae conol h 2 R g Convegence poved by Sads 979 f Lyapnov eq. solved eacly Bead & Sads sed complcaed Galen Inegals o solve Lyapnov eq. Ab Khalaf & Lews sed NN o appo. fo nonlnea sysems and poved convegence Fll sysem dynamcs ms be nown Off-lne solon

16 LQR Polcy eaon Klenman algohm. Fo a gven conol polcy L solve fo he cos: 0 A A P P A C C 2. Impove polcy: L A BL R B P L RL If saed wh a sablzng conol polcy L0 he ma P monooncally conveges o he nqe posve defne solon of he Rcca eqaon. Evey eaon sep wll en a sablzng conolle. he sysem has o be nown. Lyapnov eq. Klenman 968

17 Polcy Ieaon Solon Polcy eaon A BB P P P A BB P PBB P Q 0 hs s n fac a Newon s Mehod Rc P A P PA Q PBB P hen Polcy Ieaon s P P Rc P Rc P 0 Feche Devave Rc' P P A BB P P P A BB P

18 Dagna abe Polcy Ieaons who Lyapnov Eqaons Dynamc pogammng bl on Bellman s opmaly pncple alenave fom fo C Sysems [Ahans & Falb 966 Lews & Symos 995] mn d Δ Δ * * τ τ τ τ τ < Δ τ τ τ Q τ τ R τ f and g do no appea

19 Dagna abe Solvng fo he cos O appoach Fo a gven conol he cos sasfes L Q R d f and g do no appea LQR case P Q R d P Opmal gan s L R B P

20 Dagna abe Solvng fo he cos O appoach. Polcy eaon Cos pdae Fo LQR case Q R d L τ τ τ A and B do no appea P Q L RL d P Conol gan pdae L R B P B needed fo conol pdae Inal sablzng conol s needed

21 A B B P P P A B B P P B B P Q 0 P P R c R c P 0 P heoem Lemma s eqvalen o τ τ τ P Q L RL d P L R B P Solves Lyapnov eqaon who nowng A o B A BR B P P P A BR B P P BR B P Q 0 Poof: d d P A P P A K RK Q Q K RK dτ d P P P D. abe hs algohm conveges and s eqvalen o Klenman s Algohm Only B s needed

22 Algohm Implemenaon Cc pdae τ τ τ P Q L RL d P Use Konece podc vec ABC C A vec B o se hs p as s he qadac bass se p τ Q K RK τ dτ p p ϕ p [ ] τ Q K RK τ dτ ρ Renfocemen on me neval [ ] Qadac egesson veco Now se RLS along he ajecoy o ge new weghs p Unpac weghs no he ma P hen fnd pdaed FB gan L R B P

23 . Selec nal conol polcy 2. Fnd assocaed cos Solves Lyapnov eq. who nowng dynamcs [ ] p τ Q L RL τ dτ ρ 3. Impove conol obseve apply L obseve cos negal L obseve R B P pdae P Mease cos ncemen enfocemen by addng as a sae. hen Q R A s no needed anywhee do RLS nl convegence o P pdae conol gan o L

24 Algohm Implemenaon he Cc pdae τ τ τ P Q L RL d P can be sep as O se bach Leas-Sqaes solon along he ajecoy ϕ [ ] τ τ τ p p Q L RL d d L s he qadac bass se Evalang d L fo nn/2 ajecoy pons one can sep a leas sqaes poblem o solve 2 N X [ ϕ ϕ... ϕ ] 2 N Y d K d K d K p XX XY [... ]

25 Dagna abe Dec Opmal Adapve Conolle Aco Sysem A B; 0 mlple L ZOH Q R FB Gan L Cc Dynamc Conol Sysem A hybd connos/dscee dynamc conolle whose nenal sae s he obseved cos ove he neval

26 Gan pdae Polcy L Conol L Sample peods need no be he same hey can be seleced on-lne n eal me Connos-me conol wh dscee gan pdaes

27 Dagna abe 2. C ADP Geedy eaon Cos pdae Q R d Conol polcy L LQR τ τ τ P Q L RL d P Conol gan pdae L R B P A and B do no appea B needed fo conol pdae No nal sablzng conol needed Dec Opmal Adapve Conol fo Paally Unnown C Sysems

28 Algohm Implemenaon he cc pdae τ τ τ P Q L RL d P Use Konece podc o se hs p as vec ABC C A vec B s he qadac bass se p τ Q L RL τ dτ p Regesson veco Pevos weghs Now se RLS along he ajecoy o ge new weghs p Unpac weghs no he ma P hen fnd pdaed FB gan L R B P

29 . Selec conol polcy Solves fo cos pdae who nowng dynamcs 2. Fnd assocaed cos p τ Q L RL τ dτ p 3. Impove conol obseve apply obseve cos negal obseve L R B P pdae P Mease cos ncemen by addng as a sae. hen Q R No nal sablzng conol needed do RLS nl convegence o P pdae conol gan o L A s no needed anywhee

30 Dagna abe Dec Opmal Adapve Conolle A hybd connos/dscee dynamc conolle whose nenal sae s he obseved vale ove he neval Aco Sysem A B; 0 L ZOH Q R FB Gan L Cc Dynamc Conol Sysem Has a dffeen cc cos pdae No nal sablzng gan needed

31 Dagna abe Analyss of he algohm Fo a gven conol polcy L wh L R B P A A BR B P Geedy pdae { } Q R dτ 0 0 s eqvalen o A A A A P e Q L RL e d e Pe a sange psedo-dscezed RE c.f. D RE P A P A Q A P B P A P A Q L P B PK B B P A P B P K B L

32 Dagna abe Analyss of he algohm Lemma 2. C HDP s eqvalen o P A P e P A AP Q L RL e 0 A hs ea em means he nal Conol acon need no be sablzng d A A BR B P When ADP conveges he eslng P sasfes he Connos-me ARE!! ADP solves he C ARE who nowledge of he sysem dynamcs A

33 Solve he Rcca Eqaon WIHOU nowng he plan dynamcs Model-fee ADP Dec OPIMAL ADAPIE CONROL Wos fo Nonlnea Sysems a neal newo s sed o appomae he cos Robsness? Compason wh adapve conol mehods?

34 Polcy Evalaon Cc pdae Le K be any sae feedbac gan fo he sysem. One can mease he assocaed cos ove he nfne me hozon τ Q K RK τ dτ W whee W s an nal nfne hozon cos o go. Wha o do abo he al sses n Recedng Hozon Conol

35 Gan pdae Polcy L Conol L Sample peods need no be he same Connos-me conol wh dscee gan pdaes

36 Smlaons on: F-6 aoplo Load feqency conol fo powe sysem A ma no needed 0 Conol sgnal Sysem saes me s Conolle paamees me s me s Cc paamees P P2 P22 P - opmal P2 - opmal P22 - opmal Convege o SS Rcca eqaon soln me s

37 Nonlnea Case Wos oo f g d Q R d Polcy Ieaon Q R d h R g f no needed Pove hs s eqvalen o: 0 f h h f needed 2 h R g whch s nown o convege

38 Q R d Appomae vale by Neal Newo W φ Algohm Implemenaon W φ Q R d W φ φ φ [ ] W Q R d egesson veco Renfocemen on me neval [ ] Now se RLS along he ajecoy o ge new weghs W hen fnd pdaed FB h R g φ R g W 2 2

39

40 f d R Q d 0 H f 0 0 mn mn 0 * * f g R h * 2 * d d g gr d d Q f d d * * 4 * Sysem Cos Hamlonan Opmal cos Opmal conol HJB eqaon Connos-me Opmal Conol Bellman mn mn 0 f h h h γ c.f. D vale ecson whee f g do no appea