Notes on Optimal Control

Size: px

Start display at page:

Download "Notes on Optimal Control"

Toby McDaniel
6 years ago
Views:

Alingon Noes on Opimal Conol Sppoed by : NSF - PAUL WERBOS ARO

1 F.L. Lewis Moncief-O Donnell Endowed Chai Head Conols & Sensos Gop Aomaion & Roboics Reseach Insie ARRI he Univesiy of eas a Alingon Noes on Opimal Conol Sppoed by : NSF - PAUL WERBOS ARO RANDY ZACHERY Dagna abie al available online a hp://arri.a.ed/acs

2 Michael K. Sain Blocing zeos Zeo synhesis modle heoy of mlivaiable zeos of dynamical sysems Eacness of maps SP nonlinea feedbac synhesis James L. Massey Michael K. Sain: Invese Poblems in Coding Aomaa and Coninos Sysems FOCS 967: Michael K. Sain: Minimal osion Spaces and he Paial Inp/Op Poblem Infomaion and Conol 292: M.K. Sain B.F. Wyman J.L. Peczowsi Eended zeos and model maching SIAM Jonal on Conol and Opimizaion May 99 Cheyl B. Schade and Michael K. Sain Zeo Pinciples fo Implici Feedbac Sysems Cicis Sysems and Signal Pocessing: Special Isse on Implici and Robs Sysems ol. 3 No. 2-3 pp Michael K. Sain and Cheyl B. Schade Feedbac Zeos and Blocing Dynamics in Recen Advances in Mahemaical heoy of Sysems Conol Newos and Signal Pocessing I. H. Kima and S. Kodama eds. oyo: Mia Pess pp Michael K. Sain and Cheyl B. Schade Bilinea Opeaos and Maices in Mahemaics fo Cicis and Files. Wai-Kai Chen ed. CRC Pess pp Ronald W. Diesing Michael K. Sain and Chang-Hee Won Bi-Cmlan Games: A genealizaion of H-inf and H2/H-inf Conol" IEEE ansacions on Aomaic Conol Sbmied 2007.

3 Modle heoeic zeo sces fo sysem maices Ahos: Wyman Boswic F.; Sain Michael K. Absac: he coodinae-fee modle-heoeic eamen of ansmission zeos fo MIMO ansfe fncions developed by Wyman and Sain 98 is genealized o inclde nonconollable and nonobsevable linea dynamical sysems.... NASA Cene: NASA non Cene Specific Pblicaion Yea: 987 Added o NRS: Accession Nmbe: 87A3090; Docmen ID:

4 Did I say ha? Well ha was hen. his is now. -M.K. Sain EIC edioial IEEE Cicis & Sysems magazine v.3 no. 2003

I pmps ions acoss he cell membane o mainain homeosais and has only

5 Opimaliy in Biological Sysems Cell Homeosasis he individal cell is a comple feedbac conol sysem. I pmps ions acoss he cell membane o mainain homeosais and has only limied enegy o do so. Pemeabiliy conol of he cell membane Cellla Meabolism hp://

6 Opimaliy in Conol Sysems Design R. Kalman 960 Roce Obi Injecion Dynamics w 2 v μ F w sinφ 2 m wv F v cosφ m m Fm Objecives Ge o obi in minimm ime Use minimm fel hp://micosa.sm.bms./e-libay/lanch/dnep_geo.pdf

7 Adapive Conol is No Opimal Opimal Conol is off-line and needs o now he sysem dynamics o solve design eqs. We wan ONLINE ADAPIE OPIMAL Conol

8 f d R Q d 0 H f 0 0 min min 0 * * f g R h * 2 * d d g gr d d Q f d d * * 4 * Sysem Cos Hamilonian Opimal cos Opimal conol HJB eqaion Coninos-ime Opimal Conol Bellman min min 0 f Fo a given conol he cos saisfies his eq. In LQR his is a Lyapnov eq In LQR his is a Riccai eq

9 Linea sysem qadaic cos Sysem: Uiliy: A B Q R; R > 0 Q 0 he cos is qadaic Opimal conol sae feedbac: R BP L dτ P HJB eqaion is he algebaic Riccai eqaion ARE: 0 PA A P Q PBR B P Fll sysem dynamics ms be nown

10 C Policy Ieaion o avoid solving HJB eqaion Uiliy Q R Cos fo any given 0 f H Lyapnov eqaion Ieaive solion Pic sabilizing iniial conol Find cos 0 f h h 0 0 Updae conol h 2 R g Convegence poved by Saidis 979 if Lyapnov eq. solved eacly Bead & Saidis sed complicaed Galein Inegals o solve Lyapnov eq. Ab Khalaf & Lewis sed NN o appo. fo nonlinea sysems and poved convegence Fll sysem dynamics ms be nown

11 LQR Policy ieaion Kleinman algoihm. Fo a given conol policy L solve fo he cos: 0 A A P P A C C 2. Impove policy: L A BL R B P L RL If saed wih a sabilizing conol policy L0 he mai P monoonically conveges o he niqe posiive definie solion of he Riccai eqaion. Evey ieaion sep will en a sabilizing conolle. he sysem has o be nown. Lyapnov eq. Kleinman 968

12 Policy Ieaion Solion Policy ieaion A BB P P P A BB P PBB P Q 0 i i i i i i his is in fac a Newon s Mehod Ric P A P PA Q PBB P hen Policy Ieaion is Pi Pi Ric P Ric Pi i 0 i Feche Deivaive Ric' P i P A BB P i P P A BB P i

13 Dagna abie Policy Ieaions wiho Lyapnov Eqaions Dynamic pogamming bil on Bellman s opimaliy pinciple alenaive fom fo C Sysems [Lewis & Symos 995] min d Δ Δ * * τ τ τ τ τ < Δ τ τ τ Q τ τ R τ f and g do no appea

14 Dagna abie Solving fo he cos O appoach Fo a given conol he cos saisfies L Q R d f and g do no appea LQR case P Q R d P Opimal gain is L R B P

15 Dagna abie Solving fo he cos O appoach. Policy ieaion Cos pdae Fo LQR case Q R d L τ τ τ A and B do no appea P Q L RL d P Conol gain pdae L R B P B needed fo conol pdae Iniial sabilizing conol is needed

16 A B B P i P i P i A B B P i P i B B P i Q 0 P P R ic R ic P i 0 i i Pi i Lemma is eqivalen o τ τ τ P Q L RL d P L R B P Solves Lyapnov eqaion wiho nowing A o B A BR B P P P A BR B P P BR B P Q 0 Poof: d d P i A i P i P A i i K i RK i Q Q K RK dτ d P P P heoem i i i i i his algoihm conveges and is eqivalen o Kleinman s Algoihm Only B is needed

17 Algoihm Implemenaion Ciic pdae τ τ τ P Q L RL d P Use Konece podc vec ABC C A vec B o se his p as is he qadaic basis se p τ Q Ki RKi τ dτ p p ϕ p [ ] τ Q Ki RKi τ dτ ρ Reinfocemen on ime ineval [ ] Qadaic egession veco Now se RLS along he ajecoy o ge new weighs p Unpac weighs ino he mai P hen find pdaed FB gain L R B P

18 . Selec iniial conol policy 2. Find associaed cos Solves Lyapnov eq. wiho nowing dynamics [ ] p τ Q L RL τ dτ ρ 3. Impove conol obseve apply L obseve cos inegal L obseve R B P pdae P Mease cos incemen einfocemen by adding as a sae. hen Q R A is no needed anywhee do RLS nil convegence o P pdae conol gain o L

19 Algoihm Implemenaion he Ciic pdae τ τ τ P Q L RL d P can be sep as O se bach Leas-Sqaes solion along he ajecoy i ϕ i [ ] τ τ τ p p Q L RL d d L is he qadaic basis se Evalaing d L fo nn/2 ajecoy poins one can sep a leas sqaes poblem o solve i 2 N X [ ϕ ϕ... ϕ ] 2 N Y d Ki d Ki d Ki p XX XY [... ]

20 Dagna abie Diec Opimal Adapive Conolle his is a vey weid Conol Sce whose lies I have no seen in conol sysem heoy Aco Sysem A B; 0 - sspicios qoe by F.L. Lewis 2007 mliplie L ZOH Q R FB Gain L Ciic Dynamic Conol Sysem A hybid coninos/discee dynamic conolle whose inenal sae is he obseved cos ove he ineval

21 Gain pdae Policy L Conol L Sample peiods need no be he same hey can be seleced on-line in eal ime Coninos-ime conol wih discee gain pdaes

22 Dagna abie 2. C ADP Geedy ieaion Cos pdae Q R d Conol policy L LQR τ τ τ P Q L RL d P Conol gain pdae L R B P A and B do no appea B needed fo conol pdae No iniial sabilizing conol needed Diec Opimal Adapive Conol fo Paially Unnown C Sysems

23 Algoihm Implemenaion he ciic pdae τ τ τ P Q L RL d P Use Konece podc o se his p as vec ABC C A vec B is he qadaic basis se p τ Q L RL τ dτ p Regession veco Pevios weighs Now se RLS along he ajecoy o ge new weighs p Unpac weighs ino he mai P hen find pdaed FB gain L R B P

24 . Selec conol policy Solves fo cos pdae wiho nowing dynamics 2. Find associaed cos p τ Q L RL τ dτ p 3. Impove conol obseve apply obseve cos inegal obseve L R B P pdae P Mease cos incemen by adding as a sae. hen Q R i i No iniial sabilizing conol needed do RLS nil convegence o P pdae conol gain o L A is no needed anywhee

25 Dagna abie Diec Opimal Adapive Conolle A hybid coninos/discee dynamic conolle whose inenal sae is he obseved vale ove he ineval Aco Sysem A B; 0 L ZOH Q R FB Gain L Ciic Dynamic Conol Sysem Has a diffeen ciic cos pdae No iniial sabilizing gain needed

26 Dagna abie Analysis of he algoihm Fo a given conol policy L wih L R B P A A BR B P Geedy pdae { } Q R dτ 0 0 is eqivalen o A A A A P e Q L RL e d e Pe a sange psedo-disceized RE c.f. D RE P A P A Q A P B P A P A Q L P B PK B B P A P B P K B L

27 Dagna abie Analysis of he algoihm Lemma 2. C HDP is eqivalen o P A P e P A AP Q L RL e 0 A his ea em means he iniial Conol acion need no be sabilizing d A A BR B P When ADP conveges he esling P saisfies he Coninos-ime ARE!! ADP solves he C ARE wiho nowledge of he sysem dynamics A

28 Solve he Riccai Eqaion WIHOU nowing he plan dynamics Model-fee ADP Diec OPIMAL ADAPIE CONROL Wos fo Nonlinea Sysems a neal newo is sed o appoimae he cos Robsness? Compaison wih adapive conol mehods?

29 Policy Evalaion Ciic pdae Le K be any sae feedbac gain fo he sysem. One can mease he associaed cos ove he infinie ime hoizon τ Q K RK τ dτ W whee W is an iniial infinie hoizon cos o go. Wha o do abo he ail isses in Receding Hoizon Conol

30 Gain pdae Policy L Conol L Sample peiods need no be he same Coninos-ime conol wih discee gain pdaes

31 Simlaions on: F-6 aopilo Load feqency conol fo powe sysem A mai no needed 0 Conol signal Sysem saes ime s Conolle paamees ime s ime s Ciic paamees P P2 P22 P - opimal P2 - opimal P22 - opimal Convege o SS Riccai eqaion soln ime s

33 f d R Q d 0 H f 0 0 min min 0 * * f g R h * 2 * d d g gr d d Q f d d * * 4 * Sysem Cos Hamilonian Opimal cos Opimal conol HJB eqaion Coninos-ime Opimal Conol Bellman min min 0 f h h h γ c.f. D vale ecsion whee f g do no appea

Adaptive Optimal Control F.L. Lewis Automation & Robotics Research Institute (ARRI) The University of Texas at Arlington

Adaptive Optimal Control F.L. Lewis Automation & Robotics Research Institute (ARRI) The University of Texas at Arlington Adapive Opimal Conol F.L. Lewis Aomaion & Roboics Reseach Insie ARRI he Univesiy of eas a Alingon F.L. Lewis Moncief-O Donnell Endowed Chai Head Conols & Sensos Gop Aomaion & Roboics Reseach Insie ARRI