Robust and Learning Control for Complex Systems

Transcription

1 Robus and Learning Conrol for Complex Sysems Peer M. Young Sepember 13, 2007 &

2 Talk Ouline Inroducion Robus Conroller Analysis and Design Theory Experimenal Applicaions Overview MIMO Robus HVAC Conrol Robus Learning Conrol Concluding Remarks

3 Single-Inpu, Single-Oupu (SISO) Loopshaping Conrol r + - e u y K P L = PK M = T = Y R = PK 1+ PK = L 1+ L S E R 1 1+ PK = = = 1 1+ L

4 Loopshaping Crierion L Genle Slope Near Crossover 0dB Low Frequency Gain Desired Bandwidh GM ω High Frequency Rolloff L PM -180 o ω

5 SISO Conrol Overview Nominal Sabiliy via Nyquis/Bode Injec Low Frequency Gain for Adequae Tracking and Disurbance Rejecion Adequae Bandwidh for Desired Speed Sabiliy Margins Gain Margin, Phase Margin High Frequency Rolloff All Achieved by Shaping Loop Gain

6 Muli-Inpu, Muli-Oupu (MIMO) Sysems u G y Coninuous Time: Discree-Time:

7 Quoe From Uncle Remus I ain wha you don know in life ha ges you ino rouble. I s wha you know for sure, bu wha ain so!

8 Robus Conroller Design Seup Δ Disurbances w P z Penaly Signals Conrol Signals u y Measuremens K - Generalized (Open-Loop) Plan: P - Robus Conroller: K - Perurbed Closed-Loop Map: T zw

9 μ Upper Bound as a Scaled H Problem Consan Marix: Dynamic Sysem: μ Synhesis:

10 D-K K Ieraion for μ-synhesis Δ P D P D -1 P K D K M Compue D: μ Problem Compue K: H Problem

11 Robus Conrol Theory Sysemaic Analysis and Design Theoreical and Compuaional Issues Complex MIMO Sysems Classical SISO Tools Inadequae Fundamenal Tradeoffs and Limiaions Robus o Uncerain Signals and Sysems Coninuous/Discree Time Various Uncerainy Types (Parameric/Dynamic) Opimal/High Performance

12 Applicaion Areas Power Disribuion Sysems (PSERC) Disk Drive Servo Conrol (Seagae/NSF) Human-Compuer Inerfaces (DARPA) Flexible Srucures (NSF) HVAC Conrol (Siemens/NSF)

13 Power Sysem Analysis High Qualiy Simulaion Models Modern Tools versus Tradiional Design and Analysis

14 Inerconneced Grid of Machines Key Parameers Fied as Polynomials Polynomials Wrien as Linear Fracional Transformaions See Earlier Work by Khammash, Vial, Zhu Large Scale Problems Compuaion Tools

15 Disk Drive Servo Conrol Seagae Technology as Indusrial Parner Single and Dual Sage Acuaion Sysems High Performance in Discree-Time Modern Robus Conrol Mehodologies Implemenaion Issues In Drive Tesing

16 Robus Conrol Servo Design

17 Measured Frequency Response

18 Measured Runou In Drive

19 Human-Compuer Inerface

20 Rapid Prooyping Environmen

21 Flexible Srucure Conrol

22 Conrol of HVAC Sysems Conrol Challenges for HVAC Sysems Complex Nonlinear Time-Varying Sysem Highly Uncerain Sysem Dynamics MIMO Ineracion of Conrolled Variables Currenly Conrolled by Muliple SISO PID Loops True MIMO Analysis and Design Improved Sysem Performance Coordinaed Acion for Independen Conrol MIMO Robus Sabiliy and Performance

23 Experimenal HVAC Plaform Simple HVAC Sysem Couner Flow Ho Waer o Air Hea-Exchanger Variable Air Volume Mixing Box Elecric Ho Waer Heaer Conrolled Variables: Discharge Air Temperaure Air Flow Rae

24 Experimenal HVAC Sysem exernal air T mixing box T filer heaing coil T air flow fan discharge air P E P E T T T T valve P E reurn air Inerface inpus T AE T AR T AI T WI F W T WS waer heaer T WO T AO F A variable frequency drive oupus C DE C DR C WH C VP C BS

25 PC/MATLAB Based Conrol Sysem

26 Model Verificaion Model of Experimenal Sysem Model and Experimenal Sysem Oupus

27 Conroller Design Basic Design Goals: MIMO Sabiliy and Robusness Independen Conrol of Key Sysem Variables Discharge Air Temperaure and Flow Rae Reference Convenional PI Conroller MIMO Robus Conrollers: Minimal (3x6) ~ T WS Exernally Conrolled Consrained (4x7) ~ T WS Inegraed Full (4x7) MIMO

28 PI Conroller Design Nonlinear Sysem Tune a High Gain Parameers Conrolled: Waer Supply Temperaure Air Flow Rae Inpu Air Temperaure Discharge Air Temperaure Heaing Coil capaciy vs.. waer flow rae

29 Reference PI Conroller

30 Conroller K Experimenal PI

31 Conroller K Implemenaion R3 4x7 Conroller Air flow (F a ) Inpu Air Temperaure (T ai ) Discharge Air Temperaure (T ao ao )

32 Conroller K Experimenal R3

33 Comparison of Conroller Performance

34 HVAC Conrol Summary Designed and Buil Experimenal HVAC Sysem Developed Model of HVAC sysem Designed MIMO Robus Conrollers Implemened MIMO Conrol Reduced T ao Sele Time by over 300% Decoupling of Conrolled Variables Simulaneous, Coordinaed Conrol Acion Conrollers are Insensiive o Model Uncerainy

35 Robus Learning Conrol Robus conrol heory Guaranees sabiliy Resuls in less aggressive conrollers Reinforcemen learning Opimizes he performance of a conroller No guaranee of sabiliy while learning

36 Reinforcemen Learning Agen in Parallel wih Robus Conroller reinforcemen = e

37 { } ), ( ), ( ), ( ), ( ), ( ), ( ), ( = = = + = + = T k k k k T k k k k a s Q a s R E a s R a s R E a s R a s R E a s Q π π π π π γ γ γ γ Reinforcemen Learning Theory Reinforcemen Learning Theory { } ), ( ), ( ), ( ), ( 1 1 a s Q a s Q a s R E a s Q π π π π γ + = Δ + + Subrac righ side from lef o ge algorihm for updaing Q Subrac righ side from lef o ge algorihm for updaing Q [ ] ), ( ), ( ), ( ), ( 1 1 a s Q a s Q a s R a s Q π π π γ α + = Δ + + Replace expecaion wih sample (Mone Carlo approach) Replace expecaion wih sample (Mone Carlo approach) Temporal Temporal-difference error difference error = = + + T k k k k a s R E a s Q 0 ), ( ), ( γ π π acion acion sae sae policy funcion policy funcion value value funcion funcion discoun facor discoun facor reinforcemen ( error ) reinforcemen ( error )

38 Reinforcemen Learning Implemenaion [ R( s, a ) + γq ( s, a ) Q ( s, a )] Δ Qπ ( s, a ) = α π π This converges on bes approximaion of value funcion for policy π. To also improve he policy: Q s, a ) = α R( s, a ) + γ min Q [ ] ( s, a ) Q ( s, a ) Δ π ( π + 1 π a A Q implicily defines he policy: π ( s ) = arg min Q( s, a) a A Value funcion (Q) learned by criic nework. (Q-learning, Wakins, 1989) Policy funcion (π)( ) learned by acor nework.

39 Neural Ne for Learning Agen W anh V Acor Nework (Criic Nework no shown) anh anh linear

40 Robus Conrol based on Inegral Quadraic Consrains An Inegral Quadraic Consrain (IQC) describes he relaionship p beween signals as vˆ( jw) wˆ( jw) * Π( jw) vˆ( jw) wˆ( jw) dw 0 v Uncerainies (D) Conroller/Plan (M) w Sabiliy of he closed loop sysem is guaraneed if M ( jw) I * Π( jw) M ( jw) I εi for all w and for ε > 0. Given specific IQCs for a paricular sysem, his inequaliy problem becomes a linear, marix inequaliy (LMI) problem.

41 IQCs Robusness Calculaion Two-layer neural ne as acor, in parallel wih conroller. Now wih anh and varying parameers covered by IQCs.

42 Robus Reinforcemen Learning Approach Reinforcemen learning algorihm guides adjusmen of acor s s weighs. IQC places bounding box in weigh space, beyond which sabiliy has no been verified. Sep 1 iniial guaraneedsable region Sep 4 nex guaraneed-sable region Sep 5 Now learning can coninue unil edge of new bounding box is encounered. UNSTABLE REGION! final weigh vecor Sep 0 iniial weigh vecor rajecory of weighs Sep 2 while learning mus find Sep 3 new sable region weigh rajecory wihou robus consrains weigh rajecory wih robus consrains weigh space (high-dimensional) weigh space (high-dimensional)

43 Disillaion Column Example of ask for which conrol variables inerac in complex way.

44 Decoupling Conroller Nominal Good response Perurbed Terrible response

45 Robus Conroller Nominal Less aggressive response Perurbed Much improved response

46 Robus Reinforcemen Learning Conroller Perurbed case, no learning (from previous slide) Perurbed case, wih learning Sum Squared Error Nominal Conroller Robus Conroller Robus RL Conroller Through learning, conroller has been fine-uned o acual dynamics of real plan wihou losing guaranee of sabiliy!

47 Reinforcemen Learning wihou IQCs Ulimaely achieves same good performance, bu during learning periods of insabiliy occur.

48 Concluding Remarks Complex MIMO Sysems (Coordinaed Conrol) Uncerain Signals and Sysems Opimal/High Performance (Efficiency/Mainenance) Robus Conrol Theory (Tradeoffs) Sysemaic Analysis and Design (Compuer Aided) Many Applicaion Areas Simulaion and Experimen Robus Learning Conrol Guaraneed Sabiliy Throughou he Online Learning/Adapaion Process

49 Fuure Research New Theoreical Tools for Robus/Learning Conroller Analysis and Design Improved Compuaion Tools (Malab) Furher Developmen of Exising Applicaion Areas New Applicaion Areas?