Distributed and Real-time Predictive Control

Transcription

1 Distributed and Real-time Predictive Control Melanie Zeilinger Christian Conte (ETH) Alexander Domahidi (ETH) Ye Pu (EPFL) Colin Jones (EPFL)

2 Challenges in modern control systems Power system: - Frequency control - Voltage control Customer: - Control of building networks - Control of flexible loads and storage capacities i4energy seminar: 12pm, 310 SDH "The Role of Supply-Following Loads in Highly Renewable Electricity Grids, Jay Taneja Courtesy of Large-scale, complex system Constraints Uncertainties High performance and safety Composed of coupled subsystems Often high-speed dynamics Computation and communication constraints 2

3 Challenges in modern control systems Traffic network Power network Courte sy of Courtesy of Dr. Pu Wang Robotics Courtesy of IDSC, ETH Large-scale, complex system Constraints Uncertainties High performance and safety Composed of coupled subsystems Often high-speed dynamics Computation and communication constraints 3

4 Model Predictive Control (MPC) " A High Performance Method for Constrained Control N 1 u (x) := argmin V f (x N )+ l(x k,u k ) k=0 x 0 = x x k+1 = f (x k,u k ) (x k,u k ) X U x N X f state x system output y x 1 Each sample time: 1. Measure / estimate state 2. Solve optimization problem for entire planning window 3. Implement only the first control action x 0 = x X f x5 x4 x 2 x 3 4

5 Model Predictive Control (MPC) " A High Performance Method for Constrained Control N 1 u (x) := argmin V f (x N )+ l(x k,u k ) k=0 x 0 = x x k+1 = f (x k,u k ) (x k,u k ) X U x N X f state x system output y Classical MPC theory: J High performance J Recursive constraint satisfaction J Stability by design Established approach: Optimality Terminal cost and constraint 5

6 Real-time Model Predictive Control u (x) := argmin V f (x N )+ Embedded processor N 1 k=0 l(x k,u k ) x 0 = x x k+1 = f (x k,u k ) (x k,u k ) X U x N X f state x system output y Classical MPC theory: J High performance J Recursive constraint satisfaction J Stability by design Bounded computation time à Early termination à Invalidates MPC theory based on optimality 6

7 Distributed Model Predictive Control contr 1 contr 3 contr M sys 1 sys 3 sys M sys 2 contr 2 Classical MPC theory: J High performance J Recursive constraint satisfaction J Stability by design Local computation and information: à Restrictive local terminal conditions à Stability in exchange for significant conservatism 7

8 Outline: Distributed and Real-time MPC Centralized MPC theory: J Recursive constraint satisfaction J Stability by design Established approach: Optimality Terminal cost and constraint Real-time MPC: Flexibility and fast convergence through interior-point methods BUT: Variable solve-times Outline (Part I): Stability and constraint satisfaction for any real-time constraint à MPC for fast, safety-critical systems Distributed MPC: Reduced conservatism through distributed optimization BUT: Global terminal conditions Outline (Part II): Stability with larger region of attraction based on local information à Plug and Play MPC 8

9 Outline: Distributed and Real-time MPC Centralized MPC theory: J Recursive constraint satisfaction J Stability by design Established approach: Optimality Terminal cost and constraint Real-time MPC: Flexibility and fast convergence through interior-point methods BUT: Variable solve-times Outline (Part I): Stability and constraint satisfaction for any real-time constraint à MPC for fast, safety-critical systems Distributed MPC: Reduced conservatism through distributed optimization BUT: Global terminal conditions Outline (Part II): Stability with larger region of attraction based on local information à Plug and Play MPC 9

10 Stability and Invariance of Optimal MPC V N (x) = min V N (x,u) :=V f (x N )+ N 1 i=0 x T i Qx i + u T i Ru i x 0 = x x i+1 = Ax i + Bu i Cx i + Du i b x N X f x 1 = x + Assumptions:1. X f X is invariant x 2 X f ) Ax + Bu f (x) 2 X f x 0 = x Ax 5 + Bu f (x 5 ) X f x 2 2. V f (x) is a Lyapunov function in X f V f (Ax + Bu f (x)) V f (x) apple l(x,u f (x)) x 5 x 4 x 3 Theorem: V N (x) is a convex Lyapunov function The feasible set is invariant under the optimal MPC controller 10

11 Stability and Invariance of Optimal MPC V N (x) = min V N (x,u) :=V f (x N )+ N 1 i=0 x T i x 0 = x Qx i + u T i Ru i x i+1 = Ax i + Bu i Cx i + Du i b x N X f x 1 = x + Assumptions:1. X f X is invariant x 2 X f ) Ax + Bu f (x) 2 X f x 0 = x Ax 5 + Bu f (x 5 ) X f x 2 2. V f (x) is a Lyapunov function in X f V f (Ax + Bu f (x)) V f (x) apple l(x,u f (x)) x 5 x 4 x 3 Proof: Shifted sequenceu =[u1,...,u N 1,Kx N ] is feasible à Recursive feasibility and invariance decreases the cost V N (x + ) V N (x) VN (x + ) V N (x) l(x,u 0 ) < 0 à V N (x) is a Lyapunov function 11

12 Real-Time MPC Controller Synthesis Large-scale, ms Interior point methods Modify controller to be robust to time constraints Medium-scale, us Gradient approaches Bound computation time a priori Small-scale, ns Pre-compute controller Fixed time online x Generic u x Deterministic u x optimization optimizer code Analytic expression for control law u Flexibility Speed Ideal approach is problem specific 12

13 Real-time MPC using interior-point methods Real-time online MPC: Guarantee that within the real-time constraint a feasible solution satisfying stability criteria for any admissible initial state is found. 13

14 Real-time MPC using interior-point methods Real-time online MPC: Guarantee that within the real-time constraint Early termination a feasible solution Warm-start satisfying stability criteria for any admissible initial state is found. x + x Suboptimal solution Warm-start Online optimization 14

15 Real-time MPC using interior-point methods Real-time online MPC: Guarantee that within the real-time constraint Early termination a feasible solution Warm-start satisfying stability criteria for any admissible initial state is found. Many recent codes have demonstrated that extreme speeds are possible OOQP Object-oriented software for quadratic programming CVXGEN Code Generation for Convex Optimization qpoases Online Active Set Strategy QPSchur A dual, active-set, Schurcomplement method for quadratic programming but cannot guarantee stability in a real-time setting! 15

16 Example: Effect of limited computation time Closed loop trajectory: Optimal control law Unstable example x + = x u x 1 5, 5 x 2 1 u 1,N =5,Q= I,R =1 x x 1 Closed loop trajectory: Optimization stopped after 4 iterations = max computation time of 21ms Computation time [s] 3 2 x Time Step Limited computation time => No stability properties 16

17 Example: Stability under proposed real-time method x Closed loop trajectory: Optimal control law Proposed real-time MPC method" stopped after 4 online iterations x 1 Closed loop trajectory: Optimization stopped after 4 iterations = max computation time of 21ms Computation time [s] 3 2 x 10-4 Unstable example x + = x u x 1 5, 5 x 2 1 u 1,N =5,Q= I,R = Time Step Real-time robust MPC : Nearly optimal and satisfies time constraints 17

18 Loss of stability guarantee in real-time Requirement for stability: Lyapunov function à Use of MPC cost as Lyapunov function à Key condition: Decrease of MPC cost at every time step Using interior-point methods this condition can be violated even when initializing with a stabilizing sequence, e.g. the shifted sequence Example: Barrier interior-point method Minimize augmented cost min z f (z) Fz = Ex Gz V N (x t, u t ) <V N (x t 1, u t 1 ) d min z f (z) µ Fz = Ex m i=1 log( G i z + d i ) z * z * (10) à Decrease in augmented cost does not enforce a decrease in MPC cost à Steady-state offset for μ 0 18

19 Real-time stability guarantees Goal: Ensure that suboptimal cost is Lyapunov function Introduce Lyapunov constraint : Enforces decrease in suboptimal MPC cost at each iteration Theorem: V N (xt, u t ) V N (x t 1, u t 1 ) ɛ x t 1 2 Q (Quadratic constraint) Suboptimal cost for any feasible solution to real-time problem provides Lyapunov function à Stability for any real-time constraint If We can provide (strictly) feasible solution for Lyapunov constraint in real-time Key: Ensure that epsilon progress is always possible without optimization à Technique based on warm-starting from previous sampling time! We can solve quadratically constrained QPs with modified structure 19

20 Computation times on Intel Atom for QP CVXGEN 10 FORCES Oscillating masses: QP with - box constraints - diagonal cost Time per" 10 3 iteration (s) μs Number of masses (Number of states/2) More details in [Domahidi, et al., ACC 2012]. forces.ethz.ch 20

21 Computation times on Intel Atom for QP 1 10 FORCES RT 2 CVXGEN 10 FORCES Oscillating masses: QP with - box constraints - diagonal cost Time per" 10 3 iteration (s) FORCES RT is stabilizing for " all numbers of iterations! 207 μs is obtainable Other methods ~10 iterations μs QCQP with - quadr. terminal set - real-time constr Number of masses (Number of states/2) More details in [Domahidi, et al., ACC 2012]. forces.ethz.ch 21

22 Summary: Real-Time MPC Real-time online MPC: Guarantee that within the real-time constraint Early termination a feasible solution Warm-start satisfying stability criteria Lyapunov constraint for any admissible initial state is found. Optimal MPC requires unknown computation time à Fast systems require theory of real-time MPC Real-time method provides stability guarantees for arbitrary time constraints Extension to robust tube-based MPC Extension to tracking (more involved) Possible to achieve millisecond solve-times on inexpensive hardware Real-time MPC still faster than solvers without guarantees [Zeilinger, et al., Automatica 2013, accepted], [Domahidi, et al., CDC 2012] 22

23 Outline: Distributed and Real-time MPC Centralized MPC theory: J Recursive constraint satisfaction J Stability by design Established approach: Optimality Terminal cost and constraint Real-time MPC: Flexibility and fast convergence through interior-point methods BUT: Variable solve-times Outline (Part I): Stability and constraint satisfaction for any real-time constraint Distributed MPC: Reduced conservatism through distributed optimization BUT: Global terminal conditions Outline (Part II): Stability with larger region of attraction based on local information 23

24 Distributed Model Predictive Control (MPC) Communication with neighbours N i Cooperative objective l(x,u)= M i=1 l i(x i,u i ) contr 1 contr 3 contr M sys 1 sys 3 sys M sys 2 contr 2 Coupled linear dynamics = M j=1 A ijx j + B i u i = A Ni x Ni + B i u i x + i Independent constraints x i X i u i U i How to ensure stability and constraint satisfaction without central coordination? 24

25 Distributed Model Predictive Control (MPC) Plug and Play MPC: Allow subsystems to join or leave the network contr 1 contr 3 contr M sys 1 sys 3 sys M sys 2 contr 2 sys 4 contr 4 Modified dynamics x + i = j N A i ij x j + B i u i How to maintain stability and constraint satisfaction during network changes? 25

26 Distributed Optimization Requires Structure min f i (y i ) y i Y i A i y i = c Distributed optimization requires that the problem is structured Example: Dual Decomposition g( ) = min f i (y i )+ T A i y i c = min f i (y i )+ y i Y i y i Y i T A i y i Gradient of the dual function: g( )= A i y i ( ) c Gradient-based approach + = + g( ) Many variants on this theme (ADMM, AMA,...) Optimal values y i* Local optimization" Dual update Consensus 26

27 Two Conflicting Requirements min V f (x N )+ s.t. x 0 = x NX 1 i=0 x i+1 = Ax i + Bu i (x i,u i ) 2 X U x N 2 X f l(x i,u i ) Plant A, B structured X, U distributed l(x,u) distributed 1 Stability and invariance if: 1. X f X is invariant x X f Ax + Bu f (x) X f Dense Dense 2. V f (x) is a Lyapunov function in X f V f (Ax + Bu f (x)) V f (x) l(x,u f (x)) 2 Structured optimization Terminal cost & constraints: X f = Xf 1 X f M V f (x) = Vf k (x Nk ) k=1 u f (x) =[u 1 f (x N1 ),...,u M f (x NM )] T Goal: Satisfy both requirements without central coordination à Online & offline optimization structured according to system coupling 27

28 Structured Lyapunov Function Lyapunov requirement: V f (x + ) V f (x) l(x,u f (x)) Structure requirement: V f (x) =Vf 1 (x 1 )+ + Vf M (x M ) Idea: Allow local increase while requiring a global decrease Theorem: [Jokic, Lazar, 2009] V f (x) := M i=1 V i f (x Ni ) is a Lyapunov function if V i f (x + i ) V i f (x i ) l i (x Ni )+ i (x Ni ) M i=1 i(x Ni ) 0 Possible local increase Global decrease J Global Lyapunov function à Stability 28

29 Structured Invariant Set Invariance requirement: x X f x + X f Structure requirement: X f ( )=Xf 1 ( 1 ) Xf M ( M ) Idea: Level sets of a Lyapunov function are invariant M X f = x V f (x) = Vf i (x Ni ) i=0 Problem: This terminal constraint couples all sub-systems Want a condition that can be tested in a distributed fashion X i f ( i )={x V i f (x Ni ) i} where M i=0 i = Xf i ( i) V f (x i ) i V f (x + i ) i V i f (x + i ) V i f (x i ) l i (x Ni,u i f (x Ni )) + i (x Ni ) 0 29

30 Structured Dynamic Invariant Set Invariance requirement: x X f x + X f Structure requirement: X f ( )=Xf 1 ( 1 ) Xf M ( M ) Define auxiliary dynamics, with the same structure as the system dynamics: + i = i + i (x Ni ) Choose initial i i x Vf i(x N i ) X Theorem: 1. Time-varying terminal set Xf i ( i )={x Vf i (x i ) i} is invariant 2. All state and input constraints are satisfied in X f ( ) Proof: From 1. Vf i (x + 2. X f ( ) X X f ( + ) X x i X i f ( i ) x + i X i f ( + i ) + i = i + i(x Ni ) i i ) Vf i (x i ) l i (x Ni,uf i (x Ni )) + i (x Ni ) i + i (x Ni )= + i 30

31 Structured Dynamic Invariant Set Invariance requirement: x X f x + X f Structure requirement: X f ( )=Xf 1 ( 1 ) Xf M ( M ) Define auxiliary dynamics, with the same structure as the system dynamics: + i = i + i (x Ni ) Choose initial i i x Vf i(x N i ) X Theorem: 1. Time-varying terminal set Xf i ( i )={x Vf i (x i ) i} is invariant x i X i f ( i ) x + i X i f ( + i ) 2. All state and input constraints are satisfied in X f ( ) J Recursive feasibility 31

32 Distributed MPC Online Control Structured MPC problem min s.t. M i=1 N 1 Vf i (x i (N)) + x i (0) = x i k=0 l(x i (k),u i (k)) x i (k + 1) = A ii x i (k)+b i u i (k)+ (x i (k),u i (k)) X i U i x i (N) X i f ( i ) j N i A ij x j (k) u i + i = i + (x Ni ) 32

33 Distributed MPC - Synthesis and Online Control Vi f (x i )=xi T P i x i i (x Ni )=x T N i ix Ni u f i (x N i )=K Ni x Ni u i + i = i + x T N i (N) Ni x Ni (N) No central coordination required! 33

34 Computational example Chain of inverted pendulums (unstable) Linearized around the origin States: Angle and angular velocity of each pendulum Inputs: Torque at each pivot 34

35 Computational example Closed-Loop Simulation Angle i Input T i Simulation step Pend.1, centr. MPC (cost 9.021) Pend.2 centr. MPC (cost 9.021) Pend.1, distr. MPC (cost 9.360) Pend.2, distr. MPC (cost 9.360) Pend. 1, distr. MPC X f =0 (cost ) Pend. 2, distr. MPC X f =0 (cost ) Simulation step 5 Pendulums, alternating direction method of multipliers, 100 iterations. Initially all pendulums in origin, only pendulum 1 is deflected. Cost of proposed method only 4% higher than centralized MPC and 21% lower than for a trivial terminal set. 35

36 Computational example Local Terminal Sets Sizes of local terminal sets change dynamically 36

37 Computational example Region of Attraction φ 1,max MPC prediction horizon Centr. MPC Distr. MPC Distr. MPC X f = 0 Maximum feasible deflection of the first pendulum vs. prediction horizons Short prediction horizons: Region of attraction for proposed method significantly larger than for trivial terminal set Long prediction horizons: All methods converge to the same maximum control invariant set 37

38 Distributed Model Predictive Control (MPC) Plug and Play MPC: Allow subsystems to join or leave the network contr 1 contr 3 contr M sys 1 sys 3 sys M sys 2 contr 2 sys 4 contr 4 Modified dynamics x + i = j N A ij x j + B i u i i C R Maintain stability and recursive feasibility during network changes: Adapt local control laws of subsystems and neighbours Ensure feasibility of the modified control laws 38

39 Preparation for Plug and Play operation: Plug and Play operation = subsystems join/leave network and modified local control law is applied contr 1 sys 1 sys 2 contr 2 contr 3 sys 3 sys 4 contr 4 contr M sys M Redesign Phase: Adapt local control laws of subsystems and neighbours Compute new local terminal costs Ṽf i (x i ) and constraint sets X f i ( i ) for (virtually) modified network Transition Phase: Ensure feasibility of the modified MPC problem Compute a steady-state for Plug and Play operation such that: - Steady-state is a feasible initial state for the modified MPC problem - System can be controlled to the steady-state from the current state If steady-state can be found - Control system to steady-state - Permit plug and play operation Else - Reject plug and play operation 39

40 Preparation for Plug and Play operation: Plug and Play operation = subsystems join/leave network and modified local control law is applied contr 1 sys 1 sys 2 contr 2 contr 3 sys 3 sys 4 contr 4 contr M sys M Redesign Phase: Adapt local control laws of subsystems and neighbours Compute new local terminal costs Ṽf i (x i ) and constraint sets X f i ( i ) for (virtually) modified network Transition Phase: Ensure feasibility of the modified MPC problem Compute a steady-state for Plug and Play operation such that: - Steady-state is a feasible initial state for the modified MPC problem - System can be controlled to the steady-state from the current state If steady-state can be found - Control system to steady-state - Permit plug and play operation Else - Reject plug and play operation Plug and and play synthesis and control via distributed optimization 40

41 Computational example Area Generation Control Four power generation areas with load frequency control Model linearized around equilibrium (Saadat, 2002; Riverso, et al. 2012) ż i = A ij z i + B i v i + L i P Li 1" j N i Goals: - Restore frequency, follow load change P L1 = 0.15, P L3 = Allow fifth area to join the network 2" 3" 5" 4" x [1] 2 x [5] Area t [s] x 10 3 Area 3 x [3] t [s] Area t [s] x [2] 2 x [4] x 10 3 Area t [s] x 10 3 Area t [s] Frequency deviation is controlled to zero x [1] 3 u [1] t [s] t [s] System is first regulated to steady-state and then to the origin 41

42 Computational example Area Generation Control Four power generation areas with load frequency control Model linearized around equilibrium (Saadat, 2002; Riverso, et al. 2012) ż i = A ij z i + B i v i + L i P Li j N i Goals: - Restore frequency, follow load change P L1 = 0.15, P L3 = Allow fifth area to join the network 1" 2" 3" 5" 4" α [i] 5 x α [1] α [2] α [3] α [4] α [5] t [s] Terminal set sizes change dynamically 42

43 Summary Distributed MPC Structured Lyapunov functions and dynamic invariant sets guarantee stability and invariance by design Synthesis and control via distributed optimization [Conte, et al., ACC 2012], [Conte, et al., CDC 2012] Extension to Robust Tube-based MPC and Tracking MPC [Conte, et al., ECC 2013, Conte, et al., CDC 2013, submitted] Plug and Play MPC enables network changes during closed-loop operation [Zeilinger, et al., CDC 2013, submitted] 43

44 Distributed and Real-time MPC Centralized MPC theory: J Recursive constraint satisfaction J Stability by design Established approach: Optimality Terminal cost and constraint Real-time MPC: Flexibility and fast convergence through interior-point methods BUT: Variable solve-times Outline (Part I): Stability and constraint satisfaction for any real-time constraint Distributed MPC: Reduced conservatism through distributed optimization BUT: Global terminal conditions Outline (Part II): Stability with larger region of attraction based on local information 44