Distributed and Real-time Predictive Control

Distributed and Real-time Predictive Control Melanie Zeilinger Christian Conte (ETH) Alexander Domahidi (ETH) Ye Pu (EPFL) Colin Jones (EPFL)

Challenges in modern control systems Power system: - Frequency control - Voltage control Customer: - Control of building networks - Control of flexible loads and storage capacities i4energy seminar: 12pm, 310 SDH "The Role of Supply-Following Loads in Highly Renewable Electricity Grids, Jay Taneja Courtesy of Large-scale, complex system Constraints Uncertainties High performance and safety Composed of coupled subsystems Often high-speed dynamics Computation and communication constraints 2

Challenges in modern control systems Traffic network Power network Courte sy of Courtesy of Dr. Pu Wang Robotics Courtesy of IDSC, ETH Large-scale, complex system Constraints Uncertainties High performance and safety Composed of coupled subsystems Often high-speed dynamics Computation and communication constraints 3

Model Predictive Control (MPC) " A High Performance Method for Constrained Control N 1 u (x) := argmin V f (x N )+ l(x k,u k ) k=0 x 0 = x x k+1 = f (x k,u k ) (x k,u k ) X U x N X f state x system output y x 1 Each sample time: 1. Measure / estimate state 2. Solve optimization problem for entire planning window 3. Implement only the first control action x 0 = x X f x5 x4 x 2 x 3 4

Model Predictive Control (MPC) " A High Performance Method for Constrained Control N 1 u (x) := argmin V f (x N )+ l(x k,u k ) k=0 x 0 = x x k+1 = f (x k,u k ) (x k,u k ) X U x N X f state x system output y Classical MPC theory: J High performance J Recursive constraint satisfaction J Stability by design Established approach: Optimality Terminal cost and constraint 5

Real-time Model Predictive Control u (x) := argmin V f (x N )+ Embedded processor N 1 k=0 l(x k,u k ) x 0 = x x k+1 = f (x k,u k ) (x k,u k ) X U x N X f state x system output y Classical MPC theory: J High performance J Recursive constraint satisfaction J Stability by design Bounded computation time à Early termination à Invalidates MPC theory based on optimality 6

Distributed Model Predictive Control contr 1 contr 3 contr M sys 1 sys 3 sys M sys 2 contr 2 Classical MPC theory: J High performance J Recursive constraint satisfaction J Stability by design Local computation and information: à Restrictive local terminal conditions à Stability in exchange for significant conservatism 7

Outline: Distributed and Real-time MPC Centralized MPC theory: J Recursive constraint satisfaction J Stability by design Established approach: Optimality Terminal cost and constraint Real-time MPC: Flexibility and fast convergence through interior-point methods BUT: Variable solve-times Outline (Part I): Stability and constraint satisfaction for any real-time constraint à MPC for fast, safety-critical systems Distributed MPC: Reduced conservatism through distributed optimization BUT: Global terminal conditions Outline (Part II): Stability with larger region of attraction based on local information à Plug and Play MPC 8

Stability and Invariance of Optimal MPC V N (x) = min V N (x,u) :=V f (x N )+ N 1 i=0 x T i Qx i + u T i Ru i x 0 = x x i+1 = Ax i + Bu i Cx i + Du i b x N X f x 1 = x + Assumptions:1. X f X is invariant x 2 X f ) Ax + Bu f (x) 2 X f x 0 = x Ax 5 + Bu f (x 5 ) X f x 2 2. V f (x) is a Lyapunov function in X f V f (Ax + Bu f (x)) V f (x) apple l(x,u f (x)) x 5 x 4 x 3 Theorem: V N (x) is a convex Lyapunov function The feasible set is invariant under the optimal MPC controller 10

Stability and Invariance of Optimal MPC V N (x) = min V N (x,u) :=V f (x N )+ N 1 i=0 x T i x 0 = x Qx i + u T i Ru i x i+1 = Ax i + Bu i Cx i + Du i b x N X f x 1 = x + Assumptions:1. X f X is invariant x 2 X f ) Ax + Bu f (x) 2 X f x 0 = x Ax 5 + Bu f (x 5 ) X f x 2 2. V f (x) is a Lyapunov function in X f V f (Ax + Bu f (x)) V f (x) apple l(x,u f (x)) x 5 x 4 x 3 Proof: Shifted sequenceu =[u1,...,u N 1,Kx N ] is feasible à Recursive feasibility and invariance decreases the cost V N (x + ) V N (x) VN (x + ) V N (x) l(x,u 0 ) < 0 à V N (x) is a Lyapunov function 11

Real-Time MPC Controller Synthesis Large-scale, ms Interior point methods Modify controller to be robust to time constraints Medium-scale, us Gradient approaches Bound computation time a priori Small-scale, ns Pre-compute controller Fixed time online x Generic u x Deterministic u x optimization optimizer code Analytic expression for control law u Flexibility Speed Ideal approach is problem specific 12

Real-time MPC using interior-point methods Real-time online MPC: Guarantee that within the real-time constraint a feasible solution satisfying stability criteria for any admissible initial state is found. 13

Real-time MPC using interior-point methods Real-time online MPC: Guarantee that within the real-time constraint Early termination a feasible solution Warm-start satisfying stability criteria for any admissible initial state is found. Many recent codes have demonstrated that extreme speeds are possible OOQP Object-oriented software for quadratic programming CVXGEN Code Generation for Convex Optimization qpoases Online Active Set Strategy QPSchur A dual, active-set, Schurcomplement method for quadratic programming but cannot guarantee stability in a real-time setting! 15

Example: Effect of limited computation time 2 1.5 Closed loop trajectory: Optimal control law Unstable example x + = 1.2 1 0 1 x + 1 0.5 u x 1 5, 5 x 2 1 u 1,N =5,Q= I,R =1 x2 1 0.5 0-0.5-4 -3-2 -1 0 x 1 Closed loop trajectory: Optimization stopped after 4 iterations = max computation time of 21ms Computation time [s] 3 2 x 10-4 1 0 5 10 15 20 25 Time Step Limited computation time => No stability properties 16

Example: Stability under proposed real-time method x2 2 1.5 1 0.5 0-0.5 Closed loop trajectory: Optimal control law -4-3 -2-1 0 Proposed real-time MPC method" stopped after 4 online iterations x 1 Closed loop trajectory: Optimization stopped after 4 iterations = max computation time of 21ms Computation time [s] 3 2 x 10-4 Unstable example x + = 1.2 1 0 1 x + 1 0.5 u x 1 5, 5 x 2 1 u 1,N =5,Q= I,R =1 1 0 5 10 15 20 25 Time Step Real-time robust MPC : Nearly optimal and satisfies time constraints 17

Loss of stability guarantee in real-time Requirement for stability: Lyapunov function à Use of MPC cost as Lyapunov function à Key condition: Decrease of MPC cost at every time step Using interior-point methods this condition can be violated even when initializing with a stabilizing sequence, e.g. the shifted sequence Example: Barrier interior-point method Minimize augmented cost min z f (z) Fz = Ex Gz V N (x t, u t ) <V N (x t 1, u t 1 ) d min z f (z) µ Fz = Ex m i=1 log( G i z + d i ) z * z * (10) à Decrease in augmented cost does not enforce a decrease in MPC cost à Steady-state offset for μ 0 18

Real-time stability guarantees Goal: Ensure that suboptimal cost is Lyapunov function Introduce Lyapunov constraint : Enforces decrease in suboptimal MPC cost at each iteration Theorem: V N (xt, u t ) V N (x t 1, u t 1 ) ɛ x t 1 2 Q (Quadratic constraint) Suboptimal cost for any feasible solution to real-time problem provides Lyapunov function à Stability for any real-time constraint If We can provide (strictly) feasible solution for Lyapunov constraint in real-time Key: Ensure that epsilon progress is always possible without optimization à Technique based on warm-starting from previous sampling time! We can solve quadratically constrained QPs with modified structure 19

Computation times on Intel Atom for QP 1 10 2 CVXGEN 10 FORCES Oscillating masses: QP with - box constraints - diagonal cost Time per" 10 3 iteration (s) 4 10 77 μs 5 10 0 5 10 15 20 25 30 Number of masses (Number of states/2) More details in [Domahidi, et al., ACC 2012]. forces.ethz.ch 20

Computation times on Intel Atom for QP 1 10 FORCES RT 2 CVXGEN 10 FORCES Oscillating masses: QP with - box constraints - diagonal cost Time per" 10 3 iteration (s) FORCES RT is stabilizing for " all numbers of iterations! 207 μs is obtainable Other methods ~10 iterations 4 10 207 μs QCQP with - quadr. terminal set - real-time constr. 5 10 0 5 10 15 20 25 30 Number of masses (Number of states/2) More details in [Domahidi, et al., ACC 2012]. forces.ethz.ch 21

Summary: Real-Time MPC Real-time online MPC: Guarantee that within the real-time constraint Early termination a feasible solution Warm-start satisfying stability criteria Lyapunov constraint for any admissible initial state is found. Optimal MPC requires unknown computation time à Fast systems require theory of real-time MPC Real-time method provides stability guarantees for arbitrary time constraints Extension to robust tube-based MPC Extension to tracking (more involved) Possible to achieve millisecond solve-times on inexpensive hardware Real-time MPC still faster than solvers without guarantees [Zeilinger, et al., Automatica 2013, accepted], [Domahidi, et al., CDC 2012] 22

Outline: Distributed and Real-time MPC Centralized MPC theory: J Recursive constraint satisfaction J Stability by design Established approach: Optimality Terminal cost and constraint Real-time MPC: Flexibility and fast convergence through interior-point methods BUT: Variable solve-times Outline (Part I): Stability and constraint satisfaction for any real-time constraint Distributed MPC: Reduced conservatism through distributed optimization BUT: Global terminal conditions Outline (Part II): Stability with larger region of attraction based on local information 23

Distributed Model Predictive Control (MPC) Communication with neighbours N i Cooperative objective l(x,u)= M i=1 l i(x i,u i ) contr 1 contr 3 contr M sys 1 sys 3 sys M sys 2 contr 2 Coupled linear dynamics = M j=1 A ijx j + B i u i = A Ni x Ni + B i u i x + i Independent constraints x i X i u i U i How to ensure stability and constraint satisfaction without central coordination? 24

Distributed Model Predictive Control (MPC) Plug and Play MPC: Allow subsystems to join or leave the network contr 1 contr 3 contr M sys 1 sys 3 sys M sys 2 contr 2 sys 4 contr 4 Modified dynamics x + i = j N A i ij x j + B i u i How to maintain stability and constraint satisfaction during network changes? 25

Distributed Optimization Requires Structure min f i (y i ) y i Y i A i y i = c Distributed optimization requires that the problem is structured Example: Dual Decomposition g( ) = min f i (y i )+ T A i y i c = min f i (y i )+ y i Y i y i Y i T A i y i Gradient of the dual function: g( )= A i y i ( ) c Gradient-based approach + = + g( ) Many variants on this theme (ADMM, AMA,...) Optimal values y i* Local optimization" Dual update Consensus 26

Two Conflicting Requirements min V f (x N )+ s.t. x 0 = x NX 1 i=0 x i+1 = Ax i + Bu i (x i,u i ) 2 X U x N 2 X f l(x i,u i ) Plant A, B structured X, U distributed l(x,u) distributed 1 Stability and invariance if: 1. X f X is invariant x X f Ax + Bu f (x) X f Dense Dense 2. V f (x) is a Lyapunov function in X f V f (Ax + Bu f (x)) V f (x) l(x,u f (x)) 2 Structured optimization Terminal cost & constraints: X f = Xf 1 X f M V f (x) = Vf k (x Nk ) k=1 u f (x) =[u 1 f (x N1 ),...,u M f (x NM )] T Goal: Satisfy both requirements without central coordination à Online & offline optimization structured according to system coupling 27

Structured Lyapunov Function Lyapunov requirement: V f (x + ) V f (x) l(x,u f (x)) Structure requirement: V f (x) =Vf 1 (x 1 )+ + Vf M (x M ) Idea: Allow local increase while requiring a global decrease Theorem: [Jokic, Lazar, 2009] V f (x) := M i=1 V i f (x Ni ) is a Lyapunov function if V i f (x + i ) V i f (x i ) l i (x Ni )+ i (x Ni ) M i=1 i(x Ni ) 0 Possible local increase Global decrease J Global Lyapunov function à Stability 28

Structured Invariant Set Invariance requirement: x X f x + X f Structure requirement: X f ( )=Xf 1 ( 1 ) Xf M ( M ) Idea: Level sets of a Lyapunov function are invariant M X f = x V f (x) = Vf i (x Ni ) i=0 Problem: This terminal constraint couples all sub-systems Want a condition that can be tested in a distributed fashion X i f ( i )={x V i f (x Ni ) i} where M i=0 i = Xf i ( i) V f (x i ) i V f (x + i ) i V i f (x + i ) V i f (x i ) l i (x Ni,u i f (x Ni )) + i (x Ni ) 0 29

Structured Dynamic Invariant Set Invariance requirement: x X f x + X f Structure requirement: X f ( )=Xf 1 ( 1 ) Xf M ( M ) Define auxiliary dynamics, with the same structure as the system dynamics: + i = i + i (x Ni ) Choose initial i i x Vf i(x N i ) X Theorem: 1. Time-varying terminal set Xf i ( i )={x Vf i (x i ) i} is invariant 2. All state and input constraints are satisfied in X f ( ) Proof: From 1. Vf i (x + 2. X f ( ) X X f ( + ) X x i X i f ( i ) x + i X i f ( + i ) + i = i + i(x Ni ) i i ) Vf i (x i ) l i (x Ni,uf i (x Ni )) + i (x Ni ) i + i (x Ni )= + i 30

Structured Dynamic Invariant Set Invariance requirement: x X f x + X f Structure requirement: X f ( )=Xf 1 ( 1 ) Xf M ( M ) Define auxiliary dynamics, with the same structure as the system dynamics: + i = i + i (x Ni ) Choose initial i i x Vf i(x N i ) X Theorem: 1. Time-varying terminal set Xf i ( i )={x Vf i (x i ) i} is invariant x i X i f ( i ) x + i X i f ( + i ) 2. All state and input constraints are satisfied in X f ( ) J Recursive feasibility 31

Distributed MPC Online Control Structured MPC problem min s.t. M i=1 N 1 Vf i (x i (N)) + x i (0) = x i k=0 l(x i (k),u i (k)) x i (k + 1) = A ii x i (k)+b i u i (k)+ (x i (k),u i (k)) X i U i x i (N) X i f ( i ) j N i A ij x j (k) u i + i = i + (x Ni ) 32

Distributed MPC - Synthesis and Online Control Vi f (x i )=xi T P i x i i (x Ni )=x T N i ix Ni u f i (x N i )=K Ni x Ni u i + i = i + x T N i (N) Ni x Ni (N) No central coordination required! 33

Computational example Chain of inverted pendulums (unstable) Linearized around the origin States: Angle and angular velocity of each pendulum Inputs: Torque at each pivot 34

Computational example Closed-Loop Simulation 1.4 4 1.2 1 2 0.8 0 Angle i 0.6 0.4 Input T i 2 4 0.2 0 0.2 0.4 5 10 15 20 25 30 35 40 45 50 Simulation step 6 8 10 Pend.1, centr. MPC (cost 9.021) Pend.2 centr. MPC (cost 9.021) Pend.1, distr. MPC (cost 9.360) Pend.2, distr. MPC (cost 9.360) Pend. 1, distr. MPC X f =0 (cost 11.891) Pend. 2, distr. MPC X f =0 (cost 11.891) 5 10 15 20 25 30 35 40 45 50 Simulation step 5 Pendulums, alternating direction method of multipliers, 100 iterations. Initially all pendulums in origin, only pendulum 1 is deflected. Cost of proposed method only 4% higher than centralized MPC and 21% lower than for a trivial terminal set. 35

Computational example Local Terminal Sets Sizes of local terminal sets change dynamically 36

Computational example Region of Attraction 2.5 2 1.5 φ 1,max 1 0.5 0 5 10 15 20 25 30 MPC prediction horizon Centr. MPC Distr. MPC Distr. MPC X f = 0 Maximum feasible deflection of the first pendulum vs. prediction horizons Short prediction horizons: Region of attraction for proposed method significantly larger than for trivial terminal set Long prediction horizons: All methods converge to the same maximum control invariant set 37

Distributed Model Predictive Control (MPC) Plug and Play MPC: Allow subsystems to join or leave the network contr 1 contr 3 contr M sys 1 sys 3 sys M sys 2 contr 2 sys 4 contr 4 Modified dynamics x + i = j N A ij x j + B i u i i C R Maintain stability and recursive feasibility during network changes: Adapt local control laws of subsystems and neighbours Ensure feasibility of the modified control laws 38

Preparation for Plug and Play operation: Plug and Play operation = subsystems join/leave network and modified local control law is applied contr 1 sys 1 sys 2 contr 2 contr 3 sys 3 sys 4 contr 4 contr M sys M Redesign Phase: Adapt local control laws of subsystems and neighbours Compute new local terminal costs Ṽf i (x i ) and constraint sets X f i ( i ) for (virtually) modified network Transition Phase: Ensure feasibility of the modified MPC problem Compute a steady-state for Plug and Play operation such that: - Steady-state is a feasible initial state for the modified MPC problem - System can be controlled to the steady-state from the current state If steady-state can be found - Control system to steady-state - Permit plug and play operation Else - Reject plug and play operation 39

Computational example Area Generation Control Four power generation areas with load frequency control Model linearized around equilibrium (Saadat, 2002; Riverso, et al. 2012) ż i = A ij z i + B i v i + L i P Li 1" j N i Goals: - Restore frequency, follow load change P L1 = 0.15, P L3 =0.05 - Allow fifth area to join the network 2" 3" 5" 4" x [1] 2 x [5] 0.01 0 Area 1 0.01 0 20 40 60 80 t [s] x 10 3 Area 3 x [3] 2 2 5 0 5 0 20 40 60 80 t [s] Area 5 0.01 0 0.01 0 20 40 60 80 t [s] x [2] 2 x [4] 2 5 0 5 5 0 5 x 10 3 Area 2 0 20 40 60 80 t [s] x 10 3 Area 4 0 20 40 60 80 t [s] Frequency deviation is controlled to zero x [1] 3 u [1] 0.05 0 0.05 0.1 0.15 0.2 0 10 20 30 40 50 60 70 80 t [s] 0.1 0 0.1 0.2 0.3 0 10 20 30 40 50 60 70 80 t [s] System is first regulated to steady-state and then to the origin 41

Summary Distributed MPC Structured Lyapunov functions and dynamic invariant sets guarantee stability and invariance by design Synthesis and control via distributed optimization [Conte, et al., ACC 2012], [Conte, et al., CDC 2012] Extension to Robust Tube-based MPC and Tracking MPC [Conte, et al., ECC 2013, Conte, et al., CDC 2013, submitted] Plug and Play MPC enables network changes during closed-loop operation [Zeilinger, et al., CDC 2013, submitted] 43

Distributed and Real-time MPC Centralized MPC theory: J Recursive constraint satisfaction J Stability by design Established approach: Optimality Terminal cost and constraint Real-time MPC: Flexibility and fast convergence through interior-point methods BUT: Variable solve-times Outline (Part I): Stability and constraint satisfaction for any real-time constraint Distributed MPC: Reduced conservatism through distributed optimization BUT: Global terminal conditions Outline (Part II): Stability with larger region of attraction based on local information 44