Modeling, Control, and Reachability Analysis of Discrete-Time Hybrid Systems

Transcription

1 Modeling, Control, and Reachability Analysis of Discrete-Time Hybrid Systems Alberto Bemporad DISC Course on Modeling and Control of Hybrid Systems March 31, 23 Università degli Studi di Siena (founded in 124) COHES Group Control and Optimization of Hybrid and Embedded SystemS Outline What is a hybrid system? Models for hybrid systems Control of hybrid systems Safety analysis of hybrid systems Examples (automotive)

2 Motivating Problem GOAL: command gear ratio, gas pedal, and brakes to track a desired speed and minimize consumptions CHALLENGES: continuous and discrete inputs dynamics depends on gear nonlinear torque/speed maps Hybrid Systems (Witsenhausen, 1966) Computer Science Control Theory X = {1, 2, 3, 4, 5} U = {A, B, C} Finite state machines Continuous dynamical systems x R n,u R m y R p B A Hybrid systems B C C B A C u(t) system y(t) ú dx(t) = f(x(t),u(t)) dt y(t) = g(x(t),u(t))

3 Motivation: Embedded Systems discrete inputs symbols continuous states automaton / logic interface continuous dynamical system symbols Consumer electronics Home appliances Office automation Automobiles Industrial plants... continuous inputs Motivation: Intrisically Hybrid Transmission Discrete command (R,N,1,2,3,4,5) + Continuous dynamical variables (velocities, torques) Four-stroke engines Automaton, dependent on power train motion

4 Key Requirements for Hybrid Models Descriptive enough to capture the behavior of the system continuous dynamics (physical laws) logic components (switches, automata, software code) interconnection between logic and dynamics Simple enough for solving analysis and synthesis problems x = Ax + Bu y = Cx + Du linear systems? linear hybrid systems x = f(t, x, u) y = g(t, x, u) nonlinear systems Piecewise Affine Systems (Sontag 1981) state+input space Approximates nonlinear dynamics arbitrarily well Suitable for stability analysis, reachability analysis (verification), controller synthesis,...

5 Discrete Hybrid Automata (Torrisi, Bemporad, 23) Switched Affine Systems Linear affine dynamics depends upon the mode i(k)

6 Event Generator Generates a logic signal according to the satisfaction of a linear affine constraint Finite State Machine Discrete dynamics evolving according to a Boolean state update function

7 Mode Selector a Boolean function selects the active mode i(k) of the SAS Logic and Inequalities Glover 1975, Williams 1977

8 Transformation of Logic into Linear Integer Inequalities. Given a Boolean statement F(X 1, X 2,..., X n ) 1. Convert to Conjunctive Normal Form (CNF): V ð m W ñ Wi P j X i i N j Xö i j=1 2. Transform into inequalities: Aî ô B, î {, 1} 1 ô X î i + X (1 à î i ) i P 1 i N ô X î i + X (1 à î i ) i P m i N m Every logic proposition can be translated into linear integer inequalities General Transformation into Linear Integer Inequalities Geometric approach: The polytope convex hull of the rows of the truth table P,{î : Aî ô B} T is the Aî ô B, î {, 1} n Every logic proposition can be translated into linear integer inequalities

9 Truth Tables Linear Integer Inequalities Example: logic AND 3 2 Key idea: White points cannot be in the hull of black points conv " #, " # 1, " # 1, " #! = î : à î 1 + î 3 ô à î 2 + î 3 ô î 1 + î 2 à î 3 ô 1 1 Convex hull algorithms: cdd, lrs, qhull, chd, Hull, Porto Mixed Logical Dynamical Systems DHA Mixed Logical Dynamical (MLD) Systems (Bemporad, Morari 1999) Continuous and binary variables Computationally oriented (Mixed Integer Programming) Suitable for optimal controller synthesis, verification,...

10 System: Associate A Simple Example x(t +1)= ú.8x(t) +u(t) if x(t) õ à.8x(t) +u(t) if x(t) < à 1 ô x(t) ô 1, à 1 ô u(t) ô 1 [î(t) =1] [x(t) õ ] and transform à mî(t) ô x(t) à m à (M + ï)î(t) ôàx(t) à ï M = à m =1 ï> small Thenx(t +1) =1.6î(t)x(t) à.8x(t) +u(t) z(t) =î(t)x(t) Rewrite as linear equation z(t) ô Mî(t) z(t) õ mî(t) z(t) ô x(t) à m(1 à î(t)) z(t) õ x(t) à M(1 à î(t)) x(t +1) =1.6z(t) à.8x(t) +u(t) HYSDEL (HYbrid Systems DEscription Language) Describe hybrid systems: Automata Logic Lin. Dynamics Interfaces Constraints Automatically generate MLD models in Matlab (Torrisi, Bemporad, 23) MLD model is not unique in terms of the number of auxiliary variables optimize model (minimize # binary variables!)

11 Example 1: AD section [ = T] [ ] s h h max SYSTEM tank { INTERFACE { STATE { REAL h [-.3,.3]; } INPUT { REAL Q [-1,1]; } PARAMETER { REAL k = 1; REAL e = 1e-6; } } /* end interface */ IMPLEMENTATION { AUX { BOOL s; } AD { s = hmax - h <= ; } CONTINUOUS { h = h + k * Q; } } /* end implementation */ } /* end system */ Example 2: DA section Nonlinear amplification unit u comp u ( u < ut ) = 2.3u 1.3 ut ( u ut) SYSTEM motor { INTERFACE { STATE { REAL ucomp; } INPUT { REAL u [,1]; } PARAMETER { REAL ut = 1; REAL e = 1e-6;} } /* end interface */ IMPLEMENTATION { AUX { REAL unl; BOOL th; } AD { th = ut - u <= ; } DA { unl = { IF th THEN 2.3*u - 1.3*ut ELSE u }; } CONTINUOUS { ucomp = unl; } } /* end implementation */ } /* end system */

12 Example 3: LOGIC section SYSTEM train { INTERFACE { STATE { BOOL brake; } INPUT { BOOL alarm, tunnel, fire; } } /* end interface */ u = u ( s s ) s brake alarm tunnel fire fire u alarm IMPLEMENTATION { AUX { BOOL decision; } LOGIC { decision = alarm & (~tunnel ~fire); } AUTOMATA { brake = decision; } MUST { fire -> alarm; } } /* end implementation */ } /* end system */ Example 4: CONTINUOUS section d i = C u() t dt Apply forward difference rule: T uk ( + 1) = uk ( ) + ik ( ) C SYSTEM capacitord { INTERFACE { STATE { REAL u; } PARAMETER { REAL R = 1e4; REAL C = 1e-4; REAL T = 1e-1; } } /* end interface */ IMPLEMENTATION { CONTINUOUS { u = u - T/C/R*i; } } /* end implementation */ } /* end system */

13 Example 5: AUTOMATA section SYSTEM outflow { INTERFACE { STATE { BOOL closing, stop, opening; } INPUT { BOOL uclose, uopen, ustop; } } /* end of interface */ IMPLEMENTATION { AUTOMATA { closing = (uclose & closing) (uclose & stop); stop = ustop (uopen & closing) (uclose & opening); opening = (uopen & stop) (uopen & opening); } MUST { ~(uclose & uopen); ~(uclose & ustop); ~(uopen & ustop); } } /* end implementation */ } /* end system */ Example 6: MUST section SYSTEM watertank { INTERFACE { STATE { REAL h; } INPUT { REAL Q; } PARAMETER { REAL hmax =.3; REAL k = 1; } } /* end interface */ IMPLEMENTATION { CONTINUOUS { h = h + k*q; } MUST { h - hmax <= ; -h <= ; } h h max } /* end implementation */ } /* end system */

14 Modeling Flow g 12x(t)+h 12u(t) <k 12 x(t +1)= A1x(t)+ B1u(t)+ f1 x(t +1)= A3x(t)+B3u(t)+f3 g 23x(t)+h 23u(t) <k 23 x(t +1)= A2x(t) + B2u(t) + f2 g 13x(t)+h 13u(t) <k 13 g 32x(t)+h 32u(t) <k 32 Designer HYSDEL Source Hysdel logic A/D D/A MLD PWA continuous Mixed Logical Dynamical form Piecewise Affine form Example: Bouncing Ball x ẍ =à g x ô xç(t + )=à (1 à ë)xç(t à ) F= m g N ë [, 1] How to model this system in MLD form?

15 HYSDEL - Bouncing Ball SYSTEM bouncing_ball { INTERFACE { /* Description of variables and constants */ STATE { REAL height [-1,1]; REAL velocity [-1,1]; } PARAMETER { REAL g=9.8; REAL dissipation=.4; /* =elastic, 1=completely anelastic */ REAL Ts=.5; } } IMPLEMENTATION { AUX { REAL z1; REAL z2; BOOL negative; } AD { negative = height <= ; } }} DA { z1 = { IF negative THEN height-ts*velocity ELSE height+ts*velocity-ts*ts*g}; z2 = { IF negative THEN -(1-dissipation)*velocity ELSE velocity-ts*g}; } CONTINUOUS { height = z1; velocity=z2;} System Theory for Hybrid Systems Analysis Well-posedness Realization & Transformation Stability Reachability (=Verification) Observability Synthesis Control State estimation Identification Modeling language

16 Well-posedness Are state and output trajectories defined? Uniquely defined? Persistently defined? MLD well-posedness : î(t) =F(x(t),u(t)) z(t) =G(x(t),u(t)) {x(t),u(t)} {x(t +1)} {x(t),u(t)} {y(t)} are single valued Numerical test based on mixed-integer programming available (Bemporad, Morari, Automatica, 1999) Realization and Transformation (State-Space Space Hybrid Models)

17 Other Existing Hybrid Models Linear complementarity (LC) systems (Heemels, 1999) x(t +1)=Ax(t)+B 1 u(t)+b 2 w(t) y(t) =Cx(t)+D 1 u(t)+d 2 w(t) v(t) =E 1 x(t)+e 2 u(t)+e 3 w(t)+e 4 ô v(t) w(t) õ Ex: mechanical systems circuits with diodes etc. Extended linear complementarity (ELC) systems Generalization of LC systems (De Schutter, De Moor, 2) Min-max-plus-scaling (MMPS) systems x(t +1)=M x (x(t),u(t),d(t)) y(t) =M y (x(t),u(t),d(t)) õ M c (x(t),u(t),d(t)) Example:x(t + 1) = 2 max(x(t), ) + min(à 2 1 u(t), 1) (De Schutter, Van den Boom, 2) MMPS function: defined by the grammar M := x i ë max(m 1,M 2 ) min(m 1,M 2 ) M 1 + M 2 ìm 1 Used for modeling discrete-event systems (t=event counter) Equivalences of Hybrid Models Σ 1 x (k), y (k) u(k) x ()=x () Σ 2 x (k), y (k)

18 Equivalence Results? DHA PWA???? LC?? MMPS? MLD? ELC (Heemels, De Schutter, Bemporad, Automatica, 21) (Torrisi, Bemporad, IEEE CST, 23) (Bemporad and Morari, Automatica, 1999) (Bemporad, Ferrari-T.,Morari, IEEETAC, 2) Theoretical properties and analysis/synthesis tools can be transferred from one class to another Efficient MLD to PWA Conversion Proof is constructive: given an MLD system it returns its equivalent PWA form Drawback: it needs the enumeration of all possible combinations of binary states, binary inputs, and δ variables Most of such combinations lead to empty regions Ω IDEA: use techniques from multiparametric programming to avoid such an enumeration

19 MLD to PWA Conversion Algorithm GOAL: split a given set B of states+inputs into polyhedral cells and find the affine dynamics in each cell, therefore determining the PWA system which is equivalent to the given MLD system u (Bemporad, 22) B x Note: the cells Ω i are embedded in R n+m by treating integer states and integer inputs as continuous vars: x b,u b [,1] x b,u b [-1/2,1/2) [1/2,3/2] Example: Heat and Cool Cold air flow Hot air flow Rules of the game: T amb #1 #2 T 1 T 2

20 Hybrid HYSDEL Model MLD model Hybrid MLD Model 2 continuous states: 1 continuous input: 2 auxiliary continuous vars: (temperatures T 1,T 2 ) (room temperature T amb ) (water flows u hot, u cold ) 6 auxiliary binary vars: 2 mixed-integer inequalities (4 thresholds + 2 for OR condition) Possible combination of integer variables: 2 6 = 64

21 Hybrid PWA Model PWA model 2 continuous states: (temperatures T 1,T 2 ) 1 continuous input: (room temperature T amb ) 5 polyhedral regions (partition does not depend on input) Temperature T 2 (deg C) Temperature 1 T(deg C) Computation time:.72 s in Matlab uhot = u cold = u hot = u cold = ŪC uhot = ŪH u cold = Controller Synthesis

22 Model Predictive Control of Hybrid Systems Controller Hybrid System Reference r(t) Input u(t) Output y(t) Measurements MODEL: a model of the plant is needed to predict the future behavior of the plant PREDICTIVE: optimization is based on the predicted future evolution of the plant CONTROL: control complex constrained multivariable plants At time t : Receding Horizon Philosophy Solve an optimal control problem over a finite future horizon p : minimize y à r + ú u subject to constraints t t+1 r(t) Predicted outputs Manipulated Inputs u(t+k) t+p u min ô u ô u max y min ô y ô y max Only apply the first optimal move t+1 t+2 t+p+1 u ã (t) Get new measurements, and repeat the optimization at time t +1 Advantage of on-line optimization: FEEDBACK!

23 Receding Horizon - Example MPC is like playing chess! MPC for Hybrid Systems past future t t+1 Predicted outputs Manipulated Inputs u(t+k) y(t+k t) t+t Model Predictive (MPC) Control At time t solve with respect to U,{u(t),u(t +1)...,u(t + T à 1)} the finite-horizon open-loop, optimal control problem: min U J(U, x(t),r), P k= Tà1 kq(y(t + k +1 t) à r)k + kr(u(t + k) à ur )k subj. to + û kî(t + k t) à î r k + kz(t + k t) à z r k + kx(t + k t) à x r k ( ) MLD model x(t t) =x(t) x(t + T t) =x r Apply only u(t)=u ã (t) (discard the remaining optimal inputs) Repeat the whole optimization at time t+1

24 Closed-Loop Stability (Bemporad, Morari 1999) Proof: Easily follows from standard Lyapunov arguments Stability Proof

25 Hybrid MPC - Example Switching System: " # ô cosë(t) à sinë(t) x(t +1)=.8 x(t) + õ u(t) sinë(t) cos ë(t) 1 y(t) = [ 1] x(t) ù if [1 ]x(t) õ ë(t) = 3 ù à if [1 ]x(t) < 3 Closed loop: y(t), r(t) u(t) Constraint: à 1 ô u(t) ô 1 Open loop: time t time t State Estimation / Fault Detection

26 State Estimation / Fault Detection Problem: given past output measurements and inputs, estimate the current state/faults Solution: Use Moving Horizon Estimation for MLD systems (dual of MPC) Augment the MLD model with: Input disturbances Output disturbances ø R n ð R p At each time t solve the optimization: T min P kyê(t à k t) à y(t à k) k and get estimates xê(t) k=1 MHE optimization = MIQP Convergence can be guaranteed (Bemporad, Mignone, Morari, ACC 1999) (Ferrari-T., Mignone, Morari, 22) Fault detection: augment MLD with unknown binary distubances þ {,1} p Example: Three Tank System COSY Benchmark problem, ESF þ 1 : leak in tank 1 for 2s ô t ô 6s þ 2 : valve V 1 blocked for t õ 4s

27 Example: Three Tank System COSY Benchmark problem, ESF þ 1 : leak in tank 1 for 2s ô t ô 6s þ 2 : valve V 1 blocked for t õ 4s Add logic constraint [h 1 ô h v ] þ 2 = Optimal Control of Hybrid Systems: Computational Aspects

28 MIQP Formulation of MPC (Bemporad, Morari, 1999) min X Tà1 n o y (t + k +1 t)qy(t + k +1 t)+u (t + k)ru(t + k) k= s.t.mld dynamics Set ø,[u(t),...,u(t +T à1),î(t t),...,î(t +T à1 t),z(t t),...,z(t +T à1 t)] 1 min J(ø, x(t)) = ø Hø + x 1 (t)fø + x (t)yx(t) 2 2 ø s.t.gø ô W + Sx(t) Mixed Integer Quadratic Program (MIQP) u R n u î {, 1} n î z R n z ø R T(n u+n z ) â{, 1} Tn î MILP Formulation of MPC Tà1 min X kqy(t + k +1 t)k + kru(t + k)k k= s.t.mld dynamics (Bemporad, Borrelli, Morari, 2) Introduce slack variables: min x min ï s.t. ï õ x ï õàx ï x k ï u k õ õ kqy(t + k t)k kru(t + k)k ï x k ï x k ï u k ï u k õ [Qy(t + k t)] i i =1,...,p, k =1,..., T à 1 õ à[qy(t + k t)] i i =1,...,p, k =1,...,Tà1 õ [Ru(t + k))] i i =1,...,m, k =,...,Tà1 õ à[ru(t + k))] i i =1,...,m, k =,...,Tà1 Set ø,[ï x 1,...,ïx N y,ï u 1,...,ï Tà1,U,î,z] Mixed Integer Linear Program (MILP) Tà1 min J(ø, x(t)) = X ï x + i ïu i ø k= s.t.gø ô W + Sx(t)

29 Mixed-Integer Program Solvers Mixed-Integer Programming is NP-hard Phase transitions have been found in computationally hard problems. Cost of Computation var 4 var 2 var BUT Ratio of Constraints to Variables (Monasson et al., Nature, 1999) General purpose Branch & Bound/Branch & Cut solvers available for MILP and MIQP (Ilog CPLEX, XPRESS-MP, Sahinidis,...) More solvers and benchmarks: No need to reach global optimum (see proof of the theorem), although performance deteriorates Good for large sampling times (e.g., 1 h) / expensive hardware but not for fast sampling (e.g. 1 ms) / cheap hardware! Hybrid Control Example: Heat Exchange

30 Hybrid Control - An Example (Hedlund and Rantzer, CDC1999) Objective: Control the temperature to a given set-point T 1, T 2 ô õ Tç 1 Tç 2 ô õ 1 ô õ B i = ô 1 õ ô ô = à 1 õ T1 à 2 T 2 if first furnace heated if second furnace heated if no heating õ + B i u Constraints: Only three operation modes: 1- Heat only the first furnace 2- Heat only the second furnace 3- Do not heat any furnaces Amount of heating power u is constant Alternate Heating of Two Furnaces HYSDEL model: MLD model: h.9231 i x(t +1)=.8521 x(t) z(t) u(t) + h 1 1 i z(t) x 3(t)

31 Alternate Heating of Two Furnaces Performance index min u(t),u(t+1),u(t+2) { } P 2 k= kr(u(t +k +1)à u(t +k))k + kq(x(t +k t) à x e )k Constraints à 1 ô x 1 ô 1 à 1 ô x 2 ô 1 Closed-Loop Behavior u =.4 u =.8 X 2 X 2 X 1 Set point cannot be reached X 1 Set point is reached Computational complexity of on-line MILP (Bemporad, Borrelli, Morari, 2) Linear constraints Continuous variables Binary variables Parameters Time to solve MILP (av.) ,9 s

32 On-Line vs. Off-Line Optimization min J(U, x(t)), P U k= MLD model subj. to x(t t) =x(t) x(t + T t) = Tà1 kqy(t + k +1 t)k + kru(t + k)k On-line optimization: given x(t), solve the problem at each time step t Mixed-Integer Linear Program (MILP) Off-line optimization: solve the MILP for all x(t) min J(ø, x(t)),f ø ø s.t.gø ô W + Fx(t) multi-parametric Mixed Integer Linear Program (mp-milp) Explicit Form of Model Predictive Control via Multiparametric Programming

33 Example of Multiparametric Solution Multiparametric LP ( ) ø R 2 6 CR{1,4} CR{1,2,3} 4 2 CR{1,3} CR{2,3} x x 1 Linear MPC Linear Model: x(t +1)=Ax(t)+Bu(t) y(t) =Cx(t) u(t) =u(t) à u(t à 1) input increments (for integral action) Prediction: y(t + k t) =CA k x(t)+ Pkà1 CA i Bu(t + k à 1 à i) i= u(t + k) =u(t à 1) + Pkà1 u(t + i) i=

34 Linear MPC Quadratic Performance Index P min pà1 kw y y(t + k +1 t) à r(t) U [ ]k 2 k= Constraints min U J(U, x(t)) = 2 1 U HU +[x (t) r (t)]fu [x (t) r (t)]y ô subject to GU ô W + K x(t) õ r(t) U,{ u(t),..., u(t + m à 1)} + kw u u(t + k)k 2 + W u [ u(t + k) à u target (t)] k 2 y min ô y(t + k t) ô y max, k =1,...,p u min ô u(t + k) ô u max, k =, 1,...,mà 1 u min ô u(t + k) ô u max, k =, 1,...,mà 1 ô x(t) õ r(t) (convex) QUADRATIC PROGRAM (QP) Multiparametric Quadratic Programming (Bemporad et al., 22) Objective: solve the QP for all

35 x X Linearity of the Solution solve QP to find identify active constraints at form matrices active constraints by collecting KKT optimality conditions: From (1) : From (2) : In some neighborhood of x, λ and U are explicit affine functions of x! Determining a Critical Region Impose primal and dual feasibility: linear inequalities in x! Remove redundant constraints critical region CR x CR = {Ax ôb} CR x-space X CR is the set of all and only parameters x for which Gà,Wà, Sà is the optimal combination of active constraints at the optimizer

36 Multiparametric QP R 1 R 2 R x 3 CR CR = {Ax ôb} R i = {x X : A i x>b i, A j z ôb j, j<i} x-space R N R 4 X Theorem: {CR,R 1,...,R N } is a partition of X ò R n Proceed iteratively: for each region R i repeat the whole procedure with X R i The recursive algorithm terminates after a finite number of steps, because the number of combinations of active constraints is finite Mp-QP More efficient method (Tøndel, Johansen, Bemporad, 23) x 2 x 2 x 2 CR CR CR x 1 x 1 x 1 The active set of a neighboring region is found by using the active set of the current region + knowledge of the type of hyperplane we are crossing: The corresponding constraint is added to the active set The corresponding constraint is withdrawn from the active set

37 Multiparametric QP continuous, piecewise affine convex, continuous, piecewise quadratic, C 1 (if no degeneracy) Corollary: The linear MPC controller is a continuous piecewise affine function of the state Complexity Reduction CR1 CR2 CR3 CR4 CR5 CR6 CR7 CR8 CR CR1 CR2 CR3 CR4 CR5 CR6 CR U(x()),[u (x()),...,u Tà1 (x())] Regions where the first component of the solution is the same can be joined (when their union is convex). (Bemporad, Fukuda, Torrisi, Computational Geometry, 21)

38 System: Double Integrator Example y(t) = 1 s 2 u(t) x(t +1)= 1 1 sampling + ZOH T=1 s ô õ ô x(t)+ 1 1 y(t) = [ 1 ] x(t) õ u(t) Constraints: à 1 ô u(t) ô 1 Control objective: minimize P y (t)y(t)+ 1 t= 1 u 2 (t) u t+k = K LQ x(t + k t) k õ N u Optimization problem: for N u =2 (cost function is normalized by max λ(h)) mp-qp solution N u =2

39 Complexity N u =3 N u =4 N u =5 N u =6 CPU time # Regions # free moves Extensions Tracking of reference r(t) : îu(t) =F(x(t),u(t à 1),r(t)) Rejection of measured disturbance v(t) : îu(t) =F(x(t),u(t à 1),v(t)) Soft constraints: u(t) =F(x(t)) y min à ï ô y(t + k t) ô y max + ï Variable constraints: u(t) =F(x(t),u min (t),...,y max (t)) u min (t) ô u(t + k) ô u max (t) y min (t) ô y(t + k t) ô y max (t) Linear norms: min J(U, x(t)), P p U k= kqy(t + k t)k + kru(t + k)k (Bemporad, Borrelli, Morari, IEEE TAC, 22)

40 Closed-Loop MPC and Hybrid Systems DHA Motivation: Use hybrid techniques to analyze closed-loop MPC systems! (Bemporad, Heemels, De Schutter IEEE TAC, 22) MPC Regulation of a Ball on a Plate Task: Tune an MPC controller by simulation, using the MPC Simulink Toolbox Get the explicit solution of the MPC controller. Validate the controller on experiments.

41 B&P Experimental Setup Host Ball & Plate System TCP/IP Code Data Ball & Plate u 1, u 2 Target a,b x, y Ball&Plate Experiment y δ γ x Specifications: Angle: -17 deg +17deg Plate: -3 cm +3 cm Input Voltage: -1 V +1 V Computer: PENTIUM166 Sampling Time:3 ms α α' β' β Model: LTI 14 states Constraints on inputs and states

42 General Philosophy: (1) MPC Design Step 1: Tune the MPC controller (in simulation) E.g: MPC Toolbox for Matlab (Bemporad, Morari, Ricker, 23) General Philosophy: (2) Implementation Step 2: Solve mp-qp and implement Explicit MPC E.g: Real-Time Workshop + xpc Toolbox

43 Controller: Explicit MPC Solution x: 22 Regions, y: 23 Regions o x-mpc: sections at α x =, α x =, u x =, r x =18, r α = Region 1: Region 6: Region 16: LQR Controller (near Equilibrium) Saturation at -1 Saturation at +1 MPC Regulation of a Ball on a Plate Design Steps: Tune an MPC controller by simulation, using the MPC Simulink Toolbox. Get the explicit solution of the MPC controller. Validate the controller on experiments.

44 Comments on Explicit MPC Multiparametric Quadratic Programs (mp-qp) can be solved efficiently Model Predictive Control (MPC) can be solved off-line via mp-qp Explicit solution of MPC controlleru = f(x) is Piecewise Affine Eliminate heavy on-line computation for MPC Make MPC suitable for fast/small/cheap processes MILP Formulation of MPC Tà1 min X kqy(t + k +1 t)k + kru(t + k)k k= s.t.mld dynamics (Bemporad, Borrelli, Morari, 2) Introduce slack variables: min x min ï s.t. ï õ x ï õàx ï x k ï u k õ õ kqy(t + k t)k kru(t + k)k ï x k ï x k ï u k ï u k õ [Qy(t + k t)] i i =1,...,p, k =1,...,Tà1 õ à[qy(t + k t)] i i =1,...,p, k =1,...,Tà1 õ [Ru(t + k))] i i =1,...,m, k =,...,Tà1 õ à[ru(t + k))] i i =1,...,m, k =,...,Tà1 Set ø,[ï x 1,...,ïx N y,ï u 1,...,ï Tà1,U,î,z] Mixed Integer Linear Program (MILP) Tà1 min J(ø, x(t)) = X ï x + i ïu i ø k= s.t.gø ô W + Sx(t)

45 Multiparametric MILP min ø={øc,ø d } f ø c + d ø d ø c R n s.t. Gø c + Eø d ô W + Fx ø d {, 1} m mp-milp can be solved (by alternating MILPs and mp-lps) (Dua, Pistikopoulos, 1999) Theorem: The multiparametric solution ø ã (x) is piecewise affine Corollary: The MPC controller is piecewise affine in x Solutions via Dynamic Programming (Borrelli, Bemporad, Baotic, Morari, 23) (Mayne, ECC 21) Explicit solutions to finite-time optimal control problems for PWA systems can be obtained using a combination of Dynamic Programming Multiparametric Linear (1-norm, -norm), or or Quadratic (squared 2-norm) programming Note: in the 2-norm case, the partition may not be polyhedral

46 Hybrid Control Examples (Revisited) Hybrid Control - Example Switching System: " # ô cosë(t) à sinë(t) x(t +1)=.8 x(t) + õ u(t) sinë(t) cos ë(t) 1 y(t) = [ 1] x(t) ù if [1 ]x(t) õ ë(t) = 3 ù à if [1 ]x(t) < 3 Closed loop: y(t), r(t) u(t) Constraint: à 1 ô u(t) ô 1 Open loop: time t time t

47 Hybrid MPC - Example MLD system State x(t) Input u(t) Aux. binary vector δ(t) Aux. continuous vector z(t) 2variables 1variables 1variables 4variables mp-milp optimization problem minn oj(v 1,x(t)),P 1 kq k= 1 (v(k) à u e )k + kq 2 (î(k t) à î e )k + kq 3 (z(k t) à z e )k + kq 4 (x(k t) à x e )k v 1 subject to constraints to be solved in the region à 5 ô x 1 ô 5 à 5 ô x 2 ô 5 Computational complexity of mp-milp Linear constraints Continuous variables Binary variables Parameters Time to solve mp-milp Number of regions min 7 mp-milp Solution Prediction Horizon T=2 PWA law ñ MPC law Linear constraints Continuous variables Binary variables Parameters Time to solve mp-milp Number of regions min 7

48 mp-milp Solution X 2 X 2 X 1 Prediction Horizon T=3 Prediction Horizon T=4 X 1 Alternate Heating of Two Furnaces (Bemporad, Borrelli, Morari, 2) mp-milp optimization problem min u(t),u(t+1),u(t+2) { } à 1 ô x 1 ô 1 à 1 ô x 2 ô 1 ô u ô 1 P 2 k= Parameter set kr(u(t +k +1)à u(t +k))k + kq(x(t +k t) à x e )k parameterized! Computational complexity of mp-milp Linear constraints Continuous variables Binary variables Parameters Time to solve mp-milp Number of regions min 15

49 u =.4 mp-milp Solution u =.8 No Heat No Heat T 2 Heat 1 T 2 Heat 1 Heat 2 Heat 2 T 1 T 1 X 2 X 2 X 1 Set point cannot be reached Set point is reached X 1 NOTE: The control action of explicit and of implicit MPC are totally equal! Hybrid Control Example: Traction Control System

50 Vehicle Traction Control Improve driver s ability to control a vehicle under adverse external conditions (wet or icy roads) Model nonlinear, uncertain, constraints Controller suitable for real-time implementation MLD hybrid framework + optimization-based control strategy Simple Traction Model Mechanical system ò ó 1 ü t ωç e = ü c à b e ω e à Je gr ü t vç v = mv r t Manifold/fueling dynamics ü c = b i ü d (t à ü f ) Tire torque τ t is a function of slip ω and road surface adhesion coefficient µ ω e v v ω = à gr rt Wheel slip ω

51 Tire Force Characteristics Longitudinal Force Lateral Force Tire Forces Slip Target Zone Maximum Braking Maximum Cornering Maximum Acceleration Tire Slip Steer Angle Longitudinal Force Lateral Force Hybrid Model Torque Nonlinear tire torque τ t =f( ω, µ) µ Slip PWA Approximation (PWL Toolbox, Julian, 1999) Torque µ Slip HYSDEL (Hybrid Systems Description Language) Mixed-Logical Dynamical (MLD) Hybrid Model (discrete time)

52 MLD Model State x(t) 9 variables Input u(t) 1 variable Aux. Binary vars δ(t) 3 variables Aux. Continuous vars z(t) 4 variables The MLD matrices are automatically generated in Matlab format by HYSDEL Performance and Constraints Control objective: min P N k= subj. to. ω(k t) à ω des MLD Dynamics Constraints: Limits on the engine torque: à 2Nm ô ü d ô 176Nm Logic Constraint: Hysteresis

53 Experimental Apparatus Experimental Apparatus

54 Experimental Results 25 2 Wheel Speeds, [rad/s] Engine Torque Command, [Nm] Controller Region Time, [s] Experiment >5 regions 2ms sampling time Pentium 266Mhz + Labview

55 Hybrid Control Example: Cruise Control System Hybrid Control Problem Renault Clio 1.9 DTI RXE GOAL: command gear ratio, gas pedal, and brakes to track a desired speed and minimize consumption

56 Hybrid Model Vehicle dynamics mẍ =F e à F b à ìxç xç = vehicle speed discretized with sampling time T s =.5 s Transmission kinematics F e = traction force F b = brake force R ω = g (i) ks xç F e = R g (i) ks M ω M i = engine speed = engine torque = gear Hybrid Model Engine torque à C à e (ω) ô M ô C+ e (ω) Max engine torque C + e (ω) 18 Piecewise-linearization: (PWL Toolbox, Julián, 1999) requires: 4 binary aux variables 4 continuous aux variables Min engine torque C à e (ω) =ë 1 + ì 1 ω

57 Hybrid Model Gear selection: define a binary input for each gear #i, g i {, 1} Gear selection (traction force): F e = R g (i) ks M depends on gear #i define auxiliary continuous variables: IF g i =1 THEN F ei = R g (i) M ELSE ks ω = R g (i) ks xç F e = F er + F e1 + F e2 + F e3 + F e4 + F e5 Gear selection (engine/vehicle speed): similarly, also requires 6 auxiliary continuous variables MLD model Hybrid Model x(t +1)=Ax(t)+B 1 u(t) + B 2 î(t)+b 3 z(t) y(t) =Cx(t)+D 1 u(t) + D 2 î(t)+d 3 z(t) E 2 î(t)+e 3 z(t) ô E 4 x(t)+e 1 u(t)+e 5 2 continuous states: 2 continuous inputs: 6 binary inputs: 1 continuous output: x, v M, F b g R,g 1,g 2,g 3,g 4,g 5 v (vehicle position and speed) (engine torque, brake force) (gears) (vehicle speed) 16 auxiliary continuous vars: 4 auxiliary binary vars: 96 mixed-integer inequalities (6 traction force, 6 engine speed, 4 PWL max engine torque) (PWL max engine torque breakpoints)

58 Hysdel Model Hybrid Controller Max-speed controller max J(u t, x(t)),v(t + 1 t) ut ú MLD model subj. to x(t t) = x(t) 25 2 MILP optimization problem Linear constraints Continuous variables Binary variables Parameters Time to solve mp-milp (Sun Ultra 1) Number of regions s 11 v(t) x(t) ( x(t) is irrelevant)

59 Hybrid Controller Max-speed controller 2 Velocity (km/h) 5 Gear 1 Fraction of Max Torque (Nm) 1 Brakes (Nm) Road Slope (deg) 6 Engine speed (rpm) 2 Engine Torque (Nm) 6 Power (kw) Time (s) 5 1 Time (s) 5 1 Time (s) 5 1 Time (s) Tracking controller Hybrid Controller min J(u t,x(t)), v(t +1 t) à v d (t) + ú ω ut ú MLD model subj. to x(t t) =x(t) 25 MILP optimization problem Linear constraints Continuous variables Binary variables Parameters Time to solve mp-milp (Sun Ultra 1) Number of regions m 49 v d (t) v(t)

60 Hybrid Controller Tracking controller min ut J(u t,x(t)), v(t +1 t) à v d (t) + ú ω Velocity (km/h), Desired velocity (km/h) Gear 1.5 Fraction of Max Torque (Nm) Brakes (Nm) ú = Road Slope (deg) Engine speed (rpm) Engine Torque (Nm) Power (kw) Time (s) 1 2 Time (s) Time (s) Time (s) Hybrid Controller Smoother tracking controller min J(u t,x(t)), v(t +1 t) à v d (t) + ú ω ut v(t +1 t) à v(t) <T s a max subj. to MLD model x(t t) =x(t) 25 MILP optimization problem Linear constraints Continuous variables Binary variables Parameters Time to solve mp-milp (Sun Ultra 1) Number of regions m 54 v d (t) v(t)

61 Hybrid Controller Smoother tracking controller Velocity (km/h), Desired velocity (km/h) Gear 1.5 Fraction of Max Torque (Nm) Brakes (Nm) Road Slope (deg) 6 Engine speed (rpm) 2 Engine Torque (Nm) 1 Power (kw) Time (s) 1 2 Time (s) Time (s) Time (s) Reachability Analysis (Verification)

62 Verification GIVEN: an embedded system (continuous dynamical system + logic controller) CERTIFY that such combination behaves as desired for ALL initial conditions within a given set for ALL disturbances within a given class or PROVIDE a counterexample. Simulation: provides a partial answer (not all possibilities can be tested!) Reachability Analysis: provides the answer. Ariane For 1=1:1 For a[i]=; 1=1:1 end a[i]=; 4 end while For (x) 1=1:1 read while a[i]=; file (x) read end file Success Physical beaviour Computer code 5 + For 1=1:1 For a[i]=; 1=1:1 end a[i]=; 4 end while For (x) 1=1:1 read while a[i]=; file (x) read end file Failure Need for verification

63 Reachability Analysis/Verification (Bemporad, Torrisi, Morari, 2) X() X Z1 () Z 1 X Z2 () Z 2 X ZL () Given: A hybrid system A set of initial conditions Target sets Time horizon Σ X() Z 1, Z 2,..., Z L t ô T max Z L (disjoint) Problem: Is Z i reachable from X() in t steps? If yes, from which subset X Zi () of X()? Disturbance/input sequences driving X Zi () to Z i. Applications X() X Z1 () Z 1 Z 1, Z 2,..., Z L Safety ( = unsafe sets) Stability ( Z 1 = invariant set around the origin) X Z2 () X Z L () Z 2 Z L u(),...,u(t max) Optimal control ( = optimal strategy, Z 1 = reference set) (Bemporad, Giovanardi, Torrisi, CDC 2) (practical) Liveness ( Z 1 = set to be reached within a finite time) Robust Simulation

64 Verification Algorithm via MILP Simple solution: Solve T> the mixed-integer linear feasibility test x() X() x(t) Z i u(t) U, ô t ô T x(t +1)=Ax(t)+B 1 u(t)+b 2 î(t)+b 3 z(t) E 2 î(t)+e 3 z(t) ô E 1 u(t)+e 4 x(t)+e 5 with respect to x(), {u(t),î(t),z(t)} T t= Only practical for small problems! because number of free integer variables î(), î(1),..., î(t) grows with T Efficient Solution: Exploit the special structure of the problem. Reach-Set Evolution (Bemporad, Torrisi, Morari, 2) (Torrisi, Bemporad, Giovanardi, 23) T max <, discrete-time Decidable but NP-hard Reachability analysis algorithm: Compute the polyhedral reach set X(t) Detect switching X() Describe new intersections X(t) C j X(1) Stopping criteria for a single X(2) exploration X(3) Organize the search X(4) C i C j (Bemporad, Torrisi, Morari, 2)

65 Reach Set Computation Reach set implicitly defined by linear inequalities where is the current region Simple to compute Number of constraints grows linearly with time Explicit form also possible via projection methods (e.g. CDD, Fukuda 1997) Approximation of Intersections Simple to compute via Linear Programming (LP) Can approximate with arbitrary precision Trade off between conservativeness and complexity Both inner and outer approximations in one shot Approximate computation of projections

66 Stopping Criteria Reach set has left the current region X () C 1 C 2 X (1, X ()) X (2, X ()) X (3, X ()) Reach set is all inside a target Z i X (4, X ()) X (5, X ()) Z i t > T max (to guarantee termination) X (3, X ()) X (2, X ()) X (1, X ()) X () Graph of Evolution X 1 X Z 1 Z The graph is initialized with all the initial subsets X i =X() C i and all the target sets A node is added for each initial region of a new exploration If a set can reach another set, an oriented arc is drawn The time needed to reach the set is a weight associated with the arc

67 Conservativeness vs Graph Complexity Less conservative Graph more complicate Graph less complicate More conservative LP determines if a switching sequence is feasible Verification Example: Cruise Control System

68 Cruise Control System GOAL: Verify if a given switching controller satisfies certain specifications (Torrisi, Bemporad, 21) Cruise Control System Gear selector: Speed controller:

69 Verification Problem Question: Will the cruise control reach the desired speed reference within 1 s without exceeding the speed limit? Speed (Km/h) Safety Time (s) Liveness Safety Liveness Hysdel Model (HYbrid Systems DEscription Language) SYSTEM car { INTERFACE { STATE { REAL speed, err, vr; BOOL gear1, gear2, gear3, gear4, gear5; } PARAMETER { }} IMPLEMENTATION { AUX { REAL Fe1, Fe2, Fe3, Fe4, Fe5, w1, w2, w3, w4, w5, DCe1, DCe2, DCe3, DCe4,zut, zub, ierr, torque, F_brake; BOOL dpwl1, dpwl2, dpwl3, dpwl4, sd, su, verr, sat_torque, sat_f_brake, no_sat;} LOGIC { no_sat = ~(sat_torque sat_f_brake) & verr; } AD {dpwl1 = wpwl1 - (w1 + w2 + w3 + w4 + w5) <= ; dpwl2 = wpwl2 - (w1 + w2 + w3 + w4 + w5) <= ; dpwl3 = wpwl3 - (w1 + w2 + w3 + w4 + w5) <= ; dpwl4 = wpwl4 - (w1 + w2 + w3 + w4 + w5) <= ; sd = (w1 + w2 + w3 + w4 + w5) - wl <= ; su = wu - (w1 + w2 + w3 + w4 + w5) <= ; verr = speed - vr - 2 <= ; sat_torque = - zut + (DCe1 + DCe2 + DCe3 + DCe4) + 1 <= ; sat_f_brake = - zub + max_brake_force <= ; } DA {Fe1 = {IF gear1 THEN torque / speed_factor * Rgear1}; Fe2 = {IF gear2 THEN torque / speed_factor * Rgear2}; Fe3 = {IF gear3 THEN torque / speed_factor * Rgear3}; Fe4 = {IF gear4 THEN torque / speed_factor * Rgear4}; Fe5 = {IF gear5 THEN torque / speed_factor * Rgear5}; w1 = {IF gear1 THEN speed / speed_factor * Rgear1}; w2 = {IF gear2 THEN speed / speed_factor * Rgear2}; w3 = {IF gear3 THEN speed / speed_factor * Rgear3}; w4 = {IF gear4 THEN speed / speed_factor * Rgear4}; w5 = {IF gear5 THEN speed / speed_factor * Rgear5}; DCe1 = {IF dpwl1 THEN (apwl2) + (bpwl2) * (w1 + w2 + w3 + w4 + w5) ELSE (apwl1) + (bpwl1) * (w1 + w2 + w3 + w4 + w5)}; DCe2 = {IF dpwl2 THEN (apwl3 - apwl2) + (bpwl3 - bpwl2) * (w1 + w2 + w3 + w4 + w5)}; DCe3 = {IF dpwl3 THEN (apwl4 - apwl3) + (bpwl4 - bpwl3) * (w1 + w2 + w3 + w4 + w5)}; DCe4 = {IF dpwl4 THEN (apwl5 - apwl4) + (bpwl5 - bpwl4) * (w1 + w2 + w3 + w4 + w5)}; zut = {IF verr THEN kt * (vr - speed) + it * err}; zub = {IF ~verr THEN - kb * (vr - speed) - ib * err}; torque ={IF sat_torque THEN (DCe1 + DCe2 + DCe3 + DCe4) + 1 ELSE zut}; F_brake = {IF sat_f_brake THEN max_brake_force ELSE zub}; ierr = {IF no_sat THEN err + Ts * (vr - speed)};} CONTINUOUS { speed = speed + Ts / mass * (Fe1 + Fe2 + Fe3 + Fe4 + Fe5 - F_brake - beta_friction * speed); err = ierr; vr = vr;} AUTOMATA { gear1 = (gear2 & sd) (gear1 & ~su); gear2 = (gear1 & su) (gear3 & sd) (gear2 & ~sd & ~su); gear3 = (gear2 & su) (gear4 & sd) (gear3 & ~sd & ~su); gear4 = (gear3 & su) (gear5 & sd) (gear4 & ~sd & ~su); gear5 = (gear4 & su) (gear5 & ~sd );} MUST { -w1 <= -wemin; w1 <= wemax; -w2 <= -wemin; w2 <= wemax; -w3 <= -wemin; w3 <= wemax; -w4 <= -wemin; w4 <= wemax; -w5 <= -wemin; w5 <= wemax; -F_brake <= ; F_brake <= max_brake_force; torque - (DCe1 + DCe2 + DCe3 + DCe4) - 1 <= ; -((REAL gear1) + (REAL gear2) + (REAL gear3) + (REAL gear4) + (REAL gear5)) <= ; (REAL gear1) + (REAL gear2) + (REAL gear3) + (REAL gear4) + (REAL gear5) <= 1.1; dpwl4 -> dpwl3; dpwl4 -> dpwl2; dpwl4 -> dpwl1; dpwl3 -> dpwl2; dpwl3 -> dpwl1; dpwl2 -> dpwl1;}}

70 Hybrid Model MLD model 3 continuous states: v, v r,e (speed, reference and tracking error) 5 binary states: g 1,g 2,g 3,g 4,g 5 (gears) 19 auxiliary continuous vars: (5 traction force, 5 engine speed, 5 reset/saturation, 4 PWL max engine torque) 15 auxiliary binary vars: (4 PWL max torque breakpoints, 4 saturations 5 logic updates, 2 gear switching conditions) 173 mixed-integer inequalities Verification Results For all v r [3, 7] km/h satisfies both liveness the controller & safety properties CPU time: 2.5h (Matlab 5.3, PC65MHz) Speed (Km/h) Safety Time (s) Liveness For v r [3, 12] km/h the verification algorithm finds the first counterexample after 7m

71 Conclusions Hybrid systems as a framework for new applications, where both logic and continuous dynamics are relevant Mixed Logical Dynamical (MLD) systems as discretetime, computation-oriented models for hybrid systems HYSDEL x(t +1)=Ax(t)+B 1 u(t) + B 2 î(t)+b 3 z(t) y(t) =Cx(t)+D 1 u(t) + D 2 î(t)+d 3 z(t) E 2 î(t) +E 3 z(t) ô E 4 x(t) +E 1 u(t) +E 5 Conclusions Hybrid systems as a framework for new applications, where both logic and continuous dynamics are relevant Mixed Logical Dynamical (MLD) systems as discretetime, computation-oriented models for hybrid systems Supervisory MPC controllers and State Estimation/Fault Detection schemes can be synthesized via on-line mixedinteger programming (MILP/MIQP) Piecewise Linear Optimal Controllers can be synthesized via off-line multiparametric programming for fast-sampling applications Plant Input u(t) Output y(t) Measurements

72 Conclusions Hybrid systems as a framework for new applications, where both logic and continuous dynamics are relevant Mixed Logical Dynamical (MLD) systems as discretetime, computation-oriented models for hybrid systems Supervisory MPC controllers and State Estimation/Fault Detection schemes can be synthesized via on-line mixedinteger programming (MILP/MIQP) Piecewise Linear Optimal Controllers can be synthesized via off-line multiparametric programming for fast-sampling applications Safety Analysis properties can be formally verified References Paper download: